Mobile App Testing – Introduction for Product Owners

In this article, we will look at some of the key aspects of testing, focusing on the following:

• manual testing
• nature of the mobile app testing process
• purpose of the test pyramid
• types of tests and how to use them

Mobile application testing – basics

To fully understand what mobile application testing is all about, we need to explain what it really is at its core. 

What is mobile application testing?

As the name suggests, mobile application testing refers to the process of checking the functionality of an application developed for mobile devices based on predefined criteria. This includes both individual parts and the whole application. The testing process can be carried out manually and automatically. 

Its purpose is to check whether the application meets the technical and business requirements set for it. Mobile testing guarantees the highest quality product for end users is delivered, meeting our expectations. Thus, we can say that the testing process aims to ensure the highest product quality.

Manual testing in mobile app development

Manual testing is one of the stages of developing every app, not just a mobile application. But what does manual testing really mean and what role does it play in software development? 

Before we explain that, there’s one more thing that we need to know. It’s a fact that almost no application is created all at once but in fragments. And these fragments are the first thing that we need to test. 

However, to give you the simplest definition of manual testing, we can say that it is a software testing method that is performed manually by the tester. This means that each submitted fragment of the application is checked by a physical person imitating the end user of the mobile application.

At this stage, we check the application fragment for compliance with previous assumptions, functionality, performers, and user interface.

But that’s not all. During the execution of manual testing, we create documentation for the conducted tests, which we can later use to create automated tests and reports, as well as perform regression tests. Such documentation also makes it easier to see what kind of tests and how many of them were performed.

What is automation testing in mobile app development?

Automated testing is a hot topic in the testing field, and it’s definitely worth taking an interest in. We often hear that we should automate as much as possible. 

However, it’s worth analyzing our project, budget, and application concept before we start automating tests. We need to consider in which areas automated tests will bring the most value to the project.

Before we proceed further, we need to understand what automated tests are.  

Automated tests are scripts created by an automation tester. Such a script is designed to execute tests automatically. In this case, the tester is not responsible for manually checking the application but for writing code that, when executed, will verify the application on its own. 

In the next paragraph, we’ll cover some of the differences between manual and automated testing and discuss when it’s best to perform each type of test.

Read also: Mobile & Web App Development Process – Quality Assurance

Manual testing vs. automation testing of mobile apps

The first difference will be the execution time of the test. Manual testing can be a tedious step-by-step test process that consumes a large amount of time. On the other hand, automated tests are much faster as they can perform the same steps within a few seconds. However, these tests must be created beforehand, which is also time-consuming.

It is worth noting that, during manual test execution, a person can observe the application’s unusual behavior and quickly adapt their testing focus, making it easier to identify new defects. This can also be a drawback, as the tester may make mistakes or overlook a step in the test, which can lead to the omission of a potential defect. 

On the other hand, automated tests run according to a strictly defined scenario, so there is no mistake on their part. But, they lack human creativity, so they can potentially miss new bugs.

Test creation is also a topic worth considering. Both types of testing may initially require exploratory testing, especially when we are dealing with the application for the first time. To write automated tests, we first need to know the list of steps necessary for automation. In the case of manual tests, it is good practice to have documented test scenarios, which we will also create as a result of exploratory testing sessions.

Test repetition is one of the most significant factors when deciding on automated testing. Automated tests perform very quickly, while a human cannot achieve such fast results. However, they can take a non-standard approach to finding errors when performing tests.

Here’s a simple table comparing automation and manual testing to help you understand their differences quickly and easily.

In conclusion, both testing methods are very different from each other, so before deciding which tests to choose, we should carefully analyze our project. There’s also nothing stopping us from implementing both types of testing in our project. This will allow us to check the application very thoroughly and efficiently.

Read also: 12 Great Manual QA Testing Tools to Boost Your App’s Success

How is mobile testing different from web testing?

The difference in the method of testing a mobile application and a web application results from several factors. 

The following table compares testing mobile apps and web apps, summarizing their distinct characteristics. You can also use it as a valuable guide, illuminating the key elements that ought to be evaluated for each.

As you can see, there are many differences, and these are just a few. 

• The main difference is the environment in which the app runs. 

Mobile applications are often hybrid apps specifically designed for platforms like Android or iOS. In contrast, web applications are crafted to function within a web browser, negating the need for installation on a device.

• The type of devices that an application will operate on varies between web and mobile apps. 

Mobile applications are typically installed and used on devices like smartphones, tablets, or smartwatches, which generally have smaller screen sizes. In contrast, web applications, being browser-based, have the added flexibility to run on devices with larger displays, like computers.

• That leads to the next distinction – the user interaction approach. 

Mobile applications are tested for their performance under touch-based interactions, including various gestures. Web applications, meanwhile, are evaluated based on their functionality when operated using conventional input devices, such as a mouse and keyboard.

• Another difference is functionality in offline mode. 

Mobile applications are often designed to be used without an internet connection. Then, we verify that the application displays all the data that was intended to be displayed offline. In the case of a web application, this is more difficult because the lack of internet usually makes it impossible to use.

• The last item in the table is web browsers. 

Mobile apps operate independently of a web browser, yet they often contain features that link to web pages for extended functionality. Conversely, web applications require extensive testing across browsers to ensure consistent performance and universal compatibility.

These are the differences between mobile applications and web applications. As we see in the table, each of them has different requirements, works differently, and has different access to the functions built into the device. And it is these elements that will affect what tests will be carried out on each of them.

Why does mobile app testing matter and how does it benefit your business?

The testing process enables the detection of errors at the early stages of application production when its initial components are being developed. 

This allows us to avoid accumulating many errors that would make using the product more and more difficult. Additionally, the earlier an error is detected, the easier and cheaper it is to fix.

An untested application is highly likely to provide users with a non-working product.

And an app that doesn’t work will not be successful and neither will it bring a profit. However, if the application could be tested before showing it to users, we can avoid such problems later. Finally, it also increases our trust in the product and its quality.

Read also:

• 10 Benefits of Mobile Usability Testing | Guide for App Owners

• Mobile App for Employees: 5 Reasons Why Your Company Needs One

Who tests mobile applications and when?

Now it’s time for us to talk about who is responsible for running the tests. Because it isn’t always a tester…

Developers create the first tests. Surprised? Here is how it happens. 

• Initial testing procedures are carried out in tandem with code development. These are essential to verify particular sections, functions, or even single lines of code. Thanks to this solution, a lot of tests that check small code fragments can be created in a short time. And since developers have the best understanding of their own code, they are ideally suited to perform this task.

• Following the developers, testers step in to evaluate the finalized code segments, be it a new function or design element. Their role is to verify that these components operate as intended while scanning for any errors within the application.

• And what happens next? Before the release of the application, the client can make additional tests, after which they approve the application and allow its release. Those tests can be performed by the customer directly or by another team. They are not performed by anyone who worked on the application but by independent people.

What is a testing pyramid?

Another thing worth mentioning is the testing pyramid. The test pyramid may vary depending on the source or specific needs the project team faces. Therefore, we will focus on its basic four-level variant.

Let’s start with the definition from the ISTQB document, which unifies the concepts from the entire field of testing:

A graphical model representing the relationship of the amount of testing per level, with more at the bottom than at the top.

To simplify the above definition, we can say that the test pyramid suggests the proportions that should be maintained between different types of tests. The basis of the pyramid covers the fastest and more stable tests. 

At higher levels, some tests require more time for preparation and execution. This means that the higher a particular type of test is positioned in the pyramid, the more human effort is required to conduct it. 

We can present the test pyramid graphically.

Unit tests

Unit tests form the foundational layer of the testing pyramid, chiefly because they are the most numerous, the fastest to implement and, at the same time, the cheapest. Their primary function is to validate the logic and functionality of the code. Typically, these tests are performed in isolation, ensuring a thorough and focused examination of individual code components.

Integration tests

According to the pyramid, there are fewer of these compared to unit tests, but they are much more time-consuming. At this level of testing, we verify that the interface operation (i.e., data flow) is correctly working, as well as the interaction between various elements. To simplify, here we check the fragments of the application and the integration of these fragments with each other, not just individual elements. We don’t yet have a GUI (Graphical User Interface) at this level.

System tests

System tests are the level of functional testing where we verify the functionalities of the developed software in an integrated environment. At this point, we can test the product end-to-end. These tests aim to check the compatibility of a specific function’s behavior with its assumptions and identify any errors. Such tests are time-consuming due to their complexity. Additionally, at this stage, we also test the application’s design.

Acceptance tests

The final tier of testing evaluates the system’s readiness for deployment or release. Responsibility for their implementation rests on the client or stakeholders. Finally, they decide whether the product is ready for release. Additionally, you don’t have to perform these tests at the end of development, as you can do it during software development. These assessments are typically performed by individuals who haven’t previously participated in the product’s development, ensuring an unbiased and fresh perspective.

The key advantages of the test pyramid

Here is why you can benefit from the test pyramid:

• Showcases Complexity: The test pyramid uses simple graphics to show how complex the testing process can be.

• Indicates Work Volume: It helps visualize the amount of work required to finish a project.

• Highlights Cost: The pyramid provides a rough idea about the cost of different testing levels.

• User-friendly: It’s easy to understand, even for someone new to the subject, helping them appreciate the intricacy of testing.

• Improves flexibility: Remember, the pyramid’s shape can change based on the specific needs of your project. It’s not one-size-fits-all!

What does the testing pyramid look like in a mobile application?

In the previous paragraph, we mentioned that the test pyramid can be modified depending on the project’s needs. The same applies to mobile applications. What’s more, the pyramid can be reversed! 

Inverting the pyramid doesn’t change the cost of running tests or the time that is needed to run them. Only the quantity of specific types of tests to be conducted changes. Manual tests will still take longer than unit tests and require more preparation time. However, the reversal of the pyramid may imply that there will be more manual tests than unit tests. Why can this happen?

• Mobile applications can be about practically anything. We have a lot of mobile apps on the market for ordering food, buying items, editing photos, navigating via GPS, and many more. Each of them will require a different approach to testing. 

• Moreover, we perform tests for mobile applications on various devices, so we must conduct the same test multiple times. Unit and integration tests, positioned beneath system tests in the pyramid, don’t support this type of verification. These tests confirm functionality but don’t ensure performance across specific devices.

• As a result, additional tests are carried out to ensure the application’s operation on as many devices as possible. Importantly, these evaluations are conducted manually on real mobile devices, not simulations.

It’s crucial to remember that the testing pyramid is just a guideline illustrating what the testing process might look like. It doesn’t lay out prescriptive rules but rather provides a flexible framework to initiate our testing procedure.

Regression tests and retest – what is it and why do we need it?

Performing tests involves the possibility of detecting errors in the application. If an error is detected, the relevant element in which it occurs is submitted for correction. After the developer makes a correction, the item is tested again to see if the error has been resolved. This is a retest, re-executing the same test due to the changes made.

Meanwhile, regression tests are performed when we want to recheck all or part of the developed software. This involves re-executing all previously performed tests and performing additional ones. This allows us to explore large areas of the application, including those that haven’t been reviewed or tested before, or haven’t been reviewed for a long time. We check the application for any previously overlooked errors and any new ones that might have arisen due to recent updates.

Regression tests are crafted to ensure the application functions correctly and as anticipated, irrespective of any past changes or updates. Typically, these tests are carried out before the launch of the application or a new version.

Smoke testing – what is it?

The last type of test which we will discuss here is smoke testing. 

Smoke tests are designed to check the most important functionalities of the software without going into details. These tests allow you to get to know the application in general and check its critical points. The details are checked with other tests. In critical moments, smoke tests can replace regression tests, but we must keep in mind that, in this case, we can only check the main elements of the application.

What app features should be tested?

Now, it’s time to talk about what we should pay attention to when testing. And the answer is simple: everything we test, not just the main functionalities of the application. 

We also pay attention to things such as the design or non-functional features of the application So, we don’t only check if something works, but also how it works. These can be performance or usability tests. Design projects, requirements, and instructions can also be subject to testing.

Furthermore, our evaluation process isn’t limited to confirming the application’s desired behaviors; we also investigate the potential for undesirable ones. For instance, an undesirable behavior could be the registration allowance using a name composed solely of numbers. Such a registration attempt should be rejected unless our assumptions indicate otherwise.

It’s good practice to test the minor functionalities, as well as the most critical. Possible errors can be found even in the least expected places within the mobile application.

Choosing the right devices for effective mobile app testing

Another important aspect of testing mobile applications is the devices we choose for testing. So, how do we choose them? 

Native vs. hybrid application

First, we must know if we are making a native or hybrid application. With a native application made only for iOS or Android, we don’t need devices that are not based on the selected system. Of course, you can make a native application for both of these operating systems. Then, we go for both iOS and Android.

Know your regional target

Awareness of our app’s intended audience’s geographical scope, be it country, region, or even continent, provides crucial insights. This lets us determine the most widely used devices and system versions. Several online applications provide this data, accessible to anyone who needs it. Additionally, you can ask a client for any information or statistics they collected themself. These dates may differ from those on the Internet and are also worth considering.

Always test on more than one device 

Next, you should consider compatibility testing across various devices, not just the platform or system version. It’s important to ensure that the selected devices also differ in physical aspects. An application running on an older device may behave differently than on the newest one. Some designs can break on large screens, even though they look impeccable at smaller sizes. That’s why it’s so important to test the application on as many devices as possible.

How does Droids On Roids help with mobile app testing?

At Droids On Roids, we make every effort to ensure that the applications we create are of the highest quality so that we can be proud of them. Therefore, during the development process of a mobile application, we adhere to the fundamental principles of testing discussed in this article. 

At the same time, we strive to adjust tests individually for each application, ensuring that each one is appropriately cared for. We create test documentation and provide tips and explanations so that our clients can feel that the quality of their applications is taken care of at all stages of development. We also have QA Plans, which were created based on our experience and knowledge. Contact us, so we can find the best solution for you!

Mobile app testing – summary

In summary, testing is key to ensuring product quality and success. It is a vital stage that allows for identifying and eliminating errors, ensuring that the application functions as users expect. At the same time, our trust in the quality of the application is growing.

Proper testing also ensures optimal performance and responsiveness of the application across different mobile devices. As a result, a reliable testing process is an important element in creating user-friendly and reliable applications that meet their expectations and enhance their daily lives.

Related posts