Mobile App Testing: How to QA a Mobile Product Properly

Mobile app testing is where a lot of otherwise disciplined teams quietly lower their standards. The same people who would never merge backend code without tests will ship a release after tapping through the app on one handset, because mobile testing has a reputation for being slow, flaky and expensive to set up. That reputation is half-earned. The slowness and flakiness are real, but both are mostly consequences of testing at the wrong level, and they are fixable.

This guide covers how to structure a mobile QA process that catches real bugs: what to test at which level, how to choose between the main frameworks, when a real device is genuinely necessary, and the Play Store rule that now forces new developers to run a testing programme whether they planned to or not.

What mobile app testing actually has to cover

Mobile widens the definition of “does it work” more than most teams account for. Google’s Android testing fundamentals splits it into four questions, and they are a good checklist for any platform:

• Functional. Does the app do what it is supposed to?
• Performance. Does it do it quickly and without draining the battery?
• Accessibility. Does it work with screen readers and accessibility services?
• Compatibility. Does it work across devices and OS versions?

That last one is the mobile-specific tax. A web app targets browsers that mostly behave. A mobile app targets a spread of screen sizes, OS versions, manufacturer skins and hardware capabilities you do not control, and the bugs that reach production are disproportionately compatibility bugs rather than logic bugs.

The other mobile-specific category worth naming: interruption and state. Phone calls, notifications, backgrounding, the OS killing your process to reclaim memory, losing signal mid-request, permission prompts, and rotation. Most crash reports that baffle a team on Monday morning trace back to something in that list, and almost none of it gets exercised by tapping through the happy path on your own handset.

The test pyramid, applied to mobile

The single biggest lever on mobile QA is testing at the lowest level that can answer the question. Google’s own guidance describes three tiers by scope: unit tests covering a method or class, medium tests covering integration between units, and end-to-end tests covering a whole screen or user flow.

The distinction that matters for speed is a different axis: local versus instrumented.

• Local tests run on your machine. They are fast, they are stable, and there is no device involved.
• Instrumented tests run on a device or emulator, with your app installed alongside a test app that drives it. These are usually the UI tests, and they are where the time goes.

The mistake is treating those axes as the same. Not every unit test has to be local and not every end-to-end test has to run on a device. Tools like Robolectric let you run device-flavoured tests locally, which is how teams claw back most of their feedback loop.

Google’s underlying point is architectural rather than about tooling: if your business logic lives inside Activities and Fragments, you are forced to boot a device to test anything at all. Splitting the app into presentation, domain and data layers, keeping logic out of framework classes, and injecting dependencies is what makes cheap tests possible in the first place. A slow test suite is usually a symptom of a coupled codebase, not of mobile being hard.

Choosing a framework

There are three serious options for UI automation, and the choice is mostly determined by one question: are you testing one platform or two?

Espresso (Android) and XCUITest (iOS) are the first-party frameworks. Both are free, both are maintained by the platform owner, and crucially both run in-process with the app, so they can synchronise with the UI thread and know when the app is idle. That architectural detail is why native tests need far fewer arbitrary waits and tend to be markedly less flaky. The cost is that they are single-platform and written in the platform language, so testing both means maintaining two suites.

Appium is the cross-platform option. One suite, one language, both platforms, plus web and hybrid apps. It works by driving the OS from the outside through the WebDriver protocol, which is exactly what makes it flexible and also what makes it slower and historically flakier than native tooling, since it is inferring app state rather than sharing a process with it.

Maestro is the newer entrant, defining flows in YAML with built-in tolerance for timing. It is open source, aimed at developer-led testing where a full automation stack is overkill.

A reasonable default: if you are one platform, use the native framework. If you have two platforms and dedicated automation engineers, Appium’s shared suite may pay for itself. If you have two platforms and no automation specialist, keep the automated flows short and treat them as smoke tests rather than a safety net you trust with everything.

Be sceptical of framework benchmarks generally, since most published flakiness and speed comparisons come from vendors selling one of the tools. The architectural argument above holds regardless of whose numbers you read.

Real devices versus emulators

Emulators are for iteration. Real devices are for the truth.

Emulators and simulators are fast, free, scriptable and fine for the bulk of your suite, which is checking that logic and layout behave. What they cannot honestly reproduce is the set of things that actually break in the wild: real GPU and CPU behaviour under thermal load, battery drain, camera and sensor input, biometric prompts, patchy real-world networks, manufacturer skins that alter system UI, and the memory pressure that makes Android kill your process.

The pragmatic split most teams land on is to run everything on emulators in CI on every commit, then run a smaller smoke suite on a handful of real devices before release. Choosing which devices is a data question, not a taste question: pull your actual install base from the Play Console or App Store Connect and cover the top handful of devices and OS versions your users genuinely have, plus the oldest OS version you still support. Device cloud services rent real hardware if maintaining a device drawer is not realistic, though they are priced per parallel test and get expensive quickly.

The Play Store rule that forces the issue

If you are publishing a new Android app, testing is no longer optional or self-directed. It is a gate.

Google requires personal developer accounts created after 13 November 2023 to run a closed test before they can apply for production access. The Play Console requirements are specific: at least 12 testers must be opted in to your closed test, and they must have been opted in for the last 14 days continuously. Google reduced this from 20 testers in December 2024. Once you meet the criteria you apply for production access and answer questions about your testing process, with review typically taking 7 days or less.

Two things catch people out. The requirement is continuous, so dropping below 12 opted-in testers resets your clock rather than pausing it. And adding 12 email addresses is not the same as having 12 testers: each person has to actually opt in, which is a step they have to complete themselves.

Plan this into the schedule. A fortnight of closed testing, plus review, sits between a finished build and a public launch, and finding out about it the week you intended to ship is a self-inflicted delay.

A workable process

Nothing here requires a QA department:

1. Write local tests for logic, and structure the app so that is possible. This is where the coverage should live, because it is the only tier that stays fast.
2. Automate a short list of critical flows end to end. Sign-up, log-in, purchase, the one screen that earns the money. Not everything.
3. Test the interruptions deliberately. Background the app mid-flow, kill it, rotate it, deny the permission, turn off the network. This is manual, it takes ten minutes, and it finds real bugs.
4. Run a real-device smoke test before release on the devices your users actually hold.
5. Check accessibility, with a screen reader turned on, in the same pass.

If you are assembling the tooling around this, our best A/B testing tools roundup covers the experimentation side of shipping to mobile users, and testing augmented reality and VR goes into device testing where the hardware itself is the variable.

Frequently asked questions

What is mobile app testing? Mobile app testing is verifying that an app works correctly across the devices and conditions real users have. It covers four areas: functionality, performance, accessibility and compatibility across devices and OS versions, plus mobile-specific states such as interruptions, backgrounding and lost connectivity.

What is the best mobile app testing framework? It depends on platform coverage. Espresso for Android and XCUITest for iOS are free, first-party and run in-process with the app, so they are faster and less flaky. Appium covers both platforms with one suite at the cost of speed and stability. Maestro suits developer-led testing with simpler YAML flows.

Do I need real devices, or are emulators enough? Both. Use emulators for the bulk of your automated suite because they are fast and cheap, then run a smaller smoke suite on real devices before release. Emulators cannot faithfully reproduce thermal behaviour, battery drain, sensors, biometrics, manufacturer skins or real network conditions.

How many testers does Google Play require? At least 12 testers, opted in continuously for the last 14 days, for personal developer accounts created after 13 November 2023. This was reduced from 20 testers in December 2024. The count must stay at 12 or above throughout, as dropping below resets the period.

Why are mobile UI tests so flaky? Usually because they are running at the wrong level and the framework is inferring app state from outside the process. Native frameworks synchronise with the app’s UI thread and need fewer arbitrary waits. Moving assertions down to local tests, and keeping end-to-end tests to a short list of critical flows, removes most flakiness.

What should I test that most teams miss? Interruption and state handling. Incoming calls, notifications, backgrounding, the OS killing the process under memory pressure, permission denial, rotation and connection loss mid-request. These account for a large share of production crashes and almost never appear in happy-path testing.