Beta testing a mobile app: Android vs iOS, TestFlight limits, and what you can automate
Beta testing a mobile app on Android and iOS: TestFlight limits and review, Google Play testing tracks, platform differences, and what to automate. By betatest.ing team · September 19, 2026 · Updated October 10, 2026 · 7 min read.
Key takeaways
- TestFlight builds expire after 90 days, support up to 10,000 external testers, and external builds go through Beta App Review.
- Google Play offers internal, closed and open testing tracks; new personal developer accounts must meet closed-testing requirements before production.
- Permissions, notifications, deep links, background behavior and interruptions are where mobile betas find their worst bugs.
- Automate the repeatable device matrix work; keep humans for real-world conditions and store review readiness.
- Collect crash reports and in-app feedback automatically, because testers rarely write up crashes themselves.
In this article
Beta testing a mobile app is harder than beta testing a website for one reason: distribution. You cannot send a link and have someone use the new version in two seconds. Each platform has its own program, its own rules and its own limits. This guide covers what differs between iOS and Android, what to test on both, and which parts of the process you can hand to automation.
The distribution layer: TestFlight vs Google Play testing tracks
iOS: TestFlight
Apple's beta distribution runs through TestFlight. The key facts, from Apple's TestFlight documentation:
- Internal testing: up to 100 members of your App Store Connect team, no review required, builds available almost immediately after processing.
- External testing: up to 10,000 testers, invited by email or a public link. The first build of a version (and some later ones) goes through Beta App Review, which is lighter than App Store review but can still take a day or more and can reject.
- Build expiry: every build expires 90 days after upload. Testers lose access to expired builds, so a long beta needs regular uploads.
- Tester experience: testers install the TestFlight app, accept the invite, then see your builds. They can send feedback and screenshots from inside TestFlight, and crashes are reported to you with the tester's consent.
Practical consequences: plan your external beta so the first build is uploaded a few days before you need testers on it, keep internal testers for daily builds, and set a reminder for build expiry. Our iOS app beta testing guide covers the setup steps.
Android: Google Play testing tracks
Google Play Console provides several tracks, described in Google's Play Console help:
- Internal testing: up to 100 testers, builds available in minutes, no review. The fastest loop.
- Closed testing: invite-only via email lists or Google Groups; you can run several closed tracks at once (for example one per region or customer segment).
- Open testing: anyone can join from the Play Store listing; you can cap the number of testers.
- Pre-launch reports: Google runs your build on a set of real devices in its own lab and reports crashes, accessibility issues and screenshots. Worth reading before every release.
One more rule that catches new developers: personal developer accounts created after late 2023 must run a closed test that meets Google's requirements (a minimum number of testers opted in for a minimum period) before they can publish to production. Check the current numbers in Play Console, because they have changed before, and start the closed test early.
Our Android app beta testing guide walks through track setup and the APK vs App Bundle question.
Side by side
| iOS (TestFlight) | Android (Play Console) | |
|---|---|---|
| Internal testers | 100 | 100 |
| External testers | 10,000 | Closed: list-based; Open: capped as you choose |
| Review before external testers | Beta App Review | None for internal and closed; policy checks apply |
| Build expiry | 90 days | None, but old builds are superseded |
| Built-in feedback | Yes, in the TestFlight app | Via Play Store "send feedback" and pre-launch reports |
| Device lab | No | Pre-launch report on Google-managed devices |
What differs when you are actually testing
Devices and fragmentation
iOS has a small number of devices and most users are on a recent OS. Android has thousands of device models, several OS versions in wide use, and manufacturer customizations (battery optimizers, custom permission dialogs, different default browsers for web views). An Android beta needs more device variety; an iOS beta needs more attention to the latest OS beta, since users update quickly.
Permissions
Both platforms gate camera, location, contacts, photos and notifications behind runtime permission prompts, but the flows differ. iOS asks once and sends users to Settings afterwards; Android allows re-asking under some conditions and has "only this time" options for some permissions. Test every permission in three states: granted, denied, and denied-then-granted from Settings. Apps that assume "granted" crash in the other two.
Notifications
On iOS, notification permission must be requested explicitly. On Android 13 and later it is also a runtime permission. Test: the prompt timing, what happens when the user declines, deep links from a notification into the right screen when the app is cold, backgrounded and foregrounded, and badge counts.
Deep links and universal links
Both platforms let a web URL open your app. Both require server-side files (an Apple App Site Association file and a Digital Asset Links file) and both fail silently when those are wrong. Test links from email, from a messaging app and from the browser, with the app installed and not installed.
Background behavior and interruptions
Receive a call mid-flow. Switch apps and come back after ten minutes. Lock the screen during an upload. Lose the network in a tunnel. Rotate the device on the payment screen. These are the moments mobile betas find bugs that no simulator run ever shows, and they are the ones to assign explicitly to human testers because they happen in real life.
Payments
If you sell digital goods inside the app, both stores require their in-app purchase systems and both have sandbox environments: App Store sandbox tester accounts on iOS and license testers on Google Play. Test purchase, restore, cancellation and the subscription renewal cycle, which both sandboxes accelerate so you can see a "month" pass in minutes.
A mobile beta test plan that fits in two weeks
- Week 0: setup. Create tracks, add internal testers, upload a build, write the welcome note and the first task. Confirm crash reporting and analytics work in the beta build. The beta test plan generator gives you a plan skeleton to fill in.
- Days 1–3: internal round. Your team and a few friendly users on the most common devices. Fix the obvious.
- Days 4–10: external round. Invite the real testers. Give one clear first task, then a list of flows to try, then a short survey. Watch crash reports daily.
- Days 11–13: fix and re-test. Ship a new build, re-run the affected flows on the devices where bugs were found.
- Day 14: decision. Compare against the exit criteria you wrote in week 0, and check the store-specific pre-submission lists.
The mobile app beta testing guide expands on each stage.
What you can automate
Mobile beta testing has a large repeatable component, and that is the part worth automating.
Device matrix runs. Running the core flows on several device sizes and OS versions, every build. An AI beta tester can do this on emulated or real devices, following goals rather than brittle scripts, and it produces screenshots and recordings per device so layout problems are visible rather than described.
Smoke tests on every build. Install, launch, log in, reach the main screen, perform the one action the app exists for. Ten minutes of automation per build catches the "we shipped a build that cannot log in" class of problem before a tester does.
Crash and error collection. Use the platform crash reporting (and a third-party tool if you need more context) so you do not depend on testers writing up crashes. They rarely do.
Accessibility checks. Both platforms have accessibility scanners (Accessibility Inspector on iOS, Accessibility Scanner on Android) and Google's pre-launch report includes accessibility findings. Run them on every build; fix labels and contrast before human testers with assistive technology get involved.
Regression after each fix. Each new build should re-run the flows the previous build broke. Scheduling this is more reliable than remembering it.
What to keep for people
- Real-world interruptions and network conditions.
- First-run impressions and whether onboarding makes sense.
- Whether notifications feel helpful or annoying after three days.
- Store-readiness judgment: screenshots, description, age rating, privacy labels.
- Anything requiring a real identity: real SMS codes, real bank connections, real contacts.
Common mobile beta mistakes
- Letting TestFlight builds expire mid-beta.
- Not starting the Google Play closed test early enough to meet the production requirements.
- Testing only on the developers' personal phones, which are always high-end and always up to date.
- Shipping a beta build with debug logging that leaks data into crash reports.
- Collecting feedback with no device or build information attached.
- Fixing a crash and not re-testing on the device that produced it.
How betatest.ing handles mobile apps
You add your Android or iOS app as a project and the AI beta tester runs beta tests on phones, following goals like "sign up, enable notifications, complete a purchase in sandbox". It files bugs with steps, severity, annotated screenshots and recordings, scores each run, exports the report to PDF or Excel, and can re-run the same tests on a schedule for every new build. Use it for the device matrix and the regression grind; use TestFlight and Play testers for real-world conditions.
Start for $1 and run your first mobile beta pass today.
Related guides
Each ends with a test task you can send to your AI beta tester.
Keep reading
- iOS app beta testing with TestFlight: what nobody tells you — TestFlight is simple until it is not: builds expire after 90 days, external groups wait on Beta App Review, feedback screenshots arrive in an odd place, and the simulator cannot run TestFlight at all.
- Android app beta testing: a step-by-step guide for 2026 — From the first internal build to production: Play Console tracks, the closed-testing rule for new personal accounts, which devices to cover, what to test on each build, and where an AI tester fits.
- AI mobile app testing: what an AI beta tester can and can't do on a phone — An AI beta tester on a phone sees the accessibility tree and the screen, taps, types, reads crash logs and records everything. It cannot be a real camera, notification or wallet. Here is the line.