Smoke testing: quick checks after a release

A smoke test is a short, shallow test that answers one question: is this build working well enough to use and test further? It takes minutes, and it is the first thing to run after any deploy.

Who it’s for

  • Developers deploying to production or staging
  • Teams opening a new beta round
  • Anyone who wants a fast "is the site up and working?" check

Updated 2026-10-19.

Smoke testing is the quickest test you can run on a new build: a handful of checks that tell you whether the product basically works before anyone spends time on deeper testing. Run it after every deploy and before every beta round, and it will catch the embarrassing failures before your users or beta testers do.

What a smoke test is

The term comes from hardware: switch it on and see if smoke comes out. In software, a smoke test checks the most basic, critical functions of a build right after it is deployed. If the smoke test fails, there is no point running deeper tests or inviting beta testers until it is fixed.

A smoke test is broad and shallow: it touches every critical area once, without exploring edge cases.

What a smoke test covers

  • The site or app loads; the main pages return without errors.
  • Signup and login work.
  • The main flow can be started and completed once.
  • Key pages show real data, not errors or blank states.
  • Navigation and the most important links work.
  • No new console errors or failed requests on the main pages.
  • Payments: the checkout page loads (tested in test mode only).
  • The main page renders correctly on a phone.

How to set up smoke testing, step by step

  1. List your critical paths. Usually: home page, signup, login, the main task, and upgrade or checkout.
  2. Write one pass/fail check per path, each with a clear expected result.
  3. Prepare a test account with realistic data, and a test-mode payment setup.
  4. Order the checks so the most basic go first: if the site does not load, nothing else matters.
  5. Time it. Aim for a run that takes minutes, not an afternoon.
  6. Decide who or what runs it after each deploy, and where results go.
  7. Define the stop rule: any failure blocks further testing and the beta until fixed or rolled back.
  8. Review the list whenever you add a critical feature or a smoke check misses an outage.

Example smoke test checklist

Smoke test – production – after each deploy

[ ] 1. Home page loads in under a few seconds, no error banner
[ ] 2. Signup with a new email reaches the welcome screen
[ ] 3. Login with the test account reaches the dashboard
[ ] 4. Dashboard shows the account's existing projects (not empty, no error)
[ ] 5. Create a new project and see it in the list after reload
[ ] 6. Pricing page loads; "Upgrade" opens checkout (test mode)
[ ] 7. Main pages: no new console errors or failed requests
[ ] 8. Home page and dashboard usable at phone width
[ ] 9. Logout returns to the home page

Stop at the first failure. Report it, then decide: fix forward or roll back.

Example smoke test failure report

Title: Smoke test: login fails with "Something went wrong" after deploy

Environment: Chrome on Windows 11, production, deploy of today 14:05
Account: [email protected] (regular user)

Steps to reproduce:
1. Open /login
2. Enter the test account email and password
3. Click "Log in"

Expected: dashboard opens
Actual: red banner "Something went wrong"; the login request returns a 500 error
Severity: Critical (no existing user can log in)
Frequency: Always, also in Safari and on Android
Evidence: screenshot with the banner marked, console error copied

Smoke testing vs other tests

  • Smoke test: minutes, critical paths only, after every deploy.
  • Sanity test: a narrow check of one area that just changed.
  • Regression test: deeper re-testing of existing flows and past bugs.
  • Beta test: real-world use by outsiders, with feedback.

When to run a smoke test

  • Immediately after every production deploy.
  • After every staging deploy that testers will use.
  • Before opening a beta round.
  • After infrastructure, DNS, CDN or certificate changes.
  • On a schedule, to catch outages caused by third-party services.

Prioritizing smoke test failures

Any smoke test failure is at least major, because it means a critical path is broken. Fix in this order: the product does not load; login or signup fails; data does not save or shows wrongly; checkout or upgrade fails; mobile layout breaks the main flow. If the fix is not quick, rolling back the deploy is often the better choice.

How to keep it useful

  • Keep it short; if it takes long, it will be skipped.
  • Write it as a list of pass/fail checks.
  • Report only real problems; a smoke test is not a design review.
  • Fail fast: if login is broken, stop and report.

Common mistakes

  • Letting the smoke test grow into a full regression suite that nobody has time to run.
  • Testing only on staging when the outage is in production configuration.
  • Using an admin account, which hides permission bugs regular users hit.
  • Only checking that pages load, not that data shows and saves.
  • Running real payments instead of test mode.
  • Ignoring console errors because the page looks fine.

Running a smoke test manually vs with an AI beta tester

A manual smoke test is fine when you deploy rarely, but it is the first thing skipped on a busy day. You can describe the smoke test in a few lines and let an AI beta tester like betatest.ing run it in a real browser or on phones: open the main pages, sign in with the saved test login (stored encrypted and never shown to the AI), complete the main flow once, check the console, and report pass/fail per page. Scheduled after deploys or daily, it gives you a short report and only interrupts you when something is actually broken, with steps, an annotated screenshot and the error text. It stops before any real payment.

A smoke test, manual or AI, checks that critical paths work. It is not a load test or a security test, and it does not tell you what real users think. Keep those for dedicated tools and real beta testers.

The test task

What you send your AI beta tester. It works out the steps and tests like a real user.

Run a quick smoke test (about 10 minutes) on [YOUR APP URL] after a deploy. Open these main pages: [YOUR MAIN PAGES OR SECTIONS]. On each, check it loads, nothing is visibly broken, and there are no console errors or failed requests. Log in with the saved test login and complete the main flow once. Open the checkout or upgrade page but do not pay (test mode only). Check the home page at phone width. Stop and report immediately if login or the main flow is broken. Log only real problems as findings with steps to reproduce, the error text and an annotated screenshot. Report a short pass/fail table per page and flow.

Fill in your details

Here, or later in the chat with your tester.

  • [YOUR APP URL] The environment you just deployed.
  • [YOUR MAIN PAGES OR SECTIONS] e.g. "home, pricing, dashboard, settings".

What the AI beta tester checks

  • Each main page loads without errors
  • Login with the saved test account
  • The main flow, completed once
  • Console errors and failed requests
  • Checkout or upgrade page loads (no payment)
  • Home page at phone width

What you get

  • A pass/fail table per page and flow
  • Findings only for real problems, with error text
  • Annotated screenshots of anything broken
  • An immediate alert if login or the main flow fails

How to run it with betatest.ing

  1. Add your app as a project with a test login.
  2. List your main pages in the task.
  3. Run it after each deploy, or schedule it daily.
  4. Run a full regression test before big releases.

Tips

  • Keep a smoke test to minutes; depth belongs in regression tests.
  • Fail fast: a broken login means stop and fix.
  • Schedule it so third-party outages are caught too.

Smoke testing: FAQ

What is smoke testing?

A short, shallow test of the most critical functions right after a build is deployed, to confirm it works well enough to use and test further.

What is the difference between smoke and sanity testing?

A smoke test checks all critical areas briefly; a sanity test checks one area that just changed in a bit more depth.

How long should a smoke test take?

Minutes. If it takes much longer, move the deeper checks into a regression suite.

Can smoke tests run automatically?

Yes. You can schedule an AI beta tester to run a plain-language smoke test after deploys or daily and report only real failures.

Should smoke tests run in production?

Yes, a short smoke test right after a production deploy is common, because some failures only happen with production configuration, data or third-party services. Use a dedicated test account and keep payments in test mode or stop before paying.

What happens if a smoke test fails?

Stop further testing, report the failure with steps and evidence, and decide quickly whether to fix forward or roll back the deploy. Don't open a beta round or announce a release until the smoke test passes.

Who should write the smoke test?

Whoever knows the product's critical paths best, often a developer, product owner or QA engineer. Keep it as a short shared list of pass/fail checks that anyone on the team, or an automated or AI tester, can run.

Related beta testing guides

  • Regression testing after deploys — Regression testing checks that what worked yesterday still works after today’s change. For small teams shipping often, a short regression suite run after every deploy prevents most "it used to work" bugs from reaching users.
  • Broken links checker — Broken links and JavaScript errors are cheap to fix and expensive to leave: they break flows, hurt trust and waste crawl budget. This page shows what to collect, how to group it, and a task your AI beta tester can run across your site.
  • Beta testing checklist — Use this beta testing checklist to prepare, run and close a beta test without forgetting the basics. It works for websites, web apps and mobile apps, and you can hand the product part of it to an AI beta tester.
  • Website beta testing — A website beta test is a run-through of your site by someone who has never seen it, before real visitors do. This guide covers what breaks most often on marketing and content sites, how to set up the test, and a ready-to-run task your AI beta tester can start on right now.
  • Free tool: Test case generator — A table of test cases with steps, data and expected results, edge cases included. CSV and Markdown export.

More in guides