Smoke Testing: What It Is, When to Run It, and Best Practices

Smoke testing is the check that decides whether a build is worth testing at all. It runs first, it runs fast, and it gives you a straight yes or no before anyone spends an afternoon on a build that was broken from the start.

This guide covers the practical side: what to put in a smoke suite, how the accept and reject flow actually works, and how to automate it in your pipeline, including the API and agile-sprint cases that come up most often.

What is smoke testing?

Smoke testing is a preliminary test run that verifies the basic and critical functionality of a software build is working. It answers one question: is this build stable enough to justify further testing?

The scope is deliberately wide and deliberately shallow. You touch every critical path once and you check only that it completes. You are not verifying that a discount calculates correctly or that an error message has the right wording. You are verifying that the checkout page loads and an order can be placed at all.

In software testing you will also see it called build verification testing (BVT), build promotion testing, or confidence testing. These all describe the same practice. The name varies by team and by tooling, and none of the variations imply a different method.

What to include in a smoke test

The selection rule is simple to state and easy to get wrong: include a path if the application is unusable without it, and exclude everything else.

Take a SaaS dashboard as an example. A reasonable smoke suite checks that the application loads at its main URL without a blank screen or a crash, that a known test user can log in with valid credentials, that the main dashboard renders its navigation, header, and content area, that the primary create-or-save action completes, and that the key third-party integration responds. That is five checks, each one shallow.

What stays out matters just as much. No password reset flow, no multi-factor prompts, no permission matrix, no field validation, no expired sessions, no unusual inputs. Every one of those is worth testing somewhere. None of them belongs here, because each one you add costs runtime, and a smoke suite that stops finishing in minutes is one people start skipping. Set an explicit time budget and treat it as a limit rather than a target.

If you are building a suite from scratch, start by listing the things a user fundamentally cannot work around: signing in, navigating, and whatever your product exists to do. That list is usually shorter than teams expect.

How smoke testing works

The flow is short and the outcome is binary. A new build is produced, commonly right after code review, and the smoke suite runs against it. If the suite passes, the build is qualified for further functional and regression testing. If it fails, the build is rejected and handed straight back to the development team, and no further QA time goes into it.

That rejection is the mechanism doing the work. A smoke failure that files a ticket while the pipeline carries on regardless is not a gate, it is a notification. Decide in advance that a failure stops the build, and write that into your pipeline configuration rather than leaving it to whoever happens to be on duty.

Two things make that loop work in practice. The suite has to run automatically, so nobody has to remember to start it, and its results have to live somewhere the whole team can see, so "did this build pass smoke" has an answer rather than becoming an argument.

How to automate smoke testing

Smoke testing is the strongest automation candidate most teams have. The suite is small, its scope barely changes from month to month, and it runs on every build, so the payback arrives quickly and keeps arriving. If you are deciding what to automate first, this is usually the answer.

Wire the suite into your CI/CD pipeline

Schedule the smoke suite on your CI/CD pipeline so it triggers automatically whenever a new build is created. Place it after the build step and before anything is deployed to a test environment, and configure a failure to stop the pipeline.

This is the arrangement that gives you the most value for the least ongoing effort. It runs without anyone asking, it blocks unstable builds before they reach QA, and it guarantees the critical paths are checked on every build rather than whenever someone gets to it.

The other half of the setup is where the result lands. If a pass or fail only exists in a pipeline log, nobody outside the build system can see it. Holding run history in a shared place, such as Katalon Test Management in Katalon True Platform, is what turns the smoke result into something a release decision can actually reference.

Smoke testing an API

API smoke tests follow the same principle with a narrower surface. Pick the endpoints the system cannot function without, usually authentication, one core read, and one core write, then assert on status code and a minimal shape of the response body rather than on full payload correctness.

A useful API smoke suite answers whether the service is up, authenticating, and able to complete a round trip. Schema validation, pagination edge cases, and error-code coverage belong in your functional API suite, not here. Because API checks avoid browser startup entirely, they are typically the fastest smoke tests you can run, which makes them a good candidate for the earliest stage of a pipeline.

Smoke testing in agile sprints

On a team shipping multiple builds a day, manual smoke testing stops being viable quickly, and the failure mode is predictable: the suite gets skipped under time pressure, usually on exactly the build where it would have caught something.

Two habits keep it healthy across sprints. Review the suite at the end of each sprint and add a check only when a genuinely new critical path has shipped, so the suite tracks the product instead of drifting away from it. And treat the smoke result as a definition-of-done condition rather than a separate QA task, which stops it becoming something to negotiate away.

Choose a framework and a runner

Match the framework to your application's architecture rather than to preference. Selenium and Playwright cover browser UI, Appium covers mobile, and REST Assured or a similar HTTP client covers backend APIs. Katalon Studio spans web, API, mobile, and desktop in one project, which matters more than it sounds for a smoke suite: the critical paths of most products cross at least two of those surfaces, and keeping them in one suite is simpler than stitching three frameworks together. For breadth of browser and device coverage without maintaining that infrastructure yourself, Katalon's test execution cloud handles it.

If your framework has no built-in execution management, add a runner such as TestNG, JUnit, NUnit, or Cucumber. Runners are what give you parallel execution, grouping, data providers, and reporting, and parallel execution in particular is often what keeps a growing smoke suite inside its time budget.

Manual, automated, or hybrid

Fully manual smoke testing still has a place on projects with a single daily build or a UI in heavy flux, where a person's judgment catches things an assertion would not. It stops scaling as soon as build frequency rises.

Automated smoke testing is the default for anything shipping regularly, for the reasons above. A hybrid approach, where automation handles the stable critical paths and a tester reviews the one area currently in flux, is a reasonable middle ground and often the practical answer when a fully automated suite is still being built out. If budget or headcount is the constraint, note that automated and hybrid approaches usually cost less over a quarter than running everything manually, because the same suite runs on every build without consuming a person's time each cycle.

Where smoke testing fits alongside sanity and regression testing

Smoke testing is the first of three checks that get confused with each other, and the distinction is mostly about scope and timing.

Sanity testing runs after a specific fix or small code update and checks only the module that was modified, along with whatever sits immediately next to it. Where smoke testing is broad and shallow across a whole build, sanity testing is narrow and deeper on one change. If you need to decide which of the two a given change calls for, our guide to smoke testing vs sanity testing works through the triggers case by case.

Smoke testing vs regression testing

Both check that a build is sound, but they sit at opposite ends of the effort scale. Smoke testing is a fast gate that confirms core features load. Regression testing goes deep to confirm that recent changes have not broken existing functionality, covering full feature sets, edge cases, data behavior, and integration paths.

The two are complements, not alternatives. Smoke testing is what protects the regression run from being wasted on a build that was never going to survive it.

What smoke testing gives you, and where it falls short

The value is concentrated in a few places. You get an early read on build stability, so major failures surface within minutes of a build rather than hours into a test cycle. Developers get fast feedback while the change is still fresh, which is when it is cheapest to fix. QA time stops being spent on builds that were never viable. And because the suite is small and stable, it automates cleanly and produces consistent results run after run.

The limits are worth being equally clear about. Smoke testing only looks at the paths you chose, so anything outside that set passes through untouched by definition. A suite that has not been maintained alongside the product creates false confidence, which is worse than no suite at all because people trust it. Run manually against several builds a day, it becomes a bottleneck, and results start varying between testers. And if it is not automated and wired into CI, it tends to be skipped exactly when the schedule is tightest.

None of those are arguments against smoke testing. They are arguments for keeping the suite small, current, and automated.

Smoke testing best practices

Four habits separate a suite that earns its place from one that quietly rots.

Focus on the essentials. Center the suite on core paths only. This is what keeps it fast, and speed is what makes it usable as a gate on every build.

Keep the tests simple. Straightforward steps confirming basic behavior, with no attempt to cover variations. Anyone on the team should be able to read a smoke test and know what it proves in a few seconds.

Automate as soon as it runs often. Once a suite runs more than once a day, automation removes the repetitive work and the chance of a skipped run, and delivers consistent results faster than a person can.

Integrate with CI/CD and enforce the result. Connect the automated suite to the pipeline so every build is checked before deeper testing, and make a failure stop the pipeline. Test early and often across every sprint rather than saving it for a release candidate.

Why is it called smoke testing?

The name comes from hardware. Engineers would power up a new board and watch for literal smoke, because smoke meant a short circuit and no point continuing. Software borrowed the metaphor unchanged: switch the build on, see whether it catches fire, and only carry on if it does not.

Getting started

A useful smoke suite is smaller than most teams first assume. List the paths a user cannot work around, write one shallow check for each, run it on every build, and make a failure stop the pipeline. That is the whole practice.

The two decisions that determine whether it stays useful are both about restraint: keep the suite inside its time budget, and review it each sprint so it tracks the product rather than drifting behind it. Get those right and smoke testing quietly saves your team from the most expensive kind of wasted work, which is thorough testing of a build that was broken before anyone opened it.