Your team spends three days on full regression because of a button label change.
And last week it shipped a database migration with no checks at all — “we only touched the backend”. If this sounds familiar, congratulations: you are mixing up smoke, sanity and regression. And it is not a terminology quiz for job interviews. It is the reason your releases are slow AND leaky at the same time. Quite an achievement — most teams only manage one of the two.
Smoke testing is a fast check that the build is alive at all: the app opens, the key functions work. It runs on every build and takes minutes. Sanity testing is a narrow check of one specific change: you fixed a bug — you verify the fix and its surroundings. Regression testing is a wide check that new changes did not break old functionality; it runs before a release. Smoke answers “is it alive?”, sanity answers “did the fix work?”, regression answers “did we break anything else?”.
What smoke testing is, in plain words
The term comes from hardware: plug the board in — if no smoke comes out, you can keep testing. Your “smoke” is: the app opens, login works, the main scenario (cart, payment, order creation — whatever feeds the business) goes through end to end. That's it. Five to ten checks, ten minutes.
Smoke does not hunt for bugs. Smoke answers exactly one question: is this build worth testing further — or is it dead on arrival. Teams without smoke find out about a dead build from customers. On Friday night. By phone.
What sanity testing is, in plain words
Sanity is a common-sense check after a targeted change. You fixed the discount calculation — you check the discount calculation and the prices around it, not the whole cart, delivery and account area. Narrow, deep, fast.
Sanity is what you do instead of a full regression when the change is small and you understand its blast radius. The key word is “understand”. If the developer answers “nothing, really” to “what could this affect?” and looks away — that is not sanity, that is a lottery.
What regression testing is, in plain words
Regression checks that the old did not break because of the new. Not the new feature — you will check that anyway, it is in plain sight. The old stuff. The things that worked for years, that nobody touched, and that fell over because of a migration, a library upgrade or someone else's refactoring.
Regression is the most expensive of the three: dozens or hundreds of scenarios. Manually — days. So teams start “optimising” it: trimming, skipping, running it once a quarter. The bugs, unfortunately, were not notified about your optimisation.
Comparison table: smoke vs sanity vs regression
| Smoke | Sanity | Regression | |
|---|---|---|---|
| The question | Is the build alive? | Did the fix work? | Did the old stuff break? |
| When | Every build / deploy | After a targeted fix | Before a release |
| Scope | Narrow, critical paths only | Narrow, around the change | Wide, all existing features |
| Time | Minutes | Minutes to an hour | Hours to days |
| Who runs it | Automation (ideally) | QA | Automation + QA |
Why you are running the wrong one
Now the uncomfortable part. A typical team does the exact opposite.
Full regression on every sneeze. A label change, a new icon — “well, we have to check everything”. Three days of QA time for a change with a blast radius of one button. The sprint burns, the team starts hating releases, releases become rarer — and the next one carries twenty changes. Which, of course, “all need checking”. Congratulations: you have built a perpetual pain machine.
No smoke at all. The build lands on staging, QA starts a three-day regression — and on day two discovers that login was broken from the start. Two days spent testing a corpse.
Sanity impersonating regression. “We checked the fix, we didn't touch anything else” — and a week later it turns out “anything else” included the report export everyone forgot about. Sanity without a periodic full regression is driving without mirrors: perfectly fine as long as you only go straight ahead.
All three have the same root: when a check is expensive, people start economising on it. And they always economise on the invisible — the old functionality that “works anyway”.
What the run pyramid should look like
Notice: items 1 and 3 are automation. Smoke and regression are the same checks repeated hundreds of times without changes — exactly the work you should take away from humans. In TestManager a scenario is recorded with Recorder once and then lives in both loops: the short set runs as smoke on every deploy, the full set as regression before each release, with reports you can read in a minute. Sanity and exploratory passes stay with people — where you need a head, not stamina.
When a manual run is still the right call
Honestly: automation does not replace everything. New features, complex user journeys, “let me eyeball whether the layout broke” — that is the hands and head of a live QA. But if your people are walking the same login checklist for the hundredth time, you are paying a salary for a robot's job. The robot, by the way, is cheaper and does not quit out of boredom.
FAQ
What is the difference between smoke and sanity testing?
Smoke is wide and shallow: the whole system at the “alive or dead” level. Sanity is narrow and deep: one change and its surroundings. Smoke runs on every build, sanity after a specific fix.
How many checks should a smoke suite have?
5–15. If your smoke takes more than 15 minutes, it is not smoke any more — it is a small regression. Cut it down to the scenarios without which the business stops.
How often should full regression run?
Before every release. If regression is automated — every night if you like. If it is manual and “once a quarter, because it is expensive” — that is not a schedule, that is a confession.
Can smoke and regression be automated without a programmer?
Yes. In TestManager scenarios are recorded with Recorder from real user actions, built on Page Objects and run on schedule or from CI/CD. A manual QA manages from day one.
Smoke failed — now what?
Stop testing. The build goes nowhere, the developer fixes it immediately. A smoke whose failure is “we'll look later” does not exist — you are just burning electricity.