Automate what a script can check the same way every time: critical user flows clicked end to end, failed requests, console errors, accessibility rules, layout at each viewport. Keep manual what needs judgment: exploratory testing, whether a flow makes sense, how it works with a screen reader. Our QA Automation Foundations package does the first part for your 5 critical flows, running in CI with accessibility checks, for USD 5,000 in two weeks.
We do QA, manual and automated, on the products we build and for clients, and the rules below come from there. Here is the shape of a test we’d actually write, and where I’d stop automating.
What’s the difference between QA and testing?
Quality assurance is the whole practice of making sure software does what it should: how requirements get written, how changes are reviewed, what gates a release. Testing is one activity inside it, which is running the software and checking the result. A team can have a lot of tests and weak QA, if the tests check the wrong things or nobody acts when they fail.
It matters when you outsource. If you buy “testing”, you get test cases. If you buy QA, you should get fewer defects in production. If you want to buy it that way, QA Automation Foundations is the fixed-price package we start with.
Will AI replace QA?
It won’t replace the part that takes judgment. AI already helps write test code and generate test data, and we use it that way. But deciding what matters enough to test, noticing that a flow is technically correct and still confusing, and owning the call that a release isn’t ready are human decisions.
Playwright’s own accessibility guide says it plainly: automated tests “can detect some common accessibility problems such as missing or invalid properties. But many accessibility problems can only be discovered through manual testing.” That holds well beyond accessibility.
What should you automate with Playwright?
Anything a script can check identically on every run and that would hurt if it broke unnoticed. Start with the critical flows, clicked like a user: sign up, log in, submit the form that makes you money. The assertion is that the user gets to the end. While the flow runs, watch the network (any 4xx or 5xx, any failed request) and the runtime (uncaught exceptions and console errors, which a screenshot never shows). Add the accessibility rules axe can check, like missing labels, contrast and duplicate IDs. Run it all at each viewport you support, at minimum one desktop width and one phone width.
A test that only takes a screenshot proves the page rendered. A test that walks the flow and fails on any error proves it works. This is the shape we use:
import { test, expect } from "@playwright/test";
import AxeBuilder from "@axe-core/playwright";
test("contact journey works and is accessible", async ({ page }) => {
const problems: string[] = [];
page.on("pageerror", (e) => problems.push(`exception: ${e.message}`));
page.on("console", (m) => m.type() === "error" && problems.push(`console: ${m.text()}`));
page.on("requestfailed", (r) => problems.push(`failed: ${r.url()}`));
page.on("response", (r) => r.status() >= 400 && problems.push(`${r.status()}: ${r.url()}`));
await page.goto("/");
await page.getByRole("link", { name: "Contact" }).click();
await page.getByLabel("Email").fill("qa@example.com");
await page.getByRole("button", { name: "Send" }).click();
await expect(page.getByRole("status")).toContainText("Thanks");
const axe = await new AxeBuilder({ page })
.withTags(["wcag2a", "wcag2aa", "wcag21a", "wcag21aa", "wcag22aa"])
.analyze();
expect(axe.violations).toEqual([]);
expect(problems).toEqual([]);
});
Run it once per viewport with Playwright projects in playwright.config.ts, for example one at 1440 × 900 and one at 390 × 844. The wcag22aa tag matters. WCAG 2.2, a W3C Recommendation since October 2023 and updated in December 2024, added criteria such as 2.5.8 Target Size (Minimum) at level AA. axe-core tags its rules by standard.
The click-through on the marketing site we built for a govtech client has the same structure. On a local build, it passed at 1440 and 390 px wide, 10 steps, 0 failed requests, 0 console errors, and axe reported 0 violations on the pages judged.
What should stay manual?
What a script can’t judge. Exploratory testing comes first: a person with a goal and a time box, trying to break the feature in ways nobody wrote down. Then the question “does this make sense?”, because a flow can pass every assertion and still confuse people. Screen readers and keyboard-only use, end to end, are also manual. Automated rules catch missing labels, and they don’t tell you whether the experience works when you can’t see the screen. And new features whose behaviour hasn’t settled. Automating a flow that changes every day gives you tests that get deleted.
My rule is to automate the checking and keep the testing human. Checking confirms what you already know should be true. Testing finds what you didn’t know to ask.
How much does QA outsourcing cost?
With us there are two ways to buy it. QA Automation Foundations is USD 5,000 fixed, two weeks, for a Playwright suite over your 5 critical flows in CI plus accessibility checks. Or you take a QA engineer by the hour through staff augmentation: USD 30 for junior, 35 to 40 for mid, 45 to 50 for senior.
For context on US pay, the Bureau of Labor Statistics reports a median hourly wage of USD 50.14 for software QA analysts and testers in May 2025. That’s a wage, which leaves out the employer’s full cost, so it doesn’t compare directly with a billed rate. Our rates post explains how to compare them properly, and the pricing page has everything in one place.
What tools do we use for test automation?
For web products our default is Playwright for browser flows (Chromium, Firefox and WebKit from one API) and @axe-core/playwright for accessibility rules inside the same tests. Logic goes to the project’s own unit runner (Jest, Vitest or the language’s native one). The seams between components get integration tests against a real local server and database.
Where a product needs more, we add it. On the portal of an AI marketing platform we built, the Playwright flows also run with keyboard-only navigation, in dark mode, with forced colours and at 400% zoom, plus design-fidelity checks against approved Figma frames. On a PropTech housing platform, browser flows cover 13 variants, with pixel-fidelity checks against the approved frames and end-to-end proofs against a real local server.
Is test automation worth it?
When the tests fail for real reasons. A suite that is flaky, checks the wrong thing or runs so slowly nobody runs it just costs money. Four habits keep a suite honest.
Assert on the outcome, not the wrapper: “the status message says Thanks” beats “the button was clicked”. Collect console errors and failed requests in every test, as above, so errors you can’t see still fail the run. Use roles and labels as selectors, since getByRole and getByLabel break when accessibility breaks, which is what you want. And never weaken a test to turn it green. A test changed to pass without a changed requirement is a deleted test with extra steps.
What are the types of software testing?
By level, from the bottom up. Unit tests check one function or module in isolation. Integration tests check several components together, ideally with their real dependencies. End-to-end tests check the whole application the way a person uses it, and acceptance tests check it against the business criteria. By goal, tests are functional (does it do what it should?) or non-functional (accessibility, performance, security). A healthy suite has many unit tests, enough integration tests and a few end-to-end flows chosen with care. Those are the slowest to run and the most expensive to maintain.
How do I outsource QA without losing control?
Ask any vendor, us included, for three things in writing. Where the tests live: in your repository, running in your pipeline, readable by your team. What fails the build: which checks block a release and which only report. And what a “pass” covers: which flows, viewports and browsers, so a green run means something specific. If those answers are vague, the reports will be too.
If you want to see how this would look on your product, name your 5 critical flows and start QA Automation Foundations: USD 5,000, 2 weeks.
Sources, accessed 2026-10-06: W3C, WCAG 2.2, axe-core API documentation, Playwright, accessibility testing, BLS Occupational Employment and Wage Statistics (SOC 15-1253).