AI software testing glossary: 41 terms every QA and engineering team should know

Plain-English definitions for the terms that come up when teams talk about testing, test automation and AI in QA. Each definition is written to stand on its own, so you can link straight to a term.

Requirements and planning

Acceptance criteria. The conditions a user story must meet to be accepted, written as observable behaviour. Often phrased as Given / When / Then.

Requirement. A single, testable statement of what the software must do.

Requirement coverage. The share of requirements that have at least one test verifying them.

Requirements traceability. The links between requirements, the tests that verify them and their results. See our traceability matrix template.

Requirements traceability matrix (RTM). A table showing each requirement, its tests and their latest results.

Test plan. A document describing what will be tested, how, by whom and with which exit criteria.

Test scenario. A high-level description of what to test, such as "coupon codes at checkout".

User story. A short description of a feature from the user's point of view: "As a…, I want…, so that…".

Test design

Boundary value analysis. Testing values at the edges of a valid range and one step either side, where off-by-one errors cluster. See boundary test cases.

Equivalence partitioning. Grouping inputs the system should treat the same way and testing one value per group.

Expected result. The observable outcome a test case must see to pass.

Integration test case. A test that checks a feature working together with other components or services.

Negative test case. A test that uses invalid input and expects a clear, safe rejection with no side effects.

Positive test case. A test that uses valid input and expects success.

Test case. A precise, repeatable check: preconditions, steps, test data and one expected result.

Test data. The specific values and records a test uses.

Test oracle. The source of truth used to decide whether a result is correct, such as a requirement or a spec.

Test types

End-to-end (E2E) test. A test of a complete user flow through the real interface and its back-end services.

Exploratory testing. Unscripted testing where a person investigates the software to find unexpected problems.

Regression testing. Re-running tests after a change to make sure existing behaviour still works.

Smoke test. A quick check of the most important flows to decide whether a build is worth testing further.

Unit test. A test of a small piece of code, such as a function, in isolation.

Headed browser. A browser that runs with a visible window. A headless browser runs without one.

Automation and maintenance

Codeless test automation. Automated testing without writing test code, for example through recording or plain-language steps.

Flaky test. A test that passes and fails on the same code without changes. See flaky tests.

Quarantine (tests). Moving known-flaky tests out of the blocking pipeline, with an owner and a deadline to fix them.

Selector (locator). The expression a script uses to find an element on a page. Brittle selectors are a main source of test maintenance.

Self-healing test. A test that tries alternative locators when its original one breaks.

Test maintenance. The work of keeping existing tests correct as the product changes.

AI in testing

AI-assisted testing. Adding AI to one step of an existing testing process, such as suggesting locators.

AI-native QA. Testing in which AI runs the whole loop: reading requirements, generating tests, running them and reporting results traced to requirements. See what is AI-native QA.

AI test case generation. Using AI to produce test cases from requirements, user stories or specs.

Agentic testing. Testing performed by an AI agent that plans and takes actions, such as driving a browser, instead of executing a fixed script.

Autonomous testing. A mode where AI generates, runs and verifies tests on its own and people review the results afterwards. See copilot vs autonomous.

Copilot testing. A mode where people review AI-generated test cases and choose which ones run.

Grounding. Tying AI output to a trusted source, such as the project's requirements, so it doesn't invent behaviour.

Hallucination. AI output that is fluent but not supported by its source, such as an expected result the spec never states.

Human in the loop. A process design where a person approves or reviews AI decisions at a defined checkpoint.

Reporting and release

Escaped defect. A bug found in production that testing did not catch.

Pass rate. The share of executed tests that passed in a run.

Release readiness. Evidence that the changes in a release are tested, passing and safe to ship. See the release readiness checklist.

---

testdart is an AI-native QA platform that turns requirements into test cases, runs them in a real browser and reports every result against the requirement it proves. Learn more.