Unit 6 / 12

Test Automation and Quality Assurance

Gains:

  • Ability to produce unit, integration and edge case tests with meaningful assertion with AI
  • Ability to systematically extract test coverage, limit values and negative scenarios with AI support
  • Ability to verify that the tests produced by the AI actually verify behavior and do not just repeat existing code

Testing is the mechanism that proves that the software actually behaves as promised. A good test suite tells you in seconds whether a change breaks something and gives the engineer the freedom to act with confidence. AI speeds up the most tedious and most skipped part of test writing: generating a multitude of scenarios, breakpoints, and negative cases. But there's a sneaky trap here: AI may write tests that verify the code's current (perhaps faulty) behavior, not its supposed behavior; or it can produce empty tests that always pass, not actually checking anything. The value of a test is not in whether it passes, but in whether it checks for the right thing and turns red when it's wrong.

In this unit, you will learn how to produce unit, integration and edge case tests with meaningful assertions; how to systematically extract test coverage, breakpoints, and downside scenarios; and we'll see how you can check that the tests the AI ​​produces actually validate behavior.

Concepts: Unit testing: Tests a single function/class in isolation. Integration testing: Tests that multiple parts work together correctly. Assert: A statement that checks that a result is equal to what was expected; This is the heart of the test. Coverage: How much of the code is run by tests; High coverage does not guarantee quality.

Producing Meaningful Tests

A good test does three things clearly: it establishes a state, it performs an action, it asserts the result. When printing tests to the AI, specify what behavior you want to verify and what scenarios it should cover; Otherwise, it produces superficial tests that always pass.

  1. Define the behavior to be tested. “What counts as right?” Answer the question clearly.
  2. Ask for scenario types. Normal, limit, negative, error condition.
  3. Import meaningful assert. It didn't just "throw an error", it "returned the correct value".
  4. Check the accuracy of the test. Does the test turn red when you break the code consciously?

Comprehensive test generation prompt: "Write unit tests for the following 'applydiscount(amount, coupon)' function. Have AT LEAST one scenario in the following categories: (1) normal valid coupon, (2) breakpoints (0 amount, 100% discount), (3) negative (invalid coupon, negative amount), (4) error case (null coupon). Assert the CONCRETE expected value in each test (not just 'worked'). Name the tests readable. Code: [code]"

Boundary value extraction prompt: "Perform a boundary value analysis for the inputs of this function. For each parameter, extract the values 'just at the boundary', 'just below the boundary', 'just above the boundary' as a table. Then list the test scenarios that cover these boundaries. Don't write code yet, just analysis and scenario list. Function: [signature]"

Caution: High test coverage (e.g. 90%) does not prove that the code is correct. Coverage measures how many rows were executed; not that those lines produce the correct result. A test without meaningful assert increases coverage but doesn't guarantee anything. The content of the assert determines quality, not the number of assertions.

Testing the Test Itself: The Logic of Mutation

The most practical way to understand whether the AI-generated test actually works is to deliberately break the code (mutation testing logic). Reverse a condition, make a + sign -; If no tests turn red, your tests are not actually maintaining that behavior.

Test vulnerability hunting prompt: "Tell me which potential bugs in this code the following tests MAY NOT catch. Suggest 5 small mutations that could be made to the code (e.g. >= instead of >, - instead of +) and indicate for each whether existing tests would catch it. For those not caught, suggest testing that should be added. Code: [code] Tests: [test]"

Weak Prompt / Strong Prompt

WEAK: "Write a test to this function." (Result: usually one happy scenario, weak assert; misses errors.) STRONG: "Write a test to this 'passwordStrong' function. Rule: at least 8 characters, 1 uppercase letter, 1 digit required. Cover the following scenarios as SEPARATE tests: exactly 8 characters (limit), 7 characters (below limit), no uppercase letters, no digits, empty string, only spaces, too long (1000 characters) Explicitly assert the expected true/false value in each test and name the test according to what it checks."

Powerful prompt gives rules and full boundary scenarios. Boundary pairs like "exactly 8 / 7 characters" are the most common places to make mistakes (confusing > with >=). Weak prompt bypasses these boundaries and carries the error to production.

Test Types and Where to Use

Test type

What does it confirm?

AI contribution

Attention

unit

Single function/class

Generates multi-scenarios fast

Meaningful assert is required

integration

Parts working together

Scenario and mock data draft

True addictive behavior

end/accept

Entire user flow

Step list and expectation

prone to brittleness

regression

Old error not returning

Fault specific testing

Should be added to every fix

Mini Cases

Case 1 — The test that always passes. AI writes 12 tests to a function and they all pass. The engineer becomes suspicious and deliberately distorts the return value of the function; Only 3 of the tests turn red. The other 9 tests do not contain meaningful asserts. Testing is strengthened by mutation hunting; real protection is gained in 9 scenarios.

Case 2 — Boundary error. An age verification function should say "18 and over is valid" but >18 is written, meaning age 18 is rejected. The error shows up immediately in testing because the AI ​​generates the “exactly 18” scenario through breakpoint analysis. A single limit test prevents any real user complaints.

Case 3 — Fixing current behavior. When the AI ​​is told to "write a test based on this code", it produces a test that accepts as "correct" a rounding error that already exists in the code. When the engineer prints the test according to the requirement (expected correct value) and not the code, the test turns red and the real error occurs. Tests should be derived from expectation, not from code.

Common mistakes

  • Pointless assert. "Didn't throw an error" is not enough; The correct value must be verified.
  • Confusing scope with quality. High coverage is no guarantee of accurate results.
  • Printing the test by code. Fixes the current error to "true"; Tests should derive from expectation.
  • Skipping limit values. Confusing > with >= is the most common mistake; boundary pairs must be tested.
  • Not auditing the test itself. A test that does not turn red when you break the code does not provide protection.

In summary

A good testing suite is the key to making changes with confidence. AI quickly generates a multitude of scenarios, bounds, and negative situations; But if it derives tests from code rather than requirements, it can fix existing bugs or write meaningless tests that always pass. Assert the concrete expected value in each test, include bound pairs, and verify that your tests actually protect by deliberately breaking the code. The content of the assert, not the number of scopes, determines quality.

Application task

Select a function and have it generate tests in four categories (normal, limit, negative, error) with a comprehensive test generation prompt; Have the concrete expected value asserted in each test. Then run the test vulnerability hunting prompt, suggest 5 small mutations in the code, and run the tests to check which ones they catch. Add a new test for at least one mutation that was not caught and show that it is now in the red.

checklist

  • [ ] I printed the tests based on the expected/correct behavior, not the code.
  • [ ] I covered normal, limit, negative and error scenarios.
  • [ ] I asserted the concrete expected value in each test.
  • [ ] I tested border pairs (just above-below / just above-below).
  • [ ] By deliberately breaking the code, I confirmed that the tests turned red.
  • [ ] I added a new test for undetected mutations.