Unit 5 / 11

API Test Automation: Contract, Schema and End-to-End Validation with AI

Gains:

  • Ability to conduct API testing in depth with artificial intelligence support at status code, schema/contract, business rule and negative/authorization layers
  • Ability to generate JSON schema from sample response and avoid the pseudo-confidence of looking only at the status code with type and imperative validation
  • Ability to test security scenarios such as authorization and IDOR with synthetic data and for defensive purposes only within authorization

Most modern software talks to each other in the background via API (Application Programming Interface — the interface where two pieces of software talk according to a specific contract). When a mobile app adds items to the cart, it actually sends a request to an API on the server. API testing checks that this conversation is correct, secure, and consistent, regardless of the interface; It is faster, more stable and deeper than UI testing. Artificial intelligence (AI) is very efficient in API testing: it generates tests from an API definition, extracts the response schema (the contract that defines the structure of the data), lists edge cases. But again the central caveat applies: the AI ​​doesn't know the real business rules of your API; tends to produce superficial tests that only confirm "200 returned". Your job is to make sure the test verifies the actual contract and business logic.

In this unit, you will learn how to set up AI-supported, deep API tests with approaches such as Postman, REST Assured and schema validation.

Layers of API testing

Consider API testing in several depths, with AI helping differently at each layer:

1. Status code and basic response. Does the request return the expected HTTP status code (200/201 for success, 400/401/404 for error)? This is the most superficial layer; AI produces easily but alone gives false-trust.

2. Schema/contract validation. Does the structure of the response fit the contract — are the expected fields present, are their types correct, are required fields missing? The AI ​​can generate JSON Schema — the standard that defines the structure of a JSON document — from a sample response, and tests can validate against that schema. This is much more robust than manually writing a field-based assert.

3. Business rule validation. The real value is here: "For a 1000 TL order, the discount field should be 100", "a canceled order cannot be canceled again". AI will only verify these if you give it the rules; If you don't give it, it will jump.

4. Negative and security. 401 for invalid token, 403 for accessing someone else's data, clear 400 for bad body. Authorization tests (verifying that a user can only access their own data) are the heart of API security and are done for defensive purposes.

Tip: Don't request a test without telling the AI ​​to "validate not just the status code, but also the response schema and those business rules." Otherwise, you will be left with tests that say "200 returned, passed" but do not notice the API returning corrupted data.

Weak prompt / Strong prompt

Weak: "Write tests for this API."
Strong: "Write REST Assured (Java) tests for the POST /order endpoint. Agreement: productId and quantity are mandatory in the body; 201 and {orderId, total, discount, status} are returned on success. Business rules: 10% discount over 1000 TL; 400 if quantity<=0; 401 if invalid token; 403 when seeing another user's order. Tests: (1) status code, (2) response JSON schema validation, (3) discount business rule, (4) bind every assert to explicit business rule; don't just check 200/201.”

The powerful prompt gives the contract, business rules, security scenarios, and schema validation expectation.

Contract testing: preventing breakups between teams

In microservice architectures (the structure in which the application is divided into small services that are independent of each other and talk to the API), changing the response format of a service silently disrupts other services connected to it. Contract testing — the test that verifies that the API contract between the provider service and the consumer service is not broken on both sides — catches such breaks early. The idea is this: the consumer defines the form of response he expects from the producer as a "contract"; With each change, the manufacturer tests that it still complies with this agreement. So when the name or type of a field changes, the consumer notifies the pipeline before it crashes.

AI accelerates two tasks in this context: drafting a contract that reflects consumer expectation from an existing API response, and pre-marking which contract clause a change could break. But the contract itself is a business decision: the expert determines which areas are truly critical, which changes will break backward compatibility — old consumers continue to work. AI writes the contract; You are the one who approves it.

Tip: Deleting a field or changing the field type in an API is almost always a breaking change. Adding new fields is usually safe. Having the AI ​​classify a change as “breaking or safe” provides a quick pre-release security check.

Postman or code-based?

criterion

Postman/Newman

REST Assured / code (Java, C#, JS)

Learning

Easy, visual

Code knowledge required

Version control

Collection JSON

Directly in source code

complex logic

Limited (JS scripts)

Full programming power

CI/CD integration

with Newman

Directly dependent on build

Schema validation

With test scripts

Powerful with library

Team scale

small/medium

big, mature

AI generates code for both; Be clear which one you want.

Four copyable templates

1) Contract-based API testing:

Your role: senior API test engineer.Write tests for the following endpoint with [tool/language]: [method + path].Contract: [required fields, success code, response structure].Business rules: [rules].Test layers: (1) status code (2) response schema validation(3) each business rule (4) negative + authorization.Link each assert to the relevant rule/contract clause.

2) Schema generation from sample response:

Generate JSON Schema from the sample API response below. Specify required fields, types, format constraints (date, email, number range). Then give a test example that validates against this schema. Sample response: [paste JSON]

3) Negative and authorization scenarios:

Generate negative and security test cases for endpoint[endpoint]. Includes: missing/required field, wrong type, too large value, invalid/expired token, access to unauthorized resource (IDOR — access to someone else's record by changing ID), rate limit. Specify the expected status code and error body for each scenario. Note: will only be tested on my own API, authorized.

4) Pseudo-trust control:

Check out this API test. Would this test catch if the server returned the correct status code but FALSEbody/data? If not, add schema and business rule validation. Test: [paste test]

three mini cases

Case 1 — The power of schema validation. A team was only checking the status code in the tests it produced with AI. In one version, the API began erroneously returning the total field as text ("1200"); the tests stayed green because it was still returning 200. Mobile application crashed. After adding type validation with the "Schema generation from sample response" template, the same error was immediately caught.

Case 2 — Authority gap (IDOR). An expert ran the IDOR test between the “negative and authorization scenarios” generated by the AI: He requested the order ID of user B with the token of user A. The API returned data of 200 and B — a serious authorization vulnerability. This defensive test closed the data leak before it went live.

Case 3 — Business rule bypass. AI generated 8 tests for the discount endpoint; all were checking 200, none were verifying the discount amount. The expert added the business rules to the prompt and had them reproduced. New tests revealed that the discount was calculated incorrectly at the 1000 TL limit (the discount was also applied to 999). Contract control is not enough; Business rule control is a must.

Common mistakes

  • Just looking at the status code. To say "200 has returned and passed"; not seeing the corrupted body (false-trust).
  • Bypassing schema validation. Not checking field types and obligation; type changes pass silently.
  • Requesting testing without providing business rules. AI doesn't know the rules; it produces only technical control.
  • Forgetting negative and entitlement scenarios. Security vulnerabilities (IDOR, unauthorized access) are caught only by these tests.
  • Using real/production tokens and data. Use dedicated media and synthetic data for testing; Do not stick real keys into the vehicle.
  • Unauthorized security testing. Only run authorization tests on your own API and with permission.

In summary

API testing verifies the speech of pieces of software quickly and deeply, regardless of interface. AI; contract tests are very efficient at generating JSON schema and negative/security scenarios from the sample response. But superficial tests that only check the status code give pseudo-confidence. Require all four layers: status code, schema validation, business rule, negative, and authorization. Put the business rules and contract on the prompt; Perform security tests with synthetic data and only with authorization.

Application task

Choose an API endpoint from your own project. Have AI write four-layer tests with the “contract-based API testing” template. Then add type/enforcement validation with "schema generation from sample response" and apply "pseudo-trust checking". Run at least one IDOR/authorization scenario in your own test environment. Report any contract or business rule violations you find; If you can't find any, run the test against a deliberately garbled response to prove that it caught it.

checklist

  • [ ] I covered the four layers of testing (case, schema, business rule, negative/authorization).
  • [ ] I clearly gave the contract and business rules to AI.
  • [ ] I set up tests that validate the response schema (field, type, imperative).
  • [ ] I have tried at least one authorization/IDOR scenario defensively.
  • [ ] I used test environment and synthetic data instead of real token/data.
  • [ ] I proved with a "pseudo-confidence check" that every test catches the corrupted response.