Gains:
- Ability to verify AI output at three layers: accuracy, security and source/license
- Ability to cover risks such as injection, hallucination packages and buried secrets with safe molds and tools
- Ability to present the security-critical code to the approval of a competent engineer and understand the non-transferability of responsibility
Generating AI code is easy; Trusting him is expensive. The sole purpose of this unit is to transform the "verify" principle, which we have repeated in all previous units, into a systematic engineering discipline. Because the code produced by AI, even if it seems correct at first glance, carries three separate dangers: being non-working/incorrect (hallucination), being insecure (vulnerability) and carrying legal/licensing risks. Knowing these three and establishing a door for each of them makes you a professional.
Here we consider “validation” at three layers: correctness (does the code actually do the job?), security (does it withstand malicious input?), and provenance/license (do I have the right to use this code?). Each layer has its own means of control, and none of them can be bypassed with "that's what the AI said."
Three Layers of Risk
1. Risk of accuracy (hallucination). The model may call a non-existent function, misuse an API, silently bypass an edge case. The code looks "reasonable" but is wrong. Antidote: compilation, testing, static analysis and visual inspection.
2. Security risk. AI can repeat insecure patterns in training data: query vulnerable to SQL injection, unauthenticated user input, weak encryption, insecure deserialization, open redirection. The code works but is vulnerable to attack. Antidote: security-focused review, automated scanners (SAST), and imposing known secure patterns.
3. Source/license risk. AI may produce output that closely resembles copyrighted or restrictive licensed code, or it may suggest an inappropriately licensed dependency. Antidote: dependency and license checking, originality checking, corporate policy.
Caution: The most insidious of these three risks is security; because the code can pass testing, run smoothly in production, and the vulnerability is only revealed when an attacker finds it. “Working” is not the same as “secure.”
Step by Step: Layered Authentication Gate
- Read with understanding. Really understand the code before accepting it; Don't merge code you don't understand. If you can't explain "why it works", it hasn't been validated yet.
- Verify it exists. Confirm that every function, API and package used actually exists and is used correctly (hallucination gate).
- Run automated tools. Compiler, linter (style/error scanner), type checker, unit tests, and if possible a SAST (Static Application Security Testing — tool that scans source code for vulnerabilities).
- Look at it from a security perspective. Is the input validated? Is the query parameterized? Is the secret buried? Is there authorization control?
- Check source and license. Are new dependencies licensed? Does the output look overly similar to a known codebase?
- If it is security-critical, ask for expert approval. Independent review by an engineer competent in areas such as authentication, payment, cryptography, access control is mandatory.
Three Mini Cases
Case 1 — SQL injection caught at inspection gate. The AI generated code that concatenates user input directly into the SQL query for a search endpoint ("... WHERE name = '" + q + "'"). The code was working and passed the test. Security-focused inspection and SAST scanning caught this; It was converted into a parameterized query (prepared statement). If it hadn't been caught, it would have been a classic data leak vulnerability.
Case 2 — Hallucination package. AI suggested a non-existent npm package (fast-safe-parse) for a task. When the developer tried to install it, the package was not found. Worse: in some cases, attackers can fill such "ghost" package names with real, malicious packages (dependency confusion). Lesson: verify each recommended package against the official registry and download/maintenance history.
Case 3 — License incompatibility. A nifty companion library suggested by AI had a strong copyleft license that was incompatible with the institution's product license. Dependency license scan reported this; The team replaced the license with a suitable alternative. Without verification, a legal burden would arise in product distribution.
Four Copiable Templates
Pre-admission self-check:
Before accepting the following AI generated code, check:1) Does every function/API/package it uses actually exist? Flag the suspects.2) Are there any unvalidated input, SQL/command concatenation, buried secret, weak crypto?3) What are the unaddressed bugs/edge cases?Label each finding as "certain / probable" and suggest fixes.{{code}}
Security focused review:
Examine this code with a security eye. Look for common OWASP style vulnerabilities: injection, broken authentication/authorization, sensitive data disclosure, insecure deserialization, unauthenticated redirection. For each finding: risk, exploitation scenario, remediation. This is a preliminary screening; refer critical findings to human security review.{{code}}
Dependency and license checking:
List the dependencies added/suggested by this code. For each: does the package actually exist, is it maintained, what would its typical license be (MUST BE VERIFIED), and is it actually needed for the project or can it be done with an existing tool?{{code or dependency list}}
Safe formwork imposition (in production):
Write code for {{task}}. MANDATORY security rules:- Validate/sanitize all external input.- Only use parameterized query in database access.- Don't embed secrets in code; assume environment variable/secret manager - Don't swallow errors; Consider it meaningfully. Explain how the code complies with these rules in 3 items.
Weak prompt / Strong prompt
Weak: "Write a query that searches by username." (A code vulnerable to injection may occur.)
Strong: "Write a function that searches by username. Never join user input into a query as a string; use a parameterized query (prepared statement). Validate the input for length and character. Explain in 2 sentences why the code is closed to injection."
The strong version imposes the secure pattern from the beginning; Thus, it ensures that the vulnerability does not occur at all, rather than catching it later. However, it is essential to pass the generated code through verification gates.
Authentication layer
Tool/method
Is "AI said" enough?
accuracy
Compilation, testing, visual inspection
no
API/package reality
Official document/record control
no
Security
SAST, security review
no
License/source
Dependency & license checking
no
Security-critical logic
Expert engineer approval
Absolutely not
Responsibility Cannot Be Transferred
Responsibility for errors, vulnerabilities or violations arising from the code produced by an AI tool belongs to the team that assembles and distributes that code, not to the tool provider. This is a professional fact as well as a legal one: you sign. So "AI produced it" is not an excuse, but a justification for extra caution. Particularly in safety-critical systems, AI output is not a substitute for review and approval by a qualified engineer under any circumstances; At most, AI provides a blueprint that speeds up that engineer.
Tip: Create a short checklist on your team that you call “validation gate for AI-generated code” (build + test + security scan + visual inspection). Once this gate becomes a habit, the loss of speed is minimal and the risk reduction is maximum.
Common mistakes
- Confusing "works" with "safe". Code that passes testing may be vulnerable to attack.
- Using the package/API without verifying it. Hallucinatory packets both corrupt and pose a security risk.
- Bypassing automated tools. Linter, type checker and SAST cheaply catch what humans miss.
- Ignoring the license. Improper licensed dependency creates legal burden on distribution.
- Putting the responsibility on the vehicle. The team is responsible for the code in production; “AI did it” is no excuse.
In summary
Accepting AI output requires three layers of verification: correctness (compile, test, visual inspection), security (SAST and security-focused review), and source/license (dependency checking). Confirm that each package and API used actually exists, enforce secure patterns from the start, and submit security-critical code for approval by a qualified engineer. “Works” does not mean safe, and “AI produced” does not remove liability. The verification gate is the price of professionalism, not speed.
Application task
Deliberately give an AI a security-sensitive task (e.g. “a function that searches the database with user input”), this time without imposing a safe pattern. Pass incoming code through “pre-admission self-audit” and “security-focused review” templates: is there any injection, buried secret, hallucinated packet, or unauthenticated input? Then ask the same task again with the “secure pattern imposition” template and compare the two outputs. If possible, run a linter/SAST tool and compare the findings to the AI's self-regulation.
checklist
- [ ] I verify AI output at three layers: accuracy, security and license.
- [ ] I confirm that every function, API and package used actually exists.
- [ ] I run compile, test, linter and, if possible, SAST tools.
- [ ] I impose secure patterns (parameterized query, input validation, secret management) from the beginning.
- [ ] I check the licensing and requirement of new dependencies.
- [ ] I am submitting security-critical code for approval by a competent engineer and I understand that I am responsible.