Logging, Audit Trail and Provenability

Gains:

• Ability to design a minimum audit trail scheme sufficient to reconstruct the event
• Ability to prevent the log from being a source of leakage by masking the prompt/response
• Ability to establish verifiable logs with correlation identity, immutability and retention period

In an AI system, one day the question will surely be asked: "Why was this decision made this way, what exactly happened that day?" This question may be asked by a customer, an auditor, a regulator, or a court. Your answer will either be a verifiable audit trail or "we don't know." The latter is unacceptable in a corporate environment. In this unit, we will learn what should and should not be logged specific to AI, how to establish an audit trail, and how to keep logs in balance with security and privacy.

Why Is Logging Different in AI?

In classical software, "who did what" is logged. In AI, three new dimensions are added to this: what model/version was used, what prompt was sent, and what response was produced. When an error or complaint occurs, you cannot reconstruct the incident without these three. But this very prompt/response can contain PII, as we saw in unit 2 — meaning the log itself can become a source of leaks. This is the art of balance.

Caution: Logging is not "log everything". Too much logging creates a privacy risk, and too little logging creates lack of evidence. The goal is to keep enough of the PII to reconstruct the event by masking it.

What should be logged? Audit Trail Schema

A solid AI audit trail includes, at a minimum:

• Who: User ID and role (or service ID).
• When: Timestamp (append-only if possible).
• What: Desired action and summoned tools.
• Which model: Model name and version (e.g. claude-opus-4-8), critical parameters such as temperature.
• Input/output digest: A masked version or a digest/hash of the request and response.
• Decision: Was it processed automatically, went to a human, was it approved or rejected?
• Result: Is the operation successful or error, which resource is affected?

Step by Step: Establishing an Audit Trail

1. Set a goal. Who will read these logs and why? (Incident response, compliance auditing, debugging.) Purpose determines what you keep.
2. Enforce PII policy. Mask the prompt/response before logging (unit 2).
3. Provide immutability. Let critical logs be append-only; No one should be able to erase the past silently.
4. Define retention period. Determine the duration according to the balance of legal requirement and confidentiality; Automatically delete when time expires.
5. Limit access. Access to logs should also be protected with RBAC; Log reading should also be logged.
6. Add correlation ID (trace ID). Connect all steps of a request (input, tool call, verification, output) with a single identity.

Four Copiable Templates

Audit log schema (JSON):

{ "trace_id": "...", "time": "YYYY-MM-DDThh:mm:ssZ", "user": "...", "role": "...", "model": "claude-opus-4-8", "parameters": { "temperature": 0 }, "request_summary": "<masked>", "response_summary": "<masked>", "tools": ["tool_a", "tool_b"], "decision": "auto|human_approval", "approval": "approved|rejected|none", "result": "success|error", "affected_resource": "..."}

Log PII control prompt:

Check out the log examples below. Are the fields required for the audit trail (who, when, model, decision, result) complete? Also has raw PII been leaked? For each row, report as: "not enough / missing space: ... /PII leak: ..." <logs>{{ examples }}</logs>

Event rebuild prompt:

The following audit records belong to a single trace_id. Turn the event into a narrative in chronological order: what did the user want, what did the model do, what validations ran, how was the decision made, what was the outcome? Flag missing or inconsistent steps.<records>{{ trace_registers }}</records>

Retention policy decision rule:

For each log type, determine:- Is there a legal retention obligation? (minimum period if any)- Does it contain PII? (if included, shorten the duration, narrow the access)- Evidence of security incident? (store cannot be changed)Result: "store N days + append-only mi + access level".

Weak Prompt / Strong Prompt

poor approach

Strong approach

Not logging at all ("don't need")

Logging the minimum set to reconstruct the event

Logging raw prompt/response as is

Masked summary + trace ID logging

Store logs unlimitedly

Retention period with balance of legal + privacy

Anyone can delete logs

Critical logs are append-only, access controlled

Three Mini Cases

Case 1 — Trace ID reduced a day's investigation to 15 minutes. "My application was unfairly rejected," a customer said to a bank's credit pre-evaluation assistant. Thanks to the correlation ID, the team reconstructed that application's input, employee verifications, and decision in 15 minutes; showed that the error was caused by an incorrect threshold in a rule validation and fixed it.

Case 2 — Excessive logging was discovered in the audit. An e-commerce company was writing all prompts/responses to raw logs for debugging. During the annual audit, it was seen that these logs contained customer addresses and telephone numbers and were kept for 2 years. The finding was closed by switching to a masking + 90-day retention policy; The audit trail function was preserved.

Case 3 — Append-only log revealed internal abuse. An employee at one provider attempted to delete logs to hide an erroneous batch he made. Since the logs are append-only and log reading/deleting attempts are recorded, the attempt was immediately visible; The incident resulted in disciplinary and process correction.

Tip: Assign a correlation ID (trace ID) to each request and carry it through all steps. When a problem occurs, being able to collect "everything about that request" with a single query is the biggest accelerator of incident response.

Common mistakes

• Not logging at all, or logging so little that you cannot reconstruct the event.
• Logging the raw request/response without a mask and turning the log into a source of leakage.
• Not logging model name/version and decision (automatic/human).
• Storing logs for an unlimited period of time increases privacy risk.
• Leaving critical logs subject to change; Not logging log access.
• Not being able to connect the steps together because it does not use a correlation ID (trace ID).

In summary

• AI logging adds three dimensions to “who did what”: which model/version, which prompt, which response.
• The goal is to keep the PII minimal enough to reconstruct the event by masking it—no more, no less.
• The audit trail should include who/when/what/which model/decision/result fields.
• Critical logs should be append-only, access should be limited, and log access should also be logged.
• Correlation ID (trace ID) connects all steps of a request and speeds up incident investigation.

Application task

Select a request from your own AI flow and write the ideal audit trail for it with the JSON schema above. Then do two tests: (1) Can you tell the story from beginning to end with just this recording? (2) Is there raw PII in the record? If there is missing field, add it, if there is PII, mask it. Finally, set a retention period and access level.

checklist

• [ ] Audit trail includes who/when/what/pattern/decision/result fields.
• [ ] The prompt/response is masked before logs (no PII).
• [ ] A correlation ID (trace ID) is assigned to each request.
• [ ] Critical logs are append-only and access controlled.
• [ ] Storage period is defined by the legal + confidentiality balance, and is deleted at the end of the period.
• [ ] With logs I can reconstruct an event in less than 30 minutes.