System Prompt and Model Parameters

Gains:

• Can design how the system prompt guides the model through the entire conversation
• Understands the role and cost impact of adaptive thinking and effort parameters
• implements output controls such as max_tokens, stop sequences, and structured output

Two different products of the same model may behave completely differently. The difference is not in the model itself, but in the system prompt and parameters given to it. The system prompt is the model's "work contract" and the parameters are "work settings". In this unit, you will learn how to design a powerful system prompt, what the thinking and effort settings in modern models do, and how to control the output for format/length. Setting these settings correctly allows you to manage both quality and cost at the same time.

System Prompt: Permanent Directive of the Model

The system prompt is the high-level instruction that applies throughout the entire conversation. These rules remain valid no matter what the user types. A good system prompt includes the following components:

1. Role/identity: Who is the model? (“You are a corporate support assistant.”)
2. Scope and boundary: What does it do and what does it not do? (“Base only on the policy document provided.”)
3. Format rules: What should the output look like? ("Maximum 3 articles, official language.")
4. Behavior in uncertainty: What does one do when unsure? ("If there is no information, make it up, direct it to the relevant unit.")
5. Security/privacy: What doesn't/doesn't want? ("Request personal data.")

Tip: Keep the system prompt fixed. Do not embed information that changes with each request (current date, username, session ID). This both breaks consistency and invalidates the prompt cache on unit 6. Put the variable information in the user message.

The Overly Aggressive Instruction Trap

Modern models follow the instruction very closely. Aggressive phrases such as "MUST", "ALWAYS", "DEFINITELY do this", etc., which worked in older models, today lead to overtriggering: the model calls an agent when it is not needed or runs for an unnecessarily long time. Soften the rule: Instead of "MUST use the search tool", "If the answer is not in the conversation, use the search tool" is more accurate.

Model Parameters: Thought and Effort

Classical LLMs had a temperature parameter: a lower value produced more specific/consistent output, a higher value produced more varied/creative output. Modern generation models (such as Opus 4.8, Sonnet 5) replace this approach with two more powerful mechanisms and no longer accept sampling parameters such as temperature.

• Adaptive thinking: The model reasons step by step in its "head" before responding. The model decides how much to think based on the difficulty of the task. Significantly improves accuracy on complex, multi-step problems; He thinks less to avoid unnecessary delays on simple questions.
• Effort: High-level knob that adjusts how deeply the model dives into a task and how many tokens it spends in total. Typical levels: low, medium, high and above. High effort may improve quality, but it also increases delay and cost; Low effort brings speed and savings.

Setting

What does

when

Thinking off/low effort

Fast, cheap, superficial

Simple classification, short response, delay sensitive tasks

Adaptive thinking + medium effort

Balanced quality/cost

Most general purpose tasks

Adaptive thinking + high effort

highest accuracy

Complex reasoning, coding, long-range agent work

Caution: The "maximum effort no matter what" reflex inflates costs. Adjust effort to task; In simple tasks, low effort often gives the same accurate result at a much cheaper price. Go high where critical accuracy is needed.

Output Control: Format, Length, Structure

Besides the parameters, you also control the output itself:

• max_tokens: Hard ceiling of the output (1st and 3rd unit).
• Stop sequences: Stopping the model when it sees a certain string. Useful for setting breakpoints in structured production.
• Structured output: Force the model's response to conform to a JSON schema you provide. It ensures that the output is programmatically parsable and valid. It's more reliable than saying "just return JSON" with a prompt.

{ "output_config": { "format": { "type": "json_schema", "schema": { "type": "object", "additionalProperties": false, "properties": { "category": { "type": "string", "enum": ["invoice", "technical", "return", "other"] }, "urgency": { "type": "string", "enum": ["low", "medium", "high"] } }, "required": ["category", "urgency"] } } }}

Copiable System Prompt Templates

# Corporate support assistantYou are a corporate support assistant.- Rely solely on the policy document provided; If it is not in the document, say "I do not have this information." - Give a formal and clear answer in a maximum of 3 sentences. - Ask for personal data (TC ID number, card number) and do not repeat it in your answer. - If you are not sure, do not guess.

# Structured output forcing classifierYou are a demand classifier. The input is a customer message. Return only the requested fields, do not write comments. If you're not sure, use "other".

# Analyst with defined behavior of standing in uncertaintyYou are a data analyst. Draw only verifiable inferences from the table provided. Never make up a conclusion that does not exist in the data. If an inference is unclear, write "data insufficient."

# Content writer with tone and length controlYou are a content writer. Use a warm but professional tone. Limit each text to 120 words or less. Avoid cliché marketing language.

Weak prompt / Strong prompt

# WEAKBe helpful and give good answers. Do your best.

# STRONGRole: Technical support specialist.Scope: Product guide provided only.Format: Step-by-step, numbered list, 5 steps maximum.Limit: Recommend solution not in the guide; Say "I couldn't find it in the manual." Privacy: Do not repeat the serial number shared by the user in the response.

Powerful version; It determines the role, scope, format, boundaries and confidentiality separately. Output consistency comes directly from this clarity.

Three Mini Cases

Case 1 — Cost reduction through effort adjustment. One team was running all their calls on high effort + thinking; Even simple email digests were expensive and slow to produce. They assigned simple tasks such as summaries to low effort and contract analysis to high effort. Accuracy was maintained, average latency was halved, and monthly cost was reduced by a third.

Case 2 — JSON guarantee. An operations team asked for the classification output with a prompt saying "just give JSON", but the model would occasionally write "Here is the result:" and the parser would crash. When I connected the configured output schema, the output returned valid JSON every time; parsing errors have been reset.

Case 3 — Aggressive prompt recoil. An assistant prompt said, "MUST search for EVERY QUESTION"; The model made unnecessary searches even for simple questions to which it already knew the answer, slowing down and increasing costs. They relaxed the rule to "If the answer is not in context, search"; Unnecessary calls decreased by 70% and responses accelerated.

Common mistakes

• Embedding variable data in the system prompt: Breaks consistency and invalidates the cache.
• Overly aggressive instruction: Excessive triggering and unnecessary cost in modern models.
• High effort in every task: Waste in simple tasks; adjust effort to task.
• Requesting JSON only via prompt: It breaks occasionally; if critical, use structured output.
• Not defining boundary/ambiguity behavior: The model fills the gap with fabrication (hallucination).
• Old `temperature` habit: Modern models do not accept this; Guide behavior with prompt and effort.

Deeper: Writing the Prompt Like a Contract

Experienced teams treat the system prompt like a contract, not a literary text: clear clauses, measurable rules, unambiguous boundaries. This approach has three concrete benefits. The first is consistency: the same input gives similar output at different times. Second is testability: you can test each item separately with a sample. Third is ease of maintenance: if a behavior is wrong, you know which item to replace.

A good practice is to lead with positive examples. Rather than providing a list of "don't do this", it is much more effective in modern models to provide an example that says "this is exactly what the desired output looks like". For example, in a classifier, adding one or two samples of the expected JSON to the prompt significantly reduces formatting errors.

Another powerful technique is to write the uncertainty behavior explicitly. A clause such as "If unsure, don't guess; say 'insufficient data'" suppresses the model's tendency to fill in the blank with fabrication (hallucination). This single sentence offloads the verification layer, which we will cover in unit 11: once the model has already flagged uncertainty, it becomes easier to lead to human validation.

Finally, consider effort and prompt together. At high effort, the model explores more and sometimes does unwanted “extra work” (unnecessary explanation, additional suggestion). Saying "only give the desired output, don't add additional comments" in the prompt offsets this side effect of high effort.

In summary

The system prompt is the permanent directive of the model: it defines the role, scope, format, obscurity behavior, and confidentiality. In modern models, behavior is driven by adaptive thinking and effort parameters rather than temperature; Aligning effort to the task manages quality and cost simultaneously. You secure the output with max_tokens, stop arrays, and structured output.

Application task

Choose a task. (1) Write a system prompt with five components (role, scope, format, ambiguity, confidentiality). (2) State what level of effort you would choose for this task and why. (3) If the output should be structured, sketch out a small JSON schema. (4) Check if there is an overly aggressive pattern in your prompt and soften it.

checklist

• [ ] I can name five components of a good system prompt.
• [ ] I can explain what the adaptive thinking and effort parameters do.
• [ ] I can balance quality/cost by adjusting effort according to the task.
• [ ] I know why structured output is safer than requesting JSON via prompt.
• [ ] I can recognize the risk in modern models of overly aggressive instructions.