Gains:
- Can describe the basic structure of an LLM API request (endpoint, model, messages, max_tokens)
- Understands the difference between system, user and assistant roles and stateless conversation history
- Can read and interpret fields (content blocks, stop_reason, usage) of the returned response
In previous modules, we used artificial intelligence from a chat window. But if you want to embed AI into your own product, automation or workflow, a chat interface won't cut it; You need to connect to the model programmatically, that is, with code or an automation tool. The name of this bridge is API (Application Programming Interface, the contract that allows two software to talk with certain rules). When you finish this unit, you will know what constitutes a LLM (Large Language Model) API request, what message roles do, and how to read the response. This is the foundation on which the rest of the module will be built.
How Does the API Work?
The basic flow in the API is this: you send a request in a certain format; The server returns a response in a specific format. In LLMs, this is usually an HTTP call (HTTP: standard protocol for carrying request-response on the web) to a single address (endpoint, the fixed address on the server that handles your request). For example, in a messaging API, all requests go to a single address and are carried in the body as JSON (JavaScript Object Notation — a text format consisting of key/value pairs that can be read by both humans and machines).
In a request, you specify at least these three things:
- Model: Which model you will use (e.g. a fast and cheap model or a powerful model).
- max_tokens: The maximum number of tokens (the smallest unit in which the text is processed, which will be processed in detail in the next unit) that the model can produce; i.e. output limit.
- messages: List of messages that make up the conversation.
Step by Step: How to Set Up a Request
- Prepare the endpoint and credentials. You add your API key (the secret string that proves your identity) to the request in a header. You never embed the key in the code; We will cover safe storage in unit 9.
- Select the model and output limit. Lightweight model + small max_tokens for a simple task; Powerful model + larger limit for a complex task.
- Set up the message list. List the system instruction, user message, and past rounds (if any).
- Send the request and parse the response. Read the text content, stop reason, and token usage from the returned JSON.
Message Roles: system, user, assistant
A conversation consists of messages arranged in a sequence, and each message has a role. The role determines how the model treats that text.
Role
Who writes
Purpose
system
Developer/operator
Permanent instructions, personality and rules that apply throughout the entire conversation
user
end user
The user's current question or input
assistant
model
Response produced by the model (and previous responses)
The system role is available as a separate system field in the request body in most providers; user and assistant are listed sequentially in the messages list. Critical point: the system instruction is the high-level instruction, the user message is the request to be answered at that moment.
{ "model": "claude-opus-4-8", "max_tokens": 1024, "system": "You are a corporate support assistant. Give a short, formal and verified response. Do not make up information you are not sure about.", "messages": [ { "role": "user", "content": "How do I start my return process?" } ]}
Speech is Stateless
Here's the most common misconception: LLM API calls are stateless — the server retains no memory between two requests. The model does not remember your previous request. If you're setting up a multi-round chat, you'll need to resend past rounds with each new request. The model's "memory" consists of a list of messages you have sent.
{ "model": "claude-opus-4-8", "max_tokens": 512, "messages": [ { "role": "user", "content": "Hello, my name is Deniz." }, { "role": "assistant", "content": "Hello Deniz, how can I help you?" }, { "role": "user", "content": "I just said my name, do you remember?" } ]}
Answering the third message correctly depends on you sending both previous messages. If you don't send it, the model won't know "Sea" and will answer incorrectly. This also directly affects the cost: the longer the conversation, the larger the list, each request consuming more tokens.
Tip: In long conversations, summarizing and moving old rounds (summary + last few rounds) instead of sending the entire history reduces cost and preserves the context window. We will deepen this in units 6 and 11.
Read the Answer
When the model returns a response, you receive a structured object, not plain text. Typical areas:
{ "id": "msg_01ABC...", "model": "claude-opus-4-8", "role": "assistant", "content": [ { "type": "text", "text": "To initiate a return, go to the 'My Orders' page in your account..." } ], "stop_reason": "end_turn", "usage": { "input_tokens": 47, "output_tokens": 88 }}
- content: The response itself; It is a list of content blocks. The text field of the text block is the actual answer.
- stop_reason: Why the model stopped. end_turn = natural end; max_tokens = stuck at output limit (response may be incomplete); refusal = refused for security reasons. Your code should always look at stop_reason first.
- usage: Input and output token numbers. It is the basis of cost and limit tracking.
Attention: If stop_reason is max_tokens, the response is not completed. Treating this as a "successful response" and showing half text to the user is one of the most common mistakes in production. Either increase max_tokens or use streaming.
Weak prompt / Strong prompt
Same task with two different system prompts:
# WEAKYou are an assistant. Answer the questions.
# STRONGYou are a corporate support assistant. Rules:- Rely solely on the information in the policy document provided; If it is not in the document, say "I do not have this information, I am directing it to the relevant unit." - Answers should not exceed 3 sentences, be formal and clear. - Do not ask for personal data (TC ID number, card number) and do not repeat. - Do not guess when you are not sure.
Powerful version; It defines scope, form, safety margin, and behavior in uncertainty. The consistency of the model output comes directly from this clarity.
Three Mini Cases
Case 1 — Support bot (statelessness trap). An e-commerce team took the bot live; When the user said "cancel the previous order", the bot "forgot" the order number. Reason: they were sending each request with only the last message. Solution: they added the last 6 rounds to the messages list. Result: context preserved, but input per request increased from 40 tokens to ~600 tokens — we'll cover the cost lesson in unit 2.
Case 2 — Incomplete contract summary. A legal team was having 10-page contracts outlined; max_tokens: 300 remained low, summaries were cutting off mid-sentence. stop_reason was max_tokens every time but no one was looking. increased max_tokens to 1500 and added stop_reason check; The truncated summary rate decreased from 18% to 0%.
Case 3 — Mixing roles. A marketing team was writing all the instructions into the user message, leaving the system blank. When user input mixed with instruction, the model would sometimes comply with the user's command to "forget the previous rules." They moved permanent rules to the system; By separating user input from instruction, rule violations decreased significantly.
Common mistakes
- Forgetting to send the past: The model is thought to "not remember"; whereas it is stateless. You carry the context.
- Not looking at `stop_reason`: The response stopped with max_tokens is considered complete.
- Embedding the instruction in `user`: Persistent rules into system; instant input goes to the user. Mixing creates security vulnerabilities.
- Mistaking `content` for a plain string: The answer is a list of blocks; read the text field of the first text block, verify its type before getting content[0] with a blind index.
- Embedding the key in the code: Use an environment variable (unit 9).
Deeper: Content Blocks and Multi-Part Answers
Understanding why the content field in the response is a list is fundamental to the advanced features you'll encounter later. Sometimes the model returns not a single block of text, but several blocks: a block of thinking, followed by a block of text; or a block of text followed by a tool use block. That's why blindly counting content[0] as an "answer" is fragile. The correct approach is to go through the list and sort it by type: you collect the text content of blocks whose type field is text, and treat other types (thinking, tool) separately.
What this distinction does in practice is that you can log the model's reasoning (if any) without revealing it to the user, redirect tool calls to separate logic, and only print the actual answer on the screen. As the module progresses (especially in units 4 and 11) you will see how useful this block structure is for validating and directing output.
Another practical point: you can access the same model from different provider platforms (direct API, via a cloud provider). Although the endpoint address and authentication format may change, basic concepts such as message roles, statelessness, and response structure remain the same. So the basics in this unit apply no matter what platform you use.
In summary
An LLM API request consists of the model, output limit, and message list; roles (system, user, assistant) determine the behavior of the model. Calls are stateless: you carry the context with each request. The response is a structured object; Reading and interpreting the content, stop_reason and usage fields is the basis of durability in production.
Application task
Choose a task from your own profession (e.g. sorting incoming e-mail, creating brief summaries). On a piece of paper: (1) write the system prompt with 4-5 rules, (2) set up a sample user message and a 2-round history if any, (3) determine a reasonable value for max_tokens and write the justification, (4) list which stop_reason values you will handle in the returned response and how.
checklist
- [ ] I can count the three mandatory parts of a request (model, max_tokens, messages).
- [ ] I can explain the difference between system, user and assistant roles.
- [ ] I know that calls are stateless and that I need to carry the past.
- I can read and comment on [ ] content, stop_reason and usage fields.
- [ ] With max_tokens I can notice and handle the truncated response.