Gains:
- Recognizing light, balanced and powerful model stages and reasoning mode
- Selecting the appropriate model level based on the difficulty of a task
- Ability to consciously establish the balance between latency, accuracy and cost
We learned to calculate the price; Now let's talk about what we get for that price, namely model levels and capabilities. Three forces constantly compete when choosing a model: quality (accuracy), speed (latency), and cost. Improving two of these three often means compromising the third. In the unit, you will learn about light, balanced and powerful levels, reasoning mode and small/large model preference; You will be able to consciously establish the right balance according to the difficulty of the task.
Three Levels, Three Different Job Characters
Almost every provider offers three tiers within the same family. Let's match these with the character of the task:
- Light tier (small model): Very fast, very cheap, "good enough". Ideal for simple, repetitive, high-volume tasks: classification, briefing, tagging, data cleaning, simple response drafts.
- Balanced tier (medium model): Good quality, reasonable speed, reasonable cost. The “default” choice for most day-to-day work: content production, mid-length analysis, customer responses.
- Powerful tier (large model): Highest quality, slowest, most expensive. For complex reasoning, long and multi-step analysis, difficult code, precise legal/financial evaluation.
Tip: A good starting rule: Start with a balanced level. If the quality is poor, move up to the strong level; If the quality is extremely good and the volume is high, go down to the light level. The "start with good enough" approach, not the "start with the strongest" approach, saves both money and time.
What is Reasoning Mode?
Some models may first "think" the answer step by step rather than generating it directly. This is called reasoning mode. The model breaks a problem down into parts, processes the intermediate steps within itself, and only then gives the final answer. This is similar to a student solving a math problem step by step on paper.
When is it valuable? In multi-step, logical problems: a complex calculation, contradiction detection, a long chain of inference, a difficult code error. In this type of work, reasoning significantly increases accuracy.
What is the cost? Reasoning increases cost and latency as the model generates more “thinking tokens.” So it will be both more expensive and slower. A simple “is this review positive?” Reasoning on the question is like killing a fly with a hammer; creates unnecessary costs and delays.
Caution: Do not leave reasoning mode open for every task. It is both faster and cheaper to keep it closed for simple tasks. Some of the modern models come with an "adaptive" thinking mode that decides for itself when it needs to think; However, it is your job to manage it consciously.
Quality, Speed, Cost Triangle
priority
Appropriate level
reasoning
Lowest cost, high volume
lightweight
Closed
Balance (most jobs)
balanced
depending on the situation
Highest accuracy, critical decision
strong
open
Instant response, low latency (chat)
Light/balanced
Closed
Complex, multi-step analysis
strong
open
As you look at this table, ask yourself: “What will be the cost if I make a mistake on this task?” If the price is high (wrong legal assessment, wrong financial decision) the investment of strong caliber and reasoning is justified. If the price is low (misclassification of a product label), a light and fast model is sufficient.
Small Model or Large Model?
Contrary to popular belief, not every job requires a large model. Smaller models have become surprisingly capable, performing many tasks at a tenth the cost and several times the speed of the larger model. Three questions for decision:
- Is the task simple and formulaic? (Classification, extraction, brief summary) → Small model.
- Is the volume too high and speed critical? → Small model, because large model is expensive and slow.
- Does the task require nuanced reasoning, extended context, or creativity? → Large model.
Three Realistic Cases
Case 1 — The lightweight model is sufficient. An e-commerce team sorts 40,000 customer comments a day into positive/negative/neutral. They try it with the powerful model and then compare it with the lightweight model: accuracy is almost the same (96% vs 94%), but the lightweight model is 8 times cheaper and 3 times faster. The lightweight model is by far the right choice for this task; The extra 2% accuracy does not justify 8x the cost.
Case 2 — Strong model + reasoning is a must. A tax consulting firm wants to calculate a complex multi-stage tax scenario. The lightweight model makes mistakes in intermediate steps and gives incorrect results. When they use the powerful model with reasoning mode turned on, the model progresses step by step and reaches the correct conclusion. Here, additional costs and delays are acceptable because the cost of error is high; However, an expert confirms the result.
Case 3 — Mixed use. A call center processes call transcripts in two steps: first, it briefly summarizes each call with a lightweight model (high volume, simple job), then it gives only the calls marked as "complaints" to the powerful model and performs a root cause analysis (low volume, complex job). So they process the bulk of the volume cheaply, while reserving the power of the strong pattern for the few calls that really matter.
Weak Prompt / Strong Prompt
Weak prompt:
Solve this tax scenario. [complex scenario]
It is unclear how carefully the model will consider.
Powerful prompt:
Your role: senior tax specialist. Task: Solve the following multi-stage scenario step by step. First list the assumptions, then calculate and show each step separately, and finally give the net result. If there is a point that you are not sure about or that depends on the legislation, mark it as "expert confirmation required". Skipping intermediate steps.[scenario]
The powerful prompt guides the model through step-by-step thinking (encouraging reasoning) and makes intermediate steps visible for verifiability.
Copiable Templates
Stage 1 diagnosis:
My mission: [TASK]. Volume: [NUMBER/day]. Error cost: [LOW/MEDIUM/HIGH].Speed priority: [LOW/HIGH]. Based on this profile, what level (light/balanced/strong) and reasoning (open/closed) would you recommend? Justify.
2. Small/large model decision:
I define the following task: [TASK]. Can this problem be solved with a small (light) model, or is a large model required? Suggest 3 test cases and a success threshold for me to try solving with the small model.
3. Appropriateness of reasoning:
Would the reasoning (step-by-step thinking) mode be beneficial or costly in the following task: [TASK]? Justify your decision based on the number of steps and logical complexity of the task.
4. Two-stage workflow design:
Design a two-stage flow to reduce cost on the following high-volume task [TASK]: which part should go to the lightweight model, which part should go to the powerful model? Write the level and rationale for each step.
Common mistakes
- A powerful model for every job: Creating unnecessary costs and delays by using the most powerful model in simple and voluminous jobs.
- Always leaving reasoning on: Keeping the thinking mode on in simple tasks and wasting time and money in vain.
- Making decisions without measuring quality: Assuming "bigger is better" without comparing two levels with your own data.
- Forgetting the need for speed: In places where an instant response is required, such as chat, choosing a slow powerful model and keeping the user waiting.
- Not thinking about the two-stage flow at all: Not having the entire volume built by a single model and not saving on the part that can be done cheaply.
In summary
- Model selection is a balancing act in the triangle of quality, speed and cost; Improving two often requires compromising the third.
- Light stage is suitable for simple/bulky jobs, strong stage is suitable for complex/critical jobs; The balanced stage is a good start for most jobs.
- Reasoning mode improves accuracy in multi-step problems but increases cost and latency; It should not be left open to every task.
- "What is the cost of failure in this mission?" The question is the compass of the decision of level and reasoning. Two-stage flows provide great savings in volume jobs.
Application task
Select a task in your job and determine which stage and reasoning setting is appropriate with template 1 (stage diagnosis). If the task is high volume, come up with a design that directs some of the volume to the lightweight model and the critical portion to the strong model with template 4 (two-stage workflow). We will use this design in the model portfolio study in unit 9.
checklist
- [ ] I can match the work character of light, balanced and powerful stages.
- [ ] I can explain what reasoning mode does and what it costs.
- [ ] I can consciously establish the quality-speed-cost balance for a task.
- [ ] I can answer the small or large model question with three questions.
- [ ] I can design a two-stage flow for a high volume job.