Why your prompt costs what it costs

Last lesson you learned that models see tokens, not words. Now let’s follow the money — because every single token is billed, twice over: once going in (your prompt), once coming out (the answer). And output tokens typically cost 3–5× more than input tokens.

That price gap isn’t greed. It’s physics, and it’s the single most useful thing to understand about LLM costs. Let’s see it first, explain it second.

The pause and the drip: prefill vs. decode

Did you catch the two phases?

Phase 1 — the pause (prefill). The model reads your entire prompt in one parallel gulp. A thousand input tokens can be processed simultaneously, like a scanner slurping a whole page. This is cheap per token. This is why input is the discount aisle.

Phase 2 — the drip (decode). Generation cannot be parallelized, because of a brutal dependency: to produce token #2, the model needs to know what token #1 was. Each output token requires one full pass through the entire network — billions of calculations — to produce one chunk. Then again. Then again. The word-by-word appearance you’ve always seen isn’t a loading animation and never was: you are watching the model think in real time, and every drip is a line item.

One machine, two modes: photocopier on the way in, calligrapher on the way out. Now the pricing table of every AI provider on Earth makes sense — and so does a strategy: be generous with input, stingy with output. A longer prompt that produces a shorter answer is often the cheaper prompt.

Experiment 1: The polite tax

Being polite doesn’t improve the answer — the model has no feelings to flatter — but note the sneaky second effect: a chatty prompt often triggers a chatty response, and the response is the expensive part. (Important distinction for later modules: extra context often makes answers better. Extra padding never does. This course is largely about learning to tell them apart.)

Experiment 2: Muzzling the expensive end

Since output costs 3–5× more, the highest-leverage cost control is controlling the answer. Two muzzles, compared:

There’s also a hard muzzle: the max_tokens setting cuts the model off mid-sentence when the budget runs out. Look at the receipt for a field called finish_reason — it says stop when the model finished naturally, and length when the guillotine dropped. Every mysteriously amputated AI answer you’ve ever seen was a finish_reason: length — and now you’ll never be confused by one again.

Experiment 3: Deutsch kostet extra

Last lesson’s German-tax discovery, now with a price tag:

This isn’t just a curiosity; it’s a fairness problem baked into the technology. A user in Yangon can pay multiples of what a user in Ohio pays for the same question, because the tokenizer’s compression game was played mostly on English text. If you build for German users, your real per-request costs are higher than any English-based estimate — measurably, and now you can measure it.

Two plot twists on the bill

Twist 1: You can pay for tokens you never see. Modern “reasoning” models think before answering — producing internal chains of thought that are hidden from you but billed as output tokens, the expensive kind. Ask a hard riddle, receive a one-line answer, get billed for 2,000 tokens: the other 1,950 were the model muttering to itself. When your engine is connected to a thinking-capable model, the receipt grows a reasoning_tokens line — the muttering, itemized.

Twist 2: Repetition earns a discount. Providers can cache the prefill of a prompt-prefix they’ve recently processed — send the same long system prompt a thousand times and reruns of the cached part are often 50–90% cheaper. This is why well-built apps put the stable stuff (instructions, documents) first and the changing stuff (the user’s question) last. File that away: it becomes a design rule in the prompting module.

The scale check (why any of this matters)

For you, tonight, playing in this playground: none of this matters. A thousand experiments cost less than a coffee. The arithmetic only becomes real at scale — a support bot handling 100,000 chats a day, where shaving 50 output tokens per answer saves five million tokens daily. Token thrift is like fuel efficiency: irrelevant for a lap around the parking lot, decisive for an airline. Learn the habits now while they’re free.

Exercise: The prompt diet

Below is a bloated prompt. Your mission: get it under 15 prompt tokens without changing what the model does. This time the checker judges both halves: the prompt budget and the French translation that comes back.