What is a token in AI? A plain-language guide

So, what is a token in AI? A token is the small piece of text a language model reads and writes: often a whole
short word, sometimes part of a longer one, a number or a punctuation mark. Models never see letters or words, only
tokens, and every AI provider bills by them. If your company pays for AI, tokens are the meter that runs.

This guide is for the people who roll AI out rather than build it: HR, L&D and operations teams who keep hearing
"tokens" in budget meetings. It explains what a token is, why the same email costs more in German than in English,
why an AI agent spends far more tokens than the words you type, and how teams spend fewer. Every product figure below
was checked on 27 October 2026.

What is a token in AI? One picture

Picture a sentence cut into tiles. Counted with OpenAI's open o200k tokenizer, "Please summarize the attached report"
makes five tiles: "Please", " summarize", " the", " attached", " report". Write it the British way, "summarise", and
it makes six: the rarer spelling is cut into " summar" and "ise". Common words are usually one tile each, with the
space in front of them included; rarer words are cut into pieces the model already knows. A date such as 2026-11-02
makes six tokens ("202", "6", "-", "11", "-", "02"), so figures cost more than their length suggests.

The program that does the cutting is a tokenizer. Each model family has its own, with its own vocabulary of
pieces. OpenAI publishes one of its tokenizers as open source: tiktoken, whose
documentation says that in practice each token corresponds to about 4 bytes of text, roughly four characters of
English. Once the text is cut, each piece becomes a number, and those numbers are what the model actually processes.

You can watch this happen. Our free token counter colors each word by the number of tokens it
adds: blue for one, yellow for two, orange for three, pink for four or more. Paste a paragraph from your own inbox and the pattern is clear
within seconds: short common words are cheap, names, figures and long compounds are not.

Tokens in AI are not just words

Three things surprise most people the first time they count:

1. Spaces and punctuation count. A comma, a line break or a bullet point can each be a token, or part of one.
2. Spelling and form change the count. " summarize" (with the space before it) is one token in o200k,
   " summarise" two; "Report" at the start of a sentence and " report" after a space are two different tokens of
   the vocabulary.
3. Tokens go both ways. What you send is counted as input tokens, what the model writes back as output tokens,
   and output is usually priced higher (more on that below).

How many tokens is a word?

There is no fixed answer, because it depends on the tokenizer and the language. As a check, we counted the prose of
this article with o200k: about 2,200 words made about 2,900 tokens, about 1.3 tokens per word (it is full of
figures, which cost extra). Claude, on our measurements below, counts more again. Treat any such rule as a rough guide: the only reliable figure is a count made with the
tokenizer of the model you will actually use.

Why languages cost differently

A tokenizer's vocabulary is built from the text it was trained on, and in practice common English words tend to be
single pieces while German compound words and French accented forms are more often split into several. The result:
the same message makes more tokens in German or French than in English, and costs more.

We measured this on our own texts. On 27 October 2026 we counted 36 texts in English, German and French (emails,
prompts, meeting notes, reports, tables and code) twice: once with Claude's own token count, once with OpenAI's open
o200k tokenizer. Here is what Claude counted for each o200k token:

Two lessons follow for a company working in several languages. First, a German team sending the same requests as an
English team will spend noticeably more tokens. Second, you cannot read one model's count off another model's
tokenizer: the ratio moves from text to text, so a count made with OpenAI's tokenizer is only an estimate for Claude.

Tokenizers also change between model generations. Anthropic's own
token counting documentation notes that
the models from Claude Opus 4.7 on use a newer tokenizer on which the same text produces approximately 30 percent more
tokens than on earlier models, and advises recounting prompts for the model you plan to use. A budget sized on last
year's model may not hold on this year's.

What an agent reads at every step

Here is the part most token explainers miss, and the one that matters most for a company rolling out AI agents.

A chatbot answers once. An agent works in steps: it reads a file, makes a change, checks the result, then decides
what to do next. At every one of those steps the model reads its whole context again: its own instructions, the
conversation so far and what its tools returned. Anthropic's
context window documentation spells this
out: the system prompt, every message (including tool results and documents) and the tool definitions all count, and
each turn's input contains the whole previous conversation plus the new message.

So the cost of a request is not the length of what you type. It is the number of steps the agent takes, multiplied by
everything it re-reads at each step.

We measure this in our own training sessions, where every request is counted step by step:

• The agent of our sessions re-reads about 20,700 tokens of instructions at every step. Claude Code reads about
  8,600.
• So a single step costs at least about 21,000 tokens, even when you type one short line.
• A request that takes 2 to 5 actions costs 45,000 to 117,000 tokens.

Compare that with the message itself. "Add our phone number to the footer" is a handful of tokens. The work behind it,
looking at the page, making the change and checking it, reads tens of thousands. A team of 40 in claims handling,
each sending an agent ten such requests a day, is not spending "a few words" of tokens: at 50,000 a request, that is
20 million tokens a day.

The context window: how many tokens fit

Everything the model reads at once has a ceiling, the context window, measured in tokens. Anthropic's
documentation lists a 1M-token context window for its latest models, Claude Haiku 5.5 and Claude Sonnet 5.5
included, with up to 128k output tokens in a single request. The same page warns that more context is not
automatically better: as the token count grows, accuracy and recall degrade, which it calls context rot. A long
conversation is therefore both dearer, because every step re-reads it, and less reliable.

Tokens and money

AI providers publish their prices per million tokens (MTok), with input and output priced separately. On
Anthropic's pricing page, checked on 27 October 2026:

Three things stand out in that table:

1. Output costs five times input on both models. Long answers cost more than long questions.
2. Re-reading is discounted when it is cached. Instructions an agent reads again and again can be served from a
   cache: a tenth of the input price on Haiku 5.5, a twentieth on Sonnet 5.5. That is why agents are affordable at all, and why the number of
   steps still matters: each one still reads, and still pays.
3. Model choice multiplies everything. In our sessions, a typical step with Sonnet 5.5 costs about 12 to 14 times
   what it costs with Haiku 5.5, for the same result on a small change.

A worked example, with the list prices above and simple arithmetic. Say a team sends 1,000 agent requests a month at
about 100,000 tokens each: 100 million input tokens. At Haiku 5.5's uncached input price that is about $10; at Sonnet
5.5's, about $200. Caching lowers both, output adds to both, but the ratio between the models, and between a
two-step request and a five-step one, stays.

The money per request looks small. What makes AI budgets drift is volume: thousands of employees, each sending vague
requests that take more steps than they need.

How to spend fewer tokens

The good news: the biggest savings come from habits, not engineering. Here is what we see make the difference.

1. Say what "better" means in the first request

Vague requests are the most expensive thing an employee can type. In one of our exercises, "Ask Once, Ask Clearly",
the request "improve it" took 113,000 to 114,000 tokens: the agent took four actions and restyled the whole page,
because nothing told it what to change. The precise request took 49,000. Our landing page shows the same effect on a
two-message exchange: saying what "better" means in the first request, one message instead of two, saves about 94,000
tokens.

2. Give the four parts of a request

A request an agent can act on in one go has four parts: the context (who it is for, what exists already), the
task (what to do), the guidelines (how to do it) and the constraints (what not to touch, limits, format).
An HR team writing a job ad with AI gives the agent what it needs to get it right the first time when it names the
role, the audience, the tone and the length; give it only a title and the rounds of "can you make it shorter" begin. If you want to practise turning a
loose brief into one complete request, try Prompt Golf, our free game scored on exactly that.

3. Start a new conversation for a new task

Every step re-reads the conversation so far. When the next task has nothing to do with the last one, a fresh
conversation means the agent stops paying to re-read documents it no longer needs.

4. Pick the model for the job

A small, well-described change does not need the largest model. Reserve the dearer models for the work that needs
them, and let teams see what each choice costs.

5. Count before you scale

Before a prompt becomes a template that 2,000 people use daily, count it. Our token counter
gives an exact o200k count and an estimate of Claude's in your browser, with nothing sent anywhere. For an exact Claude
figure, Anthropic's token counting endpoint is free to use, though its documentation calls the count an estimate that
can differ slightly from what a real message uses.

Practise it, rather than read about it

Knowing what a token is changes little on its own. What changes spending is employees seeing what their own requests
cost. In Tokenacademy's sessions, each employee works with a real AI agent in the browser; some sessions cap the
tokens a learner may spend, and at the end a report goes through the attempt request by request with the tip that
saves the most. To see it with your own teams, book a demo, or read more on
the Tokenacademy blog and our hands-on AI training.

FAQ

What is a token in AI, in simple terms?

A token is a piece of text that an AI model reads or writes: often a short word, sometimes part of a longer word, a
number or a punctuation mark. Models process text as tokens, and providers bill by the number of tokens read and
written.

How many tokens is 1,000 words?

It depends on the tokenizer and the language. Counted with OpenAI's o200k tokenizer, the prose of this article makes
about 1.3 tokens per word, so 1,000 words of similar English come to roughly 1,300 tokens. The same text in German
or French makes more, and Claude's count is higher again: on our 36 measured texts, Claude made about 1.47 tokens per
o200k token in English.

Why does the same text cost more tokens in German or French?

Common English words are often a single token, while German compounds and French accented words are more often split
into several. On our measured texts, Claude counted about 2.04 tokens per o200k token
in German and 1.74 in French, against 1.47 in English.

Are input and output tokens priced the same?

No. Output tokens usually cost more. On Anthropic's published prices for Claude Haiku 5.5 and Claude Sonnet 5.5,
output costs five times input.

Why do AI agents use so many tokens?

Because they work in steps, and at each step the model re-reads its instructions, the conversation and its tools'
results. The agent of our training sessions re-reads about 20,700 tokens of instructions at every step, so even a
short request costs at least about 21,000 tokens, and one with several actions 45,000 to 117,000.

How can I count the tokens in my text?

Paste it into a token counter. Ours, at /tools/token-counter, runs in your browser, gives an exact count for OpenAI's
o200k tokenizer and an estimate of Claude's, and sends nothing anywhere.