How many tokens is 1,000 words?
1,000 words of English prose is roughly 1,330 tokens, because one English word averages about 1.33 tokens. In the other direction, 1,000 tokens is roughly 750 words and 256 tokens is roughly 192 words. Source code and JSON cost far more per word, around 2.4 and 2.7 tokens, so the same word count of structured data is close to double.
Runs in your browserAlso asked as
- 256 tokens to words
- word to token
- tokens to words
- how many words is 1000 tokens
Do it here: Words to Tokens Converter
Convert between words, characters, pages and tokens using measured ratios for prose, code, JSON and CJK.
- English prose
- 1 word is about 1.33 tokens; 1 token is about 4 characters
- Source code
- 1 word is about 2.4 tokens; 1 token is about 2.6 characters
- JSON
- 1 word is about 2.7 tokens; 1 token is about 2.6 characters
- 1,000 words
- About 1,330 tokens of prose, 2,400 of code, 2,700 of JSON
- 1,000 tokens
- About 750 words of prose
Words to tokens
| Words | Prose tokens | Code tokens | JSON tokens |
|---|---|---|---|
| 100 | ~133 | ~240 | ~270 |
| 250 | ~333 | ~600 | ~675 |
| 500 | ~665 | ~1,200 | ~1,350 |
| 1,000 | ~1,330 | ~2,400 | ~2,700 |
| 5,000 | ~6,650 | ~12,000 | ~13,500 |
| 10,000 | ~13,300 | ~24,000 | ~27,000 |
Tokens to words
| Tokens | Words of prose | Roughly |
|---|---|---|
| 256 | ~192 | A long paragraph |
| 512 | ~385 | A page |
| 1,024 | ~770 | Two pages |
| 4,096 | ~3,080 | A short article |
| 8,192 | ~6,160 | A long article |
| 128,000 | ~96,000 | A short novel |
Why it is not one token per word
Language models do not split text on spaces. They use byte pair encoding, which builds a vocabulary of the most frequent character sequences in the training data and then represents any text as a sequence of those pieces. Common short words such as "the" and "and" are single tokens. Longer or rarer words are split into several: "tokenisation" might arrive as three pieces, and a proper noun the vocabulary has never seen can become one token per syllable or worse. Punctuation, leading spaces and newlines are tokens too.
That is also why the ratio rises for code and JSON. Braces, brackets, quotes, colons and indentation are all separate tokens, and identifiers written in camelCase or snake_case split at the case change or the underscore. A JSON document is mostly structure, so it spends most of its tokens on characters a reader would not count as words at all.
What moves the number
- The tokenizer. Different model families ship different vocabularies, so the same paragraph can differ by several percent between them.
- The language. Chinese, Japanese and Korean are counted closer to one token per character, and languages with long compounds such as German or Finnish split more than English.
- Formatting. Markdown tables, code fences and heavy indentation add tokens without adding words.
- Names and identifiers. Unusual proper nouns, UUIDs and hashes are the most expensive text per word there is.
What to do with the number
- Context window calculator checks whether a document plus its prompt and expected reply fits a model.
- AI API cost calculator turns a token count into a price.
- Prompt token optimizer trims a prompt that is over budget.
- Text chunker for RAG splits a long document into pieces that fit.
- Word counter gives you the word count to start from.