Diagnosing Lexical Diversity with Type-Token Ratio (TTR) — and Its Length-Bias Trap
When judging the quality of a piece of writing, "how much variety is there instead of repeating the same words" is an important measure. TTR (Type-Token Ratio) is the number that quantifies it. The calculation itself is simple, but trusting this single figure for comparisons can lead you to entirely wrong conclusions because of text length. This guide covers what TTR is, why the value mechanically drops as text gets longer, and how linguistics and NLP correct for that problem — all with real worked numbers.
1. What is TTR?
TTR is defined as number of unique words (types) ÷ total number of words (tokens). For example, if a text has 100 total word occurrences and 70 of those are distinct words once duplicates are removed, the TTR is 0.70. A value closer to 1 means the writer barely repeated the same word, while a value closer to 0 means a small set of words was repeated heavily. The Word Counter tool displays both the total word count and the unique word count on screen, so anyone can compute TTR themselves just by dividing the two.
2. Why TTR drops on its own as text gets longer
TTR's biggest trap is that it's fundamentally sensitive to text length. A short piece hasn't had much chance yet to repeat words, so TTR comes out high; as the text gets longer, the odds of reusing an already-used word (particles, conjunctions, frequently used nouns, and so on) grow exponentially, and TTR inevitably falls. This happens as a purely mathematical consequence of text length alone, regardless of the writer's actual vocabulary skill.
So a comparison like "this piece has a TTR of 0.35, that one has 0.55, so that one has richer vocabulary" is only meaningful when both pieces are the same length. Comparing the raw TTR of a short social media post against a 5,000-word blog post is a flawed comparison from the start.
3. Measures that correct for length bias
Linguistics and corpus research use several corrected measures to address this problem.
| Measure | Calculation | Notes |
|---|---|---|
| Guiraud's Index | unique words ÷ √(total words) | Dividing by the square root softens the length effect. Simple to compute, so it's commonly used in practice |
| MSTTR (windowed TTR) | Split the text into fixed-size chunks (e.g. 100 words), compute TTR for each chunk, then average | Every chunk being the same length means length bias effectively disappears |
| MTLD | Cut a new segment ("factor") each time the cumulative TTR drops below a threshold (typically 0.72), averaging forward and backward passes | Proposed by McCarthy & Jarvis (2010); the standard measure in corpus linguistics |
Applying Guiraud's Index to the example above (10 words, 8 unique) gives 8÷√10 ≈ 2.53. The 20-word version made by repeating the same sentence twice gives 8÷√20 ≈ 1.79 — still lower, but nowhere near the raw TTR's straight halving, because the square root dampens the effect of the added length. MSTTR fixes the chunk length outright, so averaging over 100-word windows lets you compare a 1,000-word text and a 10,000-word text on the same footing.
4. Putting it to practical use
Paste your text into the Word Counter tool and it immediately shows the total word count and the unique word count. Here's how to put those numbers to use by computing TTR yourself:
- Compare segments within the same piece: Split the whole piece into a few paragraphs, paste each one into the tool separately, and compare TTR across them to spot where word repetition is heaviest (a manual version of MSTTR).
- Only compare pieces of similar length: If you want to compare this article's TTR against a previous one, make sure the two have roughly the same total word count, or convert both to Guiraud's Index before comparing.
- A separate issue from keyword repetition: Keyword repetition for SEO purposes is intentional, so a low TTR doesn't automatically mean bad writing. Checking whether a specific keyword is repeated excessively is more accurately done with the Keyword Density Checker.
For ordinary natural prose (essays, blog posts), a TTR around 0.5–0.7 for a roughly 500-word piece is generally a reasonable range, and dropping below 0.3 is worth checking for excessive repetition of a specific word — usually a topic term or a particle-like filler word. That said, this range is a reference point, not a hard rule, and it should always be weighed alongside the piece's genre and length.
Frequently Asked Questions
Q. Does a higher TTR always mean better writing?
No. TTR only shows the degree of vocabulary repetition and has nothing to do with a piece's logic, readability, or accuracy. An unusually high TTR can even mean the writer kept rephrasing the same concept with different words each time, which can actually make a piece harder to read.
Q. Is there any way to compare pieces of different lengths?
Guiraud's Index (unique words ÷ √total words) or MSTTR (averaged windowed TTR) lets you compare pieces of different lengths fairly reasonably. Academic research uses the more sophisticated MTLD as the standard.
Q. Is TTR calculation accurate for Korean text?
Korean counts "words" as whitespace-separated eojeol units. Because particles attach directly to words, Korean text tends to register a higher unique-word count than a morphologically-analyzed language would (for example, "학교는" and "학교를" get counted as different tokens), so directly comparing Korean and English TTR values is not recommended.