AI Content Detector — Perplexity and Burstiness, Measured Locally

No. No detector can, including the paid ones that quote a confident percentage. Perplexity measures how predictable text is to a language model, and clear, conventional, well-edited human writing is also predictable — that is what makes it clear. The measurement is real; the inference from it to authorship is not sound.

Because a percentage implies a calibrated probability, and there is no calibration behind it. People are failed, disciplined and fired on the strength of those numbers. Showing the underlying measurements against their baselines lets you interrogate the reading instead of deferring to it.

The exponential of the average negative log-probability the model assigned to each actual token — in plain terms, how surprised it was. Low perplexity means the text went the way the model expected. Generated text tends to be low because it was produced by maximising exactly that predictability.

The variation in perplexity from sentence to sentence. Human writing lurches: an ordinary sentence, then a strange one, then a digression. Sampling from a probability distribution produces a flatter profile. Burstiness is the more durable of the two signals because it survives light editing.

Most likely because it is clear and conventional. Technical documentation, legal text, formal academic prose and writing by non-native English speakers all score as predictable. Published research specifically found elevated false-positive rates for non-native writers — that is a property of the method, and the reason this tool refuses to convert its numbers into an accusation.

GPT-2, because it is small enough to run in a browser and is what most published detection work is calibrated against. It is a weak proxy for what a current frontier model finds predictable, so recent model output separates less cleanly than the original papers suggest. Treat the numbers as descriptive, not diagnostic.