OCR Text Recognition: How a Machine Reads Paper

Every OCR product on the market advertises a single accuracy number. Not one of them will tell you which of its six processing stages broke your page, which is the only fact that helps you fix it.

OCR text recognition sounds like one operation: look at the picture, write down the letters. It isn't. It's a pipeline, and most of the failures people blame on "bad OCR" happen in stages that never look at a letter at all.

Six Stages Between Image and Text

Optical character recognition takes an image and hands back characters. Between those two points sit six steps, each with its own way of going wrong.

Read that table backwards and it becomes a diagnostic tool. Speckled output with commas that were never on the page means the threshold step misjudged your paper. Two columns woven together line by line means layout analysis picked the wrong reading order. Nothing at all usually means the page was upside down.

Stage six deserves special attention, because it explains the errors that hurt most.

The language model exists to nudge output toward real words. Recognition produces candidates, the dictionary votes, plausible spellings win. It works well, and it works only on words. There is no dictionary for an account number. No dictionary for a dosage, a case number, a part code, or a sum of money. The step that quietly repairs the rest of your page cannot help the characters you care about most, and it will occasionally "correct" a proper noun or a reference code into something that looks tidier and means nothing.

Why "99% Accurate" Tells You Nothing

Character accuracy and word accuracy are different measurements that answer different questions, and vendors quote whichever flatters them.

Tesseract's own regression testing makes the point better than any critic could. Moving from its legacy engine to the modern LSTM engine cut character errors by 17% on a standard corpus while raising word errors by 22.6%. Same engines, same pages, opposite verdicts. Which number you publish decides whether that release was an improvement.

Then there's what the standard metric leaves out. The canonical benchmark in this field, the UNLV/ISRI accuracy reports that most later work builds on, defines word accuracy in a way that states plainly that errors in recognizing digits or punctuation have no effect on the score. Digits are simultaneously the worst-recognized class of character. On UNLV's numeric-heavy sample, digit accuracy read as low as roughly 87.6% while lowercase letters sat near 98%.

So the headline metric excludes the characters that fail most often and matter most. A page can score 99% word accuracy and still have a wrong figure in every table on it.

One more distribution problem, from the same reports: between 50% and 75% of all errors land on the worst 20% of pages. Accuracy is not spread evenly across a document. Spot-checking three random pages and finding them clean tells you almost nothing, because the damage is concentrated somewhere you didn't look.

Accuracy Is a Property of the Page

The most instructive number in OCR research comes from Google's own team adapting Tesseract for languages beyond English.

Tesseract estimates where the x-height of a line sits by assuming ascenders and short letters separate into two clear populations, which holds for most Latin fonts. Applied unchanged to Cyrillic, it doesn't. On a set of Russian books, the word error rate came out at 97%.

Not because Cyrillic is harder. Because one geometric assumption about letter heights stopped being true. After the fix, the same engine scored 1.35% character error on Russian. Hindi in the same paper still sat at 15.41% character error and 69.44% word error, mostly down to older typography missing from the training data.

An engine does not have an accuracy. It has an accuracy on a particular script, in a particular typeface, at a particular size, on a particular quality of scan.

There Is Such a Thing as Too Much Resolution

Everyone repeats "scan at 300 DPI." Tesseract's documentation does say it works best at a minimum of 300. What almost nobody mentions is that the same documentation describes an upper limit.

The recognizer wants x-height, the height of a lowercase letter without ascenders, to land in a usable pixel range. Below about 10 pixels, the docs say you have very little chance of accurate results. Above roughly 30 pixels, the LSTM engine stops producing accurate results. At 300 DPI, 10-point body text gives you an x-height around 20 pixels, sitting comfortably in the middle.

Scan that same text at 600 DPI and you push x-height outside the range the engine was built for. Researchers measured exactly this on typewritten pages: Tesseract 4 scored 90.49% character accuracy at 300 DPI and 47.60% at 600 DPI. A commercial engine on the same pages went from 95.02% to 92.89%, so this is a property of one engine's design rather than a law of scanning.

The practical version: pick the resolution that puts letter height in range, which means going above 300 only when the print is genuinely small. And note that authorities disagree on the number itself. The Library of Congress, which specifies OCR capture separately from preservation imaging, says 400 ppi.

Two settings beat resolution for return on effort. Feeding greyscale instead of your scanner's black-and-white document mode roughly halved character errors for two independent engines in the UNLV tests, taking one from 97.02% to 98.50%. And squaring the page matters more than it looks, because skew damages line segmentation rather than only letter shapes. Merged lines are unrecoverable later.

Handwriting Is a Different Problem

Handwriting recognition gets filed as "OCR, but harder." It's a different technology with a different failure mode, and one number shows why.

Take a model trained on a standard handwriting corpus and point it at a hand it has never seen: roughly 50.5% character error, which is unusable. Fine-tune that same model on 26 lines written by that specific person and error drops to 12.5%. At 526 lines, 3.9%.

Nothing about the model got better. It learned one person's handwriting. Printed text works because a typeface is shared by millions of documents. Handwriting is unique per writer, so recognition quality depends on adaptation rather than on any quality setting you can pick from a dropdown.

For classic OCR engines, the blocker isn't even the recognizer. A Tesseract maintainer has stated the layout stage can't separate text lines in handwritten text, and points people toward tools built for handwriting instead. If you're feeding cursive to a standard OCR engine and tuning settings, you're tuning the wrong thing.

Modern vision models do better here, with a caveat worth knowing. They fail differently rather than less. Their documented failure mode is fluent invention: repeating a line until the output collapses, or silently "fixing" what the page actually said. For transcription, output more correct than the source is a serious problem, and an invisible one.

Where the Recognition Runs

OCR used to mean a server, because the models were too heavy for anything else. That stopped being true.

Compiled to WebAssembly, an English recognition model plus engine is roughly 4 MB on first load, and a 300 DPI A4 page takes about two seconds on a normal desktop. Call it twice the time of the same engine running natively. Most of that gap is SIMD instruction width, which is worth around 3.4× on its own, so a browser that supports it does most of the work a native binary would.

That matters because of what gets fed to OCR. Insurance cards, driver's licences, consent forms and patient registration make up about 35% of volume in the one large peer-reviewed census of scanned health records. Identity documents, not clinical notes.

The published terms for cloud OCR vary more than most people assume, and some are genuinely good. Google's Document AI documentation commits to deleting documents immediately after processing with a one-day failsafe, states it never trains on customer content, and says no employee sees the documents. Azure's Document Intelligence deletes input and results after 24 hours, though its privacy page says nothing either way about training.

Two entries deserve a closer read. AWS Textract is opt-out, not opt-in: the AWS Service Terms grant Amazon permission to use and store processed content to improve the service unless an administrator configures an organization-level opt-out policy, and Textract is not included in the medical carve-out that covers several sibling services. Deletion is by support request. Separately, the Gemini API splits on billing status. On the unpaid tier, the terms say human reviewers may read, annotate, and process your input and output, and instruct you not to submit confidential information. On the paid tier, they don't. Same model, opposite data posture.

None of that makes cloud OCR wrong. It makes it a decision, and the honest version of the decision needs the document in front of you.

Reading Your Own Output

Once you know the pipeline, checking OCR takes about a minute.

• Search three words from different parts of the page, including one from a heading or a table.
• Verify every number by hand. Digits fail most and are excluded from the metric that scores your document.
• Check your worst-looking pages, not random ones. That's where most errors live.
• Select a paragraph and paste it somewhere you can read it, which surfaces reading-order problems that Ctrl+F won't.
• Keep the searchable PDF rather than only the extracted text, so the original pixels stay attached to the words and you never OCR the file twice.

If output looks wrong, the pipeline tells you what to change. Speckling points at thresholding, so send greyscale. Merged lines point at skew, so rescan squared. Interleaved columns point at layout analysis. Mangled small print points at x-height, so scan that document hotter. Garbled accents point at the wrong language pack.

Doing It Without Uploading Anything

OCR PDF runs in your browser. Five of its eight engines never send a page anywhere: Tesseract for Latin scripts, two sizes of PaddleOCR that handle Asian languages well, and two sizes of Florence-2 on WebGPU for messier layouts. Twenty languages, three quality tiers, and a smart mode that checks for an existing text layer first so you don't OCR a document that was already digital.

The remaining three engines are cloud models, labelled as such, opt-in, and useful for complex tables and mathematics where on-device models still struggle. You pick. Output comes back as clipboard text, a .txt file, or a searchable PDF with the text layer written over your original pages, and batch mode does a folder at once.

For the step-by-step version of that workflow, see how to make a scanned PDF searchable. If your PDF was never scanned in the first place and text still won't copy, that's a different problem with a different fix, covered in why your PDF won't give up its text.

The Useful Takeaway

Worth keeping:

• OCR is six stages, and four of them never see a letter. The shape of your bad output tells you which one failed.
• A single accuracy number is marketing. Word accuracy commonly excludes digits, and digits fail most, so check your numbers by hand every time.
• Where recognition runs is a choice now. In-browser OCR costs about 4 MB and two seconds a page, which is a small price for a document that never leaves your machine.

Paper doesn't come with a text layer. Once you know how the machine builds one, a bad result stops being bad luck and starts being a specific thing you can go fix.

Run OCR on a scanned document in your browser, with nothing uploaded.