Offline document extraction for Tunisian Arabic & French

The problem

Tunisian identity and business documents - ID cards, registration certificates - had to be turned into structured data inside a bank. The text is Arabic and French, frequently both on the same document, and Arabic is what conventional OCR handles worst.

The output feeds downstream systems, so a field that is confidently wrong costs more than a field that comes back empty.

The constraint

This is the part that shaped every other decision. The service runs fully offline on ordinary office hardware inside the bank's network: no GPU, no internet, no cloud API, and a hard 2 GB RAM budget. That budget was not a preference, it was what the machines it had to run on actually had.

Holding it meant memory-mapped weight loading, a 1024-token context, CPU-only inference, capping image width before it reaches the model, and optional KV-cache quantisation. Each one is a named lever rather than a general intention, and each is verified by a benchmark script on the target machine rather than assumed.

What I built

The pipeline reads the document image directly with a small quantised vision-language model and returns JSON. There is no Tesseract and no PaddleOCR anywhere in it: the model reads the picture, which is what makes the Arabic tractable at all.

Two models sit behind a single registry interface, so a deployment can swap between them without any calling code changing. Decoding is fully deterministic, so the same document always produces the same output. Pydantic models validate the JSON at the boundary, and a document the model cannot read is skipped rather than guessed at.

What it measured

The two models trade accuracy for latency, and the benchmark makes that trade explicit rather than leaving it to preference. The 2B model has noticeably stronger Arabic and runs in 20-40 seconds per document; the 500M model runs in 3-8 seconds, roughly five times faster. The benchmark runs on the actual target machine, so each deployment chooses on measured numbers rather than on what happened to be quick on a developer laptop.

The more important measurement was of the data, not the model. The supplied ground truth turned out to be machine-OCR pseudo-labels rather than human transcription: only 13 of 48 entries carried a valid number. The cause was upstream - 120 of the 201 cards were rotated in their source photographs, so the labelling pass had been reading them sideways.

I rebuilt the dataset by hand against a resumable runbook with an append-only ledger and a skip-rather-than-guess rule, then generated a 5,000-document synthetic corpus on top of the verified set. The operations guide says plainly where a figure rests on synthetic data rather than implying real-world performance.

What I'd change

I would test the labels before testing the model. Several days went into tuning against a ground truth that was wrong, and a ten-minute check on label validity would have caught it.

I would also generate the synthetic corpus from the verified transcriptions rather than alongside them, so the synthetic distribution follows the real one instead of approximating it.

At a glance

• Runs fully offline on CPU - no GPU, no internet, no cloud API - small enough to deploy on ordinary office hardware.
• Ships two interchangeable models behind one interface: a larger one with stronger Arabic, and a smaller one around five times faster. A benchmark script picks between them on the actual target machine rather than on assumption.
• Rebuilt the ground-truth dataset from scratch after finding the supplied labels were machine-generated guesses - only 13 of 48 entries had a valid number, because 120 of 201 cards were rotated in their source photos. Hand-transcribed against a resumable runbook with a skip-rather-than-guess rule, then generated a 5,000-document synthetic corpus.

Stack