What went in
152 test kit documents built around a deliberate trap.
The test kit came in 5 document conditions, from clean PDFs to faxes. 152 of its documents were built around a trap: a total that is not the amount owed, a bill sent twice, a due date that is missing. Below is a real page for each document condition and for 15 of the 18 kinds of trap. Every business on them is invented. The counts on this page describe the whole test kit. Its 10 duplicate trap PDFs were not sent and are not part of the results.
Document conditions
5 ways the same invoice arrives.
A vendor's PDF is the easy case. The rest were rendered as scans, phone photos, faxes and photocopies, so there is no text to read, only pixels.
Clean PDF
326 documents in the test kit · this one is a gutter supply invoice
The PDF exactly as the vendor's billing software made it, with every character still real text.
Traps
15 of the 18 kinds of trap, in 5 places on the page.
Each trap hides where a real one would: in the totals, the line table, the dates, the letterhead, or in a second copy of a bill you already have. Pick a part of the invoice to see the traps printed there.
What is owed
40 documents in the kit · 3 kinds of trap
Total is not the amount owed
The invoice totals $658.22, a $656.90 payment is already applied, and $1.32 is due.
On 20 documents in the test kit
See how the test scored on the accuracy case study, and every document it got wrong on every miss and why.