PDF Reality Drift

The Problem: Visual Fidelity vs. Semantic Consistency

PDF was designed to answer one question reliably: does this document look the same on any device? It answers that question exceptionally well. PostScript-derived positioning, embedded fonts, device-independent colour — the format is engineered for pixel-level rendering consistency across every viewer on every platform.

But “looks the same” is a rendering property, not a semantic one. The format makes no guarantee that the text a parser extracts is the text a human sees, that an OCR engine reads the same content as the embedded text layer, that an accessibility processor encounters the same meaning as the visual rendering, or that two different parsers extract the same thing from the same file.

For three decades this was a minor inconvenience. PDF was viewed by humans and occasionally machine-processed for simple extraction tasks where imprecision was acceptable. The rendering guarantee was what mattered, and it was delivered.

That assumption no longer holds. PDFs now flow directly into RAG knowledge bases, LLM training corpora, automated compliance workflows, financial ingestion systems, and agentic AI pipelines. In those contexts, the extraction layer — not the rendering layer — is the one that matters. And the extraction layer is where PDF has always been ambiguous.

Why This Matters Now

PDFs are no longer just viewed by humans. They are being used as:

• RAG knowledge sources — enterprise document corpora fed into retrieval-augmented generation systems
• LLM training material — Hugging Face’s FinePDFs dataset contains 475 million documents and 3 trillion tokens extracted from PDFs
• Compliance evidence — regulatory submissions, audit trails, and legal records processed by automated AI review systems
• Financial ingestion data — invoices, contracts, and financial instruments fed into automated payment and reconciliation workflows
• Automated workflow inputs — documents processed by agentic pipelines with no human reviewer in the loop
• Agentic AI context — LLM agents that read PDFs and take actions based on their extracted content

Traditional PDF security focused heavily on malware and active content — JavaScript shellcode, embedded executables, CVE-targeted byte sequences, behavioral sandbox execution. Those threats remain real and the 47-engine scanner addresses all of them.

But modern ingestion risk increasingly comes from semantic divergence between layers inside technically valid PDFs. A document with no malicious JavaScript and no CVE patterns can still deliver completely different content to an AI extraction pipeline than it shows to a human reviewer — and neither system has any signal that a discrepancy exists.

The attack surface is not the PDF viewer. It is the gap between what the viewer renders and what the extractor ingests.

Malicious vs. Semantically Unstable

The most important conceptual distinction in this work is one that traditional security framing actively obscures: most of the PDFs that exhibit reality drift are not malicious files.

They are not exploit kits. They are not malware. They were not authored by attackers. Many were produced by legitimate enterprise software — DocuSign, Adobe Sign, Kofax, Oracle forms, government PDF generators — using legal PDF features exactly as the specification intends.

They are simply semantically unstable across interpreters.

This distinction matters enormously for the contexts where this problem is most acute:

The security question for these documents is not “is this file hostile?” It is “is this file semantically deterministic?” — does it return the same meaning regardless of which layer, which parser, or which interpretation system reads it?

We call this property semantic nondeterminism. A semantically nondeterministic PDF is one where the answer to a factual question — what is the invoice total, what did the signatory agree to, what does this clause say — depends on which system asks. The format permits it. The specification does not prohibit it. Standard integrity checks do not detect it.

Corpus prevalence. We ran Engines 44, 45, and 46 against 181 documents across five categories: 102 adversarial proof-of-concept PDFs (corkami corpus), 29 academic papers (arXiv), 4 government publications, 44 IRS tax forms, and 2 government agency forms. Raw counts are shown below; percentages are rounded to the nearest integer. (The full corpus grew to 182 files during the parser agreement measurement phase; the additional corkami file scored 0 on all four vectors.)

Three findings stand out. First, reading order ambiguity (Engine 44) is ubiquitous in professionally produced documents: all 29 academic papers (29/29), all 4 government publications (4/4), and 42 of 44 IRS forms triggered it. These are not adversarial files. They are the PDFs that flow daily into RAG pipelines, document intelligence systems, and automated compliance workflows. Second, the adversarial PoC corpus — 102 files specifically crafted to stress PDF parsers — scored 0/102 on all four drift vectors. Semantic nondeterminism is a property of structurally legitimate documents, not of malformed ones; the two populations do not overlap. Third, accessibility structure (Engine 46) was present in 42 of 44 IRS forms. Any system that extracts field values from tax forms without reconciling the /StructTreeRoot tree against the visual layer is operating on unvalidated semantic assumptions for roughly 25% of all documents in this corpus.

The V/AP divergence engine (Engine 28) has a separate validation record: 8/8 true positives and 0/187 false positives on a 196-document form corpus.

Extraction divergence measurement. We extended the corpus measurement to quantify extractor-level disagreement directly: how often do five mainstream PDF text extractors — PyMuPDF, PDFium (pypdfium2), pdfplumber, Poppler/pdftotext, and MuPDF/mutool — produce materially different text from the same file? We ran all five against all 182 documents and computed pairwise Jaccard word-set similarity for every extractor pair on every file. Tesseract OCR was additionally run on a 14-file stratified sample across all non-adversarial categories for comparison against embedded text layers.

Divergence rate: proportion of files in each category where at least one extractor pair scored below Jaccard 0.70 — the threshold below which downstream NLP, embedding, and retrieval results are materially affected. Consensus failure rate: proportion where the mean across all 10 extractor pairs fell below 0.70 — a stricter metric requiring most pairs to disagree simultaneously:

Pairwise extractor agreement matrix (mean Jaccard, non-adversarial files). Mean word-set overlap between each extractor pair. Green (≥0.95) indicates near-identical output; amber (0.80–0.90) indicates systematic divergence affecting retrieval quality; red (<0.80) indicates material extraction differences that will produce different answers to the same query:

Four findings from the matrix. First, PyMuPDF and MuPDF agree at 0.982 — both are built on the same underlying libmupdf library, and the residual 1.8% divergence reflects only tokenisation and whitespace handling differences. Second, pdfplumber is the most divergent extractor: mean Jaccard 0.770 against PyMuPDF and 0.776 against Poppler means roughly one word in four differs on clean professionally produced documents. Third, PDFium disagrees with Poppler at 0.834 — two of the most widely deployed extraction libraries produce materially different text from the same file. Fourth and most significant: academic papers show higher any-pair divergence (62%) than the adversarial corpus (39%). Complex layouts, multi-column text, equations, and embedded figures expose structural extraction differences that purpose-built attack files do not trigger. The documents AI pipelines most commonly ingest for knowledge retrieval are the ones where extractors most often disagree.

Layer preference bias. Comparing average word counts across extractors reveals systematic layer preference differences — some extractors consistently reach more of the document than others on the same files:

The outlier rate counts documents where an extractor retrieved fewer than 50% of the words obtained by the median extractor on the same file. pdfplumber is an outlier on 50 of 131 text-bearing files (38.2%) — it systematically under-extracts complex layouts because it reconstructs text from character bounding boxes rather than raw content streams. On government agency forms, both PDFium and pdfplumber retrieve 684 words on average vs. 2,428 for PyMuPDF/MuPDF/Poppler: a 3.5× gap from the same files. These are not random errors — they reflect deterministic architectural differences in how each library traverses the PDF object graph and which layers it prioritises during extraction.

Tesseract OCR baseline. Run on page 1 of 14 stratified non-adversarial files, Tesseract scored mean Jaccard 0.67 against both PyMuPDF and Poppler embedded text layers — one word in three diverges between the OCR path and the embedded text path on the same page. This measurement calibrates Engine 45’s detection threshold: a real-world OCR/text-layer Jaccard below 0.50 is anomalous; below 0.30 is consistent with deliberate layer poisoning.

Semantic nondeterminism is not bigger than malware because it is more dangerous in the traditional sense. It is consequential because it is structurally invisible to standard tooling, deeply embedded in legitimate document workflows, and increasingly load-bearing as PDFs flow into automated systems that have no human reviewer in the loop.

A note on design intent: nothing in this analysis is a criticism of the PDF specification. PDF was designed to solve a real and hard problem — pixel-level rendering consistency across every device and platform — and it solved it. The multi-layer architecture that creates semantic nondeterminism (incremental updates, accessibility structures, OCR layers, optional content groups) reflects deliberate design decisions for legitimate purposes: archiving, accessibility, forms, and layers all serve real needs. The problem is not that the format is defective. The problem is that these features are now being used in a context — machine semantic extraction for AI — that postdates the format’s design goals by decades, and for which no single canonical interpretation was ever required or specified.

Three New Engines: Measuring Drift Directly

Each new engine was validated against a hand-crafted test PDF that triggers the specific drift vector the engine targets. All three produce confirmed live scanner output with no false positives on a clean-document corpus.

Full Coverage Map: 13 Vectors → 47 Engines

Every structural drift vector now has at least one dedicated detection engine. High-risk combinations feed into the Correlation Engine (Engine 47) for compound scoring.

Scan a PDF for Reality Drift

The PQ PDF Forensic Scanner runs all 47 engines in parallel — including OCR text layer integrity, accessibility tree forensics, reading order ambiguity detection, ToUnicode CMap analysis, cross-source metadata reconciliation, differential multi-parser comparison, and behavioral sandbox execution. No configuration required. File deleted immediately after analysis.

→ PDF Forensics Scanner — 47 Engines, Free

→ PDF Structural Problems in AI Ingestion Pipelines — V/AP Divergence & Parser Disagreement

→ Parser Disagreement Research — Six Parsers, Eleven Divergences

→ PDF Form Security — V/AP Divergence, DocMDP, FieldMDP

No account. No file retention. Differential parsing, OCR layer integrity, accessibility tree forensics, behavioral sandbox, YARA, ClamAV, offline threat intelligence (6.4M+ indicators), and AI synthesis — all running on the same file in parallel.