PDF (Portable Document Format)

PDF describes pages with a fixed layout: every glyph, line and image has an exact position, so a file looks the same on every screen and printer. A PDF file is a set of objects (dictionaries, arrays, streams) linked by references; pages draw text with embedded fonts, vector paths and images. On top of that PDF supports links, bookmarks, interactive forms, annotations, digital signatures, encryption, attachments, layers and a logical structure tree for accessibility.

PDF is the standard format for documents meant to be read, printed, signed or archived: invoices, contracts, papers, manuals, books, forms and print-ready artwork. It isn’t meant for editing: text is stored as positioned glyphs, not paragraphs, so reflowing or editing a PDF is hard and extracting text in reading order needs heuristics.

When to use PDF#

• Use to share a document that must look the same everywhere: reports, invoices, contracts, papers, manuals.
• Use for print: PDF/X is the standard exchange format for printers.
• Use for long-term archiving with PDF/A (fonts embedded, no external dependencies, no encryption).
• Use for fillable forms and digitally signed documents.
• Avoid when the recipient needs to edit the content: send DOCX, ODT or Markdown instead, or both.
• Avoid for content read mostly on phones: fixed pages don’t reflow. EPUB or HTML reflow to the screen width.
• Avoid as a data exchange format: tables in PDF are positioned text, and extracting them reliably is hard. Send CSV, XLSX or JSON alongside.

File structure#

A PDF file has 4 parts: a header, a body of numbered objects, a cross-reference table (xref) with the byte offset of every object, and a trailer that points to the root object and to the xref. A reader starts at the end of the file: it finds startxref, reads the xref, then loads objects on demand.

This is a complete, valid 1-page PDF (the %âãÏÓ comment marks the file as binary for transfer programs):

%PDF-1.7
%âãÏÓ
1 0 obj
<< /Type /Catalog /Pages 2 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 612 792]
   /Resources << /Font << /F1 4 0 R >> >> /Contents 5 0 R >>
endobj
4 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
endobj
5 0 obj
<< /Length 41 >>
stream
BT /F1 24 Tf 72 720 Td (Hello, PDF) Tj ET
endstream
endobj
xref
0 6
0000000000 65535 f 
0000000015 00000 n 
0000000064 00000 n 
0000000121 00000 n 
0000000250 00000 n 
0000000320 00000 n 
trailer
<< /Size 6 /Root 1 0 R >>
startxref
411
%%EOF

Header#

The first line is %PDF- followed by the version: 1.0 to 1.7 or 2.0. Most readers accept up to 1024 bytes of junk before it. Since PDF 1.4 the catalog can override the header version with a /Version entry, which incremental updates use to upgrade a file without rewriting it. The second line is a comment with 4 bytes above 127, so FTP clients and mail gateways treat the file as binary.

Objects#

The body is a sequence of indirect objects, each N G obj ... endobj where N is the object number and G the generation (almost always 0). Other objects refer to it with N G R. Basic object types:

Stream data is usually compressed. Common filters: FlateDecode (zlib/DEFLATE, the default for content and fonts), DCTDecode (a JPEG file as is), JPXDecode (JPEG 2000), JBIG2Decode and CCITTFaxDecode (bilevel scans), LZWDecode, RunLengthDecode, ASCIIHexDecode, ASCII85Decode.

Cross-reference table and trailer#

The classic xref table has one 20-byte line per object: a 10-digit byte offset, a 5-digit generation and n (in use) or f (free). Object 0 is always the head of the free list. The trailer dictionary follows: /Size (number of entries), /Root (the catalog), and optionally /Info (document information dictionary), /ID (two file identifiers), /Encrypt and /Prev (offset of the previous xref section). startxref gives the offset of the xref, and %%EOF ends the file.

Cross-reference streams and object streams#

Since PDF 1.5 the xref can be a binary, compressed cross-reference stream (a stream object with /Type /XRef) and its dictionary replaces the trailer. Small objects can be packed into object streams (/Type /ObjStm) and compressed together, which makes files 10-40% smaller. Most modern writers use both. “Hybrid” files have a classic table plus an /XRefStm entry so that PDF 1.4 readers still work.

Incremental updates#

A PDF can be changed by appending new and changed objects, a new xref section and a new trailer whose /Prev points to the old xref. The original bytes stay untouched. This is how annotations and form fill-ins are saved quickly, and it’s required for digital signatures: a signature covers a byte range, and later signatures or changes are appended after it. A file can contain several %%EOF markers; everything after the first one is the history of edits. Rewriting the whole file (a “full save”) drops the history and unused objects.

Linearization#

A linearized (“fast web view”) file starts with a linearization dictionary and stores the first page’s objects first, plus hint tables, so a viewer can show page 1 before the whole file is downloaded. qpdf --linearize in.pdf out.pdf creates one.

Document structure#

Content streams#

A page’s content stream is a list of operators in postfix notation: operands first, then the operator. The coordinate origin is the bottom-left corner of the page, y grows upward.

Text operators show glyph codes in the font’s encoding, not Unicode. Text extraction maps codes back to Unicode through the font’s /ToUnicode CMap or its encoding; when a PDF lacks both, extracted text is garbage even though it renders fine. Words and lines aren’t stored either: extractors rebuild them from glyph positions.

Fonts#

Writers usually embed a subset of the font (only the glyphs used); subset font names start with 6 capital letters and +, e.g. ABCDEF+Arial.

Encryption#

The standard security handler encrypts strings and streams (not the structure) with a key derived from passwords:

• the user password is needed to open the file; if it’s empty, anyone can open it.
• the owner password unlocks the permissions (print, copy, modify, fill forms, assemble). Permission flags are only honored by cooperating software: a file with an empty user password can be decrypted by any library.

Public-key (certificate) encryption also exists. PDF/A forbids encryption.

PDF/A, PDF/UA and other subsets#

ISO publishes profiles that restrict PDF for a purpose. Validate them with veraPDF, the open-source reference validator.

Tagged PDF adds a structure tree (/StructTreeRoot) of elements like H1, P, Table, Figure, linked to content with marked-content IDs. Screen readers, reflow on phones and good text extraction depend on it.

Debugging a PDF#

• qpdf --check file.pdf validates the structure and xref; qpdf --qdf --object-streams=disable in.pdf out.pdf writes an uncompressed, human-readable copy you can open in a text editor.
• qpdf --show-encryption file.pdf shows the encryption revision and permissions.
• mutool show file.pdf trailer or mutool show file.pdf 5 prints objects; mutool clean -d decompresses streams.
• pdfinfo file.pdf (Poppler) prints version, page count, page size, metadata, encryption and whether the file is tagged and linearized.
• pdffonts file.pdf lists fonts, whether they’re embedded and whether they have a ToUnicode map: the first thing to check when text extraction fails.
• A broken xref (wrong offsets after a text editor touched the file) is common; most readers rebuild it by scanning for obj keywords. qpdf in.pdf out.pdf writes a repaired file.

Specifications and documentation#

• ISO 32000-2:2020 (PDF 2.0): the current specification, free (after registration) from the PDF Association, sponsored by Adobe, Apryse and Foxit. The same page offers free copies of related standards, including PDF/UA
• Announcement of no-cost access to ISO 32000-2 (PDF Association)
• ISO 32000-1:2008 (PDF 1.7): Adobe’s free copy of the previous edition, still a good reference since most files are PDF 1.4-1.7
• Arlington PDF Model: machine-readable definition of every PDF object, key and type in PDF 2.0
• PDF 2.0 issues and errata: clarifications to the spec from the PDF Association
• veraPDF documentation: PDF/A and PDF/UA validation rules
• qpdf manual: useful explanations of xref streams, object streams, linearization and encryption
• PDF on Wikipedia, PDF/A, PDF/UA, PDF/X

Software that opens PDF files#

Libraries#

Open-source libraries that read, render, create and modify PDF files. See all PDF libraries for a comparison.

Related formats#

• PostScript and EPS: the page description language PDF’s graphics model comes from; Ghostscript converts them to PDF
• XPS: Microsoft’s XML-based fixed-layout alternative, rarely used
• DjVu: compact format for scanned documents
• EPUB: reflowable e-books, better than PDF on small screens
• OFD: the Chinese national fixed-layout document format, similar in purpose to PDF
• RTF and Markdown: editable text formats that are often converted to PDF
• Adobe Illustrator .ai files saved with PDF compatibility are PDF files with extra private data, so PDF viewers open them