How much of a PDF’s text is page furniture? 25 public documents

Page furniture is the text a PDF prints around the content of every page: running headers, footers and page numbers. We took 25 public PDFs from US federal agencies, 1,342 pages in all, and measured how much of their plain text is furniture. Then we ran them through Agnos, the document cleaner made by Gidion, and checked every line it removed and the repeated lines it kept. This page has the numbers, the method, what Agnos got wrong, and what you need to repeat it.

The short version

• Agnos removed 2,063 lines as page furniture from 24 of the 25 documents. That is 50,928 characters, 1.45% of the plain text. Per document, the median is 1.32%, and the range runs from 0% to 3.17%.
• It removed something from every text page of 3 documents, and from at least 9 in 10 text pages of 15.
• 200 of the removed lines, in 12 documents, look like content to us: table cells and column headings, phone numbers, table labels, requirement numbers. Review shows what was removed with the rule behind each piece, so you can find them.
• It kept about as much repeated text as it removed. The largest case is a NASA footer of four lines, still in the export on all 89 pages.

By volume, furniture is a small share of the text. What matters is where it sits. It comes at every page break, so it lands in the middle of any chunk that crosses one, and the same words turn up in chunk after chunk.

What we measured and why it matters for RAG

When you pull the text out of a PDF, each page’s header, footer and page number come out with the body, as ordinary lines. A RAG system cuts that text into chunks. The same footer lands in chunk after chunk, and a chunk that crosses a page break has a header in the middle of a sentence. Remove headers, footers and page numbers from PDF text before RAG explains the effect and shows three ways to take the lines out.

That guide uses a four-page sample. This study asks three questions of real documents:

• How much of a PDF’s plain text is furniture?
• How often is it on every page?
• What does Agnos remove, what does it keep, and where is it wrong?

How we did it

The documents

We wanted long, ordinary documents that anyone can download and check. So every one is a PDF from a US federal agency’s own website, and most works of the US federal government are in the public domain. We downloaded 55 candidates and chose 25 by hand. Each one opens, has a text layer, is not a scan, and has at least 10 pages. We chose:

• Group A, 21 documents, because they have running headers or footers on most pages.
• Group B, 4 documents (03, 06, 07 and 12), because they have little furniture. They show what Agnos does when there is not much to remove.

The eight kinds are handbooks and policy manuals (01 to 03), program guides (04 to 07), technical standards (08 to 10), regulatory guidance (11 to 14), annual reports (15 to 17), procurement (18 and 19), research reports (20 to 23) and form instructions (24 and 25). Sources lists each one with its link.

All 25 are text-based PDFs. Six of them have a few pages that hold only an image: 17 pages in all. Agnos treated them as image-only and named each one in its warnings, since it does not read text out of pictures. Four of them still carry a page number as text, and Agnos removed it: 04, pages 60, 100 and 118, and 12, page 44. Where this page counts text pages, it leaves all 17 out.

The plain-text baseline

For each page we took the text that a simple PDF loader passes on: pdfplumber 0.11.10, page.extract_text() with its default options. A line is a non-empty line of that text. Characters are the characters of those lines, not counting line breaks. Across the 25 documents that comes to 64,682 lines and 3,509,754 characters.

To count furniture without Agnos, we used a simple definition of a repeated line:

• Lines are compared after Unicode normalising (NFKC), lower case, single spaces, and every run of digits replaced by #. So “Page 3 of 12” and “Page 4 of 12” count as the same line. Lines with no letter or digit are left out.
• A line is repeated when it is on at least 30% of a document’s pages, and on at least 3. We also give a looser count: on 3 or more pages.
• A repeated line is at a page edge when it is one of the first 3 or last 3 lines of its page.

What counts as removed by Agnos

We used Agnos 2.3.2 with the RAG / Retrieval Corpus purpose and its default settings. Remove page furniture was on, looking at the Edges of each page, and PDF reading mode was Text only. We ran it through its command line, from Gidion’s own build of 2.3.2, with an empty library, and built an Everything export. The Agnos window does the same steps; see Reproduce it.

A line counts as removed when it has a row in the export’s rejected.jsonl. Each row holds the removed text, its reasons, the stage that removed it and its page. For every file, these counts match the ones Review shows. Review lists up to 500 removals per file, and no file reached it: the most was 379.

• Shares are the characters of the removed lines divided by the baseline characters. Agnos reads PDFs with a different library, so its lines differ a little from pdfplumber’s. Measured against Agnos’s own reading of the text, the pooled share is the same: 1.45%.
• Matching. To compare Agnos with the simple definition, a baseline line counts as removed when a removed line on the same page covers at least 80% of it. It counts as partly removed when less is covered. Each removed line is credited to one baseline line only.

Results

Totals

Per page, the median is 1.18 lines of furniture, from none in 07 to 2.94 in 10. Agnos removed only two things for other reasons: a passage in 03 that it took for a near duplicate, and a line of placeholder text in 20. Both are described under What to check after cleaning.

The 3+ page count is much larger than the 30% count mainly because of 20, a Census report full of tables. With digits masked, its table rows look alike: 10,417 of its 13,694 lines repeat on 3 or more pages.

Document by document

The most common furniture line is the one Agnos removed from the most pages, quoted from the first page it was on. Where it carries a page number, the number changes from page to page.

The numbers for every document, 46 columns of them, are in page-furniture-study.csv (7 KB). It has one row per document, and its first row names the columns. The CSV is under CC BY 4.0: you may reuse it if you credit Gidion.

By group

In group B, 07 lost nothing: no line in it repeats on 3 or more pages. 03 lost 33 page numbers and 6 labels from a sample form. 12 lost 44 page numbers, 3 appendix headings (“Appendix 1” to “Appendix 3”) and 13 phone numbers. 06 lost 5 revision tags.

Furniture on every page

• Agnos removed at least one line from every text page of 3 documents: 01 (60 of 60 pages), 05 (42 of 42) and 08 (89 of 89). It did so on at least 90% of text pages in 15 documents, and at least 75% in 21.
• The simple definition finds a repeated line at the edge of every text page in 4 documents: the same three and 25 (41 of 41). Agnos removed a line from 40 of those 41.
• Why not more. Covers and title pages rarely carry the running header: 10 has it on 127 of 129 pages. Some documents also change the header with each section. In 04, Medicare & You, no single line is on 30% of the pages, so the simple definition finds nothing at all. Agnos needs a line on only 3 pages, and removed a header or page number from 104 of its 124 text pages.
• The lowest. 07: no pages. 06: 5 of 56. 22: 9 of 37 text pages, because its long running header and some of its short ones are kept (see below). 20: 34 of 59. None on its 18 rotated table pages, which Agnos reads as one or two long lines each, or on its 6 front-matter pages and back cover.

The furniture we found

We labelled each removed line with simple text patterns, so one line can carry several labels. These are the 2,063 lines Agnos removed as page furniture:

Example lines

Quoted exactly from rejected.jsonl, with the document and page. Lines from 22 have two em spaces before the page number; we show them as one space.

• “Page X of Y”: “Page 1 of 89” (08, page 1). “NPR 7150.2D -- Chapter1 Page 8 of 89” (08, page 8). “PAGE 3 OF 34 1333ND19QNB680025” (18, page 3). “Form I-485 Instructions 09/18/26 Page 2 of 41” (25, page 2).
• Running title with a page number: “AmeriCorps NCCC Member Handbook 1” (01, page 1). “16 Section 1: How Medicare works” (04, page 16). “CRIMINAL VICTIMIZATION, 2023 | SEPTEMBER 2024 2” (21, page 2). “Concentrations of Iodine-129 in the Eastern Snake River Plain Aquifer 11” (22, page 21).
• Running title, no page number: “Guidance for Industry and FDA Staff General Principles of Software Validation” (11, page 3). “AN ACQUISITION GUIDE FOR EXECUTIVES” (19, page 3). “NIST AI 100-1 AI RMF 1.0” (13, page 5). “1/8/15 AC 120-92B” (09, page 3).
• Agency name and date: “US Department of Housing and Urban Development April 2025” (05, page 1). “Merit Systems Protection Board Annual Report for FY 2024 June 24, 2025 5” (15, page 6). “Department of Energy Office of Inspector General” (17, page 2).
• Page number on its own: “1” (03, page 4). “Page 1” (09, page 63). “2” (16, page 2).
• Proof and handling lines: “3420.14 5/15/2026 PROPERTY OF US GOVERNMENT 1” (02, page 1). “Page 2 of 59 Fileid: … ication-15/2026/a/xml/cycle09/source 12:33 - 15-Dec-2025” (24, page 2), a line from the printer’s proof.
• Table and figure furniture: “Continued on next page” (10, page 23; 13, page 27).

No confidentiality markings. These are public documents, and none carries a confidentiality line as furniture. “Confidential” appears only in body text. The nearest things are the Bureau of Prisons footer “PROPERTY OF US GOVERNMENT”, removed on 26 pages, the IRS printer’s proof lines, and NASA’s “uncontrolled when printed” footer, which was kept.

What to check after cleaning

It helps to know how the rule works. With the default settings, Agnos looks at the first 2 and last 2 lines of each PDF page. There it removes page markers such as “Page 3 of 12”, bare page numbers that count up with the pages, and any short line that repeats at the edge of 3 or more pages. A short line is 80 characters or fewer and does not read as a full sentence. Lines are compared with their digits masked, so “Handbook 1” and “Handbook 2” count as the same line. Each part of that rule shows up below.

Lines Agnos removed that look like content

We read every removed line, and looked at the page for any that did not read as furniture. By that judgement, 200 of the 2,063 lines are content rather than furniture: 3,811 characters in 12 documents. Most come from the digit masking: different text with the same shape, at the edges of 3 or more pages, is treated as one running line.

Lone numbers. Agnos keeps a lone number at a page edge unless the numbers there count up with the pages. But a lone number with its digits masked is just #, so once lone numbers sit at the edges of 3 or more pages, the repeated-line part of the rule removes them all. Of 400 lone numbers removed, 2 are not page numbers: the “2026” heading in 04 and a footnote marker, “1”, in 23 (page 8).

Two removals for other reasons.

• A whole passage, as a near duplicate. In 03, Appendix 2, “Sample Routine Telework Agreement” (page 31, 1,974 characters), was removed as a near duplicate of Appendix 1, “Sample Situational Telework Agreement”. They are different forms that share much of their wording. A reader would likely want both. In Review, Keep this anyway puts it back.
• “Lorem ipsum”, in 20. The published Census PDF has white placeholder text beside the Figure 1 title on page 8 (printed page 2): “Lorem ipsum” and, on the line under it, “dolor sit”. You can’t see it on the page, but text extraction reads it. Agnos removed the “Lorem ipsum” line as placeholder text and kept “dolor sit”. That removal is fair.

Repeated text Agnos kept

This part leaves out 20, whose plain text cannot be measured fairly (see Limits). By the simple 30% definition, the other 24 documents hold 2,129 repeated lines. Agnos removed 1,262 of them, partly removed 318 and kept 549. The repeated characters it kept come to 52,057 by this measure, a little more than the 48,957 characters of furniture it removed from the same 24 documents. The measure undercounts where pdfplumber merges lines: it gives 23,512 for 08, while the NASA footer counted in the export comes to 26,433. Not every kept line is furniture: some are table labels, notes and headings. The large cases are furniture, though:

• 08, NASA: a footer of 4 lines on all 89 pages. Agnos looks at 2 lines at each edge. In its reading of the page, the page number and the header line come after this footer, and those were removed, so the footer itself sits just outside the edge. All four lines are still in the export on all 89 pages: 297 characters a page, 26,433 in all, which is 12.8% of the document’s plain text. The lines are “This document does not bind the public, except as authorized by law or as / incorporated into a contract. This document is uncontrolled when printed. Check / the NASA Online Directives Information System (NODIS) Library to verify that / this is the correct version before use: https://nodis3.gsfc.nasa.gov.” The footer’s left-hand label, such as “NPR 7150.2D -- Chapter1”, is also kept on all 89 pages.
• 10, NIST: half of a two-line header. Agnos reads the header as 4 lines. It removed “NIST SP 800-63B-4” and “July 2025” from 125 pages or more, and kept “Digital Identity Guidelines” and “Authentication and Authenticator Management”.
• 24, IRS: the proof notice on all 59 pages. “The type and rule above prints on all proofs including departmental reproduction proofs. MUST be removed before printing.” is 121 characters, over the 80-character limit, so it stays: 7,139 characters in all. The proof’s “Page N of 59 Fileid: …” line was removed.
• 23, Copyright Office: a long header on 44 pages. “U.S. Copyright Office Copyright and Artificial Intelligence, Part 2: Copyrightability” is 85 characters, so it stays: 3,740 characters. Its page numbers were removed.
• 22, USGS: a long header on alternate pages. The header on 13 pages is over 80 characters and stays. The shorter header on odd pages was removed on 9 pages. On 4 more it names a section that runs for fewer than 3 such pages, such as “Summary” or “References Cited”, so it stays.
• 17, DOE Inspector General: headers inside the page. “Semiannual Report to Congress - April 2025 to September 2025” (46 pages) and “Summary of OIG” (15 pages) stay, because Agnos reads them in the body of the page rather than at its edge. In Agnos’s reading the first comes third on most pages, after the page number and the agency name.
• 20, Census: rotated table pages. On 18 rotated pages (22 to 36 and 42 to 44), Agnos’s reader returns the whole page as one line (two on pages 24 and 28). The running header stays at the start of that line, and no furniture can be found on those pages. We did not try the slower Layout reading mode on them.
• Uneven treatment. The same repeated text can be removed at a page edge and kept in the body. Revision tags of one form in 06: 5 removed, 27 kept. “Source:” notes in 21: 6 removed, 33 kept. “Additional Evidence Requirements” in 25: 3 removed, 12 kept.
• Roman numerals. Front-matter page numbers such as “iii” are not page furniture to Agnos, and stay.

What you can do

• Open Review, choose a file and press Removed. It shows what was removed with the rule that removed it (up to 500 per file are listed), so a phone number or a table label is easy to spot.
• Lines removed while cleaning cannot be put back one at a time. If a document needs them, switch off Remove page furniture and clean the file again; see Change the rules. A passage removed as a near duplicate can be put back with Keep this anyway.
• If your documents have footers of more than two lines or headers longer than 80 characters, open a few pages of the export and look. Where to look for page furniture also has an All setting, which looks anywhere on a page. We did not test it in this study.

Limits of this study

• 25 documents, chosen by us. They are not a random sample, and 21 were picked because they have furniture. The shares describe these documents, not PDFs in general.
• US government PDFs only. All are in English and published by federal agencies. Company reports, papers, slide decks and PDFs from other countries may look quite different.
• Text-based PDFs only. Agnos does not read text out of images. The 17 image-only pages hold at most a page number and a word or two of text, and a scanned document would give nothing to measure.
• One version and one set of settings. Agnos 2.3.2 with the RAG / Retrieval Corpus purpose and its defaults. Other purposes, settings or versions may give other results.
• One baseline library. pdfplumber with default options. Other loaders break lines differently, and so does Agnos’s own reader. For that reason, 318 baseline lines, 250 of them in 10, match an Agnos removal only in part (in 02, 08 and 10).
• Masked digits make table rows look alike. That is why 20 has so many lines on 3 or more pages. pdfplumber also reads its rotated tables backwards: “emocni” for “income”. So we leave 20 out of the kept-text figures. The medians and the 30% count are much less affected.
• Labels and judgements. The kinds of furniture are text-pattern labels. The list of removed lines that look like content is a judgement, not a rule: we read each line, and looked at its page where the line alone was not clear. None of the counts on this page is a score for Agnos.
• We make Agnos. Gidion ran this study on its own product. That is why every number is in the CSV and every step is below: you can check it.

Reproduce it

1. Download the PDFs. Use the links in Sources. Some agencies replace files at the same address, such as Medicare & You and IRS Publication 15. Compare each file’s size in bytes with ours: a different file gives different numbers.

2. Clean them in Agnos. Install Agnos from the Microsoft Store; the 14-day trial includes exports. In Projects, press New project, check that the purpose is RAG / Retrieval Corpus, and press Create project. Drop the folder of PDFs on the Import step. On Clean, wait until every file says Ready.

3. Export everything. Open Export, choose Everything and press Build and download. Unzip it. rejected.jsonl has one row for each removed line; see Export formats. Count its rows, not the per-file counts in manifest.json, which count removed pieces rather than lines.

4. Measure. Run the two scripts below. They need Python 3 and pdfplumber (pip install pdfplumber==0.11.10).

This script counts the plain text of one PDF and its repeated lines, by the simple definition above:

import re
import sys
import unicodedata
from collections import defaultdict

import pdfplumber  # pip install pdfplumber==0.11.10


def key(line):
    # NFKC, lower case, one space between words, every run of digits as "#"
    s = " ".join(unicodedata.normalize("NFKC", line).lower().split())
    return re.sub(r"\d+", "#", s)


with pdfplumber.open(sys.argv[1]) as pdf:
    pages = [[ln.strip() for ln in (page.extract_text() or "").splitlines() if ln.strip()]
             for page in pdf.pages]

on_pages = defaultdict(set)
for n, lines in enumerate(pages):
    for line in lines:
        if any(ch.isalnum() for ch in line):
            on_pages[key(line)].add(n)
need = max(3, 0.3 * len(pages))
repeated = {k for k, seen in on_pages.items() if len(seen) >= need}

hits = [(n, i, line) for n, lines in enumerate(pages) for i, line in enumerate(lines)
        if key(line) in repeated]
at_edge = [(n, i, line) for n, i, line in hits if i < 3 or i >= len(pages[n]) - 3]

print(f"{len(pages)} pages, {sum(map(len, pages)):,} lines, "
      f"{sum(len(ln) for lines in pages for ln in lines):,} characters")
print(f"{len(hits)} repeated lines, {len(at_edge)} of them at a page edge, "
      f"on {len({n for n, _, _ in at_edge})} pages")

On two of the documents it prints the same numbers as the CSV:

> python repeated_lines.py 05-hud-hcv-guidebook-fair-housing.pdf
42 pages, 1,767 lines, 153,041 characters
84 repeated lines, 84 of them at a page edge, on 42 pages

> python repeated_lines.py 08-nasa-npr-7150-2d-software-engineering.pdf
89 pages, 3,437 lines, 207,299 characters
343 repeated lines, 254 of them at a page edge, on 89 pages

This one adds up the lines and characters Agnos removed as page furniture, from rejected.jsonl:

import json
import sys
from collections import Counter

lines, chars = Counter(), Counter()
with open(sys.argv[1], encoding="utf-8") as fh:
    for row in map(json.loads, fh):
        if row["stage"] == "cleaning" and "page_furniture" in row["reasons"]:
            lines[row["source"]] += 1
            chars[row["source"]] += len(row["text"].strip())

for source in sorted(lines):
    print(f"{source}: {lines[source]} lines, {chars[source]:,} characters")

The first lines it prints for our export:

> python agnos_removed.py rejected.jsonl
01-americorps-nccc-member-handbook.pdf: 60 lines, 2,031 characters
02-bop-standards-of-employee-conduct.pdf: 84 lines, 1,819 characters
03-opm-telework-guide.pdf: 39 lines, 291 characters
04-medicare-and-you-2027.pdf: 128 lines, 3,837 characters
05-hud-hcv-guidebook-fair-housing.pdf: 84 lines, 4,485 characters
...

Divide one by the other for the share: 4,485 of 153,041 characters is 2.93% for 05. We renamed the files with their number first, such as 05-hud-hcv-guidebook-fair-housing.pdf; any names work.

Sources

Every document was downloaded from the agency’s site on 17 October 2026. The size is the file we used, in bytes.

Every Agnos result on this page comes from Agnos 2.3.2. Agnos runs on your computer, and nothing is uploaded.

Next: see how the removed lines are named and listed, in Remove headers, footers and page numbers from PDF text before RAG.