Retrieving pages as images

1.What a parser throws away

Parsing documents before you chunk them turns a page into text. For prose that is fine. For a chart the picture is the content: the height of a bar, the bend in a line, which colour is which series. A parser keeps whatever happened to be text, such as axis ticks and data labels, and drops the rest.

Page-image retrieval removes the step. Render each page to an image. Give it to a vision-language model that outputs one small vector for each patch of the picture. Store the whole set as one object per page. The query stays plain text, and the score is the MaxSim from Late interaction: one vector per token, with image patches where passage tokens were. There is no parser, no OCR and no chunker, and the page is the unit. The idea comes from the ColPali paper (June 2024, ICLR 2025), which also introduced the ViDoRe benchmark.

Here is a toy version. The deck is invented, for a fictional company, three pages long. Page 1 is a bar chart of automotive revenue over five quarters in $ millions: 312, 388, 451, 449, 520 (all invented). Page 2 is a prose slide about automotive strategy. Page 3 is a line chart of cloud revenue. The query is "how did automotive revenue change over time?" A text embedder reads what PyMuPDF extracts. A page-image model reads a picture of each page.

The text model was all-MiniLM-L6-v2, the small teaching model from What an embedding really is (as of October 2026). The page model was ModernVBERT/colmodernvbert, about 250 million parameters under an MIT licence (as of October 2026). The order flipped: the prose slide wins on words, the chart wins on pixels. Keep the claim small. This is three pages, one query, two small models and one run: a demonstration, not a benchmark. Both gaps are narrow, and the two score columns are on different scales.

A parser keeps the words on the page. A page image keeps the rest, and bills you for every patch of it.

2.What the evidence says, and what it costs

The best evidence so far is ViDoRe V3, an arXiv preprint from January 2026 (revised April 2026): 10 datasets, about 26,000 pages and 3,099 human-checked queries over public English and French PDFs. Page-image retrievers led its table. The best one scored 59.8 nDCG@10 against 51.0 for the best text embedder. That comparison is confounded, because the winner is a 3-billion-parameter late-interaction model and the text leader a larger single-vector one. The fairer test is a text pipeline with a cross-encoder from Reranking, which scored 63.6, above the best image retriever alone. Pages that mix text and images were the hardest content type. Combining the two routes gave the best end-to-end answers, by under a point overall, which is a hint and not a result. The corpora are clean public reports; scans and handwriting were not tested.

A vendor post from Weaviate (September 2026) points the same way. It tests three hand-picked queries over 92 slides from one company's decks, runs no text baseline and measures no recall, so read it as a how-to and not as evidence. This is a route to test against parsing, not a replacement for it.

What it costs

At index time you pay one vision-model pass per page instead of a parser run. The ColPali authors report their page indexing was faster than their parse-and-OCR pipeline, on their own setup. No independent per-page latency or cost figure turned up for this lesson, so time a hundred of your own pages.

Storage is the bigger surprise. The ColPali paper reports 257.5 KB per page: about 1,030 vectors of 128 float16 numbers, which is 1,030 x 128 x 2 bytes. At 100,000 pages that is about 26 GB, and a million pages is about 260 GB. Now take the same pages as four text chunks each (an invented count), one 1,024-number float32 vector per chunk. That is 16 KB a page, or 1.6 GB for 100,000 pages: about sixteen times smaller. In the toy run above, the model returned 1,149 vectors for a page rendered at 1,000 pixels and at 2,000 pixels, so extra pixels bought no extra vectors. The Weaviate post suggests rendering near 2,000 pixels, one author's default and not a measured sweep, so check what your model does first.

At query time, 25 query vectors against 1,030 page vectors is about 26,000 dot products a page. Qdrant's tutorial (undated, read in October 2026) shrinks that by averaging the patch grid by rows, so 1,024 patch vectors become 32. It searches those first and rescores the shortlist with the full vectors. It reports comparable quality and indexing an order of magnitude faster but gives no figures, and the full vectors still have to be stored, so this saves search time, not disk. The ColPali paper warns that text-dense pages may lose more under pooling. At answer time the generator must be able to read images, and every retrieved page reaches it as image tokens.

3.Building it without trusting it blindly

Try it on a hundred pages of your hardest PDF before you build anything. The code renders pages, encodes them and scores one query by brute force.

# pip install -U "sentence-transformers[image]" pymupdf peft
import os
import pymupdf
from PIL import Image
from sentence_transformers import MultiVectorEncoder

# Set PAGE_MODEL to a current page-image retriever. Read its model card first:
# some checkpoints need trust_remote_code=True or a pinned revision.
model = MultiVectorEncoder(os.environ["PAGE_MODEL"])

def render(page, long_edge=2000):
    zoom = long_edge / max(page.rect.width, page.rect.height)
    pix = page.get_pixmap(matrix=pymupdf.Matrix(zoom, zoom))
    return Image.frombytes("RGB", [pix.width, pix.height], pix.samples)

pages = [render(p) for p in pymupdf.open("deck.pdf")]
page_vecs = model.encode_document(pages, batch_size=2)  # a matrix per page

query = "how did automotive revenue change over time?"
scores = model.similarity(model.encode_query([query]), page_vecs)[0]

print("vectors per page:", [v.shape[0] for v in page_vecs])
for i in scores.argsort(descending=True).tolist():
    print(f"page {i + 1}  {scores[i].item():6.2f}")

# Output of one run with ModernVBERT/colmodernvbert (as of October 2026),
# Sentence Transformers 6.1, PyMuPDF 1.28, on the invented deck above.
# Loading it printed a warning about unexpected custom_text_proj.lora keys;
# shapes and scores looked sane, but check your own checkpoint.
# vectors per page: [1149, 1149, 1149]
# page 1   13.45
# page 2   12.89
# page 3   11.20

In production you route. Decide per page at ingestion, with cheap signals such as the share of the page covered by images, table or chart blocks your parser reports, and very few words per page (invented rules, so test them). Query both indexes and fuse the lists with the reciprocal rank fusion from Hybrid search. If routing is hard, index the chart-heavy pages both ways. Exact identifiers such as part numbers and error codes should stay on the text and keyword side. That is inference, since no source tested it, but a small serial number is the first thing to check.

The confident misread. Retrieval does its job and the generator reads a bar as 451 when it is 449, or answers from the wrong series. The Weaviate post never evaluates the answer step, and none of the sources read for this lesson isolates chart-reading errors. The second failure runs the other way: on a dense page of running text, the image route can rank below the same page parsed to text.

Evaluate with queries whose answer lives only in the picture, such as the trend of a line or one table cell by row and column. Score two things apart: did the right page come back, as in Measuring retrieval, and did the answer carry the right number, as in Measuring the whole answer. A good first score with a bad second points at the generator, not the retriever. Cite the page and show the image, as Citations and grounding describes. In ViDoRe V3 the best models' boxes around the evidence scored under 0.1 F1 where human annotators agreed with each other at 0.60, so do not claim a region. A page image also carries everything printed on it, so the permission metadata from Permissions that travel with the data travels with the page, and a model change re-embeds every page, as Changing the embedding model underneath a live index describes.

When not to use it. Skip it for text-heavy corpora such as contracts, transcripts and logs, where Weaviate's own authors say a text embedder is cheaper and about as accurate. Skip it for exact-string lookup, for a storage budget that cannot absorb sixteen times the vectors, and for pages that do not render the same way each time, because the vectors then describe your renderer. Parse and rerank is the baseline: adopt this only when your own eval set shows it finds pages the baseline misses. Even then it only finds the page; whether the number on it was read correctly is a separate question.