← Back

med-rag

med-rag answers questions about a folder of PDFs in plain language. It reads the documents once, keeps the text and the embeddings in a database on your machine, and then, for every question, retrieves the passages that bear on it and hands them to a language model. It works with a model running locally through Ollama, or with Claude through the API.

Indexing runs on the CPU, with an embedding model of about 50MB in ONNX and no GPU. With the Claude backend the machine does nothing heavier, so any ordinary laptop is enough. (Running the model locally through Ollama needs more memory, so use a model at least as capable as gemma3:12b).

It was written for an archive of medical reports large enough that finding a single value meant opening the files one after another until it turned up. It also handles simpler plain pages, where there are no tables to reconstruct.

I also used med-rag while writing my thesis and preparing it for publication, to retrieve the references across the dozens of PDFs I had accumulated. It was better than searching each PDF with Ctrl+F or using an LLM that, by reading every PDF entirely, would have cost a huge amount of tokens.

Source and instructions are available at
github.com/leonardocppn/med-rag.

112 reports, 264 pages ChromaDB 1,760 chunks embedded once, stored on this machine When was the appendix removed? ~500 tokens sent, not 100k language model Ollama on this machine, or Claude Appendicectomy, 3 June 2019, discharge letter [page 1], recovery uneventful [page 2]. indexed once per question
Figures are an example for an archive of that size.
How it works

Indexing runs once per document. The profiler measures the layout and derives the parsing parameters, the parser extracts every text element with its coordinates and classifies it as title, header, footer, table or body, and what survives is embedded by a small multilingual model and written to a local ChromaDB collection. A scanned book comes out of the profiler with an impossible column density, produced by the OCR noise, and gets a parameter set that keeps the footnotes and skips the column and title rules. Narrative documents can skip the layout work entirely with --plain.

Asking runs per question. The question is embedded the same way, the nearest chunks come back, and a cross-encoder can re-rank them when precision matters more than speed. The model receives those passages with the page they came from, and streams its answer to the terminal.

Each PDF gets its own collection, and several can be grouped into a named corpus that is then queried as one document.

indexing asking PDF a file, or a whole folder profiler + parser metrics, blocks, regions embeddings multilingual, ONNX, on CPU ChromaDB ./chroma_db question one PDF, or a corpus retrieval nearest chunks, re-ranked language model Ollama, or Claude answer with the pages it came from
The commands

Reading documents in:

index
Indexes one PDF, several of them, or every PDF in a directory, into a corpus.
sync
Indexes only what a directory has gained since last time, recognised by content hash.
delete
Drops the collection of one PDF.
delete-corpus
Drops a corpus, or a single document out of it.

Asking questions:

ask
One question against a PDF or a corpus, answered from the retrieved passages.
chat
The same, kept open across turns, with the conversation in context.
summary
A structured summary of a whole document. Long ones are summarised in batches.
search
BM25 keyword lookup, no model involved, results are instant.

Looking at the machinery:

profile
The layout metrics measured on a PDF and the parsing thresholds derived from them.
cluster
Groups the profiled PDFs by layout similarity, with DBSCAN.
inspect
The raw blocks the parser produced, for when a document comes apart badly.

Every command takes a PDF path or a corpus name in the same position, so the syntax is the same in both cases.

Setting it up

Python 3.10+ and the packages declared in pyproject.toml: pdfplumber for the PDFs, fastembed and sentence-transformers for the embeddings, chromadb to store them, rank-bm25 for the keyword search, scikit-learn for the clustering, and the two client libraries. An Anthropic API key is needed for the Claude backend. The local backend needs a running Ollama on the machine.

git clone https://github.com/leonardocppn/med-rag cd med-rag python3 -m venv venv && source venv/bin/activate python -m pip install --upgrade pip pip install -e . cp .env.example .env

Then index a folder and ask it something. The first run downloads the embedding model, about 50MB, from HuggingFace:

medrag index data/ --corpus reports medrag ask reports "When was the appendix removed?" --rerank

--corpus puts every PDF of the folder in one searchable collection, and a name is assigned automatically if you leave it out. --rerank passes the retrieved chunks through a cross-encoder before they reach the model. It costs about a second per question, and another 100MB the first time. --model claude sends the question and the retrieved passages to the API; without it the answer comes from the local model and nothing leaves the machine.

Source and instructions: github.com/leonardocppn/med-rag (MIT licence).

← Back