RAG Workshop Guide

What is RAG?

Retrieval-Augmented Generation combines a language model with a searchable knowledge base. Instead of relying on the LLM's training knowledge, matching passages from your documents are retrieved for each question and passed to the model as context.

For patent practice that means: file-based answers with citations, without blowing up the context window, and without re-training the model.

When do you need RAG?

• You have a large document collection (case files, EPO office actions, literature, internal opinions).
• You want answers with source references ("quote + citation").
• The full documentation does not fit into even the largest models' context window.
• The data is confidential and must not leave the house.
• You need reproducible results that can be checked against verifiable sources.

You don't need RAG when the document already fits into the context window (e.g. a single patent specification). Just pass it in full.

Components of a RAG pipeline

1. Chunking: documents are split into sections of typically 500-1500 tokens. Too small = missing context, too large = blurry hits. Overlap (100-200 tokens) ensures features at boundaries are not lost.
2. Embedding: each chunk is converted by an embedding model into a vector (e.g. 1024 dimensions) that numerically encodes its meaning. Similar content ends up close together.
3. Vector Store: a database (ChromaDB, Qdrant, pgvector) stores the vectors together with metadata (filename, page, paragraph).
4. Retrieval: a question is embedded the same way, and the k most similar chunks are retrieved (typically k = 4-10).
5. Generation: the retrieved chunks are prepended to the LLM prompt as context. The LLM answers with reference to these passages.
6. Citation: metadata is passed along so the model can cite sources ("see paragraph 23 of patent X").

Local setup: ChromaDB + Ollama

Fully local stack: no data leaves the machine. Prerequisite: Python 3.10+, 16 GB RAM, GPU optional.

1. Install Ollama and pull models

# Windows/Mac: installer from ollama.com
# Linux:
curl -fsSL https://ollama.com/install.sh | sh

# Generator model (answers questions)
ollama pull llama3.3:70b-instruct
# or smaller:
ollama pull llama3.2:3b-instruct

# Embedding model (creates vectors)
ollama pull nomic-embed-text

2. Python environment

python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install chromadb ollama pypdf langchain-text-splitters

3. Index documents

import chromadb
import ollama
from pypdf import PdfReader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from pathlib import Path

client = chromadb.PersistentClient(path="./rag-db")
collection = client.get_or_create_collection("patents")

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1200, chunk_overlap=150
)

for pdf_path in Path("./files").glob("*.pdf"):
    text = "\n".join(p.extract_text() or "" for p in PdfReader(pdf_path).pages)
    chunks = splitter.split_text(text)
    for i, chunk in enumerate(chunks):
        emb = ollama.embeddings(model="nomic-embed-text", prompt=chunk)
        collection.add(
            documents=[chunk],
            embeddings=[emb["embedding"]],
            metadatas=[{"source": pdf_path.name, "chunk": i}],
            ids=[f"{pdf_path.stem}-{i}"],
        )
print("Indexing complete.")

4. Query with source references

def ask(question: str, k: int = 5) -> str:
    q_emb = ollama.embeddings(model="nomic-embed-text", prompt=question)
    hits = collection.query(query_embeddings=[q_emb["embedding"]], n_results=k)

    context = "\n\n".join(
        f"[Source: {m['source']} · chunk {m['chunk']}]\n{d}"
        for d, m in zip(hits["documents"][0], hits["metadatas"][0])
    )
    prompt = f"""Answer the question strictly based on the context.
Cite the source for every statement in the form [Source: ...].
If the context does not contain an answer, reply: "Not contained in the material."

# Context:
{context}

# Question:
{question}"""
    resp = ollama.chat(
        model="llama3.3:70b-instruct",
        messages=[{"role": "user", "content": prompt}],
    )
    return resp["message"]["content"]

print(ask("Which features does D1 disclose for feature 1.3?"))

Example use case: searching an opposition file

Scenario: you have 40 documents (EP specification, D1-D8, opposition brief, reply). Typical questions:

• "Where does the opponent assert anticipation of feature 1.4?"
• "Which passages in D3 discuss the objective problem of the patent?"
• "Did the patentee in the reply narrow the claim to a specific value range?"

Each answer includes its source (file + page) so manual verification is trivial.

Advanced techniques