A production-ready Retrieval-Augmented Generation (RAG) system for intelligent question-answering over multiple PDF documents. This repository contains a notebook, helper modules, and documentation to run a hybrid retrieval pipeline that combines vector and keyword search, re-ranking, and a small web UI for interactive exploration.
This system allows users to:
- Ingest multiple PDF documents into a unified knowledge base
- Ask natural language questions and receive cited answers
- Compare information across documents
- Generate comprehensive document summaries
The architecture combines semantic (vector) search with keyword-based (BM25) retrieval, fused using Reciprocal Rank Fusion (RRF), and refined through cross-encoder re-ranking for optimal relevance.
User Query
│
├── Query Classification (factoid/summary/comparison/extraction/reasoning)
├── Multi-Query Expansion (3 alternative phrasings)
└── HyDE Generation (hypothetical answer document)
│
▼
┌──────────────────────────────────────┐
│ Hybrid Retrieval │
│ ┌─────────────┐ ┌─────────────┐ │
│ │ ChromaDB │ │ BM25 │ │
│ │ (Vector) │ │ (Keyword) │ │
│ └─────────────┘ └─────────────┘ │
│ │ │ │
│ └──────┬───────┘ │
│ ▼ │
│ RRF Fusion + Deduplication │
└──────────────────────────────────────┘
│
▼
Cross-Encoder Re-ranking
(BAAI/bge-reranker-v2-m3)
│
▼
LLM Generation (Llama 3.3 70B)
with inline source citations
│
▼
Answer Verification (for complex queries)
| Component | Technology |
|---|---|
| LLM | Llama 3.3 70B (via Groq API) |
| Embeddings | BAAI/bge-large-en-v1.5 |
| Re-ranker | BAAI/bge-reranker-v2-m3 |
| Vector Database | ChromaDB |
| Keyword Search | BM25 (rank-bm25) |
| PDF Processing | PyPDF |
| Web Interface | Gradio |
| Framework | LangChain 0.2.x |
- Python 3.10 or higher
- Groq API key (free tier available at console.groq.com)
- GPU recommended but not required (models will run on CPU if unavailable)
Developed and tested on Google Colab. Dependency versions are pinned to ensure compatibility at the time of development.
- Clone the repository:
git clone <repository-url>
cd rag-project- Install dependencies:
pip install numpy==1.26.4
pip install pandas==2.2.2
pip install scipy==1.13.1
pip install langchain-core==0.2.40
pip install langchain-community==0.2.16
pip install langchain==0.2.16
pip install langchain-groq==0.1.9
pip install langchain-text-splitters==0.2.4
pip install chromadb==0.5.5
pip install sentence-transformers==3.0.1
pip install pypdf==4.3.1
pip install rank-bm25==0.2.2
pip install gradio
pip install torch- Restart your Python environment after installation.
Open rag.ipynb in Jupyter Notebook or VS Code and run the cells in order to initialize the environment, ingest documents, and start the application. The notebook launches a Gradio web interface locally (typically at http://localhost:7860) and can optionally create a public shareable link via gradio.live for quick demos.
- Initialize: Enter your Groq API key in the Setup tab and click "Initialize"
- Upload Documents: Add one or more PDF files to build the knowledge base
- Ask Questions: Use the Chat tab to query your documents
- Summarize: Generate a comprehensive summary of all loaded documents
- HyDE (Hypothetical Document Embeddings): Generates a hypothetical answer to improve retrieval quality. Enabled by default.
- Multi-Query: Expands the original query into multiple phrasings for broader coverage. Enabled by default.
The system automatically classifies queries into five types and adjusts retrieval strategy accordingly:
| Query Type | Retrieval Depth (k) | Answer Style |
|---|---|---|
| Factoid | 6 | Direct |
| Summary | 10 | Bullets |
| Comparison | 12 | Bullets |
| Extraction | 8 | Direct |
| Reasoning | 10 | Steps |
Documents are split based on semantic similarity between sentences rather than fixed character counts, preserving coherent ideas within each chunk.
- Upload multiple PDFs to build a combined knowledge base
- Automatic PDF diversity enforcement for cross-document queries
- Clear source attribution with document name and page number
For complex queries (comparisons, summaries, reasoning), the system performs a self-verification step to ensure answers are direct, structured, and grounded in sources.
Repeated queries return cached responses for faster performance.
Single Document Analysis:
- "What is the main contribution of this paper?"
- "Explain the methodology in detail"
- "What are the limitations mentioned by the authors?"
Multi-Document Comparison (with two or more PDFs loaded):
- "Compare the approaches discussed in these papers"
- "What are the key differences between the methodologies?"
Summarization:
- Use the Summarize tab to generate a map-reduce summary of all documents
rag project/
└── rag.ipynb # Main notebook containing all code
├── Dependencies # Installation cell
├── Imports # Library imports and device setup
├── Data Classes # QueryProfile, QueryCache, SemanticChunker, RRF
├── EnhancedRAGv3 # Core RAG engine class
├── Gradio UI # Web interface definition
└── Launch # Application startup
| Operation | Typical Duration |
|---|---|
| Model initialization | 30-60 seconds |
| PDF ingestion (per doc) | 10-30 seconds |
| Simple queries | 5-8 seconds |
| Complex queries | 10-15 seconds |
| Full document summary | 30-90 seconds |
Key parameters that can be adjusted in the code:
| Parameter | Default | Description |
|---|---|---|
max_chunk_size |
1000 | Maximum characters per semantic chunk |
similarity_threshold |
0.5 | Cosine similarity threshold for chunk grouping |
chunk_size |
800 | Fallback text splitter chunk size |
chunk_overlap |
150 | Character overlap between chunks |
fetch_factor |
2 | Multiplier for initial retrieval pool |
lambda_mult |
0.6 | MMR diversity parameter (0=diverse, 1=relevant) |
cache_max_size |
100 | Maximum number of cached query responses |
- Requires active internet connection for Groq API calls
- PDF quality affects text extraction accuracy
- Large documents may take longer to process
- Query cache does not persist between sessions
- Package versions are pinned to those available during development; future updates to underlying libraries may require adjustments
This project is provided for educational and research purposes.