Stop Wasting PDFs — Build a RAG That Actually Understands Them

The rise of digital documentation has led to an overwhelming amount of PDF files being shared and stored. However, extracting valuable information from these files can be a daunting task, especially when dealing with messy scans, tables, and long paragraphs. The traditional approach of using simple retrieval models often results in inaccurate or incomplete information, leading to wasted time and resources. In this article, we will explore a production-ready RAG (Retrieval-Augmented Generator) pipeline that leverages OCR, heading-aware chunking, FAISS, cross-encoder reranking, and strict LLM prompts to turn messy PDFs into reliable, auditable answers.

Understanding the Problem

PDFs are a common file format used for sharing and storing digital documents. However, they can be notoriously difficult to work with, especially when it comes to extracting information. The main challenges with PDFs are:

• Scans and images: PDFs often contain scanned or image-based content, which can be difficult to read or extract text from.
• Tables and layouts: PDFs can have complex layouts and tables, making it hard to identify and extract relevant information.
• Long paragraphs: PDFs often contain long, dense paragraphs of text, which can be time-consuming to read and understand.

To overcome these challenges, we need a more sophisticated approach to extracting information from PDFs. This is where RAG comes in – a pipeline that combines retrieval and generation models to provide accurate and reliable answers.

Building a RAG Pipeline

A RAG pipeline consists of several components, each designed to address a specific challenge in extracting information from PDFs. The components are:

• Ingest: This step involves ingesting the PDF file and extracting the text content using OCR (Optical Character Recognition) techniques.
• Smart chunking: This step involves breaking down the extracted text into smaller, more manageable chunks, using heading-aware chunking to identify relevant sections and subsections.
• Bi-encoder shortlisting: This step involves using a bi-encoder model to shortlist the most relevant chunks of text based on the input query.
• Cross-encoder reranking: This step involves using a cross-encoder model to rerank the shortlisted chunks and provide a more accurate ranking of the most relevant text.
• Grounded LLM prompts: This step involves using strict LLM prompts to generate a final answer based on the most relevant text.

import PyPDF2
import pytesseract
from PIL import Image
from transformers import AutoModel, AutoTokenizer

# Ingest PDF file
def ingest_pdf(file_path):
pdf_file = open(file_path, 'rb')
pdf_reader = PyPDF2.PdfFileReader(pdf_file)
text = ''
for page in range(pdf_reader.numPages):
page_obj = pdf_reader.getPage(page)
text += page_obj.extractText()
pdf_file.close()
return text

# Smart chunking
def smart_chunking(text):
chunks = []
headings = []
for line in text.split('\n'):
if line.strip() and line.strip() not in headings:
headings.append(line.strip())
chunks.append(line.strip())
else:
chunks[-1] += ' ' + line.strip()
return chunks

# Bi-encoder shortlisting
def bi_encoder_shortlisting(chunks, query):
model = AutoModel.from_pretrained('sentence-transformers/all-MiniLM-L6-v2')
tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/all-MiniLM-L6-v2')
inputs = tokenizer(chunks, return_tensors='pt', padding=True, truncation=True)
outputs = model(**inputs)
embeddings = outputs.last_hidden_state[:, 0, :]
query_embedding = model(tokenizer(query, return_tensors='pt'))[0]
similarities = torch.nn.functional.cosine_similarity(embeddings, query_embedding)
shortlisted_chunks = [chunk for _, chunk in sorted(zip(similarities, chunks), reverse=True)]
return shortlisted_chunks

# Cross-encoder reranking
def cross_encoder_reranking(shortlisted_chunks, query):
model = AutoModel.from_pretrained('sentence-transformers/all-MiniLM-L6-v2')
tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/all-MiniLM-L6-v2')
inputs = tokenizer(shortlisted_chunks, return_tensors='pt', padding=True, truncation=True)
outputs = model(**inputs)
embeddings = outputs.last_hidden_state[:, 0, :]
query_embedding = model(tokenizer(query, return_tensors='pt'))[0]
similarities = torch.nn.functional.cosine_similarity(embeddings, query_embedding)
reranked_chunks = [chunk for _, chunk in sorted(zip(similarities, shortlisted_chunks), reverse=True)]
return reranked_chunks

# Grounded LLM prompts
def grounded_llm_prompts(reranked_chunks):
model = AutoModel.from_pretrained('t5-base')
tokenizer = AutoTokenizer.from_pretrained('t5-base')
prompt = 'Answer the question based on the following text: ' + ' '.join(reranked_chunks)
inputs = tokenizer(prompt, return_tensors='pt')
outputs = model.generate(**inputs)
answer = tokenizer.decode(outputs[0], skip_special_tokens=True)
return answer

Practical Tips and Best Practices

When building a RAG pipeline, there are several practical tips and best practices to keep in mind:

• Use high-quality OCR: The quality of the OCR model used can significantly impact the accuracy of the extracted text.
• Optimize chunking: The chunking step can be optimized by using heading-aware chunking and identifying relevant sections and subsections.
• Fine-tune bi-encoder and cross-encoder models: Fine-tuning the bi-encoder and cross-encoder models can improve the accuracy of the shortlisting and reranking steps.
• Use strict LLM prompts: Using strict LLM prompts can help generate more accurate and relevant answers.

Key Takeaways

• RAG pipelines can be used to extract accurate and reliable information from PDFs: By combining retrieval and generation models, RAG pipelines can provide more accurate and reliable answers than traditional retrieval models.
• High-quality OCR is essential: The quality of the OCR model used can significantly impact the accuracy of the extracted text.
• Optimizing chunking and fine-tuning models can improve accuracy: Optimizing the chunking step and fine-tuning the bi-encoder and cross-encoder models can improve the accuracy of the shortlisting and reranking steps.

Conclusion

In conclusion, building a RAG pipeline can be an effective way to extract accurate and reliable information from PDFs. By combining retrieval and generation models, RAG pipelines can provide more accurate and reliable answers than traditional retrieval models. By following the practical tips and best practices outlined in this article, developers can build a production-ready RAG pipeline that leverages OCR, heading-aware chunking, FAISS, cross-encoder reranking, and strict LLM prompts to turn messy PDFs into reliable, auditable answers. So why wait? Start building your RAG pipeline today and stop wasting PDFs!

---

🚀 Enjoyed this article?

If you found this helpful, here’s how you can support:

💙 Engage

• Like this post if it helped you
• Comment with your thoughts or questions
• Follow me for more tech content

📱 Stay Connected

• Telegram: Join our tech community for instant updates → t.me/RoboVAI
• More Articles: Check out my blog → robovai.blogspot.com

---

Thanks for reading! See you in the next one. ✌️