Unlocking Conversations: Engage with Your Documents Through Retrieval-Augmented Generation (RAG)

Picture having your own chatbot that can respond to inquiries drawn directly from your documents—whether they are PDFs, research articles, or books. With Retrieval-Augmented Generation (RAG), this becomes not only feasible but also easy to execute. In this guide, we’ll explore how to create a chatbot that engages with your documents, such as PDFs, utilizing Retrieval-Augmented Generation (RAG). We will employ Groq for language model inference, Chroma for the vector repository, and Gradio for the user interface.

By the conclusion, you will have a chatbot proficient in answering queries from your documents, maintaining the context of your dialogues, and delivering brief, precise responses.

What is Retrieval-Augmented Generation (RAG)?

Retrieval-Augmented Generation (RAG) represents an AI framework that amplifies the abilities of Large Language Models (LLMs) through the incorporation of an information retrieval mechanism. This mechanism retrieves pertinent data from outside sources, supplying the LLM with substantiated information to produce responses that are more accurate and contextually relevant. By merging the generative capabilities of LLMs with real-time data sourcing, RAG minimizes errors and guarantees that the information generated by AI is current.

Prerequisites

Python Installation: Confirm that Python 3.9+ is installed on your machine.

Groq API Key: Create a Groq account and obtain an API key:

Access Groq Console.

Head to API Keys and generate a new key.

Retain your API key for implementation in the project.

Dependencies: Install the necessary libraries:

pip install langchain langchain-community langchain-groq gradio sentence-transformers PyPDF2 chromadb

These libraries will assist in language processing, user interface development, model integration, PDF management, and vector database administration.

Downloading the PDF Resource

For this lesson, a publicly available PDF with details on diseases, their symptoms, and remedies will be utilized. Download the PDF and place it in your project directory (you may use any PDF you choose).

Step 1: Extracting Text from the PDF

We will utilize PyPDF2 to extract text from the PDF:

from PyPDF2 import PdfReader

def extract_text_from_pdf(pdf_path):
reader = PdfReader(pdf_path)
text = “”
for page in reader.pages:
text += page.extract_text()
return text

pdf_path=”diseases.pdf” # Replace with your PDF path
pdf_text = extract_text_from_pdf(pdf_path)

Step 2: Split the Text into Chunks

Lengthy documents are segment into smaller, manageable portions for processing.

from langchain.text_splitter import RecursiveCharacterTextSplitter

def split_text_into_chunks(text, chunk_size=2000, chunk_overlap=200):
text_splitter = RecursiveCharacterTextSplitter(
chunk_size=chunk_size,
chunk_overlap=chunk_overlap
)
return text_splitter.split_text(text)

text_chunks = split_text_into_chunks(pdf_text)

Step 3: Create a Vector Store with Chroma

We will embed the text chunks utilizing a pre-trained model and record them in a Chroma vector database.

from langchain.embeddings import SentenceTransformerEmbeddings
from langchain.vectorstores import Chroma

embedding_model = SentenceTransformerEmbeddings(model_name=”all-MiniLM-L6-v2″)

vector_store = Chroma(
collection_name=”disease_info”,
embedding_function=embedding_model,
persist_directory=”./chroma_db”
)

vector_store.add_texts(texts=text_chunks)

Step 4: Initialize the Groq Language Model

To leverage Groq’s language model, set your API key and create an instance of ChatGroq.

import os
from langchain_groq import ChatGroq

os.environ[“GROQ_API_KEY”] = ‘your_groq_api_key_here’ # Replace with your API key

llm = ChatGroq(model=”mixtral-8x7b-32768″, temperature=0.1)

Step 5: Create the Conversational Retrieval Chain

With LangChain’s ConversationalRetrievalChain, we can connect the language model and vector database.

from langchain.chains import ConversationalRetrievalChain

retrieval_chain = ConversationalRetrievalChain.from_llm(
llm=llm,
retriever=vector_store.as_retriever(topk=3),
return_source_documents=True
)

Step 6: Implement the Chatbot Logic

We establish the logic for preserving conversation history and formulating responses.

conversation_history = []

def get_response(user_query):
response = retrieval_chain({
“question”: user_query,
“chat_history”: conversation_history
})
conversation_history.append((user_query, response[‘answer’]))
return response[‘answer’]

Step 7: Build the User Interface with Gradio

At last, create a Gradio interface for interaction with the chatbot.

import gradio as gr

def chat_interface(user_input, history):
response = get_response(user_input)
history.append((user_input, response))
return history, history

with gr.Blocks() as demo:
chatbot = gr.Chatbot()
state = gr.State([])
with gr.Row():
user_input = gr.Textbox(show_label=False, placeholder=”Enter your question…”)
submit_btn = gr.Button(“Send”)
submit_btn.click(chat_interface, inputs=[user_input, state], outputs=[chatbot, state])

Running the Code

Save the script as app.py and execute

python app.py

Congratulations! You’re finished. The Gradio interface will open, allowing you to converse with your document.

But why halt here? You can go beyond by attempting to incorporate any of the following features into the chatbot.

Enhanced Vector Store: Utilize alternative vector databases like Milvus or Pinecone for scalability.

Fine-tuned Models: Test with finely-tuned Groq models for domain-specific precision.

Multi-Document Support: Expand the functionality to accommodate numerous documents.

Improved Context Management: Enhance conversational logic for better handling of extended chat histories.

Custom UI: Create a more refined user interface with sophisticated styling and features.

Congratulations! You have successfully established a document-based chatbot using Groq and LangChain. Experiment with enhancements and create something exceptional! 🚀

Resources:

https://nios.ac.in/media/documents/SrSec314NewE/Lesson-29.pdf

LangChain (https://www.langchain.com/)

Groq (https://groq.com/)

Moreover, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Remember to connect with our 65k+ ML SubReddit.

🚨 Recommend Open-Source Platform: Parlant is a framework that revolutionizes how AI agents make decisions in customer-facing contexts. (Promoted)

Vineet Kumar serves as a consulting intern at MarktechPost. Currently, he is acquiring his BS from the Indian Institute of Technology(IIT), Kanpur. He has a strong enthusiasm for Machine Learning. His interests lie in research and the recent progressions in Deep Learning, Computer Vision, and associated domains.