Building Your Chatbot
In this section, we are going to create a multi-turn chatbot that can use various tools to diagnose and schedule appointments for users based on their symptoms.
We will be using langchain and qdrant to build our chatbot, with functionalies including a:
- RAG pipeline to retrieve medical knowledge to diagnose patients
- Custom tools to create new appointments based on patient symptoms
- Memory system to keep track of chat histories
We'll also implement our chatbot with an independent model and system prompt variable - which we'll be evaluating in the next section.
If you already have a multi-turn chatbot that you want to evaluate, feel free to skip to the evaluation section of this tuorial.
Setup Your Model
First create a MedicalChatbot class and use langchain's chat models to call OpenAI:
from langchain_openai import ChatOpenAI
class MedicalChatbot:
def __init__(self, model: str):
self.model = ChatOpenAI(model=model)
# Choose the LLM that will drive the agent
# Only certain models support this so ensure your model supports it as well
You can also use other interfaces to call OpenAI, or any other model.
Try prompting it with a messages array:
chatbot = MedicalChatbot(model="gpt-4o-mini")
chatbot.model.invoke([{"user": "Hi!"}])
Which should let you see something like this:
AIMessage(
content="Hey, how can I help you today?",
additional_kwargs={},
response_metadata={
'prompt_feedback': {'block_reason': 0, 'safety_ratings': []},
'finish_reason': 'STOP',
'model_name': 'gpt-4o-mini',
'safety_ratings': []
},
id='run--c2786aa1-75c4-4644-ae59-9327a2e8c153-0',
usage_metadata={'input_tokens': 23, 'output_tokens': 417, 'total_tokens': 440, 'input_token_details': {'cache_read': 0}}
)
✅ Done. Now let's create some tools for the chatbot to start booking appointments.
Create RAG Pipeline For Diagnosis
Since OpenAI models weren't specifically trained on medical knowledge, we'll need to leverage RAG to provide additional context at runtime to diagnose patients that are grounded in context.
We'll be using a text version of The Gale Encyclopedia of Alternative Medicine as our knowledge base in this example. You will need to download it locally and convert it to a .txt file.
Index medical knowledge
We'll ingest "The Gale Encyclopedia of Alternative Medicine" to Qdrant, a popular vector database choice for fast and accurate retrievals:
from qdrant_client import models, QdrantClient
from sentence_transformers import SentenceTransformer
from langchain_openai import ChatOpenAI
class MedicalChatbot:
def __init__(self, model: str):
self.model = ChatOpenAI(model=model)
# For RAG engine
self.encoder = SentenceTransformer("all-MiniLM-L6-v2")
self.client = QdrantClient(":memory:")
def index_knowledge(self, document_path: str):
with open(document_path) as file:
documents = file.readlines()
# Create namespace in qdrant
self.client.create_collection(
collection_name="gale_encyclopedia",
vectors_config=models.VectorParams(size=self.encoder.get_sentence_embedding_dimension(), distance=models.Distance.COSINE),
)
# Vectorize and index into qdrant
self.client.upload_points(
collection_name="gale_encyclopedia",
points=[models.PointStruct(id=idx, vector=self.encoder.encode(doc).tolist(), payload={"content": doc}) for idx, doc in enumerate(documents)],
)
Then, simply run your index_knowledge method usign the encyclopedia you've downloaded as .txt:
chatbot = MedicalChatbot()
chatbot.index_knowledge("path-to-your-encyclopedia.txt")
✅ Done. Now let's try querying it to sanity check yourself.
You only have the run index_knowledge once.
Query your knowledge base
Simply implement a TOOL to query from qdrant. in this case retrieve_knowledge:
from qdrant_client import models, QdrantClient
from sentence_transformers import SentenceTransformer
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
class MedicalChatbot:
def __init__(self, model: str):
self.model = ChatOpenAI(model=model)
# For RAG engine
self.encoder = SentenceTransformer("all-MiniLM-L6-v2")
self.client = QdrantClient(":memory:")
@tool
def retrieve_knowledge(self, query: str) -> str:
""""A tool to retrive data on various diagnosis methods from gale encyclopedia"""
hits = self.client.query_points(collection_name="gale_encyclopedia", query=self.encoder.encode(query).tolist(), limit=3).points
contexts = [hit.payload['content'] for hit in hits]
return "\n".join(contexts)
def index_knowledge(self, document_path: str):
# Same as above
pass
The @tool decorator tells langchain that the retrieve_knowledge method can be called as a function call and will come in handy in later sections.
Now try calling it:
chatbot = MedicalChatbot()
chatbot.retrieve_knowledge("Cough, fever, and diarrhea.")
Great! Now that we have the essentials for making a diagnosis, time to move on to implementing a way to book appointments after a diagnosis.
Create Tool To Book Appointments
Since we need a way for our chatbot to book appointments based on the diagnosis at hand, this section will focus on creating the tools required to do so. There's only one tool for booking appointments for the sake of simplicity:
create_appointment: Creates a new appointment in memory (you can also use something like SQLite for persistance storage)
First, let's create a simple data model for appointments:
from pydantic import BaseModel, Field
from typing import Optional, List
from datetime import date
class Appointment(BaseModel):
id: str
name: str
email: str
date: date
symptoms: Optional[List[str]] = Field(default=None)
diagnosis: Optional[str] = Field(default=None)
Now let's implement the create_appointment tool:
import uuid
...
class MedicalChatbot:
def __init__(self, model: str):
self.model = ChatOpenAI(model=model)
# For RAG engine
self.encoder = SentenceTransformer("all-MiniLM-L6-v2")
self.client = QdrantClient(":memory:")
# For managing appointments
self.appointments: List[Appointment] = []
@tool
def create_appointment(self, name: str, email: str, date: str) -> str:
"""Create a new appointment with the given ID, name, email, and date"""
try:
appointment = Appointment(
id=str(uuid.uuid4()),
name=name,
email=email,
date=date.fromisoformat(date)
)
self.appointments.append(appointment)
return f"Created new appointment with ID: {appointment.id} for {name} on {date}."
except ValueError:
return f"Invalid date format. Please use YYYY-MM-DD format."
@tool
def retrieve_knowledge(self, query: str) -> str:
# Same as above
pass
def index_knowledge(self, document_path: str):
# Same as above
pass
Great! Now let's glue everything together using LangChain.
Implementing Chat Histories
First create a helper method that retrieves conversation histories, which would be required for our LLM:
from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.chat_history import BaseChatMessageHistory
# Simple in-memory store for chat histories
chat_store = {}
def get_session_history(session_id: str) -> BaseChatMessageHistory:
if session_id not in chat_store:
chat_store[session_id] = ChatMessageHistory()
return chat_store[session_id]
Then we'll combine the agent setup and memory functionality into one clean implementation, including the retrieve_knowledge and create_appointment tools in our agent:
from langchain.agents import create_tool_calling_agent, AgentExecutor
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_core.tools import StructuredTool
...
class MedicalChatbot:
def __init__(self, model: str, system_prompt: str):
self.model = ChatOpenAI(model=model)
self.system_prompt = system_prompt
# For RAG engine
self.encoder = SentenceTransformer("all-MiniLM-L6-v2")
self.client = QdrantClient(":memory:")
# For managing appointments
self.appointments: List[Appointment] = []
# Setup agent with memory
self.setup_agent()
def setup_agent(self):
"""Setup the agent with tools and memory"""
# Create prompt messages
prompt = ChatPromptTemplate.from_messages([("system", self.system_prompt), MessagesPlaceholder(variable_name="chat_history"), ("human", "{input}")])
# Create agent
tools = [
StructuredTool.from_function(func=self.retrieve_knowledge),
StructuredTool.from_function(func=self.create_appointment)
]
agent = create_tool_calling_agent(self.model, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools)
self.agent_with_memory = RunnableWithMessageHistory(
agent_executor,
get_session_history,
input_messages_key="input",
history_messages_key="chat_history",
)
# Other methods from above goes here
...
🎉🥳 Congratulations! You've just created a fully functional medical chatbot with memory, the abiliy to diagnose users, and book appointments when needed.
Eyeball Your First Output
Now that you have your chatbot, its time to query it to see if it lives up to your expectations. Create a method so you can interact with it in the CLI, and supply your model and choice and system prompt:
def start_session(session_id: Optional[str] = None):
"""Start an interactive session with the chatbot"""
print("Hello! I am Baymax, your personal healthcare companion.")
print("How are you feeling today? (type 'exit' to quit."))
while True:
if session_id is None:
session_id = str(uuid.uuid4())
user_input = input("Your query: ")
if user_input.lower() == 'exit':
break
response = chatbot.agent_with_memory.invoke(
{"input": user_input},
config={"configurable": {"session_id": session_id}}
)
print("Baymax:", response["output"])
# These parameters will be evaluated later
MODEL = "gpt-4o-mini"
SYSTEM_PROMPT = """"""
# Initialize chatbot and start session
chatbot = MedicalChatbot(model=MODEL, system_prompt=SYSTEM_PROMPT)
chatbot.index_knowledge("path-to-your-encyclopedia.txt")
start_session()
The model and system prompt are the variables you'll want to be improving in the next section.
Then simply execute the file in the CLI:
python main.py
Here's an example conversation:
Hello! I am Baymax, your personal healthcare companion.
How are you feeling today? (type 'exit' to quit.")
Your query: Hello Baymax, I've been feeling unwell. I have a fever and a sore throat.
Baymax: I'm sorry to hear you're not feeling well. A fever and sore throat are common symptoms of upper respiratory infections. Based on medical knowledge, rest, fluids, and over-the-counter pain relievers can help. If your symptoms are severe or persistent, it's best to consult a doctor. Would you like me to help you schedule an appointment?
Your query: Yes, please book an appointment for tomorrow at 2 PM.
Baymax: I'll help you create an appointment. I need your name and email to proceed. Could you please provide those details?
Your query: My name is John Doe and my email is john@example.com
Baymax: Created new appointment with ID: 550e8400-e29b-41d4-a716-446655440000 for John Doe on 2024-01-16.
Was this what you really wanted? Is this diagnosis a good one? Was the appointment booked appropriately? The process of making these judgements yourself are known as eyeballing LLM outputs. It works, but isn't very scalable nor reliable - especially when conversations get long and you find yourself skimming instead of evaluating.