🔥 DeepEval 4.0 just got released. Read the announcement.

Building Your Chatbot

In this section, we are going to create a multi-turn chatbot that can use various tools to diagnose and schedule appointments for users based on their symptoms. We will be using langchain and qdrant to build our chatbot, with functionalies including a:

  • RAG pipeline to retrieve medical knowledge to diagnose patients
  • Custom tools to create new appointments based on patient symptoms
  • Memory system to keep track of chat histories

We'll also implement our chatbot with an independent model and system prompt variable - which we'll be evaluating in the next section.

Setup Your Model

First create a MedicalChatbot class and use langchain's chat models to call OpenAI:

main.py
from langchain_openai import ChatOpenAI

class MedicalChatbot:
    def __init__(self, model: str):
        self.model = ChatOpenAI(model=model)
        # Choose the LLM that will drive the agent
        # Only certain models support this so ensure your model supports it as well

Try prompting it with a messages array:

main.py
chatbot = MedicalChatbot(model="gpt-4o-mini")
chatbot.model.invoke([{"user": "Hi!"}])

Which should let you see something like this:

AIMessage(
    content="Hey, how can I help you today?",
    additional_kwargs={},
    response_metadata={
        'prompt_feedback': {'block_reason': 0, 'safety_ratings': []},
        'finish_reason': 'STOP',
        'model_name': 'gpt-4o-mini',
        'safety_ratings': []
    },
    id='run--c2786aa1-75c4-4644-ae59-9327a2e8c153-0',
    usage_metadata={'input_tokens': 23, 'output_tokens': 417, 'total_tokens': 440, 'input_token_details': {'cache_read': 0}}
)

✅ Done. Now let's create some tools for the chatbot to start booking appointments.

Create RAG Pipeline For Diagnosis

Since OpenAI models weren't specifically trained on medical knowledge, we'll need to leverage RAG to provide additional context at runtime to diagnose patients that are grounded in context.

Index medical knowledge

We'll ingest "The Gale Encyclopedia of Alternative Medicine" to Qdrant, a popular vector database choice for fast and accurate retrievals:

main.py
from qdrant_client import models, QdrantClient
from sentence_transformers import SentenceTransformer
from langchain_openai import ChatOpenAI

class MedicalChatbot:
    def __init__(self, model: str):
        self.model = ChatOpenAI(model=model)
        # For RAG engine
        self.encoder = SentenceTransformer("all-MiniLM-L6-v2")
        self.client = QdrantClient(":memory:")

    def index_knowledge(self, document_path: str):
        with open(document_path) as file:
            documents = file.readlines()

        # Create namespace in qdrant
        self.client.create_collection(
            collection_name="gale_encyclopedia",
            vectors_config=models.VectorParams(size=self.encoder.get_sentence_embedding_dimension(), distance=models.Distance.COSINE),
        )

        # Vectorize and index into qdrant
        self.client.upload_points(
            collection_name="gale_encyclopedia",
            points=[models.PointStruct(id=idx, vector=self.encoder.encode(doc).tolist(), payload={"content": doc}) for idx, doc in enumerate(documents)],
        )

Then, simply run your index_knowledge method usign the encyclopedia you've downloaded as .txt:

main.py
chatbot = MedicalChatbot()
chatbot.index_knowledge("path-to-your-encyclopedia.txt")

✅ Done. Now let's try querying it to sanity check yourself.

Query your knowledge base

Simply implement a TOOL to query from qdrant. in this case retrieve_knowledge:

main.py
from qdrant_client import models, QdrantClient
from sentence_transformers import SentenceTransformer
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool

class MedicalChatbot:
    def __init__(self, model: str):
        self.model = ChatOpenAI(model=model)
        # For RAG engine
        self.encoder = SentenceTransformer("all-MiniLM-L6-v2")
        self.client = QdrantClient(":memory:")

    @tool
    def retrieve_knowledge(self, query: str) -> str:
        """"A tool to retrive data on various diagnosis methods from gale encyclopedia"""
        hits = self.client.query_points(collection_name="gale_encyclopedia", query=self.encoder.encode(query).tolist(), limit=3).points

        contexts = [hit.payload['content'] for hit in hits]
        return "\n".join(contexts)

    def index_knowledge(self, document_path: str):
        # Same as above
        pass

Now try calling it:

main.py
chatbot = MedicalChatbot()
chatbot.retrieve_knowledge("Cough, fever, and diarrhea.")

Great! Now that we have the essentials for making a diagnosis, time to move on to implementing a way to book appointments after a diagnosis.

Create Tool To Book Appointments

Since we need a way for our chatbot to book appointments based on the diagnosis at hand, this section will focus on creating the tools required to do so. There's only one tool for booking appointments for the sake of simplicity:

  • create_appointment: Creates a new appointment in memory (you can also use something like SQLite for persistent storage)

First, let's create a simple data model for appointments:

main.py
from pydantic import BaseModel, Field
from typing import Optional, List
from datetime import date

class Appointment(BaseModel):
    id: str
    name: str
    email: str
    date: date
    symptoms: Optional[List[str]] = Field(default=None)
    diagnosis: Optional[str] = Field(default=None)

Now let's implement the create_appointment tool:

main.py
import uuid

...

class MedicalChatbot:
    def __init__(self, model: str):
        self.model = ChatOpenAI(model=model)
        # For RAG engine
        self.encoder = SentenceTransformer("all-MiniLM-L6-v2")
        self.client = QdrantClient(":memory:")
        # For managing appointments
        self.appointments: List[Appointment] = []

    @tool
    def create_appointment(self, name: str, email: str, date: str) -> str:
        """Create a new appointment with the given ID, name, email, and date"""
        try:
            appointment = Appointment(
                id=str(uuid.uuid4()),
                name=name,
                email=email,
                date=date.fromisoformat(date)
            )
            self.appointments.append(appointment)
            return f"Created new appointment with ID: {appointment.id} for {name} on {date}."
        except ValueError:
            return f"Invalid date format. Please use YYYY-MM-DD format."

    @tool
    def retrieve_knowledge(self, query: str) -> str:
        # Same as above
        pass

    def index_knowledge(self, document_path: str):
        # Same as above
        pass

Great! Now let's glue everything together using LangChain.

Implementing Chat Histories

First create a helper method that retrieves conversation histories, which would be required for our LLM:

from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.chat_history import BaseChatMessageHistory

# Simple in-memory store for chat histories
chat_store = {}
def get_session_history(session_id: str) -> BaseChatMessageHistory:
    if session_id not in chat_store:
        chat_store[session_id] = ChatMessageHistory()
    return chat_store[session_id]

Then we'll combine the agent setup and memory functionality into one clean implementation, including the retrieve_knowledge and create_appointment tools in our agent:

main.py
from langchain.agents import create_tool_calling_agent, AgentExecutor
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_core.tools import StructuredTool
...

class MedicalChatbot:
    def __init__(self, model: str, system_prompt: str):
        self.model = ChatOpenAI(model=model)
        self.system_prompt = system_prompt
        # For RAG engine
        self.encoder = SentenceTransformer("all-MiniLM-L6-v2")
        self.client = QdrantClient(":memory:")
        # For managing appointments
        self.appointments: List[Appointment] = []

        # Setup agent with memory
        self.setup_agent()

    def setup_agent(self):
        """Setup the agent with tools and memory"""

        # Create prompt messages
        prompt = ChatPromptTemplate.from_messages([("system", self.system_prompt), MessagesPlaceholder(variable_name="chat_history"), ("human", "{input}")])

        # Create agent
        tools = [
            StructuredTool.from_function(func=self.retrieve_knowledge),
            StructuredTool.from_function(func=self.create_appointment)
        ]
        agent = create_tool_calling_agent(self.model, tools, prompt)
        agent_executor = AgentExecutor(agent=agent, tools=tools)
        self.agent_with_memory = RunnableWithMessageHistory(
            agent_executor,
            get_session_history,
            input_messages_key="input",
            history_messages_key="chat_history",
        )

    # Other methods from above goes here
    ...

🎉🥳 Congratulations! You've just created a fully functional medical chatbot with memory, the abiliy to diagnose users, and book appointments when needed.

Eyeball Your First Output

Now that you have your chatbot, its time to query it to see if it lives up to your expectations. Create a method so you can interact with it in the CLI, and supply your model and choice and system prompt:

main.py
def start_session(session_id: Optional[str] = None):
    """Start an interactive session with the chatbot"""
    print("Hello! I am Baymax, your personal healthcare companion.")
    print("How are you feeling today? (type 'exit' to quit."))

    while True:
        if session_id is None:
            session_id = str(uuid.uuid4())

        user_input = input("Your query: ")
        if user_input.lower() == 'exit':
            break

        response = chatbot.agent_with_memory.invoke(
            {"input": user_input},
            config={"configurable": {"session_id": session_id}}
        )
        print("Baymax:", response["output"])


# These parameters will be evaluated later
MODEL = "gpt-4o-mini"
SYSTEM_PROMPT = """"""

# Initialize chatbot and start session
chatbot = MedicalChatbot(model=MODEL, system_prompt=SYSTEM_PROMPT)
chatbot.index_knowledge("path-to-your-encyclopedia.txt")
start_session()

Then simply execute the file in the CLI:

python main.py

Here's an example conversation:

Hello! I am Baymax, your personal healthcare companion.
How are you feeling today? (type 'exit' to quit.")

Your query: Hello Baymax, I've been feeling unwell. I have a fever and a sore throat.
Baymax: I'm sorry to hear you're not feeling well. A fever and sore throat are common symptoms of upper respiratory infections. Based on medical knowledge, rest, fluids, and over-the-counter pain relievers can help. If your symptoms are severe or persistent, it's best to consult a doctor. Would you like me to help you schedule an appointment?

Your query: Yes, please book an appointment for tomorrow at 2 PM.
Baymax: I'll help you create an appointment. I need your name and email to proceed. Could you please provide those details?

Your query: My name is John Doe and my email is john@example.com
Baymax: Created new appointment with ID: 550e8400-e29b-41d4-a716-446655440000 for John Doe on 2024-01-16.

Was this what you really wanted? Is this diagnosis a good one? Was the appointment booked appropriately? The process of making these judgements yourself are known as eyeballing LLM outputs. It works, but isn't very scalable nor reliable - especially when conversations get long and you find yourself skimming instead of evaluating.

Introduction

Previous Page

Evaluate Multi-Turn Convos

Next Page

On this page

Setup Your ModelCreate RAG Pipeline For DiagnosisIndex medical knowledgeQuery your knowledge baseCreate Tool To Book AppointmentsImplementing Chat HistoriesEyeball Your First Output