🔥 DeepEval 4.0 just got released. Read the announcement.

Trajectory-Based Evaluation

Trajectory-based evaluation assesses the entire chain of decisions and actions an AI agent takes to complete a task. It is designed for agents and long-horizon agents whose quality depends not only on the final answer, but also on the plans, tool calls, handoffs, and intermediate steps that produced it.

What Are Trajectory-Based Evals?

An agent trajectory is the ordered sequence of steps between receiving a task and producing a result. A trajectory may contain planning, LLM generations, tool calls, retries, sub-agent handoffs, and other intermediate operations.

Trajectory-based evals score this sequence as a whole. They can determine whether an agent completed its task, followed its plan, and avoided unnecessary steps by analyzing the full trace produced during execution.

This makes trajectory-based evaluation especially useful for:

  • Tool-using agents that make multiple decisions before responding.
  • Long-horizon agents that plan, act, observe, and revise over many steps.
  • Multi-agent systems where work is delegated between agents.
  • Agent workflows where two runs can produce the same final answer through very different paths.

Trajectory-Based vs Other Evals

The difference is the scope visible to the metric:

  • End-to-end evaluation treats your application as a black box and evaluates its observable input and output.
  • Trajectory-based evaluation looks inside an agent but evaluates the complete ordered chain of steps as one unit.
  • Component-level evaluation evaluates one internal span, such as a single retriever, LLM, tool, or sub-agent invocation.

Use trajectory-based evals when the path itself affects quality. You can combine all three approaches in one evaluation strategy: score the final result end-to-end, the complete agent trajectory, and selected critical components.

How Trajectory-Based Evals Work

Each evaluation task produces one trace containing the agent's complete execution tree. The trace's ordered spans—such as plans, LLM calls, tool calls, and sub-agent handoffs—form the trajectory that the metrics evaluate.

  1. evals_iterator() yields a golden containing the agent's task.
  2. Your instrumented agent runs the task and emits a trace of its internal steps.
  3. deepeval associates the completed trace with that golden.
  4. Trajectory metrics analyze the full trace and return a score and reason.
  5. The trajectory and its metric results are stored together in the test run.

Unlike component-level evaluation, the metric is not attached to one span. It receives the complete trace so it can judge relationships between steps and the agent's overall execution.

Evaluate an Agent Trajectory

Trajectory-based evaluation requires tracing, because the metrics need access to the agent's internal execution steps. The evals_iterator() method associates each dataset golden with its captured trace and evaluates that trajectory.

Build Dataset

Create an EvaluationDataset containing representative tasks for your agent. Each Golden supplies the input for one agent trajectory.

from deepeval.dataset import Golden, EvaluationDataset

goldens = [
    Golden(input="What is your name?"),
    Golden(input="Choose a number between 1 and 100"),
    # ...
]

dataset = EvaluationDataset(goldens=goldens)

The dataset lives only for this run — no push, no save. Perfect for quickstarts and one-off evaluations.

You can load entire datasets on Confident AI's cloud in one line of code.

from deepeval.dataset import EvaluationDataset

dataset = EvaluationDataset()
dataset.pull(alias="My Evals Dataset")

Non-technical domain experts can create, annotate, and comment on datasets on Confident AI. You can also upload datasets in CSV format, or push synthetic datasets created in deepeval to Confident AI in one line of code.

For more information, visit the Confident AI datasets section.

from deepeval.dataset import EvaluationDataset

dataset = EvaluationDataset()
dataset.add_goldens_from_csv_file(
    file_path="example.csv",
    input_col_name="query",
)

For more advanced options, like loading context and tools_called columns or renaming every column, see loading a dataset.

from deepeval.dataset import EvaluationDataset

dataset = EvaluationDataset()
dataset.add_goldens_from_json_file(
    file_path="example.json",
    input_key_name="query",
)

For more advanced options, like loading context and tools_called keys or reading goldens a line at a time from a .jsonl file, see loading a dataset.

Instrument Agent

Instrument your agent using deepeval's native tracing or the integration for your framework. Every decision, tool call, and nested operation captured as a span becomes part of the trajectory available to the metrics.

Wrap the top-level function with @observe and call update_current_trace(...) to set the trace-level test case fields:

main.py
import asyncio
from deepeval.tracing import observe, update_current_trace
from deepeval.metrics import TaskCompletionMetric
...

@observe()
async def my_ai_agent(query: str) -> str:
    answer = "..."  # await your LLM call here
    update_current_trace(input=query, output=answer)
    return answer

for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    task = asyncio.create_task(my_ai_agent(golden.input))
    dataset.evaluate(task)
main.py
from deepeval.evaluate import AsyncConfig
from deepeval.tracing import observe, update_current_trace
from deepeval.metrics import TaskCompletionMetric
...

@observe()
def my_ai_agent(query: str) -> str:
    answer = "..."  # call your LLM here
    update_current_trace(input=query, output=answer)
    return answer

for golden in dataset.evals_iterator(
    metrics=[TaskCompletionMetric()],
    async_config=AsyncConfig(run_async=False),
):
    my_ai_agent(golden.input)

See tracing for the full @observe and update_current_trace surface.

Build your agent with create_agent, then pass deepeval's CallbackHandler to its invoke / ainvoke method inside the loop:

langchain_app.py
import asyncio
from langchain.agents import create_agent
from deepeval.integrations.langchain import CallbackHandler
from deepeval.metrics import TaskCompletionMetric
...

def multiply(a: int, b: int) -> int:
    """Multiply two numbers."""
    return a * b

agent = create_agent(
    model="openai:gpt-4o-mini",
    tools=[multiply],
    system_prompt="Be concise.",
)

async def run_agent(prompt: str):
    return await agent.ainvoke(
        {"messages": [{"role": "user", "content": prompt}]},
        config={"callbacks": [CallbackHandler()]},
    )

for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    task = asyncio.create_task(run_agent(golden.input))
    dataset.evaluate(task)
langchain_app.py
from langchain.agents import create_agent
from deepeval.evaluate import AsyncConfig
from deepeval.integrations.langchain import CallbackHandler
from deepeval.metrics import TaskCompletionMetric
...

def multiply(a: int, b: int) -> int:
    """Multiply two numbers."""
    return a * b

agent = create_agent(
    model="openai:gpt-4o-mini",
    tools=[multiply],
    system_prompt="Be concise.",
)

for golden in dataset.evals_iterator(
    metrics=[TaskCompletionMetric()],
    async_config=AsyncConfig(run_async=False),
):
    agent.invoke(
        {"messages": [{"role": "user", "content": golden.input}]},
        config={"callbacks": [CallbackHandler()]},
    )

See the LangChain integration for the full surface.

Wire your StateGraph, then pass deepeval's CallbackHandler to its invoke / ainvoke method inside the loop:

langgraph_app.py
import asyncio
from langchain.chat_models import init_chat_model
from langgraph.graph import StateGraph, MessagesState, START, END
from deepeval.integrations.langchain import CallbackHandler
from deepeval.metrics import TaskCompletionMetric
...

llm = init_chat_model("openai:gpt-4o-mini")

async def chatbot(state: MessagesState):
    return {"messages": [await llm.ainvoke(state["messages"])]}

graph = (
    StateGraph(MessagesState)
    .add_node(chatbot)
    .add_edge(START, "chatbot")
    .add_edge("chatbot", END)
    .compile()
)

async def run_graph(prompt: str):
    return await graph.ainvoke(
        {"messages": [{"role": "user", "content": prompt}]},
        config={"callbacks": [CallbackHandler()]},
    )

for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    task = asyncio.create_task(run_graph(golden.input))
    dataset.evaluate(task)
langgraph_app.py
from langchain.chat_models import init_chat_model
from langgraph.graph import StateGraph, MessagesState, START, END
from deepeval.evaluate import AsyncConfig
from deepeval.integrations.langchain import CallbackHandler
from deepeval.metrics import TaskCompletionMetric
...

llm = init_chat_model("openai:gpt-4o-mini")

def chatbot(state: MessagesState):
    return {"messages": [llm.invoke(state["messages"])]}

graph = (
    StateGraph(MessagesState)
    .add_node(chatbot)
    .add_edge(START, "chatbot")
    .add_edge("chatbot", END)
    .compile()
)

for golden in dataset.evals_iterator(
    metrics=[TaskCompletionMetric()],
    async_config=AsyncConfig(run_async=False),
):
    graph.invoke(
        {"messages": [{"role": "user", "content": golden.input}]},
        config={"callbacks": [CallbackHandler()]},
    )

See the LangGraph integration for the full surface.

Drop-in replace from openai import OpenAI with from deepeval.openai import OpenAI (or AsyncOpenAI). Wrap the call in with trace(): so the LLM call becomes a trace:

openai_app.py
import asyncio
from deepeval.openai import AsyncOpenAI
from deepeval.tracing import trace
from deepeval.metrics import TaskCompletionMetric
...

client = AsyncOpenAI()

async def call_openai(prompt: str):
    with trace():
        return await client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": prompt}],
        )

for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    task = asyncio.create_task(call_openai(golden.input))
    dataset.evaluate(task)
openai_app.py
from deepeval.openai import OpenAI
from deepeval.tracing import trace
from deepeval.evaluate import AsyncConfig
from deepeval.metrics import TaskCompletionMetric
...

client = OpenAI()

for golden in dataset.evals_iterator(
    metrics=[TaskCompletionMetric()],
    async_config=AsyncConfig(run_async=False),
):
    with trace():
        client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": golden.input}],
        )

See the OpenAI integration for streaming and tool-calling.

Pass DeepEvalInstrumentationSettings() to your Agent's instrument keyword:

pydanticai_agent.py
import asyncio
from pydantic_ai import Agent
from deepeval.integrations.pydantic_ai import DeepEvalInstrumentationSettings
from deepeval.metrics import TaskCompletionMetric
...

agent = Agent(
    "openai:gpt-4.1",
    system_prompt="Be concise.",
    instrument=DeepEvalInstrumentationSettings(),
)

for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    task = asyncio.create_task(agent.run(golden.input))
    dataset.evaluate(task)
pydanticai_agent.py
from pydantic_ai import Agent
from deepeval.evaluate import AsyncConfig
from deepeval.integrations.pydantic_ai import DeepEvalInstrumentationSettings
from deepeval.metrics import TaskCompletionMetric
...

agent = Agent(
    "openai:gpt-4.1",
    system_prompt="Be concise.",
    instrument=DeepEvalInstrumentationSettings(),
)

for golden in dataset.evals_iterator(
    metrics=[TaskCompletionMetric()],
    async_config=AsyncConfig(run_async=False),
):
    agent.run_sync(golden.input)

See the Pydantic AI integration for the full surface.

Call instrument_agentcore() before creating your agent. The same call also instruments Strands agents running inside AgentCore:

agentcore_agent.py
import asyncio
from strands import Agent
from deepeval.integrations.agentcore import instrument_agentcore
from deepeval.metrics import TaskCompletionMetric
...

instrument_agentcore()

agent = Agent(model="amazon.nova-lite-v1:0")

for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    task = asyncio.create_task(agent.invoke_async(golden.input))
    dataset.evaluate(task)
agentcore_agent.py
from strands import Agent
from deepeval.evaluate import AsyncConfig
from deepeval.integrations.agentcore import instrument_agentcore
from deepeval.metrics import TaskCompletionMetric
...

instrument_agentcore()

agent = Agent(model="amazon.nova-lite-v1:0")

for golden in dataset.evals_iterator(
    metrics=[TaskCompletionMetric()],
    async_config=AsyncConfig(run_async=False),
):
    agent(golden.input)

See the AgentCore integration for the full surface (including the BedrockAgentCoreApp entrypoint pattern).

Call instrument_strands() before invoking your Strands agent (for AgentCore-hosted Strands, use the AgentCore tab instead):

strands_agent.py
import asyncio
from strands import Agent
from strands.models.openai import OpenAIModel
from deepeval.integrations.strands import instrument_strands
from deepeval.metrics import TaskCompletionMetric
...

instrument_strands()

agent = Agent(
    model=OpenAIModel(model_id="gpt-4o-mini"),
    system_prompt="You are a helpful assistant.",
)

for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    task = asyncio.create_task(agent.invoke_async(golden.input))
    dataset.evaluate(task)
strands_agent.py
from strands import Agent
from strands.models.openai import OpenAIModel
from deepeval.evaluate import AsyncConfig
from deepeval.integrations.strands import instrument_strands
from deepeval.metrics import TaskCompletionMetric
...

instrument_strands()

agent = Agent(
    model=OpenAIModel(model_id="gpt-4o-mini"),
    system_prompt="You are a helpful assistant.",
)

for golden in dataset.evals_iterator(
    metrics=[TaskCompletionMetric()],
    async_config=AsyncConfig(run_async=False),
):
    agent(golden.input)

See the Strands integration for the full surface.

Drop-in replace from anthropic import Anthropic with from deepeval.anthropic import Anthropic (or AsyncAnthropic). Wrap the call in with trace(): so the LLM call becomes a trace:

anthropic_app.py
import asyncio
from deepeval.anthropic import AsyncAnthropic
from deepeval.tracing import trace
from deepeval.metrics import TaskCompletionMetric
...

client = AsyncAnthropic()

async def call_claude(prompt: str):
    with trace():
        return await client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1024,
            messages=[{"role": "user", "content": prompt}],
        )

for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    task = asyncio.create_task(call_claude(golden.input))
    dataset.evaluate(task)
anthropic_app.py
from deepeval.anthropic import Anthropic
from deepeval.tracing import trace
from deepeval.evaluate import AsyncConfig
from deepeval.metrics import TaskCompletionMetric
...

client = Anthropic()

for golden in dataset.evals_iterator(
    metrics=[TaskCompletionMetric()],
    async_config=AsyncConfig(run_async=False),
):
    with trace():
        client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1024,
            messages=[{"role": "user", "content": golden.input}],
        )

See the Anthropic integration for streaming and tool-use.

Register deepeval's event handler against LlamaIndex's instrumentation dispatcher. agent.run(...) is async-only, so the sync variant uses asyncio.run(...):

llamaindex_agent.py
import asyncio
from llama_index.llms.openai import OpenAI
from llama_index.core.agent import FunctionAgent
import llama_index.core.instrumentation as instrument
from deepeval.integrations.llama_index import instrument_llama_index
from deepeval.metrics import TaskCompletionMetric
...

instrument_llama_index(instrument.get_dispatcher())

def multiply(a: float, b: float) -> float:
    return a * b

agent = FunctionAgent(
    tools=[multiply],
    llm=OpenAI(model="gpt-4o-mini"),
    system_prompt="You are a helpful calculator.",
)

for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    task = asyncio.create_task(agent.run(golden.input))
    dataset.evaluate(task)
llamaindex_agent.py
import asyncio
from llama_index.llms.openai import OpenAI
from llama_index.core.agent import FunctionAgent
import llama_index.core.instrumentation as instrument
from deepeval.evaluate import AsyncConfig
from deepeval.integrations.llama_index import instrument_llama_index
from deepeval.metrics import TaskCompletionMetric
...

instrument_llama_index(instrument.get_dispatcher())

def multiply(a: float, b: float) -> float:
    return a * b

agent = FunctionAgent(
    tools=[multiply],
    llm=OpenAI(model="gpt-4o-mini"),
    system_prompt="You are a helpful calculator.",
)

for golden in dataset.evals_iterator(
    metrics=[TaskCompletionMetric()],
    async_config=AsyncConfig(run_async=False),
):
    asyncio.run(agent.run(golden.input))

See the LlamaIndex integration for the full surface.

Register DeepEvalTracingProcessor once, then build your agent with deepeval's Agent and function_tool shims:

openai_agents_app.py
import asyncio
from agents import Runner, add_trace_processor
from deepeval.openai_agents import Agent, DeepEvalTracingProcessor, function_tool
from deepeval.metrics import TaskCompletionMetric
...

add_trace_processor(DeepEvalTracingProcessor())

@function_tool
def get_weather(city: str) -> str:
    return f"It's always sunny in {city}!"

agent = Agent(
    name="weather_agent",
    instructions="Answer weather questions concisely.",
    tools=[get_weather],
)

for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    task = asyncio.create_task(Runner.run(agent, golden.input))
    dataset.evaluate(task)
openai_agents_app.py
from agents import Runner, add_trace_processor
from deepeval.evaluate import AsyncConfig
from deepeval.openai_agents import Agent, DeepEvalTracingProcessor, function_tool
from deepeval.metrics import TaskCompletionMetric
...

add_trace_processor(DeepEvalTracingProcessor())

@function_tool
def get_weather(city: str) -> str:
    return f"It's always sunny in {city}!"

agent = Agent(
    name="weather_agent",
    instructions="Answer weather questions concisely.",
    tools=[get_weather],
)

for golden in dataset.evals_iterator(
    metrics=[TaskCompletionMetric()],
    async_config=AsyncConfig(run_async=False),
):
    Runner.run_sync(agent, golden.input)

See the OpenAI Agents integration for the full surface.

Call instrument_google_adk() once before building your LlmAgent. ADK's runner.run_async(...) is async-only, so the sync variant uses asyncio.run(...):

google_adk_agent.py
import asyncio
from google.adk.agents import LlmAgent
from google.adk.runners import InMemoryRunner
from google.genai import types
from deepeval.integrations.google_adk import instrument_google_adk
from deepeval.metrics import TaskCompletionMetric
...

instrument_google_adk()

agent = LlmAgent(model="gemini-2.0-flash", name="assistant", instruction="Be concise.")
runner = InMemoryRunner(agent=agent, app_name="deepeval-quickstart")

async def run_agent(prompt: str) -> str:
    session = await runner.session_service.create_session(
        app_name="deepeval-quickstart", user_id="demo-user",
    )
    message = types.Content(role="user", parts=[types.Part(text=prompt)])
    async for event in runner.run_async(
        user_id="demo-user", session_id=session.id, new_message=message,
    ):
        if event.is_final_response() and event.content:
            return "".join(part.text for part in event.content.parts if getattr(part, "text", None))
    return ""

for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    task = asyncio.create_task(run_agent(golden.input))
    dataset.evaluate(task)
google_adk_agent.py
import asyncio
from google.adk.agents import LlmAgent
from google.adk.runners import InMemoryRunner
from google.genai import types
from deepeval.evaluate import AsyncConfig
from deepeval.integrations.google_adk import instrument_google_adk
from deepeval.metrics import TaskCompletionMetric
...

instrument_google_adk()

agent = LlmAgent(model="gemini-2.0-flash", name="assistant", instruction="Be concise.")
runner = InMemoryRunner(agent=agent, app_name="deepeval-quickstart")

async def run_agent(prompt: str) -> str:
    session = await runner.session_service.create_session(
        app_name="deepeval-quickstart", user_id="demo-user",
    )
    message = types.Content(role="user", parts=[types.Part(text=prompt)])
    async for event in runner.run_async(
        user_id="demo-user", session_id=session.id, new_message=message,
    ):
        if event.is_final_response() and event.content:
            return "".join(part.text for part in event.content.parts if getattr(part, "text", None))
    return ""

for golden in dataset.evals_iterator(
    metrics=[TaskCompletionMetric()],
    async_config=AsyncConfig(run_async=False),
):
    asyncio.run(run_agent(golden.input))

See the Google ADK integration for the full surface.

Call instrument_crewai() once, then build your crew with deepeval's Crew, Agent, and @tool shims:

crewai_app.py
import asyncio
from crewai import Task
from deepeval.integrations.crewai import instrument_crewai, Crew, Agent
from deepeval.metrics import TaskCompletionMetric
...

instrument_crewai()

tutor = Agent(
    role="Math Tutor",
    goal="Answer math questions accurately and concisely.",
    backstory="An experienced tutor who explains simple math clearly.",
)
answer_task = Task(
    description="{question}",
    expected_output="An accurate, concise answer.",
    agent=tutor,
)
crew = Crew(agents=[tutor], tasks=[answer_task])

for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    task = asyncio.create_task(crew.kickoff_async({"question": golden.input}))
    dataset.evaluate(task)
crewai_app.py
from crewai import Task
from deepeval.evaluate import AsyncConfig
from deepeval.integrations.crewai import instrument_crewai, Crew, Agent
from deepeval.metrics import TaskCompletionMetric
...

instrument_crewai()

tutor = Agent(
    role="Math Tutor",
    goal="Answer math questions accurately and concisely.",
    backstory="An experienced tutor who explains simple math clearly.",
)
task = Task(
    description="{question}",
    expected_output="An accurate, concise answer.",
    agent=tutor,
)
crew = Crew(agents=[tutor], tasks=[task])

for golden in dataset.evals_iterator(
    metrics=[TaskCompletionMetric()],
    async_config=AsyncConfig(run_async=False),
):
    crew.kickoff({"question": golden.input})

See the CrewAI integration for the full surface.

Evaluate Trajectory

Pass trajectory metrics to evals_iterator(), then invoke your traced agent once for each golden. This guide uses only TaskCompletionMetric, StepEfficiencyMetric, and PlanAdherenceMetric; see the metrics introduction for guidance on metrics and their behavior.

evaluate_agent.py
from deepeval.metrics import (
    TaskCompletionMetric,
    StepEfficiencyMetric,
    PlanAdherenceMetric,
)

metrics = [
    TaskCompletionMetric(),
    StepEfficiencyMetric(),
    PlanAdherenceMetric(),
]

for golden in dataset.evals_iterator(metrics=metrics):
    my_ai_agent(golden.input)

The iterator captures one trace per golden and evaluates the complete trajectory after the agent finishes. Each metric score and reason is stored alongside the trace, allowing you to connect a failure to the exact execution path that produced it.

In CI/CD

Run trajectory-based evals on every pull request by moving the same dataset, traced agent, and metrics into a pytest test. A test fails when any trajectory metric falls below its threshold, allowing the eval to block a regression from shipping.

test_agent_trajectory.py
from deepeval.metrics import (
    TaskCompletionMetric,
    StepEfficiencyMetric,
    PlanAdherenceMetric,
)
from deepeval.dataset import EvaluationDataset, Golden
from deepeval import assert_test
from app import my_ai_agent
import pytest

dataset = EvaluationDataset(
    goldens=[Golden(input="Plan a three-day trip to Paris")]
)
metrics = [
    TaskCompletionMetric(),
    StepEfficiencyMetric(),
    PlanAdherenceMetric(),
]

@pytest.mark.parametrize("golden", dataset.goldens)
def test_agent_trajectory(golden: Golden):
    my_ai_agent(golden.input)
    assert_test(golden=golden, metrics=metrics)
deepeval test run test_agent_trajectory.py

Your agent must remain instrumented in CI so the assertion can capture and evaluate its complete trace. See unit testing in CI/CD for pipeline configuration and command options.

FAQs

What is trajectory-based evaluation?
Trajectory-based evaluation scores the complete ordered path an AI agent takes through planning, model calls, tools, handoffs, and other intermediate steps while completing one task.
How is trajectory-based evaluation different from end-to-end and component-level evaluation?
End-to-end evaluation treats the application as a black box and scores its observable result. Component-level evaluation scores one internal span. Trajectory-based evaluation looks inside the agent but scores the entire chain of spans as one execution.
Why does trajectory-based evaluation require tracing?
The metrics need the agent's internal execution steps, not only its final output. Tracing captures those steps as an ordered tree and associates the resulting trajectory with the current golden.
Can trajectory-based evals run in CI/CD?
Yes. Run your traced agent inside a pytest or vitest test and pass trajectory metrics to the assertion. The test fails when a metric misses its configured threshold.
Which metrics does this guide use?
This guide uses TaskCompletionMetric, StepEfficiencyMetric, and PlanAdherenceMetric. See the metrics introduction for metric guidance and configuration.
Can I combine trajectory-based and component-level evaluation?
Yes. Pass trajectory metrics to the iterator or test assertion, then attach component metrics to the individual spans you also want to inspect. Both scopes can be evaluated during the same traced run.

Multi-Turn

Previous Page

Component-Level Evals

Next Page

On this page

What Are Trajectory-Based Evals?Trajectory-Based vs Other EvalsHow Trajectory-Based Evals WorkEvaluate an Agent TrajectoryBuild DatasetInstrument AgentEvaluate TrajectoryIn CI/CDFAQs