Portkey

deepeval's integration with Portkey AI allows you to use the portkey gateway to connect to any model to power all of deepeval's metrics.

Command Line

To configure your Portkey model through the CLI, run the following command:

deepeval set-portkey \
    --model "your-model" \ # Ex: gpt-4.1
    --provider "your-provider" \ # Ex: openai
    --base-url "your-base-url" \
    --temperature=0

The CLI command above sets Portkey as the default provider for all metrics, unless overridden in Python code. To use a different default model provider, you must first unset Portkey:

deepeval unset-portkey

Python

Alternatively, you can define PortkeyModel directly in python code:

from deepeval.models import PortkeyModel
from deepeval.metrics import AnswerRelevancyMetric

model = PortkeyModel(
    model="gpt-4.1",
    provider="openai",
    api_key="your-api-key",
    base_url="your-base-url"
)

answer_relevancy = AnswerRelevancyMetric(model=model)

To use any Portkey model directly in deepeval, set the USE_PORTKEY_MODEL=1 in your env and simply pass the name of your desired model in your metric initialization:

from deepeval.metrics import AnswerRelevancyMetric

answer_relevancy = AnswerRelevancyMetric(
    model="gpt-4.1",
)

You should also set the other necessary vars like PORTKEY_API_KEY to be able to use the Portkey models as shown above.

There are ZERO mandatory and SIX optional parameters when creating a PortkeyModel:

[Optional] model: A string specifying the name of the Portkey model to use. Defaults to PORTKEY_MODEL_NAME if not passed; raises an error at runtime if unset.
[Optional] api_key: A string specifying your Portkey API key for authentication. Defaults to PORTKEY_API_KEY if not passed; raises an error at runtime if unset.
[Optional] provider: A string specifying the Portkey provider of your model. Defaults to PORTKEY_PROVIDER_NAME if not passed; raises an error at runtime if unset.
[Optional] base_url: A string specifying the base URL for the model API. Defaults to PORTKEY_BASE_URL if not passed; raises an error at runtime if unset.
[Optional] temperature: A float specifying the model temperature. Defaults to TEMPERATURE if not passed; falls back to 0.0 if unset.
[Optional] generation_kwargs: A dictionary of additional generation parameters forwarded to Portkey's chat.completions.create(...) call.

Any additional **kwargs you would like to use for your Portkey client can be passed directly to PortkeyModel(...). These are forwarded to the underlying OpenAI client constructor. We recommend double-checking the parameters supported by your chosen model and provider in the official Portkey docs.

Command Line

Python

On this page