Gemini

deepeval allows you to directly integrate Gemini models into all available LLM-based metrics, either through the command line or directly within your python code.

Command Line

Run the following command in your terminal to configure your deepeval environment to use Gemini models for all metrics.

deepeval set-gemini \
    --model=<model> # e.g. "gemini-2.5-flash"

Python

Alternatively, you can specify your model directly in code using GeminiModel from deepeval's model collection. By default, model is set to gemini-2.5-pro.

from deepeval.models import GeminiModel
from deepeval.metrics import AnswerRelevancyMetric

model = GeminiModel(
    model="gemini-2.5-pro",
    api_key="Your Gemini API Key",
    temperature=0,
    cost_per_input_token=0.00000125,
    cost_per_output_token=0.00000500
)

answer_relevancy = AnswerRelevancyMetric(model=model)

from deepeval.metrics import AnswerRelevancyMetric

answer_relevancy = AnswerRelevancyMetric(
    model="gemini-2.5-pro",
)

There are ZERO mandatory and SIX optional parameters when creating a GeminiModel:

• [Optional] model: A string specifying the name of the Gemini model to use. Defaults to GEMINI_MODEL_NAME if not passed; raises an error at runtime if unset.
• [Optional] api_key: A string specifying the Google API key for authentication. Defaults to GOOGLE_API_KEY if not passed; raises an error at runtime if unset.
• [Optional] temperature: A float specifying the model temperature. Defaults to TEMPERATURE if not passed; falls back to 0.0 if unset.
• [Optional] cost_per_input_token: A float specifying the cost per input token for the provided model. Defaults to GEMINI_COST_PER_INPUT_TOKEN if set; falls back to deepeval's model cost registry, else None.
• [Optional] cost_per_output_token: A float specifying the cost per output token for the provided model. Defaults to GEMINI_COST_PER_OUTPUT_TOKEN if set; falls back to deepeval's model cost registry, else None.
• [Optional] generation_kwargs: A dictionary of additional generation parameters forwarded to the Gemini API generate_content(...) call.

Parameters may be explicitly passed to the model at initialization time, or configured with optional settings. The mandatory parameters are required at runtime, but you can provide them either explicitly as constructor arguments, or via deepeval settings / environment variables (constructor args take precedence). See Environment variables and settings for the Gemini-related environment variables.

Available Gemini Models

Below is a list of commonly used Gemini models:

gemini-3-pro-preview gemini-3-flash-preview gemini-2.5-pro gemini-2.5-flash gemini-2.5-flash-lite gemini-2.0-flash gemini-2.0-flash-lite gemini-pro-latest gemini-flash-latest gemini-flash-lite-latest