💥 BREAKING CHANGE: All metric scores are now HIGHER THE BETTER. Read changelog →
Speech-To-Text (STT)

OpenAI STT

In deepeval, you can use OpenAI to transcribe speech into text when running voice simulations. Transcription is what turns your agent's spoken replies into the Turn.content your multi-turn metrics judge, so its accuracy directly bounds how faithfully those metrics see the conversation.

Setting Up Your API Key

DeepEval autoloads .env.local then .env at import time (process env -> .env.local -> .env).

Recommended (local dev):

# .env.local
OPENAI_API_KEY=<your-openai-api-key>

Alternative (Shell/CI):

export OPENAI_API_KEY=<your-openai-api-key>

Alternative (notebook):

If you're working in a notebook environment (Jupyter or Colab), set your OPENAI_API_KEY in a cell:

%env OPENAI_API_KEY=<your-openai-api-key>

Command Line

To transcribe with OpenAI in every voice simulation, run:

deepeval set-stt openai --model=whisper-1

--model is optional and defaults to OpenAI's own default model.

In Code

Pass an OpenAISTTModel to VoiceConfig to transcribe with a model other than the default, or to pin a language:

from deepeval.models import OpenAISTTModel
from deepeval.voice import VoiceConfig

stt_model = OpenAISTTModel(
    model="gpt-4o-transcribe",
    language="en",
)
voice_config = VoiceConfig(
    stt_model=stt_model,
    ...,
)

There are ZERO mandatory and EIGHT optional parameters when creating an OpenAISTTModel:

  • [Optional] model: A string specifying the name of the transcription model to use. Defaulted to gpt-4o-transcribe.
  • [Optional] api_key: A string specifying your OpenAI API key. Defaults to OPENAI_API_KEY if not passed; raises an error at runtime if neither is set.
  • [Optional] base_url: A string specifying a custom endpoint to reach the OpenAI API through. Defaulted to None.
  • [Optional] language: A string ISO-639-1 code (e.g. "en") pinning the spoken language, which improves accuracy and latency. Pass "auto" to detect the language per utterance instead. Defaulted to None, which leaves the decision to OpenAI.
  • [Optional] cost_per_1m_input_tokens: A float overriding the input token price used for cost accounting. Defaults to the built-in price for the model.
  • [Optional] cost_per_1m_output_tokens: A float overriding the output token price used for cost accounting. Defaults to the built-in price for the model.
  • [Optional] cost_per_minute: A float overriding the per-minute price used for cost accounting. Only applies to models priced by duration, such as whisper-1.
  • [Optional] transcription_kwargs: A dictionary of additional parameters forwarded to OpenAI's audio.transcriptions.create(...) call.

First set your API key:

# .env.local
OPENAI_API_KEY=<your-openai-api-key>
DEEPEVAL_STT_MODEL=whisper-1

OpenAI is the transcription provider when no USE_*_STT flag is set, so USE_OPENAI_STT=1 is only needed to switch back after selecting another provider. deepeval set-stt openai --model whisper-1 writes both for you.

You can also pass the model name in code, which wins over DEEPEVAL_STT_MODEL:

from deepeval.voice import VoiceConfig

voice_config = VoiceConfig(
    stt_model="whisper-1",
    ...,
)

A bare model name is shorthand for OpenAISTTModel(model=...), so you can leave stt_model out entirely and transcription still runs on gpt-4o-transcribe. Pass the model object itself to pin a language.

Available OpenAI Models

  • gpt-4o-transcribe (default)
  • gpt-4o-mini-transcribe
  • whisper-1

The gpt-4o transcription models are billed per token, while whisper-1 is billed per minute of audio.

On this page