Generate Goldens From Goldens

Q: Does it require context to work?

No. If your existing goldens include context, it grounds generation in that context; otherwise it falls back to generate_goldens_from_scratch using your existing inputs. Note expected_outputs are only generated when the seed goldens contain context.

deepeval enables you to generate synthetic goldens from an existing set of goldens, without requiring any documents or context. This is ideal for quickly expanding or adding more complexity to your evaluation dataset.

Generate Your Goldens

To get started, simply define a Synthesizer object and pass in your list of existing goldens. Note that you can only generate single-turn goldens from existing single-turn ones, and vice versa.

from deepeval.synthesizer import Synthesizer

synthesizer = Synthesizer()
goldens = synthesizer.generate_goldens_from_goldens(
  goldens=goldens,
  max_goldens_per_golden=2,
  include_expected_output=True,
)

There is ONE mandatory and TWO optional parameter when using the generate_goldens_from_goldens method:

goldens: a list of existing Goldens from which the new Goldens will be generated.
[Optional] max_goldens_per_golden: the maximum number of goldens to be generated per golden. Defaulted to 2.
[Optional] include_expected_output: a boolean which when set to True, will additionally generate an expected_output for each synthetic Golden. Defaulted to True.

from deepeval.synthesizer import Synthesizer

synthesizer = Synthesizer()
conversational_goldens = synthesizer.generate_conversational_goldens_from_goldens(
  goldens=goldens,
  max_goldens_per_golden=2,
  include_expected_outcome=True,
)

There is ONE mandatory and TWO optional parameter when using the generate_conversational_goldens_from_goldens method:

goldens: a list of existing Goldens from which the new Goldens will be generated.
[Optional] max_goldens_per_golden: the maximum number of goldens to be generated per golden. Defaulted to 2.
[Optional] include_expected_outcome: a boolean which when set to True, will additionally generate an expected_outcome for each synthetic ConversationalGolden. Defaulted to True.

FAQs

When should I use generate_goldens_from_goldens?

Use it when you already have a set of seed goldens and want to quickly expand or add complexity to your dataset — no documents or contexts required. Control volume with max_goldens_per_golden.

Does it require context to work?

No. If your existing goldens include context, it grounds generation in that context; otherwise it falls back to generate_goldens_from_scratch using your existing inputs. Note expected_outputs are only generated when the seed goldens contain context.

Can I mix single-turn and multi-turn goldens?

No. You can only generate single-turn goldens from single-turn ones, and multi-turn ConversationalGoldens from multi-turn ones.

Can my team generate goldens from our existing goldens without code?

Yes. On Confident AI you can connect your knowledge bases and run the generation pipeline no-code — tweak filtration, evolutions, and styling, experiment with variations, and collaborate on the resulting dataset as a team.

Generate Your Goldens

FAQs

On this page