Generate Goldens From Scratch
You can also generate synthetic Goldens from scratch, without needing any documents or contexts.
Generate Your Goldens
Since there is no grounded context involved, you'll need to provide a StylingConfig when instantiating a Synthesizer for deepeval's Synthesizer to know what types of goldens it should generate:
from deepeval.synthesizer import Synthesizer
from deepeval.synthesizer.config import StylingConfig
styling_config = StylingConfig(
input_format="Questions in English that asks for data in database.",
expected_output_format="SQL query based on the given input",
task="Answering text-to-SQL-related queries by querying a database and returning the results to users",
scenario="Non-technical users trying to query a database using plain English.",
)
synthesizer = Synthesizer(styling_config=styling_config)from deepeval.synthesizer import Synthesizer
from deepeval.synthesizer.config import ConversationalStylingConfig
conversational_styling_config = ConversationalStylingConfig(
conversational_task="Answering text-to-SQL-related queries by querying a database and returning the results to users",
scenario_context="Non-technical users trying to query a database using plain English.",
participant_roles="Non-technical users trying to query a database using plain English."
)
synthesizer = Synthesizer(conversational_styling_config=conversational_styling_config,)Finally, to generate synthetic goldens without provided context, simply supply the number of goldens you want generated:
from deepeval.synthesizer import Synthesizer
...
goldens = synthesizer.generate_goldens_from_scratch(num_goldens=25)
print(goldens)from deepeval.synthesizer import Synthesizer
...
conversational_goldens = synthesizer.generate_conversational_goldens_from_scratch(num_goldens=25)
print(conversational_goldens)There is ONE mandatory parameter when using the generate_goldens_from_scratch method:
num_goldens: the number of goldens to generate.
FAQs
When should I generate goldens from scratch?
Use it when your application doesn't rely on RAG, or when you want to test queries beyond your existing knowledge base. Supply
num_goldens to set how many to generate.Why is generate_goldens_from_scratch the least grounded method?
Because no documents or contexts are provided, goldens aren't grounded in a knowledge base — making this the least grounded of the four generation methods. Use generate_goldens_from_docs if you need grounded goldens.
How does the Synthesizer know what to generate without context?
You must provide a
StylingConfig (or a ConversationalStylingConfig for multi-turn) defining the task, scenario, and output formats, which steer generation in place of grounded context.Can my team generate goldens from scratch without code?
Yes. On Confident AI you can connect your knowledge bases and run the generation pipeline no-code — tweak filtration, evolutions, and styling, experiment with variations, and collaborate on the resulting dataset as a team.