Datasets

Choose your language:

Datasets provide scenarios used to evaluate a system.

client.datasets.generate(systemId, options)

Start asynchronous dataset generation for a system.

SDK	client.datasets.generate(system_id, options=None, idempotency_key=None)
Returns	JobRef

Parameters:

Location	Parameter	Type	Required	Default	Description
Path	system_id	string	Required	—	System UUID.
Body	scenario_count	integer	Optional	10	Number of scenarios to generate. Minimum 1, maximum 500.
Body	user_dataset_path	string	Optional	—	Optional path/reference to a user-provided dataset source.
SDK option	idempotency_key	string	Optional	—	Sent as the Idempotency-Key header.

client.datasets.import(...)

Import dataset scenarios directly. Python uses import_ because import is a reserved keyword.

SDK	client.datasets.import_(system_id, scenarios, config=None)
Returns	Dataset

Parameters:

Location	Parameter	Type	Required	Description
Path	system_id	string	Required	System UUID.
Body	scenarios	list[dict]	Required	At least one scenario. Each scenario requires input.
Body	config	dict	Optional	Dataset-level metadata/configuration.

Dataset scenario fields:

Field	Type	Required	Description
input	string or object	yes	Prompt/input sent to the system.
expected_outputs	string[]	no	Expected response hints or labels.
source	string	no	Scenario source.
test_category	string	no	Scenario category.
tested_aspect	string	no	Aspect being tested.

client.datasets.get(systemId)

Read the current dataset for a system.

SDK	client.datasets.get(system_id)
Returns	Dataset | null

Parameters:

Location	Parameter	Type	Required	Description
Path	system_id	string	Required	System UUID.

client.datasets.update(systemId, patch)

Patch the dataset scenarios and/or dataset config.

SDK	client.datasets.update(system_id, patch)
Returns	Dataset

Parameters:

Location	Parameter	Type	Required	Description
Path	system_id	string	Required	System UUID.
Body	scenarios	list[dict]	Optional	Replacement scenario list.
Body	config	dict	Optional	Dataset config patch.

If scenarios is omitted and config is provided, the backend merges config into the existing dataset metadata config.

client.datasets.validate(systemId)

Validate the current dataset for a system.

SDK	client.datasets.validate(system_id)
Returns	Dataset

Parameters:

Location	Parameter	Type	Required	Description
Path	system_id	string	Required	System UUID.