Skip to main content
Version: 25.9

snorkelai.sdk.develop.ErrorAnalysis

final class snorkelai.sdk.develop.ErrorAnalysis(provenance, error_analysis_run_uid)

Bases: Base

Provides methods for creating, monitoring, and retrieving results from error analysis clustering runs.

The clustering algorithm identifies patterns in LLM evaluation failures and groups similar errors together. This enables systematic analysis of model performance issues and identification of common failure modes.

Read more in the Error Analysis Guide.

Using the ErrorAnalysis class requires the following import:

from snorkelai.sdk.develop import ErrorAnalysis

__init__​

__init__(provenance, error_analysis_run_uid)

Initializes an ErrorAnalysis instance.

Parameters​

NameTypeDefaultInfo
provenanceDict[str, int]Tracking information that uniquely identifies the inputs used to create this cluster analysis.
error_analysis_run_uidintUnique identifier for this specific cluster analysis run.

Methods

__init__(provenance, error_analysis_run_uid)Initializes an ErrorAnalysis instance.
create(prompt_execution_uid, *[, sync])Creates and triggers error analysis clustering job for LLM evaluation results.
delete(error_analysis_run_uid)Deletes this error analysis run and all associated cluster data.
get(error_analysis_run_uid)Retrieves an existing error analysis by its unique run identifier.
get_clusters()Fetches clusters from a completed error analysis.
get_latest(prompt_execution_uid)Gets the most recent error analysis run for a specific prompt execution.
update()Update an error analysis run

Attributes

uidReturn the UID of the error analysis run

create​

classmethod create(prompt_execution_uid, *, sync=False)

Creates and triggers error analysis clustering job for LLM evaluation results.

Parameters​

NameTypeDefaultInfo
prompt_execution_uidintUnique identifier of the prompt execution.
syncboolFalseIf True, method blocks until clustering completes and returns ready ErrorAnalysis. If False, returns immediately with ErrorAnalysis that requires waiting to get clusters.

Returns​

An ErrorAnalysis instance representing the clustering job. If sync = False, get_clusters will fail until analysis is complete. If sync = True, results are immediately available.

Return type​

ErrorAnalysis

Raises​

  • ValueError – If benchmark, prompt execution or criteria don’t exist or are not valid.

  • ValueError – If prompt execution has no evaluation results or insufficient error cases to cluster.

Examples​

Example 1​

Create error analysis asynchronously:

from snorkelai.sdk.develop import ErrorAnalysis
error_analysis = ErrorAnalysis.create(
    prompt_execution_uid=456,
    sync=False
)

Example 2​

Create error analysis synchronously:

from snorkelai.sdk.develop import ErrorAnalysis
error_analysis = ErrorAnalysis.create(
    prompt_execution_uid=456,
    sync=True
)

delete​

classmethod delete(error_analysis_run_uid)

Deletes this error analysis run and all associated cluster data.

Parameters​

NameTypeDefaultInfo
error_analysis_run_uidintThe unique identifier of the error analysis run to delete.

Raises​

ValueError – If the error analysis run has already been deleted or does not exist.

Return type​

None

Example​

from snorkelai.sdk.develop import ErrorAnalysis
ErrorAnalysis.delete(error_analysis_run_uid=42)

get​

classmethod get(error_analysis_run_uid)

Retrieves an existing error analysis by its unique run identifier.

Parameters​

NameTypeDefaultInfo
error_analysis_run_uidintThe unique identifier of the error analysis run to retrieve.

Returns​

The ErrorAnalysis instance for the specified run, ready for status checks and results retrieval.

Return type​

ErrorAnalysis

Raises​

ValueError – If no error analysis run exists with the given ID.

Example​

from snorkelai.sdk.develop import ErrorAnalysis
error_analysis = ErrorAnalysis.get(error_analysis_run_uid=42)

get_clusters​

get_clusters()

Fetches clusters from a completed error analysis.

Returns​

List of cluster objects.

Return type​

List[Cluster]

Raises​

  • RuntimeError – If called before analysis is complete.

  • ValueError – If analysis failed or was deleted.

Example​

from snorkelai.sdk.develop import ErrorAnalysis
analysis = ErrorAnalysis.get(error_analysis_run_uid=42)
clusters = analysis.get_clusters()

get_latest​

classmethod get_latest(prompt_execution_uid)

Gets the most recent error analysis run for a specific prompt execution.

Parameters​

NameTypeDefaultInfo
prompt_execution_uidintThe unique identifier of the prompt execution.

Returns​

The most recent ErrorAnalysis for the prompt execution or None if no error analysis has been run for this prompt execution.

Return type​

ErrorAnalysis or None

Raises​

ValueError – If prompt execution does not exist.

Example​

from snorkelai.sdk.develop import ErrorAnalysis
latest_analysis = ErrorAnalysis.get_latest(
    prompt_execution_uid=456
)

update​

update()

Update an error analysis run

Return type​

None

property uid: int

Return the UID of the error analysis run