Version: 25.4

snorkelflow.client.ctx.SnorkelFlowContext

class snorkelflow.client.ctx.SnorkelFlowContext(tdm_client, minio_args=None, studio_client=None, storage_client=None, workspace_name=None, set_global=True, debug=False)

Bases: object

The SnorkelFlowContext object provides client context for the Snorkel Flow SDK. It allows the Snorkel Flow SDK to recognize a Snorkel Flow instance by identifying Snorkel Flow’s essential API services (TDM, Studio, MinIO) via user-provided parameters, a YAML config file, or a Snorkel Flow API key.

Examples

import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_kwargs(...)

The keyword arguments are presented in the documentation below. Alternatively, if you have a .snorkel-flow.yaml config file locally, you can use it to create a context:

import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_user_config("path/to/.snorkel-flow.yaml")

__init__(tdm_client, minio_args=None, studio_client=None, storage_client=None, workspace_name=None, set_global=True, debug=False): Initialize a SnorkelFlowContext.

Methods

`__init__`(tdm_client[, minio_args, ...])	Initialize a SnorkelFlowContext.
`from_endpoint_url`([endpoint, ...])	Initialize a SnorkelFlowContext from keyword arguments.
`from_kwargs`([tdm_scheme, tdm_host, ...])	Initialize a SnorkelFlowContext from keyword arguments.
`from_user_config`(cls[, user_config_yaml, ...])	Method to create a SnorkelFlowContest object from a `.snorkel-flow.yaml` file.
`get_global`()	Retrieve the global SnorkelFlowContext object.
`get_minio_client`()	Retrieve an S3 resource for the Minio client.
`set_debug`(debug)	Set the verbosity of warnings, details, and stacktraces
`set_global`([ctx])	Set a SnorkelFlowContext object globally.

Attributes

`storage_client`
`workspace_name`	SnorkelFlowContext objects are only scoped to work in a particular workspace.

from_endpoint_url

classmethod from_endpoint_url(endpoint=None, minio_endpoint=None, api_key=None, minio_access_key=None, minio_secret_key=None, workspace_name=None, set_global=True, debug=False)

Initialize a SnorkelFlowContext from keyword arguments.

Examples

# Instantiate with default kwargs and a custom url
import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_endpoint_url("https://edge.k8s.g498.io/")

# Instantiate with custom kwargs
import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_endpoint_url(
    endpoint="https://edge.k8s.g498.io/",
    minio_endpoint="https://edge-minio-api.k8s.g498.io",
    minio_access_key="minio",
    minio_secret_key="minio123",
    workspace_name="my-workspace",
)

# Instantiate with an API key
import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_endpoint_url(
    endpoint="https://edge.k8s.g498.io/",
    api_key="my-api-key",
)

Parameters Parameters
Returns Returns: A SnorkelFlowContext object that can be used to interact with the MinIO, TDM, and Studio clients directly.
Return type Return type: SnorkelFlowContext

Name	Type	Default	Info
endpoint	`Optional[str]`	`None`	The baseurl to a snorkelflow instance.
minio_endpoint	`Optional[str]`	`None`	The url to a minio service.
minio_access_key	`Optional[str]`	`None`	The access key to use for MinIO. If not provided, look for the access key at an environment variable MINIO_ACCESS_KEY. If this environment variable is not defined, then look for a file identified by an environment variable MINIO_ACCESS_KEY_FILE.
minio_secret_key	`Optional[str]`	`None`	The secret key to use for MinIO. If not provided, look for the secret key at an environment variable MINIO_SECRET_KEY. If this environment variable is not defined, then look for a file identified by an environment variable MINIO_SECRET_KEY_FILE.
workspace_name	`Optional[str]`	`None`	The workspace name, which determines the workspace a dataset and application will be created in and queried from.
api_key	`Optional[str]`	`None`	The API key to use for the TDM, Studio, and Storage APIs.
set_global	`bool`	`True`	Whether to set the context as the global context, accessible across all notebooks in-platform.
debug	`bool`	`False`	Whether to enable debug mode. Debug mode will print more verbose errors to the screen inside of the Python notebook.

from_kwargs

classmethod from_kwargs(tdm_scheme='http', tdm_host='tdm-api', tdm_port=8686, tdm_url_prefix='', minio_scheme='http', minio_host='minio', minio_port=9001, minio_access_key=None, minio_secret_key=None, studio_scheme='http', studio_host='studio-api', studio_port=8484, studio_url_prefix='', storage_scheme='http', storage_host='storage-api', storage_port=31315, storage_url_prefix='', workspace_name=None, api_key=None, set_global=True, debug=False)

Initialize a SnorkelFlowContext from keyword arguments.

Examples

# Instantiate with default kwargs
import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_kwargs()

# Instantiate with custom kwargs
import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_kwargs(
    tdm_port=8080,
    minio_port=9000,
    minio_access_key="minio",
    minio_secret_key="minio123",
    studio_port=8080,
    workspace_name="my-workspace",
)

# Instantiate with an API key
import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_kwargs(
    api_key="my-api-key",
)

Parameters Parameters
Returns Returns: A SnorkelFlowContext object that can be used to interact with the MinIO, TDM, and Studio clients directly.
Return type Return type: SnorkelFlowContext

Name	Type	Default	Info
tdm_scheme	`str`	`'http'`	The scheme to use for the TDM API.
tdm_host	`str`	`'tdm-api'`	The host to use for the TDM API.
tdm_port	`int`	`8686`	The port to use for the TDM API. Contact an administrator for information.
tdm_url_prefix	`str`	`''`	The url prefix to use for the TDM API.
minio_scheme	`str`	`'http'`	The scheme to use for MinIO.
minio_host	`str`	`'minio'`	The host to use for MinIO.
minio_port	`int`	`9001`	The port to use for MinIO. Contact an administrator for information.
minio_access_key	`Optional[str]`	`None`	The access key to use for MinIO. If not provided, look for the access key at an environment variable MINIO_ACCESS_KEY. If this environment variable is not defined, then look for a file identified by an environment variable MINIO_ACCESS_KEY_FILE.
minio_secret_key	`Optional[str]`	`None`	The secret key to use for MinIO. If not provided, look for the secret key at an environment variable MINIO_SECRET_KEY. If this environment variable is not defined, then look for a file identified by an environment variable MINIO_SECRET_KEY_FILE.
studio_scheme	`str`	`'http'`	The scheme to use for the Studio API.
studio_host	`str`	`'studio-api'`	The host to use for the Studio API.
studio_port	`int`	`8484`	The port to use for the Studio API. Contact an administrator for information.
studio_url_prefix	`str`	`''`	The url prefix to use for the Studio API.
storage_scheme	`str`	`'http'`	The scheme to use for the Storage API.
storage_host	`str`	`'storage-api'`	The host to use for the Storage API.
storage_port	`int`	`31315`	The port to use for the Storage API. Contact an administrator for information.
storage_url_prefix	`str`	`''`	The url prefix to use for the Storage API.
workspace_name	`Optional[str]`	`None`	The workspace name, which determines the workspace a dataset and application will be created in and queried from.
api_key	`Optional[str]`	`None`	The API key to use for the TDM and Studio APIs.
set_global	`bool`	`True`	Whether to set the context as the global context, accessible across all notebooks in-platform.
debug	`bool`	`False`	Whether to enable debug mode. Debug mode will print more verbose errors to the screen inside of the Python notebook.

from_user_config

classmethod from_user_config(cls, user_config_yaml=None, protocol='http', set_global=True, debug=False)

Method to create a SnorkelFlowContest object from a .snorkel-flow.yaml file.

Examples

import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_user_config("path/to/your/.snorkel-flow.yaml")

Parameters Parameters
Returns Returns: A SnorkelFlowContext object that can be used to interact with the MinIO, TDM, and Studio clients directly.
Return type Return type: SnorkelFlowContext

Name	Type	Default	Info
user_config_yaml	`Optional[str]`	`None`	Path to the `.snorkel-flow.yaml` file.
protocol	`str`	`'http'`	The scheme to use for TDM, Studio, and MinIO.
set_global	`bool`	`True`	Whether to use this YAML file to determine the context globally.
debug	`bool`	`False`	Whether to enable debug mode. Debug mode will print more verbose errors to the screen inside of the Python notebook.

get_global

classmethod get_global()

Retrieve the global SnorkelFlowContext object.

Examples

import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.get_global()

Returns Returns: A SnorkelFlowContext object that can be used to interact with the MinIO, TDM, and Studio clients directly.
Return type Return type: SnorkelFlowContext
Raises Raises: AttributeError – If no global context has been set.

get_minio_client

get_minio_client()

Retrieve an S3 resource for the Minio client. All methods for the retrieved resource can be found in the Boto3 documentation.

Examples

import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_kwargs()
mc = ctx.get_minio_client()
mc.create_bucket(Bucket="my-bucket")

# Create a new bucket
mc = ctx.get_minio_client()
mc.create_bucket(Bucket="my-bucket")

# List all objects in a bucket
bucket = mc.Bucket("my-bucket")
for obj in bucket.objects.all():
    print(obj)

Return type Return type: S3.ServiceResource
Raises Raises: RuntimeError – If no valid connection to the MinIO client can be established.

set_debug

set_debug(debug)

Set the verbosity of warnings, details, and stacktraces

Parameters Parameters
Return type Return type: None

Name	Type	Default	Info
debug	`bool`		If True, SDK operations will print more verbose errors to the screen inside of the Python notebook.

set_global

classmethod set_global(ctx=None)

Set a SnorkelFlowContext object globally. This context object will be used by all Snorkel Flow SDK functions.

Examples

import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_kwargs(...)
sf.SnorkelFlowContext.set_global(ctx)

Parameters Parameters
Return type Return type: None

Name	Type	Default	Info
ctx	`Optional[SnorkelFlowContext]`	`None`	A context to set as global. If no context is provided, will attempt to create one with default parameters.

property storage_client: StorageClient

property workspace_name: str

SnorkelFlowContext objects are only scoped to work in a particular workspace.

Returns Returns: The name of the workspace belonging to this context object.
Return type Return type: str

Examples​

\_\_init\_\_

__init__​

from\_endpoint\_url

from_endpoint_url​

Examples​

Parameters

Parameters​

Returns

Returns​

Return type

Return type​

from\_kwargs

from_kwargs​

Examples​

Parameters

Parameters​

Returns

Returns​

Return type

Return type​

from\_user\_config

from_user_config​

Examples​

Parameters

Parameters​

Returns

Returns​

Return type

Return type​

get\_global

get_global​

Examples​

Returns

Returns​

Return type

Return type​

Raises

Raises​

get\_minio\_client

get_minio_client​

Examples​

Return type

Return type​

Raises

Raises​

set\_debug

set_debug​

Parameters

Parameters​

Return type

Return type​

set\_global

set_global​

Examples​

Parameters

Parameters​

Return type

Return type​

Returns

Returns​

Return type

Return type​

Examples

init

from_endpoint_url

Examples

Parameters

Returns

Return type

from_kwargs

Examples

Parameters

Returns

Return type

from_user_config

Examples

Parameters

Returns

Return type

get_global

Examples

Returns

Return type

Raises

get_minio_client

Examples

Return type

Raises

set_debug

Parameters

Return type

set_global

Examples

Parameters

Return type

Returns

Return type