snorkelflow.client.ctx.SnorkelFlowContext

class snorkelflow.client.ctx.SnorkelFlowContext(tdm_client, minio_args=None, studio_client=None, workspace_name='default', set_global=True, debug=False)

Bases: object

The SnorkelFlowContext object provides client context for the Snorkel Flow SDK. It allows the Snorkel Flow SDK to recognize a Snorkel Flow instance by identifying Snorkel Flow’s essential API services (TDM, Studio, MinIO) via user-provided parameters, a YAML config file, or a Snorkel Flow API key.

Examples

import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_kwargs(...)

The keyword arguments are presented in the documentation below. Alternatively, if you have a .snorkel-flow.yaml config file locally, you can use it to create a context:

import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_user_config("path/to/.snorkel-flow.yaml")

\_\_init\_\_

__init__

__init__(tdm_client, minio_args=None, studio_client=None, workspace_name='default', set_global=True, debug=False)

Initialize a SnorkelFlowContext.

Methods

__init__(tdm_client[, minio_args, ...])	Initialize a SnorkelFlowContext.
from_kwargs([tdm_scheme, tdm_host, ...])	Initialize a SnorkelFlowContext from keyword arguments.
from_user_config([user_config_yaml, ...])	Method to create a SnorkelFlowContest object from a .snorkel-flow.yaml file.
get_global()	Retrieve the global SnorkelFlowContext object.
get_minio_client()	Retrieve an S3 resource for the Minio client.
set_debug(debug)	Set the verbosity of warnings, details, and stacktraces
set_global([ctx])	Set a SnorkelFlowContext object globally.

Attributes

workspace_name

SnorkelFlowContext objects are only scoped to work in a particular workspace.

from\_kwargs

from_kwargs

classmethod from_kwargs(tdm_scheme='http', tdm_host='tdm-api', tdm_port=8686, tdm_url_prefix='', minio_scheme='http', minio_host='minio', minio_port=9001, minio_access_key=None, minio_secret_key=None, studio_scheme='http', studio_host='studio-api', studio_port=8484, studio_url_prefix='', workspace_name='default', api_key=None, set_global=True, debug=False)

Initialize a SnorkelFlowContext from keyword arguments.

Examples

# Instantiate with default kwargs
import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_kwargs()

# Instantiate with custom kwargs
import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_kwargs(
    tdm_port=8080,
    minio_port=9000,
    minio_access_key="minio",
    minio_secret_key="minio123",
    studio_port=8080,
    workspace_name="my-workspace",
)

# Instantiate with an API key
import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_kwargs(
    api_key="my-api-key",
)

Parameters

Parameters

Name	Type	Default	Info
tdm_scheme	str	'http'	The scheme to use for the TDM API.
tdm_host	str	'tdm-api'	The host to use for the TDM API.
tdm_port	int	8686	The port to use for the TDM API. Contact an administrator for information.
tdm_url_prefix	str	''	The url prefix to use for the TDM API.
minio_scheme	str	'http'	The scheme to use for MinIO.
minio_host	str	'minio'	The host to use for MinIO.
minio_port	int	9001	The port to use for MinIO. Contact an administrator for information.
minio_access_key	Optional[str]	None	The access key to use for MinIO. If not provided, looks for the access key as an environment variable.
minio_secret_key	Optional[str]	None	The secret key to use for MinIO. If not provided, looks for the access key as an environment variable.
studio_scheme	str	'http'	The scheme to use for the Studio API.
studio_host	str	'studio-api'	The host to use for the Studio API.
studio_port	int	8484	The port to use for the Studio API. Contact an administrator for information.
studio_url_prefix	str	''	The url prefix to use for the Studio API.
workspace_name	str	'default'	The workspace name, which determines the workspace a dataset and application will be created in and queried from.
api_key	Optional[str]	None	The API key to use for the TDM and Studio APIs.
set_global	bool	True	Whether to set the context as the global context, accessible across all notebooks in-platform.
debug	bool	False	Whether to enable debug mode. Debug mode will print more verbose errors to the screen inside of the Python notebook.

Returns

Returns

A SnorkelFlowContext object that can be used to interact with the MinIO, TDM, and Studio clients directly.

Return type

Return type

SnorkelFlowContext

from\_user\_config

from_user_config

classmethod from_user_config(user_config_yaml=None, protocol='http', set_global=True, debug=False)

Method to create a SnorkelFlowContest object from a .snorkel-flow.yaml file.

Examples

import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_user_config("path/to/your/.snorkel-flow.yaml")

Parameters

Parameters

Name	Type	Default	Info
user_config_yaml	Optional[str]	None	Path to the .snorkel-flow.yaml file.
protocol	str	'http'	The scheme to use for TDM, Studio, and MinIO.
set_global	bool	True	Whether to use this YAML file to determine the context globally.
debug	bool	False	Whether to enable debug mode. Debug mode will print more verbose errors to the screen inside of the Python notebook.

Returns

Returns

A SnorkelFlowContext object that can be used to interact with the MinIO, TDM, and Studio clients directly.

Return type

Return type

SnorkelFlowContext

get\_global

get_global

classmethod get_global()

Retrieve the global SnorkelFlowContext object.

Examples

import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.get_global()

Returns

Returns

A SnorkelFlowContext object that can be used to interact with the MinIO, TDM, and Studio clients directly.

Return type

Return type

SnorkelFlowContext

Raises

Raises

AttributeError – If no global context has been set.

get\_minio\_client

get_minio_client

get_minio_client()

Retrieve an S3 resource for the Minio client. All methods for the retrieved resource can be found in the Boto3 documentation.

Examples

import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_kwargs()
mc = ctx.get_minio_client()
mc.create_bucket(Bucket="my-bucket")

# Create a new bucket
mc = ctx.get_minio_client()
mc.create_bucket(Bucket="my-bucket")

# List all objects in a bucket
bucket = mc.Bucket("my-bucket")
for obj in bucket.objects.all():
    print(obj)

Return type

Return type

S3.ServiceResource

Raises

Raises

RuntimeError – If no valid connection to the MinIO client can be established.

set\_debug

set_debug

set_debug(debug)

Set the verbosity of warnings, details, and stacktraces

Parameters

Parameters

Name	Type	Default	Info
debug	bool		If True, SDK operations will print more verbose errors to the screen inside of the Python notebook.

Return type

Return type

None

set\_global

set_global

classmethod set_global(ctx=None)

Set a SnorkelFlowContext object globally. This context object will be used by all Snorkel Flow SDK functions.

Examples

import snorkelflow.client as sf
ctx = sf.SnorkelFlowContext.from_kwargs(...)
sf.SnorkelFlowContext.set_global(ctx)

Parameters

Parameters

Name	Type	Default	Info
ctx	Optional[SnorkelFlowContext]	None	A context to set as global. If no context is provided, will attempt to create one with default parameters.

Return type

Return type

None

property workspace_name: str

SnorkelFlowContext objects are only scoped to work in a particular workspace.

Returns

Returns

The name of the workspace belonging to this context object.

Return type

Return type

str