snorkelflow.utils.open_file

warning

This is a beta function in 25.4. Beta features may have known gaps or bugs, but are functional workflows and eligible for Snorkel Support. To access beta features, contact Snorkel Support to enable the feature flag for your Snorkel-hosted instance.

snorkelflow.utils.open_file(path, mode='r', **kwargs)

Opens a file at the specified path and returns the corresponding file object for user files.

Currently supports MinIO URLs in the following format: minio://{bucket}/{key}, local file paths, and S3 URLs.

New MinIO paths paths are resolved to the current workspace by default, ensuring seamless compatibility. Workspace-scoped paths (e.g., minio://workspace-1/file.txt) are also supported if explicitly provided and match the current workspace.

Existing non-workspace-scoped MinIO paths paths will continue to work as expected. However, if a file with the same name is created in a workspace-scoped path, it will take priority moving forward.

Examples

>>> # Open a file in MinIO
>>> f = open_file("minio://my-bucket/my-key")

>>> # Open a file in MinIO outside of the in-platform Notebook
>>> f = open_file(
        "minio://my-bucket/my-key",
        key={MINIO-ACCESS-KEY},
        secret={MINIO-SECRET-KEY},
        client_kwargs={"endpoint_url": {MINIO-URL}}
    )

>>> # Open a local file
>>> f = open_file("./path/to/file")

>>> # Open a file in S3
>>> f = open_file("s3://my-bucket/my-key", key={YOUR-S3-ACCESS-KEY}, secret={YOUR-S3-ACCESS-KEY})

Parameters

Parameters

Name	Type	Default	Info
path	str		The path of the file to be opened.
mode	str	'r'	'w', 'rb', etc.
kwargs	Optional[Dict[str, Any]]		Extra options that make sense to a particular storage connection, e.g. host, port, username, password, etc. These options are passed directly to fsspec.open.

Returns

Returns

A file object opened in the specified mode

Return type

Return type

OpenFile