snorkelflow.utils.open_file

snorkelflow.utils.open_file(path, mode='r', **kwargs)

Opens a file at the specified path and returns the corresponding file object.

Currently supports MinIO URLs in the following format: minio://{bucket}/{key}, local file paths, and S3 URLs.

Examples

>>> # Open a file in MinIO
>>> f = open_file("minio://my-bucket/my-key")

>>> # Open a file in MinIO outside of the in-platform Notebook
>>> f = open_file(
        "minio://my-bucket/my-key",
        key={MINIO-ACCESS-KEY},
        secret={MINIO-SECRET-KEY},
        client_kwargs={"endpoint_url": {MINIO-URL}}
    )

>>> # Open a local file
>>> f = open_file("./path/to/file")

>>> # Open a file in S3
>>> f = open_file("s3://my-bucket/my-key", key={YOUR-S3-ACCESS-KEY}, secret={YOUR-S3-ACCESS-KEY})

Parameters

Parameters

Name	Type	Default	Info
path	str		The path of the file to be opened.
mode	str	'r'	'w', 'rb', etc.
kwargs	Optional[Dict[str, Any]]		Extra options that make sense to a particular storage connection, e.g. host, port, username, password, etc. These options are passed directly to fsspec.open.

Returns

Returns

A file object opened in the specified mode

Return type

Return type

OpenFile