Skip to main content
Version: 25.6

Managing files in data storage

User files are the assets related to data points in a given data source. These files can be associated with your data sources.

Permissions​

To upload user files, ensure you have these instance-level permissions:

  • Direct file uploads from the computer that is running the Snorkel AI Data Development Platform.
  • Remote file transfer from a cloud storage provider, such as Amazon S3 or Google Cloud Storage.

An administrator may disable upload permissions for the instance you are working on, which could prevent you from uploading or transferring user files.

To upload user files​

  1. To upload user files, select Data Storage.

  2. Select the Import data button.

    data storage step 1

  3. Choose an existing folder from the dropdown or create a new folder.

    data storage step 2

  4. Choose your data source. The available sources are Amazon S3, Google Cloud Storage, and Local File Upload.

    data storage step 3

  5. Select the file type from the use case dropdown. Supported use cases include uploading images, PDFs, CSVs, parquet, and arrow files.

    data storage step 4

  6. Enter your file paths or select files from local storage. Select Import data to finish the data upload.

    data storage step 5
    • If you select the Remote Storage option, your files are transferred in the background. You can check the status of this transfer using the Jobs icon. - If you select the Local Storage option, your files are uploaded. A loading spinner appears, and the user interface is disabled.
    important

    Do not close the tab. Wait for an alert to confirm the upload finished successfully. It may take several minutes to finish uploading.

Once you have uploaded or transferred files to your Snorkel AI Data Development Platform, you can associate them with your data sources when uploading new datasets.

To download files from a folder​

  1. In the left-hand sidebar, select the file folder from which you want to download.
  2. Select the ... icon next to an individual file to open the file options menu.
  3. Select Download. Download-File-Menu-Option.webp The file downloads to your browser's default download location.