Skip to main content
Version: 0.91

Deploying Snorkel-built models to the Databricks Workspace Model Registry and Unity Catalog

Snorkel Flow supports integrations for Databricks Workspace Model Registry and Unity Catalog, which is a unified governance solution for managing data and AI assets in the Databricks platform.

This tutorial demonstrates how to connect Snorkel-built models to the Databricks platform in three steps:

  1. Connect to your organization's Databricks Model Registry.
  2. Push the model object from Snorkel to the Model Registry and optional Unity Catalog.
  3. Deploy the model in Databricks.

Prerequisites​

Ensure you have the CREATE_MODEL privilege for the catalog or schema in Databricks. For more about registry and catalog permissions, see Unity Catalog privileges and securable objects.

Connect to your Databricks Model Registry​

Using the Deploying Snorkel-built models tutorial, bundle the Snorkel workflow for deployment.

Instead of selecting Snorkel's default model registry, choose the Register your own option:

  • **Registry name
    **The name of the project. This value shows up as the Name in the Registered Models Section in Databricks' Model Registry.
  • **Tracking server URL
    **Your organization's base Databricks URL, For example, dbc-8db1117a-9cc7.cloud.databricks.com.
  • **Authentication Method
    **Username/password or authentication token. Snorkel recommends using authentication tokens to comply with security best practices. For more information, see Authentication for Databricks automation - overview.

Push the model object from Snorkel to the Databricks Unity Catalog​

  1. Enter in the relevant fields, and then Validate your credentials and connectivity to your new registry. 
    **
    NOTE
    **To register the asset with the Databricks Unity Catalog, the Deployment Name must be in the form of <catalog>.<schema>.<model>.
  2. If you'd like to track each model as an Experiment in Databricks, ensure the Experiment Name follows Databricks naming convention pattern as an absolute path. For example, /Users/<username>/<experiment_name>.
  3. After configuring the required fields, select Deploy to push the model asset to Databricks Model Registry and Databricks Unity Catalog.

Deploy the model in Databricks​

  1. After successfully pushing the model to Databricks, login to the Databricks portal.
  2. Connected to the Unity Catalog to find your newly pushed model in the Catalog Explorer.
  3. Select Serve this model to deploy the model for testing or production.
  4. Configure the remaining model serving settings for your use case.
  5. After creating a serving endpoint, test the endpoint to confirm a valid model response.

Conclusion​

In this tutorial, you connected to the Databricks Model Registries and the Databricks Unity Catalog, pushed Snorkel-built models to Databricks, and deployed them for production workloads.

If you have any questions about this workflow, contact the Snorkel support team.