Applications and operators: Build end-to-end AI applications

An application is an end-to-end pipeline of data frame transformations implemented as an operator (see Operators) graph. You can use Application Studio to instantiate operator blocks or iterate on individual operators. Once you’ve finished building your graph, you can package and export the application to make it available for deployment.

This article explains the process for customizing your application.

Application creation

Snorkel recommends creating an application with the guided application creation workflow. Alternatively, you can create an application using templates. This flow is more customizable as described below.

After you create your application, you can customize the application by modifying the Directed Acyclic Graph (DAG) on the Pipeline (DAG) page from the left panel.

Instantiating nodes from application templates

If you created your application based on an application template from the home page, you will see a corresponding block of nodes Application Studio.

Committing operators to nodes

Each node in the operator DAG is a “placeholder” with an expected operator type (indicated by the dotted black outline). You may fill a node by committing a specific operator to the graph. Once an operator is committed, it will be styled accordingly (see the legend in the top-right of the view).

Heuristic operators

To commit a heuristic (or non-model) operator, click on the corresponding node. You’ll see a search box and dropdown listing all operators with a matching operator type. Select the desired operator, and fill out corresponding arguments to commit.

Model operators

Models are learned operators — they need to be trained using a Snorkel-generated training dataset. To commit a model operator, click on the desired node with the dotted outline. This will take you to the Overview page, where you’ll be able to iteratively develop the model by labeling a dataset, training models, and performing error analysis. Once you’re happy with a model that you’ve built in this ML model development loop, commit the corresponding model from the Train page.

Compose complex operator graphs

If you’d like to compose complex operator graphs in Application Studio, you will need to enable an advanced setting.

To do this: click your user name in the bottom left > “User settings” > “Advanced” > Enable “Application Studio advanced mode”. Once this toggle has been enabled, you will be able to build more complex applications using the GUI:

• Add new blocks based on application templates by clicking on the “more options” icon on each block.
• Add featurizers prior to a node.
• Add arbitrary nodes after the current node.

Refreshing stale operators

An operator may become “stale” if you update an upstream operator. Application Studio will detect and indicate stale operators (see the legend in the top right of the view). When an operator is stale, you can refresh its underlying input dataframe by clicking the Refresh icon.

Warning

If you refresh a stale operator, the rows in the dataframe will update in place. You will need to re-tune labeling functions and models correspondingly.

Deploying completed applications

Once you’ve finalized your application and committed an operator to all nodes, you can export the application. This will package your application and make it available for deployment. See Deploying Snorkel-built models for more information about how deployments.

Copying blocks

As part of your iterative process, you may need to modify high-level information about your application, such as specifying new labels or new label columns. To do this, create a copy of your application from the All Applications screen.

Within Application Studio, you can enable DAG editing by toggling the dropdown menu in the top right from READ to EDIT. You should then see a three dot menu in the top right of each block in the DAG.

Clicking on Copy block within that menu will open a dialog which allows you to configure new settings – custom ground truth columns, new labels, and renaming existing labels. Under Advanced options, the checkbox Import LFs/labels/tags will copy over any existing LFs from the existing block. You’ll then be able to interact with the operators within this block as normal.