LF management tab
The Labeling Functions (LF) Management Tab in Snorkel Flow provides an interface to organize and manage labeling functions (LFs), crucial for your Snorkel Flow applications. It allows you to filter, sort, and evaluate LFs, helping you streamline the labeling process and optimize model performance.
By managing LFs effectively, you ensure consistent, high-quality labels, which are vital for improving your model’s accuracy.
Now, let’s explore the key features and functionalities of the LF Management Tab.
As you develop your model in Develop (Studio), the Labeling Functions accordion provides a range of tools to view and manage your Labeling Functions.
There are 4 types of Labeling Functions (LFs) in Snorkel Flow:
- Active: The set of LFs actively used in labeling. When creating a new training set this is the group of LFs that will be packaged for model training.
- Suggested: LFs suggested by the Snorkel Flow based on your data. You can modify the parameters of these suggestions as needed.
- Inactive: LFs that you have archived from the active list.
- In Progress: LFs that are being generated and have not yet been activated.
All LFs are displayed in a table format, with different tables for each type.
The following section explains most commonly used functions in the Labeling Function tables:
Filter LFs
To filter LFs in the table, click on the filter icon above the table. You can filter by LF name, pattern, author, and label. Once applied, that filter appears above the table, and the table itself shows only the LFs that match the filter.
Filter LF is available in the Active and Suggested tables. It is also available in the LF details modal.
When filtering by label for multi-label apps, we will show that label first followed by a count of other labels.
Sort LFs
Users can sort LFs, including multi-label LFs, by clicking on the "Label" header in the LF table.
Multi-label LFs are sorted by the voted labels alphabetically. If an LF votes on more than one label, we will first look at the present labels alphabetically, and then at absent labels alphabetically to determine the order of the multi-label LFs.
View active LF details
Click on the launch icon in the table to open a model where details of each LF can be reviewed in a table format.
Rename LFs
There are multiple locations where you can rename an LF: in the LF table, in the active LF modal, and in the LF composer.
In the LF table, click on an LF (LF edit mode), click on the overflow icon and select Rename this LF in the menu. Keyboard Enter to submit edits.
In the active LF modal, click on the pencil (edit) icon to enter name edit mode. Keyboard Enter to submit edits.
In LF edit mode, go to the LF composer and click on the pencil (edit) icon to enter name edit mode. Keyboard Enter to submit edits.
LF metrics
Single label applications
Both the LF table and the LF composer displays Prec. (GT) and Recall (GT) for each LF by default. Numbers above 90.0% is colored green (considered high); numbers below 70.0% is colored red ewith a warning icon (considered low).
Prec. (GT) and Recall (GT) values may not be numerics in the following conditions:
- No LF coverage: when a LF does not vote any data points in the current split, Prec. (GT) and Recall (GT) cannot be calculated and reads No LF coverage.
- No GT: when none of the data point a LF voted on has any Ground Truth, Prec. (GT) and Recall (GT) cannot be calculated and reads No GT.
Multi-label applications: standard & per-class metrics
In multi-label applications, two types of metrics are available:
- Standard metrics: in a multi-label setting, LF metrics are displayed in the macro-averaged form (i.e. average of per-class metrics over all classes). We only calculate these metrics over points where neither the Ground Truth nor the prediction are
ABSTAINs. - Per-class metrics: when a LF votes on multiple classes, click on the launch icon to see the metrics breakdown per class.
