LF management tab
As you develop your model in Studio, the Labeling Functions accordion provides a range of tools to view and manage your Labeling Functions.
There are 4 types of Labeling Functions (LFs):
- Active: The set of LFs actively used in labeling. When you opt to create a new training set when training a new model, this is the set of LFs that will be packaged.
- Suggested: A set of LFs suggested by the Snorkel platform. You have the option to modify the suggestion paramter.
- Inactive: All LFs that you opt to archive from the active list.
- In Progress: The LFs that are being generated (to become active LFs).
Labeling Functions of each type is displayed in a table format.
The following section explains most commonly used functions in the Labeling Function tables:
Filter LFs
To filter LFs in the table, click on the filter icon above the table, select your desired filter logic (by LF name, pattern, author, and label), and upon filtering a filter pill will appear along with list of qualified LFs.
Filter LF is available in the Active and Suggested tables. It is also available in the LF details modal.
When filtering by label for multi-label apps, we will show that label first followed by a count of other labels.
Sort LFs
Users can sort LFs, including multi-label LFs, by clicking on the "Label" header in the LF table.
Multi-label LFs are sorted by the voted labels alphabetically. If an LF votes on more than one label, we will first look at the present labels alphabetically, and then at absent labels alphabetically to determine the order of the multi-label LFs.
View active LF details
Click on the launch icon in the table to open a model where details of each LF can be reviewed in a table format.
Rename LFs
There are multiple locations where you can rename an LF: in the LF table, in the active LF modal, and in the LF composer.
In the LF table, click on an LF (LF edit mode), click on the overflow icon and select Rename this LF in the menu. Keyboard Enter to submit edits.
In the active LF modal, click on the pencil (edit) icon to enter name edit mode. Keyboard Enter to submit edits.
In LF edit mode, go to the LF composer and click on the pencil (edit) icon to enter name edit mode. Keyboard Enter to submit edits.
LF metrics
Single label applications
Both the LF table and the LF composer displays Prec. (GT) and Recall (GT) for each LF by default. Numbers above 90.0% is colored green (considered high); numbers below 70.0% is colored red ewith a warning icon (considered low).
Prec. (GT) and Recall (GT) values may not be numerics in the following conditions:
- No LF coverage: when a LF does not vote any data points in the current split, Prec. (GT) and Recall (GT) cannot be calculated and reads No LF coverage.
- No GT: when none of the data point a LF voted on has any Ground Truth, Prec. (GT) and Recall (GT) cannot be calculated and reads No GT.
Multi-label applications: standard & per-class metrics
In multi-label applications, two types of metrics are available:
- Standard metrics: in a multi-label setting, LF metrics are displayed in the macro-averaged form (i.e. average of per-class metrics over all classes). We only calculate these metrics over points where neither the Ground Truth nor the prediction are
ABSTAINs. - Per-class metrics: when a LF votes on multiple classes, click on the launch icon to see the metrics breakdown per class.
