Using multi-schema annotations

Multi-schema annotations unlock complex workflows by letting you collect annotations across multiple schemas at one time. This feature empowers subject matter experts to work more efficiently.

With multi-schema annotations, datasets become the new home for all of your annotations and ground truth (GT). The GT is stored in a label schema for a dataset, which can be used by all of the  downstream model nodes.

Multi-schema annotations are available for text-only datasets. Multi-schema annotations are not supported for PDF or image datasets.

To upload a multi-schema annotation dataset

1. To create a new dataset, select Datasets > Upload new dataset.
2. Enter the required information for creating your dataset. For more, see Uploading a dataset.
3. Check Enable multi-schema annotations while creating your dataset.
   NOTE: You cannot enable multi-schema annotations later. If you do not opt in to enabling multi-schema annotations when creating your dataset, you must create a new dataset to enable it for that dataset.
4. Select Verify data source(s).
5. Within Datasets > Label Schemas, select + Create new label schema.
6. Enter a name, description, data type, task type, and additional fields for each task type.
   • Classification tasks: Select Text as the Primary text field. Select Single label, Multi-label, or Text label. Text label allows for free text in your labels instead of a defined label or labels for the other options.
   • Extraction tasks: Select Sequence tagging. Enter the label for sequence tagging with a defined primary text field.
7. Select + Add label schema.
8. In Batches,select + Create new batch.
9. Enter the batch name. Select your split and your label schema. Enter your batch numbers and batch sizes. If you want, you can assign users to annotate the batch. Select Create batch.

To annotate multiple schemas

1. In the Batches tab, select Annotate beside the batch you want to annotate.
2. Select the labels that apply to the data point.
3. Select the previous and next arrows to move between data points.
4. Continue annotating each data point until you have completed your annotations. These annotations are applied to the data points across any batch in which they are used.

To review annotations

1. To see the annotations from other annotators, enable Reviewer mode with the toggle to see all of the annotations for a dataset.
   
   You can see the annotations each annotator made.
2. Continue reviewing each data point until you have determined the correct annotations. These annotations are applied to the data points across any batch in which they are used.
3. After all of the annotations are complete for a dataset and ready to be used for the ground truth, select Batches > Commit for the dataset.

To configure your annotation display

1. In the Batches tab, select Annotate beside the batch you want to annotate.
2. If you want to filter the data points to annotate, you can select your filtering options with fields and operators.
3. To change your display settings, select the gear icon. You can change the displayed columns and order, text direction, prioritize unlabeled documents, and set a default multi-label class.

To see annotation progress

In the Overview, you can select your Label schema from the dropdown menu to see the current status and how much each annotator has finished.

You can select filters for Annotator progress to see the progress for specific annotators.

In the Review tab, you can see can see that annotators, annotations, and batches.