Skip to main content

Snorkel Flow v25.2 (STS) release notes

Breaking changes​

SDK​

  • When a datasource is uploaded via a CSV file, a value: "None" used to stay as "None" but is now recognized as NaN.
  • Some syntaxes were removed in Pandas 2.0 (see here for more details https://pandas.pydata.org/docs/dev/whatsnew/v2.0.0.html). As a result, a user-defined function (e.g., custom operator) would break if any of the removed syntaxes is used. Please reach out to the Snorkel support team if you need any assistance in updating your code.

Deprecations​

SDK​

  • Moved rich_doc_wrapper to snorkelflow/.

Features and improvements​

Documentation​

  • Added PDF versions of the documentation to the homepage of the Snorkel AI docs site.
  • Enabled filtering in search results by guide type: User guide, Admin guide, SDK reference, Release notes, and Glossary. filter-search-results.webp

Data upload​

  • You can upload ground truth for classification label schemas directly to a dataset.

Application setup​

  • When you open the Analysis modal, it displays the previously selected view.
  • Renamed token metrics to word metrics for fixed migration tests.

Annotation​

  • Users can now use multi-schema annotation (MSA) for text candidate extraction applications. They are able to create datasets using the new dataset template and annotate documents across multiple label schemas (that include text candidate extraction) all at once.
  • Implemented golden response request when requesting annotations.
  • Made sidebar resizable to allow for more space for annotating.
  • Added label ordinality for single label schemas in dataviewer.
  • Added support of reviewer mode for text annotations.
  • Reviewers can see multiple free text field annotations with the associated annotator.
  • User can commit a criteria's free text field as ground truth.
  • Label schema panel in annotation is resizable to allow for more space for annotating.

Data development​

  • Added slice count cell to evaluation dashboard.
  • Constrained Model Predictions and GT on word boundaries.

Prompt development​

  • Added ability to view GT annotations from input dataset in prompt workflow.
  • Added empty state for when base version has no data.
  • Added ability to open record view on click of record in table view.
  • Added table view for records.
  • Added streaming of prompt job responses in GUI.
  • Disabled Run Prompt button until user selects a model.
  • Added prompt job progress to user.
  • Added ability to delete a prompt version.
  • Added ability to view multiple data sources in a dataset in prompt workflow.
  • Added rename for prompt versions.
  • Added SME feedback to prompt development.
  • Added option to star prompt version.
  • Added column sorting and reordering to prompt.

Evaluation​

  • Enabled criteria to be reordered in the evaluation dashboard.
  • Updated evaluation dashboard view with the count of datapoints per slice.
  • Reordered slices in the evaluation dashboard.
  • Renamed span metrics to word entity metrics.

SDK​

  • Consolidated [data] extra into [sdk].
  • Added workspace scoped path to sf.download_remote_object.
  • Upgraded Pandas to 2.2.3.

Bug fixes​

Application setup​

  • Prevented duplicate labels in dataset label schema.
  • Added check for existing label state when generating default labels.
  • Added support for having multiple output node ids when creating a node.
  • Set default scheduler value for DAG processing.

Data upload​

  • Fixed split data by % creates the data sources.

Annotation​

  • Fixed annotation dataviewer throwing error of "Must specify at least one label_schema_uid".
  • Fixed label schema ordinality values not showing in case of 0.
  • Added support for text label schemas in GT commit.
  • Removed selection after adding annotation.

Data development​

  • Fixed the logic for % of dataset values in slice-based analysis.
  • Fixed no data found for filter that yields fewer results than current pagination.
  • Fixed duplicate data showing up in table view.
  • PDF: Fixed issues with PDF label map for classification.
  • PDF: Fixed HVLines and other RichDocs columns so that they can be deserialized in Pandas 2.2.3.
  • Text: Fixed issue with sorting based on context uid.

Foundation model development​

  • Updated base behavior for predictors to allow errors in prompting request.
  • Set prefect logging settings to empty to prevent logs being lost.
  • Disabled duplicates for prompt LFs.
  • Fixed broken filters when previewing a prompt LF.

Prompt development​

  • Fixed initial prompt not loading.
  • Fixed SME feedback only showing up for first page.
  • Fixed sme feedback error showing up in prompt development.
  • Fixed comparison view base swap bug.
  • Added handling for comparison view when there's only one prompt version.
  • Fixed prompt response column overlap.
  • Fixed responses endpoint being called before prompt job finish.
  • Fixed prompt duplicate name error coming after prompt creation.
  • Fixed prompt create modal appearing very small in height with first time load.
  • Fixed transparent background of "creating batch" notification.
  • Integrated create batch endpoint.
  • Removed extra space from top of workflow header.

Evaluation​

  • Fixed the parsing for metric plot values.

SDK​

  • Fixed get_preprocessing_issues function.

User interface​

  • Prevented image previews from flashing when clicking generate preview.
  • Updated doc links with correct version.
  • Fixed <ExpandableColumn /> not rendering falsy values.
  • Added useCachedJobPoller to send fresh swr requests for polling.

Infrastructure​

  • Enabled jupyterhub to run in the strict mTLS mode with Istio.

Known issues​

Application setup​

  • Users may encounter an error stating a new application cannot be created because it already exists.
  • An error message is needed when data sources have inconsistent schemas during application creation.
  • PDFv2 application creation from template fails at initialization.
  • Errors occur when creating an application with scanned PDFs in air-gapped environments.
  • The development split is not maintained for PDF IE when copying an application.
  • Errors occur when duplicating chatbot applications.

Data upload​

  • Users experience internal server errors when uploading large CSV data to S3.
  • Uploading large CSV files shows unrelated errors during data upload.
  • New datasources do not have embeddings generated if the feature is not activated.

Annotation​

  • There is a type error in the dataset batch filter.
  • Users are unable to export dataset annotations.
  • Users cannot switch labels from one label schema to another.
  • The reviewer workflow is broken for sequence tagging with overlapping spans.
  • The Annotator Agreement filter doesn't work.
  • Annotation filters with Annotation = negative or GT = negative don't work in PDFv2.
  • Snorkel Flow generated labels are not updated for PDFv2.

Data development​

  • Studio /dataset and /advanced-lf-state endpoints error out with a cryptic error message when there is no span.
  • Drop context_pages and page_docs need to be dropped at /dataset & /context-dataset.
  • The populator saves two different sets of arrow files at the same path when applying.
  • The "Override LF labels with ground truth labels where available" feature doesn't override LF labels with negative GT labels.
  • Server errors occur when creating an LF.
  • Errors with resampled development split show "datapoints not in index."
  • Confusion and clarity matrix numbers don't add up for sequence tagging applications.

Prompt development​

  • Text input for prompts changes the viewport position unexpectedly.

Evaluation​

  • Evaluation data popup does not show any data.
  • In a populated evaluation report, the user who created the report is shown as the requester user rather than the original user from the populated app.

Model development​

  • Model training should error out if any of the vectorizers is not fit.
  • Modeling fails when columns are categorical.
  • Errors occur when deleting custom models.