Skip to content

Add label version history methods (PLT-4661) - #2086

Merged
mnoszczak merged 2 commits into
developfrom
mnoszczak/plt-4661
Oct 19, 2026
Merged

mnoszczak merged 2 commits into
developfrom
mnoszczak/plt-4661

Conversation

@mnoszczak

@mnoszczak mnoszczak commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

PLT-4661

API: Labelbox/intelligence#31624 (the labelVersions / labelVersion GraphQL queries this calls). Merge after that PR is deployed.

Description

Labelbox records a version of a label at each hand-off: a submit or save from the editor, a review approve/reject, a rework submit, a move into rework. Each version has who made the hand-off, when, in which workflow step, and the label's annotations at that moment. Autosaves and drafts record none, and versions are never changed or deleted. Recording is enabled per project (label-version-history), and there are no versions from before it was enabled. Airbnb asked for this to compute IAA/ARA/RAA and audit QA (packet AN-3).

# One label's history, oldest first
for v in label.versions():
    print(v.created_at, v.action, v.created_by_id, v.stage and v.stage.name)
    annotations = v.label_json          # same shape as Label.label

# A project's versions, filtered
versions = project.label_versions(
    data_row=data_row,                  # or label=..., created_by=user
    created_after="2026-10-01T00:00:00Z",
    created_before="2026-11-01T00:00:00Z",
    include_annotations=False,          # metadata only
)

version = client.get_label_version(version_id)   # None if missing or not readable

What's added:

  • Project.label_versions(label=None, data_row=None, created_by=None, created_after=None, created_before=None, include_annotations=True, page_size=100). Filters take IDs or SDK objects. Times take a datetime (naive ones are read as UTC) or an ISO 8601 string. A bad page size or an empty time range raises ValueError before any request.
  • Label.versions(include_annotations=True) and Client.get_label_version(version_id, include_annotations=True).
  • Typed rows:
    • LabelVersion: id, label_id, data_row_id, project_id, action, created_at, created_by_id, stage, annotation_count, label, label_json.
    • LabelVersionStage: task_queue_id, name, type.
    • LabelVersionAction: SUBMITTED, SAVED, APPROVED, REJECTED, REWORK_SUBMITTED, SENT_TO_REWORK. Unknown actions are kept as strings.
  • Cursor paging: the GraphQL query pages by an opaque cursor (≤100 per page). PaginatedCollection follows it as you iterate.
  • Exported from labelbox and listed in the Sphinx docs (label-version.rst). The version and changelog are left to the Prep PR.

Readable by whoever can read the labels:

  • without permission to see other users' labels, only versions of your own labels;
  • in another organization's project without permission to see all labels, only labels your organization made;
  • another organization gets AuthorizationError.

End-to-end evidence

This branch's SDK against the intelligence PR's API, running locally on :8661. The workflow is Labeling → Review 1 → Audit → Done, with rejects going to Rework. The versions come from real editor/review hand-offs (see the API PR):

labelbox 7.15.0 | project cmvapvpo1000gncua5dlv4v28

# project.label_versions() -- all versions, oldest first
  09:03:18.996 SUBMITTED        cmplt4392labeler00000001a @ Initial labeling task label 2fbesg
  09:03:40.907 SAVED            cmplt4392reviewer00000001 @ Review 1 label 2fbesg
  09:04:00.704 REJECTED         cmplt4392reviewer00000001 @ Review 1 label 2fbesg
  09:04:44.002 SAVED            cmplt4392labeler00000001a @ Rework (all rejected) label 2fbesg
  09:05:04.965 REWORK_SUBMITTED cmplt4392labeler00000001a @ Rework (all rejected) label 2fbesg
  09:05:08.917 APPROVED         cmplt4392reviewer00000001 @ Review 1 label 2fbesg
  09:05:17.119 APPROVED         cmplt4392auditor000000001 @ Audit label 2fbesg
  09:06:09.190 SENT_TO_REWORK   clxw4f1io000ty4klcmdgcexr @ Review 1 label w0e4jq
  09:09:56.190 SUBMITTED        cmplt4392labeler00000001a @ Initial labeling task label rjbg6e

# project.label_versions(label=..., page_size=2) -- pages of 2 behind the scenes
  SUBMITTED box [{'top': 60, 'left': 40, 'height': 120, 'width': 200}] quality ['Good']
  SAVED box [{'top': 60, 'left': 55, 'height': 120, 'width': 200}] quality ['Blurry']      <- the reviewer's edit
  REJECTED box [{'top': 60, 'left': 55, 'height': 120, 'width': 200}] quality ['Blurry']
  SAVED box [{'top': 60, 'left': 45, 'height': 120, 'width': 200}] quality ['Good']        <- rework
  REWORK_SUBMITTED box [{'top': 60, 'left': 45, 'height': 120, 'width': 200}] quality ['Good']
  APPROVED box [{'top': 60, 'left': 45, 'height': 120, 'width': 200}] quality ['Good']
  APPROVED box [{'top': 60, 'left': 45, 'height': 120, 'width': 200}] quality ['Good']

# label.versions() from the SDK Label object
   ['SUBMITTED', 'SAVED', 'REJECTED', 'SAVED', 'REWORK_SUBMITTED', 'APPROVED', 'APPROVED']

# filters: created_by (reviewer), created_after/created_before
  rafael: ['SAVED', 'REJECTED', 'APPROVED']
  created_after=2026-10-19T09:04:00.704000+00:00: ['REJECTED', 'SAVED', 'REWORK_SUBMITTED', 'APPROVED', 'APPROVED']
  created_before=2026-01-01: []

# client.get_label_version(id)
  cmvf11ugb0015t8ua8wmq39jl SAVED cmplt4392reviewer00000001 Review 1 Blurry

# denials
  leo (labeler, only his own labels): []
  leo get_label_version of lena's label: None
  outsider (other organization): AuthorizationError - You are unauthorized to perform this action Missing: VIEW_ALL_LABELS, VIEW_OTHER_LABELS, VIEW_OWN_LABELS
  outsider get_label_version: None
  page_size=500: ValueError - page_size must be between 1 and 100

Tests

tests/unit/schema/test_label_version.py: 16 tests. They cover:

  • cursor paging across two pages and parsing, including stages, enums and label_json;
  • that only the filters given are sent, with IDs or SDK objects, and times normalized to UTC;
  • include_annotations=False leaving the content fields out of the query;
  • an unknown action kept as a string, and labelJson sent as text;
  • ValueError for a bad page size, an empty or unreadable time range, and an empty ID;
  • get_label_version returning a version or None;
  • the Project, Label and Client entry points.

Results:

  • pytest tests/unit: 471 passed. The one collection error is test_label_data_type.py, which needs the data extra (numpy) that isn't in my environment.
  • mypy src: clean.
  • ruff check / ruff format --check (ruff 0.9, as rye pins): clean.

CI note (not this change): unit-data fails on Python 3.11–3.13 in tests/data/serialization/ndjson/test_image.py::test_mask* with ValueError: Invalid mode for data type. This run resolved the new pillow==13.0.0; develop's last green run (Oct 16) used 12.3.0. I reproduced the error locally with Pillow 13 on this branch's untouched mask code. unit (474 passed), lint and integration pass on every version. Follow-up: pin pillow<13 or adapt Image.fromarray for Pillow 13.

🤖 Generated with Claude Code

Labelbox now records a version of a label at each hand-off (submit or save
from the editor, review approve/reject, rework submit, move into rework),
readable through the labelVersions / labelVersion GraphQL queries. This adds
them to the SDK:

- Project.label_versions(label=, data_row=, created_by=, created_after=,
  created_before=, include_annotations=, page_size=)
- Label.versions(include_annotations=)
- Client.get_label_version(version_id, include_annotations=)
- Typed LabelVersion / LabelVersionStage and the LabelVersionAction enum,
  exported from labelbox and listed in the Sphinx docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pillow 13 rejects Image.fromarray(bool_array, "L") with "Invalid mode for
data type", which fails tests/data/serialization/ndjson/test_image.py on
Python 3.11+ (they now resolve Pillow 13). Converting the bool mask to uint8
first produces byte-identical PNGs on Pillow 12 and works on 13.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mnoszczak
mnoszczak merged commit bc70439 into develop Oct 19, 2026
24 checks passed
@mnoszczak
mnoszczak deleted the mnoszczak/plt-4661 branch October 19, 2026 12:34
@mnoszczak mnoszczak mentioned this pull request Oct 19, 2026

This branch was previously deployed

1 inactive deployment
Test-PyPI — 588be0c7 Deployed Oct 19, 2026 by mnoszczak via test-pypi #2547
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants