Skip to content

Add user group and member organization filters to workspace performance reports - #2081

Merged
manuaero merged 1 commit into
developfrom
feat/performance-workspace-groups-and-member-organizations
Oct 14, 2026
Merged

manuaero merged 1 commit into
developfrom
feat/performance-workspace-groups-and-member-organizations

Conversation

@manuaero

@manuaero manuaero commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Description

The Performance REST API's two workspace reports now take a user group filter and a member organization filter, and its member tables say which organization each member's account belongs to (Labelbox/intelligence#31561, deployed to production). This adds them to the SDK methods from #2078. No ticket.

# Workspace reports for the members of user groups (IDs or UserGroup objects)
client.get_workspace_performance_metric(
    "labels_created", "2026-01-01", "2026-01-31", interval="week",
    user_group_ids=["<USER_GROUP_ID>"])
client.get_workspace_labeler_performance(
    "2026-01-01", "2026-01-31", user_group_ids=["<USER_GROUP_ID>"])

# ... or for the members of other organizations, such as a workforce provider
client.get_workspace_labeler_performance(
    "2026-01-01", "2026-01-31", member_organization_ids=["<ORGANIZATION_ID>"])

# Each row says whose organization the member belongs to
for row in project.get_labeler_performance("2026-01-01", "2026-01-31"):
    print(row.email, row.member_organization_id, row.member_organization_name)

What changes:

  • Two new filters on Client.get_workspace_performance_metric() and Client.get_workspace_labeler_performance(): user_group_ids (up to 25 groups of the caller's organization, at most 5,000 members between them) and member_organization_ids (up to 100). Both take IDs or SDK objects, and an empty list raises ValueError, as the existing filters do. They are appended after the existing keyword arguments, so positional calls are unchanged.
  • New row fields on ProjectLabelerPerformance, ProjectReviewerPerformance and WorkspaceLabelerPerformance: member_organization_id and member_organization_name (masked as org.name.<id> where the caller may not see it). Both default to None, so a reply without them still parses.
  • New series fields on PerformanceMetricSeries: member_organization_ids and user_group_ids, the filters a workspace series was given.
  • A table that changes mid-read is reported. Under a user group filter the API works out who is in the groups for every page, so a member who joins or leaves while the pages are read shifts the rows. Iterating PerformanceRows now raises the new PerformanceTableChangedError (a LabelboxError, exported from labelbox) when a page reports a different total from the first page, instead of returning rows that can miss or repeat one. This applies to every performance table, since a project table is also read one request per page. A join and a leave between the same two pages leave total unchanged and cannot be seen; the PerformanceRows docstring says so. This is the one change in behaviour for existing callers; it goes under Changed in the release notes.
  • Docstrings say what the filters need: the organization-wide permission to view user groups (without it every group adds no one and no error is raised) and permission to view all performance for a narrowed metric. They also say that a group of another organization adds no one, and that more than 5,000 members raises InvalidQueryError.

Checked against production

The branch's methods were run against production with a real API key (counts printed, no names or IDs):

  • A user group with 4,986 members: the workspace table returned 486 rows over 5 pages, every row with a member organization, total matching the rows read; the same group in upper case gave the same total; the metric echoed user_group_ids.
  • The whole workspace table (7,336 rows, 74 pages) read without PerformanceTableChangedError firing.
  • member_organization_ids returned only rows of that organization, and the metric echoed it.
  • Project labeler rows carry member_organization_id.
  • An empty list raised ValueError before any request; 26 groups raised InvalidQueryError naming userGroupIds; an unknown group returned an empty table.

The weekly metric under a group filter came out 5 labels above the table's total. The same gap exists with no filter at all (71 labels on the whole workspace), so it is a property of the Monitor's chart and table and not of this change.

Tests

tests/unit/schema/test_performance.py: the filters are sent under the API's names and only when given; empty lists are refused; UserGroup and other SDK objects are accepted; the echoes and row fields are parsed, including null and a reply that leaves them out; the over-5,000-members 400 is raised as InvalidQueryError naming the parameter; a total that changes between pages raises PerformanceTableChangedError, on a project table and on a group-filtered workspace table. All 100 tests in the file pass, and ruff format, ruff check and mypy are clean on the changed files.

Before opening, the change and the docs were reviewed against the API's code by four independent reviewers, with every finding checked by two more. The 18 upheld findings (wording, an over-strong paging promise, the version note in the guide) are fixed here, and the fixes were checked again.

The user-facing guide is updated in Labelbox/lbx-docs-mintlify#297, to merge once 7.14.0 is on PyPI. Release as 7.14.0 after this merges.

Type of change

  • New feature (non-breaking change which adds functionality)

All Submissions

  • Have you followed the guidelines in our Contributing document?
  • Have you provided a description?
  • Are your changes properly formatted?

New Feature Submissions

  • Does your submission pass tests?
  • Have you added thorough tests for your new feature?
  • Have you commented your code, particularly in hard-to-understand areas?
  • Have you added a Docstring?

Changes to Core Features

  • Have you written new tests for your core changes, as applicable?
  • Have you successfully run tests with your changes locally?

🤖 Generated with Claude Code


Note

Medium Risk
New optional filters are additive, but existing callers iterating large performance tables may now hit PerformanceTableChangedError if totals change between pages—a deliberate behavior change for data consistency.

Overview
Workspace performance APIs gain user_group_ids and member_organization_ids filters on get_workspace_performance_metric and get_workspace_labeler_performance, wired through to the REST query params and echoed on PerformanceMetricSeries.

Labeler/reviewer/workspace performance rows now include member_organization_id and member_organization_name so callers can attribute work to vendor orgs.

Behavior change: paginated PerformanceRows iteration raises the new exported PerformanceTableChangedError when a later page’s total disagrees with the first page, instead of silently duplicating or skipping rows when reports shift mid-read (notably under user-group filters). Docstrings and Sphinx docs are updated accordingly.

Reviewed by Cursor Bugbot for commit 4220c5a. Bugbot is set up for automated code reviews on this repo. Configure here.

…ce reports

Client.get_workspace_performance_metric() and
Client.get_workspace_labeler_performance() take user_group_ids and
member_organization_ids, which the REST API now accepts on both
workspace endpoints. Rows of the three performance tables carry
member_organization_id and member_organization_name, and a workspace
series echoes member_organization_ids and user_group_ids.

Iterating a performance table now raises PerformanceTableChangedError
when a page reports a different total from the first page. Under a user
group filter, group membership is worked out for every page, so a member
joining or leaving mid-read shifts the rows; before, the SDK returned
them without saying so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@manuaero
manuaero requested a review from a team as a code owner October 14, 2026 04:50
@manuaero
manuaero merged commit 94eb7e8 into develop Oct 14, 2026
23 of 25 checks passed
@manuaero
manuaero deleted the feat/performance-workspace-groups-and-member-organizations branch October 14, 2026 05:02
@manuaero manuaero mentioned this pull request Oct 14, 2026

This branch was previously deployed

1 inactive deployment
Test-PyPI — 4220c5ac Deployed Oct 14, 2026 by manuaero via test-pypi #2537
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant