Repository navigation
Using external_id with Label container #1941
Copy link
Copy link
Open
Description
Describe the bug
Hi, I am uploading annotations to a project using the SDK. When refencing the image to upload the annotations to this can be done using external_id, row data id or global key.
I am successful in doing this by using the row data id but this adds some complexity to my pipeline I can avoid by using the external_id.
There is a key which can be used for this as mentioned here but I run into a validation error when using this that is unclear to me.
Would appreciate some help in better understanding this, it may be that I am making a mistake somewhere.
Thank you
To reproduce
import labelbox as lb
import labelbox.data.annotation_types as lb_types
from labelbox import MALPredictionImport
import uuid
bbox_annotation_1 = lb_types.ObjectAnnotation(
name="something",
value=lb_types.Rectangle(
start=lb_types.Point(x=1000, y=2000),
end=lb_types.Point(x=1000, y=30)
)
)
annotations = [
bbox_annotation_1,
]
#use the labels container (we can use global or external key instead of data row id)
labels = [
lb_types.Label(
data={
"external_id": "name.jpg"
},
annotations=annotations,
)
]
# Upload labels for one image
upload_job = MALPredictionImport.create_from_objects(
client = client,
project_id = project_id,
name = "mal_job"+str(uuid.uuid4()),
predictions = labels)
upload_job.wait_till_done()
#error check
print(f"Errors: {upload_job.errors}", )
print(f"Status of uploads: {upload_job.statuses}")
Error
ValidationError: 1 validation error for DataRow
Value error, Must set either id or global_key [type=value_error, input_value={'id': None, 'global_key': None}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.10/v/value_error
File <command-510266779839629>, line 2
1 # Upload labels for one image
----> 2 upload_job = MALPredictionImport.create_from_objects(
3 client = client,
4 project_id = project_id,
5 name = "mal_job"+str(uuid.uuid4()),
6 predictions = labels)
8 upload_job.wait_till_done()
10 #error check
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/pydantic/main.py:214, in BaseModel.__init__(self, **data)
212 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
213 __tracebackhide__ = True
--> 214 validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
215 if self is not validated_self:
216 warnings.warn(
217 'A custom validator is returning a value other than `self`.\n'
218 "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
219 'See the `model_validator` docs ([https://docs.pydantic.dev/latest/concepts/validators/#model-validators](https://adb-377525844578231.11.azuredatabricks.net/editor/notebooks/%3Ca%20class=)" target="_blank" rel="noopener noreferrer">[https://docs.pydantic.dev/latest/concepts/validators/#model-validators</a>](https://docs.pydantic.dev/latest/concepts/validators/#model-validators%3C/a%3E)) for more details.',
220 stacklevel=2,
221 )
Additional context
Add any other context about the problem here.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity
vbrodsky commentedon Jan 13, 2025
Please try with global_key instead of external_id. The value can remain the same. i.e. try
Issa-Bqain commentedon Jan 14, 2025
I tried this however it returns an error message stating that it could not find the image which otherwise works using the data row key.
How could it identify an image using the external_id key value through the global_key key? based on my interpretation of the class this should not be possible?
sowndappan5 commentedon Jun 30, 2026
I investigated this issue & found that during NDJSON serialization, the SDK attempts to build a DataRow model requiring either id or global_key. Because the Labelbox import API only supports referencing data rows via id or globalKey (not externalId), providing only external_id throws a generic Pydantic ValidationError.
To address this, I evaluated two approaches:
Demerit: This would require making synchronous API/network requests during serialization to fetch data row IDs. This adds significant overhead and could fail if the external_id is not unique across datasets in the project.
I went ahead with Option 1 to keep the serialization fast, predictable, and avoid unexpected network round-trips.
You can check out the PR here: #2059