A target which finishes processing in a non-success state fails the test rather than being retried #3426

What happens

tests/mock_vws/test_update_target.py::TestTargetName::test_name_valid[Real Vuforia-Short name] failed in CI:

        vws_client.wait_for_target_processed(target_id=target_id)
>       vws_client.update_target(target_id=target_id, name=name)
E       vws.exceptions.vws_exceptions.TargetStatusNotSuccessError

response   = Response(text='{"result_code":"TargetStatusNotSuccess","transaction_id":"2351abe61cc3445d8906c8b751c05192"}',
         url='https://vws.vuforia.com/targets/32cc74d0f5d54296b1ca1e86d26a61ff',
         status_code=403, ...)

https://github.com/VWS-Python/vws-python-mock/actions/runs/31577443258/job/94052660625

The test waits for processing to finish and then updates the target's name. Real Vuforia answered 403 TargetStatusNotSuccess, which means the target's status was not success when the update arrived. wait_for_target_processed returns once the status is no longer processing, so the target had come out of processing in the failed state.

The target_id fixture in tests/conftest.py:67 adds a target with image_file_success_state_low_rating, whose whole purpose is to reach success with a low rating. Real Vuforia rated it badly enough to fail it instead. That is presumably the same underlying behaviour as #3354, where the same class of image gave an unexpected tracking rating.

It passed on a re-run of the same job, so it does not reproduce reliably.

Why it matters

Nothing retries it. pytest-retry is configured with retries = "10", but pytest_set_filtered_exceptions in conftest.py limits retries to TRANSIENT_VWS_EXCEPTIONS, which is (TooManyRequestsError, ServerError, RequestsTimeout) — TargetStatusNotSuccessError is not in it, and should not be, because for most tests that exception is exactly the thing under test.

So a target which real Vuforia happens to fail turns into a red required check on an unrelated pull request, and the only remedy is to notice it is unrelated and re-run the job. Every test using the target_id fixture against Real Vuforia can hit it.

Suggested resolution

The fixture, not the assertion, is where the flake lives: a test which asks for "a target which processes successfully" and receives a failed one has had its precondition broken, not its expectation.

Options, roughly in order of how much they change:

• Have the target_id fixture check the status after wait_for_target_processed and, if it is failed, add a new target and try again a bounded number of times. This keeps the retry where the precondition is established and out of the tests.
• Use an image which real Vuforia rates more comfortably. That trades this flake for less coverage of the low-rating path, and image_file_success_state_low_rating presumably exists deliberately.
• Add a marker so specific tests treat TargetStatusNotSuccessError as transient. This is the least appealing: the same exception is a real assertion elsewhere in the same file.

Worth checking how often this happens before choosing — the first option costs an extra target per occurrence against the request quota.