Skip to content

Stamp per-worker-process execution metadata on test results - #420

Open
bitwise-aiden wants to merge 2 commits into
mainfrom
parallel-worker-metadata
Open

Stamp per-worker-process execution metadata on test results#420
bitwise-aiden wants to merge 2 commits into
mainfrom
parallel-worker-metadata

Conversation

@bitwise-aiden

Copy link
Copy Markdown
Contributor

What

Stamps three per-worker-process execution metadata fields on every minitest result, carried nil-safely through TestData#to_h into log/test_data.json:

  • parallel_worker_pidProcess.pid at result creation.
  • parallel_worker_test_index — 0-based per-process monotonic counter, incremented per execution (requeued runs get their own index). Fork-safe: restarts when the pid changes, so each process incarnation gets a clean 0,1,2,… sequence.
  • parallel_worker_id — identifier injected by the embedding environment (e.g. a Rails parallel-testing worker number) via Minitest::Queue.parallel_worker_id= or CI_QUEUE_PARALLEL_WORKER_ID; nil when not applicable.

This makes per-worker-process execution order reconstructable downstream:

PARTITION BY job_id, parallel_worker_id, parallel_worker_pid
ORDER BY parallel_worker_test_index

i.e. a SQL query can reproduce any worker's test_order-w{N}-{pid}.log from warehouse rows — the prerequisite for warehouse-native test-pollution / requeue-overlap analysis.

How

  • New Minitest::ParallelWorkerMetadata accessors module prepended to Minitest::Result (fields ride the result object through Marshal/DRb).
  • Stamping happens at the top of Minitest::Queue.handle_test_result — in the process that ran the test for all in-process flows.
  • First-writer-wins: embedders that run tests in forked workers and transport results to a central reporting process (e.g. Rails parallelization over DRb, where handle_test_result runs server-side) must call Minitest::Queue.stamp_parallel_worker_metadata(result) in the worker before sending. Pre-stamped results pass through reporting untouched — otherwise the reporting-side stamp would carry the server's pid and an arrival-order index interleaved across workers.

Impact on existing consumers

  • Fields are additive and nil-safe; TestDataReporter is structurally unchanged.
  • rspec-queue, junit.xml, test_order.log, and Redis build-status/error reports are untouched.
  • Downstream Monorail wrappers whitelist payload keys, so the new keys are inert until schema/field lists are updated deliberately.

Testing

  • New unit suite for stamping (setter/env precedence, fork reset, first-writer-wins, nil-safety).
  • TestData unit tests for stamped/unstamped/accessor-less results.
  • Integration assertions in test_test_data_reporter: worker id from env, single pid, indexes exactly 0..N-1, requeued execution ordered before its final run.

Also bumps the version to 0.98.0 for release.

Adds three fields to each recorded result, stamped in the process that
ran the test (worker-side, before any DRb send in embedding
environments):

- parallel_worker_pid: Process.pid at result creation
- parallel_worker_test_index: 0-based per-process monotonic counter,
  incremented per execution (requeued runs get their own index), fork-safe
- parallel_worker_id: injected by the embedding environment via
  Minitest::Queue.parallel_worker_id= or CI_QUEUE_PARALLEL_WORKER_ID;
  nil when not applicable

Stamping is first-writer-wins: embedders that run tests in forked
workers and transport results to a central reporting process (e.g.
Rails parallel testing over DRb, where handle_test_result runs
server-side) must call Minitest::Queue.stamp_parallel_worker_metadata
in the worker before sending; pre-stamped results pass through
reporting untouched. Otherwise the reporting-side stamp would carry
the server's pid and an arrival-order index interleaved across workers.

The fields are carried nil-safely through TestData#to_h into
log/test_data.json (TestDataReporter unchanged), so per-worker-process
execution order is reconstructable downstream:

  PARTITION BY parallel_worker_id, parallel_worker_pid
  ORDER BY parallel_worker_test_index

Assisted-By: devx/77f6d45d-84ba-4c9e-983d-5ec948229a08
Assisted-By: devx/77f6d45d-84ba-4c9e-983d-5ec948229a08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants