Skip to content

.NET: Fix off-thread RunStatus race where GetStatusAsync can return Running after ResumeAsync halts - #5412

Merged
Peter Ibekwe (peibekwe) merged 3 commits into
mainfrom
peibekwe/workflow-test-fix
Apr 23, 2026
Merged

Peter Ibekwe (peibekwe) merged 3 commits into
mainfrom
peibekwe/workflow-test-fix

Conversation

@peibekwe

Copy link
Copy Markdown
Contributor

Motivation and Context

In StreamingRunEventStream.RunLoopAsync, after every _inputWaiter.WaitForInputAsync wake-up, _runStatus was being flipped to Running unconditionally. This can lead stale signal to flip status back to running after consumer already observed halt signal.

Fixes #5411

Description

Only flip RunStatus to Running when there are messages to process.
LockstepRunEventStream is not affected — it runs supersteps synchronously on the consumer thread and has no comparable wake-up race.

This addresses one of the intermittent workflow test failures in the pipeline.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

Copilot AI review requested due to automatic review settings April 21, 2026 21:23
@moonbox3 Evan Mattson (moonbox3) added .NET Usage: [Issues, PRs], Target: .Net workflows Usage: [Issues, PRs], Target: Workflows labels Apr 21, 2026
@github-actions github-actions Bot changed the title Fix off-thread RunStatus race where GetStatusAsync can return Running after ResumeAsync halts .NET: Fix off-thread RunStatus race where GetStatusAsync can return Running after ResumeAsync halts Apr 21, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a race in the off-thread workflow run loop where RunStatus could transiently flip back to Running after a halt had already been observed, and adds a stress regression test to prevent reintroduction.

Changes:

  • Update StreamingRunEventStream.RunLoopAsync to set RunStatus.Running only when there are unprocessed messages to run.
  • Add an off-thread stress regression test that repeatedly runs the Step9 sample to catch the previously intermittent status race.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
dotnet/src/Microsoft.Agents.AI.Workflows/Execution/StreamingRunEventStream.cs Prevents stale input signals from transiently setting RunStatus back to Running when there is no work to process.
dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/SampleSmokeTest.cs Adds a stress regression test targeting the Step9 multi-response resume scenario in off-thread execution.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@peibekwe
Peter Ibekwe (peibekwe) added this pull request to the merge queue Apr 22, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 22, 2026
@peibekwe
Peter Ibekwe (peibekwe) added this pull request to the merge queue Apr 22, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 22, 2026
@peibekwe
Peter Ibekwe (peibekwe) added this pull request to the merge queue Apr 23, 2026
@lokitoth

Copy link
Copy Markdown
Contributor

Ah, guess I missed it, but for information purposes:

I remember looking at this earlier in #2646; when I added tests for this, I noticed that Lockstep was semantically different in how it handled things, so there were some things to be done to get it to a happy state, but may be out of scope of this PR.

Might be worth bringing over the tests though?

I'll bring over tests + any changes that are needed for them separately.

Merged via the queue into main with commit 69adf6d Apr 23, 2026
21 checks passed
@baywet
Vincent Biret (baywet) deleted the peibekwe/workflow-test-fix branch September 17, 2026 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

.NET Usage: [Issues, PRs], Target: .Net workflows Usage: [Issues, PRs], Target: Workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.NET: [Bug]: Workflows - Fix off-thread RunStatus race where GetStatusAsync can return Running after ResumeAsync halts

6 participants