.NET: Add Magentic E2E workflow coverage by Copilot · Pull Request #5833 · microsoft/agent-framework

Copilot · 2026-05-14T01:56:33Z

Motivation and Context

This PR completes the Magentic end-to-end workflow coverage described in MagenticE2E_TestPlan.md and summarized in MagenticE2E_ImplementationReview.md.

The Magentic orchestrator has several user-visible branches that are difficult to validate with isolated unit tests: planning and replanning, human plan review, participant routing, stall/reset handling, progress-ledger retry behavior, final-answer termination, checkpoint/resume, and invalid or erroneous workflow inputs. This change adds fully built workflow tests that exercise those behaviors through MagenticWorkflowBuilder.Build() and the in-process streaming workflow runtime.

It also fixes minor issues discovered while adding the E2E tests:

Empty Magentic teams now fail during workflow build instead of producing a workflow that can fail later during execution.
New top-level messages sent after Magentic termination now surface the orchestrator's terminal-state error as a workflow error, even though the workflow framework accepts queued messages.

Description

This PR adds a Magentic E2E test suite and supporting implementation updates.

Key changes:

Added MagenticOrchestrationTests.cs with 23 end-to-end tests that run fully built Magentic workflows through the streaming workflow runtime.
Covered happy-path completion, participant delegation, multi-round coordination, plan signoff, plan approval/revision flows, multiple revisions, stall-triggered replanning, checkpoint/resume behavior, round/reset limits, progress-ledger retry/reset handling, next-speaker validation, warning/event emission, instruction-message flow, empty-team validation, and post-termination message rejection.
Updated MagenticWorkflowBuilder.Build() to throw InvalidOperationException when no participants have been added.
Updated MagenticOrchestrator.TakeTurnAsync() to reject new messages after the Magentic task context is terminated, matching the existing terminal-state protection on plan-review responses.
Aligned Magentic orchestration behavior with the expected coordination loop: normal participant returns resume progress-ledger coordination instead of replanning.
Ensured stall handling uses the intended StallCount > MaxStallCount semantics and preserves stall context for plan review when replanning after a stall.
Ensured the orchestrator declares the final-answer output protocol used by fully built workflow execution.

Validation performed:

dotnet build tests/Microsoft.Agents.AI.Workflows.UnitTests/Microsoft.Agents.AI.Workflows.UnitTests.csproj -f net10.0
dotnet test --project tests/Microsoft.Agents.AI.Workflows.UnitTests/Microsoft.Agents.AI.Workflows.UnitTests.csproj -f net10.0 --filter-class '*MagenticOrchestrationTests'
Parallel validation completed with no review comments or security alerts.

Contribution Checklist

The code builds clean without any errors or warnings
The PR follows the Contribution Guidelines
All unit tests pass, and I have added new tests where possible
~~[ ] Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.~~

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/96d76349-1ffd-482b-a3ee-ed208778b1bb Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/44a4fd8a-3828-40e5-9435-90381aeffdb8 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/322c9e2d-59bc-42ad-9a1e-f6fd4c866b26 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/b2c60ce7-4d05-4a0d-b05d-d4284f5b7bb3 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/6e8bca46-448d-4f21-a7e9-240179571970 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/1f878ef4-61b0-410a-a8bc-ebf618b3e5de Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/aba19507-7c7e-40dd-850d-d1fabb5dfa65 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/6fe88a80-2e05-40d5-9539-ca7c59b9022b Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/125f6628-6b3b-4c51-9a51-ae84baece6bb Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/21f3b1ae-183e-4fea-99ad-14efc19f084d Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/1b9e74e8-69e1-43f2-8467-c5ba963c2622 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

…outing assertion Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/9b34e409-61b8-4650-ae55-34efad034ed0 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

…t decrement avoids reset Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/9b34e409-61b8-4650-ae55-34efad034ed0 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

…ld reset Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/9b34e409-61b8-4650-ae55-34efad034ed0 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Copilot AI and others added 30 commits May 13, 2026 20:22

Add E2E test plan for Magentic orchestrator

90ec8e4

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/96d76349-1ffd-482b-a3ee-ed208778b1bb Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add MagenticOrchestrationTests.cs scaffold for Magentic E2E tests

68fbead

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/44a4fd8a-3828-40e5-9435-90381aeffdb8 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Fix MagenticOrchestrator output declaration and add first E2E test

b207921

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/322c9e2d-59bc-42ad-9a1e-f6fd4c866b26 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add plan review test and event emission tests

cd7e8c8

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/322c9e2d-59bc-42ad-9a1e-f6fd4c866b26 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add next speaker validation test

00a132a

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/322c9e2d-59bc-42ad-9a1e-f6fd4c866b26 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add Magentic E2E implementation review

8a6a594

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/b2c60ce7-4d05-4a0d-b05d-d4284f5b7bb3 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add PlanSignoff_Disabled_Proceeds_Immediately E2E test

c0bd2a3

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/6e8bca46-448d-4f21-a7e9-240179571970 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add NextSpeaker_Empty_Falls_Back_To_First E2E test

d1a66a8

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/6e8bca46-448d-4f21-a7e9-240179571970 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add Task_Completes_After_Multiple_Rounds E2E test

1b609b2

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/6e8bca46-448d-4f21-a7e9-240179571970 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add PlanReview_Revised_Triggers_Replan E2E test

bbd94f7

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/6e8bca46-448d-4f21-a7e9-240179571970 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add MaxRoundLimit_Terminates_Workflow E2E test

b7fd354

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/6e8bca46-448d-4f21-a7e9-240179571970 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add MaxStallCount_Triggers_Reset E2E test

d52de26

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/6e8bca46-448d-4f21-a7e9-240179571970 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Update MagenticE2E_ImplementationReview.md with full coverage status

4cb3010

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/6e8bca46-448d-4f21-a7e9-240179571970 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Rewrite Magentic E2E implementation review

eb3e0e4

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/1f878ef4-61b0-410a-a8bc-ebf618b3e5de Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add MaxResetLimit_Terminates_Workflow E2E test

4601c1a

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/aba19507-7c7e-40dd-850d-d1fabb5dfa65 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add PlanReview_On_Stall_Replan E2E test

2673054

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/aba19507-7c7e-40dd-850d-d1fabb5dfa65 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add Instruction_Message_Sent_When_Present E2E test

1ff632d

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/aba19507-7c7e-40dd-850d-d1fabb5dfa65 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Update ImplementationReview.md to reflect 14 tests

9a955c3

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/aba19507-7c7e-40dd-850d-d1fabb5dfa65 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Rewrite Magentic E2E implementation review

584d9b1

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/6fe88a80-2e05-40d5-9539-ca7c59b9022b Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add ProgressLedger_Retry_On_Parse_Failure E2E test

375356f

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/125f6628-6b3b-4c51-9a51-ae84baece6bb Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add ProgressLedger_Max_Retries_Triggers_Reset E2E test

600bc91

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/125f6628-6b3b-4c51-9a51-ae84baece6bb Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add Stall_NoProgress_Increments_StallCount E2E test

34b2709

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/125f6628-6b3b-4c51-9a51-ae84baece6bb Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add PlanReview_Multiple_Revisions E2E test

5063540

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/125f6628-6b3b-4c51-9a51-ae84baece6bb Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Update ImplementationReview.md to reflect 18 tests and new coverage

f566d49

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/125f6628-6b3b-4c51-9a51-ae84baece6bb Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Rewrite Magentic E2E implementation review

1c8e16b

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/21f3b1ae-183e-4fea-99ad-14efc19f084d Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Preserve IsStalled on stall-triggered plan review requests

c46ce45

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/1b9e74e8-69e1-43f2-8467-c5ba963c2622 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Rename isStalled parameter to replanAfterStall for clarity

b68cfc0

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/1b9e74e8-69e1-43f2-8467-c5ba963c2622 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add Task_Delegates_To_Correct_Agent E2E test with multi-participant r…

72c79ae

…outing assertion Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/9b34e409-61b8-4650-ae55-34efad034ed0 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add Progress_Made_Decrements_StallCount E2E test verifying stall coun…

e8cbd1c

…t decrement avoids reset Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/9b34e409-61b8-4650-ae55-34efad034ed0 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Add Consecutive_Stalls_Trigger_Reset E2E test for multi-stall thresho…

1e242c2

…ld reset Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/9b34e409-61b8-4650-ae55-34efad034ed0 Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>

Evan Mattson (moonbox3) added orchestration Usage: [Issues, PRs], Target: multi-agent orchestration (high-level patterns) and removed agent orchestration labels Jun 17, 2026

This was referenced Jun 17, 2026

Bump the microsoft group with 22 updates razeone/CloudEngAgent#53

Closed

Bump Microsoft.Agents.AI and 19 others razeone/CloudEngAgent#55

Closed

Bump the microsoft group with 22 updates razeone/CloudEngAgent#58

Closed

dependabot Bot mentioned this pull request Jul 26, 2026

Bump the nuget-all group with 6 updates dotnet-presentations/ai-workshop#547

Merged

This was referenced Aug 26, 2026

Bump the microsoft group with 3 updates JoshuaRowePhantom/Phantom.Workspaces#1361

Closed

Bump Microsoft.Agents.AI and 2 others JoshuaRowePhantom/Phantom.Workspaces#1387

Open

dependabot Bot mentioned this pull request Sep 10, 2026

Bump the microsoft-dotnet group with 18 updates clawdotnet/SharpClawCode#19

Open

Vincent Biret (baywet) deleted the copilot/add-e2e-tests-for-magentic-again branch September 17, 2026 14:36

Navigation Menu

.NET: Add Magentic E2E workflow coverage - #5833

.NET: Add Magentic E2E workflow coverage#5833
Jacob Alber (lokitoth) merged 47 commits into
mainfrom
copilot/add-e2e-tests-for-magentic-again

Copilot AI commented May 14, 2026 •

edited by lokitoth

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

Copilot AI commented May 14, 2026 • edited by lokitoth Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation and Context

Description

Contribution Checklist

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Copilot AI commented May 14, 2026 •

edited by lokitoth

Loading