Wrapping combinators (TaskSeq.map, taskSeq { for .. }) yield the final item twice over an external IAsyncEnumerable on a non-default TaskScheduler (Orleans grain)

Summary

TaskSeq.map and taskSeq { for item in upstream do yield ... } over an externally produced IAsyncEnumerable<T> each yield the final item twice when the consumption runs on a non-default TaskScheduler — observed deterministically inside a Microsoft Orleans grain activation (Orleans' per-activation scheduler). Enumerating the same IAsyncEnumerable<T> directly (the exact await foreach desugaring: GetAsyncEnumerator / MoveNextAsync / Current / DisposeAsync) yields the correct count.

Environment

FSharp.Control.TaskSeq 0.6.0
.NET 10 (net10.0), F#
Reproduces identically on Microsoft Orleans 10.1.0 and 10.2.2 (the enumerable being wrapped is produced by Orleans' IAsyncEnumerableGrainExtension machinery — batched MoveNext pulls over grain calls)

The discriminator (what isolates it to the wrapping construct)

One grain method consumes the same 3-item upstream stream three ways and returns the three counts as an ordinary unary reply, so nothing about our own streaming-reply transport participates in the measurement:

let! direct = TaskSeq.toListAsync (upstream.watch (label, 3))          // plain enumeration of the source

let! viaMap =
    upstream.watch (label2, 3)
    |> TaskSeq.map (fun tick -> tick.note)                             // wrapped
    |> TaskSeq.toListAsync

let! viaFor =
    TaskSeq.toListAsync (taskSeq {                                     // wrapped
        for tick in upstream.watch (label3, 3) do yield tick.note
    })

Result, measured 2026-08-18, stable across runs and across both Orleans versions:

(List.length direct, List.length viaMap, List.length viaFor) = (3, 4, 4)

direct = 3 is correct; both wrapping forms report 4 items for a 3-item stream, the last item duplicated.

What tracing showed

Instrumenting all enumerator layers on the failing path localized it precisely: the taskSeq wrapper yielded one extra MoveNextAsync = true with a stale Current immediately after the inner enumerator had already answered false.

What we ruled out

Our own enumerable implementation: direct enumeration is correct everywhere, including batched sources, empty sources, and a throwing producer.
The obvious structural suspects: four progressively more faithful offline models on the ordinary thread pool — an all-synchronous enumerator, a batched one, one whose terminating MoveNextAsync completes asynchronously, and a full in-process re-implementation of the Orleans pull loop driving two stacked legs — none reproduced it. The trigger appears to require the custom TaskScheduler an Orleans activation runs on.

Repro status (honest)

We could not reduce it to a standalone console repro — that is the main obstacle to a better report, and why this issue points at a live test instead. In our repository it reproduces deterministically:

The discriminator test (asserts direct = 3; deliberately does not pin the divergent value so an upstream fix cannot break our suite): https://github.com/Neftedollar/orleans-fsharp/blob/e31b2e3c5190acb1cbcaadd4078dccd8d67b6b95/tests/Orleans.FSharp.Integration/FunctionalPhaseFIntegrationTests.fs#L598-L627
The in-grain probe that performs the three consumptions: https://github.com/Neftedollar/orleans-fsharp/blob/e31b2e3c5190acb1cbcaadd4078dccd8d67b6b95/tests/Orleans.FSharp.Integration/FunctionalPhaseFFixture.fs#L305-L324

To observe the raw counts: clone the branch, change the discriminator's assertion to print the tuple, and run dotnet test tests/Orleans.FSharp.Integration --filter "FullyQualifiedName~consuming an upstream stream inside a grain".

Happy to run any diagnostic build or instrumented package against this environment if that helps narrow it down.

Addendum — a second, independent manifestation of TaskSeq 0.6.0's missing dynamic path, hit in the same repository today:

.NET SDK 10.0.400's F# compiler sends taskSeq { } bodies down the dynamic resumable-code path that SDK 10.0.201 compiled statically, and every enumeration then throws:

System.NotImplementedException : No dynamic implementation for TaskSeq yet.
  at FSharp.Control.TaskSeqBuilder.Run[T](ResumableCode`2 code)

Observed as a clean A/B: the identical code (a set of taskSeq producers with try/finally, gates, and let!-over-task bodies — e.g. https://github.com/Neftedollar/orleans-fsharp/blob/6d06fe8/tests/Orleans.FSharp.Integration/FunctionalPhaseFFixture.fs) is green on SDK 10.0.201 in both Debug and Release, and fails wholesale on 10.0.400 (GitHub's ubuntu-latest runners picked .400 up around 2026-08-12, which is what first surfaced it — CI floated 10.0.x while local machines were on .201). Our mitigation is pinning the SDK to 10.0.201 (Neftedollar/orleans-fsharp#28), which is obviously temporary.

Two asks, either of which unblocks unpinning:

A dynamic implementation for TaskSeqBuilder.Run — the NotImplementedException branch — so compiler-driven static/dynamic flips stop being fatal;
or guidance on which body shapes are guaranteed static-compilable across compiler versions, so libraries can write to that contract.

Happy to run candidate builds against the failing matrix; the repro is deterministic per SDK version.

.NET SDK 10.0.400's F# compiler sends taskSeq { } bodies down the dynamic resumable-code path that SDK 10.0.201 compiled statically,

This looks like a compiler regression. Could you repost this issue to dotnet/fsharp?

/repo-assist Look into the bug in the dynamic implementation and see if you can fix it, else analyse it, add test cases even if failing (mark them skipped if so).

You may need to update SDK etc.

✓ Repo Assist completed successfully, see workflow run.

Generated by 🌈 Repo Assist, see workflow run. Learn more.

test: add regression tests investigating #452 dynamic-path duplicate-…

Merge pull request #458 from fsprojects/repo-assist/investigate-issue…

Thanks — I reran the real Orleans reproduction against the current released package instead of 0.6.0.

Matrix:

.NET SDK 10.0.400
FSharp.Control.TaskSeq 1.1.1
Orleans 10.1.0 and 10.2.2

Both Orleans versions produce the same tuple:

direct enumeration = 3
TaskSeq.map         = 4
taskSeq { for ... } = 3

So there are now two separate conclusions:

The dynamic-path NotImplementedException is gone with TaskSeq 1.1.1. We can move the repository from SDK 10.0.201 to 10.0.400, so I do not think a new dotnet/fsharp issue is warranted for the current released package.
The original Orleans-specific duplication remains, but is now narrowed to TaskSeq.map. The taskSeq { for x in source do yield x } manifestation is fixed. The standalone scheduler approximations added in [repo-assist] Investigate #452: add regression tests for dynamic-path scheduler bug #458 are useful guards, but they do not cover the real Orleans scheduling/pull path which still returns 4 here.

The integration probe is in FunctionalPhaseFIntegrationTests.fs, with the grain-side enumeration in FunctionalPhaseFFixture.fs. I have also run the exact probe in both supported Orleans matrix legs.

I can extract those two actors into a small standalone Orleans repro project if that would make the remaining TaskSeq.map issue easier to debug.

Important correction to my previous comment: an SDK A/B shows that the apparent taskSeq { for ... } fix is specific to TaskSeq 1.1.1's dynamic path, and that path has two more serious Orleans regressions.

Same package and Orleans code, changing only the SDK/compiler:

SDK	direct	`TaskSeq.map`	`taskSeq { for ... }`	producer disposal/finally	`asTaskSeq` delivery
10.0.201 (static path)	3	4	4	passes	passes
10.0.400 (dynamic path)	3	4	3	fails (20 s timeout)	fails (no item in 30 s)

The two dynamic-path failures reproduce individually, not only in the full suite. With SDK 10.0.201 and the same TaskSeq 1.1.1 package, both pass together in 5 seconds.

So:

1.1.1 removes the NotImplementedException, but the dynamic implementation is not yet safe for this Orleans workload.
On the verified static path, the original issue still affects both TaskSeq.map and taskSeq { for ... }.
We therefore have to keep the repository pinned to SDK 10.0.201 for now.

This changes my earlier conclusion: the SDK/compiler flip is the trigger, but the resulting failures are semantic differences in TaskSeq's dynamic implementation rather than only the old missing-implementation exception. I can extract the Orleans repro if useful.

Standalone reproduction is now public:

https://github.com/Neftedollar/taskseq-orleans-repro

It has no Orleans.FSharp package/project reference. A tiny C# boundary exists only so Orleans' official Roslyn generator can emit proxies/metadata; every TaskSeq producer and consumer is an F# delegate executed inside the real Orleans grain turn.

The public CI runs eight cells: SDK 10.0.201/10.0.400 × Orleans 10.1.0/10.2.2 × Debug/Release:

https://github.com/Neftedollar/taskseq-orleans-repro/actions/runs/32856726674

SDK	Configuration / path	direct	`TaskSeq.map`	`taskSeq { for ... }`	disposal reaches producer `finally`
10.0.201	Debug or Release / static	3	4	4	yes
10.0.400	Debug / dynamic	3	4	3	no
10.0.400	Release / static	3	4	4	yes

The repository's normal tests characterize that table and stay green. Running with -p:AssertCorrect=true instead asserts healthy (3,3,3) plus disposal reaching finally, producing the corresponding failing tests and exact actual values.

One correction to my previous A/B comment: the SDK 10.0.400 disposal regression is specifically the Debug/dynamic path in this minimized repro. Release takes the static path and behaves like 10.0.201. Also, the channel-backed asTaskSeq delivery symptom from the larger suite did not survive minimization, so this repository deliberately does not claim it.

Smoke tests now fail in debug configuration in VS test explorer. So I tried dotnet test with sdk 10.0.400, same failure. Wrong items produced, sequences differ.

🤖 This is an automated response from Repo Assist.

Follow-up on @majocha's report (2026-09-06) that the smoke tests fail in Debug: I've confirmed this and root-caused it to be the same upstream compiler regression already tracked in #473, not a new symptom.

Reproduction: with .NET SDK 10.0.400 and global.json temporarily rolled forward to pick it up, the two existing regression tests in TaskSeq.Issue452.Tests.fs (TaskSeq.map / taskSeq { for .. } over an external IAsyncEnumerable on a custom TaskScheduler) both fail in Debug, returning [] instead of [10; 20; 30] — the sequence is truncated to nothing rather than duplicating the final item. The same tests pass in Release on the identical SDK. Repo-pinned SDK 10.0.111 passes in both configurations.

This matches the dotnet/fsharp#20466 Debug-mode state-machine lowering bug (fix in progress at dotnet/fsharp#20469) documented in #473 and in this repo's new README "Known issues" section: any taskSeq { } with awaits between yields, compiled with SDK 10.0.400+ in Debug, can silently drop items. The Orleans "duplicated final item" symptom reported originally in this issue may be a distinct, real bug in this library's handling of externally-produced enumerators on non-default schedulers — but the newest failure mode reported here (wrong/missing items in Debug on 10.0.400) is the external compiler bug, not something fixable in this repo's source.

No source change is needed for the Debug/SDK-10.0.400 symptom — it will resolve once the upstream fix ships. The original Orleans duplication question (last confirmed reproducing only inside real Orleans grain scheduling, not in the standalone regression tests) remains open and still needs a maintainer or Orleans-environment reproduction to make further progress, as noted in the previous investigation.

Recommend keeping this issue open, but noting in the description/labels that the Debug-mode symptom is external (tracked via #473 / dotnet/fsharp#20466) while the original Orleans duplication remains unconfirmed outside Orleans itself.

Generated by 🌈 Repo Assist, see workflow run. Learn more.
_{Comment /repo-assist to run again}

_{Add this agentic workflow to your repo}

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repo-assist.md@4bc8419fad05e6b032741cbfd189986700bcf71c

Navigation Menu

Repository navigation

Wrapping combinators (TaskSeq.map, taskSeq { for .. }) yield the final item twice over an external IAsyncEnumerable on a non-default TaskScheduler (Orleans grain) #452

Summary

Environment

The discriminator (what isolates it to the wrapping construct)

What tracing showed

What we ruled out

Repro status (honest)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Wrapping combinators (TaskSeq.map, taskSeq { for .. }) yield the final item twice over an external IAsyncEnumerable on a non-default TaskScheduler (Orleans grain) #452

Description

Summary

Environment

The discriminator (what isolates it to the wrapping construct)

What tracing showed

What we ruled out

Repro status (honest)

Activity

Neftedollar commented on Aug 18, 2026

majocha commented on Aug 24, 2026

dsyme commented on Aug 24, 2026

github-actions commented on Aug 24, 2026

github-actions commented on Aug 24, 2026

Neftedollar commented on Aug 25, 2026

Neftedollar commented on Aug 25, 2026

Neftedollar commented on Aug 25, 2026

github-actions commented on Aug 28, 2026

majocha commented on Sep 6, 2026

github-actions commented on Sep 18, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions