Skip to content

fix(event_store): defer stream_forward telemetry until stream consumed - #64

Merged
yordis merged 1 commit into
mainfrom
fix/stream-forward-telemetry-read-duration
Apr 1, 2026
Merged

yordis merged 1 commit into
mainfrom
fix/stream-forward-telemetry-read-duration

Conversation

@yordis

@yordis yordis commented Apr 1, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Defer [:commanded, :event_store, :stream_forward, :stop] telemetry until stream enumeration completes for lazy-stream adapters, so duration reflects actual read-from-store latency
  • Adapters returning plain lists emit :stop immediately, preserving backwards compatibility
  • Emit [:commanded, :event_store, :stream_forward, :exception] with kind, reason, and stacktrace when adapter resolution or stream_forward raises, matching :telemetry.span/3 behavior

@cursor

cursor Bot commented Apr 1, 2026 •

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes stream_forward/4 telemetry timing and adds new :exception emission, which can affect observability assumptions and duration metrics for adapters returning lazy streams.

Overview
Updates EventStore.stream_forward/4 telemetry so [:commanded, :event_store, :stream_forward, :stop] is deferred until a lazy stream finishes (or is halted), while preserving immediate :stop for list and {:error, _} results.

Replaces the prior :telemetry.span/3 wrapper with explicit :start/:stop execution, adds a correlated telemetry_span_context, and emits [:commanded, :event_store, :stream_forward, :exception] (with kind, reason, stacktrace) when adapter resolution or the stream_forward call raises.

Extends telemetry tests to cover list vs lazy-stream behaviour, early halt, and exception emission, and updates the shared test handler to also subscribe to :exception events.

Written by Cursor Bugbot for commit 57ed763. This will update automatically on new commits. Configure here.

@coderabbitai

coderabbitai Bot commented Apr 1, 2026 •

Copy link
Copy Markdown

Warning

Rate limit exceeded

@yordis has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 2 minutes and 25 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 2 minutes and 25 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a8b30548-4cd5-421a-9de9-ba79ebe3efc3

📥 Commits

Reviewing files that changed from the base of the PR and between 3703033 and 57ed763.

📒 Files selected for processing (2)
  • lib/commanded/event_store.ex
  • test/event_store/telemetry_test.exs

Walkthrough

EventStore.stream_forward/4 now emits explicit telemetry: it emits a start event with span context and times, resolves the adapter and calls adapter.stream_forward/4, emits stop immediately for eager lists or defers stop until enumeration for lazy streams, and emits exception telemetry (with kind/reason/stacktrace) on errors.

Changes

Cohort / File(s) Summary
EventStore Telemetry Refactor
lib/commanded/event_store.ex
Added explicit telemetry emissions for :start, :stop, and :exception in stream_forward/4; ensures telemetry_span_context is present; emits :stop immediately for eager lists or wraps lazy enumerables to defer :stop until enumeration; re-raises after emitting exception telemetry.
Streaming Telemetry Tests
test/event_store/telemetry_test.exs
Added tests validating start/stop emission timing and metadata for eager and lazy adapter results, partial enumeration behavior, and exception telemetry (validating kind, reason, stacktrace, shared telemetry_span_context, and duration).

Sequence Diagram(s)

sequenceDiagram
    participant Client as Client
    participant ES as EventStore
    participant Adapter as Adapter
    participant Telemetry as Telemetry

    Client->>ES: call stream_forward(...)
    ES->>Telemetry: emit [:commanded,:event_store,:stream_forward,:start] (monotonic/system, telemetry_span_context)
    ES->>Adapter: resolve & call adapter.stream_forward(...)
    alt Adapter returns immediate list
        Adapter-->>ES: list
        ES->>Telemetry: emit [:...:stream_forward,:stop] (duration)
        ES-->>Client: return list
    else Adapter returns lazy enumerable
        Adapter-->>ES: enumerable
        ES-->>Client: return wrapped enumerable (stop deferred)
        Client->>ES: enumerate wrapped stream
        ES->>Telemetry: emit [:...:stream_forward,:stop] when enumeration completes
    end
    opt resolution or adapter raises
        ES->>Telemetry: emit [:...:stream_forward,:exception] (kind, reason, stacktrace, duration, span context)
        ES-->>Client: re-raise exception
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Poem

🐰 I hopped to start with a bright little ping,

Sent span and times on a telemetry string,
I waited to finish when streams take their swing,
Snatched faults in a tumble and gave them a ring,
A rabbit's small trace — soft hops for your logging 🎋

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: deferring stream_forward telemetry until stream consumption completes, which is the primary objective of this pull request.
Description check ✅ Passed The description is directly related to the changeset, detailing the three main behavioral changes: deferred telemetry for lazy streams, immediate emission for lists, and exception telemetry with proper metadata.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/stream-forward-telemetry-read-duration

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Comment thread lib/commanded/event_store.ex

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@lib/commanded/event_store.ex`:
- Around line 308-321: The telemetry block inside the Stream.transform call can
fail to emit any telemetry if the downstream consumer raises (leaving the
OpenTelemetry span orphaned); update the stream-wrapping logic (the
Stream.transform usage in wrap_stream_forward_telemetry or the function that
creates the stream telemetry) so that exceptions during enumeration emit a
[:commanded, :event_store, :stream_forward, :exception] telemetry event with
duration, monotonic_time, and the original span_context, and always ensure the
span_context is included in the telemetry metadata; implement this by wrapping
the stream enumeration with a construct that catches errors (e.g., convert to a
Stream.resource-based wrapper or add a try/rescue around enumeration) to call
:telemetry.execute for :exception before re-raising, and keep the existing :stop
telemetry on normal completion.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 2ce22e2e-15e4-4007-bce9-062201c798e3

📥 Commits

Reviewing files that changed from the base of the PR and between c4d1d97 and 768eb19.

📒 Files selected for processing (4)
  • lib/commanded/event_store.ex
  • test/application/dynamic_applications_test.exs
  • test/commands/dispatch_command_test.exs
  • test/event_store/stream_forward_deferred_telemetry_test.exs

Comment thread lib/commanded/event_store.ex
@yordis
yordis force-pushed the fix/stream-forward-telemetry-read-duration branch from 768eb19 to 44048d2 Compare April 1, 2026 02:13

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/event_store/telemetry_test.exs (1)

107-107: Avoid runtime-generated atoms for telemetry handler IDs.

The handler ID is created via dynamic atom interpolation with System.unique_integer/1. Runtime-generated atoms accumulate in the BEAM atom table and are never garbage collected. Since :telemetry.attach_many/4 accepts any term, use a non-atom identifier instead.

♻️ Suggested change
-      handler = :"stream_forward_ex-#{System.unique_integer([:positive])}"
+      handler = {__MODULE__, :stream_forward_ex, make_ref()}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/event_store/telemetry_test.exs` at line 107, The test currently builds a
runtime atom for the telemetry handler ID via handler =
:"stream_forward_ex-#{System.unique_integer([:positive])}", which leaks atoms;
change the handler to a non-atom term (for example a tuple or string) and pass
that to :telemetry.attach_many/4 instead (e.g., use a tuple like
{:stream_forward_ex, System.unique_integer([:positive])} or a string
interpolation) so the handler id is not an atom while keeping uniqueness for
tests.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@test/event_store/telemetry_test.exs`:
- Line 107: The test currently builds a runtime atom for the telemetry handler
ID via handler = :"stream_forward_ex-#{System.unique_integer([:positive])}",
which leaks atoms; change the handler to a non-atom term (for example a tuple or
string) and pass that to :telemetry.attach_many/4 instead (e.g., use a tuple
like {:stream_forward_ex, System.unique_integer([:positive])} or a string
interpolation) so the handler id is not an atom while keeping uniqueness for
tests.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 0b1c71e8-180d-4792-84fa-1cbfa26c1bb5

📥 Commits

Reviewing files that changed from the base of the PR and between 768eb19 and 44048d2.

📒 Files selected for processing (2)
  • lib/commanded/event_store.ex
  • test/event_store/telemetry_test.exs
🚧 Files skipped from review as they are similar to previous changes (1)
  • lib/commanded/event_store.ex

@yordis
yordis force-pushed the fix/stream-forward-telemetry-read-duration branch from 44048d2 to 94de2c0 Compare April 1, 2026 02:40

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/event_store/telemetry_test.exs (1)

193-197: Minor inconsistency: missing duration >= 0 assertion.

The full enumeration test (line 152) asserts both is_integer(stop_meas.duration) and stop_meas.duration >= 0, but this halted-stream test only asserts the integer check. For consistency and completeness, consider adding the non-negative assertion here as well.

Proposed fix
       assert_receive {:deferred, [:commanded, :event_store, :stream_forward, :stop], stop_meas, _}
       assert is_integer(stop_meas.duration)
+      assert stop_meas.duration >= 0
     end
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/event_store/telemetry_test.exs` around lines 193 - 197, The deferred
stop measurement assertion is missing a non-negative check: after the existing
assert_receive that binds stop_meas (matching {:deferred, [:commanded,
:event_store, :stream_forward, :stop], stop_meas, _}), add an assertion that
stop_meas.duration >= 0 to mirror the full enumeration test; keep the existing
is_integer(stop_meas.duration) check and simply append assert stop_meas.duration
>= 0.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@test/event_store/telemetry_test.exs`:
- Around line 193-197: The deferred stop measurement assertion is missing a
non-negative check: after the existing assert_receive that binds stop_meas
(matching {:deferred, [:commanded, :event_store, :stream_forward, :stop],
stop_meas, _}), add an assertion that stop_meas.duration >= 0 to mirror the full
enumeration test; keep the existing is_integer(stop_meas.duration) check and
simply append assert stop_meas.duration >= 0.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 79aab0c4-fd0b-4714-8d6e-99b4e55c2386

📥 Commits

Reviewing files that changed from the base of the PR and between 44048d2 and 94de2c0.

📒 Files selected for processing (2)
  • lib/commanded/event_store.ex
  • test/event_store/telemetry_test.exs
✅ Files skipped from review due to trivial changes (1)
  • lib/commanded/event_store.ex

@yordis
yordis force-pushed the fix/stream-forward-telemetry-read-duration branch from 94de2c0 to 3703033 Compare April 1, 2026 02:50
Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
@yordis
yordis force-pushed the fix/stream-forward-telemetry-read-duration branch from 3703033 to 57ed763 Compare April 1, 2026 03:11
@yordis
yordis merged commit 588b801 into main Apr 1, 2026
5 checks passed
@yordis
yordis deleted the fix/stream-forward-telemetry-read-duration branch April 1, 2026 03:17
@sht-bot sht-bot mentioned this pull request Apr 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant