Skip to content

fix(telemetry): add peer.service and DB semantic attributes to EventStore spans - #91

Merged
yordis merged 1 commit into
mainfrom
fix-otel-2
Apr 21, 2026
Merged

yordis merged 1 commit into
mainfrom
fix-otel-2

Conversation

@yordis

@yordis yordis commented Apr 20, 2026

Copy link
Copy Markdown
Member

Summary

  • APM backends like Datadog cannot distinguish the EventStore as a separate downstream service because spans use kind: :internal and lack database connection attributes — everything is lumped under the parent BEAM node's service.name
  • Changing span kind to :client and adding db.system, server.address, server.port, db.namespace, and peer.service from the EventStore's runtime connection config enables APM service map discovery
  • Connection attributes are resolved at span start time via EventStore.Config.lookup/1 with a rescue fallback for adapters that don't use it (e.g. InMemory in tests)

@cursor

cursor Bot commented Apr 20, 2026 •

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes OpenTelemetry span kind/attributes for all EventStore operations and adds runtime connection lookup, which can affect APM service maps and span cardinality but is low functional risk to event store behavior.

Overview
Improves EventStore OpenTelemetry spans to be recognized as downstream dependencies. Event store operation spans are now started as kind: :client and include DB semantic attributes (e.g. db.system) plus connection/service identification fields (server.address, server.port, db.namespace, peer.service) derived from the adapter’s runtime config.

Adds helper logic to safely enrich spans from EventStore.Config.lookup/1 (with defensive fallbacks) and updates tests to assert the new span kind and added attributes across event store operations and error/exception paths.

Reviewed by Cursor Bugbot for commit 8796a3f. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai

coderabbitai Bot commented Apr 20, 2026 •

Copy link
Copy Markdown

Warning

Rate limit exceeded

@yordis has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 12 minutes and 27 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 12 minutes and 27 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f45dca78-8a3c-4bc5-a345-045d1cdecff4

📥 Commits

Reviewing files that changed from the base of the PR and between b711c23 and 8796a3f.

📒 Files selected for processing (3)
  • lib/commanded/opentelemetry/event_store.ex
  • lib/commanded/opentelemetry/helpers.ex
  • test/opentelemetry/event_store_test.exs

Walkthrough

The OpenTelemetry instrumentation for EventStore is refactored to derive span metadata through adapter resolution. Previously, destination was computed directly from event metadata; now, the adapter is fetched and queried for connection configuration. The span kind changes from :internal to :client, and database system and connection attributes are conditionally added based on adapter type.

Changes

Cohort / File(s) Summary
Core EventStore Instrumentation
lib/commanded/opentelemetry/event_store.ex
Refactored adapter resolution flow to fetch adapter metadata via fetch_event_store_adapter/1, derive event store name, and look up connection configuration. Span kind changed from :internal to :client. Conditionally adds database system attributes (:postgresql for EventStore, :in_memory for InMemory) and connection-related attributes via helper.
OpenTelemetry Helpers
lib/commanded/opentelemetry/helpers.ex
Added semantic convention aliases (DBAttributes, PeerAttributes, ServerAttributes) and introduced maybe_add_connection_attributes/2 to enrich attribute lists with server address, port, database namespace, and peer service from connection config. Added maybe_add_attr/3 utility for conditional attribute inclusion.
EventStore Telemetry Tests
test/opentelemetry/event_store_test.exs
Updated span assertions across multiple EventStore operations to expect kind: :client instead of :internal and added "db.system": :in_memory attribute expectations to all span assertions for consistency.

Sequence Diagram

sequenceDiagram
    actor Start as Telemetry Event
    participant EST as EventStore Module
    participant AR as Adapter Resolution
    participant Config as Connection Config Lookup
    participant SA as Span Attributes Builder
    participant OT as OpenTelemetry Span

    Start->>EST: telemetry :start event
    EST->>AR: fetch_event_store_adapter(application)
    AR->>AR: resolve adapter & meta
    AR-->>EST: {adapter, adapter_meta}
    EST->>Config: lookup_connection_config(adapter, event_store_name)
    Config-->>EST: connection_config or []
    EST->>SA: build attributes list
    SA->>SA: maybe_add_db_system(adapter_type)
    SA->>SA: maybe_add_connection_attributes(config)
    SA-->>EST: enriched attributes
    EST->>OT: create span(kind: :client, attributes)
    OT-->>Start: span recorded
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Poem

🐰 Adapters now resolve with grace,
Connection config finds its place,
From :internal to :client we stride,
With database attributes as our guide,
The spans dance brighter, refined! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: adding peer.service and DB semantic attributes to EventStore spans, which is the core objective of the pull request.
Description check ✅ Passed The description clearly explains the problem (APM backends cannot distinguish EventStore as a service), the solution (adding semantic attributes and changing span kind to :client), and implementation details (using EventStore.Config.lookup with fallback).

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-otel-2

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a4b7b85. Configure here.

Comment thread lib/commanded/opentelemetry/event_store.ex
Comment thread lib/commanded/opentelemetry/event_store.ex Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (2)
test/opentelemetry/event_store_test.exs (2)

622-624: Nit: same whitespace issue as line 451

Trailing spaces before the closing brace after removing the last comma.

Proposed diff
-               "error.type": "Elixir.RuntimeError"             }
+               "error.type": "Elixir.RuntimeError"
+             }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/opentelemetry/event_store_test.exs` around lines 622 - 624, The map
literal contains trailing whitespace before the closing brace (after the
"error.type": "Elixir.RuntimeError" entry); remove the extra spaces so the
closing brace immediately follows the value (matching the style used at line
451). Locate the map in the test (the block containing
"commanded.expected_version", "erlang.exception.kind", and "error.type") and
delete the trailing spaces before the closing brace to keep consistent
whitespace/formatting.

449-451: Nit: stray whitespace before closing brace

When the last map entry's trailing comma was removed, leftover spaces/indentation were left before the }. Same thing at line 624. A formatter pass will clean this up.

Proposed diff
-               "error.type": "Elixir.MatchError"             }
+               "error.type": "Elixir.MatchError"
+             }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/opentelemetry/event_store_test.exs` around lines 449 - 451, The closing
brace of the map literal that contains keys like "commanded.expected_version",
"erlang.exception.kind" and "error.type" has stray whitespace/indentation before
the `}` (also present at the other map near line 624); remove the extra spaces
so the `}` directly follows the last entry (and run the Elixir formatter);
locate the map in test/opentelemetry/event_store_test.exs (the failing map in
the test assertions) and trim the indentation/whitespace before the closing
brace to match standard formatting.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@lib/commanded/opentelemetry/event_store.ex`:
- Around line 208-219: The db_system_for/1 function currently returns atoms
(:postgresql, :in_memory) which do not match OTel SemConv canonical string
values and use an unsupported "in_memory" token; change db_system_for/1 to
return the canonical string "postgresql" for
Commanded.EventStore.Adapters.EventStore and return nil (or skip setting
db.system) for the InMemory adapter; separately, avoid a hard compile‑time
dependency on EventStore.Config.lookup/1 in lookup_connection_config/2 by doing
a runtime-safe call (e.g., check Code.ensure_loaded?(EventStore.Config) and
function_exported?(EventStore.Config, :lookup, 1) before invoking, otherwise
return []) so you don’t emit Xref/compile warnings while preserving the current
rescue behavior.

In `@lib/commanded/opentelemetry/helpers.ex`:
- Around line 115-123: The helper maybe_add_connection_attributes currently sets
PeerAttributes.peer_service() to config[:database], which is wrong under
OpenTelemetry conventions and duplicates db.namespace; change the call so the
peer attribute is a service identifier (preferably service.peer.name for forward
compatibility) by accepting an extra parameter (e.g., event_store_name) or
letting the caller supply it; update maybe_add_connection_attributes to call
maybe_add_attr(PeerAttributes.peer_service() or the newer service.peer.name
attribute, event_store_name) instead of config[:database], and update the caller
in event_store.ex to pass event_store_name (e.g., inspect(event_store_name) or a
sensible default like "eventstore").
- Around line 115-126: The code calls the non-existent
EventStore.Config.lookup/1 which causes the rescue path to always return [] —
replace that call with EventStore.Config.parsed(event_store_name, :commanded)
(or EventStore.Config.get/2 as appropriate) where the lookup was used, and
update the input validation on maybe_add_connection_attributes to use an
explicit Keyword test (e.g., Keyword.keyword?(config)) instead of the pattern
guard [_ | _] = config so the function only attempts to add attributes when
config is a keyword list; keep the maybe_add_attr helper behavior intact.

---

Nitpick comments:
In `@test/opentelemetry/event_store_test.exs`:
- Around line 622-624: The map literal contains trailing whitespace before the
closing brace (after the "error.type": "Elixir.RuntimeError" entry); remove the
extra spaces so the closing brace immediately follows the value (matching the
style used at line 451). Locate the map in the test (the block containing
"commanded.expected_version", "erlang.exception.kind", and "error.type") and
delete the trailing spaces before the closing brace to keep consistent
whitespace/formatting.
- Around line 449-451: The closing brace of the map literal that contains keys
like "commanded.expected_version", "erlang.exception.kind" and "error.type" has
stray whitespace/indentation before the `}` (also present at the other map near
line 624); remove the extra spaces so the `}` directly follows the last entry
(and run the Elixir formatter); locate the map in
test/opentelemetry/event_store_test.exs (the failing map in the test assertions)
and trim the indentation/whitespace before the closing brace to match standard
formatting.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 58523421-cfde-4832-8f56-a9a7980fdf17

📥 Commits

Reviewing files that changed from the base of the PR and between 9a6ee07 and b711c23.

📒 Files selected for processing (3)
  • lib/commanded/opentelemetry/event_store.ex
  • lib/commanded/opentelemetry/helpers.ex
  • test/opentelemetry/event_store_test.exs

Comment thread lib/commanded/opentelemetry/event_store.ex
Comment thread lib/commanded/opentelemetry/helpers.ex Outdated
Comment thread lib/commanded/opentelemetry/helpers.ex Outdated
Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
@yordis
yordis merged commit d09a91e into main Apr 21, 2026
3 checks passed
@yordis
yordis deleted the fix-otel-2 branch April 21, 2026 00:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant