fix(self-driving): enable Logs and Endpoints signal sources in setup - #412
andrewm4894 merged 2 commits into
Conversation
Step 4 said `logs` / `alert_state_change` has no responder and never mentioned Endpoints. PostHog emits that pair for firing and auto-disabled Logs alerts, and emits `endpoints` / `endpoint_execution_failed` plus `endpoints` / `endpoint_breakdown_limit_exceeded`. `emit_signal` drops a signal when its source row is not enabled, so those findings never reached the Inbox. Step 4 now enables the three rows when the project uses the matching product. Step 2 gains an `endpoints-get-all` probe, because neither the repo nor the project profile reports that product. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: 697c7c73-18b5-4946-a3e8-e956d023acc3
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Test against a wizard branch:
Add Results will be posted here when complete. |
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: 697c7c73-18b5-4946-a3e8-e956d023acc3
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 52b65f606a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| | Scout gate | **On by default — never create a row.** The server lets scout findings into the inbox with no config row; a `signals_scout` / `cross_source_issue` row only exists to opt OUT. If you find one with `enabled: false` (an earlier opt-out), flip it back on with `inbox-source-configs-partial-update`; with no row, do nothing and record "on by default" | `signals_scout` / `cross_source_issue` — existing disabled row only | | ||
| | Health checks | **Always** — instrumentation issues (missing events, proxy gaps, outdated SDKs) are always actionable and a good thing for the agent to fix | `health_checks` / `health_issue` | | ||
| | Error tracking | **Enable by default**, even with no current signal — teams adopt error tracking sooner or later, and with no errors there are no findings and no cost. Evidence (report, exception autocapture ON, or error issues from the step-2 probe) only raises confidence; its absence is **not** a reason to skip | **All three rows**: `error_tracking` / `issue_created`, `error_tracking` / `issue_reopened`, `error_tracking` / `issue_spiking` — the product UI treats them as one switch | | ||
| | Logs | **Only when the project uses Logs**, per your step-2 checklist. The pair carries alerts that fire and alerts the server auto-disables, so a project with no logs never produces one | `logs` / `alert_state_change` | |
There was a problem hiding this comment.
When Logs ranks among a project's most-used products, this enables the native responder while references/6-scouts.md still treats signals-scout-logs as an eligible specialist and excludes only error tracking and session replay from scout routing. A normal setup can therefore enable both recurring Logs pipelines, contrary to the skill's “one surface, one route” invariant, causing duplicate coverage and unnecessary LLM runs; propagate the new native route to the route map and force the Logs scout off.
Useful? React with 👍 / 👎.
| - **Surveys**: does the code or profile show PostHog surveys in use? | ||
| - **AI/LLM**: are there `$ai_*` events, an LLM SDK, or LLM analytics in the profile? | ||
| - **Logs**: is the PostHog logs product in use (per the profile)? | ||
| - **Logs**: is the PostHog logs product in use (per the profile)? Step 4 needs the answer. |
There was a problem hiding this comment.
On a newly integrated project where Logs was successfully configured, context/skills/integration-v2/report/description.md explicitly requires that outcome in posthog-setup-report.md, but step 1 limits report-derived facts to events, error tracking, and feature flags while this line relies only on the profile. Because a profile 404 is explicitly expected on first-run teams, Logs becomes unknown and step 4 skips the new source row, continuing to drop Logs findings despite the report already proving usage; treat a successful Logs result in the setup report as affirmative evidence before falling back to the profile.
Useful? React with 👍 / 👎.
2 participants
Problem
emit_signalreturns before it creates a signal when the(source_product, source_type)row is missing or disabled. Step 4 told the wizard not to create these rows, so every one of those emissions was dropped.logs/alert_state_changealert_signal_emitter.py— alert firing, or auto-disabled as brokenendpoints/endpoint_execution_failedlogic/execution.pyendpoints/endpoint_breakdown_limit_exceededlogic/strategies.pyOrigin
Changes
inbox-source-configs-partial-updateand an already-enabled row is left alone.endpoints-get-allprobe. Neither the repo norscout-project-profile-getreports the Endpoints product, so the probe is the only evidence step 4 can gate on. Logs already had a profile question; it now says step 4 consumes the answer.evaluation. That type genuinely has no emitter left — the taxonomy keeps the value only so older signals still resolve to a label.Risk
Note
Both new rows are conditional, so a project that uses neither product sees no change. An enabled row with no matching product stays idle and costs nothing, so the failure mode of a false positive is a dead switch in the Inbox settings, not noise.
Verification
npm test— 215 tests pass.npm run build— the packagedself-driving-setup.zipcarries both new rows.PostHog/posthogmaster.products/logs/backend/test/test_alert_signal_emitter.pyasserts thelogs/alert_state_changepair. The PostHog tests were not run here: this sandbox has no monorepo checkout, and nothing in PostHog changes.Agent context
Two adjacent gaps in the same table, verified but deliberately not changed, because they are outside what this report asked for:
llm_analyticsis still described as internal with no user-facing responder.eval_reports/emit_signal.pyemitsllm_analytics/evaluation_report, and the Slack onboarding flow enables that pair by default.analytics/anomaly_investigationhas an emitter inanomaly_investigation/workflow.pyand is missing from step 4 entirely. It is also an onboarding default. Enabling it applies to every project, so it deserves its own decision.Created with PostHog Desktop from this inbox report.
🤖 Generated with Claude Code