Repository navigation
docs: long-format TSDB schema + retention reference - #2
Merged
Merged
Conversation
Document the ts_drivers/ts_metrics/ts_samples schema, the deadlock pattern in RecordSamples (intern Exec inside a tx blocks on the single-conn pool), the hourly Parquet rolloff, the HTTP surface, and the full list of metrics emitted by the bundled Sungrow + Ferroamp drivers. All examples cite file:line in Go and Lua sources.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6b3e74d4fc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| The HTTP API (go/internal/api/api.go:123-124, 721-771) exposes: | ||
|
|
||
| - `GET /api/series?driver=&metric=&range=&points=` -- one metric over a time window. `range` accepts `1h`, `24h`, `7d`, etc.; `points` caps the row count |
There was a problem hiding this comment.
/api/series does not currently support 7d, but this doc says it does. In go/internal/api/api.go (parseRange), only 5m, 15m, 1h, 6h, 24h, and 3d are handled, and any other value falls back to 5 minutes; so users following this reference can silently fetch a much shorter window than intended and make incorrect diagnostics/export decisions.
Useful? React with 👍 / 👎.
| Defined in `go/internal/state/parquet.go`: | ||
|
|
||
| - **Entrypoints**: `RolloffToParquet(ctx, coldDir)` and `LoadSeriesFromParquet(coldDir, driver, metric, since, until)` | ||
| - **File layout**: `<coldDir>/YYYY/MM/DD.parquet`. Default `coldDir` is `cold/` next to `state.db`, configurable via `state.cold_dir` in `config.yaml` (go/internal/config/config.go:148-156) |
There was a problem hiding this comment.
This line implies the default cold storage directory is resolved relative to state.db, but runtime wiring in go/cmd/forty-two-watts/main.go sets independent defaults (statePath="state.db", coldDir="cold") and does not derive coldDir from the configured state DB path. If an operator customizes state.path but omits state.cold_dir, roll-off files will land in ./cold instead of alongside the DB, which is an operational surprise for backups and disk planning.
Useful? React with 👍 / 👎.
3 of 4 tasks
Contributor
|
🎉 This PR is included in version 2.0.1 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
erikarenhill
added a commit
that referenced
this pull request
Apr 16, 2026
fix(ci): switch semantic-release to conventionalcommits preset
erikarenhill
added a commit
that referenced
this pull request
Apr 20, 2026
5-agent vote per comment accepted 6 of 7: - Offload HealthTick observeAt to a goroutine so a slow ntfy Publish (10 s timeout) can't stall the control loop. The event bus runs handlers inline on the publisher, and HealthTick is emitted every control tick. (Copilot #1) - Nil-publisher guard in dispatch(): cold-start or reload with an unknown/unconfigured provider used to panic at s.pub.Publish; now it's treated as a failed dispatch with a history entry. (Copilot #3) - Emit NotificationDispatched on template render failures too, so the history UI surfaces template errors instead of only publish errors. (Copilot #4) - activeAlert leak fix: when no driver_recovered rule is enabled, a post-pass cleanup clears activeAlert for any driver whose telemetry is fresh (<30 s). Preserves the original lifecycle: when recovered IS enabled, the recovered rule still owns the clear. (Copilot #5) - Stop seeding backend default templates into currentConfig. Render title/body template fields as raw inputs whose placeholder is the server default. Blank stays blank, backend fallback keeps working, and future server-side default changes apply without operator action. (Copilot #6) - driver_recovered ignores threshold_s on the backend (hardcoded 30 s staleness window). Don't render the threshold input for that event type — show a clarifying note instead. (Copilot #7) Tests: new TestActiveAlertClearsWhenRecoveredDisabled and TestNilPublisherNoPanic. Full suite still green (35 packages). Skipped Copilot #2 (SQLite write in NotificationDispatched subscriber blocks the publisher): the write runs after HTTP publish on a path that already blocks for up to 10 s, a sub-millisecond SQLite insert adds nothing measurable. 3-of-5 agent votes INVALID. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2 of 3 tasks
erikarenhill
added a commit
that referenced
this pull request
Apr 30, 2026
Five comments, four logical fixes: #1, #2, #3 — config.SafetyMarginA: float64 → *float64 so we can distinguish "unset → use 0.5 default" from "explicit 0 → operator chose to disable margin". Single source of truth via new DefaultFuseSafetyMarginA constant + Fuse.EffectiveSafetyMarginA() helper. main.go startup + hot-reload both call the helper instead of duplicating `<= 0 → 0.5` shortcuts that clobbered the disable case. Validate() updated to dereference; explicit 0 is accepted (and verified by a new round-trip test). #4, #5 — direction attribution in applyFuseGuard + forceFuseDischarge now gates on `currentGrid` (live aggregate at the meter) instead of `predicted` (post-target hypothetical). Per-phase amps and currentGrid come from the same DerMeter sample; predicted mixes in sumTarget which can swing across 0 on a large planner request and silently flip the gate, attributing a current-export overage to the import path (or vice versa). New tests: - TestFuseSafetyMarginNilUsesDefault — back-compat for unset YAML - TestFuseSafetyMarginExplicitZeroDisables — locks the new contract - TestFuseSafetyMarginExplicitValuePassesThrough - TestValidateRejectsNegativeSafetyMargin (parses YAML) - TestValidateRejectsSafetyMarginAtOrAboveMaxAmps - TestValidateAcceptsExplicitZeroSafetyMargin (round-trips through Parse + EffectiveSafetyMarginA to verify 0 stays 0)
frahlg
added a commit
that referenced
this pull request
Jun 5, 2026
Codex returned NO-GO on the parallel build; these close the findings relevant to the shippable relay-hardening (slices 1+2): - #1 CRITICAL: require -home-web when a home host is set (else the relay fell back to forwarding anonymous GETs to the Pi — the exact surface we forbid). requireHomePin now enforces it. - #2 HIGH: device_pubkeys are now covered by the registration signature (MeRegisterSigningStringV2); a captured /me/register can no longer be replayed with a swapped/added device key. + regression test. - #4 MED: symlink-escape in the -home-web static server closed (EvalSymlinks + re-check the resolved path stays under the web root). Deferred to the C2-enforcement phase (inert until the device-key gate is ON), with TODOs: #3 (bind device_pubkey to the enroll ceremony) and #5 (separate per-IP limiter on challenge issuance). All Go unit + 97 web tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
frahlg
added a commit
that referenced
this pull request
Jun 5, 2026
* feat(home-route): inline passkey sign-in on the dashboard The dashboard now reveals a discreet sign-in banner + header key when whoami reports the viewer isn't signed in on a remote origin, and runs the passkey login ceremony in place over the same strict P2P channel (ownerFetch) — no redirect to /owner-access/login.html, which would spawn a fresh channel with no session. Re-checks on P2P (re)connect. LAN bypass + signed-in views never show it. Closes the discoverability gap (part of #438 seamless UX). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(home-route): full sign-in gate + informational transport indicator Evolve the inline-signin banner into a real gate: a logged-out remote visitor now sees ONLY a clean sign-in card, never the empty dashboard chrome (which rendered 'No devices configured' and read as an unconfigured instance). The gate covers the dashboard until whoami confirms a signed-in owner; the passkey ceremony runs in place over the strict P2P channel. Suppresses the no-devices prompt while logged out. The transport indicator is now informational (direct/ relayed/connecting), not a confusing toggle that broke the P2P-only channel. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(home-route): no dashboard flash on reload behind the sign-in gate A reload briefly painted the dashboard before next-app.js could cover it with the gate. Add a pre-paint inline <head> script (mirroring the theme-flash guard) that marks <html class=ftw-gated> on a remote origin from the first frame; CSS shows the opaque full-screen gate immediately, and setupAuth() clears ftw-gated once whoami confirms a signed-in owner. The gate is now a class toggle (not the hidden attribute) so the pre-paint CSS can win. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: hardened-relay + device-key design spec (no anonymous path to a Pi) Spec for the relay-hardening Fredrik approved: relay serves the sign-in shell + /api/identity from its own embedded copy (Pi not touched), and forwards signaling to the Pi ONLY when it carries a device-key proof the relay can verify. The device-key (LAN-born, wallet-bound) becomes the sole credential that can reach a Pi from the internet, and also mints the session silently. Surface-by-surface threat model + build slices + one open trade-off for review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(home-route): hardened relay + device-key remote access (no anonymous path to a Pi) Relay serves the sign-in shell (-home-web) + /api/identity (from -home-pubkey) itself, so anonymous visitors never reach the Pi. Signaling is gated by a device-key proof (relay-issued nonce, signed by a non-extractable WebCrypto ECDSA P-256 key minted at LAN enrollment, verified against the device-pubkeys the Pi publishes on /me/register) — else 403, Pi never woken. The device-key also mints the session silently over the channel (device-PoP); step-up keeps a passkey; revocation drops the key on Pi + relay. Built in parallel across relay/ Pi/browser against locked wire contracts (C1-C4), then integrated. All Go unit tests + 97 web tests pass. Pending Codex security GO before live. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(home-route): address Codex security review of the device-key build Codex returned NO-GO on the parallel build; these close the findings relevant to the shippable relay-hardening (slices 1+2): - #1 CRITICAL: require -home-web when a home host is set (else the relay fell back to forwarding anonymous GETs to the Pi — the exact surface we forbid). requireHomePin now enforces it. - #2 HIGH: device_pubkeys are now covered by the registration signature (MeRegisterSigningStringV2); a captured /me/register can no longer be replayed with a swapped/added device key. + regression test. - #4 MED: symlink-escape in the -home-web static server closed (EvalSymlinks + re-check the resolved path stays under the web root). Deferred to the C2-enforcement phase (inert until the device-key gate is ON), with TODOs: #3 (bind device_pubkey to the enroll ceremony) and #5 (separate per-IP limiter on challenge issuance). All Go unit + 97 web tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(relay): -require-device-key rollout flag (C2 gate dormant by default) Slices 1+2 (relay serves the shell + /api/identity itself, closing the anonymous-FETCH surface) ship independently of the device-key signaling gate: -require-device-key defaults OFF, so signalBrowserOffer parks the raw offer with no proof (pre-C2 behaviour) and a home Pi that doesn't yet publish device-keys keeps working. Flip it on once device-keys are enrolled to close the anonymous-SIGNALING surface. C2 tests now assert the enforced path; a new test asserts the dormant default. All Go unit + web tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docs/tsdb.mddocumenting the long-format time-series storage: SQLite schema, storage estimate, write/read path, Parquet cold tier, thehost.emit_metricdriver API, and the list of metrics actually emitted by the bundled driversstate.Store.RecordSamples(single-conn pool -- internExecinside a tx blocks forever) so future contributors don't trip itfile:linein the Go or Lua sources (store.go, store_ts.go, parquet.go, telemetry/store.go, drivers/host.go, drivers/lua.go, main.go, sungrow.lua, ferroamp.lua)Test plan
cd go && go build ./...cleancd go && go test -timeout 120s -count=1 ./...all greengo test -count=1 -run TestLuaDriverLifecycle ./internal/drivers/passes (validates Lua code block in section 7)host-api.md,lua-drivers.md,configuration.mdexist indocs/)🤖 Generated with Claude Code