improved sync, storage tables and compute - #25
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the app’s local-first pipeline by adding durable compute scheduling, richer diagnostics surfaces, expanded decoded-storage coverage, and upgraded charting/UI elements to better reflect “building vs ready” day states.
Changes:
- Adds a durable derive scheduler + periodic history backfill flow, and wires pipeline status into UI/diagnostics.
- Introduces new diagnostics screens and expands persistence/compute tests (including derive preparation coverage).
- Replaces simple sparklines with an interactive time-series chart and updates Today/Journey UI to surface more context.
Reviewed changes
Copilot reviewed 24 out of 25 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| test/local_persistence_test.dart | Expands DB integration tests to cover new diagnostics helpers, decoded substrate tables, and band signals. |
| test/derive_prepare_test.dart | Adds unit tests for prepareDerivationPayload day filtering behavior. |
| test/derivation_pipeline_test.dart | Refactors fixture discovery and adjusts derive-path tests (currently can no-op if fixture missing). |
| pubspec.yaml | Bumps flutter_blue_plus dependency version. |
| pubspec.lock | Updates locked transitive dependency versions/hashes. |
| lib/ui/today/today_screen.dart | Adds status card + replaces HR sparkline with interactive time-series chart and extra HR stats. |
| lib/ui/profile/profile_screen.dart | Adds navigation to new diagnostics screens and formatting updates. |
| lib/ui/kit/charts.dart | Adds TimeSeriesChart widget with interactive tooltip/selection behavior. |
| lib/ui/journey/journey_screen.dart | Switches timeline/movement charts to TimeSeriesChart and reshapes summary UI. |
| lib/ui/debug/metrics_diagnostics_screen.dart | Adds a new metrics-focused diagnostics screen driven by DB + pipeline snapshots. |
| lib/ui/debug/diagnostics_screen.dart | Adds a general diagnostics screen and SQLite export/share flow. |
| lib/state/app_state.dart | Adds derive scheduler integration, periodic backfill timer, battery sample persistence, and pipeline status snapshot. |
| lib/models/payloads.dart | Adds TodayStatus model and exposes TodayData.status. |
| lib/main.dart | Disables FlutterBluePlus logging when supported. |
| lib/data/models.dart | Extends Sample with decoded 1Hz/rr/IMU/spo2/temp fields and related helpers. |
| lib/data/local_repository_impl.dart | Changes Today composition to merge “activity vs overnight” bundles and provide status metadata. |
| lib/compute/substrate.dart | Adds substrate JSON (de)serialization and improves sleep-session search window/history use. |
| lib/compute/onehz_pipeline.dart | Adjusts RMSSD computation and adds NOOP-style nightly HRV helper; currently emits empty baselines. |
| lib/compute/derive_scheduler.dart | Adds durable queued derive scheduler that gates compute behind capture/offload settling. |
| lib/compute/derive_prepare.dart | Adds isolate worker + payload format for preparing day derivation inputs from substrate pages. |
| android/gradle.properties | Adds Flutter migrator flags for Kotlin/DSL toggles. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
1617
to
1621
| activeWorkout = LiveWorkoutState( | ||
| startTime: start, | ||
| targetKcal: targetKcal, | ||
| workoutId: id, | ||
| type: type, |
Comment on lines
1687
to
1688
| activeWorkout = null; | ||
| notifyListeners(); |
Comment on lines
37
to
40
| test('decodeRecTs reads the frame\'s real ts, not the fallback', () { | ||
| final candidates = ['../whoop_hist.jsonl', '../../whoop_hist.jsonl', 'whoop_hist.jsonl']; | ||
| File? f; | ||
| for (final c in candidates) { | ||
| if (File(c).existsSync()) { f = File(c); break; } | ||
| } | ||
| expect(f, isNotNull, reason: 'whoop_hist.jsonl fixture not found'); | ||
| final f = fixtureFile(); | ||
| if (f == null) return; | ||
|
|
Comment on lines
64
to
67
| test('V2 path: decodeSubstrate → segmentation → deriveDayBundle is sane', () { | ||
| // The fixture sits next to the worktree: whoop-master/whoop_hist.jsonl. | ||
| final candidates = [ | ||
| '../whoop_hist.jsonl', | ||
| '../../whoop_hist.jsonl', | ||
| 'whoop_hist.jsonl', | ||
| ]; | ||
| File? f; | ||
| for (final c in candidates) { | ||
| final file = File(c); | ||
| if (file.existsSync()) { | ||
| f = file; | ||
| break; | ||
| } | ||
| } | ||
| expect(f, isNotNull, reason: 'whoop_hist.jsonl fixture not found'); | ||
| final f = fixtureFile(); | ||
| if (f == null) return; | ||
|
|
Comment on lines
+594
to
+598
| @@ -577,35 +595,7 @@ Map<String, dynamic> deriveDayBundle(Map<String, dynamic> inputJson) { | |||
| // layer can consume; the existing readiness/skin_temp_z headlines are untouched. | |||
| // skin_temp is intentionally EXCLUDED — its series is raw ADC, not the °C the | |||
| // skin_temp cfg bounds expect, so feeding it would hard-reject every night. | |||
| // Fold the trailing history ONLY (today is the value being scored, never folded | |||
| // into its own baseline), then report today + its deviation + the cold-start | |||
| // status. Status/nValid reflect the history nights. | |||
| Map<String, dynamic> baselineBlock( | |||
| List<double> history, double? today, MetricCfg cfg) { | |||
| final state = | |||
| Baselines.foldHistory(<double?>[for (final v in history) v], cfg); | |||
| final dev = today == null ? null : Baselines.deviation(today, state); | |||
| return <String, dynamic>{ | |||
| ...state.toJson(), | |||
| 'value': today, | |||
| 'z': dev == null ? null : _round(dev.z, 3), | |||
| 'delta': dev == null ? null : _round(dev.delta, 3), | |||
| 'ratio': dev == null ? null : _round(dev.ratio, 4), | |||
| 'in_normal_range': dev?.inNormalRange, | |||
| }; | |||
| } | |||
|
|
|||
| // hrv baseline: history + today are BOTH the robust nocturnal RMSSD (the | |||
| // `rmssd` series), so the center and the value being scored are the same metric. | |||
| // skin_temp uses the raw-ADC cfg (relative deviation only; no absolute °C). | |||
| final baselines = <String, dynamic>{ | |||
| 'resting_hr': | |||
| baselineBlock(d.rhrHistory, rhrScalar, Baselines.restingHRCfg), | |||
| 'hrv': baselineBlock(d.rmssdHistory, rmssdScalar, Baselines.hrvCfg), | |||
| 'resp': baselineBlock(d.respHistory, respToday, Baselines.respCfg), | |||
| 'skin_temp': | |||
| baselineBlock(d.skinTempAdcHistory, skinTempAdc, _skinTempAdcCfg), | |||
| }; | |||
| final baselines = const <String, dynamic>{}; | |||
Comment on lines
+845
to
+866
| if (startSec <= 0 || endSec <= startSec) return null; | ||
| const windowS = 300; | ||
| final vals = <double>[]; | ||
| var t = startSec; | ||
| while (t < endSec) { | ||
| final bucket = <double>[]; | ||
| for (var i = 0; i < math.min(rrTsMs.length, rrMs.length); i++) { | ||
| final tsSec = (rrTsMs[i] / 1000.0).round(); | ||
| if (tsSec >= t && tsSec < t + windowS) { | ||
| bucket.add(rrMs[i]); | ||
| } | ||
| } | ||
| final cleaned = _noopCleanRr(bucket); | ||
| if (cleaned.length >= 2) { | ||
| final r = _rmssdRaw(cleaned); | ||
| if (r != null) vals.add(r); | ||
| } | ||
| t += windowS; | ||
| } | ||
| if (vals.isEmpty) return null; | ||
| return vals.reduce((a, b) => a + b) / vals.length; | ||
| } |
Collaborator
|
yeah copilot's got a point on the perf but honestly thats the small part. the bigger thing for me is we now have a second copy of the hrv math living in here - _noopSessionAvgHrv / _noopCleanRr / _noopRejectEctopic / _rmssdRaw are basically redoing what openstrap_analytics already does (correctRr, rmssd, the ectopic rejection). two versions of the same physiology will drift sooner or later and the package one is the tested one, so id rather these call into ana.* than keep their own copy. on the actual loop: yep it rescans the whole rr array for every 5min window, and even scans beats outside start/end each pass, so O(windows x beats). for a normal night its only a couple ms so not urgent, but rrTsMs is already time sorted so one pass bucketing by (tsSec-start)~/300 gets it to O(n). easy fix. not blocking from my side, happy for it to be a followup. just wanted to flag the duplication before it spreads. |
Contributor
Author
|
I have resolved the AI comments and moved the HRV to analytics: OpenStrap/analytics#10 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.