Repository navigation
Enable hypercore compression by default on the asset_datapoint table - #2352
Merged
Merged
Conversation
… exceeded by operation"
richturner
requested changes
Dec 22, 2025
richturner
left a comment
richturner
left a comment
Member
There was a problem hiding this comment.
Going to want to change DATA_POINTS_MAX_AGE_DAYS to be something that aligns with DB chunks i.e. weeks as discussed in calls.
Will need a DB migration script to convert old DAYS to weeks
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request enables TimescaleDB HyperCore (columnstore compression) for the asset_datapoint table to achieve storage savings and improve purge performance. The changes include a database migration to enable columnstore with a 3-month auto-compression policy, an optimized purge strategy using TimescaleDB's drop_chunks() function to avoid decompression overhead, a utility method for datetime conversions, and a comprehensive test validating compression and purging with 2.5 million datapoints.
Key Changes:
- Database migration enables columnstore compression with configurable compression interval
- Purge logic enhanced to use
drop_chunks()for efficient chunk removal, falling back to DELETE when necessary - New
TimeUtil.toInstant()utility method for handling various datetime types
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| V20251216_01__EnableHyperCore.sql | Enables columnstore compression and sets up auto-compression policy for datapoint tables |
| AssetDatapointService.java | Adds optimized purge logic using TimescaleDB's drop_chunks() to avoid decompression overhead |
| TimeUtil.java | Adds utility method to convert various datetime types to Instant |
| PersistenceService.java | Adds configuration property for columnstore compression interval with Flyway placeholder support |
| AssetDatapointHypercoreTest.groovy | Comprehensive test validating compression and purging with 2.5M datapoints across storage measurements |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Member
|
Saw this error in the logs of an instance running compression: |
Contributor
Author
|
@richturner which instance was it? Do you know which version of TimescaleDB is running there? |
# Conflicts: # manager/src/main/java/org/openremote/manager/datapoint/AssetDatapointService.java
|
Member
|
Need to decide what the fallback is if the postgres instance doesn't support hypercore, should configuration be part of DB image and then manager warns if |
# Conflicts: # manager/src/main/java/org/openremote/manager/datapoint/AssetDatapointService.java
Contributor
Author
|
@richturner Looks good. I cannot approve it, because I'm the author |
Member
|
The quick look over is fine, I will do the review of the code you wrote as well |
richturner
approved these changes
May 7, 2026
|
richturner
enabled auto-merge (squash)
May 7, 2026 13:15
MartinaeyNL
changed the title
4 tasks
This was referenced Oct 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enables columnstore compression on the
asset_datapointtable for storage savings and faster purges.Changes
Benefits