Skip to content

60 replace typeid storage with recordid recordkey architecture - #62

Merged
lxsaah merged 10 commits into
mainfrom
60-replace-typeid-storage-with-recordid-recordkey-architecture
Dec 6, 2025
Merged

60 replace typeid storage with recordid recordkey architecture#62
lxsaah merged 10 commits into
mainfrom
60-replace-typeid-storage-with-recordid-recordkey-architecture

Conversation

@lxsaah

@lxsaah lxsaah commented Dec 6, 2025

Copy link
Copy Markdown
Contributor

Added

  • RecordId + RecordKey Architecture (Issue Replace TypeId Storage with RecordId + RecordKey Architecture #60): Complete rewrite of internal storage for stable record identification
    • RecordId: u32 index wrapper for O(1) Vec-based hot-path access
    • RecordKey: Hybrid &'static str / Arc<str> with Borrow<str> for zero-alloc static keys and flexible dynamic keys
    • O(1) key resolution via HashMap<RecordKey, RecordId>
    • Type introspection via HashMap<TypeId, Vec<RecordId>>
  • Key-Based Producer/Consumer API:
    • produce_by_key::<T>(key, value): Produce to specific record by key
    • subscribe_by_key::<T>(key): Subscribe to specific record by key
    • producer_by_key::<T>(key): Get key-bound producer
    • consumer_by_key::<T>(key): Get key-bound consumer
  • New Types: ProducerByKey<T, R> and ConsumerByKey<T, R> for key-bound access with .key() accessor
  • Introspection Methods:
    • records_of_type::<T>(): Returns &[RecordId] for all records of type T
    • resolve_key(key): O(1) lookup returning Option<RecordId>
  • New Error Variants:
    • RecordKeyNotFound: Key doesn't exist in registry
    • InvalidRecordId: RecordId out of bounds
    • TypeMismatch: Type assertion failed during downcast
    • AmbiguousType: Multiple records of same type (use key-based API)
    • DuplicateRecordKey: Key already registered
  • RecordMetadata Extensions: Now includes record_id: u32 and record_key: String fields

Changed

  • Breaking: configure<T>() Signature: Now requires key parameter: configure::<T>("key", |reg| ...)
  • Breaking: Internal Storage: Changed from BTreeMap<TypeId, Box<dyn AnyRecord>> to:
    • Vec<Box<dyn AnyRecord>> for O(1) hot-path access by RecordId
    • HashMap<RecordKey, RecordId> for O(1) name lookups
    • HashMap<TypeId, Vec<RecordId>> for type introspection
  • Breaking: Type-Based Lookup: produce(), subscribe(), producer(), consumer() now return AmbiguousType error when multiple records of the same type exist
  • SecurityPolicy: ReadWrite variant now uses HashSet<String> for writable record keys
  • Dependencies: Added hashbrown for no_std-compatible HashMap with default-hasher feature
    ´

… indexing

- Added RecordId and RecordKey types for O(1) record access and identification.
- Implemented key-based producer/consumer API with produce_by_key and subscribe_by_key methods.
- Enhanced introspection capabilities with records_of_type and resolve_key methods.
- Introduced new error variants for better error handling.
- Updated RecordMetadata to include record_id and record_key fields.
- Refactored internal storage for improved performance and stability.
- Updated SecurityPolicy to use RecordKey for writable records.
- Added multi-instance tests to validate new architecture.
- Updated documentation and migration guide to reflect breaking changes.
@lxsaah
lxsaah requested a review from Copilot December 6, 2025 17:01
@lxsaah lxsaah self-assigned this Dec 6, 2025
@lxsaah lxsaah linked an issue Dec 6, 2025 that may be closed by this pull request

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements a comprehensive architectural refactoring (Issue #60) that replaces TypeId-based record storage with a new RecordId/RecordKey system. This enables multiple records of the same Rust type to coexist with different keys (e.g., "sensors.indoor" and "sensors.outdoor" can both store Temperature data).

The changes include:

  • New RecordId (u32 index) and RecordKey (hybrid static/dynamic string) types for record identification
  • Internal storage refactor from BTreeMap<TypeId> to Vec + multiple HashMap indexes for O(1) access
  • Key-based producer/consumer API methods (produce_by_key, subscribe_by_key, etc.) with new ProducerByKey and ConsumerByKey types
  • Breaking change: configure<T>() now requires a key parameter
  • Type-based methods return AmbiguousType error when multiple records of the same type exist
  • Security policy migration from TypeId-based to key-based writable records
  • Comprehensive test coverage and updated examples

Reviewed changes

Copilot reviewed 32 out of 33 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
aimdb-core/src/record_id.rs New module with RecordId and RecordKey types, including Borrow<str> for O(1) HashMap lookups
aimdb-core/src/builder.rs Refactored storage from BTreeMap to Vec + HashMap indexes; updated configure() to require key parameter
aimdb-core/src/typed_api.rs Added ProducerByKey and ConsumerByKey types for key-bound access
aimdb-core/src/error.rs New error variants: RecordKeyNotFound, InvalidRecordId, TypeMismatch, AmbiguousType, DuplicateRecordKey
aimdb-core/src/remote/config.rs Security policy migrated from TypeId to key-based writable records
aimdb-core/src/remote/handler.rs Remote access handlers updated for key-based lookups instead of TypeId
aimdb-core/src/remote/metadata.rs Added record_id and record_key fields to RecordMetadata
aimdb-tokio-adapter/tests/multi_instance_tests.rs Comprehensive test suite for multi-instance scenarios
examples/* All examples updated to use new key-based configure() API
docs/design/015-M6-record-id-architecture.md Design documentation with implementation decisions
CHANGELOG.md Migration guide and breaking changes documentation

Comment thread CHANGELOG.md Outdated
Comment thread aimdb-core/src/builder.rs Outdated
Comment thread aimdb-core/src/remote/config.rs Outdated
push spawn functions only for new records

improve error handling
@lxsaah
lxsaah requested a review from Copilot December 6, 2025 17:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 33 out of 34 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

tools/aimdb-mcp/CHANGELOG.md:1

  • The date '2025-11-20' in line 16 is in the future relative to the current date (December 6, 2025 is after November 20, 2025). However, this appears to be an existing entry that should remain unchanged.

Comment thread examples/remote-access-demo/src/server.rs
Comment thread aimdb-core/src/remote/config.rs
Comment thread aimdb-core/src/builder.rs
@lxsaah
lxsaah merged commit a9eb9cf into main Dec 6, 2025
12 of 13 checks passed
@lxsaah
lxsaah deleted the 60-replace-typeid-storage-with-recordid-recordkey-architecture branch December 6, 2025 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace TypeId Storage with RecordId + RecordKey Architecture

2 participants