Skip to content

test: add Unicode and large blob storage regression test - #346

Merged
MPCoreDeveloper merged 3 commits into
MPCoreDeveloper:masterfrom
saltus7:test/storage-regression-tests
Aug 30, 2026
Merged

test: add Unicode and large blob storage regression test#346
MPCoreDeveloper merged 3 commits into
MPCoreDeveloper:masterfrom
saltus7:test/storage-regression-tests

Conversation

@saltus7

@saltus7 saltus7 commented Aug 30, 2026

Copy link
Copy Markdown
Collaborator

Description

Summary

Adds two self-contained regression test suites targeting edge cases in the single-file storage layer. No production code changes.

What's included

UnicodeStorageTests.cs (8 tests)

  • 3-byte CJK roundtrip (Japanese, Chinese, Korean) with WHERE clause matching
  • 4-byte emoji including ZWJ sequences (family emoji, skin tone modifiers)
  • RTL scripts (Arabic, Hebrew)
  • Combining character equivalence — verifies the engine performs byte-exact comparison without normalization (precomposed é vs decomposed e+́ are treated as distinct)
  • Large mixed-script payload roundtrip at the block level

LargeBlobCompressionTests.cs (8 tests)

  • Roundtrip integrity for 1KB, 64KB, 1MB, and 16MB payloads using SHA256 hash verification
  • 16MB test forces ~4,000 page chains, validating block chaining and reassembly
  • High-entropy (incompressible) data to verify compression fallback path
  • Multiple large blobs in the same file to verify registry isolation

Motivation

These tests cover scenarios that are easy to get silently wrong:

  • Unicode normalization bugs are invisible until a user queries café and gets no results because the database stored cafe+́
  • Block chaining bugs on large blobs produce silent data corruption — the read succeeds but the bytes don't match
  • Compression fallback failures only manifest on incompressible data, which is rarely tested

All tests use temp file cleanup via IDisposable and don't require any external infrastructure.

Test results

UnicodeStorageTests: 8/8 passed (1.15s)
LargeBlobCompressionTests: 8/8 passed (2.13s)

Breaking changes

None. Test-only PR.

Note: This PR includes the compression feature from PR #344 as a dependency. The test files reference BlockCompressionMode introduced in that commit. If the maintainer prefers, these tests can be merged together with the compression feature, or rebased onto master after #344 merges.

scdb-dev added 3 commits August 29, 2026 17:18
Validates block chaining integrity for payloads up to 16MB spanning
~4000 pages, compression fallback for incompressible data, and
registry isolation across multiple large blocks in the same file.
Validates roundtrip integrity for 3-byte CJK, 4-byte emoji (including
ZWJ sequences), RTL scripts, and combining characters. Verifies the
engine performs byte-exact comparison without normalization.
@saltus7
saltus7 force-pushed the test/storage-regression-tests branch from 187542a to 6cf8455 Compare August 30, 2026 04:43
@sonarqubecloud

Copy link
Copy Markdown

@MPCoreDeveloper
MPCoreDeveloper merged commit 84bbf8c into MPCoreDeveloper:master Aug 30, 2026
7 checks passed
MPCoreDeveloper pushed a commit that referenced this pull request Aug 30, 2026
… document compression/encryption/metadata sizing (#344 #345 #346)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants