Add query planner diagnostics and regression reproductions

[magiccodingman]

Purpose

This PR intentionally does not fix the reproduced query-engine defects. It establishes a stronger semantic-test boundary and structured query-planner observability first, so the follow-up repair work can make independently proven red regressions turn green.

The final branch is a single commit on top of master and retains IndexedDB 2.0 as Magic IndexedDB's supported compatibility contract. IndexedDB 3.0 is still draft and is not used as a correctness assumption or workaround.

Structured planner diagnostics

When Magic IndexedDB debug mode is enabled, the JavaScript query pipeline now records a passive structured trace for the most recent query. The trace does not participate in planning decisions.

It records:

• logical flattening and branch counts;
• optimizer dispatch/result and expected-vs-actual input shape;
• cursor-forcing decisions;
• branch classification (single-index, compound-index, cursor);
• compound-index consumed vs. residual condition counts;
• pagination reconvergence and undefined/invalid cursor entries;
• physical indexed optimization, including input branch counts and output operations;
• indexed/cursor execution selection and result counts where execution completes;
• final partition counts.

Browser tests consume the structured object directly rather than parsing console text. This is intentionally a lightweight last-query development probe, not concurrent correlated telemetry.

Initial reproduced planner defects

The first audit established seven regressions:

1. independent indexed conditions joined by AND can lose intersection semantics;
2. selecting a compound index can discard a residual predicate outside that index;
3. multiple indexed OR branches plus Take/pagination can reconverge as undefined cursor condition sets;
4. multiple indexed StartsWith branches can be rewritten into an incorrect physical In/anyOf query;
5. the live advanced-optimizer dispatch passes an incompatible outer container shape;
6. direct advanced-optimizer invocation violates OR absorption/subset semantics;
7. advanced-optimizer canonicalization can deduplicate equal operation/value pairs across different properties.

Second semantic audit

A broader semantic pass added regression coverage for additional confirmed defect families:

• Cursor branch correlation: (A=1 && B=2) || (A=3 && B=4) can be reconstructed as independent value sets and admit cross-pairs.
• Indexed range provenance: disjoint OR ranges and same-direction range alternatives can lose boolean provenance or become branch-order dependent.
• String/path parity: case-insensitive StartsWith can be sent to a case-sensitive IndexedDB prefix lookup, while ordinary C# string equality can acquire different semantics on the cursor path.
• Case-aware compatibility pruning: a satisfiable equality + OrdinalIgnoreCase prefix can be discarded as contradictory.
• Empty membership: an empty external Contains set can collapse to an empty filter and return the entire table instead of no rows.
• Mixed boolean constants: constants inside larger expressions can become synthetic record conditions instead of preserving identities such as true && P == P and true || P == true.
• DateTime.Date boundaries: non-equality comparisons can compare the original timestamp against target midnight instead of whole-day boundaries; unit contracts also cover Date != target.
• C#→JS operation vocabulary: negated equality/inequality, string.Equals, and collection-property Contains currently emit operation tokens that do not match the canonical JS evaluator vocabulary.
• Empty quantifiers: the Any/All preprocessing visitor throws on empty sequences instead of applying the standard false/true identities.
• Compound-index metadata: components of a compound index are reported as standalone indexes even though IndexedDB 2.0 does not create those independent indexes.
• Compound-key de-dup identity: delimiter-joined string normalization can collide for distinct compound keys such as ["a|b", "c"] and ["a", "b|c"].

A suspected sibling issue involving a live query on one component of a compound primary key did not reproduce. That query currently returns the correct result set, so the test is retained only as a positive semantic guard and is not listed as a bug.

Testing layers

The expanded coverage deliberately separates three kinds of proof:

1. C# translator/unit contracts verify the expression-tree and operation-token semantics before JavaScript is involved.
2. Direct JS truth/metadata tests validate optimizer, cursor-rebuilder, compatibility, index-metadata, and key-identity invariants without depending on the whole database pipeline.
3. Browser semantic-oracle tests execute the real IndexedDB/Dexie path and compare results against equivalent in-memory C# predicates.

This lets a follow-up fix distinguish "translation was wrong", "a transformation changed truth", and "physical execution changed semantics" rather than treating every failure as an opaque browser mismatch.

Browser coverage

The regression suites run on:

• Chromium / Linux;
• Firefox / Linux;
• WebKit / Linux;
• WebKit / macOS query validation.

The macOS open/registration and CRUD/streaming suites remain separate controls and stay green.

Verified final red baseline

Validation on final one-commit head 7c06c6e0 produced the intended result.

Core validation

Restore, package-version checks, clean NuGet packing, and package-content verification all pass.

The semantic unit/contract suite is intentionally red:

• 60 total
• 49 passed
• 11 failed
• 0 skipped

Those 11 failures are only the newly corrected semantic contracts: operation vocabulary/case sensitivity, .Date boundaries, and empty Any/All behavior. There are no compile or harness failures.

Browser validation

• Chromium / Linux: 139 total — 114 passed / 25 failed
• Firefox / Linux: 139 total — 114 passed / 25 failed
• WebKit / Linux: 139 total — 114 passed / 25 failed
• WebKit / macOS query suite: 100 total — 75 passed / 25 failed
• WebKit / macOS CRUD + streaming: 21 passed / 0 failed
• WebKit / macOS open/registration: 1 passed / 0 failed

The same 25 test names fail on every browser engine. There are no Chromium-only, Firefox-only, or WebKit-only semantic failures and no unrelated browser failures. Several defect families intentionally have both a direct transformation-law test and a live end-to-end reproduction, so 25 failing tests should not be interpreted as 25 independent root causes.

Representative reproduced outputs include:

• empty membership: expected 0 rows, actual 44 (entire fixture table);
• cursor OR branch correlation: expected 2 rows, actual 4;
• disjoint indexed range OR: expected 9 rows, actual 0;
• case-insensitive indexed prefix: expected 2 rows, actual 0;
• mixed true || P: expected 44 rows, actual 15;
• mixed true && P: expected 15 rows, actual 0;
• cursor case-sensitive equality with lowercase literal: expected 0 rows, actual 1;
• distinct compound keys normalize to the same "a|b|c" de-dup identity.

Expected CI state

This PR is intentionally red. The red tests are the deliverable: they prove the pre-existing behavior before repair work begins. Build/package infrastructure and unrelated test surfaces remain healthy.

The follow-up bug-fix work should use this merged baseline and drive these semantic regressions green without weakening or deleting the contracts.

[magiccodingman]

Second semantic audit — additional regressions added

After the initial seven planner regressions were isolated, I did one more semantic pass over the query pipeline before treating this PR as the reproduction baseline. That pass found several additional, independently testable correctness gaps, and the PR now captures them without applying production fixes.

Additional confirmed defect families

• Cursor OR-branch correlation: cursor reconstruction can turn correlated branches such as (A=1 && B=2) || (A=3 && B=4) into independent value sets and admit cross-pairs.
• Indexed range provenance: range alternatives can lose their original AND/OR provenance; disjoint OR ranges and same-direction alternatives are now covered explicitly.
• String execution-path parity: case-insensitive StartsWith can reach a case-sensitive indexed prefix lookup, while ordinary C# string equality can acquire different semantics when forced through the cursor.
• Case-aware compatibility pruning: satisfiable equality + ignore-case string-prefix combinations can be discarded as contradictory.
• Empty membership: an empty external Contains collection currently reproduces as the whole table instead of the empty set.
• Mixed boolean constants: true && P and true || P do not preserve their boolean identities end-to-end when the constant appears inside a larger expression.
• Whole-day .Date semantics: Date >, Date <=, and the unit contract for Date != need day-boundary translations rather than raw comparisons against midnight.
• C#→JS operation vocabulary: negated equality/inequality, string.Equals, and collection-property Contains expose mismatched operation tokens (NotEquals, StringEquals, ArrayContains) relative to the JS evaluator's canonical vocabulary.
• Empty Any / All: the preprocessing visitor throws on empty sequences because the expansion uses unseeded Aggregate; the expected identities are false for empty Any and true for empty All.
• Compound-index metadata: compound-index members are currently surfaced as if each were a standalone IndexedDB 2.0 index.
• Compound-key identity collision: delimiter-joined key normalization allows distinct compound keys such as ["a|b", "c"] and ["a", "b|c"] to collapse to the same de-duplication identity.

I also tested a suspected sibling case involving querying one component of a compound primary key. That live query currently returns the correct result set, so I did not classify it as a bug; the test remains as a passing positive regression guard.

Test structure added

The additional coverage is intentionally split across three boundaries:

1. C# translator/unit contracts for expression and wire-token semantics;
2. direct-JS truth/metadata invariants for optimizer/rebuilder/compatibility/index metadata/key identity;
3. real browser IndexedDB/Dexie tests compared against equivalent in-memory C# predicates.

That gives the follow-up repair PR much better localization than a single end-to-end red test.

Final verified baseline

Final squashed head: 7c06c6e0

• Core semantic contracts: 60 total / 49 pass / 11 intentional failures. Restore, package-version checks, clean NuGet pack, and package-content verification are green.
• Chromium/Linux: 139 total / 114 pass / 25 intentional failures.
• Firefox/Linux: 139 / 114 / 25.
• WebKit/Linux: 139 / 114 / 25.
• WebKit/macOS query suite: 100 total / 75 pass / 25 intentional failures.
• WebKit/macOS CRUD + streaming: 21/21 green.
• WebKit/macOS open/registration: 1/1 green.

The same 25 browser test names fail on all four browser engines; there are no browser-specific extra failures. Some defect families intentionally have both a direct truth-law test and a live reproduction, so 25 failing browser tests are not being claimed as 25 separate root causes.

No production query fix is included here. This comment and the updated PR description are the checkpoint: merge the reproductions/diagnostics first, then drive these contracts green in the follow-up bug-fix work.

IndexedDB compatibility remains intentionally scoped to IndexedDB 2.0; IndexedDB 3.0 draft behavior is not used as an assumption or workaround.