A scute is one of the bony plates that make up a turtle's shell. A shell is made of plates; a database file is made of pages. That is the whole idea, and every layer here is built on it.
This is a learning project built in the open, not a product. Each step starts with the problem before the code, ships with a test, and where bytes are involved, a real hexdump you can run yourself.
| Repo | What |
|---|---|
| scutedb | The engine. Go 1.22+, MIT, zero dependencies. |
Six phases, roughly 18 weeks.
| Phase | What | Status |
|---|---|---|
| 0 | Foundations — interfaces, the naive database, pages | done |
| A | Bytes & the B+Tree | done |
| B | Persistence — storage manager, buffer pool, locking | next |
| C | A real data store — schema, rows, indexes | |
| D | Transactions — WAL, recovery, 2PL, MVCC | |
| E | Beyond — LSM engine, Raft, server, query planner |
Phase 0 is complete. Three interfaces defined before any implementation
(File, Index, Engine), a deliberately naive append-only database measured
until it broke four separate ways — O(n) lookups, 116,475× write amplification,
497 of 1,000 records silently lost to a data race, 2,439 records vanishing on
SIGKILL — and then fixed 4 KB pages with a self-describing 16-byte header to
fix the root cause behind all four: no fixed unit.
Phase A is complete: bytes first, then a B+Tree. internal/codec is split in
two, because keys and values have different jobs: values optimize for size
(varint, zigzag), keys optimize for sorting correctly as raw bytes (fixed-width
big-endian, sign-flipped). Conflating them is a common mistake and is not
recoverable later — an index built on a non-order-preserving key encoding returns
wrong answers for every range query. Null bitmaps then make "no value" a property
of the row rather than a magic value hidden inside it, and aligned slots buy O(1)
addressing inside a page for a few bytes of padding.
Then the tree. Search, insert, node splits and root splits; leaves chained together so a range costs one descent and then a sideways walk rather than one descent per key; and deletion, which is the half most from-scratch engines skip — underflow repaired by borrowing from the left sibling, else the right, else merging, with merges cascading upward and the root collapsing when it runs out of keys. 2,000 keys sit at height 7; deleting 1,990 of them brings it back to height 3 through 1,487 borrows, 1,485 merges and 4 collapses.
99 tests, 9 fuzz targets, go test -race clean, zero dependencies. The test
suite itself was audited by mutation testing — deliberately breaking the
implementation to check the tests notice — which found one invariant that had
never once fired.
Next: Phase B. Every node is still a Go pointer, so nothing survives a restart. Replacing those pointers with page IDs is what turns a data structure into a database.
- Go 1.22+, standard library only. No dependencies, on purpose.
- Explain first, then build. The reasoning lives in the README and commit messages.
- Every step ships a test. Where bytes are involved, a hexdump.
gofmtandgo vetclean before every commit.
MIT licensed · Built by @suhailopensource
