Skip to content

Migrate dependency management from git submodules to soldeer #194

Closed

Description

@thedavidmeister

Migrate dependency management from git submodules to soldeer

The repo currently uses 6 git submodules (forge-std, rain.string, rain.datacontract, rain.math.fixedpoint, rain.deploy, rain.sol.codegen) plus an openzeppelin-contracts submodule under rain.math.fixedpoint/lib/. Switch to soldeer for dependency management.

Why

  • Smaller clones — soldeer fetches archives, not full git history of each dep.
  • Cleaner versioning — semver lock via soldeer.lock rather than tracking arbitrary commit SHAs across .gitmodules + foundry.lock.
  • First-class foundry support (forge soldeer install).
  • Easier transitive resolution; today each indirect dep has to be added explicitly as a submodule.
  • CI simpler: no submodules: recursive checkout.

What changes

  1. Delete .gitmodules. Remove lib/ from version control (or keep it as the soldeer install target).
  2. Replace foundry.toml's remappings = [...] with a [soldeer] and/or [dependencies] section.
  3. Add soldeer.lock (committed) that pins resolved versions.
  4. Add forge soldeer install to every CI job that builds (rainix.yaml, manual-sol-artifacts.yaml, etc.) and drop submodules: recursive from actions/checkout@v4.
  5. nix develop shell needs to provide a soldeer-aware forge (current nix develop already does, modulo flake.lock bumps).

Per-dependency decisions

Each rain.* dep needs to be either:

  • Published to the soldeer registry (https://soldeer.xyz) — clean semver consumption.
  • Declared as a git-source dependency in foundry.toml — rain.deploy = { url = "...", ref = "<sha>" }. Equivalent to submodules with different syntax; doesn't gain semver benefits.

Recommendation: publish rain.* to soldeer first; consume via published versions in this repo.

Bytecode-reproducibility check (must pass before merge)

The Zoltu deterministic deploy address is a function of creation code, which transitively depends on every byte of every dependency. Confirm keccak256(type(DecimalFloat).creationCode) is identical before and after the migration on the same source revision; if it shifts, LibDecimalFloatDeploy.ZOLTU_DEPLOYED_DECIMAL_FLOAT_ADDRESS and the contract codehash constants must be regenerated and a redeploy triggered.

Same check for LibDecimalFloatDeploy.combinedTables() initcode (log tables data contract).

Open questions to resolve before starting

  • Which rain.* deps are already published to soldeer? Which need to be published as part of this migration?
  • Are downstream consumers of rain.math.float on soldeer, submodules, or both?
  • Does rain.deploy itself need to migrate first (it uses submodules today per its CLAUDE.md)?

Out of scope

  • Switching off forge entirely.
  • Changing the deterministic deploy address intentionally (any change must be unintentional and verified zero-impact).

Activity

  1. thedavidmeister commented on May 9, 2026

    @thedavidmeister
    ContributorAuthor

    Dependency graph

    Verified by git submodule status --recursive and grep -hrE '^import.*from' over src/test/script at HEAD of branch 2026-05-09-189.

    Direct deps (6, declared in .gitmodules)

    Dep SHA Imports used in this repo
    forge-std 1801b054 (v1.14.0) Script.sol, Test.sol
    rain.datacontract 3bff3078 (2026-02-07-creation-code-2-g3bff307) lib/LibDataContract.sol
    rain.deploy ccfdb378 lib/LibRainDeploy.sol
    rain.math.fixedpoint 8308cbb6 (transitively used; no direct imports)
    rain.sol.codegen dfe95884 src/lib/LibCodeGen.sol, src/lib/LibFs.sol
    rain.string 488f237c error/ErrParse.sol, lib/parse/LibParseChar.sol, lib/parse/LibParseDecimal.sol

    Transitive deps (6 unique repos beyond top level)

    Repo Pulled in by SHAs observed
    openzeppelin-contracts rain.math.fixedpoint, rain.string fcbae5394 (v4.8.0-1034) — single SHA across 3 sites
    rain.solmem rain.datacontract, rain.string 2e47e41a (datacontract), f28da2f0 (string) — 2 versions
    erc4626-tests openzeppelin-contracts 232ff9ba8 (v0.1.1) — same SHA across 3 sites
    halmos-cheatcodes openzeppelin-contracts 7328abe1 — same SHA across 3 sites
    rain.math.binary rain.sol.codegen f44f846a
    rain.math.saturating rain.math.fixedpoint 22e58d70

    Version drift

    • forge-std at 3 versions in the tree:
      • 1801b054 (v1.14.0) — top level + datacontract + deploy + solmem (datacontract path)
      • b8f065fd (v1.11.0-5) — fixedpoint + sol.codegen + math.saturating + math.binary + rain.solmem (string path)
      • 3b20d60d (v1.9.6) — all openzeppelin-contracts copies
    • rain.solmem at 2 versions (2e47e41a vs f28da2f0).
    • Soldeer dedup forces resolution to a single version; today these are silently inconsistent across the tree.

    Imports actually used in this repo

    forge-std/{Script,Test}.sol
    openzeppelin-contracts/contracts/utils/Strings.sol
    rain.datacontract/lib/LibDataContract.sol
    rain.deploy/lib/LibRainDeploy.sol
    rain.sol.codegen/src/lib/{LibCodeGen,LibFs}.sol
    rain.solmem/lib/LibBytes.sol
    rain.string/{error/ErrParse,lib/parse/LibParseChar,lib/parse/LibParseDecimal}.sol
    

    rain.math.fixedpoint has no direct imports — verify whether it's reachable transitively (e.g. via shared types in rain.string) before declaring it droppable.

    Current foundry.toml remappings

    rain.solmem/=lib/rain.datacontract/lib/rain.solmem/src/
    openzeppelin-contracts/=lib/rain.math.fixedpoint/lib/openzeppelin-contracts/
    rain.deploy/=lib/rain.deploy/src/
    

    The other deps rely on foundry's default lib/<name> resolution. Soldeer would generate all remappings explicitly — a one-time tabling exercise.


    Migration topological order

    External sources first; Rain-controlled in dependency order. Within each level the items are independent and can be migrated in parallel.

    Level 0 — external (no Rain migration work)

    • forge-std
    • erc4626-tests
    • halmos-cheatcodes
    • openzeppelin-contracts (treat as leaf; already published)

    Level 1 — Rain leaves (depend only on external)

    • rain.solmem → forge-std
    • rain.math.binary → forge-std
    • rain.math.saturating → forge-std
    • rain.deploy → forge-std

    Level 2 — depend on level 1

    • rain.datacontract → rain.solmem
    • rain.string → openzeppelin-contracts, rain.solmem
    • rain.sol.codegen → rain.math.binary

    Level 3

    • rain.math.fixedpoint → openzeppelin-contracts, rain.math.saturating, rain.string

    Level 4

    • rain.math.float (this repo) → rain.datacontract, rain.deploy, rain.math.fixedpoint, rain.sol.codegen, rain.string

    Bottleneck: rain.math.fixedpoint. Migration of this repo is blocked until levels 1–3 are complete.

    Per-repo migration recipe (apply bottom up):

    1. All deps already migrated and published.
    2. Replace lib/ submodules with soldeer entries in foundry.toml.
    3. Generate soldeer.lock. Update CI to drop submodules: recursive and add forge soldeer install.
    4. Verify keccak256(creationCode) diff = 0 vs the pre-migration HEAD on the same source revision (especially critical for rain.deploy and rain.math.float, which have deterministic deploys shipped to prod).
    5. Publish to soldeer (or commit a release tag if going git-source).
  2. thedavidmeister commented on May 9, 2026

    @thedavidmeister
    ContributorAuthor

    Progress 2026-05-09

    rain.solmem is the first leaf migrated end-to-end:

    Side-effects from this leaf surfaced two rainix bugs that affect every sol-only consumer:

    • rust toolchain leaking into sol-shell via the rustfmt-conditional pre-commit hook
    • nodejs leaking into sol-shell via the prettier hook's default package

    Both fixed in rainlanguage/rainix#134 with a closure regression test (mutation-tested).

    What's next

    Migrate the next leaves in the topo-sort published higher up this issue:

    • rain.math.binary
    • rain.math.saturating
    • rain.deploy

    Each leaf gets the same shape: soldeer init → ignore depth-anchoring → REUSE coverage → publish workflow as the rainix wrapper → tag v0.1.0 → confirm publish.

    The registry-side project must exist on https://soldeer.xyz before the first publish lands; first attempt fails with project <name> not found and the project must be created manually.

  3. thedavidmeister commented on May 9, 2026

    @thedavidmeister
    ContributorAuthor

    Progress 2026-05-10

    Levels 1-3 of the topo-sort are now migrated end-to-end. rain.math.float itself (level 4) is unblocked.

    Level Repo PR Published
    1 rain.solmem #34, #36, #40, #38, #37 rain-solmem v0.1.3
    1 rain.math.binary #11, #12, #13, #14, #15 rain-math-binary v0.1.1
    1 rain.math.saturating #10, #11, #12, #13, #14 rain-math-saturating v0.1.1
    1 rain.deploy #7, #8, #9 rain-deploy v0.1.2
    2 rain.datacontract #18, #19 tag v0.1.0 pushed (registry-side project may need creating)
    2 rain.string #21 rain-string v0.1.0
    2 rain.sol.codegen #18 tag v0.1.0 pushed (registry-side project may need creating)
    3 rain.math.fixedpoint #17 tag v0.1.0 pushed (registry-side project may need creating)

    The first PR per repo is the soldeer migration; subsequent PRs collapse the per-task workflow wrappers into the rainix-sol composite reusable.

    Rainix infrastructure shipped along the way

    PR What
    rainlanguage/rainix#134 Stop leaking rust-toolchain and nodejs into sol-shell (rustfmt-conditional + prettier hook fixes; closure regression test, mutation-tested)
    rainlanguage/rainix#136 Reusable publish-soldeer.yaml workflow
    rainlanguage/rainix#137 Reusable rainix-sol-static.yaml
    rainlanguage/rainix#138 Reusable rainix-sol-legal.yaml
    rainlanguage/rainix#139 Reusable rainix-sol-test.yaml
    rainlanguage/rainix#140 Composite rainix-sol.yaml that fans out static/legal/test in parallel as separate runners
    rainlanguage/rainix#141 (open) Mirror rust-shell and js-shell slim devShells from the sol-shell precedent

    What's left

    • rain.math.float itself, the only level-4 leaf and the original target. Now unblocked since rain.math.fixedpoint, rain.datacontract, rain.deploy, rain.sol.codegen, rain.string all have soldeer publishes (or tags ready pending registry-side project creation).
    • Bytecode-reproducibility check before merge: keccak256(type(DecimalFloat).creationCode) and the log-tables initcode must be identical pre/post migration. If they shift, LibDecimalFloatDeploy.ZOLTU_DEPLOYED_DECIMAL_FLOAT_ADDRESS and the codehash constants need regenerating with a fresh deploy.

    Operational note

    soldeer.xyz rejects pushes to nonexistent projects with error during IO operation for "": not connected (misleading message — actually a 404). First-time tag for any new package needs a one-time manual project creation on soldeer.xyz before the publish workflow lands. Confirmed against rain-solmem, rain-math-binary, rain-math-saturating, rain-deploy, rain-string. Pending: rain-datacontract, rain-sol-codegen, rain-math-fixedpoint.

  4. added a commit that references this issue on May 9, 2026
    7e3dd01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions