Skip to content

build: absolute filesystem paths baked into cache keys — breaks cross-runner / cross-workspace reuse #148

Description

@zackees

Problem

fbuild's build-cache keys include absolute filesystem paths in two places. On the same runner image these paths are stable, so they don't explain the iter3 warm regression tracked by #146. But for cross-runner / cross-workspace / containerized / cross-OS cache reuse, they guarantee a cache miss even when source content and toolchain are byte-identical.

Sub-task of #147.

Locations

1. Compiler rebuild signature — crates/fbuild-build/src/compiler.rs:303

let mut hasher = Sha256::new();
hasher.update(compiler_path.to_string_lossy().as_bytes());  // ← absolute
hasher.update([0]);
for group in [flags, pre_flags, extra_flags] { ... }

compiler_path is typically something like /home/runner/.fbuild/packages/toolchain-atmelavr/.../bin/avr-gcc. Change the runner home, change the ~/.fbuild root, mount a different volume → key diverges, full rebuild.

2. Watch-set fingerprint — crates/fbuild-build/src/build_fingerprint/mod.rs:106, 127, 236

All three call sites do hasher.update(normalize_path(&file)), where normalize_path only rewrites separators — absolute paths stay absolute. Lands in hash_files, hash_watch_set, and hash_watch_set_stamps.

Fix direction

compiler.rs:303

Hash compiler identity, not on-disk path:

  • version string (avr-gcc -dumpversion output, captured once at package-resolve time)
  • toolchain package fingerprint (already tracked in fbuild-packages)
  • ABI / target triple

The goal: two runners with the same toolchain package installed at different paths produce the same rebuild signature.

build_fingerprint/mod.rs

Hash paths relative to their watch root, not absolute. FingerprintWatch.root is already the natural anchor — store path.strip_prefix(&watch.root) in the hash.

TDD plan

RED

crates/fbuild-build/tests/cache_keys_stable_across_workspace_rename.rs:

#[test]
fn rebuild_signature_ignores_absolute_compiler_path() {
    // Same compiler version + flags at two different absolute locations → same key.
    let a = CompilerBase { compiler_path: \"/opt/toolchain-a/bin/avr-gcc\".into(), ... };
    let b = CompilerBase { compiler_path: \"/home/runner/.fbuild/…/avr-gcc\".into(), ... };
    assert_eq!(a.rebuild_signature(...), b.rebuild_signature(...));
}

#[test]
fn fingerprint_ignores_workspace_root() {
    // Same project tree under two different parent dirs → same fingerprint.
    let project_a = create_test_project_at(\"/tmp/ws-a/proj\");
    let project_b = create_test_project_at(\"/tmp/ws-b/proj\");
    assert_eq!(hash_watch_set_stamps(&watches_for(&project_a))?,
               hash_watch_set_stamps(&watches_for(&project_b))?);
}

Both fail today. Second test also depends on #146 (mtime) passing so mtime doesn't mask the absolute-path difference.

GREEN

  1. Extract a new CompilerIdentity { version: String, package_fingerprint: String, target_triple: String } — hash this instead of compiler_path.
  2. In hash_watch_set_stamps, change normalize_path(&file) to normalize_path(file.strip_prefix(&watch.root).unwrap_or(&file)).
  3. Same in hash_files and hash_watch_set.
  4. Migrate existing build_fingerprint.json — bump schema version, force one-time recompute.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority: p1Important follow-up after p0 foundations

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions