cargo build-dir incompatibility: no output generated for "lib…" "rmeta"

[kornelski]

Summary

Bootstrap searches target_dir for compiled dependencies, but when build-dir feature is enabled, the rmeta files are stored elsewhere.

Command used

./x build clippy

My ~/.cargo/config.toml contains:

[build]
build-dir = "/Users/me/Library/Caches/Cargo/{workspace-path-hash}"

[unstable]
build-dir = true

Expected behaviour

It should support the build-dir setting, or at least disable it.

Actual behaviour

no output generated for "libunicode_ident-61f54578ab0e0070" "rmeta"
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/788da80fcfcef3f34c90def5baa32813e39a1a41/library/std/src/panicking.rs:697:5
   1: core::panicking::panic_fmt
             at /rustc/788da80fcfcef3f34c90def5baa32813e39a1a41/library/core/src/panicking.rs:75:14
   2: bootstrap::core::build_steps::compile::run_cargo
             at ./src/bootstrap/src/core/build_steps/compile.rs:2505:21
   3: <bootstrap::core::build_steps::compile::Rustc as bootstrap::core::builder::Step>::run
             at ./src/bootstrap/src/core/build_steps/compile.rs:1138:9
   4: bootstrap::core::builder::Builder::ensure
             at ./src/bootstrap/src/core/builder/mod.rs:1706:36
   5: <bootstrap::core::build_steps::compile::Assemble as bootstrap::core::builder::Step>::run
             at ./src/bootstrap/src/core/build_steps/compile.rs:2155:36
   6: bootstrap::core::builder::Builder::ensure
             at ./src/bootstrap/src/core/builder/mod.rs:1706:36
   7: bootstrap::core::builder::Builder::compiler
             at ./src/bootstrap/src/core/builder/mod.rs:1341:14
   8: bootstrap::core::build_steps::tool::RustcPrivateCompilers::build_compiler_from_stage
             at ./src/bootstrap/src/core/build_steps/tool.rs:1384:21
   9: bootstrap::core::build_steps::tool::RustcPrivateCompilers::new
             at ./src/bootstrap/src/core/build_steps/tool.rs:1343:30
  10: <bootstrap::core::build_steps::tool::Clippy as bootstrap::core::builder::Step>::make_run
             at ./src/bootstrap/src/core/build_steps/tool.rs:1445:32
  11: bootstrap::core::builder::StepDescription::maybe_run
             at ./src/bootstrap/src/core/builder/mod.rs:519:13
  12: bootstrap::core::builder::StepDescription::run
             at ./src/bootstrap/src/core/builder/mod.rs:651:22
  13: bootstrap::core::builder::Builder::run_step_descriptions
             at ./src/bootstrap/src/core/builder/mod.rs:1314:9
  14: bootstrap::core::builder::Builder::execute_cli
             at ./src/bootstrap/src/core/builder/mod.rs:1294:14
  15: bootstrap::Build::build
             at ./src/bootstrap/src/lib.rs:767:25
  16: bootstrap::main
             at ./src/bootstrap/src/bin/main.rs:107:11

Bootstrap configuration (bootstrap.toml)

None

config.toml is

profile = "tools"
change-id = 132494

[rust]
debug = true
debug-logging = true
debug-assertions = false

Operating system

macOS 15.6

HEAD

2886b36

[Kobzol]

Hi, if the fix is simple, I'd be happy to review a PR that adds support for this, but otherwise I'd label this as unsupported for now.

[epage]

but otherwise I'd label this as unsupported for now.

That seems unfortunate. Users would be setting this for all regular development but then fall down flat when trying to contribute to rustc. We also plan to eventually change the default for build-dir from "unset" to something like {cargo-cache-home}/{workspace-path-hash} which means not supporting it would require users to explicitly configure the feature to match target-dir ({workspace-root}/target).

[Kobzol]

Just to clarify, the bootstrap codebase has a lot of tech debt and issues, and very few maintainers, so my default is kind of to say "no" to new features (similar to what Cargo does :D), unless there's a clear use-case. I didn't know what's the current status of this feature, and we can't support all experimental/unstable features of Cargo in bootstrap, it's already super complicated.

But if you think that this feature will be used a lot in the future, and is heading towards stabilization, then I'm fine with supporting it, of course. As I wrote before, happy to review a PR :)

[epage]

Thanks for the clarification! I assume its clear from my message, that on stabilization the usage will likely be small at first but we will hit a point where it will be pervasive (changing of the default).

As for where its at in terms of stabilization, the FCP has closed (rust-lang/cargo#14125 (comment)) on build-dir though minor a problem was identified when creating the stabilization PR but that should be addressed soon.

[Kobzol]

I took a very brief look at this, and in theory it might be fixable on a small number of places. How can I get the information about where is the build-dir pointing to for a given Cargo invocation? Currently, bootstrap pretty much just hardcodes target/<target>/release/deps.

[epage]

You can lookup the build-dir via cargo metadata.

If there are problems, a workaround would be for you to just overwrite build-dir to point to your hard coded target-dir.

[Kobzol]

Running cargo metadata before every cargo execution isn't really feasible, but we do already run it once at the beginning of bootstrap, to figure out properties of the whole workspace, we could read it there. It's not a trivial refactoring though, so for now I would just override the build-dir using CARGO_BUILD_BUILD_DIR. bootstrap does some funky things with these artifacts anyway.

Done that in #145156.