Target directory size reduction

Summary

Reduce disk space used by Rust build directories (e.g. target/) through iterative, measurement-guided optimizations in rustc and Cargo, focusing on intermediate artifact sizes, debuginfo, and garbage-collection.

Motivation

Disk space usage is a recurring concern; the 2025 State of Rust survey shows it as the second-most common complaint about the Rust toolchain, surpassed only by slow compilation.

We expect that reducing disk space usage will benefit all environments that build Rust programs, but especially:

• developers who heavily use multiple git worktrees or jj workspaces, such as in AI-first workflows or when contributing to the Rust compiler itself;
• CI jobs that cache intermediate artifacts, both in storage space and in upload/download speeds;
• persistent remote caches, such as used by Bazel and Buck2, primarily in upload/download speeds and disk IO usage.

We also theorize that by reducing disk IO, we can speed up wall time for compilation itself. If our efforts are sufficiently fruitful, we could further decrease compilation time by allowing target directories to fit on a RAM disk (sometimes known as tmpfs).

Note that disk space used by build directories includes both intermediate artifacts and final artifacts. The binary size reduction roadmap tracks final artifacts; this goal is focused primarily, but not solely, on intermediate artifacts. Debuginfo size reductions will benefit both.

The status quo

Compiling Rust programs takes three primary resources: time, memory, and disk space. CPU time, and to some extent wall time and memory use, has been relentlessly optimized and micro-optimized for many years in Rustc. Disk space usage, by contrast, has received far less attention. There have been various proposals to reduce the target directory size on the Cargo side, but little actual work due to a lack of funding, and hardly anything has been done on the Rustc side.

We believe the primary obstacles to shrinking intermediate artifacts are that it’s currently difficult to measure sizes, and a lack of funding for the work. The closest tools are llvm-size to show overall section sizes; rustc -Z meta-stats to show .rmeta contents; and size:query_cache on perf.rust-lang.org, but none give a breakdown of .rmeta or incremental files by query, and are difficult to correlate across invocations.

We are aware of the following ongoing work:

• The cross workspace cache goal and deduplicating build artifacts across workspaces, both of which reduce the number of total artifacts, but not the size of each artifact.
• The garbage-collection work to reclaim space between builds has several tracking issues (cargo#5026, cargo#13136, cargo#13060), but not a Project Goal.
• Reusing caching between check and build is being investigated in the incremental system redesign goal, but cannot help with initial full builds.
• -Zembed-metadata=no avoids creating unnecessary metadata sections, but does not shrink the sections themselves when they are created.
• Reduce debuginfo to line-tables-only in the dev profile reduces the size of debuginfo in the most common scenarios, but does not improve builds that use full debuginfo.

Furthermore, the Cargo team has suggested several possible improvements to disk size which could be incorporated into this goal:

• Reduce how frequently build scripts need to be written
• Give build scripts a dedicated scratchpad for temporary artifacts
• Convert build scripts to artifact dependencies
• Pipeline build scripts, not just library crates

What we propose to do about it

We plan a two-phase goal. The first is a small initiative that has only Jynn as the owner, planned at 3-6 months, with an aim of 30-40% decrease in size for fresh builds. In the second phase, we will organize a team of 3 engineers, plus team champions, planned at 12 months, and aim to decrease the size of the directory by 60% or more for fresh builds and by 80% or more for repeated builds across version branches.

We will work on both decreases in artifact size and garbage-collection within a target directory, such as artifacts from an earlier edit. We will benchmark based on real crates in the Rust ecosystem, focusing on disproportionately large and widely used crates. After benchmarking, we will work on multi-pronged approaches that allow parallel work on high-impact interventions.

At a high level, our plan is:

1. Profile disk space usage by builds, and include that data in rustc-perf so it is triaged alongside other performance improvements and regressions.
2. Near-term (1-6 months), use those measurements to guide simple improvements, with a focus on “easy wins” that greatly reduce size without much implementation work.
3. Mid-term (3-12 months), use measurements to guide structural improvements that need design work or extensive review.
4. Long-term (6-18 months), make large refactors, and change best-practices in the ecosystem to avoid disk usage.

We will document our approach and tooling so that sizes can be further reduced even after the goal period ends.

We do not believe our long-term goals will be necessary for our proposed 60-80% size improvement in the first year. We expect mid-term and long-term redesigns to “compose” with the short-term improvements; in other words, major improvements before a redesign will also be major improvements after the redesign. If we find this not to be the case after experience during implementation, we will revisit our approach.

All structural redesigns will be conditional on benchmarks that show that they’re actually a problem.

Axioms

• Avoid an all-or-nothing approach. Steps can depend on previous steps, but between each step we should have a coherent work product that will be an improvement to the ecosystem even if later steps are never completed. Size improvements should stand on their own without requiring a rewrite of subsystems of Rustc or Cargo. Only work on sweeping refactors once we have fixed all the “low-hanging fruit”.

• Focus on architectural improvements. Micro-optimizations look good in the short-term and are hard to maintain in the long-term. They can also easily be specific to the precise hardware that was used to run the benchmark. By prioritizing algorithmic and architectural improvements, we hope to end up with code that is more maintainable than it was to start, while still generating significantly smaller artifacts.

• Work together. There are many people who want to work on the compiler and few who have the expertise to do so. We have an explicit goal of teaching people about the compiler, especially about the query system and rustc_metadata, so that they can continue to maintain these subsystems even after the goal is closed. As a corollary, we will choose optimizations that can be worked on in parallel, so that multiple people can be working at once, collaborating and bouncing ideas off of each other.

  Note: A common problem when optimizing CPU time is that improvements in one area regress other areas, especially when touching the query system, forcing work to essentially be done in serial. We do not expect that to be the case here, because so little optimization has been done to date. We will benchmark each change to confirm, and will revist our scheduling assumptions every 3 months based on observed conflicts.

• Use good defaults but allow configuration. Many software projects are either highly configurable (e.g. Kakoune) or work well out of the box (Helix), but few are both. We aim to make improvements that greatly improve disk space without needlessly regressing performance, while still allowing developers to choose higher reductions in exchange for slower compilation time, or vice-versa. For example, environments that already use an in-kernel compressed file system for target/ may wish to disable debuginfo compression, or remote caching servers that rarely rebuild but have to maintain artifacts indefinitely may wish to prioritize size over speed.

Work items over the next year

See hackmd for a detailed workplan.

Team asks

Suggested reviewers:

• cargo: Ed Page, Weihang Lo, Ross Sullivan
• compiler: Vadim Petrochenkov, Nicholas Nethercote
• compiler/performance: Jakub Beránek
• infra/bootstrap: Jakub Beránek

Help wanted

Note: at this stage we are primarily looking for contributors who are new to this area of the compiler, or new to the Rust project overall, and wish to be mentored as part of this goal. We have sufficient contributors with existing experience.

Funding

We are looking for sponsors to fund work on this goal. We suggest funding in two phases: first, a short-term phase, with a small group; then a mid-term phase with a larger group.

Phase 1: Measurements and near-term improvements

This goal needs a minimum of $60k to run, which will allow us to do preliminary research and high-leverage low-hanging improvements, as described in the “Measurements” and “Near-term improvements” section of the HackMD. Our best estimate is that this minimum funding would give around a 30% improvement in incremental target sizes over the next 6 months.

I have extensive experience working on the compiler, including the query system and integration work between Rustc and Cargo. I also have project management experience from my time at Ferrocene, as well as consensus building experience on policy, design, and implementation work within the Rust project. As part of this work, I would lead implementation, mentor contributors, and coordinate between the Compiler and Cargo teams.

Phase 2: mid-term improvements

There are over a dozen workstreams within this proposed goal. By myself, I cannot implement them all within a year. Additional funding will allow me to hire and mentor other compiler and build system engineers to split up the work, not only accelerating timelines but also completing more overall workstreams. Furthermore, it will allow me to focus on design work rather than implementation, unlocking the larger structural improvements that require coordination between teams.

This amount of funding will allow us to reach the 60% (per-directory) and 80% (cross-directory) targets from the proposal section.

Weihang Lo is a current member of the Cargo team and a member of Amazon’s build performance team. He has experience implementing, reviewing, and designing changes to Cargo’s garbage collection and build configuration, and has made -Z embed-metadata the default in Cargo, reducing metadata size by half on nightly toolchains. He has written in detail about the changes that would improve binary size.

Matyáš Racek is a current member of the Compiler Performance working group and has ongoing work to reduce encoding sizes in rmeta files and incremental caches. He has experience measuring, triaging, and improving the compiler’s performance, and his assistance will be invaluable in optimizations within the compiler itself.

Jakub Beránek has been responsible for implementing and stabilizing -Z embed-metadata=no, and is a current member of the Council. embed-metadata has reduced the size of release builds by almost 30% in the past, and we are confident that further similar improvements are possible.

Zeklandia has a non-traditional programming background, but has the logistical flexibility to do short-term contract work, and I hope to mentor her to be a long-term maintainer on these compiler subsystems.

Frequently asked questions

Q: Does this goal include garbage-collection across directories, such as deleting cargo -Z script artifacts?

No. There is parallel work on that from the Cargo team, but it is not currently part of this goal.