Disk usage 2-3x somewhere around 1.48

[infinity0]

Not sure if anyone else is seeing this, but the Debian builds of rustc started to take up way more disk space around 1.48, in some cases more than a 3x increase:

arch	1.47 → 1.48	1.48 → 1.48	1.49 beta → 1.49
all [*]	7 GB	20.32 GB	21.73 GB
amd64	8.36 GB	32.69 GB	34.62 GB
arm64	7.61 GB	25.03 GB	26.85 GB
armel	7.39 GB	9.95 GB	11.21 GB
armhf	7.38 GB	9.94 GB	11.2 GB
i386	7.74 GB	27.29 GB	28.84 GB
mips64el	9.47 GB	33.6 GB	36.63 GB
mipsel	8.94 GB	11.5 GB	12.85 GB
powerpc	7.64 GB	9.88 GB	(failed)
ppc64	8.12 GB	33.06 GB	35.1 GB
ppc64el	14.4 GB	39.48 GB	41.75 GB
riscv64	7.7 GB	37.61 GB	(failed)
s390x	8.24 GB	39.31 GB	42.09 GB
sparc64	7.97 GB	31.57 GB	33.39 GB

Data from Debian rustc logs; the column headings mean (version of stage0 used -> stable version being built)

[*] architecture-independent artifacts such as libstd:wasm32 and docs; builds rustc during the process but this is discarded

LLVM 11 was used for all of these builds.

[jyn514]

@infinity0 is this with incremental enabled? Could you paste debian/config.toml?

[infinity0]

We are not enabling incremental, the relevant parts of config.toml is as follows, but this hasn't changed across the 3 different builds listed in the above table:

[build]
submodules = false
vendor = true
locked-deps = false
verbose = 2

[install]
prefix = "/usr"

[target.DEB_BUILD_RUST_TYPE]
llvm-config = "LLVM_DESTDIR/usr/lib/llvm-LLVM_VERSION/bin/llvm-config"
linker = "DEB_BUILD_GNU_TYPE-gcc"

[target.DEB_HOST_RUST_TYPE]
llvm-config = "LLVM_DESTDIR/usr/lib/llvm-LLVM_VERSION/bin/llvm-config"
linker = "DEB_HOST_GNU_TYPE-gcc"

[target.wasm32-wasi]
wasi-root = "/usr"

[llvm]
link-shared = true

[rust]
jemalloc = false
optimize = true
dist-src = false

channel = "stable"

debuginfo-level = 2
debuginfo-level-std = 2
rpath = false
remap-debuginfo = true

verbose-tests = true
backtrace-on-ice = true

We also have some Debian-specific patches but these are generally fairly trivial and also haven't changed significantly across the builds.

One thing that may be significant, is the first column used LLVM 11.0.0 but the others used LLVM 11.0.1. Does rust official builds record the disk usage? If so it would be interesting to see this info for before and after PR #80796.

[Mark-Simulacrum]

We don't track those metrics (yet), but it may be possible to pull them from logs. Do you have a tarball or some other way of seeing what is using the disk space in your case?

[klensy]

I've see

1.40.0+dfsg1-5 (sid)

amd64	Maybe-Successful	2020-01-05 17:11:43	x86-ubc-01	1h 50m	20.07 GB

So it isn't specific to version mentioned?

[nagisa]

What is the actual thing that's being measured here?

[infinity0]

@Mark-Simulacrum no sadly not, it all gets discarded. I can probably reproduce by rebuilding an old version but that'll take some time.

@nagisa Extra disk space consumed by the build process.

It's possibly debuginfo-related - the architectures using the least space are the ones where we are setting debuginfo-level = 0 due to #45854, generally the 32-bit architectures. But it's weird that it's changing across versions when the config is staying the same.

[apiraino]

Issue has been discussed during the compiler team meeting on Zulip.

We acknowledge this is definitively a major annoyance and we need to improve/investigate our reporting on our builds (currently lacking a bit of depth in this regard).

@rustbot label -I-prioritize +P-medium