You descend into the pit of exit codes and one-character flags,
The sunken abyss of the Makefiles,
The mill of 11-year-old legacy scripting.
The writhen shell commands serve in adoration,
Immured beneath a rottenrun-makedirectory.
Before you lies the servitor of order,
The cleaver,
The redeemer,
The recreator,
Therun-make-supportcrate.
Vessel of rust-lang,
You are the unmaker,
You bring the rapture,
The murmurs of error logs rise...
The theatre of CI failures begins.
Let's say, theoretically, that you are some average every day nerd who wants to git gud and do a cool techy thing for once. Unfortunately, you have been locked into various educational institutions since the age of 5 and have been constantly at the centre of shallow praise, leading you to believe you do not need to work hard to succeed in life. Now, painfully mediocre compared to the legends writing operating systems in the womb, how will you accomplish this feat?
The answer, of course, is relentless violence.
Before me stood 352 Makefiles, used for various compiler integrity and correctness checks in the master Rust repository. Note the past tense - each day of the summer of 2024, I have hacked away at them without pause, rewriting them in Rust with the use of the run-make-support crate - which also needed multiple extensions to emulate the various features present in the Makefile swarm.
Here is an example of one such rewrite:
# needs-profiler-support
# ignore-cross-compile
include ../tools.mk
COMPILE_FLAGS=-g -Cprofile-generate="$(TMPDIR)"
all:
$(RUSTC) $(COMPILE_FLAGS) test.rs
$(call RUN,test) || exit 1
[ -e "$(TMPDIR)"/default_*.profraw ] || (echo "No .profraw file"; exit 1)
which becomes:
// -C profile-generate, when used with rustc, is supposed to output
// profile files (.profraw) after running a binary to analyze how the compiler
// optimizes code. This test checks that these files are generated.
// See https://github.com/rust-lang/rust/pull/48346
//@ needs-profiler-support
//@ ignore-cross-compile
use run_make_support::{cwd, has_extension, has_prefix, run, rustc, shallow_find_files};
fn main() {
rustc().arg("-g").profile_generate(cwd()).input("test.rs").run();
run("test");
let profraw_files = shallow_find_files(cwd(), |path| {
has_prefix(path, "default") && has_extension(path, "profraw")
});
assert!(!profraw_files.is_empty(), "no .profraw file generated");
}
All progress was neatly catalogued in this issue, acting as a Most Wanted hitlist of sorts for the most resilient of Makefiles. As of writing this, 11/352 Makefiles remain, which mean the porting efforts are 97% complete. This project was sponsored as part of the Rust Foundation's first year entry into Google Summer of Code.
Due to the extremely commit-spammy nature of this work, this project has rocketed me to - at the time of writing - position #153 of all time in the Rust Contributors Leaderboard. I even soared past my mentor jieyouxu, who is an established maintainer. Anyone with two neurons and a functional synapse in between will understand that this is laughably meaningless, especially considering they made the #1 a bot to remind the monomanic maintainers to touch grass and stop burning themselves out. But, hey, it might deceive some foolish figure of authority one day to give me some respect.
Now that your attention is starting to wane, let me proceed right away to stories of various fun moments throughout this project. I wanted to call this part the "House of Horrors", though it's more a "House of Curiosities" by now.
What Blasphemy Really Looks Like
Let's start with a demonstration of how cursed these Makefiles can get:
ifneq (,$(findstring x86,$(TARGET)))
What do you think this means? "If the string x86 is not found in the target name, run this block"? WRONG! findstring will return x86 if it finds it in TARGET, and an empty string otherwise. This is compared with the first argument of the expression, (, (did you miss it?), which is an empty string. That means "if TARGET contains x86, then empty string is not equal to x86 (returned by findstring), then run this block." It's the OPPOSITE of my first impression.
Yes, this is real Rust repository code, running on every single pull request merge as part of the test suite, and it has been present there for years.
Until I arrived, of course. You may start applauding now.
Traps and Tricks
Some parts of Makefile syntax truly cut a line between the wheat and the chaff, with the measured quality being an obsessive tendency to read between the lines. Observe:
FLAGS := -C link-args=-Wl,--no-undefined
No whitespace after the comma? Is the formatter broken? Much the opposite: adding a space here would cause --no-undefined to become an extra argument. The test will then crash and burn. You, acute dear reader, have obviously spotted the other whitespace in this string - the third character. That one, as you have guessed with your unmatched intellect, is completely unnecessary:
FLAGS := -Clink-args=-Wl,--no-undefined
Perfectly acceptable.
FLAGS := -C link-args=-Wl, --no-undefined
Cast down a Hell pit to be pinched by crabs for all eternity.
Another fun one:
# Check that a primary bundle can be loaded and will be preferentially used
# where possible.
custom: test.rs working.ftl
$(RUSTC) $< -Ztranslate-additional-ftl=$(CURDIR)/working.ftl 2>&1 | $(CGREP) "this is a test message"
Please divert your attention for now from the glorious intuitiveness of $< (replaced by test.rs, the first dependency) and 2>&1 (prints out errors alongside normal output). There is something much more evil at play. Normally, RUSTC calls that are expected to fail will be written like this:
$(RUSTC) broken-file.rs && exit 1 || exit 0
This basically means, "if we run RUSTC and it returns 1 (fail), instead, return 0 (success)". This pattern is extremely common. However, the RUSTC call shown in the bundle test will fail, but has no indication of this. How does the test pass at all? Because the pipe | throws away any resulting exit codes, completely ignoring whether compilation succeeded or failed.
This time, it's supposed to fail, and the error should contain "this is a test message", but a rust program composed of simply println!("this is a test message") would pass this test just as well.
Thankfully, the test rewrite roots out such glaring weaknesses:
fn custom_bundle() {
// Check that a primary bundle can be loaded and will be preferentially used
// where possible.
rustc()
.arg("-Ztranslate-additional-ftl=working.ftl")
.input("test.rs")
.run_fail()
.assert_stderr_contains("this is a test message");
}
Non-Euclidean Symlinks
Here is something much less bad, but that launched me into a Schrödinger's cat-approved adventure due to a misunderstanding:
NAME := $(shell $(RUSTC) --print file-names foo.rs)
mkdir -p $(TMPDIR)/outdir
$(RUSTC) foo.rs -o $(TMPDIR)/outdir/$(NAME)
ln -nsf outdir/$(NAME) $(TMPDIR)
RUSTC_LOG=rustc_metadata::loader $(RUSTC) bar.rs
The pertinent line is the one starting with ln, which creates a systemic link (basically, a portal leading from one place to another) in the file system. We are creating a symlink from TMPDIR to outdir/libfoo.rlib (NAME is libfoo.rlib), which is itself contained inside TMPDIR. What? This "portal" leads to itself? It's a box containing the box?? A house in which the house is stored???
No, I just forgot that command-line utilities tend to auto-complete the destination, and that the operation is actually creating a symlink from TMPDIR/<any symlink cool name> to outdir/libfoo.rlib.
Final Testination
Porting tests requires opening up a massive amount of pull requests. Pull requests require branches. Branches need to be named. And, so it is said...
There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton
None of that "informative naming convention" nonsense. Names should be cute, not descriptive.
Which led to the glorious rise of:
- final-testination
- dwarf-fortestress (on a test about the DWARF data format!)
- infinite-test-a-novel
- one-flew-over-the-cuckoo's-test
- exitestial-crisis
- bootest-contestllation
- testigitation-cantrip
- no-test-for-the-wicked
- the-intelligent-intestor
- ...and too many others to list.
The Feline Professionals
Many, many tests involved looking at symbols in an object file, and doing something with them, such as checking that a certain one is present. In the Makefiles, this used the nm command-line utility. How to port this over? "I'll just rewrite nm in Rust", I thought. "it will be so easy."
It wasn't. After a few days, I modestly accepted my fate and used llvm-readobj instead.
Much, much later in the summer, a new contributor arrived and wrote a custom object file traversal to help port a Makefile. I clicked on her website, where I discovered ASCII-art cats and links leading to other sites containing pride flags.
Sometimes, you need to let the professionals do the job.
Assorted Fun Comments
# Check that the compiler errors out when the sysroot requested cannot be
# found. This test might start failing if there actually exists a Klingon
# translation of rustc's error messages.
sysroot-missing:
$(RUSTC) $< -Ztranslate-lang=tlh 2>&1 | $(CGREP) "missing locale directory"
// This is a test which attempts to blow out the system limit with how many
// arguments can be passed to a process. This'll successively call rustc with
// larger and larger argument lists in an attempt to find one that's way too
// big for the system at hand. This file itself is then used as a "linker" to
// detect when the process creation succeeds.
// Tests are run in alphabetical order, and the second test is dependent on the
// first to set THREAD_ID. Do NOT rename the tests in such a way that `test_run_in_same_thread`
// would run before `spawn_thread_would_block`.
// See https://doc.rust-lang.org/rustc/tests/index.html#--shuffle
// A very specific set of circumstances (mainly, implementing Deref, and
// having a procedural macro and a Debug derivation in external crates) caused
// an internal compiler error (ICE) when trying to use rustdoc. This test
// reproduces the exact circumstances which caused the bug and checks
// that it does not happen again.
// See https://github.com/rust-lang/rust/issues/38237
A Glimpse
Rewriting 300+ Makefiles is, by all sane metrics, an exercise in tedium. What I did here was repetitive and somewhat detached from the "puzzly" part of programming that most nerds seem to name as the reason why they love what they do. According to everything I know about myself, I should have given up way earlier. I should have said "this sucks" or something like that. But I didn't. I actually liked doing it. I put on some dark ambient music albums, entered a trance state and just did it.
I announced at the beginning of the summer that I had a full time 40 hours/week internship that had unexpectedly sprung up. I completed both it and this open source project simultaneously. How I was able to find the time to fulfill my project and still work full time at a software company without getting burnt out of looking at computer screens will be left as an exercise to the reader.
I was assigned tasks, of course. The main one - avoiding excessive detail - was related to implementing an AI product into their application. The task itself probably has more technological complexity than the Makefiles, even though I have a sneaking suspicion that a sizable portion of people part of a community like Rust are going to be at the very least business-AI-skeptics.
The central point: even though it was more "crunchy" in raw programming terms, I found myself enjoying much more the Google Summer of Code to an extent that's not even comparable. And it's only now that I learn something important:
It's not about the tedium of the task, it's about the context of the task.
Pretty much everyone I talked to throughout the summer has been way more knowledgeable than me, and yet I still felt among equals. I wasn't working for them, I was working with them. At no point did my heart ache with the paranoia I associate with an authority figure prowling about the creaky floorboards of a cubicle maze.
At my "official" workplace, I saw the masquerade of meetings, the theatrics of project managers. I felt like in a playground where the children are replaced by adults pretending to play with the multicoloured slides and swings, their smiles saccharine and hollow.
So much of our world still seems ordered in masters and servants. But, this summer, I saw a glimpse of something different.
The next time a fellow undergraduate student imagines with glee their future in the forges of FAANG/MAMAA and asks me to reciprocate, I'll answer with respectful doubt instead of shared excitement.
Conclusion
I will be in person at Rustconf on September 11-12. I'll be accompanied by an amazing friend whom I'll be seeing in real life for the first time. My excitement is barely contained.
I went from not knowing Rust at all 1 year ago to having my life rewritten in it. Who could have thought the Rewrite It In Rust evangelists had their dominion extend even to people's lives?
If anyone reading this blog is coming to the conference in person, feel free to let me know!
Reddit: u/oneirical
Discord: oneirical