J@ArangoDB

{ "subject" : "ArangoDB", "tags": [ "multi-model", "nosql", "database" ] }

Less Intrusive Linking

A while ago our continuous integration builds on TravisCI began to fail seemingly randomly because the build worker was killed without an apparent reason. Obviously the build process reached some resource limits though we couldn’t find any documented limit that the build obviously violated.

Some builds still succeeded without issues, but those builds that were killed had one thing in common: they were all stuck waiting the linker to finish.

The default linker used on TravisCI is GNU ld. After some research, it turned out that replacing GNU ld with GNU gold not only made the linking much faster, but also less resource-intensive. Linking ArangoDB on my local machine is almost twice as fast with gold as with ld. Even better, after reconfiguring our TravisCI builds to also use gold, our builds weren’t killed anymore by TravisCI’s build scheduling system.

To make TravisCI use gold instead of ld, add the following to your project’s .travis.yml in the install section (so it gets execute before the actual build steps):

commands for wrapping gold
1
2
3
4
5
6
7
sudo apt-get -y install binutils-gold
mkdir -p ~/bin/gold
echo '#!/bin/bash' > ~/bin/gold/ld
echo 'gold "$@"' >> ~/bin/gold/ld
chmod a+x ~/bin/gold/ld
export CFLAGS="-B$HOME/bin/gold $CFLAGS"
export CXXFLAGS="-B$HOME/bin/gold $CXXFLAGS"

The script downloads and installs gold and creates a tiny wrapper script in a file named ld in the user’s home directory. The wrapper simply calls gold with all the arguments passed to the wrapper. Finally, the script modifies the environments CFLAGS and CXXFLAGS by setting the -B parameter to the wrapper script’s directory.

-B is the option for the compiler’s search path. The compiler (g++) at least will look in this path for any helper tools it invokes. As we have a file named ld in this directory, g++ will use our wrapper script instead of the original ld binary. This way we can keep the original version of ld in /usr/bin, and only override it using environment variables. This is also helpful in other contexts, e.g. when ld shall remain as the system’s default linker but goldshall only be used for linking a few selected components.