J@ArangoDB

{ "subject" : "ArangoDB", "tags": [ "multi-model", "nosql", "database" ] }

Using Ccache When Working With Different Branches

Git makes working with many different branches in the same local repository easy and efficient.

In a C/C++ project, the code must be re-compiled after switching into another branch. If the branches only differ minimally, running make will only re-compile the parts that are actually different, and after that re-link them. That won’t take too long, though especially link times can be annoying.

However, if there are differences in central header files that are included from every file, then make has no option but to re-compile everything. This can take significant amounts of time (and coffee).

I just realized that there is a solution to speed up re-compilation in this situation: ccache!

Why ccache can help

ccache is a wrapper for the actual compiler command. It will call the compiler with the specified arguments, and capture the compiler output. When called again with the same arguments, it will look in its internal cache for a ready-to-serve result. If one is present, it will return it without invoking the compiler again. Otherwise, or if it detects some changes that forbid serving outdated results from the cache, it will transparently invoke the compiler.

When switching back to another branch that you had already compiled before, running make may re-build everything due to changes in headers. But it is not unlikely that you had built the branch before already. If so, and ccache was involved in the previous build, it may still have all the info required for re-compilation in its cache.

And everyone will be happy: make will run its full rebuild, but most operations won’t be handed to the compiler because ccache is sitting in between, serving results from its cache. And you as a developer won’t lose that much time.

Some figures

Following are some figures demonstrating its potential when running a make in the devel branch after having returned from a different branch with significant changes.

With ccache, but cache empty

time make
1
2
3
real  12m43.501s
user  11m52.550s
sys 0m44.110s

With ccache, everything in cache

time make
1
2
3
real  0m55.572s
user  0m26.346s
sys 0m7.551s

That’s a build time reduction of more than 90 %!

This is already the optimal result, as everything was already present in the cache. However, the situation was not unrealistic. I often switch into another branch, try something out or commit a small change, and the return to the original branch. I already started having many separate directories for the different branches to avoid frequent recompilation. ccache can be relief here.

By the way, timing results are from my laptop. I did not bother to run make with parallel jobs as this has limited effect on my laptop, though on more decent hardware it may be beneficial both with and without ccache, though I guess, with many parallel jobs and a full cache, linking will become the most expensive part.

How to use ccache

For Ubuntu, ccache is available in package ccache. You can easily install it with:

Installing ccache on Ubuntu
1
sudo apt-get install ccache

The most convenient way to use ccache in your build is to change your CC and CXX environment variables as follows:

setting compilers environment variables
1
2
export CC="ccache gcc"
export CXX="ccache g++"

I suggest putting that into .bashrc so the variables will be set in every session and not just once. After that, running configure will write a Makefile that will use ccache for building object files.

Note: that will change these environment variables globally, so ccache may be used for other projects, too.

What ccache cannot do

I already forgot about ccache because when working in a single branch it does not provide that many benefits. When making changes to your code, you can be pretty sure the new code won’t be in the cache yet. Running make then will invoke ccache, but this will result in a cache miss. It cannot help here, because the new code was never compiled before and thus in no cache.

Additionally, make is smart enough on its own to only re-build the parts of the program that have actually changed or depend on the changes you made.