Note: this post is about the ArangoDB 2.x series

Though we provide a lot of pre-built packages for the stable versions of ArangoDB here, it is often more interesting to play with the bleeding edge development version. New ArangoDB features are normally added to the devel branch, where they can be tested, documented and improved. When a feature matures, it is either backported to a stable branch or will eventually be released when the next stable branch is forked from devel.

Contributing to the core of ArangoDB is also much easier with a ready-to-go devel version. This post explains how to set one up from scratch.

The following instructions are for Ubuntu 14.04 LTS, which seems to be quite popular at the moment. Other flavors of Linux are probably quite similar, though package manager and packages names will likely be somewhat different.

Using Vagrant

If you don’t have an Ubuntu 14 installation yet, you can easily install one using Vagrant. If you happen to have a Linux installation already and are familiar with it, just skip this section.

After installing Vagrant on your system, pick a suitable Vagrant box from here. For example, I picked this 32 bit box from the list:

vagrant box add ubuntu-14.04-32 https://cloud-images.ubuntu.com/vagrant/trusty/current/trusty-server-cloudimg-i386-vagrant-disk1.box

After downloading the box, it can be made available via these commands:

mkdir temp
cd temp
vagrant init ubuntu-14.04-32
vagrant up

After the VM is booted, connect to it via SSH:

vagrant ssh

Cloning the repository

You’re now on the Ubuntu VM. Next step is fetch the ArangoDB source code from Github. Cloning the repository from there requires git. Let’s install it and clone the devel branch of the repository into a directory named devel on the VM:

sudo apt-get install git 
git clone -b devel https://github.com/arangodb/arangodb.git

The repository contains a lot of history so cloning may take a while. In case you don’t need the full history, you can create a shallow clone like this:

git clone -b devel --single-branch --depth 1 https://github.com/arangodb/arangodb.git 

This will reduce the download size from (currently) 375 MB to 56 MB and should be much faster. The downside of using a shallow copy is that there is no history and pushing and merging won’t work most of the time. So it’s better used for throw-away tests only.

Installing build tools and libraries

Now that the repository has been cloned into directory arangodb, we can install the required tools and libraries we need to build from source:

sudo apt-get install automake g++ libssl-dev libreadline-dev

If you prefer to install a different C++ compiler, please make sure it has proper support for C++11.

Go 1.4 is also required. The official list of downloadable Go versions can be found here. In the example, I am using the 32 bit version in this example:

wget https://storage.googleapis.com/golang/go1.4.2.linux-386.tar.gz
sudo tar -C /usr/local -xzf go1.4.2.linux-386.tar.gz
export PATH=$PATH:/usr/local/go/bin
echo "export PATH=\$PATH:/usr/local/go/bin" >> $HOME/.profile

Compiling ArangoDB

With all prerequisites set up, it’s now time to compile ArangoDB.

You probably noticed that no configure file is shipped with ArangoDB in the devel branch. To create it, we need to execute make setup once. After that, configure can be executed to create the Makefile. The Makefile finally contains the stuff that make needs:

make setup
./configure --enable-relative 
make

There first make run will take a while as it will compile all support libraries (ICU, V8, libev, zlib) before it will actually compile ArangoDB. Further invocations of make will not build these libraries again. Only any changed code will be rebuilt.

Note that make can be parallelized if you have multiple processors available. For 4 parallel make processes, use make -j4.

make will produce a lot of output. The most important information, whether or not an error occurred, can be found in its last line of its output. If it does not say something like this, make has probably succeeded:

make: *** [all] Error 2

You can also execute the following command directly after make to check the exit status of the make process:

echo $?

This will print 0 if make was successful, and a non-zero value otherwise.

Starting ArangoDB

When finished, make should have created all binaries in the bin subdirectory. We can now start arangod and the binaries directly from there without running a make install. In fact, make install is awkward to do if you do many change-compile-test cycles.

mkdir data          # creates a data directory
bin/arangod data    # starts the server

The server will be started as a foreground process (which is ideal when developing the server). To stop the server, simply press CTRL-C.

Connecting to ArangoDB

To verify ArangoDB is actually working, open a separate terminal and connect to it with the ArangoShell.

Note that if you used Vagrant, you will first need to connect to the Vagrant box in the other terminal using vagrant ssh from the directory you ran the vagrant init in. When connect to the Vagrant box, don’t forget to switch into the arangodb directory.

Once you’re in the correct directory, just issue this:

bin/arangosh

This should bring up the ArangoShell connected to your devel ArangoDB instance.

Making changes

Time to make some changes in the code. A good place to start is usually main. Here are a few places to get you started:

~/arangodb$ grep -r "int main" arangod/ arangosh/
arangod/RestServer/arangod.cpp:int main (int argc, char* argv[]) {
arangosh/Benchmark/arangob.cpp:int main (int argc, char* argv[]) {
arangosh/V8Client/arangorestore.cpp:int main (int argc, char* argv[]) {
arangosh/V8Client/arangodump.cpp:int main (int argc, char* argv[]) {
arangosh/V8Client/arangoimp.cpp:int main (int argc, char* argv[]) {
arangosh/V8Client/arangosh.cpp:int main (int argc, char* argv[]) {

Once you’re done with your changes, you need to re-compile and run:

make
bin/arangod data

Don’t worry, make will only recompile what you changed plus what depends on it and finally link it all together. This won’t take as long as on the previous run.

If you are serious about contributing to the server code, please let us know here so we can assist you.

Getting updates

We keep developing ArangoDB! To keep up to date and retrieve the latest changes from our repository, issue the following commands:

git pull origin devel
make

If make complains about files not found etc., the Makefile may have changed. Then it’s time for a cleanup:

make clean
make setup
./configure --enable-relative 
make

By the way, if you used special configure options and forgot them, you can retrieve your previous options by typing head config.log. Note: as of December 2014 (ArangoDB 2.4), most configure options starting with --enable-all-in-one- have been removed. If your config.log still contains them, you can safely omit them in any future invocations of configure.

Sometimes make clean isn’t enough. make creates cache files for the object files, and if they are too outdated, it will complain about files which have been renamed already etc. In this case, you can forcefully delete its caches by running an additional

make superclean

This also cleans already built libraries in the 3rdParty directory. It also removes the configure file, so after make superclean everything will need to be built from again. This may take a long time, so you should only run make superclean if after a make clean run there are still linker errors or complaints about missing files.

Debugging

If you are making changes in the ArangoDB C++ code and want to test them, it will be sensible to install debugging tools such as gdb and valgrind:

sudo apt-get install gdb valgrind   

Additionally, you may want to enable core files. This can be done by executing

ulimit -c unlimited

in the shell, and by putting the above line in your home directory’s .bashrc file to make it a permanent setting.

Additionally, you may want to set environment variables in your .bashrc that control compilation options. For development, I’d recommend using the following options:

export CFLAGS="-g -O0"
export CXXFLAGS="-g -O0"

This will turn off all optimizations and also generate debug symbols.

Enjoy!