Once you have set up a debug version of ArangoDB, it is quite helpful to also enable the Address Sanitizer and its companion tools.

These sanitizers provide runtime instrumentation for executables and check for common C++ programming errors such as buffer overflows, use-after-free bugs and memory leaks. The sanitizers supported natively by recent versions of g++ and clang++.

The general runtime overhead of the sanitizers is neglectable compared to other instrumentation tools such as Valgrind, so it can well be used in the day-to-day development process.

Configuring the build to use ASAN

To build with ASAN (Address Sanitizer) and UBSAN (Undefined Behavior Sanitizer), the following environment variables need to be set when invoking cmake:

1

CXXFLAGS="-fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fno-sanitize=vptr"

The full command to configure a debug build with ASAN and UBSAN support is:

1

(cd build && CXXFLAGS="-fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fno-sanitize=vptr cmake -DCMAKE_BUILD_TYPE=Debug -DUSE_MAINTAINER_MODE=On ..)

This will configure the build so it will pass the sanitizer options to the C++ compiler. Note that you’ll need a recent version of g++ or clang++ for this to work.

After that, build arangod normally. The binaries produced will be instrumented by the sanitizers, allowing many nasty memory errors to be detected very early.

Checking if an arangod binary is using ASAN

Whether or not an ArangoDB binary is instrumented can be found out at any time by calling the binary with the --version option. This will print some version information for ArangoDB itself and the other required libraries, and it will also print in the line starting with asan whether the binary was compiled with ASAN support or not:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

> build/bin/arangod --version

3.0.x-devel

architecture: 64bit
asan: true                       # !!!!
asm-crc32: true
boost-version: 1.61.0b1
build-date: 2016-06-09 12:07:07
compiler: gcc
cplusplus: 201103
endianness: little
fd-client-event-handler: poll
fd-setsize: 1024
icu-version: 54.1
jemalloc: false
libev-version: 4.22
maintainer-mode: true
openssl-version: OpenSSL 1.0.2g-fips  1 Mar 2016
rocksdb-version: 4.8.0
server-version: 3.0.x-devel
sizeof int: 4
sizeof void*: 8
sse42: true
tcmalloc: false
v8-version: 5.0.71.39
vpack-version: 0.1.30
zlib-version: 1.2.8

Controlling ASAN runtime behavior

ASAN behavior can also be controlled at runtime, after the binaries have been produced, by adjusting the environment variable ASAN_OPTIONS. The setting I use for this is:

1

ASAN_OPTIONS="handle_ioctl=true:check_initialization_order=true:detect_container_overflow=1:detect_stack_use_after_return=false:detect_odr_violation=1:allow_addr2line=true:strict_init_order=true"