I recently had to deal with some data processing operation that took about 20 minutes to complete. When looking into this, I found that the easiest and most beneficial change to the whole setup was to make the operation a server-side operation instead of executing it client-side.
This change reduced the operation’s total execution time to a few seconds.
Sometimes it is easier to make server-side operations run a bit faster. In the following post I’ll show a few low-level optimizations that can be applied to user-defined JavaScript code that is executed inside the ArangoDB server.
We’re often asked why ArangoDB uses JSON as its
data-interchange format for transferring documents from clients to
the database server and back. This is often accompanied by the
question if we could use <insert fancy format here> instead.
In the following article, I’ll try to outline the reasons for why we picked JSON as the interchange format, and why we still use it.
I’ll start with a discussion of the pros and cons of JSON, look at some alternative formats and present my personal opinion on why using a different format may not provide too much benefit for us, at least at the moment.
One of the major changes in ArangoDB 2.2 was the introduction of the write-ahead log (abbreviated WAL).
The introduction of the WAL changed how documents are stored internally in ArangoDB. A lot of things have been changed for it under the hood, and it has been a lot of work to implement it.
During the implementation, we refactored some code parts and made them considerably faster. From these changes we expected a positive effect on the database performance. But due to the fact that shape information is now also saved in the write-ahead log, there may also be some negative effect.
We developers were of course very interested in seeing the net effects, so we ran some tests for a few use cases. We compared ArangoDB 2.1.2 (still without the WAL) with ArangoDB 2.2.1 (with the WAL). The results are interesting.
Since version 2.2, ArangoDB stores all data-modification operations in its write-ahead log (abbreviated WAL). The introduction of the WAL massively changed how data are stored in ArangoDB.
Note: this post is about the ArangoDB 2.x series
In a relational database, all rows in a table have the same structure. The structure is saved once for the table, and the invidiual rows only contain the row’s values. This is an efficient approach if all records have the exact same structure, i.e. the same attributes (same names and same data types).