What is Tarantool?
Tarantool is a NoSQL database running and a Lua application server. The code is available for free under the terms of BSD license. Supported platforms are GNU/Linux, Mac OS and FreeBSD.
An overview of the architecture
The server maintains all its data in random-access memory, and therefore has very low read latency. At the same time, a copy of the data is kept on non-volatile storage (a disk drive), and inserts and updates are performed atomically.
To ensure atomicity, consistency and crash-safety of the persistent copy, a write-ahead log (WAL) is maintained, and each change is recorded in the WAL before it is considered complete. The logging subsystem supports group commit.
If update and delete rate is high, a constantly growing write-ahead log file
(or files) can pose a disk space problem, and significantly increase time
necessary to restart from disk. A simple solution is employed: the server
can be requested to save a concise snapshot of its current data. The
underlying operating system's "
copy-on-write" feature is employed to take
the snapshot in a quick, resource-savvy and non-blocking manner. The
copy-on-write" technique guarantees that snapshotting has minimal impact
on server performance.
Tarantool is lock-free. Instead of the operating system's concurrency primitives, such as threads and mutexes, Tarantool uses a cooperative multitasking environment to simultaneously operate on thousands of connections. A fixed number of independent execution threads within the server do not share state, but exchange data using low overhead message queues. While this approach limits server scalability to a few CPU cores, it removes competition for the memory bus and sets the scalability limit to the top of memory and network throughput. CPU utilization of a typical highly-loaded Tarantool server is under 10%.
Unlike most of NoSQL databases, Tarantool supports primary, secondary keys, multi-part keys, HASH, TREE and BITSET index types.
Tarantool supports Lua stored procedures, which can access and modify data atomically. Procedures can be created, modified and dropped at runtime.
Use of Lua as an extension language does not end with stored procedures: Lua programs can be used during startup, to define triggers and background tasks, interact with networked peers. Unlike popular application development frameworks implemented around "reactor" pattern, networking in server-side Lua is sequential, yet very efficient, as is built on top of the cooperating multitasking environment used by the server itself.
Extended with Lua, Tarantool typically replaces more not one but a few existing components with a single well-performing system, changing and simplifying complex multi-tier Web application architectures.
Tarantool supports replication. Replicas may run locally or on a remote host. Tarantool replication is asynchronous and does not block writes to the master. When or if the master becomes unavailable, the replica can be switched to assume the role of the master without server restart.
How stable is the software?
The software is production-ready. Tarantool has been created and is
actively used at Mail.Ru, one of the leading Russian web content providers.
At Mail.Ru, the software serves the "
hottest" data, such as online users
and their sessions, online application properties, mapping between users
and their serving shards, and so on.
Outside Mail.Ru the software is used by a growing number of projects in online gaming, digital marketing, social media industries. While product development is sponsored by Mail.Ru, the roadmap, bugs database and the development process are fully open. The software incorporates patches from dozens of community contributors, and most of the programming language drivers are written and supported by the community.