Fjall, the Rust-based key-value store, has a new release. Version 3 features better performance for large data sets, improved memory usage, more extensive configuration, and a new disk format made for longevity and forwards compatibility, along with fully revamped APIs.

Fjall (which is nordic for mountain) is a log-structured, embeddable key-value storage engine written in Rust. It features a thread-safe BTreeMap-like API, and offers LSM-tree-based storage similar to RocksDB. It provides range and prefix searching with forward and reverse iteration; and multiple keyspaces (a.k.a. column families) with cross-keyspace atomic semantics. It has built-in compression, defaulting to LZ4. It also offers optional serializable transactions, and optional key-value separation for large blob use cases.

It is not a standalone database server, or a relational or wide-column database: it has no built-in notion of columns or query language.

While Fjall offers similar features to RocksDB, it has the added benefit of being written in Rust. The developer says that Fjall 3.0 has performance improvements that make it now very competitive. They say V3 is much more scalable than previous versions for large datasets and pretty comfortably beats sled and redb in most workloads.

As a key-value storage engine, Fjall is well suited for working with non-relational data, and with very large datasets where space and write amplification become important factors. The fact it has a full-Rust API without other language dependencies means it should appeal to developers who want a fast on-device state store without the need for a full database server when working with event queues, rolling telemetry buffers, or device inventories on devices such as the Raspberry Pi and embedded Linux appliances. Fjall uses log-structured merge (LSM)-trees, providing high random write throughput, and low disk space usage.

Fjall’s API is map-like, and supports multiple column-family keyspaces. The developer says Fjall is generally very similar to RocksDB architecturally; an LSM-tree with variable-sized pages (blocks) which can be optionally compressed, arranged into disjoint runs.

This release centers on a reworking of Fjall’s block format, though the work has extended to more or less the whole codebase. The old block format was very simple, offered fast reads in blocks, but this resulted in higher CPU usage and higher memory usages in workloads that are I/O heavy such as large data sets. Compression is now used by default when large values are written to the journal, and Fjall now makes use of the zlib-rs code.

Versioning support has also been added. The internal structure of the LSM-tree is now held in a (Super)Version, where a version is a point-in-time state of the LSM-tree’s tables (and blob files, if they exist). Whenever a compaction changes the level structure or a flush adds a new table, a new version is created copy-on-write-style. Alongside this change, value-log, which was a standalone crate for the... value log implementation, has been moved into lsm-tree, making it much easier to handle the list of blob files because they can be stored in the same version. The developers say this makes it easier to reason about operations that change blob files such as flushes and garbage collection.

Fjall 3 is available on GitHub now.

More Information

Fjall On GitHub

Related Articles

Microsoft Showcases Key Value Store

Microsoft Expands Redis Support

Apple Open Sources FoundationDB

RocksDB on Steroids

RocksDB - Facebook’s Database Now Open Source

To be informed about new articles on I Programmer, sign up for our weekly newsletter,subscribe to the RSS feed and follow us on Facebook or Linkedin.

Comments

or email your comment to: comments@i-programmer.info