Objective: mobile, offline first storage choices for an event storage system.
I started to write a storage engine building my on file storage with reads and writes routed from my own memtable to sstable, using mmap + my own event loop rolled.
I realized that it was too complex, it worked but I needed secondary indexing etc. to support a lot of practical usecases, a problem that had long been solved.
I then moved to LMDB, it does work and is quick, however mmap has some issues when dealing with iOS and ipad and many other things for example: the unsafe code for a new into Rust guy like me slows down my development much much faster. RocksDB was another choice and so was LevelDB but leveldb I had heard from anecdotal evidence that crashes a lot
I pivoted to SQLLite - things were so …
Objective: mobile, offline first storage choices for an event storage system.
I started to write a storage engine building my on file storage with reads and writes routed from my own memtable to sstable, using mmap + my own event loop rolled.
I realized that it was too complex, it worked but I needed secondary indexing etc. to support a lot of practical usecases, a problem that had long been solved.
I then moved to LMDB, it does work and is quick, however mmap has some issues when dealing with iOS and ipad and many other things for example: the unsafe code for a new into Rust guy like me slows down my development much much faster. RocksDB was another choice and so was LevelDB but leveldb I had heard from anecdotal evidence that crashes a lot
I pivoted to SQLLite - things were so simple after that. But I am not set on using sqllite, I want to try other options as well
BTW: I only started Rust recently and still reading books and doing so please excuse me if this type of question is silly for Rustaceans.
Can someone point me to a place to look at when looking at storage engine choices for tiny dbs:
write amplification 1.
read amplification 1.
SSD wear and tear. 1.
Concurrency support, how tokio plays into it and how threads can be used/ 1.
support for aligned zero copy reads.
I used rkyv and bytemuck, pretty happy with those two.