hamsterdb: next release comes soon…
I’m working hard on the next release. It will feature a couple of minor bug fixes and memory leaks, and significant performance improvements.
Most important changes:
- replaced linear search in pages with binary search
- completely rewrote the blob data management, which led to performance improvements of up to 300% for big blobs!
- completely rewrote the freelist – entries are now merged, which results in smaller files
I still have one big item on my todo list: make extended keys faster. Currently, if a key does not fit in the btree index, an overflow area is allocated (this is what i call “extended key”). I use the normal blob routines to store and load the overflow areas. I will add a cache for extended keys, then they don’t have to be fetched from disk whenever they are needed.
I have done a lot of profiling lately, and so far the improvements are awesome. Some of my unit tests, which ran up to 30 minutes, are now processed in 2 or 3 minutes.
I have also rewritten my test skripts. I have nearly 1 GB of test scripts, and they run about 30 to 60 minutes, depending on the configuration. One full test run of all scripts in all configurations (with/without mmap, as in-memory-db, with different cache sizes, page sizes, key sizes, with overwriting keys etc etc) takes nearly a day.
I expect that in three or four weeks I can finally release the new version.