Another stabilization release of the 2.0.0 branch – the last few known issues were fixed, and performance was improved.
The next release might already be the final 2.0.0 release. It will feature more performance improvements (i will get rid of the physical log for most of the transactional operations), more refactoring and most likely some small API changes. The file format will continue to be compatible to 1.x and the ABI will also be compatible (the header file will change, the library interface not.)
Everything’s available for download, including precompiled win32 libraries.
http://hamsterdb.com/download.
Even when 2.0 is stable i will continue to maintain 1.x over the next few years (at least).
chris Coding, hamsterdb, Libraries
This is a minor bug fix release.
The bug was a type conversion of the cache size to a 32bit value. If you use cache sizes > 2 GB then you should upgrade. If not then you can skip this release.
Get the new files here: here.
chris Coding, Databases, hamsterdb, Libraries
This is the latest stable release from the 1.1.x branch.
This release improves performance and scalability for databases with several million items by avoiding two expensive memcpy’s and a linked list traversal when aborting or committing transactions. If you do not use transactions then don’t bother, but if you do (and your database is big enough) then you should see a significant performance gain.
This release also fixes a bug when erasing extended keys caused an SMO (btree re-balancing). This bug was present since the beginning of time and it took 6 years till it was noticed.
A new function ham_cursor_get_record_size was introduced. The cache size is now a 64bit variable and can be > 4 GB. Large files were enabled on 32bit linux.
All listed issues were reported by (mostly) commercial licensees, including code to reproduce. Thanks for all the contributions!
As usual, everything’s available for download here: http://hamsterdb.com/download.
chris Coding, Databases, hamsterdb, Libraries
This is the first stabilization release of the new (unstable) 2.0.0 branch which re-implemented the Transaction handling. Several bugs were fixed, and most (all?) of the code was refactored. A C++ compiler is now required to compile hamsterdb. All new features and bug fixes of 1.1.14 (will be released soon) were merged. This release is not ready for production! There are still two tests that currently fail and fixing these requires me to rewrite most of ham_cursor_move. Also, some of the Transaction routines (esp. in combination with extended keys and/or duplicates) have performance issues. There will be at least two more stabilization releases till 2.0.0 is ready for production.
Sources and CHANGELOG are available for download here!
chris Coding, Databases, hamsterdb, Libraries
Another bugfix release on the “stable” development line. From the readme:
This release fixes a performance regression when aborting Transactions. During abort, hamsterdb tried to re-create pages from the transaction log, even if they were not modified in the current Transaction. This was correct but unnecessary and caused a performance drop.
Tarball and precompiled win32 binaries are available for download.
chris Coding, Databases, hamsterdb, Libraries
Done!! After about 10 months i finally finished re-implementing the Transaction support.It’s a bit ambitious to call it 2.0.0, but i think the new functionality is justifying a major version increase.
So what’s so great about the new release? So far not much. You can create an unlimited number of parallel Transactions, and they will not block each other. The isolation level is “READ COMMITTED”. But the changes have been so big that i cannot guarantee as much stability as i’d like to do. And actually there are some known issues – a couple of tests are currently failing and performance is slow. Why did i still release? The failing tests show some weaknesses of the current implementation. Some routines have to be rewritten from scratch, and some modules need serious refactoring. But after 10 months i was really eager to release this beast and i really did not want to wait another 4 or 8 weeks.
The failing issues are described in the README file.
If you want to get an idea of the changes then look at the TODO list. It’s long. Really long.
Please treat this release as a “technical preview”. It will require one or two more releases till it’s ready for production. But comments are VERY welcome!
chris Coding, Databases, hamsterdb, Libraries
History repeating – another bugfix release was necessary.
1.1.11 had an issue in the cache purge routines – they were called, but they never completed successfully. This is now fixed – the memory consumption is now stable.
In addition, i exported a few internal functions that help me keep track of memory consumption, I/O behavior and performance. Every acceptance test will now create metrics like the following:
[OK] ../../testfiles/1/200.tst
perf-misc 0.010000 sec
perf-insert 0.048000 sec
perf-erase 0.008000 sec
perf-find 0.000000 sec
perf-cursor 0.000000 sec
perf-total 0.066000 sec
mem-num-allocs 400
mem-peak-bytes 84913 bytes
mem-total-bytes 86449 bytes
io-num-flushes 2
io-num-read 0
io-num-read-page 1
io-num-write 0
io-num-write-page 8
io-num-alloc 0
io-num-alloc-page 6
In total there are more than 45000 tests. A few perl scripts are helping me to pick out tests that have regressions. I hope that will avoid the necessity for releases like this one and the two previous ones.
All files are available for download here!
chris Coding, Databases, hamsterdb, Libraries
This is a bugfix release for 1.1.10.
When using transactions, 1.1.10 would sometimes flush the file handles even if HAM_WRITE_THROUGH is not enabled. This caused a performance regression of up to 30%.
In addition to this bugfix, i removed the ./configure switch “–enable-internal”. It was a relict from older times and did not make sense anymore. All “internal” functions (i.e. ham_check_integrity) are now always compiled.
In addition, my acceptance test suite (which was extended to ~ 45000 tests in the meantime) is now also tracking the performance. It already had profiling built-in, but the performance results were not stored. Such a regression should hopefully never happen again.
All files are available for download here!
chris Coding, Databases, hamsterdb, Libraries
… related to performance – if you use Transactions, then there are some unnecessary fsync/fdatasync calls, slowing down performance by 30%.
I will release a fixed version (1.1.11) tonight.
And i’ll extend my test suite to have a poor man’s performance tracking.
chris Coding, Databases, hamsterdb, Libraries
From the README:
This release fixes a bug in the cache, which caused the cache to grow and exceed the cache limits. A few other minor bugs were fixed, and the recovery process was improved: changes in the header page (i.e. when adding new Databases or when changing the address of a root page) were not correctly logged. The flag HAM_WRITE_THROUGH has a new meaning: instead of immediately flushing all pages it will now flush all file handles after a Transaction was committed or aborted. This has a performance impact, but improves the Durability of the Transactions. Sources, CHANGELOG and precompiled win32 libraries are available for download here!
This release was long overdue. There were some bugs reported that were hard to reproduce and to track down, family also required more time than usual, and in the same time i was also working on the first release of 2.0.0 (rc1), which will implement the new transaction handling. I hope i can release it in the next few weeks – i have to port a couple of patches from 1.1.10 to the new branch and also make sure that everything compiles and works on Windows. The first release will be unstable – there were so many code changes and the test coverage for some parts of the new code is still not satisfying. In addition i have some ideas how to boost performance, but so far i had no time to implement them.
Stay tuned 
chris Coding, Databases, hamsterdb, Libraries