Archive

Archive for the ‘Libraries’ Category

1.1.10 has issues…

June 28th, 2011

… related to performance – if you use Transactions, then there are some unnecessary fsync/fdatasync calls, slowing down performance by 30%.

I will release a fixed version (1.1.11) tonight.

And i’ll extend my test suite to have a poor man’s performance tracking.

chris Coding, Databases, hamsterdb, Libraries

hamsterdb: release 1.1.10

June 25th, 2011

From the README:

This release fixes a bug in the cache, which caused the cache to grow and exceed the cache limits. A few other minor bugs were fixed, and the recovery process was improved: changes in the header page (i.e. when adding new Databases or when changing the address of a root page) were not correctly logged. The flag HAM_WRITE_THROUGH has a new meaning: instead of immediately flushing all pages it will now flush all file handles after a Transaction was committed or aborted. This has a performance impact, but improves the Durability of the Transactions. Sources, CHANGELOG and precompiled win32 libraries are available for download here!

This release was long overdue. There were some bugs reported that were hard to reproduce and to track down, family also required more time than usual, and in the same time i was also working on the first release of 2.0.0 (rc1), which will implement the new transaction handling. I hope i can release it in the next few weeks – i have to port a couple of patches from 1.1.10 to the new branch and also make sure that everything compiles and works on Windows.  The first release will be unstable – there were so many code changes and the test coverage for some parts of the new code is still not satisfying. In addition i have some ideas how to boost performance, but so far i had no time to implement them.

Stay tuned :)

chris Coding, Databases, hamsterdb, Libraries

A short status update

April 6th, 2011

The next release is due, but it will week another 2 weeks. The last release was about 6 weeks ago, and my release cycle is usually not more than 4-6 weeks. There were some bug reports for 1.1.9 – but nothing critical, therefore I’m not seeing a need to release a 1.1.10 immediately.

But since 1.1.9 is the last stable release, and the new transaction implementation (which will be 1.2.0) will be unstable for the next few months, there definitely WILL be a 1.1.10 coming soon.

In the meantime let me give you an update about the re-implementation of the Transaction implementation, which was already described here.

A quick reminder about the new functionality:

  • You can have as many parallel Transactions as you want
  • Transactions are stored in RAM in a tree structure
  • For recovery, they are also written to a journal (and right now also to a physical log, which will be removed in the release after)
  • When Transactions are committed, they’re flushed to the disk. When they’re aborted then they’re just removed from the in-memory structures.

At runtime, hamsterdb consolidates the two trees (the Transaction tree in RAM and the BTree on disk). This is a complex task, especially when traversing the Database with a cursor. And things get even more complex when the Database has duplicate keys, because duplicate keys are not necessary inserted in the same order as they’re stored in the Database.

Right now most of the duplicate functionality is implemented and working fine. A few things are still missing, and should be ready in about 2 weeks or so.

After the first (unstable) release, there are a couple of things that are in urgent need of attention.

  • The test coverage and automization has to be improved
  • The high-level APIs (Java, .NET etc) also need some love
  • Improve performance by getting rid of the physical log file
  • Flush committed Transactions to disk in the background
  • etc, etc, etc…

chris Coding, hamsterdb, Libraries

hamsterdb on iPhone and Android

April 6th, 2011

I received two mails regarding support of Android and iOS. And hamsterdb works on both of them.

I even discovered a github project about Android support:

https://github.com/glacefullife/Hamster-DB-for-Android

This reminds me that the Java API (just like Python and .NET) was terribly neglected. Shame on me. My plans for the high-level APIs are to fully automize their releases. With every new release, the high-level APIs will automatically be built and tested. I plan to create the necessary infrastructure shortly after the next big release with the new Transaction code. I’ll write another blogpost about the current status and the next steps later today.

And finally, a screenshot of an app based on hamsterdb running on iOS. Thanks for sending, SSB! If someone is interested in the xcode project then please drop me a mail.

hamsterdb on iOS

an iPhone/iOS app powered by hamsterdb

chris Coding, hamsterdb, Libraries

hamsterdb: release 1.1.9!

February 18th, 2011

This release fixes a bug in the Transaction handling, and an out-of-memory condition with long-running Transactions. It fixes several other issues detected by static code analysis tools. A .spec file was added for RPM generation. A 64bit incompatibility was fixed in the remote functions/hamserver. Sources, CHANGELOG and precompiled win32 libraries are available for download here:

http://hamsterdb.com/download

The new transaction implementation is still in development; right now the only missing big piece is the support for duplicate keys. Especially the implementation of the cursors will be tricky and take a few weeks. After that some cleanup is required and then the first (unstable) release will be available!

chris Coding, hamsterdb, Libraries

hamsterdb: release 1.1.8!

December 6th, 2010

1.1.8 is a bugfix release. The following issues were fixed:

  1. bugfix with HAM_DIRECT_ACCESS and cursors which insert duplicates at the beginning of the duplicate list; if record size is <= 8 bytes then invalid data was returned. Parts of this bug were fixed in 1.1.7.
  2. bugfix in transaction handling – in certain cases with long running transactions, a circular reference in a linked list causes an endless loop (thanks, Mark)
  3. configure.in will no longer overwrite -Ox flags; the default compilation option is now -O2

Files are here: http://hamsterdb.com/download!

chris hamsterdb, Libraries

hamsterdb: release 1.1.7!

November 4th, 2010

This release fixes a small memory leak when using remote transactions. It fixes HAM_DIRECT_ACCESS with records <= 8 bytes. And it implements some significant performance improvements.

- the shutdown time improved a lot when using many extended keys

- when appending (or prepending) keys to a database then most inserts are now extremely fast in constant time

As always, you can get everything from here: http://hamsterdb.com/download.

The new transactional code is not yet included.

chris Coding, hamsterdb, Libraries

hamsterdb: release 1.1.6!

September 17th, 2010

This release stabilized the remote functionality. New samples for client/server scenarios are provided. The remote server can now write access logfiles. A unix daemon and a win32 Service were added. The dependency to Google Protocol Buffers for C was removed; instead, Google Protocol Buffers (C++) are now used. Compilation on ArchLinux (gcc 4.5.1) was fixed. Several functions are now deprecated. The source repository was moved to github. Sources, CHANGELOG and precompiled win32 libraries are available for download here!

chris Coding, hamsterdb, Libraries

hamsterdb: release 1.1.5 UNSTABLE

July 25th, 2010

This release implements remote functionality over http. The protocol is implemented with Google’s Protocol Buffers, the client uses libcurl for network handling and the server is based on mongoose, an embedded micro http server.
The server interface is very simple, and the server can easily be embedded into an application.

There are no known bugs, but currently the remote functionality is only supported on Unix/Linux. The next version 1.1.6 will also support the new features on Win32; several other improvements are also pending for the next release: a unix daemon, a win32 service and some other minor issues.

Download the sources at http://hamsterdb.com/download.

chris Coding, hamsterdb, Libraries

hamsterdb-dotnet: release 0.0.3

April 19th, 2010

… is available for download, including most of the functionality in hamsterdb 1.1.4.

chris Coding, hamsterdb, Libraries