cruppstahl’s blog

Yesterday i was invited to give a short presentation about hamsterdb for OpenHUG (Hadoop User Group) in Munich.

And i really enjoyed this meeting – most of the audience was coming from the Web 2.0/Enterprise area, where i do not have that many insights.

The OpenHUG was organized by Bob Schulze – thanks for the invitation, and thank you for the beer

Here’s the presentation.

hamsterdb

chris Coding, hamsterdb

Release 1.1.2 is available for download!

New features

• Duplicate keys can now be sorted. Sorting has to be enabled with HAM_SORT_DUPLICATES. The default collation can be overwritten with ham_set_duplicate_compare_func.

Bugfixes

• If multiple Databases in an Environment are open, an open Transaction was not immediately closed when the first Database was closed (with ham_close)
• If multiple Databases in an Environment are open, and just one Database was closed, then the pages of this Database were not flushed correctly
• Fixed crash on Linux 64bit/gcc 4.4 with compiler optimization

Other Changes

• The documentation is now reduced to public interfaces; internal documentation is only built when requested
• A lot of refactoring and code improvements

Everything’s available on http://hamsterdb.com/download.

chris Coding, hamsterdb, Libraries

I like the title – “A Small Furry Database”

http://cplus.about.com/b/2009/12/15/a-small-furry-database.htm

chris Coding, hamsterdb

I received several reports that hamsterdb crashes on 64bit linux with gcc 4.4.1 on Ubuntu (but maybe not restricted to Ubuntu).

I did some tests with gcc and the following gcc versions work fine:

4.1.3 20080704 (prerelease) (Debian 4.1.2-27)

4.3.4 (Debian 4.3.4-6)

4.4.3 20100108 (prerelease) (Debian 4.4.2-9)

It seems that the problem is only in 4.4.1 and maybe 4.4.2. To find out if your gcc has the same issue, just make a clean build and start the unittest; they would crash more or less immediately:

./configure –enable-internal
make clean
make
make test

I’ll try to get more test results…

update (jan 17, 2010) – i fixed the issues. They were caused by gcc’s “strict antialiasing” which caused crashes when compiled with -O3.

chris Coding, hamsterdb

This release fixes a few minor bugs and adds a performance for In-Memory Databases: Users can now request a pointer to the stored record data by specifying the flag HAM_DIRECT_ACCES to ham_find, ham_cursor_find[_ex] and ham_cursor_move. Thus they can overwrite the record data without calling ham_insert or ham_cursor_overwrite. Also, a few flags were removed and the interface for ham_get_key_count was changed – please see the README.

As usual everything’s available for download here.

chris Coding, hamsterdb, Libraries

After months of work and receiving/integrating tons of patches, i’m happy that 1.1.0 is finally released!

I have increased the minor version because major new functionality was added and the database file format is not backwards compatible (however, 1.1.0 can read/write databases from 1.0.x).

Most of the patches were contributed by Ger Hobbelt (http://www.hobbelt.com, http://www.hebbut.net). Thank sa lot for the great work!

What’s new?

• “Approximate matching” – retrieves the nearest neighbour if a key is not found. Useful i.e. for time-based lookup (timestamp used as index)
• Major performance improvements, esp. in freelist management and in the caching
• Data Access Modes optimize runtime behaviour for performance depending on your Database characteristics (i.e. “read/write” vs “sequential inserts”)
• New functions to get runtime information (ham_env_get_parameter,  ham_get_parameter)
• A new function ham_get_key_count() returns the number of keys
• several bugfixes

As usual everything’s available for download here.

What’s on the plate for the next release? I have more patches from ger to merge, i will clean up/refactor some parts of the code and i want to give direct access to records in in-memory databases (means you can retrieve the pointer to the stored record instead of a copy, and you can directly modify/overwrite it).

The long-term plans for hamsterdb are to merge the functionality of hamsterdb Transactional Storage into hamsterdb Embedded Storage. I decided that i don’t have the resources to develop two different databases, and since Embedded Storage already has many users and i get frequent feedback it makes sense to concentrate my energy to it.

chris Coding, hamsterdb, Libraries

I’m very busy with merging changes and hundreds of patches, most sent by Ger. My estimation of “two weeks” in my last post was way too optimistic… family and work life are taking it’s toll. Another few weeks and i hope i’ll be done. Everything compiles now, but some unittests are still failing and valgrind detects some memory leaks. If someone wants to have an unstable tarball, just drop me a mail or pull it from svn://www.hamsterdb.org/home/chris/repos/hamsterdb/trunk.

chris Uncategorized

I’m really happy (and a little bit proud) that Tony Bain mentions hamsterdb in his newest post.

What else is on the plate? I’m currently merging lots of patches submitted by Ger Holland with bug fixes, performance improvements and new functionality (“Approximate Matching”) form hamsterdb Embedded Database. I hope we can make a release in the next two weeks.

Afterwards, i’ll work on the next release for hamsterdb Transactional Database – performance improvements and reclaiming deleted disk space.

chris Coding, hamsterdb

I’m really happy to announce the first release of hamsterdb Transactional Storage – an ACID compliant, concurrent key/value store.In a nutshell:

• ACID compliant transactions in a lock-free architecture; Transactions do not hold locks during their life-time. Conflicts are resolved in memory, without accessing the disk
• Fast; performance-relevant functions are moved to the background
• Logical, idempotent logging and Recovery
• Multithreaded; every handle can be used from arbitrary threads
• portable ANSI-C code (Linux/Unix/Win32)
• Easy to use

Early adaptors can download it here:

http://hamsterdb.com/public/dl/hamsterdb2-0.0.1.tar.gz

The linux/unix version is stable – but win32 still has some issues (although it works most of the time). This release is not intended for production use.

The next releases will improve performance, improve win32 and add more features. See the TODO file in the source package for more details.

I also wrote a Whitepaper to explain the architecture of hamsterdb Transactional Storage. It’s available for download here:

http://hamsterdb.com/public/dl/hamsterdb2_technical_overview.pdf

chris Coding, hamsterdb, Libraries

I decided to rename hamsterdb to “hamsterdb Embedded Storage” and hamsterdb2 to “hamsterdb Transactional Storage”. hamsterdb2 was never intended to replace the previous hamsterdb. Instead, both will coexist in parallel because they’re targeting completely different markets.

hamsterdb Embedded Storage concentrates on raw performance and a high level of configurability. It will run great on embedded devices and high-speed applications.

hamsterdb Transactional Storage is meant to run on modern hardware with multi-core CPUs, lots of RAM and is suited for applications which are in need of ACID compliant transactions.

I hope the new names will make it clear that both libraries will coexist and live on.

chris Uncategorized hamsterdb, Libraries