cruppstahl’s blog

I like the title - “A Small Furry Database”

http://cplus.about.com/b/2009/12/15/a-small-furry-database.htm

chris Coding, hamsterdb

I received several reports that hamsterdb crashes on 64bit linux with gcc 4.4.1 on Ubuntu (but maybe not restricted to Ubuntu).

I did some tests with gcc and the following gcc versions work fine:

4.1.3 20080704 (prerelease) (Debian 4.1.2-27)

4.3.4 (Debian 4.3.4-6)

4.4.3 20100108 (prerelease) (Debian 4.4.2-9)

It seems that the problem is only in 4.4.1 and maybe 4.4.2. To find out if your gcc has the same issue, just make a clean build and start the unittest; they would crash more or less immediately:

./configure –enable-internal
make clean
make
make test

I’ll try to get more test results…

update (jan 17, 2010) - i fixed the issues. They were caused by gcc’s “strict antialiasing” which caused crashes when compiled with -O3.

chris Coding, hamsterdb

This release fixes a few minor bugs and adds a performance for In-Memory Databases: Users can now request a pointer to the stored record data by specifying the flag HAM_DIRECT_ACCES to ham_find, ham_cursor_find[_ex] and ham_cursor_move. Thus they can overwrite the record data without calling ham_insert or ham_cursor_overwrite. Also, a few flags were removed and the interface for ham_get_key_count was changed - please see the README.

As usual everything’s available for download here.

chris Coding, Libraries, hamsterdb

After months of work and receiving/integrating tons of patches, i’m happy that 1.1.0 is finally released!

I have increased the minor version because major new functionality was added and the database file format is not backwards compatible (however, 1.1.0 can read/write databases from 1.0.x).

Most of the patches were contributed by Ger Hobbelt (http://www.hobbelt.com, http://www.hebbut.net). Thank sa lot for the great work!

What’s new?

• “Approximate matching” - retrieves the nearest neighbour if a key is not found. Useful i.e. for time-based lookup (timestamp used as index)
• Major performance improvements, esp. in freelist management and in the caching
• Data Access Modes optimize runtime behaviour for performance depending on your Database characteristics (i.e. “read/write” vs “sequential inserts”)
• New functions to get runtime information (ham_env_get_parameter,  ham_get_parameter)
• A new function ham_get_key_count() returns the number of keys
• several bugfixes

As usual everything’s available for download here.

What’s on the plate for the next release? I have more patches from ger to merge, i will clean up/refactor some parts of the code and i want to give direct access to records in in-memory databases (means you can retrieve the pointer to the stored record instead of a copy, and you can directly modify/overwrite it).

The long-term plans for hamsterdb are to merge the functionality of hamsterdb Transactional Storage into hamsterdb Embedded Storage. I decided that i don’t have the resources to develop two different databases, and since Embedded Storage already has many users and i get frequent feedback it makes sense to concentrate my energy to it.

chris Coding, Libraries, hamsterdb

I’m very busy with merging changes and hundreds of patches, most sent by Ger. My estimation of “two weeks” in my last post was way too optimistic… family and work life are taking it’s toll. Another few weeks and i hope i’ll be done. Everything compiles now, but some unittests are still failing and valgrind detects some memory leaks. If someone wants to have an unstable tarball, just drop me a mail or pull it from svn://www.hamsterdb.org/home/chris/repos/hamsterdb/trunk.

chris Uncategorized

I’m really happy (and a little bit proud) that Tony Bain mentions hamsterdb in his newest post.

What else is on the plate? I’m currently merging lots of patches submitted by Ger Holland with bug fixes, performance improvements and new functionality (”Approximate Matching”) form hamsterdb Embedded Database. I hope we can make a release in the next two weeks.

Afterwards, i’ll work on the next release for hamsterdb Transactional Database - performance improvements and reclaiming deleted disk space.

chris Coding, hamsterdb

I’m really happy to announce the first release of hamsterdb Transactional Storage - an ACID compliant, concurrent key/value store.In a nutshell:

• ACID compliant transactions in a lock-free architecture; Transactions do not hold locks during their life-time. Conflicts are resolved in memory, without accessing the disk
• Fast; performance-relevant functions are moved to the background
• Logical, idempotent logging and Recovery
• Multithreaded; every handle can be used from arbitrary threads
• portable ANSI-C code (Linux/Unix/Win32)
• Easy to use

Early adaptors can download it here:

http://hamsterdb.com/public/dl/hamsterdb2-0.0.1.tar.gz

The linux/unix version is stable - but win32 still has some issues (although it works most of the time). This release is not intended for production use.

The next releases will improve performance, improve win32 and add more features. See the TODO file in the source package for more details.

I also wrote a Whitepaper to explain the architecture of hamsterdb Transactional Storage. It’s available for download here:

http://hamsterdb.com/public/dl/hamsterdb2_technical_overview.pdf

chris Coding, Libraries, hamsterdb

I decided to rename hamsterdb to “hamsterdb Embedded Storage” and hamsterdb2 to “hamsterdb Transactional Storage”. hamsterdb2 was never intended to replace the previous hamsterdb. Instead, both will coexist in parallel because they’re targeting completely different markets.

hamsterdb Embedded Storage concentrates on raw performance and a high level of configurability. It will run great on embedded devices and high-speed applications.

hamsterdb Transactional Storage is meant to run on modern hardware with multi-core CPUs, lots of RAM and is suited for applications which are in need of ACID compliant transactions.

I hope the new names will make it clear that both libraries will coexist and live on.

chris Uncategorized hamsterdb, Libraries

Currently a lot of brouhaha is generated around NoSQL databases. If you’re interested, here are some meeting points:

NoSQL LinkedIn Group: http://www.linkedin.com/groups?home=&gid=2085042&trk=anet_ug_hm

NoSQL Google Group: http://groups.google.com/group/nosql-discussion

Highavailability.com featuring this week’s best blog entry title: “It must be crap on relational databases week”:

http://highscalability.com/it-must-be-crap-relational-dabases-week

http://highscalability.com/no-sql-anti-database-movement-gains-steam-my-take

And from the other side - Curt Monash on NoSQL (BI point of view): http://www.dbms2.com/2009/07/01/nosql-sql-alternative/

chris Uncategorized

I just found a really cool trick to get a list of all predefined gcc macros:

echo | gcc -dM -E -

Found it here: http://www.willus.com/mingw/_predefined.shtml

chris Coding