More than one way to skin the Database Cat

I’m in the planning stages of a Perl application where I need some sort of database engine back-end with pretty basic requirements. I’ve surprised myself in picking what, today, has become an unconventional choice; DBM. For long time Unix practitioners, DBM is well known and there have been many re-implementations of the original idea (NDBM, SDBM, GDBM, etc.). Whatever the iteration, the basic idea is the same; a library of routines that are loaded into the application’s own address space and provides a basic key/value based mechanism for storing and retrieving records. See Wikipedia DBM article for a brief explanation and history of DBM.

Of course many application developers have forgotten all about DBM, assuming that it is an obsolete technology. The assumption today has pretty much become that the database back-end engine will of course be a relational DB with an SQL query language interface. But between applications that need all the power and features of a full-blown relational database, and those that only need low-level read/write operations of general purpose file system, there is a middle ground that a DBM-like database engine fills very nicely.

As it turns out DBM style databases are not dead at all but are actively being developed. QDBM (Quick DataBase Manager) is just one of several modern DBM-like open source database libraries available for deployment. The QDBM web page graciously lists a short description of some of its “brothers” where author Mikio Hirabayashi writes:

There are many followers of UNIX DBM. Select the best suited one for your products. NDBM is ancient and you should not use it. SDBM is maintained by Apache Project, and GDBM is maintained by GNU Project. They are most popular and time-tested. TDB is maintained by Samba Team. It allows multiple simultaneous writers. While CDB does not support updating at a runtime, it is the fastest. Berkeley DB is very multifunctional and ACID compliant. It is used in many commercial products. Finally, QDBM is balanced of performance, functionality, portability, and usability.

For my own project I’ve decided to give Berkeley DB a try. It appears to be feature rich, well supported, and perhaps most importantly, is already installed on the system where I will be deploying my application. :-)

For more information about the DBM approach to database management for applications, and about Berkeley DB in particular, I recommend checking out the first chapter of the Berkeley DB Programmer’s Reference Guide. It’s an interesting read and spells out the case for when a DBM style database is, and is not a good fit an application.

5 Responses to “More than one way to skin the Database Cat”

  1. Bill Costa says:

    The first time I visited the QDBM web page, the title was:

    “QDBM: Quantumtheoretically Dangerous Blast Machine”

    but now it is just:

    “QDBM: Quick Database Manager”

    Right now I’ve actually got both versions of the page in different browser tabs; the URLs are identical and title is the only difference I can see between the two versions of the page. Even the time stamp shown on the page is identical. I’ve refreshed the page many times but have failed to get the humorous title to re-appear. If I didn’t still have the funny version of the page in my browser I would doubt myself as to actually having seen it.

  2. Paul Sand says:

    Not to mention that it’s really, really easy to tie a database to a Perl hash with the BerkeleyDB module. Who needs SQL? CIS Unix user databases have been handled by Berkeley DB files for years; although I wouldn’t hold the system up as a model for software design, it’s been very solid and (cross fingers) reliable. (“We haven’t deleted users by accident for… well, it’s been weeks now!”)

  3. Paul Sand says:

    Oh, yeah: viewing the QDBM page‘s source code shows the Javascript behind the title switch. (There’s more than one.)

  4. Bill Costa says:

    “…source code shows the Javascript…”

    Use the source, Luke! (Says the Jedi Code Master)

  5. Kirit Basu says:

    SQLite ( is quite an amazing embedded database system; fully transactional, has plugs from most well known programming languages, amazingly lightweight and fast (can also run in-memory).
    be sure to check out the faq about how to think about concurrency though.

    also check out if you want to blow your mind about how they do testing.

Leave a Reply

Panorama theme by Themocracy