I’m in the planning stages of a Perl application where I need some sort of database engine back-end with pretty basic requirements. I’ve surprised myself in picking what, today, has become an unconventional choice; DBM. For long time Unix practitioners, DBM is well known and there have been many re-implementations of the original idea (NDBM, SDBM, GDBM, etc.). Whatever the iteration, the basic idea is the same; a library of routines that are loaded into the application’s own address space and provides a basic key/value based mechanism for storing and retrieving records. See Wikipedia DBM article for a brief explanation and history of DBM.

Of course many application developers have forgotten all about DBM, assuming that it is an obsolete technology. The assumption today has pretty much become that the database back-end engine will of course be a relational DB with an SQL query language interface. But between applications that need all the power and features of a full-blown relational database, and those that only need low-level read/write operations of general purpose file system, there is a middle ground that a DBM-like database engine fills very nicely.

As it turns out DBM style databases are not dead at all but are actively being developed. QDBM (Quick DataBase Manager) is just one of several modern DBM-like open source database libraries available for deployment. The QDBM web page graciously lists a short description of some of its “brothers” where author Mikio Hirabayashi writes:

There are many followers of UNIX DBM. Select the best suited one for your products. NDBM is ancient and you should not use it. SDBM is maintained by Apache Project, and GDBM is maintained by GNU Project. They are most popular and time-tested. TDB is maintained by Samba Team. It allows multiple simultaneous writers. While CDB does not support updating at a runtime, it is the fastest. Berkeley DB is very multifunctional and ACID compliant. It is used in many commercial products. Finally, QDBM is balanced of performance, functionality, portability, and usability.

For my own project I’ve decided to give Berkeley DB a try. It appears to be feature rich, well supported, and perhaps most importantly, is already installed on the system where I will be deploying my application.

For more information about the DBM approach to database management for applications, and about Berkeley DB in particular, I recommend checking out the first chapter of the Berkeley DB Programmer’s Reference Guide. It’s an interesting read and spells out the case for when a DBM style database is, and is not a good fit an application.

Just read an unexpected article by Garrison Keillor (American author and humorist) that pays tribute to the accomplishments of scientists, inventors, nerds in general, and in particular, to John W. Backus, Computer Science pioneer, who died in 2007 at the age of 82.

Backus directed the team that invented Fortran, the first widely used high-level programming language. Fortran was the second programming language I learned (after Pascal).  In the following semester I learned an assembly language (TOPS10).  Learning assembler certainly gave you an appreciation for the magic performed by a high level language like Fortran.  While the original Fortran language was crude by today’s standards, it was a real breakthrough of early Computer Science.  As expert assembly language programmers, the engineers on the development team put all of their skills and favorite tricks, into the first Fortran compiler.  When examining the resulting assembly language output generated by their first Fortran programs, they would often be surprised at the novel code that would emerge when the compiler unexpectedly combined two or more techniques from different practitioners.

Backus also co-invented the Backus-Naur Form (BNF) meta-syntax. This notation provides a precise and elegant method for defining the lexical and syntactic rules of a formal language such as programming language or data structure syntax.  My introduction to BNF in a Sophomore level programming class was another a-ha! moment for me as a developing programmer.  Learning BNF was relatively easy, you can learn it in an hour or two.  But by methodically applying a simple set of rules, it was possible to design the syntax of a complex formal language and specify it precisely using BNF.  From there it was then fairly straightforward to mechanically translate that BNF specification by hand into a program that could automatically parse and recognize the syntax you designed.

After our introduction to BNF, our assignment was to use BNF as an aid in the design and creation of a program that would correctly translate a number (such as 2.387) into proper English (Two thousand, three hundred, eighty seven). The program was to be written in Algol (my first exposure to that language) and after exorcising the compile time errors, it was the first non-trivial computer program I had ever written that had no logic errors on its first execution. At the time, without the road-map provided by the BNF, I would have been at a loss as to how to even begin to solve the problem in any programming language, let alone one that I had just been exposed to the week before. This was powerful stuff!

Could you imagine spending 20% of your time doing peer reviews of other colleague’s code? Well according to I, Cringely that’s what happens at Google.  There are a number of other practices that are not the usual norm as well.  Interesting reading.