To prove myself wrong, I built interfaces into this term’s Advanced Database Structures final exam practical. The course has changed a bit since I started teaching it almost 2 years ago (holy crap!). The core objectives are the same, but the underlying technology has been migrated to CouchDB and MongoDB.

From the exam instructions:

You will be refactoring an existing application to work against a MongoDB database and a CouchDB database. The codebase has been set up so that the application works as-is against a MySQL database and you can easily switch back and forth between each of the databases without having to modify anything other than the two Model classes you are responsible for.

The application I give them is a really simple proof-of-concept link shortening service called “fsa.il” and works the same way any other does: you input a long URL and your name, and you get a short URL back. You can also come back later and track click-through and referring URL statistics.

Along the top of the page are tabs for three persistence layer choices: MySQL, MongoDB, and CouchDB. The code they are given to start has a service that keeps track of your preferred data layer in your Session and handles all of the object creation for you. The students, who built the MySQL version of the app for last month’s final exam, are given a working MySQL class, an interface for the persistence layer, and two stubbed-out skeleton classes for the other two layers. Oh, and data dumps for quick import into all three layers.

That’s it. That’s the exam. The students just need to make the app work against the other two databases. The interface definition is only five functions, and ends up yielding ~150-200 lines of code they need to bang out over 3½ hours.

I think it’s a fun exam, but more importantly it’s a great piece for each student’s portfolio. How much of a rock star would you look like on an interview if you could show a piece of code that worked against both relational and non-relational persistence layers without a truckload of third-party code?

It’s interesting writing code that will work against all three layers at the same time. Naming conventions play a big part. It’s easy in SQL to write a query that aliases column names to something friendlier to code:

SELECT
    id AS linkID,
    long_url AS longURL,
    creator_name AS fullName,
    created_date AS createdDate
FROM link_links
WHERE (id = :linkId)

And while you could use Map functions in CouchDB and MongoDB to alias your document field names, that seems like a heckuva waste. Instead, you have to think about it a little bit harder beforehand. If you don’t, you end up doing translation of array key names at the Model layer, which again seems sub-optimal.

The seemingly-minor differences between the implementations of Map/Reduce in CouchDB and MongoDB also some into play. For example, when using the group() method in MongoDB you get an array back instead of a cursor, which means you can’t chain the sort() or limit() methods and have to do those in your Model.

Sorting in CouchDB can also be tricky. CouchDB assumes that you want to sort by your key and doesn’t give you any way to sort by value, making Top 10 lists a real pain to do. You’re back to sorting in your Model.

String manipulation functions are another friction point. SQL has a bevy of them, such as LEFT() and even the dreaded LIKE. No such luck in either MongoDB or CouchDB. You have to fake them in JavaScript. Compare:

WHERE (LEFT(:link, link_len) = link_start)

Versus:

function(obj, prev) {
    if("http://".substr(0, obj.linkLen) === obj.linkStart)
        prev.count++;
}

While that’s doable, and arguably potentially more powerful, it certainly isn’t what I would call elegant. Similarly, CouchDB’s startkey and endkey parameters are nice and speedy, but there are times when I’d kill for a keycontains parameter.

More to the point, I think the course is teaching great skills. Not only do students get to play with a variety of databases¹, but they also get to see how easy it is to decouple layers of an application. It’s nice that they get to muck around in Map/Reduce and REST APIs and what not, but the architectural and problem-solving lessons will take them so much further.

Oh, and just to head anyone off at the pass, the exam is done in PHP because working with MongoDB in PHP is stupid easy, while in ColdFusion it’s not for the faint of heart². The underlying Java library is usable, but requires a whole lot of syntactic icing. That is, a metric ton of javaCast calls and type paranoia. While that might be great for the students to encounter on a take-home assignment, it’s not something they need to be tripping over on a timed exam. And yes, there are some great MongoDB libraries available for ColdFusion, but the point of the exam is to have the students show that they can do it themselves.

1. If I had another 3 hours on the exam I would totally make the students add in layers for Sqlite and XML.

2. JSON-related quirks aside, using CouchDB requires an equivalent effort level from both PHP and ColdFusion. Yay REST API.

The Web SQL Database API is the logical progression from Google Gears: a programmatic way of accessing a SQL database that’s built into your web browser. The current crop of browsers also have this all nice and sandboxed, so we shouldn’t end up suffering from some of the cross-site data access issues that were a worry in the previous generation.

So far, I think the best user-space access to the databases has to go to Safari. You can get at them in the Security tab of the Preferences window:

But, even better than that, the Developer tools have an actual browser and query console—with smart SQL tab-completion!

(I didn’t do anything magic to hide that rs_hints object—it’s a view. I guess the Safari browser doesn’t show views.)

All of the current browser implementations use SQLite as their storage engine. SQLite has robust SQL support—it’s not meant to be a data warehouse, but it’ll probably do pretty much anything you’d think was sane to do inside a web browser. I’ve successfully been able to use views, unions, several types of joins, indexes, and primary and foreign keys.

I’ve whipped up a quick app that shows off some of the technology: Sudoku + HTML5. The storage layer for the app is done using the SQL engine, as is the business logic (game rules) of the Sudoku game itself. The presentation layer is, of course, done in JavaScript using the HTML5 canvas API. I’ve tested it in Safari, Opera, and Chrome. Firefox, as of 3.5, does not yet support the SQL Database API. I don’t have access to Internet Explorer to test it, but 8 shouldn’t work due to the use of the canvas element.

Here’s a really quick overview of the Web SQL Database API:

var DB_NAME    = "ricko-sudoku";
var DB_VERSION = "1.0";
var DB_TITLE   = "Rick O's SQL Sudoku";
var DB_BYTES   = 500000;
var db = window.openDatabase(DB_NAME, DB_VERSION, DB_TITLE, DB_BYTES);

The first part is very straightforward: I define a few constants and make the openDatabase call. I’m using the constants to illustrate the variable placement—your app may just use literals in the function call.

The openDatabase call takes one additional optional argument that I have elided: a function callback. If the database needs to be created, this being the first time you’ve ever tried to open this database, the extra function will be called. The theory is that you could write all of your table creation and other DDL queries in this callback. However, since SQLite supports the IF EXISTS syntax, I find it easier to run my schema setups immediately no matter if it’s the first time or not.

Before we run any queries, though, it would be helpful to have debugging functions for when our queries go pear-shaped:

function sqlFail(err) { alert("SQL failed: " + err.message); }
function txFail(err) { alert("TX failed: " + err.message); }

Again, nothing tricky here: we’ll use these functions as callback error handler in future queries. For illustrative purposes, we could also have a logger for when our queries succeed:

function sqlWin(tx, response) { console.log("SQL succeeded: " + response.rows.length + " rows."); }
function txWin(tx) { console.log("TX succeeded."); }

A little trickier, but we’ll go into more detail on result sets in a bit. On with the database setup:

db.transaction( function(tx) {
    var queryParams = [];
    tx.executeSql(
        "CREATE TABLE IF NOT EXISTS animals (id INT NOT NULL PRIMARY KEY AUTOINCREMENT, name VARCHAR(20) NOT NULL UNIQUE);",
        queryParams,
        sqlWin,
        sqlFail
    );
}, txFail, txWin);

Again, I’m a little verbose and heavy with the variables for illustrative purposes—you’d use a literal [] instead of a queryParams local variable. Inside of a transaction function, your queries are executed atomically and serially, but also asynchronously so be careful—hence all of the callback functions. The transaction function even has optional failure/success callbacks of its own, but again be careful: they’re in the opposite order of the query functions.

If we insert a row, we can get back the ID of an autogenerated number:

db.transaction( function(tx) {
    var animalName = "Cheetah";
    tx.executeSql("INSERT INTO animals (name) VALUES (?);",
        [ animalName ],
        function(tx, results) {
            var animalId     = results.insertId;
            var animalsAdded = results.rowsAffected;
            // for this simple insert, we should only see 1 row
        },
        sqlFail
    );
});

Using SELECT queries is just as easy:

db.readTransaction( function(tx) {
    var animalName = "%Lion%";
    tx.executeSql("SELECT id, name FROM animals WHERE (name LIKE ?);",
        [ animalName ],
        function(tx, results) {
            for(var i = 0; i < results.rows.length; i++) {
                var row = results.rows.item(i);
                console.log("Animal[" + row.id + "] = " + row.name);
            } // for i
        },
        sqlFail
});

Still nothing too tricky here. Query parameters work just like you’d expect: replace the parameter with a question mark in the query, then pass it in the array afterward. The callback here iterates over the records, each of which is a simple object with column names for keys, and prints them on the debugging console.

There’s not much more to it than that—it’s surprisingly easy for something designed by committee!

• Fastest Father-Son Team: Robert and Harrison Gordon, coming in with a 1:03:15 chip time.
• Fastest Father-Daughter Team: Greg Lawrence and Amy Eller, coming in with a 1:08:52 chip time.
• Fastest Husband-Wife Team: Scott and Laura Flint, coming in with a 1:15:02 chip time.
• Best Spirit of Cooperation: Kara Mathias and Manny Gutierrez, a Gator and a ‘Nole, running the entire race together and finishing in just under 66 minutes.
• Most Supportive Running Group: Dominic, William, Tim, Jamie, and Brian from Ponte Vedra, FL, coming in at a chip time of 1:46:48.
• The Family That Runs Together Stays Together: The Gobold family of Jacksonville—Chris and Lesa, with children Jena and Tyler, and friends Christopher and Mary—staying together and finishing in 2:23:30.

I am truly impressed how many husband and wife teams were seeded. That’s absolutely awesome. (And that they can match eachother’s pace the entire race!)

Despite an enormous number of Father-Son and Husband-Wife teams, Mother-Daughter teams seemed to be thin on the ground. My best guess? Jessica and Anna of Jacksonville, FL with a finish time of 1:18:21. Finding them is harder, because the surnames may not be the same. The age difference helps, but isn’t a guarantee. (Woe unto me if I call out a mother-daughter pair only to be wrong! And I’m nowhere near stupid enough to try Father-Daughter or Mother-Son pairs!)

Someone named Nicole Norris of Roswell, GA kept pace with my buddy and I throughout the entire race, crossing each split marker within 3-4 seconds of us each time. Nicole: we were the guys in the red shirt and hat and yellow shirt (mustard, not the greenish of the official race shirts). Sorry we didn’t say hello!