I’ve mostly got my process down, but there continue to be new levels of dumb to deal with. As it stands, the process goes something like:

1. Buy an audiobook. Generally, this is on CD because I trust Audible about as much as I trust Ubisoft, and for the exact same reason.
2. Rip the discs to high-quality MP3. On OSX I use Max, and on Windows I built a custom version of CDex to support disc numbers.
3. Queue up all of the individual tracks and play the first few seconds of each, looking for chapter breaks. For the breaks, title the track something smart like 4: The Leaky Cauldron.
4. Use the audiobookify Perl script I wrote to bundle tracks into chapters and convert from MP3 to M4B format so that I get nice chapter markers, cover art, etc. (And because pretty much every device plays M4B at this point.)

It’s a real PITA most of the time, but manageable. Sometimes, if I’m lucky, the CDDB titles for the tracks will be something like Chapter 4c so I don’t have to manually scan and tag.

But sometimes it’s just profoundly dumb.

Some audiobook producers (Blackstone, for one) think it’s amusing to disregard the actual spoken breaks in the book and instead use some fixed amount, such as tracks every 3 minutes. Some producers find it necessary to use all 99 available tracks on a CD. Some include intro and outro music on each disc. Some include a minute or two of the end of the previous disc as the start of the next disc (and then don’t put the overlap into its own track). Some audiobooks are only available as giant disc-length MP3 files with no breaks at all.

Why am I so hung up on chapter breaks? I spend most of my time listening to audiobooks while running. Trying to manage iPod volume and sport accoutrements while running involves a whole lot of fumbling. Occasionally, I’ll accidentally hit the track forward or back buttons. This is compounded by the length of audiobooks, which are often packaged into 5½ hour chunks, owing to a longstanding iPod firmware issue that does hinky things with longer audiobooks. Without proper breaks, scanning through a 5½ hour file is a nontrivial task.

When I am confronted with such lunacy, the easiest solution I’ve found is:

1. Join together all of the track files into one giant file.
2. Use mp3splt to find likely chapter breaks based on silence. But since I can’t under-split, as it would offend my delicate sensibilities to have 18 of 20 chapters marked, I have to over-split. And mp3splt is a bit of a hammer, so you have to really over-split. I find that a 20:1 ratio is what generally works: for every 1 chapter in the audiobook, split the book into 20 parts.
3. Pick up on step 3 above.

I occasionally run into audiobooks that are just a bit too long to fit neatly into 5½ hour chunks. In such a case, SoX has functionality to let you change tempo without changing pitch, which works well on audiobooks. I’ve found that speeding up most audiobooks up to 20% is almost negligible for impacting comprehension. This is also useful for the narrators that are just a bit too slow for my liking.

I’ve been looking into speech recognition software to help me out. You would think that it would be trivial: obtain an electronic text of the book, give the audio and the text to some magical program that parses the speech and associates timecodes with the text, and then do a phenome search for words like chapter, book, and part.

Unfortunately, it looks like that magical software doesn’t exist yet. The closest thing might be Google Audio Indexing, the technology that powers Google Voice. But as near as I can tell, it’s not for public use yet. Something might be built atop CMU Sphinx, but it would take a significant effort.

Oh, and on a side note: M4B files will upload to Google Music (with some limitations on file size), but are automagically converted to MP3 format on their servers, thus removing any chapter breaks. Amazon’s Cloud Player does not yet support M4B files.

Side note the second: a 5-hour M4B file of decent quality weighs in at around 100MB. Even really long audiobooks, Neal Stephenson’s Anathem for example, don’t generally run more than 30-35 hours. You could fit even that entire book as a series of 7 M4B files on a single CD. Up that to a 1GB flash drive and get ridiculous audio quality instead of just decent. And again, pretty much every device supports M4B files these days. Or, you know, you could go on selling a 2lb stack of 28 CDs.

I had a problem earlier tonight where the slave DNS server my domains wouldn’t return anything but SERVFAIL for any domain. The server was up and running and the config files hadn’t changed, but it just wouldn’t give any useful answers.

The log (in /var/log/named/bind.log) only gave messages like:

general: warning: zone rickosborne.org/IN: expired

I tried using rndc on the slave to retransfer the zones, upped the debug trace logging, and even eventually stopped and restarted Bind. No change.

On a whim, I restarted the master Bind. This produced a more useful error:

security: error: client a.b.c.d#nnn: request has invalid signature: TSIG rickosborne: tsig
 verify failure (BADTIME)

That BADTIME message comes up when the clocks on the master and slave get more than 5 minutes out of sync. The master server’s clock was off by ~15 minutes.

The virtual server, running on Xen, ignored any attempts to set the clock. I tried plain old date and ntpdate, and both appeared to work, but were actually silently ignored. There’s a magical incantation to fix that:

echo 1 > /proc/sys/xen/independent_wallclock

So, yeah. That was a fun 2 hours of my life to track down and fix. I hope it helps someone else.

To prove myself wrong, I built interfaces into this term’s Advanced Database Structures final exam practical. The course has changed a bit since I started teaching it almost 2 years ago (holy crap!). The core objectives are the same, but the underlying technology has been migrated to CouchDB and MongoDB.

From the exam instructions:

You will be refactoring an existing application to work against a MongoDB database and a CouchDB database. The codebase has been set up so that the application works as-is against a MySQL database and you can easily switch back and forth between each of the databases without having to modify anything other than the two Model classes you are responsible for.

The application I give them is a really simple proof-of-concept link shortening service called “fsa.il” and works the same way any other does: you input a long URL and your name, and you get a short URL back. You can also come back later and track click-through and referring URL statistics.

Along the top of the page are tabs for three persistence layer choices: MySQL, MongoDB, and CouchDB. The code they are given to start has a service that keeps track of your preferred data layer in your Session and handles all of the object creation for you. The students, who built the MySQL version of the app for last month’s final exam, are given a working MySQL class, an interface for the persistence layer, and two stubbed-out skeleton classes for the other two layers. Oh, and data dumps for quick import into all three layers.

That’s it. That’s the exam. The students just need to make the app work against the other two databases. The interface definition is only five functions, and ends up yielding ~150-200 lines of code they need to bang out over 3½ hours.

I think it’s a fun exam, but more importantly it’s a great piece for each student’s portfolio. How much of a rock star would you look like on an interview if you could show a piece of code that worked against both relational and non-relational persistence layers without a truckload of third-party code?

It’s interesting writing code that will work against all three layers at the same time. Naming conventions play a big part. It’s easy in SQL to write a query that aliases column names to something friendlier to code:

SELECT
    id AS linkID,
    long_url AS longURL,
    creator_name AS fullName,
    created_date AS createdDate
FROM link_links
WHERE (id = :linkId)

And while you could use Map functions in CouchDB and MongoDB to alias your document field names, that seems like a heckuva waste. Instead, you have to think about it a little bit harder beforehand. If you don’t, you end up doing translation of array key names at the Model layer, which again seems sub-optimal.

The seemingly-minor differences between the implementations of Map/Reduce in CouchDB and MongoDB also some into play. For example, when using the group() method in MongoDB you get an array back instead of a cursor, which means you can’t chain the sort() or limit() methods and have to do those in your Model.

Sorting in CouchDB can also be tricky. CouchDB assumes that you want to sort by your key and doesn’t give you any way to sort by value, making Top 10 lists a real pain to do. You’re back to sorting in your Model.

String manipulation functions are another friction point. SQL has a bevy of them, such as LEFT() and even the dreaded LIKE. No such luck in either MongoDB or CouchDB. You have to fake them in JavaScript. Compare:

WHERE (LEFT(:link, link_len) = link_start)

Versus:

function(obj, prev) {
    if("http://".substr(0, obj.linkLen) === obj.linkStart)
        prev.count++;
}

While that’s doable, and arguably potentially more powerful, it certainly isn’t what I would call elegant. Similarly, CouchDB’s startkey and endkey parameters are nice and speedy, but there are times when I’d kill for a keycontains parameter.

More to the point, I think the course is teaching great skills. Not only do students get to play with a variety of databases¹, but they also get to see how easy it is to decouple layers of an application. It’s nice that they get to muck around in Map/Reduce and REST APIs and what not, but the architectural and problem-solving lessons will take them so much further.

Oh, and just to head anyone off at the pass, the exam is done in PHP because working with MongoDB in PHP is stupid easy, while in ColdFusion it’s not for the faint of heart². The underlying Java library is usable, but requires a whole lot of syntactic icing. That is, a metric ton of javaCast calls and type paranoia. While that might be great for the students to encounter on a take-home assignment, it’s not something they need to be tripping over on a timed exam. And yes, there are some great MongoDB libraries available for ColdFusion, but the point of the exam is to have the students show that they can do it themselves.

1. If I had another 3 hours on the exam I would totally make the students add in layers for Sqlite and XML.

2. JSON-related quirks aside, using CouchDB requires an equivalent effort level from both PHP and ColdFusion. Yay REST API.