My project today was to take this SQL statement:

SELECT "goalType",
  SUM("distancekm") AS "totalkm",
  COUNT(*) AS "workouts",
  COUNT("powerSongAlbum") AS "songcount",
  AVG("distancekm") AS "avgkm",
  MAX("distancekm") AS "maxkm",
  MIN("distancekm") AS "minkm"
FROM "workouts"
GROUP BY "goalType"

… and create a set of ColdFusion components to transform that query into a MapReduce function that would run on MongoDB. Mapping basic SQL to MongoDB MapReduce isn’t too hard. Getting the SQL parser from Derby working within ColdFusion was significantly harder.

But I did it. This is the completely automated result:

db.runCommand({
mapreduce: "workouts",
map: function () { emit(
  this.goalType,
  {
    '_cfcount': 1,
    'distancekm_cfsum': isNaN(this.distancekm) ? null : this.distancekm,
    'distancekm_cfnum': isNaN(this.distancekm) ? 0 : 1,
    'powerSongAlbum_cfcount': (this.powerSongAlbum == null) ? 0 : 1,
    'distancekm_cfmax': isNaN(this.distancekm) ? null : this.distancekm,
    'distancekm_cfmin': isNaN(this.distancekm) ? null : this.distancekm
  }
); },
reduce: function (key,vals) {
  var ret = {
    'distancekm_cfmax': null,
    'distancekm_cfsum': null,
    'distancekm_cfmin': null,
    'distancekm_cfnum': 0,
    'powerSongAlbum_cfcount': 0,
    '_cfcount': 0
  };
  for(var i = 0; i < vals.length; i++) {
    var v = vals[i];
    ret['distancekm_cfnum'] += v['distancekm_cfnum'];
    if(!isNaN(v['distancekm_cfmax'])) ret['distancekm_cfmax'] = (ret['distancekm_cfmax'] == null) ? v['distancekm_cfmax'] : (ret['distancekm_cfmax'] > v['distancekm_cfmax']) ? ret['distancekm_cfmax'] : v['distancekm_cfmax'];
    ret['_cfcount'] += v['_cfcount'];
    if(!isNaN(v['distancekm_cfmin'])) ret['distancekm_cfmin'] = (ret['distancekm_cfmin'] == null) ? v['distancekm_cfmin'] : (v['distancekm_cfmin'] > ret['distancekm_cfmin']) ? ret['distancekm_cfmin'] : v['distancekm_cfmin'];
    ret['powerSongAlbum_cfcount'] += v['powerSongAlbum_cfcount'];
    if(!isNaN(v['distancekm_cfsum'])) ret['distancekm_cfsum'] = v['distancekm_cfsum'] + (ret['distancekm_cfsum'] == null ? 0 : ret['distancekm_cfsum']);
  }
  return ret;
},
finalize: function (key,val) {
  return {
    'totalkm'   : val['distancekm_cfsum'],
    'workouts'  : val['_cfcount'],
    'songcount' : val['powerSongAlbum_cfcount'],
    'avgkm'     : (isNaN(val['distancekm_cfnum']) || isNaN(val['distancekm_cfsum'])) ? null : val['distancekm_cfsum'] / val['distancekm_cfnum'],
    'maxkm'     : val['distancekm_cfmax'],
    'minkm'     : val['distancekm_cfmin']
  };
},
out: "s2mr",
verbose: true
});

And here’s the output when I run that against my MongoDB collection:

{ "_id" : null, "value" : {
    "totalkm" : 451.6752000000001,
    "workouts" : 54,
    "songcount" : 53,
    "avgkm" : 8.364355555555557,
    "maxkm" : 19.7502,
    "minkm" : 0.0194
} }
{ "_id" : "Distance", "value" : {
    "totalkm" : 304.76879999999994,
    "workouts" : 27,
    "songcount" : 27,
    "avgkm" : 11.287733333333332,
    "maxkm" : 26.2581,
    "minkm" : 4.0486
} }
{ "_id" : "Time", "value" : {
    "totalkm" : 19.221,
    "workouts" : 2,
    "songcount" : 2,
    "avgkm" : 9.6105,
    "maxkm" : 9.9224,
    "minkm" : 9.2986
} }

w00t!

I’ve left the code modular enough that I can make a CouchDB version almost as easily.

Update: Better yet, I can now get all the way to an actual query data type:

(Note for potential cynics: no, I’m not missing the point. This is not meant as a production tool—it’s a learning tool. If my students can start with SQL and see the end result in MapReduce, then they have that much better chance of grokking all of it.)

“Intended to require near-zero configuration, FW/1 lets you build your application without worrying about a framework getting in your way.”

This is mostly dead-on. I’m not a frameworks guy, and even I thought FW/1 was pretty simple to get up and running around with. It wasn’t perfect—but it was noticeably easier than the other CF frameworks I’ve used.

As I mentioned in my previous post, in the ColdFusion courses I teach the two final lectures are on ORM and Frameworks. Both lectures use a simple blog front-end as example code. I translated that code over to FW/1, mostly via copy-and-paste. You can download it, if you like:

Simple Blog Front-End using CF9 ORM and FW/1
Zip Archive, 12KB

There’s not much to it, really. I suspect you’d gain more from Ray’s posts on the subject.

I ran into the same problem that Ray initially had: the default method is named “default”, which is a reserved word in CFScript. This code breaks:

public function default(required struct rc) {

This means you’re limited to either CFML-based Controllers or you need to be extra careful about using defaults. I admit that the new CFScript is growing on me—especially for Controllers, Models, and Services—so this is a bit of a drawback. (Hacking your local copy of the framework to use defaultItem instead of default may not be a horrible idea, but may potentially limit the distributability of your code … if you care about such things.)

My other nitpick is that the FW/1 documentation is upside-down. The Controller examples show how to add service calls to the queue:

variables.fw.service( "myservice.myaction", "myresult" );

But it’s not until you get to the very end of that page that you find out that to get this variable you have to set it up yourself—it’s not one of the magic variables that the framework provides:

function init(fw) { variables.fw = fw; }

I spent half of my time banging my head against the wall with writeDump() statements trying to figure out what I was doing wrong … before I scrolled down to see that.

Beyond those two relatively minor nitpicks, it’s not a bad framework. All of the implicit invocation is nice, and the structure is logical and easy to work with. I love the way the Service objects are called with argumentCollection so that they don’t have to care about the framework. I’m not a huge fan of the way Views are done, though: there’s too much hand-wavey magic. Personally, I’d prefer to code Views as functions, too, if only to have nice, explicit arguments like the Services do. But I admit that might just be my OCD kicking in. The fallback for Views and Layouts is nice.

In all, FW/1 doesn’t suck. That’s high praise coming from an anti-framework guy like me.

What is it that makes ColdFusion into ColdFusion? For me, it’s that it’s not tied down into boring object-imperative code:

var m = new cfmail();
m.from    = "whatever";
m.to      = "whoever";
m.subject = "however";
m.body    = "blah blah blah";
m.send();

Ugh. That kind of blocky syntax makes it look and feel like ASP. If I wanted to code for ASP, I’d code for ASP. (And get a lobotomy.)

The beauty and elegance of ColdFusion is its declarative nature. I don’t create an email object or new() up a query object, I just declare that I want an email or a query. Done. I think declarative E4X-style literals—turning XML and CF tags into primitives like strings and numbers—keeps the “just give it to me” feel in CF.

CF was built for moving data between web pages and databases. Upgrading XML fragments to first-class primitives fits with that theme. Once you’ve got that in your head, it’s not hard to see CF code fragments as primitives, too.

Here’s a more detailed example of the kind of cfscript I’d love to see:

import org.rickosborne.util.*;
component extends="CachedObject" {
    public query queryUserByID( id, userType = "" ) {
        // Return a query of the user information
        var ret = cachePeek(id);
        if(isQuery(ret) and (ret.recordCount gt 0)) return ret;
        ret = <cfquery datasource="#this.dsn#">
          SELECT *
          FROM users
          WHERE (id = <cfqueryparam value="#arguments.id#" cfsqltype="CF_SQL_INTEGER">)
            <cfif structKeyExists(arguments, "userType") and (arguments.userType neq "")>
            AND (type = <cfqueryparam value="#arguments.type#" cfsqltype="CF_SQL_VARCHAR">)
            </cfif>
          </cfquery>;
        if(ret.recordCount gt 0) cachePoke(id, ret);
        return ret;
    } // getUserByID
    public xml xmlUserByID( id ) {
        // Return an XML representation of user data
        // which can then be transformed with XSLT
        var ret = <user />;
        var user = queryUserByID(arguments.id);
        var colName = "";
        if(!(isQuery(user) and (user.recordCount eq 1))) return ret;
        for(colName in listToArray(user.columnList)) {
            ret.appendChild(<#lcase(colName)#>#xmlFormat(user[colName][1])#</#lcase(colName)#>);
            // or maybe ...
            ret.childNodes += <#lcase(colName)#>#xmlFormat(user[colName][1])#</#lcase(colName)#>;
            // or if that's too hard on the parser, go the long way around
            ret.appendChild(xmlParse("<#lcase(colName)#>#xmlFormat(user[colName][1])#</#lcase(colName)#>"));
        } // for colName
        return ret;
    } // formatUser
} // component

(But, honestly, at that point why start with a ColdFusion engine? Why not just start with a JavaScript/EcmaScript engine such as Rhino, then just extend it to support CF functions and objects?)