My girlfriend Emily Benjamin is something of a special photographer and she is finally putting together a site to show off some of her great pieces. Naturally, I put her onto WordPress. Emily wants the photos to go front and centre, which is a fine idea, so we put together a very simple one-column theme based on QuickPic, which will be uncluttered and allow the photos to shine.

Problems arose when trying to lay out images though. Many of the photos are cropped for emphasis and impact, and being different dimensions, laying them all out on one page can look messy. This can be avoided by cheating a bit and pre-processing them in pairs and combining them into a single image, so that each constituent photos’ heights match. Arty people like because it allows for some cool dynamic looking layout. It also helps reduce the amount of HTTP requests which techy people care about.

How though to generate these tiled images? You could hand-crank one with GIMP, but that gets pretty boring after a couple of times. There are also a couple of commercial products out there that could help, but that seems like cheating. No, Team America said it best – you need a montage! ImageMagick’s montage command to be precise. (I happen to use GraphicsMagick, as ImageMagick was giving me insidious “Bus Errors” on my MacBook, but the same commands will work with both tools.) [Image|Graphics]Magick are some of the finest free tools available, and a shining example of the power of both open source communities and command line applications.

We start with two fine images, a church in Budapest and a view at sunset from the Ponte Vecchio, Florence. They’re both huge, because DSLRs now take photos at up to 20MB in JPEG format (!). So, we need to shrink them down a little bit and glue them together with a bit of whitespace, ensuring that the whole resulting image is 800 pixels wide (our chosen width for the main column).

The montage command makes this simple enough:

gm montage -geometry x800+5 -background yellow lamppost.png bell.png -resize 800 output.png

You’ll notice that I’ve made the background yellow, just so we can see where the borders are here. The geometry parameter says to make sure each component image fits within 800px vertically, leaving a 5px border on each side. We decided that we wanted a 10px internal space between the photos, so 5px on the side of each photo works nicely. The -resize command at the end takes effect after the montage has been created and resizes the entire outputted image to 800px wide, preserving the aspect ratio. This is the size of photos that has been chosen for the host site, but on this site with a narrower main column I’ve forced it to 500px. You can click on the images to see them at full 800px size if you want to. I’ve chosen to output it as a PNG. The image looks like this:

That’s not a bad first go, but there’s plenty we can improve. The sunset photo is narrower than the church photo, so it has been centred within its cell. There’s also an extraneous border around the outside. We handle these with two additional parameters, gravity and trim. Gravity allows us to ‘pull’ each photo in a given direction within its cell, and trim gets rid of a border, whatever colour it is.

For the gravity, we want to pull the left photo rightwards, and the right photo leftwards, so the central spacing is consistent. We do that by specifying -gravity East for the first photo and -gravity West for the second.

gm montage -geometry x800+5 -background yellow -gravity East lamppost.png -gravity West bell.png -resize 800 output.png

That’s looking better, but now we just have to get rid of that outer border. We do that using the -trim command, and at the same time I’m going to make the background colour transparent:

gm montage -quality 100 -geometry x800+5 -background transparent -gravity East lamppost.png -gravity West bell.png -trim -resize 800 output.png

Notice that I do the trim before the final resize. ImageMagick commands are executed sequentially from left to right. I’ve also added a -quality parameter to keep the quality of the final image high. (In this case it’s probably too high, but we won’t worry about that.) Our image now looks like this:

and it’s looking pretty groovy. But what happens if we want to tile two photos of vastly differing heights? The ‘x800′ is a decent enough guess, but if we start off with source images that are smaller than that (but which would still comfortably make a nice component image in a larger montage), we can end up with a pretty crazy result:

Not awesome. The problem is that we are only guessing as to the height of the component images. We can do better than that though, using *Magick’s identify command. This tool gives everything you could possibly want to know about an image, but for now we’re only interested in the height, and that’s relatively straightforward:

gm identify -format "%h"

which gives us a simple single number. Ideally, we only want to initially shrink the component images enough so that they are the same size vertically as the smallest vertical of the two. In practise, this means we are only shrinking the image that is larger vertically in the first stage. Once they are composed at the same height, we can glue them and shrink the resulting image. A little bit of bash scripting gets this done for us in a couple of lines:

#!/bin/bash
HEIGHT_ONE=`gm identify -format %h $1`;
HEIGHT_TWO=`gm identify -format %h $2`;

if [ "$HEIGHT_ONE" -lt "$HEIGHT_TWO" ]
then
    MIN_HEIGHT="$HEIGHT_ONE"
else
    MIN_HEIGHT="$HEIGHT_TWO"
fi

gm montage -quality 100 -geometry x${MIN_HEIGHT}+5 -background transparent -tile 2x1 -gravity East $1 -gravity West $2 -trim -resize 800 output.png

(If you’re copying and pasting that, ensure you paste as backticks on lines 3 and 4, not quotes or apostrophes.)

Save that as ‘you-need-a-montage’, then make it executable and you can do:

./you-need-a-montage left-image.jpg right-image.jpg

to end up with something like this:

Be sure to check out more of what ImageMagick and GraphicsMagick can do for you – this is just scratching the surface. And keep an eye out for Emily Benjamin too, once the site is up and running!

I was using Amazon’s Reading List application on LinkedIn and writing short reviews there, but I quite like the idea of having a history of what I’ve read on my own site. There are a couple of “now reading” plugins for WordPress, but none of them really did what I wanted, so I decided to scratch the itch and put something together. 30 minutes later and you can see the results in the sidebar on the homepage! Writing plugins for WordPress hasn’t really changed much since I was active in that area a few years ago, but the main posts table now seems to be used for all kinds of content. In that spirit, I just created a custom post type of ‘book’, inserted some data into the database and wrote a little widget to extract it. Reusing the guid column for ISBNs lets me make use of the wonderful Open Library for covers and the post content just becomes the text. I suppose I could have monetised it with Amazon links, but I’m sure I can live without the 5 bucks that would bring in a year! There’s a little more to do yet, like flesh out a main library URL, and introduce a simple backend page for adding books, but for now it does what I want.

Now I only wish I’d done this years ago, as the list appears pretty thin at the moment! I’ll just take that as yet another incentive to read more!

Erlang perhaps isn’t the best language for dealing with matrices, with its ‘one-time’ mathematical approach to assignment, however the existing methods all seemed to make use of tuple_to_list(), list_to_tuple() conversion and excessive list copying. Below I present a simple module which is at least more efficient than the above, and relies on a list of lists, using lists:nth() to retrieve elements. Perhaps in the future this will be superceded by new array syntax, but for now it suits my needs.

Matrices are generally stable, by which I mean unchanging in dimensions. Of course matrices can be different sizes, but the majority of matrix operations that return a matrix result return one the same size as at least one of the inputs. Given that we know the matrix dimensions, in theory this makes tuples a suitable choice for implementation, but in practice the immutability of tuples (even more immutable than lists) make them unwieldy and involve a lot of overhead when building them. And with Erlang’s binding there will be lots of rebuilding.

The main approach is to accept that we will be rebuilding the matrix in full each time and generalise to a standard matrix building function that takes matrix size and cell content generator function parameters:

new(Columns, Rows, ContentGenerator) ->
  [
    [ContentGenerator(Column, Row, Columns, Rows)
      || Column <- lists:seq(1, Columns)
    ]
    || Row <- lists:seq(1, Rows)
  ].

which reads roughly “for each row, for each column on that row, call the content generation function for that cell”.

Each content generator function has the following spec:

fun((pos_integer(), pos_integer(), pos_integer(), pos_integer()) -> any()

where the four parameters are current column, current row, number of columns, number of rows. A simple identity function is as follows:

fun(Column, Row, _, _) ->
  case Column of Row -> 1; _ -> 0 end
end

which will generate the identity matrix:

[[1 0 0]
 [0 1 0]
 [0 0 1]]

A sequential matrix might be generated with the following function:

fun(Column, Row, Columns, _) ->
  Columns * (Row - 1) + Column
end

giving:

[[1 2 3]
 [4 5 6]
 [7 8 9]]

while a matrix composed of pseudo-random numbers 1 to MaxValue could be:

fun(_, _, _, _) ->
  random:uniform(MaxValue)
end

where MaxValue is bound in a closure, giving (for example):

[[3 9 6]
 [1 1 4]
 [6 2 1]]

With this general mechanism, matrix operations such and addition or multiplication become a lot simpler. All that is required is to define a generator function that sets each cell correctly according to whatever operation you are performing. Matrix addition is defined thus:

% Adds two matrices together.
-spec add(num_matrix(), num_matrix()) -> num_matrix().
add(A, B) ->
  new(length(lists:nth(1, A)), length(A),
    fun(Column, Row, _, _) ->
      element_at(Column, Row, A) + element_at(Column, Row, B)
    end
  ).

Performance isn’t great for modification of a single cell, as it rebuilds the entire matrix. A future optimisation could include recognising that an entire row hasn’t changed and simply rebinding that. However, for my current use cases, I haven’t needed efficient single-cell manipulation. There are specs for the functions, but very little in the way of error checking. There are no checks for the addition of matrices of two different sizes, for example.

Future extensions will include the ‘Hungarian’ approach to the assignment problem, retrieval functions for entire columns or rows, some unit testing code, and confirmed support for NxM matrices. (Non-square matrices currently work in theory, but not all functions have been tested to work with them.) Check out the code on GitHub. It’s freely available to use and fork, and if you do, be sure to send a pull request!

The open source world has enriched my life significantly, both professionally, in providing the databases, languages, operating systems and IDEs that are my livelihood, and personally, in providing communications tools, entertainment and even this blogging platform. Over the course of my career, I’ve tried to give back to these communities, both in time and code submissions, and I’ve contributed in some small way to a number of projects. The challenging nature of my recent full-time employment has unfortunately meant that more recently I’ve only been able to do this in limited ways.

With my impending hiatus and upcoming travels, I hope to have more time to dedicate to the various online communities I follow. This might be through forums such as stackoverflow where I’ve received answers to numerous difficult questions, or simply helping people in chatrooms. It might be answering questions on Quora. It will certainly involve code contributions, though to which projects I’m not sure. No deed is entirely selfless, and regular development will ensure I stay sharp and widen my employment opportunities, but primarily this is about passion for development, which I’m certain I couldn’t live without for any length of time. I’m very excited about CouchDB, which would also develop my capabilities in Erlang. I value Zend Framework very highly, which would deepen my expertise in PHP. Perhaps it will be both, or others instead. But as the new year begins, I will make this my manifesto – that I will give back to those people and projects that have made my life, and many others’ lives better.

My job title from January 22nd – Open Source Contributor.

In a recent post, I outlined a method to abstract away the complexity of creating an encapsulated jQuery plugin. However, as was pointed out in the comments, there was a missing piece that didn’t allow for arguments to be passed through. More seriously, there was an issue with the binding of the facade function, which meant that only the last defined public function in the class could be called.

As an aside, if you need to bind a variable at the time that it is defined, it isn’t enough to define an anonymous function with a reference to the closure, as that can change by the time the function is called. The bug in my code was in the piece that bound the facade function to the set member functions.

	for ( var i in template) {
		if (typeof (template[i]) == 'function') {
			result[i] = function() {
				this.each(function() {
					this[i]();
				});
			};
		}
	}

By the time the inner function is called, “i” has already been re-bound to the final function name in the template. The solution is to bind the inner function name to the outer function name at the time of definition, which we can do by wrapping it in (yet another!) function.

	// Iterates through the set calling the specified function.
 	function makeIteratorFunction(f, set) {
		return function() {
			for ( var i = 0; i < set.length; i++) {
				set[i][f].apply(set[i][f], arguments);
			}
		};
	}

and then calling that function:

	if (template) {
		for ( var i in template) {
			if (typeof (template[i]) == 'function') {
				result[i] = makeIteratorFunction(i, result);
			}
		}
	}

At this point, although “i” will continue to change as the loop continues, the function call is bound through the closure on “f” in the auxiliary function, which remains fixed. Google Chrome’s developer tools certainly make following all that a lot simpler! I’ve updated the plugin with these improvements, which also enables us to pass arguments through, finally allowing:

$('#foo').myplugin().publicMethodWithArguments('hello');

You can get the updated plugin here. Feedback welcomed!

Update: As pointed out in the comments, the first version of this code didn’t allow for methods with parameters. Although the code in the main post below is unchanged, the code linked at the bottom of the page and here has been updated to reflect some improvements, that now allow this and some other niceties. The main body of the post should still be worth reading for the derivation though!

I’ve recently begun the process of porting some javascript library code from Prototype to jQuery, and on the whole it hasn’t been too problematic. I really like the element-centric nature of jQuery, whereas Prototype is more like a excellent set of useful static methods. There were only a couple of things I found myself really missing, and they were enumerables, and elegant plugin encapsulation. The first problem was solved with Xavier Shay’s nice enumerables plugin. The second one was more of a challenge.

The lack of encapsulation for jQuery plugins seems to be a common frustration, and there are lots of queries on the web along the lines of “how do I add public methods to a jQuery plugin”. The best solution I found came from Hector Virgen, which allows you to do something like this:

var pluginInstance = $('#foo').data('myplugin');
pluginInstance.publicMethod();

That’s pretty neat, but I didn’t really like the idea of having to go through the data object each time. A couple of other people in the comments felt the same way, but there didn’t seem to be any solutions forthcoming, so I had a crack. What follows is largely based on Hector’s code, so you should go and read that first before you go through this.

We start off with a basic plugin shell, slightly modified from Hector’s, per my taste:

(function($) {
	var Celsus = Celsus || {};
	Celsus.MyPlugin = function(element, options) {

		// Private members
		var elem = $(element);

		var settings = $.extend({}, options || {});

		// Private methods
		function _privateMethod() {
			console.log("This is a private method!");
		}

		return {
			publicMethod: function() {
				console.log(elem);
				_privateMethod();
				return true;
			}
		};
	};

	$.fn.myplugin = function(options) {
		return this.each(function() {
			var element = $(this);
			if (element.data('myplugin')) {
				return;
			}
			var myplugin = new Celsus.MyPlugin(this, options);

			// Store the new plugin definition in a data object.
			element.data('myplugin', myplugin);
		});
	};
})(jQuery);

This is a pretty good start. However, as Hector points out, the main issue is that the plugin returns a jQuery object to enable chaining. In many complex instances, chaining isn’t necessarily something you’re going to want to do, so we make a small sacrifice and forego that convenience. Instead, we are going to return a set of plugin instance objects:

$.fn.myplugin = function(options) {
	var result = [];
	this.each(function() {
		var element = $(this);

		if (!element.data('myplugin')) {
			// Initialise
			var myplugin = new Celsus.MyPlugin(this, options);

			// Store the new functions in a validation data object.
			element.data('myplugin', myplugin);
		}
		result.push(element.data('myplugin'));
	});
};

At this point, it’s not looking too great. We’ve broken chaining because we no longer return a jQuery object, but if you try and call

$('#foo').myplugin().publicMethod();

it still doesn’t work. This is because, although each plugin instance has the publicMethod() function, they are contained inside a bare array. At this point we could actually do:

($('#foo').myplugin()[0]).publicMethod();
($('#foo').myplugin()[1]).publicMethod();

Or something similar with each(), but this is very messy. We need a bit of syntactic sugar. To achieve this, the next step is to take that array, turn it into something we can work with, and add a facade, so that every public function we’ve just mixed in is presented as an option on the plugin instance set. We do that by adding the following to the plugin definition:

result = $(result);
var template = result[0];
if (template) {
	for ( var i in template) {
		if (typeof (template[i]) == 'function') {
			result[i] = function() {
				this.each(function() {
					this[i]();
				});
			};
		}
	}
}

Firstly, we convert the array to a jQuery object. Then, we look at the first instance in the set and use that as a template. It should contain all the public functions we’ve defined in our definition class, and all the instances are of the same type, so we can safely use the first entry’s template for all of them. We then enumerate through all the public functions and create a proxy or facade function on the set, which simply calls the closure of that function for each element in the set. This ensures that when you call

$('#foo').myplugin().publicMethod();

it is functionally equivalent to:

$('#foo').myplugin().each(function(instance) {
	instance.publicMethod();
});

So that’s pretty cool. We’ve avoided namespace pollution, we can mix in any number of public methods, and private members and methods behave as you’d expect them to. It isn’t possible to access public variables in this manner of course, but that is easily remedied by using public getters and setters. We can even add in a reference back to the jQuery object, so we can get chaining back in in some form:

result.$ = this;

Which lets us do:

$('#foo').myplugin().$.addClass('bar');

This is useful when you’re doing plugin initialisation, but thereafter it’s a bit redundant as the actual plugin call simply returns an object which you then ignore. The only final point is that there’s quite a lot of boilerplate going on here just to get set up. In actual fact, the actions are pretty generic, so we can extract all that code and put it in its own plugin. A plugin to generate a plugin!

(function($) {
	$.fn.encapsulatedPlugin = function(plugin, definition, objects, options) {
		var result = [];
		objects.each(function() {
			var element = $(this);

			if (!element.data(plugin)) {
				// Initialise
				var instance = new definition(this, options);

				// Store the new functions in a validation data object.
				element.data(plugin, instance);
			}
			result.push(element.data(plugin));
		});

		// We now have a set of plugin instances.
		result = $(result);

		// Take the public functions from the definition and make them available across the set.
		var template = result[0];
		if (template) {
			for ( var i in template) {
				if (typeof (template[i]) == 'function') {
					result[i] = function() {
						this.each(function() {
							this[i]();
						});
					};
				}
			}
		}

		// Finally mix-in a convenient reference back to the objects, to allow for chaining.
		result.$ = objects;

		return result;
	};

})(jQuery);

With this little plugin, our plugin initialisation code is a lot lighter:

	$.fn.myplugin = function(options) {
		return $.fn.encapsulatedPlugin('myplugin', Celsus.MyPlugin, this, options);
	};

Not too bad! The actual code that specialises a plugin is tucked away in a neatly encapsulated object, we have access to all the public methods defined on it and multiple instances can happily live side by side and be invoked separately without trampling on each other. You can grab a copy of the plugin generating plugin from here. This is still new cod and there might be the odd glitch, so if you spot any, or have other ideas, be sure to leave a comment!

A typical use-case of the new syntax is to lazy-load static variables in inherited child classes, which offers you the choice of avoiding the overhead of object-instantiation to provide class differentiation. Unfortunately, it is not quite as straightforward as you might think. Consider the following piece of code:

class BaseClass {

	static $_value = null;

	public static function getValue() {
		if (null === static::$_value) {
			static::$_value = get_called_class();
		}
		return static::$_value;
	}
}

class ChildAClass extends BaseClass {}

echo BaseClass::getValue() . "\n";
echo ChildAClass::getValue() . "\n";

What’s going on here then? When I look at that piece of code, I expect the output to be:

BaseClass
ChildAClass

Instead, it appears the call to ChildAClass::getValue() fails the conditional test and returns the static value of BaseClass, giving the following output:

BaseClass
BaseClass

Not very helpful. How can we work around this? The answer is to ensure you declare the static variable in each of your child classes as well as the parent. It’s straightforward enough, but a pain to remember every time. The following example demonstrates:

class ChildBClass extends BaseClass {

	static $_value = null;
}

echo BaseClass::getValue() . "\n";
echo ChildBClass::getValue() . "\n";

BaseClass
ChildBClass

So it appears (at least for PHP 5.3.3 (OS X)), that static only really means static if you declare it yourself each time. This might be expected behaviour, but I consider it a bug, and it can certainly lead to hard to debug situations. It doesn’t seem to be documented clearly in any standard location that I could find.

Short story – if you are looking to lazily-populate static variables using the new syntax, be sure to declare the variables in all of the child classes you are going to use!

It might just be the alcohol talking, or end-of-financial-year madness, but with WordPress 3.0 recently out and a couple of requests already this week, I’m seriously considering starting development again. I do have a couple of projects I want to do though, so I only want to dedicate time to Gengo if it’s going to be used. My Japanese has sadly diminished to the point where I’m practically mono-lingual again, so I’m not going to be blogging in multiple languages any time soon, and it would purely be for others to use. So, if enough people email me or comment in the next week or so, I’ll give it a crack. Second time around, I’ll hopefully be able to design it more cleanly and more quickly, and WordPress’ plugin architecture hopefully hasn’t changed too much while I’ve been away.

So if you want to see a new version of Gengo, please let me know. On the other hand, I hear very good things about WPML, so If Gengo’s time has passed, feel free to say that too! I won’t have hurt feelings, promise

This can be handled with a function that emits keys like:

["Category A", "2010", "03", "Post 1"]
["Category B", "2010", "03", "Post 2"]
["Category A", "2010", "03", "Post 3"]

and then use:

startkey=["Category A","2010","03"]&endkey=["Category A","2010", "03",{}]

However find its reciprocal “All March posts regardless of category” is problematic. You can’t do:

startkey=[*,"2010", "03"]&endkey=[*,"2010", "03",{}]

where * (or _, or nil, or pass) would represent “all”.

To handle this, there are currently only 2 options; design a new view with the the key components ordered differently, such that they emit:

["2010", "03", "Post 1"]
["2010", "03", "Post 2"]
["2010", "03", "Post 3"]

or, make multiple connections to the database like

startkey=["Category A","2010","03"]&endkey=["Category A","2010", "03",{}]
startkey=["Category B","2010","03"]&endkey=["Category B","2010", "03",{}]
startkey=["Category C","2010","03"]&endkey=["Category C","2010", "03",{}]

where you have a query for each category.

Neither approach is particularly satisfactory. On a recent particular problem set, a single view would be many hundreds of gigabytes of data, and while space is cheap, it’s not that cheap. Additional views were not an option. That same data set contained around 2000 different categories (or their equivalent) and 2000 connections for a particular query seemed excessive.

Since 0.9, Couch has had a way of passing multiple keys to a query in the post body of a view request. Unfortunately, this only supported precise keys, not start-end key ranges. There has been a ticket in the issue tracker to add this additional support since October, but it’s classed as a minor priority and nothing had been done on it. So I decided to have a crack.

On the face of it, it seems like a fairly simple change, only affecting the HTTP View Erlang module. On the other hand, I’ve probably written about 100 lines of Erlang in my life and never looked at the CouchDB code before, so it’s entirely possible I’ve done something wrong. Regardless, the following is a simple solution that appears to work correctly.

The output_map_view and output_reduce_view functions already had the ability to handle start and end keys, but they were being artificially restricted to treat the supplied keys and both start and end. I used Erlang’s pattern matching to make this a little richer:

case Key of
	{[{<<"startkey">>,StartKey},{<<"endkey">>,EndKey}]} ->
		nil;
	_ ->
		StartKey = Key,
		EndKey = Key
end

and then passing those new variables in the appropriate place. This seemed to work well. I presume that the Keys parameter is processed just like multiple connections, and then the results aggregated, because the results are exactly the same as a call with the same parameters in the query string.

One final change was that group_level=X is mysteriously disallowed for Multikey queries. I took a punt and removed this restriction and it all seemed to work fine. I can only guess that this restriction didn’t make sense when you had to pass precise keys.

I then query using the following as POST data:

{
    "keys": [
        {
            "startkey": ["Category A","2010","03"],
            "endkey": ["Category A","2010","03",{}]
        },
        {
            "startkey": ["Category B","2010","03"],
            "endkey": ["Category B","2010","03",{}]
        }
    ]
}

With this solution, I’m able to query 2000 services simultaneously, group them at any level I like, and get back the results at the lightning speed I’ve become accustomed to.

One small caveat: If I want to get back keys across non-contiguous blocks like this:

startkey=["Category A","2010","03"]&endkey=["Category A","2010", "03",{}]
startkey=["Category A","2010","06"]&endkey=["Category A","2010", "06",{}]
startkey=["Category B","2010","03"]&endkey=["Category B","2010", "03",{}]
startkey=["Category B","2010","06"]&endkey=["Category B","2010", "06",{}]

To get all posts in Category A and B in March and June, I can. However, if I have a reduce function and group at level 1, I still end up with 4 rows, 2 for Category A, 2 for Category B. I think this is because the queries are being run independently, without reference to the other. To do a full aggregation across time periods (for example to get the total number of posts by category in March and June), I’d still need to do a client aggregation on the resulting data-set. This may or may not be a big problem for you; it’s certainly something I can live with.

The CouchDB issue lives here, and the patch to 0.10.1 lives here.

I still have a couple of gripes, the largest one of which is that you can’t use a wildcard parameter at the beginning of your view keys, so if you need to get “items by user by category” and “items by category by user”, you need two views. I’m sure there are good architectural reasons for this, but for me it’s the one place where collation lets me down. For at least one of the solutions I’m working on, multiple views are a major problem, as even one takes up 120GB (and counting).

But, to the main point. Native Erlang views are now possible, and if you can create them, potentially significantly faster than Javascript ones. There are a couple of gotchas though, not least for me the handling of JSON objects.

We start with a document like this:

{
   "_id": "36kem",
   "_rev": "1-c895d5a55945a9898880bf870a3b3025",
   "type": "usage",
   "timestamp": [
       "2010",
       "02",
       "28",
       "23",
       "10"
   ],
   "data": [
       {
           "t": "E000005861",
           "i": "232920",
           "o": "2365730"
       },
       {
           "t": "E000006504",
           "i": "15784",
           "o": "17786"
       },
       {
           "t": "E000006505",
           "i": "16661",
           "o": "17786"
       }
   ]
}

In reality there are thousands of entries in the data array, but this will do. Our aim is to emit one key-value pair for each item in the “data” field of each document of type “usage”. In Javascript this is pretty trivial. Erlang however, proves more of a challenge.

Based on pointers from the CouchDB Wiki, I started with:

fun ({Doc}) ->
  case proplists:get_value(<<"type">>, Doc) of
    <<"usage">> ->
      Emit(proplists:get_value(<<"_id">>, Doc), null);
    _ ->
      ok
  end
end.

and was very happy to see that work. Two things to note here: Don’t forget the {} around the Doc in the function definition or you’ll get strange errors, and; to get the value of a field in a document, you can use the standard proplists:get_value(<<"fieldname">>, Doc) construct. So far so good.

The main issue for me came with manipulating the “data” field. I didn’t actually want to emit null, but instead the “i” and “o” parts of the data field. First off, I tried:

  lists:foreach(fun(Item) -> Emit(null, [proplists:get_value(<<"i">>, Item), proplists:get_value(<<"o">>, Item)]) end, proplists:get_value(<<"data">>, Doc)

But met with some (very long) errors. (Gripe number two – they could really do with humanising the Erlang crash dump.)

It took me quite a few attempts, including stripping it right back to confirm that I had an array to iterate and that each object does in fact contain an “i” and an “o” field, before I found the problem, which is this:

Even though Documents are defined within {} braces, and JSON objects within that definition are also defined within {} braces, you cannot access them the same way in an Erlang view.

proplists:get_value(<<"field">>, Doc) is fine for the document as a whole, but you can’t access JSON objects the same way. Bad assumption on my part. Luckily, the answer I got to another Stack Overflow question recently pointed the way.

To access the data we need to pattern match the components using the Erlang representation of a JSON object, like so:

  {[{<<"t">>, TrackingID},{<<"i">>, In},{<<"o">>, Out}]} = Row

Ugly, hey? Useful though, as it extracts the TrackingID, In and Out values all in one go, kind of like a list() statement on steroids.

With that in place, and a little more tidying up of the code, we arrive at:

fun({Doc}) ->
	case proplists:get_value(<<"type">>, Doc) of
		<<"usage">> ->
			[Year, Month, Day, Hour, Minute | _] = proplists:get_value(<<"timestamp">>, Doc),
                        lists:foreach(fun(Row) ->
                                {[{<<"t">>, TrackingID},{<<"i">>, In},{<<"o">>, Out}]} = Row,
				Emit([TrackingID, Year, Month, Day, Hour, Minute],[In, Out])
			end, proplists:get_value(<<"data">>, Doc));
		_ ->
			ok
	end
end.

That little beauty lets me query the usage of a service at any granularity over data from the last 7 years in a faster time than the browser can render it. Across an HTTP connection to a data source 1000km away. On development hardware.