I've been very busy this year, but not got much work done...

Atticus Robert David Perdeaux. Practising the Mexican Wave with his Dad.

More pics...

There's too many things about this website that I love to list here, but the thing that really struck me this time was in the automated emails. The confirmation email you get when you place an order begins with:

Hello

I'm Little MOO - the bit of software that will be managing your order with us. It will shortly be sent to Big MOO, our print machine who will print it for you in the next few days. I'll let you know when it's done and on its way to you.

Which is quite nice because it makes it clear its an automated message in a relaxed and humorous way. The email I got the next day to confirm dispatch begins with:

Hello

it's Little MOO again. I thought you'd like to know, the following items from your order are now in the mail:

Which I had to read a few times to realise what was striking about it. It seems really simple, the system knows it has already sent an email to the user, so acknowledges the fact in the follow-up. Giving a conversational aspect to the machine-generated content makes a difference. That's one of the reasons I keep going back to them - these little things really matter. Excellent stuff.

A quick trick that I'm finding very useful for those little deployments that don't require a full export of the Subversion repository. A little regex on the standard verbose output of the Subversion log like so:

svn log -r HEAD -v | egrep '^ +[A-Z]'

will output a list of the files included in the specified revision, e.g.:

M /trunk/apps/Feedback/views.py
A /trunk/fabfile.py

The letters "A", "M" or "D" denote whether the file was added, modified or deleted.

A little bit of Python looping and manipulation later, and there's a very nice little code snippet:

pipe = os.popen("svn log -r "HEAD -v | egrep '^ +[A-Z]'")

lstUpload = []
lstRemove = []
for i in pipe.readlines():
  file = i.strip().replace('/trunk/','').split(' ')
  if file[0] == 'A' or file[0] == 'M': lstUpload.append(file[1])
  if file[0] == 'D': lstRemove.append(file[1])


Which gives us a list of files to upload to the live server, and a list of files to delete from it. Very quick and easy - bung it in your Fabric script and rest easy pilgrim.

Well, it seems that it's not just me that found this particular idea interesting. I was recently contacted by a PhD student at the Music Technology Group of the Universitat Pompeu Fabra in Barcelona, who wanted to research people's views on the different approaches to ranking their music listening habits.

The upshot is a short survey that has just been added to the chart pages of the Normalisr, and we're hoping for around a thousand users to fill it in to give Dmitry a nice big chunk of data to analyse. So, if you are a user, please get over to the Normalisr and complete the survey.

After a few solid days of Linux server wrangling, I found getting Munin to monitor our Apache server more hassle than I thought it would be.

Salient system details:

• Ubuntu Hardy
• Nginx web server (port 80) proxy
• Apache web server (port 8080)

So, if only for my own future reference, here's how I managed it:

1. Make sure the WWW Protocol library for Perl is installed:
   sudo apt-get install libwww-perl
2. Apache needs to be running mod_status (which serves of the stats that Munin draws charts for). Add the following somewhere in /etc/apache2/apache2.conf:
   
ExtendedStatus On
<Location /server-status>
    SetHandler server-status
    Order deny,allow
    Deny from all
    Allow from 127.0.0.1
</Location>

3. Restart Apache:
   sudo /etc/init.d/apache2 restart
   
4. The Munin plugins actually live in /usr/share/munin/plugins, and are symlinked to in /etc/munin/plugins:
   
   
   
sudo ln -s /usr/share/munin/plugins/apache_accesses /etc/munin/plugins/apache_accesses
sudo ln -s /usr/share/munin/plugins/apache_activity /etc/munin/plugins/apache_activity
sudo ln -s /usr/share/munin/plugins/apache_processes /etc/munin/plugins/apache_processes
sudo ln -s /usr/share/munin/plugins/apache_volume /etc/munin/plugins/apache_volume

   
   
   NB - If you have uploaded any additional plugins, ensure root has execute permission:
   sudo chmod 755 apache_activity
   
5. Make sure the Munin plugins are monitoring port 8080 - in this setup I've got Nginx on the default port. Add the following to /etc/munin/plugin-conf.d/munin-node:
   
[apache_*]
env.ports 8080

   
6. Restart the Munin node:
   sudo /etc/init.d/munin-node restart
   

Hooray! It's live! Your all-new "Normalisr" experience can now boast the following improvements:

• A new URL - http://www.normalisr.com/
• A slick new design, including gig photographs
• Thumbnail views of artist and album charts
• The ability to manually find artists and albums that come from last.fm without a proper Musicbrainz ID - we're hoping that this will improve the accuracy of your charts
• Proper Unicode support (fingers crossed!)
• A graphical widget of your artist chart to add to your blog or last.fm profile

We hope you enjoy it. If you have any questions, feedback or issues to report, please use the feedback form.

PS - Here's a nice example of the widget:

• Proper unicode support.
• The ability to find artists & albums that have a blank Musicbrainz ID in last.fm. This will involve users doing an additional search for each artist, but it should vastly improve most people's charts. We will try to make the search process as easy as possible (see screenshots below).
• A whizzy new design using gig photos from Flickr.
• Updated XML format to mirror v2 of last.fm's data feeds (we will keep the older XML versions available on the same URL).
• Hopefully, some form of export code that will allow charts to be shown on user blogs, etc.
• We're still scratching our heads trying to think of ways we can make a few quid out of all the work we're putting into this...

Anyway, some work in progress screenshots below.

Homepage

Artist chart listing

Album chart listing

Find an album in the Musicbrainz database

Of course, if you see one of those cars, chances are you may appear in the photographs they take. My wife remembered a particular location where we saw one last summer, and we've tracked ourselves down, strolling back home from a trip to Broadway Market in Hackney:

View Larger Map

Which actually feels a bit strange. We've been Google-papped. Lucky it was my wife I was out with that day ;)

This is worth knowing if you're testing things that rely on the tests being run in order, such as creating, editing and deleting of records in the database.

For instance:

import unittest
from myapp.models import MyModel

class MyModelTestCase(unittest.TestCase):
def test_create_item(self):
obj = MyModel.objects.create(title="hello")
.... run tests ....

def test_edit_item(self):
obj = MyModel.objects.get(id=1)
obj.title = "goodbye"
obj.save()
.... run tests ....

def test_delete_item(self):
obj = MyModel.objects.get(id=1)
obj.delete()
.... run tests ....


In the above example, when the tests are sorted alphabetically, "test_create_item" is run first, but "test_delete_item" will be run before "test_edit_item", leading to a few unexpected test failures for no apparent reason.

It looks like this is all to do with the "sortTestMethodsUsing" attribute of the TestLoader class, and I daresay there is a way of overriding it in Django, but for now, I'm just happy knowing how my tests are being sorted so I can name them sensibly.

The issue is all about widows - single words that appear on their own when text wraps onto multiple lines. Design-wise, these are bad. So I've been using the rather nifty typogrify extension to Django that makes it very simple to replace the final space in a block of dynamic text with a " " so you don't need to worry about the final word wrapping onto its own line.

Fantastic, very straightforward, on to next thing. But, I simply cannot decide whether I should be doing this for headlines. The homepage of my app has 3 columns of imported feed content, with the most recent headline in quite large type. Depending on the headline content, the widow filter sometimes makes an obvious change to how the headline is rendered. The problem is, the relatively narrow column width means that it can often make the penultimate headline row seem a bit small.

For instance, the following before/after shows a pretty standard headline both with and without widow filtering:

widow

No widow

This one seems pretty cut and dried, the widowless headline looks better and is easier to read. But problems occur with this kind of example:

widow

No widow

To me, its not obvious which one of these headlines is easier to read. If I had a choice, I'd maybe go with the widow version. But this is all dynamic text, so I need to choose whether to run ALL headlines through the widow filter or not.

Currently, I am keeping the widow filter on. But I reserve the right to change my mind about five times a day before this thing is finished.