Last.fm normaliser - algorithm change
02 July 2007 | Matt Perdeaux | Mashups, Technology & applications
After much head-scratching and a few hairy moments with the database this afternoon, we have updated the Last.fm Normaliser to use median track length values in its calculations, rather than the arithmetic mean values used previously.
Hopefully, this should smooth out a lot of the issues people were reporting with a handful of extra-long or extra-short tracks skewing the figures for a particular artist or album.
see http://www.associativetrails.com/stuff/normalisefm/index.cfm?user=inkcow&chart=artist&format=xml
for an example.
02 Jul 2007 at 10:28 PM | Ed Summers
Maybe consider some kind of outlier-removal algorithm (eg take an average track length of all tracks except those > 1 std dev from the mean).
03 Jul 2007 at 12:56 PM | harveydrone
06 Jul 2007 at 09:26 PM | henriquemaia
12 Jul 2007 at 08:02 AM | error
13 Jul 2007 at 04:05 PM | Rednyrg721
20 Jul 2007 at 09:59 PM | adrian
I am last.fm/user/Monkbel, and N.R.M. doesn't show up, though their musicbrainz page is full of info.
28 Jul 2007 at 07:43 AM | Monk
Could you please escape special chars? See here for related XML error with the ñ char, for instance:
http://www.validome.org/xml/validate/?lang=en&url=http://www.associativetrails.com/stuff/no
rmalisefm/index.cfm%3fuser=csaavedra%26chart=album%26format=xml
31 Jul 2007 at 02:33 PM | Claudio
14 Aug 2007 at 12:29 PM | Jason Daniels
20 Aug 2007 at 05:21 AM | Sergej Barbarisch
could you make the normaliser work for users with spaces in their names?
16 Sep 2007 at 03:39 PM | ice cream
20 Sep 2007 at 02:08 PM | Steve
24 Sep 2007 at 09:27 AM | Matt Perdeaux
Just wondering why all the artists who do not yet have a 'known name' dropped off my statistics on this.
Like: Titee, Electromagnetic Impulses, Remergence, Man Made Man, Phillip Wilkerson, DJ Satori The Nucleator - all in my Top Ten artists at Last.fm and nowhere to be seen on the results here.
27 Sep 2007 at 04:33 AM | Pixieguts
There are numerous reasons why a certain artist may not appear in the normalised rankings. The Help/FAQ page goes through the various different causes, but if you're talking about emerging artists, its probably that they are either not in MusicBrainz yet, or the Last.fm data feed hasn't made the link to the MusicBrainz ID.
27 Sep 2007 at 06:31 AM | Matt Perdeaux
And clicking on "Refresh Cache" doesn't change anything.
18 Nov 2007 at 06:47 AM | inkbot
I get the same problem with Nick Cave & The Bad Seeds. The problem is that the data feed from last.fm does not contain a unique Musicbrainz ID for the band - see http://ws.audioscrobbler.com/1.0/user/iinkbot/topartists.xml
19 Nov 2007 at 05:44 AM | Matt Perdeaux
21 Nov 2007 at 02:59 PM | inkbot
04 Dec 2007 at 10:03 PM | Lawliet
04 Dec 2007 at 10:05 PM | Lawliet
15 Dec 2007 at 11:10 PM | Herbstlied
07 Mar 2008 at 05:05 PM | Angel
14 Apr 2008 at 07:20 AM | http://www.last.fm/user/umbrella-ella/
You currently list for your input boxes in forms the following CSS:
background: #fff url("../_img/fieldbg.gif") repeat-x top;
You forget to declare the foreground colour. My theme says the text colour is white. So now I am typing this on white-on-white.... It is rather annoying, please fix.
(I hope I did not make many typos here...)
20 Apr 2008 at 09:35 AM | aoeu
26 May 2008 at 02:43 PM | fnoll
14 Jun 2008 at 03:52 PM | nick
06 Jul 2008 at 07:38 PM | Freeman
06 Jul 2008 at 07:39 PM | Freeman
11 Jul 2008 at 11:14 AM | DeadLugosi
16 Jul 2008 at 08:04 PM | malpa
However, I would like to pinpoint two little problems I have with the 'normalisation' of my charts, concerning two classical music composers: Camille Saint-Saëns and Edvard Grieg.
The normaliser says Saint-Saëns must be considerably higher (8 positions higher) but I mostly listen to "Le carnaval des animaux" in which most songs are less than 2 minutes long.
At the same time, it says Edvard Grieg should be lower (3 positions lower), as if his songs were shorter than the average, but I've listened very often to his "Concerto For Piano In A Minor, Op 16, 1st movement: Allegro Molto Moderato", which is around 13 minutes long.
So, I would suggest you re-calculate the average song length for each of these two composers, if possible.
Great job besides that :)
10 Nov 2008 at 03:48 PM | Aragorn_54
03 Dec 2008 at 10:12 PM | Justin T.
Apart from that, congratulations :-)
05 Dec 2008 at 01:08 AM | Avinash Meetoo
I thought earlier that it might be caused by the information about the albums/song length is missing in Last.fm, but it's still the same after it was added. Then I read on the page about MusicBrainz, checked if it's there and added the missing album to the database (the other one was there already). I refreshed it on this site, but Innocens sttill don't appear in the charts... I waited few weeks, refreshed again, waited again... Nothing happens. What's wrong?
I've read something about Last.fm connecting teh feeds to MusicBrainz. Is there a way I can add the relations? I would do so for more bands in the future whenevr I find there's something missing.
05 Dec 2008 at 02:18 PM | John Beak
@Avinash - I've checked a few last.fm feeds (including my own), and "In Rainbows" doesn't have a Musicbrainz ID associated with it. Shame really.
@John Beak - Innocens do seem to be in the cache. Does your feed associate a Musicbrainz ID with them?
12 Dec 2008 at 05:47 AM | Matt Perdeaux
Is there anything more I can do?
21 Dec 2008 at 05:41 PM | John Beak
16 Jan 2009 at 08:57 AM | John Beak
18 Jan 2009 at 06:42 AM | lauschemaedchen
23 Jan 2009 at 06:35 AM | Matt Perdeaux
24 Jan 2009 at 08:54 AM | John Beak
This would the best of the two worlds.
08 Feb 2009 at 11:08 AM | exdeath
09 Jun 2009 at 04:14 AM | miriam
This site is great... Musicbrainz isn't
28 Jun 2009 at 12:52 PM | J. Lizard
28 Jul 2009 at 09:51 PM | Angela Close
Please support UNICODE!
01 Aug 2009 at 03:57 PM | KZ
05 Sep 2009 at 01:53 AM | Kennet Klotkuk