Metallica, Slayer, Megadeth, and Anthrax [click on the links to see their Fantracker stats pages], known collectively to fans of thrash metal as ‘The Big Four’, performed together for the first time at the Sonisphere Festival in Warsaw in Poland in June 2010. This Friday 8th July at Sonisphere in Knebworth, Hertfordshire, they share a bill for the first time on a British stage.

Festival season is upon us one more and with Glastonbury kicking it all off this past weekend – take a look here at our latest blog post on the success of breaking bands at Glasto – it’s off to a cracking start.

Loud music and frolicking in a large field are all very well, but how does playing a festival affect a band or artists’ wider popularity with the masses? Does it generate a wider awareness of their brand, does it gain them any more fans? Does it have any effect at all many a mud soaked artist may ponder. One obvious hypothesis is that the boost might be bigger for a lesser-known band as opposed to an act who already boasts a large following. However, luckily for you there’s no need to speculate when the numbers are so readily available from us and our dinky little app.

Below we’ve profiled a few of our top festival picks of the summer. Taking a moment to discuss some choice acts and their Musicmetric stats we’re also having a look at how playing a festival might impact on signs of the all important online success or otherwise, which gets right to the heart of how the Musicmetric app delivers valuable insight into the relationship of the online and offline world.

Anyone who has been keeping an eye on the Glastonbury 2011 Band Tracker, which the Guardian put together using the Musicmetric API, will have noticed that rather like the top of the Football Premiership, the top 5 has been rendered virtually impenetrable by the big names. Led by Beyoncé, who ‘shall not be moved’ from the number 1 spot, the top 5 also includes Ke$ha, Coldplay, Jessie J and U2.

Looking at some unexpected artists who have been powering their way up the charts, we noticed that there were some truly interesting things happening with their online data.

LMFAO

First up is American electro hop duo, LMFAO, who have soared to fame after their song ‘Party Rock Anthem’ stormed the charts, gaining them a number 2 position. ‘Party Rock Anthem’ is the first single to be taken from their sophomore album, Sorry For Party Rocking, so we’re going to try and predict whether LMFAO are going to be a flash in the pan, or the new Lady Gaga.

An update from the development team…

Our aim at musicmetric is quite simple: We will collect and analyse all the data on the web (and some that isn’t) related to trends in music and present it to our users in an easily accessible and actionable format. Over the next few months we will have downloaded and analysed a large proportion of all relevant published articles, and will continue to do so as they are written to keep right up to date with opinions, trends and buzz.

Our aims are simple, but the challenges we’ve faced over the last year and a half approaching our launch have been far from trivial, and hopefully this post will give some insight into the technical side of what we’re doing.

Gathering the data, although the easy part, needs an extensive hardware infrastructure to download, extract and archive text from millions of pages a month. Accurately analysing, scaling and detecting patterns in the data locked up in these terabytes of text is the real challenge and most interesting part of working on musicmetric. It would be naive to simply present raw data as trends in the global music landscape (although we do supply raw data), the trend tracking methods we have developed would be useless if not scaled by accurate influence ranking for the sources of these trends, and simply calculating these scores is a huge task in itself.

Likewise, following activity on just one or two social media websites and presenting this as trends would give a massively biased view of where an artist is actually popular. For example, the social media website Orkut is hugely popular in Brazil, so all data originating from this website would be biased towards that country. Likewise with Twitter, trends would lean towards the UK / USA and not necessarily reflect a global view. We are rolling out tracking for multiple social networks over the next month.

Another challenge faced are the methods we have developed for text mining and sentiment analysis (and not just the fact that we need to analyse over a million documents per day). An example would be the band Pavement. How does a machine know if a piece of text is referring to the band, or a pavement alongside a road. What about two artists with the same name? There are three artists that go by the name Nirvana, seven are called Justice. Which one does our customer care about? Perhaps all of them? Disambiguation is key for these applications to work correctly. The methods we use for sentiment analysis also have to cope with changing vocabulary, or even different languages so adaptive methods are key, for this reason we employ a machine learning approach to this problem, which again has taken a long time in development.

Because we know our customers are using this data to make important decisions in how they run their business or manage their artists, we are making absolutely sure that the data is reliable, trustworthy and complete. Traceability of data sources is paramount to reliability. Our infrastructure allows full audit of any piece of data at any time, from how it was scaled or normalised, right back to which one of our servers originally collected the raw version. This is important for a variety of reasons, particularly the ability to show exactly why trends are occurring, and improves trust in our analytics. It is one thing displaying a line chart or an index showing success for an artist, it is quite another presenting a full breakdown of each source of data and how it was included in the analysis, giving clear perspective on how that line chart or index was calculated.

musicmetric is a well funded team of 6 fulltime staff (and growing) with extensive backgrounds and deep knowledge in the field, we are using cutting edge technology and work closely with our partners to solve difficult problems and have spent the last year and a half working these out. We are extremely excited to be coming towards the end of our development / alpha stage and into our official beta, then preparing for our full launch in November.

Here at musicmetric we’ve been doing research into the intriguing puzzle of inferring similarity between artists and attaching a quantitative value to the match. Lots of methods already exist, tag based metrics, crowd sourcing recommendations, manual annotation and waveform analysis, which all perform well under certain conditions. Many of these methods are used by successful music recommendation websites and work well for more popular artists but can give odd suggestions for less well known acts. With the exception of waveform analysis which can suffer from lack of sources of the music as well as being computationally expensive.

Our business is based on the analysis of all artists, including those in the long tail of popularity, up and coming artists of all genres who may not be so well tagged as well as high profile acts. This coupled with the fact that knowing the similarity between artists is essential for some of our analytics tools we decided it was beneficial to develop a custom method for inferring similarities between artists.

At present we are experimenting with a modified version of the iterative network ranking algorithm. More usually employed by search engines to rank the relevance of results, we have combined them with machine learning algorithms trained to spot relevant features and to correctly identify the subject of the text being analysed gathered by our web crawlers. This allows us to accurately classify artists into parent classes and child classes. Artists can partially belong to multiple classes and are weighted by their relevance. This data is then used in a clustering algorithm which successfully gauges how similar two artists are.