September 19, 2012 - Comments Off on Using Data to Delight and Excite

Using Data to Delight and Excite

Last Thursday evening, Bloom were invited to give a talk at Londata (a regular big data event in the Big Smoke) on the issues we’re facing in dealing with big data, and how we’re using our findings to delight and excite our clients. The event gave us an opportunity to share with the world what we’ve been developing over the last 18 months: “Clarity”- a next generation social media monitoring tool.

The Bloom team analyse #Londata in real-time

Introducing Clarity

Clarity enables marketers to find people and networks that are actually influencing a topic of conversation. We believe influence changes significantly depending on the conversation you are having.

Lets take Bloom as an example. We should be influential in the conversation surrounding how integrated agencies can tackle big data challenges, and we’d be a bit surprised if we were influential about baking or speciality cheese (which is perhaps what Klout would say!)

Unlike first generation Social Media monitoring tools – e.g. Klout and Peer Index, Clarity focuses on nodes that are actually influencing a conversation, rather than just ‘likely’ to influence.

The conversation on Twitter using #Londata

The way in which we measure influence has been developed in partnership with the Universities of Reading and Strathclyde and is the subject of a number of academic papers, which will be published this winter. The key idea is that we understand who speaks to whom on social media and understand what they say to infer influence.

Bloom have spent the last 12 months researching new technologies and techniques to allow us to measure influence in this way. We receive real time data from Social Media sources which we process and use to calculate our influence metrics. This involves storing hundreds of thousands of rows of data every minute and performing many millions of calculations per second. The difficulty we face is not in dealing with the volume of data we need to process but the velocity at which we need to analyse and interpret the results.

Londata VI: Using Data to Delight and Excite

During the Londata event, our team of mathematicians used the tool to calculate influence and visualise the conversation happening on Twitter about the event.

Before the talk, our Insight team ran a query in Clarity to assess the influence of the people who had registered for the event. The results are shown below, in order of our “Broadcast” score:

This analysis wasn’t conversation specific: it just took the 150 people registered for Londata and considered how messages could flow between these 150 people. Specifically, it measures how well each individual can “broadcast” information to their followers.

The @LondataEvent account is of interest here. It is ranked third, showing that in our opinion the account is highly influential. The account has a small following but the people who follow the account would amplify the messages passed out by @LondataEvent. This highlights that our method of influence goes far deeper than just measuring the number of followers of each Twitter account. Careful analysis of both Klout and Peer Index measures of influence also suggest our measure is different enough to merit a lot of attention.

During the event, we analysed 290 tweets made during the event, analysing influence and sentiment every 5 minutes.

Visualisation and Sentiment

As shown during our talk, Clarity is able to visualise the messages in real time.

This visualisation clearly shows the different communities involved in the Londata conversation and which twitter account was spreading the message.

Our “weighted sentiment” measures the sentiment of each tweet and then weights that over the influence score we’ve calculated for each tweeter.

Alex began Bloom’s talk at 19:20hrs and was welcomed with a burst of positive sentiment. The drop in sentiment was caused by Peter discussing the three V’s of big data; a number of people were wondering whether the talk would have new and interesting content. The big spike of sentiment above 30 was seen at around 20:00hrs- we were discussing their work on Eurovision and how the measurement of social media was beginning to have an effect on our clients. The two further peaks correspond to discussion around the work Bloom is doing on identifying Twitter bot accounts.

The final influence scores are shown below, in order of “broadcast” score.

We believe this method of scoring influence ties in with what people would intuitively think, looking at the actual tweets. We’ve conducted a number of studies in conjunction with our University partners to look into whether our scores correlate with the scores of Social Media experts. The full details of that study will be published later in the year but we are happy to say that the correlations are good.

What about retweets?

A question was asked during the talk about whether retweets are taken account of in our model. The results above do take into account tweets and whether they get retweeted. Just for interest’s’ sake, we’ve removed the retweets from our model and we get the following result:

This provides a useful way of assessing the power of retweets. Clearly retweeting is one measure of influence: if you have retweeted someone , you give them a vote of confidence for what they say. Without retweets, @YvesMulkers is very influential. During our talk, however, no-one retweeted his content and so he slipped back in the final analysis. On the other hand, @bloomagency were retweeted so much during the night that our influence rose.

Clarity measures all of these subtleties every few minutes. It uses the data to build sophisticated predictive models based on the information it’s observed.  We believe this level of fine detail will be of great use to international brands, trying to understand what is happening in the conversation specific word of social media.

Behind Clarity lies Bloom’s ability to handle big data and generate rich insight from data. We’ve recently been nominated for a SomeComms award for the innovative work that we’ve done using Clarity for Anglian Home Improvements, and we believe that the future really is bright for our monitoring tool. We’d love to know what you think about the results from our experiment at Londata.