PeerIndex Blog

Our Twitter stream has been abuzz with people complaining about how Klout, a company measuring social influence, changed its scoring methodology last night.

As another player in a similar space, we can speak from some experience when we say to consumers: ‘Don’t worry too much’.

Why?

Social media metrics are in their infancy. Social media has only been mainstream for a few years. It took us to more 20 years to go from the first digital text stores to Google. It took 21 years to go from the first Motorola handset to the iPhone and Android ecosystems.

So during this period of development expect to see competing approaches, different paradigms, wrong turns and frequent readjustments. (Third party Macs anyone?)

Understand – there are many ways of cutting the influence pie, defining it and understanding it. At PeerIndex we focus on understanding influence & opinion leadership within topics and within specific contexts.

We believe that, over time, people accumulate influence and authority in specific topical areas. A PeerIndex profile does not simply show their overall influence, rather it features your topics and topical PeerIndex scores (those little guys in green boxes). These scores are a measure of your influence in that topic.

We strongly believe our goal is to present predictive metrics, not simply after-the-fact metrics.

A topical PI can be used to determine how likely someone is to say something that other people find interesting in that topic. If you are interested in Audio Engineering, I would recommend you follow someone with a high topical PI in audio engineering – even if they have a low PI or Klout.

The overall PI (our yellow number) is an aggregate measure of all of this – in general how likely will someone say something other people will respond to – and yes it’s one proxy for influence. We even have discussed the general principles of our influence algorithm in a blog post from March 2011.

At Klout, whose CEO Joe kindly hosted me for a visit in March, the focus is essentially on ‘tweetability’. How many ripples does this person create when they fire off a social action into social media? This is why Justin Bieber scores a 100 on Klout – when Justin makes noise, people respond to that in droves. If you want to benchmark yourself daily on your ability to create ripples, then a Klout score will do that. And yes, the Klout score is one proxy for influence.

The good folk at Kred have another approach, which looks like it is driven more from metrics and looking at the impact of individual actions – see how our CTO wrote about it here.

The guys at SocialIQ and TweetLevel have their own takes.

But none of these companies is more than 3 or 4 years old. It is early days. Big social data is still in its infancy. The tools are new, the math is young and the academics are still battling it out.

Expect changes, expect variability, expect bots and expect confusing results. We’re all doing our best, there is no standard.

No one agrees – for good reason

No one agrees on a single definition of influence. Because there isn’t a single definition. As best as I can tell, social influence is as generic as the ‘physical property’ of an apple. An apple has different attributes: weight, color, density, volume, length of stalk.

The guy measuring weight may will come up with a different ‘number’ from the guy measuring volume – and they will disagree. It’s no surprise. They are measuring different things. And each thing that is measured is valuable, but they are different.

So each player is as accurate as it is. Bar none, PeerIndex is the most accurate service at measuring PeerIndex and topical PeerIndex in the world. We are probably pretty bad at estimating Klout or Kred.

And Klout is the most accurate at measuring Klout in the world.

At the Kred guys rock at Kred-scoring. Go Kred!

But which is weight and which is volume? And which one do you need for your application?

A varied ecosystem is beneficial

Having many players with different approaches bodes well for both companies and consumers. Industries with monopolies around ratings and metrics often suffer.

Nielsen’s near monopoly control of TV ratings led to decades of absurd ad spending and social (and shareholder) value was destroyed through a lack of competition. In financial markets, the oligopoly led by S&P and Moody’s played a critical role in catalysing the financial crisis that had devastated the global economy for the past five years.

It is early days – expect changes and manage your risk appropriately

As last night’s changes in Klout’s algorithm showed, there are risks in relying exclusively on a proprietary number from a single vendor for these types of applications. What was ‘right’ last week, was deemed to be ‘wrong’ today. What if you had systematically made lots of ‘right’ decisions last week, that now turn out to have been ‘wrong’ ones?

Our data changes as well – based on your behavior and also from fine tuning of our algorithms. We believe that influence and authority are things that accumulate and diminish over time, not overnight, but that won’t stop our PeerIndex algorithm having systemic changes every few months.

It is early days. Algorithms need to be tuned. Dead ends are discovered, and code is unwound. Scores will change. Relative positions between people may also change.

As consumers you’ll have to live with it. As commercial clients, you need to manage your risk appropriately.

Are you after weight or volume (or something else)?

Influence measurement is still in its infancy. Are you after weight or volume? Or something else? Which one best suits your objectives. Choose the one or more that works best for you.

Does PeerIndex or Kred or Klout best suit this application?

If you can, A/B test your campaigns using multiple sources of data and see which actually performs better under which circumstances.

To conclude

It is very early days in the growth of influence measurement and social scoring. Today even the most thoughtful players are innovating heavily and improving constantly – which means that tomorrow is better than yesterday. And that means, yes you’re scores and social profile will change alongside. Klout will fix and improve in much the same way that we will.

The direction of travel though is without question. Influence is important. Identifying opinion leaders (and being identified as one) is valuable. And social scoring is here to stay – for all of us, consumers, developers and brands.

Don’t worry. Be circumspect (and happy)