A few months ago, towards the end of 2011, as I delivered a pitch on social media ROI/ROE at the iStrategy conference in Amsterdam, I asked my team at Orange to put a social media dashboard together, one that would be monthly, more appealing to managers than the weekly slapdash XL fires were produce for ourselves, and yet would be a useful tool for decision-making. Here is – below – its December 2011 version.
As I pointed out in my presentation in Amsterdam, this dashboard is in work-in-progress mode, and will always be. It is not meant to be used to show that we are stronger, better or unduly proud of ourselves but on the contrary, that we aim at doing well and getting better.
to Klout or not to Klout?
A lot of the metrics in this dashboard are here, for want of a better purpose, to underline that new metrics are needed. Sometimes, these metrics merely require fine tuning. Klout for instance, has been heavily criticised, including by myself. We aim to use Edelman’s Tweetlevel as a replacement very soon, because of the level of noise and fuzziness around the Klout measure, and because nonetheless we need some kind of measurement. Edelman provides this, without the hype.
engagement rates … what are we measuring here?
As to engagement rates, this is yet another kettle of fish. We realised, in the course of our comparisons, that we were being penalised by social bakers for the wrong reasons. For instance, our account was filed as a British account (vs. global) which was penalising us. Secondly, we use polls a lot and they are supposedly not considered as part of the engagement rates by the system.
Yet, we have found repeatedly that polls are a very valid way of engaging with our community and we have no intention to suppress them. Last but not least, our account is multilingual and multinational. This is perfectly normal for a company which is operating globally, that is to say in 35 countries from the B2C perspective, and 220 countries and territories from the B2B point of view. Whereas many of our competitors are not that global and therefore enjoy a much better engagement rate which is not justified, we actually choose to speak Polish to the Poles, Spanish to the Spaniards and French to the French-speaking audiences of Africa and other countries as well as English to everybody else, and to us this is perfectly normal ; yet, we are being penalised for this by social bakers.
note: no hard feelings against social bakers I think this is a very good tool, and a great company. I’m a happy customer. I am only aiming at improving the system/measure.
We are trying to work with them in order to correct this measurement, but might also introduce our own calculation in which we will introduce a new way of measuring engagement rates in order to correct this bias.
What we’re trying to prove here is not that measuring return on engagement and social media isn’t worth doing, on the contrary, but that there is a lot in our plates in order to correct all these numbers and systems, before we can rely on these numbers and move forward.
here is our Orange december 2011 dashboard: