We had a great turnout here at Critical Mass for our Social Media Week presentation on measuring Social Media ROI. As the standing room only crowd attested, this is a topic that many people are grappling with right now.
Social Media Week’s blogger Kerry Sugrue developed a good summary of the presentation, and we’ve made the slides available as well.
ROI Is Not Always the Right Fit
Perhaps the most well-received point of our presentation was the observation that the measurement of ROI is not always the best way to evaluate the value of social media engagement to an organization.
ROI has a clear definition; the measure of financial results against a defined investment. It is also unfortunately a management catch phrase.
Often, when management asks for the ROI on social media, what they are really asking for is the value of social media engagement to the business. Some value can be measured financially, and thus be converted to ROI, but with social media, much value must be measured in other terms.
Anyone who is asked to evaluate the ROI of their social media effort should ask back in return if ROI is really the right measure, or if management is really seeking the most relevant measures of value available. If the true interest is around understanding the effectiveness of engagement, then many alternative measures can be used.
If however the interest is truly on getting to an ROI number by any means necessary, then – returning to the title of this post – getting to ROI for firms that cannot directly link social media activity to sales may require investments in primary research that will allow accurate estimates of value on measures not directly related to financial returns. The cost of that research (which may be significant) will need to be factored in to the “investment” variable in the social media ROI calculation.
Awareness as a Value Metric
An intuitive perspective on social media almost always results in the sense that social media should be a very effective as a channel for building awareness. However, an accurate measurement of levels of awareness and the resulting benefit (or harm) of this awareness on a business are more difficult to develop than most managers would think.
Most of what is called “measurement” of awareness is actually an estimate rather than a measurement, since awareness, being a cognitive state, is hard to measure at a distance and without direct response.
Buzz
In social media, the measurement of awareness is often framed within the concept of “buzz”. The most highly desired form of “buzz” is on-message consumer generated content. Seeding and developing such “buzz” is the ultimate objective of the community moderators who tend social media for their brand. In the ideal scenario, consumers create content, often in response to a seed planted by a moderator, which directly supports the desired brand image.
The most feared form of “buzz” is of course negative conversation about a brand and its products. When such conversation around large brands reaches through a large enough network in social media, it tends to get picked by traditional media outlets who amplify it into the mainstream market, which constitutes a brand crisis.
Automated Analysis of Reach and Sentiment
Most large brands can pick up hundreds or thousands of brand conversation points daily across multiple social networks, and this can reach even tens of thousands if monitoring includes competitors and industry-specific topics.
The evaluation of “reach” for each of these messages is relatively straight forward, since most “outlets” – be they a twitter user or a major blog – have easily accessible data around their network size (not that such numbers are always integrated into the measurement approach being used).
The evaluation of “sentiment” for each message is less straightforward. Obviously, when put in terms of possible contribution to purchase intent, positive awareness is worth more than neutral awareness which is worth more than negative awareness.
With so much content being generated, the manual evaluation of each piece of content for its sentiment value (positive, neutral, negative) is impossible. Thus, automated “text analytics” software is often utilized. Unfortunately, the best of this software is accurate in the 60% to 80% range, thus there can still be a great deal of error in an estimate of ‘awareness impact” based on automated sentiment analysis.
For What It’s Worth
While the measurement of “buzz” or awareness is often the first thing measured around a social media effort, awareness based metrics are the least accurate and thus least valuable metrics available for determining the value or impact of social media on business. Given that they are based on an (often computer generated) assumption of internal perception versus measureable behaviors, they can only ever provide an estimate of impact.
Unfortunately, because such measurement is the easiest thing to draw from most social media “listening” platforms, and because the method for determining value aligns so closely with the PR practices of valuing impressions and scoring media clips, this approach to valuing social media for business is the most common form in practice today.
As outlined in the ROI presentation, these metrics are not only inaccurate by nature, but are also, even when estimated very well, are many degrees removed from the purchase behaviors that drive the bottom line variable needed for an ROI calculation. Even with perfect scoring of sentiment, further primary research would still be required to link the act of perceiving something positive about a brand and the commercial behavior associated with that perception.
For these reasons, awareness metrics are best understood as directional metrics at best which can be used to provide context around the more specific engagement and conversion metrics to be discussed in a subsequent post.
