As an experienced customer reference professional you notice certain topics that come up very frequently indeed. They include “write it into the contract so customers have to be references” and “how can I measure the ROI on case studies”. What follows is an expanded response to a recent post on CRKSN on the latter topic.
Can you measure the ROI of customer case studies?
Mixing metaphors, my view is that having REAL metrics on reference material usage, influence and therefore ROI is pretty much a holy grail but also an area in which there are some folks selling snake oil :o).
Yes we can look at web-stats, make folks register to view assets (poor practice) or view asset usage statistics linked to associated deals via reference fulfilment processes or systems and this does give us numbers but it tells us nothing really about the business impact of case studies.
We operate global reference helpdesks for a few organisations, managing thousands of requests per year and yes, we can say how often specific case studies were supplied and (in some circumstances) even if they were used and then we can correlate these against specific deal values. It’s way better than nothing but does not measure ROI.
Here’s the thing:
Imagine you are trying to close a ten-year, billion dollar outsourcing deal and you are part of the pursuit/bid team. You might invest half a million dollars in the bid itself and one of the items you will use at some point will be a written or video case study, assets that might exist anyway that cost $2,000 to $20,000.
Even if the case study was used right at the beginning of the process in order for you to get through to the last round of the pitch, and it gets used just once, no-one is going to say that it was not worth creating a $2,000 case study for just one $1b use. Nor can anyone say that the case study had an ROI of (close to) 5,000,000%.
It’s the same in the volume space. I’ve seen all sorts of claims about video, and white papers etc. yet we all know that direct sales teams sell most of their stuff using PowerPoint slides, most of which would be unrecognisable to the teams that lovingly created them and also many of which will be for customers that are not approved references!
A correlation between two sets of numbers (we changed this, the numbers did that) is not the same as cause and effect and I’m not always sure the due diligence has been done to independently verify results.
What we DO know is that every piece of research out there and also research we have done for clients, shows pretty much the same thing; irrespective of format, customer-approved content in the form of a case study is the most-requested deliverable, used right throughout the purchase cycle (with an emphasis on the early stages) and that measured use is only a small indication of real use because folks keep their reference libraries on their laptops and like to share.
Short of interviewing the entire customer team that was involved with the purchase, the influencers, the decision makers, as well as the sales team themselves and asking them which materials were effective and by what per cent (and even then you will get subjective answers), measuring usage via request tracking and downloads is pretty much it.
My advice to anyone in this space? CRPs have much better measurement than other influencer marketing activities and we can measure much more interesting and business-impacting stuff with ROI numbers attached than the number of times case studies are used.
Footnote – Measuring ROI
Just in case you are not sure of the correct calculation to use when working out the ROI of your programme, the one that makes the most sense to me is this:
True ROI calculation = Gross profit (not revenue) minus the programme cost, divided by the programme cost
If you use any sort of measurement process for how your requests are used you will end up with some nice ‘influence values’ i.e. the total value of deals which your CRP assisted.
It does not matter if the deals were won or lost; they needed references and you supplied them.
If you don’t know your gross margin then use your gross revenue (turnover) and gross profit numbers to calculate this.
What closed the deal anyway? and how much can the CRP claim?
Then there’s the final calculation. If you ask a salesperson what percentage of a deal can be attributed to the customer references she/he will not say 100%!
In the Enterprise IT space you can often give the CRP just 1% of the deal value and it will more than pay for itself so it’s easy to be generous with the numbers :o)