Love and carnations

The truth is that you can measure love

Gen Maitland Hudson

Head of Evaluation and Impact Assessment

Adapted from a keynote address to the Social Value Matters conference in Istanbul, Tuesday 11 April.

A couple of years ago, I was involved in what became a major charity scandal: the collapse of Kids Company. My involvement started with an article I wrote about the charity’s impact measurement, its statistical claims, and how well those claims stood up to scrutiny.

It was my view that a bit of due diligence on the available public data showed that the charity had questions to answer, and that publication of its operational data was the best way to answer them. I invited them to make that data publicly available.

Kids Company chose not to take up my invitation.

What the charity did instead was to say, loudly and repeatedly, that their work wasn’t measurable because you can’t measure love.

That struck me then, and still strikes me now, as a very odd defence.

It points to a common misconception about social measurement, one worth taking up and clarifying because if you leave it hanging it gets in the way of good quality data collection. It also entrenches a lot of the problems that Kids Company had, notably a confusion between theory, experimental research, and day-to-day operational measurement.

Because the truth is that you can measure love. Not only can you measure it, but we’re all involved in that measurement, all the time. It’s an entirely normal part of our lives. It takes up quite a lot of our time, in fact; to the point that we have all sorts of social and cultural weigh ins: father’s day, mother’s day, and of course Valentine’s Day, all occasions when we pop love onto the scales and make an assessment.

There is, naturally, quite a bit of public debate about this. This tends to take the form of criticism of the commercialisation of our annual love calibration. But on the whole, no one seems to much mind the principle of the thing: the comparative valuation itself.

To give an example of that, I’ll describe the work of a couple of youthful social entrepreneurs.

These two were well ahead of the curve, getting in on social entrepreneurship in the early 1990s, way before it was fashionable. I was at school with them. They were teenage girls and they decided to make the most of Valentine’s Day and its love measurement with a bit of charitable and profitable business.

They started up a carnation enterprise. They would buy carnations in bulk, and anyone at school could buy them singly or in bunches of six, and have them delivered anonymously to the object of their affection, or lust, in the common room on the morning of Valentine’s Day itself.

The carnations would arrive in buckets, and you’d scramble to see how many you’d received, and to guess who might have sent them.

We all understood exactly what this meant, and that it was more than a simple numbers game. Obviously you did a count, but what was important here wasn’t only volume, it was the number of suitors, and how easily you could work out who they were.

For instance a girl in the year above, Sophie, had a boyfriend from outside school. This boyfriend drove a car, and had a job, and therefore more disposable income than the rest of us.

Sophie persuaded the boyfriend to go large on the carnations, and on Valentine’s Day she had an overflowing bucket all to herself.

Instead of being admired for this feat though, we all relentlessly mocked her for it. She’d entirely misunderstood the measure and how it helped us to interpret and value affection.

We all knew about the boyfriend, who was paraded up and down on a regular basis. His carnations were worthless in our eyes, much to Sophie’s chagrin. She’d gone for sheer quantity, when the carnations were a three-part measure of quantity, coverage and mystery. Mystery, of course, in the form of anonymity, also stood for sincerity. We took it for granted that love dared not speak its name too loudly, for fear of rejection. That’s what teenage love is like. Sophie and her boyfriend were straying into quite a different sort of territory.

The buckets and carnations were a great measure for our context. They worked beautifully for the callow and competitive lot that we were. We weren’t out to measure deep and long-lasting love, we were after a measure that accounted for fleeting popularity and unrequited longing.

I’m not now going to argue for a standard carnation unit of love. What I want to underline is that this kind of measure isn’t unusual. This is the way we go about valuing all kinds of social goods that matter to us. We use representational measures. These are measures that are embedded in our everyday understanding of life and ourselves, and which help us to develop meaning.

The buckets of carnations weren’t only expressing so many unformed inner emotional states. More importantly, they were helping us to develop an outer, social understanding of suitably expressed affection.

Now if a social impact consultant was hired to develop a framework for the carnation social enterprise, I worry that consultant would skip over the bucket and the carnations inside them almost entirely. Instead of counting flowers, you’d get a three-tier bucket-carnation theory of self-esteem, and a pre- and post-carnation measure of confidence: a 17-point scalar questionnaire that would never, in practice, be used.

Both the theory enshrined in a PDF and the scalar questionnaire would be mistakes because, rough and ready measures though they are, there’s a lot to be gained from knowing the numbers of carnations and buckets, the cost of each, the turnover and the business growth that these all represented.

There is simply no need at all to produce an alien ‘validated’ measure that would ostensibly provide a ‘purer’ assessment of impact. Both would miss the point.

We need to be far more pragmatic in our measurement and in the data that we use to take account of social programmes, and recognise that representational measures and operational data go a long way in telling us about effectiveness.

The fact of the matter is that all social measurement is impure. It all operates in context. It is inevitably creative and dynamic. It produces meaning rather than taking a reading of states of mind like a mental litmus test. A yearning for perfectionist measurement will only ever be disappointed. There is no perfectly-calibrated universal and ahistorical ruler that can be held up against a social programme without having some kind of effect on it. This is a much-studied truism of social statistics. The impact measurement community isn’t going to be able to avoid that truth. We are as firmly under the net as everybody else.

We need to embrace representational measurement and make far better use of operational data than we do. We need to stop hoping that if we try hard enough we’ll finalise a set of universals and validated indicators that will somehow resist the relentless human ability to interpret and make meaning.

And, just as a quick aside, we add social interpretation even when it comes to the standard units that measure height: why else is every Mills & Boon hero always and inevitably over six foot?

This is, I think, where Kids Company perpetuated a misconception with its repeated claim that love can’t be measured.

They misunderstood both love and measurement. They focused their attention, and a good deal of their spending, on complex theory-building without ever quite knowing how many young people attended their centres and what they were doing when they did. The important day-to-day data was considered far less important than experimental laboratory research conducted, for the most part, far from their centres. This led to spectacularly inflated numbers and perceptions of quality that were very far from the reality of the centres themselves.

In focusing on theory, Kids Company were falling into a common trap. Not for nothing does NPC suggest in a recent report that their “…belief is that a developed, intentional impact measurement process is likely to be associated with a greater focus on impact… We therefore suggest that this can be a reasonable, if not perfect, proxy measure for level of impact.” As a sector we far too readily treat theory-building and framework-definition as stand-ins for actual information. We do that because it plays to our strengths. We’re good at producing theory, and a nicely workshopped theory is easily turned into a consultancy product larded with a suitable layer of jargon. This makes us feel important, and contributes to the pervasive sense that measurement is abstruse and needs experts to put it in place.

It doesn’t. Measuring isn’t an expert practice. It’s pervasive and ordinary.

The Kids Company example shows how developing an intentional impact theory may do nothing at all for impact itself, and may even detract from it. A nice theory can all too easily be entirely disconnected from the delivery of actual services.

Instead of getting theoretical, and seeking perfectionist standard units, we could more usefully start systematically counting carnations.