Developmental evaluation, a form of real-time evaluation focused on innovation and complexity, is gaining interest and attention with funders, program developers, and social innovators. Yet, it’s popularity is revealing fundamental misunderstandings and misuse of the term that, if left unquestioned, may threaten the advancement of this important approach as a tool to support innovation and resilience.
If you are operating in the social service, health promotion or innovation space it is quite possible that you’ve been hearing about developmental evaluation, an emerging approach to evaluation that is suited for programs operating in highly complex, dynamic conditions.
Developmental evaluation (DE) is an exciting advancement in evaluative and program design thinking because it links those two activities together and creates an ongoing conversation about innovation in real time to facilitate strategic learning about what programs do and how they can evolve wisely. Because it is rooted in both traditional program evaluation theory and methods as well as complexity science it takes a realist approach to evaluation making it fit with the thorny, complex, real-world situations that many programs find themselves inhabiting.
I ought to be excited at seeing DE brought up so often, yet I am often not. Why?
Building a better brand for developmental evaluation?
Alas, with rare exception, when I hear someone speak about the developmental evaluation they are involved in I fail to hear any of the indicator terms one would expect from such an evaluation. These include terms like:
- Program adaptation
- Complexity concepts like emergence, attractors, self-organization, boundaries,
- Strategic learning
- Co-development and design
- System dynamics
DE is following the well-worn path laid by terms like systems thinking, which is getting less useful every day as it starts being referred as any mode of thought that focuses on the bigger context of a program (the system (?) — whatever that is, it’s never elaborated on) even if there is no structure, discipline, method or focus to that thinking that one would expect from true systems thinking. In other words, its thinking about a system without the effort of real systems thinking. Still, people see themselves as systems thinkers as a result.
I hear the term DE being used more frequently in this cavalier manner that I suspect reflects aspiration rather than reality.
This aspiration is likely about wanting to be seen (by themselves and others) as innovative, as adaptive, and participative and as being a true learning organization. DE has the potential to support all of this, but to accomplish these things requires an enormous amount of commitment. It is not for the faint of heart, the rigid and inflexible, the traditionalists, or those who have little tolerance for risk.
Doing DE requires that you set up a system for collecting, sharing, sensemaking, and designing-with data. It means being willing to — and competent enough to know how to — adapt your evaluation design and your programs themselves in measured, appropriate ways.
DE is about discipline, not precision. Too often, I see quests to get a beautiful, elegant design to fit the ‘social messes‘ that represent the programs under evaluation only to do what Russell Ackoff calls “the wrong things, righter” because they apply a standard, rigid method to a slippery, complex problem.
Maybe we need to build a better brand for DE.
Much ado about something
Why does this fuss about the way people use the term DE matter? Is this not some academic rant based on a sense of ‘preciousness’ of a term? Who cares what we call it?
This matters because the programs that use and can benefit from DE matter. If its just gathering some loose data, slapping it together and saying its an evaluation and knowing that nothing will ever be done with it, then maybe its OK (actually, that’s not OK either — but let’s pretend here for the sake of the point). When real program decisions are made, jobs are kept or lost, communities are strengthened or weakened, and the energy and creative talents of those involved is put to the test because of evaluation and its products, the details matter a great deal.
If DE promises a means to critically, mindfully and thoroughly support learning and innovation than it needs to keep that promise. But that promise can only be kept if what we call DE is not something else.
That ‘something else’ is often a form of utilization-focused evaluation, or maybe participatory evaluation or it might simply be a traditional evaluation model dressed up with words like ‘complexity’ and ‘innovation’ that have no real meaning. (When was the last time you heard someone openly question what someone meant by those terms?)
We take such terms as given and for granted and make enormous assumptions about what they mean that are not always supported). There is nothing wrong with any of these methods if they are appropriate, but too often I see mis-matches between the problem and the evaluative thinking and practice tools used to address them. DE is new, sexy and a sure sign of innovation to some, which is why it is often picked.
Yet, it’s like saying “I need a 3-D printer” when you’re looking to fix a pipe on your sink instead of a wrench, because that’s the latest tool innovation and wrenches are “last year’s” tool. It makes no sense. Yet, it’s done all the time.
Qualities and qualifications
There is something alluring about the mysterious. Innovation, design and systems thinking all have elements of mystery to them, which allows for obfuscation, confusion and well-intentioned errors in judgement depending on who and what is being discussed in relation to those terms.
I’ve started seeing recent university graduates claiming to be developmental evaluators who have almost no concept of complexity, service design, and have completed just a single course in program evaluation. I’m seeing traditional organizations recruit and hire for developmental evaluation without making any adjustments to their expectations, modes of operating, or timelines from the status quo and still expecting results that could only come from DE. It’s as I’ve written before and that Winston Churchill once said:
I am always ready to learn, but I don’t always like being taught
Many programs are not even primed to learn, let alone being taught.
So what should someone look for in DE and those who practice it? What are some questions those seeking DE support ask of themselves?
- What familiarity and experience do you have with complexity theory and science? What is your understanding of these domains?
- What experience do you have with service design and design thinking?
- What kind of evaluation methods and approaches have you used in the past? Are you comfortable with mixed-methods?
- What is your understanding of the concepts of knowledge integration and sensemaking? And how have you supported others in using these concepts in your career?
- What is your education, experience and professional qualifications in evaluation?
- Do you have skills in group facilitation?
- How open and willing are you to support learning, adapt, and change your own practice and evaluation designs to suit emerging patterns from the DE?
- Are you (we) prepared to alter our normal course of operations in support of the learning process that might emerge from a DE?
- How comfortable are we with uncertainty? Unpredictability? Risk?
- Are our timelines and boundaries we place on the DE flexible and negotiable?
- What kind of experience do we have truly learning and are we prepared to create a culture around the evaluation that is open to learning? (This means tolerance of ambiguity, failure, surprise, and new perspectives?)
- Do we have practices in place that allow us to be mindful and aware of what is going on regularly (as opposed to every 6-months to a year)?
- How willing are we to work with the developmental evaluator to learn, adapt and design our programs?
- Are our funders/partners/sponsors/stakeholders willing to come with us on our journey?
Of both evaluators and program stakeholders
- Are we willing to be open about our fears, concerns, ideas and aspirations with ourselves and each other?
- Are we willing to work through data that is potentially ambiguous, contradictory, confusing, time-sensitive, context-sensitive and incomplete in capturing the entire system?
- Are we willing/able to bring others into the journey as we go?
DE is not a magic bullet, but it can be a very powerful ally to programs who are operating in domains of high complexity and require innovation to adapt, thrive and build resilience. It is an important job and a very formidable challenge with great potential benefits to those willing to dive into it competently. It is for these reasons that it is worth doing and doing well.
In order for us to get there this means taking DE seriously and the demands it puts on us, the requirements for all involved, and the need to be clear in our language lest we let the not-good-enough be the enemy of the great.
Photo credit: Highline Chairs by the author
Posted on July 14, 2014
In social innovation we are at risk of confusing our stories of success for real, genuine impact. Without theories, implementation science or evaluation we risk aspiring to travel to the moon, yet leaving our rockets stuck on the launchpad.
There is a Buddhist expression that goes like this:
Be careful not to confuse the finger pointing to the moon for the moon itself. *
It’s a wonderful phrase that is playful and yet rich in many meanings. Among the most poignant of these meanings is related to the confusion between representation and reality, something we are starting to see exemplified in the world of social innovation and its related fields like design and systems thinking.
On July 13, 2014 the earth experienced a “supermoon” (captured in the above photograph), named because of its close passage to earth. While it may have seemed also close enough to touch, it was still a distance unfathomable to nearly everyone except a handful on this planet. There was a lot of fingers pointed to the moon that night.
While the moon has held fascination for humans for millennia, it’s also worth drawing our attention to the pointing fingers, too.
How often do you hear “we are doing amazing stuff“when hearing about leaders describe their social innovations in the community, universities, government, business or partnerships between them? Thankfully, it’s probably a lot more than ever because the world needs good, quality innovative thinking and action. Indeed, judging from the rhetoric at conferences and events and published literature in the academic literature and popular press it seems we are becoming more innovative all the time.
We are changing the world.
…Except, that is a largely useless statement on its own, even if well meaning.
Without documentation of what this “amazing stuff” looks like, a theory or logic explaining how those activities are connected to an outcome and an observed link between it all (i.e., evaluation) there really is no evidence that the world is changed – or at least changed in a manner that is better than had we done something else or nothing at all. That is the tricky part about working with complex systems, particularly large ones. How the world is changed is subtitle of the the book by Brenda Zimmerman, Frances Westley and Michael Quinn Patton on complexity and evaluation in social change, Getting to Maybe. It is because change requires theory, strategic implementation and evaluation that these three leaders in such topics came together to discuss what can be called social innovation. They introduce theory, strategy and evaluation ideas in the book and — while the book has remained a popular text — I rarely see them referred to in serious conversations about social innovation.
Unfortunately, concrete discussion of these three areas — theory, strategic implementation, and evaluation — is largely absent from the dialogue on social innovation. No more was this evident than in the social innovation week events held across Canada in May and June of this year as part of a series of gatherings between practitioners, researchers and policy makers from all kinds of different sectors and disciplines. The events brought together some of the leading thinkers, funders, institutes and social labs from around the world and was as close to the “social innovation olympics” as one could get. The stories told were inspirational, the diversity in the programming was wide, and the ideas shared were creative and interesting.
And yet, many of those I spoke to (including myself) were left with the question: What do I do with any of this? Without something specific to anchor to that question remained unanswered.
Lots of love, not enough (research) power
As often happens, these gatherings serve more as a rallying cry for those working in a sector — something that is quite important on its own as a critical support mechanism — but less about challenging ourselves. As Geoff Mulgan from Nesta noted in the closing keynote to the Social Frontiers event in Vancouver (and riffing off Adam Kahane’s notion of power and love as a vehicle for social transformation), the week featured a lot of love and not so much expression of power (as in critique).
Reflecting on the social innovation events I’ve attended, the books and articles I’ve read, and the conversations I’ve had in the first six months of 2014 it seems evident that the love is being felt by many, but that it is woefully under-powered (pun intended). The social innovation week events just clustered a lot of this conversation in one week, but it’s a sign of a larger trend that emphasizes storytelling independent of the kind of details that one might find at an academic event. Stories can inspire (love), but they rarely guide (power). Adam Kahane is right: we need both to be successful.
The good news is that we are doing love very well and that’s a great start. However, we need to start thinking about the power part of that equation.
There is a dearth of quality research in the field of social innovation and relatively little in the way of concrete theory or documented practice to guide anyone new to this area of work. Yes, there are many stories, but these offer little beyond inspiration to follow. It’s time to add some guidance and a space for critique to the larger narrative in which these stories are told.
What often comes from the Q & A sessions following a presentation of a social innovation initiative are the same answers as ‘lessons learned’:
- Partnerships and trust are key
- This is very hard work and its all very complex
- Relationships are important
- Get buy-in from stakeholders and bring people together to discuss the issues
- It always takes longer than you think to do things
- It’s hard to get and maintain resources
I can’t think of a single presentation over the past six months where these weren’t presented as ‘take-home messages’.
Yet, none of these answers explain what was done in tangible terms, how well it was done, what alternatives exist (if any), what was the rationale for the program and any research/evidence/theory that underpins that logic, and what unintended consequences have emerged from these initiatives and what evaluated outcomes they had besides numbers of participants/events/dollars moved.
We cannot move forward beyond love if we don’t find some way to power-up our work.
Theories of change: The fingers and the moons
Perhaps the best place to start to remedy this problem of detail is developing a theory of change for social innovation**.
Indeed, the emergence of discourse on theory of change in worlds of social enterprise, innovation and services in recent years has been refreshing. A theory of change is pretty much what it sounds like: a set of interconnected propositions that link ideas to outcomes and the processes that exist between them all. A theory of change answers the question: Why should this idea/program/policy produce (specific) changes?
The strengths of the theory of change movement (as one might call it) is that it is inspiring social innovators to think critically about the logic in their programs at a human scale. More flexible than a program logic model and more detailed than a simple hypothesis, a theory of change can guide strategy and evaluation simultaneously and works well with other social innovation-friendly concepts like developmental evaluation and design.
The weaknesses in the movement is that many theories of change fail to consider what has already been developed. There is an enormous amount of conceptual and empirical work done on behaviour change theories at the individual, organization, community and systems level that can inform a theory of change. Disciplines such as psychology, sociology, political theory, geography and planning, business and organizational behaviour, evolutionary biology and others all have well-researched and developed theories to explain changes in activity. Too often, I see theories developed without knowledge or consideration of such established theories. This is not to say that one must rely on past work (particularly in the innovation space where examples might be few in number), but if a theory is solid and has evidence behind it then it is worth considering. Not all theories are created equal.
It is time for social innovation to start raising the bar for itself and the world it seeks to change. It is time to start advancing theories, strategic implementation and evaluation practice and research so that the social innovation events of the future foster real power for change and not just inspiration and love.
* one of the more cited translated versions of this phrase has been attributed to Thich Nhat Hanh who suggests the Buddha remarked: “just as a finger pointing at the moon is not the moon itself. A thinking person makes use of the finger to see the moon. A person who only looks at the finger and mistakes it for the moon will never see the real moon.”
** This actually means many theories of change. A theory of change is program-specific and might be identical to another program and built upon the same foundations as others, but just as a program logic model is unique to each program, so too is a theory of change.
Some fields stagnate because they fail to take the bold steps into the unknown by taking chances and proposing new ideas because the research isn’t there to guide it while social innovation has a different twist on the problem: it has plenty of ideas, but little research to support those ideas. Unless the ideas and research match up it is unlikely that either area will develop.
Social innovation is a space that doesn’t lack for dreamers and big ideas. That is a refreshing change of pace from the world of public policy and public health that are well-populated by those who feel chained down to what’s been done as the entry to doing something new (which is oxymoronic when you think about it).
Fields like public health and medicine are well-served by looking to the evidence for guidance on many issues, but an over-reliance on using past-practice and known facts as the means to guide present action seriously limits the capacity to innovate in spaces where evidence doesn’t exist and may not be forthcoming.
The example of eHealth, social media and healthcare
A good example of this is in the area of eHealth. While social media has been part of the online communication landscape for nearly a decade (or longer, depending on your definition of the term), there has been sparse use of these tools and approaches within the health domain by professionals until recently. Even today, the presence of professional voices on health matters is small within the larger discourse on health and wellbeing online.
One big reason for this — and there are many — is that health systems are not prepared for the complexity that social media introduces. Julia Belluz’s series on social media and healthcare at Macleans provides among the best examples of the gaps that social media exposes and widens within the overlapping domains of health, medicine, media and the public good. Yet, such problems with social media do not change the fact that it is here, used by billions worldwide, and increasingly becoming a vehicle for discussing health matters from heart disease to weight management to smoking cessation.
Social innovation and research
Social innovation has the opposite problem. Vision, ideas, excitement and energy for new ideas abound within this world, yet the evidence generation to support it, improve upon it and foster further design innovations is notably absent (or invisible). Evaluation is not a word that is used much within this sphere nor is the term research applied — at least with the rigour we see in the health field.
In late May I participated in a one-day event in Vancouver on social innovation research in Vancouver organized by the folks at Simon Fraser University’s Public Square program and Nesta as part of the Social Innovation Week Canada events.Part of the rationale for the event can be explained by Nesta on its website promoting an earlier Social Frontiers event in the UK:
Despite thriving practitioner networks and a real commitment from policymakers and foundations to support social innovation, empirical and theoretical knowledge of social innovation remains uneven.
Not only is this research base uneven, it’s largely invisible. I choose to use the word invisible because it’s unclear how much research there is as it simply isn’t made visible. Part of the problem, clearly evident at the Vancouver event, is that social innovation appears to be still at a place where it’s busy showing people it exists. This is certainly an important first step, but as this was an event devoted to social innovation research it struck me that most attendees ought to have already been convinced of that.
Missing was language around t-scores, inter-relater reliability, theoretical saturation, cost-benefit analysis, systematic reviews and confidence intervals – the kind of terms you’d expect to hear at a research conference. Instead, words like “impact” and “scale” were thrown out with little data to back them up.
Bring us down to earth to better appreciate the stars
It seems that social innovation is a field that is still in the clouds with possibility and hasn’t turned the lights on bright enough to bring it back down to earth. That’s the unfortunate part of research: it can be a real buzz-kill. Research and evaluation can confirm what it means for something to ‘work’ and forces us to be clear on terms like ‘scale’ and ‘impact’ and this very often will mean that many of the high-profile, well-intentioned initiatives will prove to be less impactful than we hope for.
Yet, this attention to detail and increase in the quality and scope of research will also raise the overall profile of the field and the quality and scope of the social innovations themselves. That is real impact.
By bringing us down to earth with better quality and more sophisticated research presented and discussed in public and with each other we offer the best opportunity for social innovation to truly innovate and, in doing so, reach beyond the clouds and into the stars.
As social innovation grows in popularity the inevitable questions about what we mean when we use the term crop up. How we define, categorize and utilize the language of social innovation may require a taxonomy to enable us to better identify and understand its different species in the wild.
One of the early things we learn as children is to distinguish things from one another. We start to learn that ‘cat’ is different than ‘dog’ and, soon after, that ‘white cat’ is different than ‘black and white cat’.
Classification systems become useful once the level of our understanding exceeds simple descriptors and requires more nuanced, detailed information to fully explain or interpret signals from it.
noun chiefly Biology
the branch of science concerned with classification, esp. of organisms; systematics.
• the classification of something, esp. organisms: the taxonomy of these fossils.
• a scheme of classification: a taxonomy of smells.
Over the past six weeks it has become increasingly evident to me that a taxonomy of social innovation might be required if we are to advance our thinking about what it is, what it does, and what it can become. Saying the term “social innovation” is becoming increasingly problematic without some qualifier or additional information, otherwise we are left asking: “What do you mean when you say that term?”.
Having attended numerous events, developed client projects, and taken some time to reflect it seems the time is right to start considering what a taxonomy of social innovation might look like. To start, I consider the two main ‘domains’ of social innovation as I see them: Big and Not-big social innovation.
Big social innovation
One of the most notable classifications of social innovation that I’ve seen is what I call Big and Not-Big social innovation. Big social innovation is the kind that is most likely to find itself discussed in places like the Stanford Social Innovation Review, find itself with design partners like IDEO or the Bill and Melinda Gates Foundation or the World Bank. This species of social innovation is intensely focused on scale, global impact, and large-scale partnerships between institutions often operating transnationally. This is a domain of big business, big philanthropy, big data, academics, policy wonks, media, design competitions and governments.
Suits, ballrooms, celebrity spokespeople, ‘rockstar’ scientists, TED talks and frequent flier miles play a big role in Big social innovation and success is measured in dollars saved/raised, thousands if not millions of units of something done, and global reach. Africa, India and the developing world are a popular point of focus for big social innovation. Terms like ‘best practice’, ‘innovating at scale’, ‘impact assessment’, ‘targets’ and ‘global reach’ are often heard in this community.
Big social innovation is usually top-down with grassroots connected through intermediary organizations.
Not-big social innovation
Not-big social innovation is a more wiley, diffuse, less-visible, but no less important part of this field of practice. It’s typically bottom-up and represented by many small and mid-size organizations of different stripes, relies on coalitions, and is often scrambling to maintain itself as so many of its participants are struggling to fund and support the operations. As such, its a much more fluid, dynamic environment. This is akin to the ‘pre-VC start-up’ world we see in the tech sector. It’s more often characterized by a highly volunteer-ized workforce, small-but-intense program delivery, with prominent roles for local leaders.
Not-big social innovation uses terms like ‘engagement’, ‘community’, ‘context-sensitive’, ‘co-design’ and ‘sustainability’ a lot, and when ‘outcomes’ are spoken of, they often are referred to on a program-by-program basis rather than in large, aggregated numbers. Qualitative research is popular and a deep understanding of ‘complexity’ (even when that term isn’t used) is felt, largely because those involved in Not-big social innovation often have to play multiple roles at once and don’t have the luxury of specializing on one thing.
Evaluation in Not-big social innovation might be done in-house with staff trained in evaluation methods or via a single-practitioner consultant who is likely to play multiple roles (e.g., evaluator, facilitator, designer, sense-maker, and educator), rather than through a large firm that may be more likely to deliver something focused more tightly on a large, single task.
Not-big does not equal small. Backbone organizations like the Tamarack Institute for Community Engagement represent an enormous community of small and medium-sized groups who are having a big impact through coordinated effort.
Does one of the two forms of social innovation strongly resonate with you? Do you think one is better than the other? If so, you’re not alone.
I’ve had the privilege of traveling within and between both worlds and often see, hear and experience the subtle and not-so-subtle prejudices within and between these communities. And they are communities with norms, values, beliefs and expectations as well as language, idols and aspirations just like any other.
However, the rush to judge either as good or bad fits a model that says that there ought to be one type of social innovation for all and implicitly presumes that there is a standard for practice that we should follow. This creates barriers to learning and communication along with dischord that is not useful. (Debate, argument, and disagreement are healthy, but only when it facilitates understanding and communication even if it doesn’t lead to agreement).
If we shift our thinking towards creating a taxonomy of social innovation we might better serve all in the long run. For example, the frog above is the Bufo viridis, a European toad. The image below is of the Hyla cinerea, a type of tree frog. Would we be so quick to say one is better than the other? Does that make any sense at all?
Biologists use taxonomies to identify animals, understand them, compare them, and share that knowledge with the world. By doing this same kind of thing with social innovation we might better produce the same kind of understanding and with it the love that comes from contemplative inquiry. In doing so, we create opportunities to invite people in, discover and share without the need for self-righteousness, prejudice and exclusion that undermines so many other social movements and could easily derail social innovation (taking all the benefits with it without giving us a chance to remedy its problems).
Big and Not-big changes to our understanding
There is a benefit to getting exposure, money and scale to certain social innovations in certain places at certain times (Big social innovation — BSI) just as we need intense local connections to grassroots, the diversity of participation, the lessons learned through struggles and wide participation for many other issues, places and contexts (Not-big social innovation, NBSI).
Social innovation is very young, yet mature enough that we might now need to consider it as something more than a single, solitary animal.
both used under Creative Commons License via Flickr
Innovation is a term commonly associated with ‘new’ and sparkly products and things, but that quest for the bigger and more shiny in what we do often obscures the true innovative potential within systems. Rethinking what we mean by innovation and considering the role that quality plays might help us determine whether bigger and glossy is just that, instead of necessarily better.
Einstein’s oft paraphrased line about new thinking and problems goes something like this:
“Problems cannot be solved with the same mind set that created them.”
In complex conditions, this quest for novel thinking is not just ideal, it’s necessary. However genuine this quest for the new idea and new thing draws heavily upon widely shared human fears of the unknown it is also framed within a context of Western values. Not all cultures revere the new over what came before it, but in the Western world the ‘new’ has become celebrated and none more so than through the word innovation.
Innovation: What’s in a word?
A look at some of the terms associated with innovation (above) finds an emphasis on discovery and design, which can imply a positive sense of wonder and control to those with Westernized sentiments. Indeed, a survey of the landscape of actors, services and products seeking to make positive change in the world finds innovation everywhere and an almost obsessive quest for ideas. What is less attended to is providing a space for these ideas to take flight and answer meaningful, not trivial, questions in an impactful way.
I recently attended an event with Zaid Hassan speaking on Social Labs and his new book on the subject. While there was much interest in the way a social lab engages citizens in generating new ideas I was pleased to hear Hassan emphasize that the energy of a successful lab must be directed at the implementation of ideas into practice over just generating new ideas.
Another key point of discussion was the overall challenge of going deep into something and the costs of doing that. This last point got me thinking about the way we frame innovation and what is privileged in that discussion
Innovating beyond the new
Sometimes innovation takes place not only in building new products and services, but in thinking new thoughts, and seeing new possibilities.
Thinking new thoughts requires asking new or better questions of what is happening. As for seeing new possibilities, that might mean looking at things long forgotten and past practices to inform new practice, not just coming up with something novel. Ideas are sexy and fun and generate excitement, yet it is the realization of these ideas that matter more than anything.
The ‘new’ idea might actually be an old one, rethought and re-purposed. The reality for politicians and funders is often confined to equating ‘new’ things with action and work. Yet, re-purposing knowledge and products, re-thinking, or simply developing ideas in an evolutionary manner are harder to see and less sexier to sell to donors and voters.
When new means better, not necessarily bigger
Much of the social innovation sector is consumed or obsessed with scale. The Stanford Social Innovation Review, the key journal for the burgeoning field, is filled with articles, events and blog posts that emphasize the need for scaling social innovations. Scaling, in nearly all of these contexts, means taking an idea to more places to serve more people. The idea of taking a constructive idea that, when realized, benefits as many as possible is hard to argue against, however such a goal is predicated highly upon a number of assumptions about the intervention, population of focus, context, resource allocations and political and social acceptability of what is proposed that are often not aligned.
What is bothersome is that there is nowhere near the concern for quality in these discussions. In public health we often speak of intervention fidelity, intensity, duration, reach, fit and outcome, particularly with those initiatives that have a social component. In this context, there is a real threat in some circumstances of low quality information lest someone make a poorly informed or misleading choice. We don’t seem to see that same care and attention to other areas of social innovation. Sometimes that is because there is no absolute level of quality to judge or the benefits to greater quality are imperceptibly low.
But I suspect that this is a case of not asking the question about quality in the first place. Apple under Steve Jobs was famous for creating “insanely great” products and using a specific language to back that up. We don’t talk like that in social innovation and I wonder what would happen if we did.
Would we pay more attention to showing impact than just talking about it?
Would we design more with people than for them?
Would we be bolder in our experiments?
Would we be less quick to use knee-jerk dictums around scale and speak of depth of experience and real change?
Would we put resources into evaluation, sensemaking and knowledge translation so we could adequately share our learning with others?
Would we be less hyperbolic and sexy?
Might we be more relevant to more people, more often and (ironically, perhaps) scale social innovation beyond measure?
Marketoonist Cartoon used under license.
Who would have thought that failure would be held up as something to be desired just a few years ago? Yet, it is one thing to extol the virtues of failure in words, it is quite another to create systems that support failure in action and if the latter doesn’t follow the former, failure will truly live up to its name among the innovation trends of the 21st century.
Ten years ago if someone would have said that failure would be a hot term in 2014 I would have thought that person wasn’t in their right mind, but here we are seeing failure held up as an almost noble act with conferences, books and praise being heaped on those who fail. Failure is now the innovator’s not-so-secret tool for success. As I’ve written before, failure is being treated in a fetishistic manner as this new way to unlock creativity and innovation when what it might be is simply a means reducing people’s anxieties.
Saying it’s OK to fail and actually creating an environment where failure is accepted as a reasonable — maybe even expected — outcome is something altogether different. Take strategic planning. Ever see a strategic plan that includes failure in it? Have you ever seen an organization claim that it will do less of things, fail more often, and learn more through “not-achieving” rather than succeeding?? Probably not.
How often has a performance review for an individual or organization included learning (which is often related to failure) as a meaningful outcome? By this I refer to the kind of learning that comes from experience, from reflective practice, from the journey back and forth through confusion and clarity and from the experimentation of trying and both failing and succeeding. It’s been very rare that I’ve seen that in either corporate or non-profit spaces, at least in any codified form.
But as Peter Drucker once argued: what gets measured, get’s managed.
If we don’t measure failure, we don’t manage for it and nor do our teams include failure as part of their core sets of expectations, activities and outcomes and our plans or aspirations.
Failure, mindfulness and judgement
In 2010 post in Harvard Business Review, Larry Prusak commented on the phenomenon of measurement and noted that judgement — something that comes from experience that includes failure — is commonly missing from our assessments of performance of individuals and organizations alike. Judgement is made based on good information and knowledge, but also experience in using it in practice, reminding me of a quote a wise elder told me:
Good judgment comes from experience, but experience comes from bad judgment.
One of the persistent Gladwellian myths* out there is that of the 10,000 hours rule that suggests if we put that amount of time into something we’re likely to achieve a high level of expertise. This is true only if most of those 10,000 hours were mindful, deliberate ones devoted to the task at hand and involve learning from the successes, failures, processes and outcomes associated with those tasks. That last part about mindful, reflective attention or deliberate practice as the original research calls it (as so many Gladwellian myths suffer from) is left off of most discussions on the subject.
To learn from experience one has to pay attention to what one is doing, what one is thinking while doing it, and assessing the impact (evaluation) of that action once whatever is done is done. For organizations, this requires alignment between what people do and what they intend to do, requiring that mindful evaluation and monitoring be linked to strategy.
If we follow this lead where it takes us is placing failure near the centre of our strategy. How comfortable are you with doing that in your organization?
A failure of failure
Failure is among the most emotionally loaded words in the English language. While I often joke that the term evaluation is the longest four-letter word in the dictionary, failure is not far off. The problem with failure, as noted in an earlier post, is that we’ve been taught that failure is to be avoided and the opposite of success, which is viewed in positive terms.
Yet, there is another reason to question the utility of failure and that is also related to the term success. In the innovation space, what does success mean? This is not a trivial question because if one asks bold questions to seek novel solutions it is very likely that we don’t know what success actually looks like except in its most general sense.
A reading of case studies from Amazon to Apple and Acumen to Ashoka finds that their success looks different than the originators intended. Sometimes this success is far better and more powerful and sometimes its just different, but in all cases the path was littered with lessons and few failures. They succeeded because they learned, not because they failed.
Why? Because those involved in creating these ‘failures’ were paying attention, used the experience as feedback and integrated that into the next stage of development. With each stage comes more lessons and new challenges and thus, failure is only so if there is no learning and reflection. This is not something that can be wished for; it must be built into the organization.
So what to do?
- Build in the learning capacity for your organization by making learning a priority and creating the time, space and organizational support for getting feedback to support learning. Devoting a small chunk of time to every major meeting to reflecting back what you’re learning is a great way to start.
- Get the right feedback. Developmental evaluation is an approach that can aid organizations working in the innovation space to be mindful.
- Ask lots of questions of yourself, your stakeholders, what you do and the systems you’re in.
- Learn how to design for your particular program context based on feedback coming from the question asking and answering. Design is about experimenting without the expectation of immediate success.
- Develop safe-fail experiments that allow you to try novel approaches in a context that is of relatively low risk to the entire organization.
There are many ways to do this and systems that can support you in truly building the learning capacity of your organization to be better at innovating while changing the relationship you have with ‘failure’.
For more information about how to do this, CENSE Research + Design offers consultation and training to get organizations up to speed on designing for social innovation.
* Refers to ideas popularized by journalist and essayist Malcolm Gladwell that are based on the scientific research of professionals and distilled into accessible forms for mass market reading that become popular and well-known through further social discussion in forms that over-simplify and even distort the original scientific findings. It’s a social version of the “telephone game“. The 10,000 hour ‘rule’ was taken from original research by K. Anders Ericsson and colleagues on deliberate practice and is often discussed in the context of professional (often medical) training, where the original research was focused. This distortion is not something Gladwell intends, rather becomes an artifact of having ideas told over and again between people who may have never seen the original work or even Gladwell’s, but take ideas that become rooted in popular culture. A look at citations on failure and innovation finds that the term deliberate practice is rarely, if ever, used in the discussion of the “10,000 rule”.
Posted on March 27, 2014
Innovators transform the world around them in big and small ways and while a successful effort can be lauded by pundits, politicians and the public there is a long road to making change happen. That road is also a lonely one and doing things different means more than just innovating and experiencing what it means to be resilient firsthand.
Clayton Christensen’s seminal book The Innovator’s Dilemma has been one of the leading sources of thinking-inspriation in business and social innovation. The book reflects the challenges with those seeking to introduce new ideas, products or services into established markets (or ecosystems) in the aim of addressing both people’s present and future needs.
These innovators — change-makers — risk disrupting the very markets they seek to influence bringing uncertainty for everyone. What innovators bet on is that the changes they introduce will have wide-ranging, positive benefits even if they don’t fully know what those are before setting out. Not surprisingly, these efforts are not always welcome at first and the road toward understanding and acceptance is a long one.
Innovation means doing something new and while we like to talk about new, many don’t actually like doing ‘new’ because that means questioning and changing things. Indeed, change — profound change — in thinking is often vigorously opposed as Albert Einstein pointed out in a quote that is paraphrased as:
Great spirits have always encountered violent opposition from mediocre minds
This opposition is a challenge for anyone, but the long slog towards innovation is not only hard on the spirit, it is often a lonely path.
The lonely lives of leaders
To innovate means to lead through ideas and products. We live in a society that admires and elevates the innovators. No better or perhaps inspiring example is the 1997 advertisement from Apple as part of the Think Different campaign in the 1990′s.
What is missing from the platitudes, plaudits and celebrations is the quiet, often lonely, life away from the attention that successful innovations bring (nevermind those that are not deemed successful). To innovate is to lead and to lead is often to be lonely by definition because there are few leading and more following. This leadership by thought or action is often what makes leaders appear creative, innovative and — as Seth Godin affectionately calls being weird. A study discussed in the Harvard Business Review and dissected in Forbes pointed to high rates of loneliness among those at the CEO level, which is among those who “made it”. Consider those who haven’t yet “made it”, who haven’t had their idea “succeed” or take off and it might feel even more lonely.
At a recent workshop I conducted a participant expressed publicly a sense of gratitude for simply having the opportunity to connect with others who were simply open to seeing the world in the same way that they were. In hosting a learning workshop for social innovators a positive byproduct was that attendees who might have been isolated in their activities and thinking in one context could come together in another.
Innovation, because it is new, means that innovators have few peers available to directly commiserate with and may need to find ways to connect on idea, method, philosophy or role, but rarely something direct. That requires extra work in the search and more effort to connect in the finding, which takes time and energy — two things innovators are often short of.
But that doesn’t diminish the value and importance of time and energy and directing it towards efforts to reduce isolation.
Creating deep community
Paul Born, Director of the Tamarack Institute for Community Engagement, recently published a book on creating deep community connections as a necessary means of fostering transformative change. Born offers four pillars to a deepening community are: 1) sharing stories, 2) taking the time to enjoy one another, 3) taking care of one another, and 4) working together towards a bigger social goal.
While there is little to argue with here, these pillars rest on the ability to locate, co-locate and create the space to share, enjoy, care and collaborate in the first place. For many innovators this is the hardest part. Where do we find the others like ourselves and how do begin to frame this journey?
There is a reason that innovators have flocked to tools like the Business Model Canvas and the Lean Startup method to help people define, refine and develop their products and mission. It’s easy to point to firms like Apple as examples of clear-focused innovators now, but 20 or 30 years ago it wasn’t so clear. Apple’s overall mission and vision are easy to see lived out in hindsight, not at the beginning. A read of Steve Jobs’ biography illustrates how often his way of approaching the world clashed with nearly everyone and everything and how difficult life was for him.
But Steve Jobs happened to be challenging the world in a place that would come to be known as Silicon Valley. For the last thirty years the San Francisco bay area has been a spark for creative thinking and innovation, one of many hotbeds of business and cultural transformation that Richard Florida documented as home of the Creative Class(es). But not all innovation takes place in these centres and even within such centres it might be hard to connect when an idea is ill-formed or new. We lose out when innovation is only done in certain places by certain people.
(Social) innovators are part of a diffuse and sometimes lost tribe.
If you look at the language that we frame innovation we reveal many of the problems with not only our ideas, but what we do with them. As mentioned in previous posts, we privilege terms like creativity, but often ignore craft. We aspire to be learners, but often don’t like real learning. We tout the role of failure in design and innovation, yet our overloaded cultural baggage attached to the term prevents us from really failing (or asking such tepid questions we don’t really stretch ourselves).
Having access to social media and electronic communities offer a lot and something we didn’t have before, but its very difficult to forge strong, connective bonds mediated through a technological interface. Technology is good at initiating superficial connections or maintaining deeper connections, but not so good at creating deep connections. Those deeper connections as Paul Born points out are the things that sustain us and allow us to do our best work.
The dilemma is how to allocate time and resources in cultivating uniqueness, depth and connecting to similar innovators when that pool is small or integrating more with those in the convention system. Of course innovators need to relate to both groups at some level because an innovation doesn’t grow if we only connect to ‘true believers’, but at different stages it matters how we’re allocating our time, energy and enthusiasm particularly along that journey up Mt. Isolation.
There is no ready answer for this problem. Indeed, the lonely path to being different, weird or constructively challenge the harmful or less effective parts of the status quo may be one of the most wicked ones innovators face.
For those interested in social innovation there are a few examples for those who want to find peers and connect:
- The Tamarack Institute for Community Engagement (mentioned earlier) has different communities of practice focused on various aspects of community building and social innovation. They host events and have created a vibrant community of learners and action-oriented professionals across Canada and the United States;
- LinkedIn has a number of topical groups that have evolved on a variety of social and innovation topics that include local, global and topical foci;
- The Social Innovation Generation Group convenes formal and informal events connecting those working in the social innovation space in the Greater Toronto Area and across Canada;
- Meetups are self-organized gatherings on virtually every topic under the sun in communities across the globe. Check out and see if there is something near you;
- In Toronto and New York City, the Centre for Social Innovation is a part co-working space, social action community, and venture incubation support group that connects and enlivens the work that social innovators do. They have many events (many are free and low cost) organized by their members that seek to bring people together and offer skill development. If you’re in Ottawa, check out The Hub. In Calgary? Check out EpicYYC ; In Vancouver, visit the great folk at the HiVE. Throughout the United States Impact Hub spaces offer innovators options to work and connect and in Cambridge, MA there is the amazing Cambridge Innovation Centre for innovation more broadly. MaRS in Toronto offers another option.
- Lastly, CENSE Research + Design hosts a series of webinars and free and paid workshops to create capacity for social innovation. For more information visit: www.cense.ca/learning .
Born, P. (2014). Deepening Community: Finding Joy Together in Chaotic Times (p. 216). San Francisco, CA: Berrett-Koehler Publishers.
Wheatley, M. (2006). Leadership and the New Science: Discovering Order in a Chaotic World (3rd. ed., p. 218). San Francisco, CA: Berrett-Koehler Publishers.
Wheatley, M. (2007). Finding Our Way: Leadership for an Uncertain Time (p. 300). San Francisco, CA: Berrett-Koehler Publishers.
Wheatley, M. (2010). Perseverance (p. 168). San Francisco, CA: Berrett-Koehler Publishers.
Creativity is a word shrouded in myth that has been held up as this elusive, seductive object that will reveal the true secrets of innovation if ever reached. Creativity is something we all have, but not all of us are craftspeople and knowing where these two are separate and meet is the difference between myth and the muscles needed to turn creativity into innovations.
A tour of blogs, journals, and magazines that cover innovation from Inc, Fast Company, Harvard Business Review, The Atlantic, Entrepreneur and all the way to Brain Pickings will find one topic more visible than most: creativity.
Creativity is one of those terms that everyone knows, many use, has multiple meanings and is highly dependent on person and context. It’s also something that many of us feel we lack. This is not surprising given the way we set our schools and workplaces up as Sir Ken Robinson has discussed throughout his career.
Robinson has delivered perhaps one of the best and certainly most viewed talks on this at TED a few years ago illustrating the ways creativity gets ‘schooled’ out of us early on:
A look at the evidence base — which is enormous, unstructured, and varied in quality and scope — finds that creativity is hardly the mythical thing it gets made out to be and, following Sir Ken’s points raised in his TED talk, something we all have in us that may simply be hidden. More than anyone, Dr Keith Sawyer knows this having put together perhaps the strongest collection of evidence for the application of creativity in his books Explaining Creativity and, more recently, Zig Zag. (Both books are highly recommended).
Sawyer dispels such myths of the creative genius or the “flash of insight” as a linear process, rather pointing to creativity as often the cultivation of practices and habits that people go through to generate insights and products. This ‘zig zag’ represents metaphorically taking switchbacks to climb a mountain rather than going straight uphill. As you engage in creative thinking and action you build a deeper knowledge base, hone and acquire skills, and simply become more creative. “Creative people” are those that engage in these practices, build the habits of mind of creativity, and persist through each zig and zag along the way.
Design and design thinking is often associated with creativity because it is, in part, about creatively finding, framing and addressing problems through a structured process of inquiry, prototyping and revision. David and Tom Kelley in their recent book Creative Confidence point to design thinking as a layered foundation that is what much creativity is built upon. The disciplined, guided process that design thinking (well applied) offers is a vehicle for building creative confidence in those who might not feel very creative in what they do.
The process of design thinking — illustrated in the CENSE model of innovation development below — fits with Sawyer’s assertion of how creativity unfolds.
The role of craft
What Robinson, Sawyer, the Kelley brothers and others have done is dispelled the myths that creativity is some otherworldly trait and shown that its something for all of us. What can get lost in the blind adoption of this way of thinking is attention to craft.
Craft is the technical skill of applying creativity to a problem or task and that is something that is quite varied. The debate over whether or not the term designer belongs to everyone who applies creativity to solving problems or those with formal design training largely is one of craft.
Craft is the thing that brings wisdom from experience and technical skill in transforming creative ideas into quality products, not just interesting ones.
In our efforts to free people from the shackles of their education and a social world that told them they weren’t creative we’ve put aside discussion of craft in the hopes that we simply get people moving and creating. That is so very important to unlocking creative confidence and ensuring that our efforts to develop social innovations are truly social and engage the widest possible numbers of participants. However, it will be craft that ensures these solutions don’t turn into what George Carlin referred to as (great) ideas that suck.
Building design practice in the everyday
The habits of creativity are just that, habits. And if design is the way of applying creativity to problems then building a design practice is key. This means bringing elements of design into the way you operate your enterprise. Spend a lot of time finding the right problems is a start (as discussed in a previous post). Discover, inquire and be curious. Visualize, prototype, create small ‘safe-fail’ experiments, and ensure that there is a learning mechanism through the evaluation to allow your enterprise to adapt.
This is all easier said than done. It can be easy to be satisfied with being creative, but to be excellent involves craft and that requires something beyond creativity alone. It may involve training (formal or otherwise), it most certainly involves mindful attention to the work (which is what underlies the ‘10,000 hour rule’ of practice that make someone an expert), but it also requires skill. Many will find their creative talents in art, management, leadership, or service, but not all will be remarkable in exercising that skill.
To put it another way; it’s like a muscle. Everyone can work their muscles and develop them with training, nutrition, rest, and attention, but some will respond to this differently for a variety of reasons due to how all of those activities come together. This is what helps contribute to reasons why someone might be better adept at long-distance running, while others are good at bulking up and still others are far more flexible on the yoga mat.
We are all creative. We are all designers, too. But not all of us are stellar designers for all things and its important to build our collective design literacy, which includes knowing when and how to cultivate, hire and retail craftspeople and not just assume we can design think our way through everything. This last point is what will ensure that design thinking doesn’t fade away as a fad after it “didn’t produce results” because people have confused creativity with craft, myth with muscle.
Kelley, T., & Kelley, D. (2013). Creative confidence: unleashing the creative potential within us all. New York, N.Y.: Crown Publishers.
Sawyer, R. K. (2012). Explaining creativity: The science of human innovation (2nd Edition.). Oxford, U.K.: Oxford University Press.
Sawyer, R. K. (2013). Zig Zag: The Surprising Path to Greater Creativity. San Francisco, CA: Jossey-Bass.
Beautiful answers require beautiful (and better) questions and Warren Berger’s new book looks at this very phenomenon of inquiry and asks: What does it mean to ask better questions and what does that mean for the answers we seek and receive?
Warren Berger recently published A More Beautiful Question, a book looking at something we take for granted and yet is the foundational building block for all great designs and innovations: the question.
Perhaps more specifically, Berger is looking at hundreds of questions as he delves into the process of questioning, the kind of questions that lead to provocative and insightful answers, and the habits of good questioning that make for sustained innovation over time. Berger is well suited to this inquiry having penned the book Glimmer, which profiled designer Bruce Mau and explored the concept of design thinking in great detail.
Asking good questions is perhaps the (often unstated, missed and neglected) foundation of what design thinking is all about and seeing that design is the foundation of innovation it therefore means that questioning is at that foundation, too. This is important stuff.
Finding the right problem by asking better questions
A look at any bookstore, blog roll, or journal dealing with the topic of innovation and you’ll inevitably find the word “creativity” used a lot. Creativity — the act and process of creating things — is highly correlated with the questions that spur the creation in the first place. Education professor J.W. Getzels did some of the earliest research on creativity and questioning (which is interestly absent from Berger’s book) and found that those who took more time to find the best problem to solve – and thus, asked better and deeper questions of their world and subject matter — came up with more creative ideas than those who dove quickly into solving the problem as they initially saw it.
The simple take-away is:
At the root of an answer is a question – J.W. Getzels
The better the question, the better the answer.
In complexity terms, the questions asked often create the path dependencies that entrench practices that come after it. So by asking better or ‘more beautiful’ questions and giving that attention we are not only doing ourselves a service, but are acting more ethical as well. This ethical foundation is what underlies mindfulness practice. Jon Kabat Zinn has written extensively on the importance of grounding oneself to ask better questions of the world, something that I’ve done through CENSE Research + Design in developing a mindful organization model.
In his 2004 presidential address to the Canadian Psychological Association Pat O’Neill looked at how sub-fields like community psychology changed the nature of how many “problems” in psychology were framed at the outset. Issues like poverty, drug addiction and unemployment were often (and still are in many domains) framed as personal, moral failings or just bad choices. By asking different questions of these problems, community psychologists were able to see how social policies, neighbourhood structures, social networks, and historical social exclusion — all systems issues — factor in to frame and constrain individual’s choices and risk behaviours. Suddenly, what had been framed as a personal problem, became a shared one that we all had at least some stake in.
It is this thinking that has led to greater awareness of how social change is inextricably linked to systems change and why we need to understand systems at the individual, organizational, community and societal level if we wish to address many of our social problems. Asking systems questions is asking different, sometimes more beautiful questions that get at the root of problems and inspire social innovation.
Finding the beautiful question
In his book, Berger finds that those best equipped to solve or at least address these big wicked questions in business, philanthropy and social innovation are those that ask ‘beautiful questions’ and do it often. Berger cites studies that have shown a clear relationship between success in leadership and a propensity to ask good questions. Asking good questions however takes time and the willingness to take time to question, think and question some more is another stand-out feature of these successful leaders.
It is why good questioning is also a leadership issue. Effective leaders often take the time needed to fully process the most important decisions to form what Gary Hamel and C.K. Pralahad refer to as strategic intent. Psychologist Daniel Goleman recently summarized the research linking mindfulness to focus and leadership, showing how leaders are able to better focus on what they do by being mindful. This mindful attention clears away much of the cognitive clutter to enable better question finding and asking.
Berger shows that finding the question requires some persistence. Good questioners are able to live with not having an answer or even the right question for a while. They have great patience. That ability to stand back and think, see, reflect and think some more while prototyping questions is what separates those who ask the better questions from those who don’t.
Creative collisions also helps. By mixing up ideas and connections with others, good questioners give themselves the raw material to work with. However, many of the best questioners that Berger spoke to also advocated for the need for some solitude and time to process these ideas and questions on their own. This mix of collaboration, collision, and independence is a key factor in developing the beautiful idea.
Designing better question-making
What jumped out at me in this book was how little support most organizations offer themselves for asking better, beautiful questions. Berger noted that the need for ‘serial mastery’ and constant learning is a staple of the new work environment, which should lend itself to question asking. However, if organizations are unwilling or unable to provide time for reflection, training, knowledge integration and ongoing discovery through better questions how likely is it that the workforce is going to respond to this need for new skills?
Are organizations willing to invest in a culture of inquiry? Are organizations able to make the leap from knowing things to asking things? How many public sector, non-profit, social and health service organizations (let alone industry groups) would be willing to follow companies like Google who create space — literally and figuratively — for questioning? These are some of the questions I asked myself as I read Berger’s book.
These are design questions. Berger notes how Google’s founders, Larry Page and Sergey Brin, were both Montessori school graduates. The Montessori system of education is based on question asking and Google is run as an organization largely framed around questions (and queries as noted by the very notion of “googling” something). Google has been designed to support better questions in its literal architecture of its software, its hardware, its office space, and the ‘20 per cent time‘ they offer employees to explore questions they have and projects that are of personal importance to them.
True to the idea of questions being worthy of paying attention to, Warren Berger’s book is filled with them including some answers. I liked the book and believe that he has tapped into something very big. Whether or not organizations and leaders will be inspired to ask better questions from this or simply try to find better answers in the processes they have is perhaps the big question next.
On a related note, March 14th has been dubbed Question Day by Berger and his colleagues at the Right Question Institute, a non-profit organization that provides support for teachers and students to ask better questions in school as a foundation for a lifetime of learning.
Berger, W. (2009). Glimmer: How Design Can Transform Your Life, Your Business, and Maybe Even the World. Toronto, ON: Random House Canada.
Berger, W. (2014). A more beautiful question: The power of inquiry to spark breakthrough ideas. New York, N.Y.: Bloomsbury Publishing.
Chand, I., & Runco, M. A. (1993). Problem finding skills as components in the creative process. Personality and Individual Differences, 14(1), 155–162.
Getzels, J. W. (1979). Problem Finding: a Theoretical Note. Cognitive Science, 3(2), 167–172. doi:10.1207/s15516709cog0302_4
Getzels, J. W. (1980). Problem Finding and Human Thought. The Educational Forum, 44(2), 243–244.
Goleman, D. (2013). Focus: The Hidden Driver of Excellence. New York, N.Y.: Harper Collins.
O’Neill, P. (2005). The ethics of problem definition. Canadian Psychology, 46(13-22).
Photo credit: Why? by Whitespeed via DeviantArt
The term evaluation has at its root the term value and to evaluate innovation means to assess the value that it brings in its product or process of development. It’s remarkable how much discourse there is on the topic of innovation that is devoid of discussion of evaluation, which begs the question: Do we value innovation in the first place?
The question posed above is not a cheeky one. The question about whether or not we value innovation gets at the heart of our insatiable quest for all things innovative.
A look at Google N-gram data for book citations provides a historical picture of how common a particular word shows up in books published since 1880. Running the terms innovation, social innovation and evaluation through the N-gram software finds some curious trends. A look at graphs below finds that the term innovation spiked after the Second World War. A closer look reveals a second major spike in the mid-1990s onward, which is likely due to the rise of the Internet.
In both cases, technology played a big role in shaping the interest in innovation and its discussion. The rise of the cold war in the 1950′s and the Internet both presented new problems to find and the need for such problems to be addressed.
Below that is social innovation, a newer concept (although not as new as many think), which showed a peak in citations in the 1960′s and 70s, which corresponds with the U.S. civil rights movements, expansion of social service fields like social work and community mental health, anti-nuclear organizing, and the environmental movement. This rise for two decades is followed by a sharp decline until the early 2000′s when things began to increase again.
Evaluation however, saw the most sustained increase over the 20th century of the three terms, yet has been in decline ever since 1982. Most notable is the even sharper decline when both innovation and social innovation spiked.
Keeping in mind that this is not causal or even linked data, it is still worth asking: What’s going on?
The value of evaluation
Let’s look at what the heart of evaluation is all about: value. The Oxford English Dictionary defines value as:
1 the regard that something is held to deserve; the importance, worth, or usefulness of something: your support is of great value.
• the material or monetary worth of something: prints seldom rise in value | equipment is included up to a total value of $500.
• the worth of something compared to the price paid or asked for it: at $12.50 the book is a good value.
2 (values) a person’s principles or standards of behavior; one’s judgment of what is important in life: they internalize their parents’ rules and values.
verb (values, valuing, valued) [ with obj. ]
1 estimate the monetary worth of (something): his estate was valued at $45,000.
2 consider (someone or something) to be important or beneficial; have a high opinion of: she had come to value her privacy and independence.
Innovation is a buzzword. It is hard to find many organizations who do not see themselves as innovative or use the term to describe themselves in some part of their mission, vision or strategic planning documents. A search on bookseller Amazon.com finds more than 63,000 titles organized under “innovation”.
So it seems we like to talk about innovation a great deal, we just don’t like to talk about what it actually does for us (at least in the same measure). Perhaps, if we did this we might have to confront what designer Charles Eames said:
Innovate as a last resort. More horrors are done in the name of innovation than any other.
At the same time I would like to draw inspiration from another of Eames’ quotes:
Most people aren’t trained to want to face the process of re-understanding a subject they already know. One must obtain not just literacy, but deep involvement and re-understanding.
Innovation is easier to say than to do and, as Eames suggested, is a last resort when the conventional doesn’t work. For those working in social innovation the “conventional” might not even exist as it deals with the new, the unexpected, the emergent and the complex. It is perhaps not surprising that the book Getting to Maybe: How the World is Changed is co-authored by an evaluator: Michael Quinn Patton.
While Patton has been prolific in advancing the concept of developmental evaluation, the term hasn’t caught on in widespread practice. A look through the social innovation literature finds little mention of developmental evaluation or even evaluation at all, lending support for the extrapolation made above. In my recent post on Zaid Hassan’s book on social laboratories one of my critique points was that there was much discussion about how these social labs “work” with relatively little mention of the evidence to support and clarify the statement.
One hypothesis is that evaluation can be seen a ‘buzzkill’ to the buzzword. It’s much easier, and certainly more fun, to claim you’re changing the world than to do the interrogation of one’s activities to find that the change isn’t as big or profound as one expected. Documentation of change isn’t perceived as fun as making change, although I would argue that one is fuel for the other.
Another hypothesis is that there is much mis-understanding about what evaluation is with (anecdotally) many social innovators thinking that its all about numbers and math and that it misses the essence of the human connections that support what social innovation is all about.
A third hypothesis is that there isn’t the evaluative thinking embedded in our discourse on change, innovation, and social movements that is aligned with the nature of systems and thus, people are stuck with models of evaluation that simply don’t fit the context of what they’re doing and therefore add little of the value that evaluation is meant to reveal.
If we value something, we need to articulate what that means if we want others to follow and value the same thing. That means going beyond lofty, motherhood statements that feel good — community building, relationships, social impact, “making a difference” — and articulating what they really mean. In doing so, we are better position to do more of what works well, change what doesn’t, and create the culture of inquiry and curiosity that links our aspirations to our outcomes.
It means valuing what we say we value.
(As a small plug: want to learn more about this? The Evaluation for Social Innovation workshop takes this idea further and gives you ways to value, evaluate and communicate value. March 20, 2014 in Toronto).