Category: evaluation

complexityevaluationjournalismpsychologystrategic foresight

Unpossible

Yinka's Ship

‘Post-truth’ was named the Oxford Dictionary word of the year. No fitting word reflects the strangeness of 2016 with the exception of unpossible, a word I made up and in a post-truth world might as well be as legitimate as many of the arguments being made about the most important things of the day, which is why we need to rethink how and what we pay attention to. 

When I was a little kid I was absolutely fascinated by ships in bottles (and still am). To me this was the embodiment of the impossible made possible. I’ve been shown how its done, read about it and still can’t really believe it despite seeing many ships in bottles over my lifetime. Gothic boxwood prayer beads are in the same category: they are both of the world and otherworldly at the same time. Brilliant stuff.

These are creations of human ingenuity, craft, patience and beauty.

What we have started to see in the social world are acts that are equally implausible to comprehend, yet lack all of these qualities but share one feature: creativity.

It may be time to examine what creativity means and what its impacts are because what might have been harmless chatter is now becoming big business and its transforming our world in ways we never could imagine and ways we might not really want.

In short: we are creating the unpossible.

Truthiness of fiction

Writing in Salon, Erin Keane reminds us that it was ten years ago that the concept of ‘truthiness’ was first floated out by Stephen Colbert and went on to become the 2006 Word of the Year by Mirriam-Webster. Keane reflects on the genesis of the word and how it articulated how a feeling of something being true could override the availability of evidence to support its existence without necessarily creating an entirely new reality.

With truthiness, though, we still recognized that truth exists, just that it could be overridden and bent to serve our own emotional purposes.

In a truthy world the absence of clear evidence didn’t mean that something didn’t exist if our feelings suggested that it might. Hence, we had an assault of Iraq and search for weapons of mass destruction based on a feeling that someone like Saddam Hussein would want to deploy them if he had them (which might have been true, but he didn’t have them and there was no evidence to suggest he did so it wasn’t true).

Now, those logical or hypothetical — if unproven — suppositions matter less. We’ve taken out ‘facts’ from the middle of the equation separating truth from fantasy.

In the US election, ‘fake news’ sites outperformed ‘not-fake news’ sites. In other words: those peddling fictions about the world drew more attention than those who sought to share what actually happened in the world. Except, what also actually happened was that people were reading, maybe believing, but certainly sharing and endorsing these made up stories, which were once referred to by names such as ‘lies’, ‘propaganda’ and ‘slander’. Now, it’s called reporting in a post-truth environment.

When the head of a news organization that promotes people who believe there ought to be a cap on women and girls in science and attacks citizen movements focused on social justice like Black Lives Matter is promoted to the role of chief strategist for the White House to serve as a representative of the people in strategy, that is post-truth at work. **

Tardigrade amnesia

The Tardigrade is perhaps the most remarkable animal on the planet. They can survive in temperatures close to absolute zero and over 150 degrees centigrade. If resilience had a mascot, it would be the tardigrade (pictured below — with credit to Bob Goldstein and Vicky Madden).

waterbear

While the effect of an election on policies and practices from healthcare, environmental protection, human rights, and safety and security may be wide-reaching and last beyond the term of office for most politicians the response can’t simply be to ‘toughen up’ and accept what’s being done, even if it is done under the banner of electoral legitimacy. Resilience is not about just absorbing shocks, but also about adapting to prevent the shocks from coming, to lessen their intensity, and also about systems change wherever possible.

The tardigrade is an expert on resiliency. It is as if it decided that, rather than plan for the best-case scenario, it figured out what the worst case would be and developed itself for that context first. Even if the tardigrade doesn’t encounter absolute zero temperatures that much in the world, it is ready for it.

Resiliency in social systems requires the same thinking.

In the US election and Brexit vote we saw politicians, pollsters and the media all get it wrong: they didn’t assess the mood and mindset of voters accurately. More importantly, voters may not have voted for what they are getting, but against what they got. In that case, what they ‘got'(i.e., had) was a sense of falling behind, perceived unfairness, absence of connection between their social world and the one talked about on TV or in government, and isolation from the economy, society and a world they thought they knew and were promised — something that built up over decades.

The voters wanted something different than what they had, but they may not have understood what they might get from this difference.

Foresight, in hindsight

Strategic foresight is a discipline that combines creative thinking, data, and planning together. It’s a burgeoning field of practice-based inquiry that offers an opportunity to explore various hypotheses about possible futures. We cannot reliably predict the future, particularly in complex systems, however it is possible to anticipate events based on trends, forecasts and signals that emerge from the data we have about the past and present when applied to the planning for the future.

Strategic foresight is a relatively young discipline, yet it holds much promise in aiding our ability to be resilient in the face of adversity and guide our actions to prevent problems and amplify those factors that can generate solutions. The result are ‘evidence-informed imaginations’ like the one that my colleague Peg Lahn and I did on the future of the neighbourhood in a growing city like Toronto, Canada. Ahead of legislation curbing the way high-rise building were built, we anticipated massive problems for Toronto’s high-rise condominiums based on the data we gathered and scenarios we developed. Falling glass was largely an ‘isolated’ incident 5 years ago and soon became a massive problem across the city and will continue to plague these buildings that will likely need to be completely ‘re-skinned’ in less than 20 years due to their reliance on poor design choices based on the city’s climate.

Our work bucked the trend toward optimism in condo development toward evidence-informed pessimism. Neither optimism or pessimism are ‘good’ or ‘bad’, rather what’s key is creating the kind of storyline that fits evidence, emotion and provides a narrative for what might happen. In doing so, a strategic plan can develop the kind of performance measures and monitoring and evaluation plans that help detect whether a particular scenario is starting to play out in the world. If so, it’s possible to correct things before they get problematic.

Strategic foresight combined with resiliency and systems thinking can be a way to envision the impossible as possible to prevent what becomes unpossible.

Consider what systems you’re working in and ask yourself if you’re seeing all (or many of) the pertinent possibilities and how they might play out. This is where fiction can be an asset, not a symptom, related to a larger issue. If you want some initial foresight into the current state of affairs in Western politics — from Le Pen in France, Farage in the UK,  Hofer in Austria, Wilders in the Netherlands, Trump in the United States, Kellie Leitch in Canada — dive into Sinclair Lewis’ 1935 classic “It Can’t Happen Here” .

It can.

The unpossible can only happen if we collectively create it.

Image credit: Yinka’s ship by Garry Knight used under Creative Commons License. Garry’s work is amazing and worth checking out. Thanks for sharing your art with the world!

** I struggled with the notion of even linking to this content, but also feel that I’m contributing to an echo chamber if those views aren’t seen and experienced, even if it’s just a small dose.

If we are to address truths — hard ones, complicated ones, ugly ones — we need to speak with truth and not pretend these voices aren’t there or comment on them if we are unwilling to expose ourselves to some of it in its original form and not solely filtered through other perspectives. One of the issues we face is that too often we (humans) speak about groups we know nothing about from any source that came from that perspective.

complexityeducation & learningevaluationinnovationknowledge translation

You Want It Darker?

shutterstock_150465248

It is poetic irony on many levels that weeks after Leonard Cohen releases his album about the threat of death that he passes on, mere days after we saw the least poetic, most crass election campaign end in the United States with an equally dramatic outcome. This points to art, but also to the science of complexity and how we choose to approach this problem of understanding– and whether we do at all — will determine whether we choose to have things darker or not. 

A million candles burning for the love that never came
You want it darker
We kill the flame

Canadian-born and citizen-of-the-world poet, literary author, and songwriter Leonard Cohen passed away last night and the words above were part of his final musical contribution to the world. It is fitting that those words were penned at time not only when Cohen was ill and dying, but also as we’ve witnessed the flames of social progress, inclusion, and diversity fall ill.

Donald Trump is the president-elect of the United States, a fact that for many is not only unpalatable, but deeply troubling for what it represents. A Trump presidency and the social ills that have been linked to his campaign are just the latest sign that we are well into a strange, fear-ful, period of history within Western democracies. His was not a win for ideas, policy, but personality and as a vector for many other things that simply cannot be boiled down exclusively to racism, sexism, celebrity, or education — although all of those things played some part. It was about the complexity of it all and the ability for simplicity to serve as a (false) antidote.

No matter what side of the political spectrum you sit, it’s hard to envision someone less suited to the job of President of a diverse, powerful nation like the United States than Donald Trump using any standard measure of leadership, personality, experience, personal integrity or record of public conduct. Yet, he’s in and his election provides another signal that we are living in complex times and, like with Brexit, the polls got it very wrong.

We are seeing global trade shrink at a time when globalization is thought to be at its highest. We are witnessing high-profile acts of hatred, discrimination and abuse at at time when we have more means to be socially connected across contexts than ever before. We are lonely when the world and connection is at our fingertips.  It is a time of paradox and when we have so many means to cast light on the world, we seem to find new ways to kill the flame.

It is for this reason that those who deal with complexity and seek positive social change in the world need to take action lest things get darker.

Complexity just got real

The election of Donald Trump and the Brexit vote are two examples that should serve to wake-up anyone who seeks greater accounting of complexity in the making of social decisions.

This is not about voting for a Republican President or for citizens wanting greater control of Britain, it’s about understanding the premise of which those decisions were based on. The amount of cognitive dissonance required to assume that Donald Trump has the qualities befitting a leader of a country like the United States is truly astounding. And just like Brexit, the theories and models proposed post-event by the same people who predicted the opposite outcome pre-event will be just words, backed with too little understanding of complexity or why things actually happened.

Those who understand complexity know that these simplistic explanations are likely to be problematic. But that doesn’t make us better people, but it does mean we have certain responsibilities.

Complexity rhetoric vs science

For those who rely on complexity science as a means of understanding these kinds of events its now time to start matching the science to our rhetoric so we can back up the talk. In crude, but truth-speaking pop culture parlance: “This shit just got real“.

As complexity and systems thinking has gained attention in social science and policy studies we are seeing much more attention to the idea of complexity. Yet, the level of rhetoric on social complexity has overwhelmed any instances of evidence of how complexity actually is manifest, emergent, harnessed, or accounted for in practical means.

This isn’t to say that the tenets of complexity for understanding social systems aren’t true, but rather we don’t know that it’s true for sure and to what extent in what situations. I write this as a true-believer, but also as one who believes in science. Science is about challenging our beliefs and only if we cannot refute our theories through our best efforts can claim something is true. Thus, if we can’t show consistently how the principles of complexity are employed to make useful choices and inform the documentation of some of the outcomes related to our actions based on those choices, we are simply making fables not flourishing organizations, communities and societies.

Showing our work

Without something more than rhetoric to back our claims up we become no better than a politician claiming to make America great again because we’ve got great ideas and will be the greatest president ever because we have great ideas.

This is not about reverting to positivist science to understand the entire world, but about responsible practice in evaluation and research that allows us to document what we do and explore the consequences in context. Powered by complexity theory and the appropriate methods, we can do this. Yet, too often I hear reference to complexity theories in presentations, discussions and papers without any reference to how its been used in real terms (and not just extracted from some other realm of science like bee colonies, natural ecosystems and simulation models) to influence something of value beyond serving as an organizing framework.

Like little kids in math class: we need to show our work.

How did complexity manifest in practice in this case? What methods were used to systematically document the process? How does this fit / challenge the theories we know? These are questions that are what responsible scientists and evaluators ask of their subjects and its time to do this with complexity, regularly and often. No longer can we give it the relatively unchallenged ride it’s been given since first being introduced as a viable contributor to social theory about 20 years ago.

The reasons have to do with what happens when we stop trying to understand complex systems.

Evaluators and social sciences’ new moral imperative

As the US election was unfolding I became aware of some prescient, wise words that were uttered by former US Supreme Court Justice David Souter speaking at a town hall prior to the last election. His words were chilling to anyone paying attention to the world today. In the quote and interview (see link) he says on the matter of government and democracy:

What I worry about is that when problems are not addressed, people will not know who is responsible.

His words are not just about the United States or even politics alone. The further we get from understanding how our social, economic, political and environmental systems work the more we all become vulnerable to the kind of simplistic thinking that leads us to someone that embodies H.L. Mencken’s mis-paraphrased words*:

There is always an easy solution to every human problem — neat, plausible, and wrong

It is our duty as scientists and evaluators to show the world the work of the programs, policies and initiatives that are aimed at changing systems — no matter what that system is. We need to be better at telling the story of programs using data and communicating what we learn to the world. It’s our role to show the work of others and to let others see our work in the process. By doing so we can make a contribution to helping address what Justice Souter meant about people not knowing who is responsible.

And like Mencken’s message, our answer won’t be one that is all that neat, but we if we approach our work with the wisdom and knowledge of how systems work we can avoid Mencken’s trap and avoid presenting the complex as simple, but we will go further and illustrate what complexity means.

It is our moral duty to do this. For if not us, who?

People do understand complexity. Anyone with a child or garden knows that there is no ‘standard practice’ that applies to all kids or any years’ crop of vegetables all the time in all cases. It’s evident all around us. We have the tools, theories and models to help illuminate this in the world and a duty to test them and make this visible to help shed that light on how our increasingly complex world works. Without that we are at risk of demagogues and the darker forces of our nature taking hold.

We have the means for people to see light through the work of those who build programs, policies and communities to illuminate our world. In doing so we not only create the candles as Leonard Cohen speaks of, but the curiosity and love that keeps that flame burning. We can’t kill the flame.

And we could use some love right now.

Thanks Leonard for sharing your gifts with us. I hope your art inspires us to reflect on what world you left to better create a world we move to.

*Mencken’s original quote was: “Explanations exist; they have existed for all time; there is always a well-known solution to every human problem — neat, plausible, and wrong.” Alas, this doesn’t make as pithy, Powerpoint worthy comment. Despite the incorrectness of the paraphrased quote attributed to Mencken, it’s fair to say that in many organizations we see this as a true statement nonetheless.

Image Credit: Shutterstock, used under licence.

behaviour changeevaluationpsychologysocial systemssystems thinking

Three Lessons From Summer Vacation

This year I took a summer holiday — something I’ve not done in years — and was reminded what literally stepping away from your everyday life and journeying to other spaces and places does for the mind, the heart and the soul. As kids (and adults) all over head back to school and tell their stories about their summer break, here is some of what I took away from my trip to the Netherlands and England. 

This time of year in the Northern Hemisphere is typically called “back to school” time as students return to their classrooms or start a new educational journey altogether. For young and old this new beginning signals a change of state and a great opening to experience, new knowledge and new people.

Among the first things students in grade school do upon their return is share what they did on their summer vacation. This year I am not returning to school, but I did have a summer vacation and in the spirit of ‘back-to-school’ I wanted to share what I learned.

My journey to the Netherlands and England allowed me to partake in sightseeing, walking for hours through some of Europe’s most beautiful towns and cities, taking in the art and history of two incredible regions, getting inspired, learning from other cultures, taking (and making) a rest, and also figuring out how to become a better football goalkeeper (more on that some other time). However, like any trip it is often about the people not just the places that make the difference and that is what I wish to focus on here.

The importance of beauty

P1020141

Jordaan District, Amsterdam, NL

Amsterdam is a beautiful city and so it was fitting that I spent one of my first days in that great European capital having lunch with author, consultant and designer, Steven de Groot talking about — among other things — beauty in organizations. Steven wrote what I believe is the most interesting doctoral theses I’ve ever read (PDF) on the role of beauty in organizations and has been developing new thinking on organizational aesthetics ever since. What makes Steven’s work so interesting is that he has taken a collection of ideas that are straightforward and simple on their own and brought them together to reveal something that is (paradoxically) incredibly obvious and yet completely unnoticed in most organizations.

Truth, beauty, goodness — these are things that we are attracted to, yet rarely identify as fundamental qualities of a high performing organization. Steven inspires people to rethink this through his writing and consulting.

I was speaking to a bartender at the local pub near my hotel about Steven’s work the evening before he and I met and the bartender went through the same stages as I did upon first encounter with the topic of organizational aesthetics: puzzlement, uncertainty, curiosity, wonder, confusion and then the big “a-ha!” where he realized how much sense it all makes (and asking why is this the first time contemplating all of this?). Why should we not value beauty in our work and workplaces and spaces? After all, we do it in almost every other facet of life and yet rarely do we consciously consider the role that aesthetics play in our organizational creations even if it is an enormous driver of behaviour and contributor to our wellbeing and quality of life.

Steven and I spoke of the challenges and opportunities inherent in inspiring people to think like a designer, wrestle with change in organizations, and overcome the (largely) self-imposed constraints to possibility that groups place on their perspectives about what is possible. We also spoke of the Dutch approach to constraints and how they’ve managed to work with a series of physical and social ones to create a society that largely supports innovative design in cities and organizations. The big challenge in drawing lessons from the Dutch (or anyone) is dealing with scale and determining how best to take ideas from one context further into others and what the implications are for transporting designs in one space to another. Beauty however exists everywhere in its own space and time, which is why it offers so much to designers working in different contexts: it’s inherently a local and global phenomenon simultaneously.

We make selections of friends, partners, places to live, products and services all based on some connection to beauty — even if that definition of beauty is different between each of us. Human beings have their own sense of beauty and are attracted to things we find beautiful so why would we not collectively nurture those qualities in the work processes, outcomes and environments we spend time in by design? We can design beauty into our work and Steven’s research and practice have pointed to ways in which people build appreciation for beauty, nurture it, and design it into the environments they inhabit everyday to enhance wellbeing, creativity and productivity. This is powerful stuff.

The power to change

Rothko_Snapseed

Red on Maroon by Mark Rothko (Tate Modern, London, UK)

Power is no more evident than in the process of change-making and I have found few more thoughtful on this topic than John Wenger. John is a London-based (via New Zealand) Scot who is a psychotherapist, organizational change consultant, community animator and writer. John seeks to find, build and nurture what Meg Wheatley calls “islands of sanity” in a complex and chaotic world through his therapy, community and consulting work. In that vein he and I met at the Victoria & Albert museum for a coffee and a stroll through some of the collections as an island of cool on a hot summer’s day.

This was just the start our our journey as we soon found ourselves zipping over to the Tate Modern and then for a long walk through the Borough Market and along the Thames talking all the way about the role of capitalism, community, individual behaviour, organizational design and social connection in shaping the world we work in. In the frenetic chaos of London’s core during the height of tourist season John and I created our own island of sanity (and inspiration) through our walk and time together.

At the heart of our art visit was a trip to the Tate Modern to see a collection of works by Mark Rothko, the Seagram Murals immortalized in John Logan’s stage play Red. These works embody the kind of complexity that anyone working with human systems knows, but rarely can communicate so eloquently: the (appearing simple) works take vastly different shapes depending on where you sit in relation to them. Rothko’s pieces, like the one pictured above, look one way in a photograph, another from across the room, and something else when closer. Lighting matters, too as I suspect they would look considerably different in different gallery spaces. Rothko knew this and that was part of his genius. Each of these perspectives provides a new layer of information, wonder and reveals new patterns within what appears to be a simplistic frame of four lines connected into a box.

John understands that the same qualities that make Rothko’s work so mesmerizing is what also makes human relationships so important. It’s easy to make a judgement from afar, but it is only through getting closer, stepping away, turning things on their head that we begin to see things differently and, with it, open up possibilities. In the calamitous wake of Brexit John is leading workshops to help people make sense of what it means for Britain (and themselves) and is encouraging healing through use of reflective dialogue and sociodrama. He sees the rift created when, no matter what your position on Brexit might be, you see one half of your country holding a counter position to your own on a matter of great importance. How do we live together, see the differences, embrace the opportunities that come from difference and bridge the gap between what we see, what we know and what we do?

It is fitting that our day of conversation, food, and walking would be filled with art because that artistry and the attention to the way we co-create reality through art is what John brings to the world. It was a reminder of the power of  relationships to bring out our best and reveal new pathways to those islands of sanity that we might miss if we simply approach the world head-on in a cognitive-rational manner and take the world as its presented to us by media, social norms and our past interpretations of history. Art is a gift embraced with others.

Conserving the planet, humbly

IMG_4749

Twins of Brick Lane by Zabou, Brick Lane, Spitalfields, London, UK

Alas, there are times when history cannot be avoided as a means of understanding our future and ecological sustainability is one of those issues. Right now, humanity is much like the twins above peering at us in Brick Lane: some of we call ‘art’ is a little noxious for the planet.

Andrew Knight and I along with a small group of conservation biologists, systems thinkers and evaluators are part of the Silwood Group: a ‘praxis’ tank (as opposed to a think tank) that seeks to bring new thinking about conservation and ecological systems together with concrete action to advance our protection of the planet. It sounds like heady stuff, but like an inversion of Rothko’s paintings there are areas of simplicity within the complexity presented by these issues that Andrew and I sought to work on over two days spent at Imperial College, where Dr. Knight is a Senior Lecturer.

Aside from some light-hearted discussion on the comic genius of the Flight of the Conchords or the maddening systems of administration within universities, our time together dealt with the ways in which we, as a transdisciplinary blend of scholars, practitioners and ‘pracademics’ from across the world and from different sectors who are the Silwood Group, could make the biggest difference in the shortest amount of time with our limited resources — a typical conservation problem if ever there was one!

This is really a contemplative problem that combines many of the aspects of what Steven de Groot and I spoke of in Amsterdam and the conversations I had with John Wenger strolling along the Thames.

It is about creating / designing spaces and products that allow people to engage with complexity and the volume of issues that are entangled within conservation and to do so through by anchoring the work to beauty and to relationships. One avenue is through education (*and by education, we were thinking about real praxis-led learning and not just packaged toolkits, lesson plans and classrooms with rows and PowerPoint) because it is through curiosity, exchange, exposure to new thinking and the opportunity to try things out that we build the kind of relationships to people, organizations and ideas that allow them to stick.

Two days was not enough to flesh this idea out much further, but needless to say that there will certainly be much reflecting upon my summer vacation in the months to come in ways that I am only now, getting back to everyday life in Canada, appreciating had such an impact on my thinking. More, much more is to come.

Thanks Steven, John and Andrew for providing such inspiration, insight, camaraderie and intellectual and social companionship on my journey this summer. I am looking forward to building on that with you in the days and months to come and, like a Rothko painting, finding new meanings and layers to the work every time I encounter it.

Photo credits: Author.

Notes:

Mark Rothko’s work at the Tate is staggering in what it elicits when seen in person. Go see it. For more information click here.

Zabou’s street art can be found here, but like all art it’s better to engage with it up close if you can find it and Brick Lane in London is as good as anywhere to see some beautiful street art.

If you are in the UK (or even if you aren’t, I suppose) and interested in John Wenger’s post-Brexit workshop entitled Who Shall Survive Brexit on October 6th you can register for it here.

 

complexityevaluationsystems thinking

Diversity / Complexity in Focus

5901575754_a7ca159cd4_b

Diversity in focus?

 

As cities and regions worldwide celebrate Pride the role of diversity, understanding and unity has been brought to mind just as it has contemplating the British public’s choice to leave the EU with Brexit. Both events offer lessons in dealing with complexity and why diversity isn’t about either/or, but more both and neither and that we might learn something not just from the English, but their gardens, too. 

It’s a tad ironic that London has been celebrating its Pride festival this past week, a time when respect and celebration of diversity as well as unification of humanity was top-of-mind while the country voted to undo many of the policies that fostered political and economic union and could likely reduce cultural diversity with Europe. But these kind of ironies are not quirky, but real manifestations of what can happen when we reduce complexity into binaries.

This kind of simplistic, reductionist thinking approach can have enormously harmful and disrupting effects that ripple throughout a system as we are seeing with what’s happened so far in the United Kingdom, Europe and the world in the past week.

Complexity abhors dichotomies

There are two kinds of people in the world: those who believe there are two kinds of people and those who don’t

The above quote (which has many variations, including one attributed to author Tom Robbins that I like) makes light of the problem of lumping the complex mass of humanity into two simple categories. It abstracts variation to such a level that it becomes nearly meaningless. The Brexit vote is similar. Both are lessons in complexity lived in the world because they reflect a nuanced, mutli-faceted set of issues that are reduced into binary options that are clustered together.

It is no surprise that, in the days following the Brexit vote in the UK, that there is much talk of a divided, rather than a united kingdom.

Diversity is difficult to deal with and is often left unaddressed as a result. The benefits to having diversity expressed and channeled within a complex system are many and articulated in research and practice contexts across sectors and include protection from disruption, better quality information, a richer array of strategic options and, in social contexts, a more inclusive social community.

The risks are many, too, but different in their nature. Diversity can produce tension which can be used for creative purposes, liberation, insight as well as confusion and conflict, simultaneously. This as a lot do with humans uneasy relationship with change. For some, change is easier to deal with by avoiding it — which is what many in the Leave camp thought they could do by voting the way they did. The darker side of the Leave campaign featured change as an image of non-white immigrant/refugees flooding into Britain, presumably to stoke those uncomfortable with (or outwardly hostile) to others to fear the change that could come from staying in the European Union.

Staying the same requires change

The author Guiseppe de Lampedussa once wrote about the need to change even when desiring to keep things as they are, because even if we seek stability, everything around us is changing and thus the system (or systems) we are embedded in are in flux. That need to change to stay the same was something that many UK citizens voiced. What was to change and what was to stay the same was not something that could be captured by a “Leave” or “Remain” statement, yet that is what they were given.

It should come to no surprise that, when presented with a stark choice on a complex matter, that there would be deep dissatisfaction with the result no matter what happened. We are seeing the fallout from the vote in the myriad factions and splintering of both of the main political parties — Conservative and Labour — and a House of Commons that is now filled with rebellion. Is the UK better off? So far, no way.

This is not necessarily because of the Leave vote, but because of what has come from the entire process of mis-handling the campaigns and the lack of plan for moving forward (by both camps). Further complicating matters is that the very EU that Britain has voted to leave is now not the same place as it was when the Brexit vote was held just five days ago. It’s also faced with rising voices for reform and potential separation votes from other member states who saw their causes bolstered or hindered because of the UK referendum. This is complexity in action.

Tending the garden of complex systems

The English know more about complexity than they might realize. An English garden is an example of complexity in action and how it relates to the balance of order, disorder and unordered systems. A look at a typical English garden will find areas of managed beauty, wildness, and chaos all within metres of one another.

9556155847_5eaf2a7eee_h

What also makes a garden work is that it requires the right balance of effort, adaptive action, planning and compensating as well as the ability to let go all at the same time. Gardening requires constant attention to the weather, seasons, the mix of diversity within the system, the boundaries of the system itself (lest weeds or other species seek to invade from outside the garden or plants migrate out to a neighbours place) and how one works with all of it in real time.

Social systems are the same way. They need to be paid attention to and acted upon strategically, in their own time and way. This is why annual strategic planning retreats can be so poorly received. We take an organization with all it’s complexity and decide that once per year we’ll sit down and reflect on things and plan for the future. Complexity-informed planning requires a level of organizational mindfulness that engages the planning process dynamically and may involve the kind of full-scale, organization-wide strategy sessions more frequently or with specific groups than is normally done. Rather than use what is really arbitrary timelines — seen in annual retreats, 5-year plans and so forth — the organization takes a developmental approach, like a gardener, and tends to the organizations’ strategic needs in rhythms that fit the ecosystem in which it finds itself.

This kind of work requires: 1) viewing yourself as part of a system, 2) engaging in regular, sustained planning efforts that have 3) alignment with a developmental evaluation process that continually monitors and engages data collection to support strategic decision-making as part of 4) a structured, regular process of sensemaking so that an organization can see what is happening and make sense of it in real-time, not retrospectively because one can only act in the present, not the future or past.

Just as a garden doesn’t reduce complexity by either being on or off, neither should our social or political systems. Until we start realizing this and acting on it — by design — at the senior strategic level of an organization, community or nation, we may see Brexit-like conditions fostered in places well beyond the white cliffs of Dover into governments and organizations globally.

Photo Credits: The London Eye Lit Up for Pride London by David Jones and Hidcote Manor GardenHidcote Manor GardenHidcote Manor Garden by David Catchpole both used under Creative Commons License via Flickr. Thanks to the Davids for sharing their work.

evaluationsocial innovation

Flipping the Social Impact Finger

7150802251_e1c559d402_k

Look around and one will notice a lot of talk about social enterprise and social impact. Look closer and you’ll find a lot more of the former and far less of the latter. 

There’s a Buddhist-inspired phrase that I find myself reflecting on often when traveling in the social innovation/entrepreneurship/enterprise/impact sphere:

Do not confuse the finger pointing to the moon for the moon itself

As terms like social enterprise and entrepreneurship, social innovation, social laboratories and social impact (which I’ll lump together as [social] for expediency of writing) become better known and written about its easy to caught up in the excitement and proclaiming its success in changing the world. Indeed, we are seeing a real shift in not only what is being done, but a mental shift in what can be perceived to be done among communities that never saw opportunities to advance before.

However exciting this is, there is what I see as a growing tendency to lose the forest amid the trees by focusing on the growth of [social] and less on the impact part of that collection of terms. In other words, there’s a sense that lots of talk and activity in [social] is translating to social impact. Maybe, but how do we know?

Investment and ROI in change

As I’ve written before using the same guiding phrase cited above, there is a great tendency to confuse conversation about something with the very thing that is being talked about in social impact. For all of the attention paid to the amount of ventures and the amount of venture capital raised to support new initiatives across the social innovation spectrum in recent years, precious little change has been witnessed in the evaluations made available of these projects.

As one government official working in this sector recently told me:

We tend to run out of steam after (innovations) get launched and lose focus, forgetting to evaluate what kind of impact and both intended and unintended consequences come with that investment

As we celebrate the investment in new ventures, track the launch of new start-ups, and document the number of people working in the [social] sector we can mistake that for impact. To be sure, having people working in a sector is a sign of jobs, but the question of whether they are temporary, suitably paying, satisfactory, or sustainable are the kind of questions that evaluators might ask and remain largely unanswered.

The principal ROI of [social] is social benefit. That benefit comes in the form of improved products and services, better economic conditions for more people, and a healthier planet and wellbeing for the population of humans on it in different measures. These aren’t theoretical benefits, they need to be real ones and the only way we will know if we achieve anything approximating this is through evaluation.

Crashing, but not wrecking the party

Evaluation needs to crash the party, but it need not kill the mood. A latent fear among many in [social] is likely that, should we invest so much energy, enthusiasm, money and talent on [social] and find that it doesn’t yield the benefits we expect or need a fickle populace of investors, governments and the public will abandon the sector. While there will always be trend-hunters who will pursue the latest ‘flavour of the month’, [social] is not that. It is here to stay.

The focus on evaluation however will determine the speed, scope and shape of its development. Without showing real impact and learning from those initiatives that produce positive benefit (or do not) we will substantially limit [social] and the celebratory parties that we now have at the launch of a new initiative, a featured post on a mainstream site, or a new book will become fewer and farther between.

Photo credit: Moonrise by James Niland used under Creative Commons licence via Flickr. Thanks for sharing your art, James.

education & learningevaluation

Reflections said, not done

27189389172_cb3f195829_k

 

Reflective practice is the cornerstone of developmental evaluation and organizational learning and yet is one of the least discussed (and poorly supported) aspects of these processes. It’s time to reflect a little on reflection itself. 

The term reflective practice was popularized by the work of Donald Schön in his book The Reflective Practitioner, although the concept of reflecting while doing things was discussed by Aristotle and serves as the foundation for what we now call praxis. Nonetheless, what made reflective practice as a formal term different from others was that it spoke to a deliberative process of reflection that was designed to meet specific developmental goals and capacities. While many professionals had been doing this, Schön created a framework for understanding how it could be done — and why it was important — in professional settings as a matter of enhancing learning and improving innovation potential.

From individual learners to learning organizations

As the book title suggests, the focus of Schön’s work was on the practitioner her/himself. By cultivating a focus, a mindset and a skill set in looking at practice-in-context Schön (and those that have built on his work) suggest that professionals can enhance their capacity to perform and learning as they go through a series of habits and regular practices by critically inquiring about their work as they work.

This approach has many similarities to mindfulness in action or organizational mindfulness, contemplative inquiry, and engaged scholarship among others. But, aside from organizational mindfulness, these aforementioned approaches are designed principally to support individuals learning about and reflecting about their work.

There’s little question that paying attention and reflecting on what is being done has value for someone seeking to improve the quality of their work and its potential impact, but it’s not enough, at least in practice (even if it does in theory). And the evidence can be found in the astonishing absence of examples of sustained change initiatives supported by reflective practice and, more particularly, developmental evaluation, which is an approach for bringing reflection to bear on the way we evolve programs over time. This is not a criticism of reflective practice or developmental evaluation per se, but the problems that many have in implementing it in a sustained manner. From professional experience, this comes down largely to the matter of what is required to actually do reflective practice or any in practice. 

For developmental evaluation it means connecting what it can do to what people actually will do.

Same theories, different practices

The flaw in all of this is that the implementation of developmental evaluation is often predicated on implicit assumptions about learning, how it’s done, who’s responsible for it, and what it’s intended to achieve. The review of the founding works of developmental evaluation (DE) by Patton and others point to practices and questions that that can support DE work.

While enormously useful, they make the (reasonable) assumption that organizations are in a position to adopt them. What is worth considering for any organization looking to build DE into their work is: are we really ready to reflect in action? Do we do it now? And if we don’t, what makes us think we’ll do it in the future? 

In my practice, I continually meet organizations that want to use DE, be innovative, become adaptive, learn more deeply from what they do and yet when we speak about what they currently do to support this in everyday practice few examples are presented. The reason is largely due to time and the priorities and organization of our practice in relation to time. Time — and its felt sense of scarcity for many of us — is one of the substantive limits and reflective practice requires time.

The other is space. Are there places for reflection on issues that matter that are accessible? These twin examples have been touched on in other posts, but they speak to the limits of DE in affecting change without the ability to build reflection into practice. Thus, the theory of DE is sound, but the practice of it is tied to the ability to use time and space to support the necessary reflection and sensemaking to make it work.

The architecture of reflection

If we are to derive the benefits from DE and innovate more fully, reflective practice is critical for without one we can’t have the other. This means designing in reflective space and time into our organizations ahead of undertaking a developmental evaluation. This invites questions about where and how we work in space (physical and virtual) and how we spend our time.

To architect reflection into our practice, consider some questions or areas of focus:

  • Are there spaces for quiet contemplation free of stimulation available to you? This might mean a screen-free environment, a quiet space and one that is away from traffic.
  • Is there organizational support for ‘unplugging’ in daily practice? This would mean turning off email, phones and other electronic devices’ notifications to support focused attention on something. And, within that space, are there encouragements to use that quiet time to focus on looking at and thinking about evaluation data and reflecting on it?
  • Are there spaces and times for these practices to be shared and done collectively or in small groups?
  • If we are not granting ourselves time to do this, what are we spending the time doing and does it add more value than what we can gain from learning?
  • Sometimes off-site trips and scheduled days away from an office are helpful by giving people other spaces to reflect and work.
  • Can you (will you?) build in — structurally — to scheduled work times and flows committed times to reflect-in-action and ensure that this is done at regular intervals, not periodic ones?
  • If our current spaces are insufficient to support reflection, are we prepared to redesign them or even move?

These are starting questions and hard ones to ask, but they can mean the difference between reflection in theory and reflection in practice which is the difference between innovating, adapting and thriving in practice, not just theory or aspiration.

 

 

 

 

evaluationsocial innovation

Benchmarking change

The quest for excellence within social programs relies on knowing what excellence means and how programs compare against others. Benchmarks can enable us to compare one program to another if we have quality comparators and an evaluation culture to generate them – something we currently lack. 

5797496800_7876624709_b

A benchmark is something used by surveyors to provide a means of holding a levelling rod to determine some consistency in elevation measurement of a particular place that could be compared over time. A benchmark represents a fixed point for measurement to allow comparisons over time.

The term benchmark is often used in evaluation as a means of providing comparison between programs or practices, often taking one well-understood and high performing program as the ‘benchmark’ to which others are compared. Benchmarks in evaluation can be the standard to which other measures compare.

In a 2010 article for the World Bank (PDF), evaluators Azevedo, Newman and Pungilupp, articulate the value of benchmarking and provide examples for how it contributes to the understanding of both absolute and relative performance of development programs. Writing about the need for benchmarking, the authors conclude:

In most benchmarking exercises, it is useful to consider not only the nature of the changes in the indicator of interest but also the level. Focusing only on the relative performance in the change can cause the researcher to be overly optimistic. A district, state or country may be advancing comparatively rapidly, but it may have very far to go. Focusing only on the relative performance on the level can cause the researcher to be overly pessimistic, as it may not be sufficiently sensitive to pick up recent changes in efforts to improve.

Compared to what?

One of the challenges with benchmarking exercises is finding a comparator. This is easier for programs operating with relatively simple program systems and structures and less so for more complex ones. For example, in the service sector wait times are a common benchmark. In the province of Ontario in Canada, the government provides regularly updated wait times for Emergency Room visits via a website. In the case of healthcare, benchmarks are used in multiple ways. There is a target that is used as the benchmark, although, depending on the condition, this target might be on a combination of aspiration, evidence, as well as what the health system believes is reasonable, what the public demands (or expects) and what the hospital desires.

Part of the problem with benchmarks set in this manner is that they are easy to manipulate and thus raise the question of whether they are true benchmarks in the first place or just goals.

If I want to set a personal benchmark for good dietary behaviour of eating three meals a day, I might find myself performing exceptionally well as I’ve managed to do this nearly every day within the last three months. If the benchmark is consuming 2790 calories as is recommended for someone of my age, sex, activity levels, fitness goals and such that’s different. Add on that, within that range of calories, the aim is to have about 50% of those come from carbohydrates, 30% from fat and 20% from protein, and we a very different set of issues to consider when contemplating how performance relates to a standard.

One reason we can benchmark diet targets is that the data set we have to set that benchmark is enormous. Tools like MyFitnessPal and others operate to use benchmarks to provide personal data to its users to allow them to do fitness tracking using these exact benchmarks that are gleaned from having 10’s of thousands of users and hundreds of scientific articles and reports on diet and exercise from the past 50 years. From this it’s possible to generate reasonably appropriate recommendations for a specific age group and sex.

These benchmarks are also possible because we have internationally standardized the term calorie. We have further internationally recognized, but slightly less precise, measures for what it means to be a certain age and sex. Activity level gets a little more fuzzy, but we still have benchmarks for it. As the cluster of activities that define fitness and diet goals get clustered together we start to realize that it is a jumble of highly precise and somewhat loosely defined benchmarks.

The bigger challenge comes when we don’t have a scientifically validated standard or even a clear sense of what is being compared and that is what we have with social innovation.

Creating an evaluation culture within social innovation

Social innovation has a variety of definitions, however the common thread of these is that its about a social program aimed at address social problems using ideas, tools, policies and practices that differ from the status quo. Given the complexity of the environments that many social programs are operating, it’s safe to assume that social innovation** is happening all over the world because the contexts are so varied. The irony is that many in this sector are not learning from one another as much as they could, further complicating any initiative to build benchmarks for social programs.

Some groups like the Social Innovation Exchange (SIX) are trying to change that. However, they and others like them, face an uphill battle. Part of the reason is that social innovation has not established a culture of evaluation within it. There remains little in the way of common language, frameworks, or spaces to share and distribute knowledge about programs — both in description and evaluation — in a manner that is transparent and accessible to others.

Competition for funding, the desire to paint programs in a positive light, lack of expertise, not enough resources available for dissemination and translation, absence of a dedicated space for sharing results, and distrust or isolation from academia among certain sectors are some reasons that might contribute to this. For example, the Stanford Social Innovation Review is among the few venues dedicated to scholarship in social innovation aimed at a wide audience. It’s also a venue focused largely on international development and what I might call ‘big’ social innovation: the kind of works that attract large philanthropic resources. There’s lot of other types of social innovation and they don’t all fit into the model that SSIR promotes.

From my experiences, many small organizations or initiatives struggle to fund evaluation efforts sufficiently, let alone the dissemination of the work once it’s finished. Without good quality evaluations and the means to share their results — whether or not they cast a program in positive light or not — it’s difficult to build a culture where the sector can learn from one another. Without a culture of evaluation, we also don’t get the volume of data and access to comparators — appropriate comparators, not just the only things we can find — to develop true, useful benchmarks.

Culture’s feast on strategy

Building on the adage attributed to Peter Drucker that culture eats strategy for breakfast (or lunch) it might be time that we use that feasting to generate some energy for change. If the strategy is to be more evidence based, to learn more about what is happening in the social sector, and to compare across programs to aid that learning there needs to be a culture shift.

This requires some acknowledgement that evaluation, a disciplined means of providing structured feedback and monitoring of programs, is not something adjunct to social innovation, but a key part of it. This is not just in the sense that evaluation provides some of the raw materials (data) to make informed choices that can shape strategy, but that it is as much a part of the raw material for social change as enthusiasm, creativity, focus, and dissatisfaction with the status quo on any particular condition.

We are seeing a culture of shared ownership and collective impact forming, now it’s time to take that further and shape a culture of evaluation that builds on this so we can truly start sharing, building capacity and developing the real benchmarks to show how well social innovation is performing. In doing so, we make social innovation more respectable, more transparent, more comparable and more impactful.

Only by knowing what we are doing and have done can we really sense just how far we can go.

** For this article, I’m using the term social innovation broadly, which might encompass many types of social service programs, government or policy initiatives, and social entrepreneurship ventures that might not always be considered social innovation.

Photo credit: Redwood Benchmark by Hitchster used under Creative Commons License from Flickr.

About the author: Cameron Norman is the Principal of Cense Research + Design and works at assisting organizations and networks in supporting learning and innovation in human services through design, program evaluation, behavioural science and system thinking. He is based in Toronto, Canada.