Social innovation, social inclusion

Inclusion means everyone

Social innovation is about bringing new ideas, products and services out into society with others for social benefit and improving the lives of our communities. While not every innovation will benefit everyone, there is a need to examine more deeply the question of who benefits(?) when we consider social innovation and that means taking some hard looks at who we are innovating for. 

On August 15th 2015 the New York Times ran a feature story titled Inside Amazon, which looked at the corporate culture inside one of the largest, most innovate retailers in the world. In the piece written by award-winning journalists Jodi Kantor and David Strietfeld, they interview more than 100 current and former employees of Amazon and find a culture that is fast-paced, exciting, dynamic, creative and sometimes cruel, relentless in its expectations of its employees, overwhelming and harsh. What was interesting is that many interviewees spoke in conflicting terms about working for the company which offered great compensation and a stimulating workplace with lots of opportunities to grow while simultaneously burning them out and challenging their sense of self in the process of delivering feedback that wasn’t always experienced as constructive.

Across the news aisle we find another example of innovation in the news. In the September issue of the Walrus Magazine, editor-in-chief Jonathan Kay returns to the front lines of reporting with a feature story called Uber v. Taxi (or The Truth about Uber on the cover), which takes a comparative examination of changing business models and culture around cars-for-hire comparing tech start-up Uber with the traditional taxi model. The piece involves Kay signing up to be an Uber driver and also completing the City of Toronto taxi school to get a first-hand look at both systems from the perspective of driver and passenger. In an interview on CBC Radio, Kay was asked about the differences between the two and commented on how Uber was working well for the young, the mobile and able-bodied whereas traditional taxis were left with the others, creating a gap in income and opportunity between the two services:

That’s where drivers make a ton of money. Uber is taking that. Taxis are being left with older people, people with special needs, people who require wheelchair access and the visually impaired. Those are the people who require special training and vehicles that taxi fleets can provide but that’s not a particularly profitable part of the trade. Those trips take a lot of time and effort and passenger care. There’s not enough money on the table left for the taxi drivers to make a living.

Innovating for whom?

What these two stories have in common is that it profiles the way innovation spaces can divide as much as unite. On the surface, we see two examples of ways in which new thinking, careful product design and marketing, and a focused attention on user experience can generate value for consumers. However, what they also illustrate is that what is perceived as value is largely contingent on whom it is being asked and that this perception is not a minority position. This is not a case of blacksmiths getting outraged at the dwindling market for horseshoes due to the automobile or manufacturers of picture tubes castigating people for buying digital televisions. This is a case of entire segments of the population being left out.

Both of these examples are based on age to illustrate a point of commonality.

In the case of Uber, its the young, urban professional who does well by its innovative model. It’s the person who has few things to carry, needs little assistance, and likes to travel to the popular places where there are many others like them, which creates an ideal marketplace. For taxis, they are being asked to go to out-of-the-way places (like doctors appointments), deliver people and their parcels (for people who aren’t highly mobile), and are bound by a set of rules that Uber is not to ensure that they assist those who need it in using their service. Uber gets the cream of the market, while taxis are left with what’s left and that is mostly older adults.

But what ‘older’ means is a matter of perspective as we see with Amazon. As the reporters explain, old age isn’t what it once was:

In interviews, 40-year-old men were convinced Amazon would replace them with 30-year-olds who could put in more hours, and 30-year-olds were sure that the company preferred to hire 20-somethings who would outwork them. After Max Shipley, a father of two young children, left this spring, he wondered if Amazon would “bring in college kids who have fewer commitments, who are single, who have more time to focus on work.” Mr. Shipley is 25.

Every innovation produces ‘winners’ and ‘losers’, but what is striking in both articles is that the ‘winners’ are a very narrow band of the population, young, urban professionals. A look across what we often gets heralded as innovation (pick up any issue of Fast Company magazine to see it) and you’ll see a world dominated by (mostly) young, (mostly) white, (mostly) male, (mostly) middle class, and (mostly) tech-driven innovations that come from places and cultures like Silicon Valley. Facebook, Apple, Google, Uber, AirBnB — they are all based in Silicon Valley.

How we design innovations and the cultures we create in that process can have enormous implications. Are we creating our own silicon valley for social innovation?

“Slamming the Door on Silicon Valley”

Jess Zimmerman, writing in The Guardian, remarked on how Silicon Valley’s culture is one of entitlement and male hegemony, pointing to work of women’s groups aimed at making the work culture in the valley more female-friendly. Even though Sheryl Sandberg’s Lean In is a product of that environment, it not of that environment. “The Valley” is an environment that fosters both Uber and Amazon (which is should be noted is based in Washington State and not Silicon Valley, but nonetheless is part of the same cultural milieu discussed here). That ethos is one that is characterized by cultures of hard work, long hours, dynamism and youth. As a result, a path dependence is created based on the design specifications proposed at the start and leads to products that are, no surprise, a reflection of their makers.

Facebook’s features of ‘extreme openness’ as evidenced by it’s settings that make it hard to keep things private and rules against using pseudonyms can be traced back to Mark Zuckerberg’s dorm room at Harvard and his personality and personal belief system about what social life is to be like. As a result, Zuckerberg’s design has influenced online interactions of more than one billion users worldwide and continues today.

So what does this have to do with social innovation? Consider the literature — wide in scope, thin in detail as it may be — on social innovation methods and tools from social labs to design thinking. What we might find is an incomplete list of items that looks something like this:

  1. Be bold, bring wild ideas to the table and lots of them to the table; no idea is a bad idea
  2. Co-create with others
  3. We live in a VUCA (Volitile, unpredictable, complex, ambiguous) world and need to work accordingly
  4. Flat organizational structures work best for innovation
  5. Innovation doesn’t happen during 9-5, it happens anytime
  6. Information technology will leverage creative innovation potential everywhere, anywhere: it always wins
  7. You have to ‘move fast and break stuff‘, including the rules

The list can go on.

While I have  belief in what is contained in this list, it’s a restrained belief. Each of these points (and there are many others) can be upended to illustrate how social innovation can exclude people, ideas, cultures and possibilities that are as harmful as helpful. As I’ve argued before, social innovation has embedded in it an ethic of social justice if it’s to truly be a true social innovation. This requires attention to the ‘winners’ and ‘losers’ of innovation in ways that go beyond a call to innovate and change, it means paying attention to the cultures we impose through the innovation process.

Do we place too much emphasis on disruption vs harmony?

Where is the role for contemplation in the speed to create new things?

Is there a place for an introvert in the innovation table?

While innovative ideas might not respect the 9-5 clock, many paycheques, office spaces, commuter schedules, daycares and employee benefits do, what does that mean for those who rely on this?

Are these values those of innovation or those of a particular type of innovation from a particular context?

The Trickle of Innovation Streams Through the Valley

If we are to adopt social innovation on a wide scale we need to create a culture of innovation that is more than just a new version of a trickle-down model. Indeed, as Geoff Mulgan from Nesta writes, innovation has the potential to be another ‘trickle down theory’ that rewards the most advantaged first and then eventually to others in some modest form, creating inequities.

Yes, we now know much more about how to cultivate buzzing creative industries, universities, knowledge intensive industries and so on. But we have almost nothing to say to around half of our population who face the prospect of bad jobs or no jobs, and look on with dismay and envy at the windfall gains accruing to the elite insiders.

Silicon Valley is currently the place of privilege in the innovation world. If you have the privilege of not needing add-ons to your taxi ride, require assistance or have to drive to a neighbourhood that’s off the beaten path or have to pay by cash, Uber is great. If you can work flex hours and long hours, are gregarious and extroverted, and aren’t temporally limited by the needs of a spouse or partner, children, a loved one who requires care, or pets (that can’t be brought to work for obvious reasons — and I’m thinking of you cat owners) then a place like Amazon is maybe for you.

When we use these spec’s as our models to design innovation more widely, including social innovation, we create systems that exclude as much as include and that might get us innovations, but not necessarily real social ones.

Social innovation, social justice and the emotional link between them

Justice

Social innovations are judged by their impact, but in the quest to assess what it does we can miss the way it does it and that is where justice and the emotional connections that justice deals with come into play. Unless we consider social justice a part of social innovation we are likely to exclude as much as we include the very people we need to help bring good ideas to light and promote true social change and development. 

Social innovation is most often characterized with emphasis on new ideas and products generated in social ways. The social part of social innovation is what distinguishes it from other forms that don’t require that same social engagement.

Social innovation has been defined in the following ways such as:

” a novel solution to a social problem that is more effective, efficient, sustainable, or just than existing solutions and for which the value created accrues primarily to society as a whole rather than private individuals.” – Phills, Deiglmeier, & Miller (2008)

Social Innovation Generation and Frances Westley describe social innovation as:

“Social innovation is an initiative, product, process or program that profoundly changes the basic routines, resource and authority flows or beliefs of any social system.”

And Geoff Mulgan, the CEO of Nesta in the UK provides perhaps the simplest of definitions:

“Social innovation is a new idea that meets social goals” — Geoff Mulgan (2013)

In all of these definitions the emphasis is on the new idea and the social environment in which that idea is cast. The first of these definitions above is the most detailed and includes mention of those new ideas being more just than those that are being replaced. Frances Westley’s definition speaks to authority flows and Mulgan’s addresses social goals. How social innovation addresses justice, authority flows and social goals is not suggested in these definitions. Indeed, a review of the literature and popular discourse on social innovation finds remarkably little mention of social justice.

Perhaps it is because there is an assumption that social innovation is a positive thing for society that justice is simply assumed to be part of the act. Yet, that is hardly the case in practice. While we may use terms like participatory, engagement, and co-creation in our discussion of social innovation, the manner in which society is part of the process and involved is not well-articulated or is described in vague terms such as “engage diversity”. What does that actually mean? And what does this mean for our connection to community?

The emotional connection

Part of the problem is that innovation gets defined in terms of the product produced and the methods of engagement used to produce that innovation. What doesn’t get discussed is the emotional connection to the innovation and the way that guides participation and engagement. That emotional connection is what sits at the seat of justice.

“Full membership in a community depends on certain feelings, and these feelings are easily starved. A community is a circle of respect, and respect is felt. When any of us don’t feel respected by the community, we withdraw”

Paul Woodruff’s book the Ajax Dilemma explores the matter of social justice and one can’t help but think of how we often neglect this important concept and the emotional way in which people connect to their community or are excluded from that community via social innovation.

Woodruff’s excellent book looks at the complex relationship between people, their community and the means that hold them together, which is justice. He maintains:

“The purpose of justice is to maintain the integrity of a community. It’s not merely what you decide that matters, but how you decide it, and how you communicate the decision”

For social innovation this means ensuring that our ideas are not only sound, but that we have generated them in a manner that promotes justice within the community and that we are clear in how we communicate the purpose and impact of our innovation to the world. This challenges the impression that good ideas are self-evident and that the ends justify the means even if they are well-intended and co-creative. This means that the innovation itself needs to fit and enhance the integrity of the community while simultaneously challenging it.

The communication imperative

The last part of Woodruff’s quote above is the piece that ties justice to making our innovations social. It’s not enough to engage others in our innovation efforts, its about communicating what we’re doing to those that are participating and those that are not at the same time. It means evaluating what we do and documenting what decisions we make along the way to ensure that we make our ideas and their implications transparent to others because, ultimately, an innovation that seeks to transform society is one that won’t always involve everyone, but it needs to consider them.

That consideration provides that emotional attachment between individuals and the ideas that we generate to serve the society in which those societies belong. In doing so we create these new ideas that preserve integrity while pushing the bounds of what communities are and the status quo that isn’t always serving the best interests of society. By communicating ourselves and our intentions and putting justice at the heart of what we do social innovators are more likely to do well and do good at the same time.

(For those interested in learning more about Paul Woodruff’s perspective the lecture below gives a sense of what justice means in general as he discusses what the Ajax dilemma really is).

The Power, Peril and Promise of Health Journalism

Online Prescription Concept

The Toronto Star, Canada’s most widely read newspaper known for its investigative reporting gifted anti-vaccination audiences armament by using poor science to point to a spurious connection between an HPV vaccine and illness. The issue points to journalism’s power to shape the discourse of health issues and it points to the power, promise and peril associated with good (and not so good) science reporting. 

With great power comes great responsibility – Uncle Ben, Spiderman

It started with a story

On Thursday February 5th, 2015 the Toronto Star, Canada’s most widely read newspaper that has a reputation for solid investigative journalism, published an story that connected the experience of young girls and negative health effects with the receiving the Gardasil HPV vaccine. The story was immediately and widely criticized by experienced science journalists and health professionals alike, who argued that it was based on terribly flawed science.

The Toronto Star’s reaction was to defend itself, arguing in many different fora that they indeed mentioned that there was little scientific evidence that supported the link between the vaccine and the negative health effects being discussed in the article. The problem is that these links are buried deep in the article and certainly are not its focus: the hypothesized harms are.

Two days later, the Star published a follow-up op-ed letter which was authored by two health professionals and co-signed / supported by dozens of Toronto’s leading physicians condemning the original article. However, by that time the damage is likely to have been done and one more bit contribution to the fictitious ‘evidence’ for vaccine harms had been added to the anti-vaccine movement’s war chest.

Perpetuating harm

This matter of poor reporting is not a trivial issue. The fraudulent science performed by Andrew Wakefield linking autism to vaccines helped spur an evidence-thin anti-vaccination movement. Today, we are seeing the resurgence of diseases once thought to be eliminated in North America (like measles) because so many people are not having their children vaccinated. Jenny McCarthy is among the celebrities who have taken up the cause of anti-vaccination and has written about and spoken at length about what she sees as the connection between autism and vaccines, using her son’s experience as an ‘example’. Oprah Winfrey, perhaps unwittingly, gave McCarthy a platform to speak about her beliefs on her show offering wider possible credibility to something that has been thoroughly discredited in the scientific literature (PDF).

For the Toronto Star, it was bad enough that the story was published — and is now online, likely for all time in various forms thanks to the Web — but what made it worse was that the Star was so vigorous in its defence of it, unwilling or unable to recognize their role in public health. Medical evidence champion, author, physician and columnist Ben Goldacre was among the many who counter-attacked, pointing to what he called The Star’s ‘smear campaign‘ against the story’s critics.

For an interesting discussion of the issue of just how the Star got it wrong, listen to Vox health reporter Julia Belluz, interviewed on the CBC’s radio show The Current. Belluz, a past MIT Knight Journalism Fellow, is one of a dwindling number of journalists who understand the practice of reporting, science, and medicine and wrote a stellar critique of the Toronto Star article, but as importantly makes the case for why there is a need for specialized, trained, supported journalists out there doing this kind of work.

…and health

I’ve argued in the past that journalism is very much a pillar of public health. When it fails, so does public health. Journalism is not and should not be an arm of public health for the very independence that good, professional journalism strives to maintain is a reason it’s often called the fourth estate, keeping governments and other forces in check to ensure they are not abusive. Yet, that distance is also what makes it a part of public health. Public health is better for journalism and journalism certainly can benefit from health stories as they continue to be popular and sought after by readers.

As a group, scientists and many clinicians are not great at communicating what they do, why their research is important to others outside their field, and what the implications of their findings are for the public and science as a whole. Some are, most are not. It’s for this reason that the entire sub-field of health sciences focused on knowledge translation, exchange and mobilization has emerged. Just as we value the ability of a graphic designer to make visuals come alive, so too have we learned to value those with the skills to communicate information well and that is what journalists are trained and paid to do. They are a big part of this process, or at least should be.

Healthy journalism, healthy science, healthy people

Science journalism is too important to be ignored. There is much skepticism of journalists by scientists and clinicians and indeed, as the Toronto Star shows, journalists sometimes get things wrong. But its one thing to get it wrong through errors of judgement or interpretation it’s quite another to get things wrong by design. The Toronto Star has some good health reporters, but they weren’t the ones on this story. Nor did they bring in the health reporters to consult on this or other health professionals prior to publication– at least as far as one can tell.

The importance to the public’s health of good reporting requires that health and science journalists have more than a rudimentary knowledge of the topics they are covering. What’s strange is how we understand this with our sports reporting, weather forecasts and foreign correspondents. You wouldn’t watch someone who has little understanding of a sport covering it in depth, would you? It’s one thing to read scores, it’s another to provide investigative and deep coverage of a game if you don’t know the players, the rules, the criteria for quality and success and so forth.

Why do we do this with health journalism and science?

Yet, journalism is under pressure and no doubt the Toronto Star, for whatever genuine contrition they experience from what happened, have to like that they are being talked about. The reason is that journalism is under threat for market reasons, the Internet and the changing ways we get our news. It is, as Jürgen Krönig wrote way back in 2004, “A crisis of the Fourth Estate”. That crisis is only getting worse.

As anyone interested in public health, we need to take actions to ensure that the fourth estate is protected, supported and not ignored. Our health might just depend on it.

Image: iStockphoto, used under licence.

The Ecology of Innovation: Part 1 – Ideas

Innovation Ecology

Innovation Ecology

There is a tendency when looking at innovation to focus on the end result of a process of creation rather than as one node in a larger body of activity, yet expanding our frame of reference to see these connections innovation starts to look look much more like an ecosystem than a simple outcome. This first in a series examines innovation ecology from the place of ideas.

Ideas are the kindling that fuels innovation. Without good ideas, bold ideas, and ideas that have the potential to move our thinking, actions and products further we are left with the status quo: what is, rather than what might be.

What is often missed in the discussion of ideas is the backstory and connections between thoughts that lead to the ideas that may eventually lead to something that becomes an innovation*. This inattention to (or unawareness of) this back story might contribute to reasons why many think they are uncreative or believe they have low innovation potential. Drawing greater attention to these connections and framing that as part of an ecosystem has the potential to not only free people from the tyrrany of having to create the best ideas, but also exposes the wealth of knowledge generated in pursuit of those ideas.

Drawing Connections

Connections  is the title of a book by science historian James Burke that draws on his successful British science documentary series that first aired in the 1970’s and was later recreated in the mid 1990’s. The premise of the book and series is to show how ideas link to one another and build on one another to yield the scientific insights that we see. By viewing ideas in a collective realm, we see how they can and do connect, weaving together a tapestry of knowledge that is far more than the sum of the parts within it.

Too often we see the celebration of innovation as focused on the parts – the products. This is the iPhone, the One World Futbol, the waterless toilet, the intermittent windshield wiper or a process like the Lean system for quality improvement or the use of checklists in medical care. These are the ideas that survive.

The challenge with this perspective on ideas is that it appears to be all-or-nothing: either the idea is good and works or it is not and doesn’t work.

This latter means of thinking imposes judgement on the end result, yet is strangely at odds with innovation itself. It is akin to judging flour, salt, sugar or butter to be bad because a baker’s cake didn’t turn out. Ideas are the building blocks – the DNA if you will — of innovations. But, like DNA (and RNA), it is only in their ability to connect, form and multiply that we really see innovation yield true benefit at a system level. Just like the bakers’ ingredient list, ideas can serve different purposes to different effects in different contexts and the key is knowing (or uncovering) what that looks like and learning what effect it has.

From ideas to ecologies

An alternative to the idea-as-product perspective is to view it as part of a wider system. This takes James Burke’s connections to a new level and actually views ideas as part of a symbiont, interactive, dynamic set of relations. Just like the above example of DNA, there is a lot of perceived ‘junk’ in the collection that may have no obvious benefit, yet by its existence enables the non-junk to reveal and produce its value.

This biological analogy can extend further to the realm of systems. The term ecosystem embodies this thinking:

ecosystem |ˈekōˌsistəm, ˈēkō-| {noun}

Ecology

a biological community of interacting organisms and their physical environment.

• (in general use) a complex network or interconnected system: Silicon Valley’s entrepreneurial ecosystem | the entire ecosystem of movie and video production will eventually go digital.

Within this perspective on biological systems, is the concept of ecology:

ecology |iˈkäləjē| {noun}

1 the branch of biology that deals with the relations of organisms to one another and to their physical surroundings.

2 (also Ecology) the political movement that seeks to protect the environment, especially from pollution.

What is interesting about the definitions above, drawn from the Oxford English Dictionary, is that they focus on biology, the discipline where it first was explored and studied. The definition of biology used in the Wikipedia entry on the topic states:

Biology is a natural science concerned with the study of life and living organisms, including their structure, function, growth, evolution, distribution, and taxonomy.[1]

Biologists do not look at ecosystems and decide which animals, plants, environments are good or bad and proceed to discount them, rather they look at what each brings to the whole, their role and their relationships. Biology is not without evaluative elements as judgement is still applied to these ‘parts’ of the system as there are certain species, environments and contexts that are more or less beneficial for certain goals or actors/agents in the system than others, but judgement is always contextualized.

Designing for better idea ecologies

Contextual learning is part of sustainable innovation. Unlike natural systems, which function according to hidden rules (“the laws of nature”) that govern ecosystems, human systems are created and intentional; designed. Many of these systems are designed poorly or with little thought to their implications, but because they are designed we can re-design them. Our political systems, social systems, living environments and workplaces are all examples of human systems. Even families are designed systems given the social roles, hierarchies, expectations and membership ‘rules’ that they each follow.

If humans create designed systems we can do the same for the innovation systems we form. By viewing ideas within an ecosystem as part of an innovation ecosystem we offer an opportunity to do more with what we create. Rather than lead a social Darwinian push towards the ‘best’ ideas, an idea ecosystem creates the space for ideas to be repurposed, built upon and revised over time. Thus, our brainstorming doesn’t have to end with whatever we come up with at the end (and may hate anyway), rather it is ongoing.

This commitment to ongoing ideation, sensemaking and innovation (and the knowledge translation, exchange and integration) is what distinguishes a true innovation ecosystem from a good idea done well. In future posts, we’ll look at this concept of the ecosystem in more detail.

Brainstorming Folly

Brainstorming Folly

Tips and Tricks:

Consider recording your ideas and revisiting them over time. Scheduling a brief moment to revisit your notebooks and content periodically and regularly keeps ideas alive. Consider the effort in brainstorming and bringing people together as investments that can yield returns over time, not just a single moment. Shared Evernote notebooks, Google Docs, building (searchable) libraries of artifacts or regular revisiting of project memos can be a simple, low-cost and high-yield way to draw on your collective intellectual investment over time.

* An innovation for this purpose is a new idea realized for benefit.

Image Credits: Top: Evolving ecology of the book (mindmap) by Artefatica used under Creative Commons License from Flickr.

Bottom: Brainstorming Ideas from Tom Fishburne (Marketoonist) used under commercial license.

Of tails, dogs and the wagging of both

Who's wagging whom?

Who’s wagging whom?

Evaluation is supposed to be driven by a program’s needs and activities, but that isn’t always the case. What happens when the need for numbers, metrics, ‘outcomes’ and data shape the very activities programs do and how that changes everything is something that is worth paying some attention to. 

Since the Second World War we’ve seen a gradual shift towards what has been called presence of neo-liberal values across social institutions, companies, government and society. This approach to the world is characterized, among other things, by its focus on personal and economic efficiency, freedom, and policies that support actions that encourage both. At certain levels of analysis, these policies have rather obvious benefits.

Who wouldn’t like to have more choice, more freedom, more perceived control and derive more value from their products, services and outputs? Not many I suspect. Certainly not me.

Yet, when these practices move to different levels and systems they start to produce enormous complications that are at odds with — and produce distortions of — the very values that they espouse. We’ve seen the same happen with other value systems that have produced social situations that are highly beneficial in some contexts and oppressive and toxic in others – capitalism and socialism both fit this bill.

Invisible tails and wags

What makes ‘isms’ so powerful is that they can become so prevalent that their purpose, value and opportunity stop being questioned at all. It is here that the tail starts to wag the dog.

Take our economy (or THE economy as it is somewhat referred to). An economy is intended to be a facilitator and product of activities used to create certain types of value in a society. We work and produce goods (or ideas), exchange and trade them for different things, and these allow us to fulfill certain human goals. It can take various shapes, be regulated more or less, and can operate at multiple scales, but it is a human construction — we invented it. Sometimes this gets forgotten and in times when we use the economy to justify behaviour we forget that it is our behaviour that is the economy.

We see over and again with neoliberalism (which is among the most dominant societal ‘ism’ of the past 50 years in the West and more reflected globally all the time) taken at the broadest level, the economy becomes the central feature of our social systems rather than a byproduct of what we do as social beings. Thus, things like goods, experiences, relations and so on we used to consider as having some type of inherent value suddenly become transformed into objects that judgements can be made.

The role of systems

This can make sense where there are purpose-driven reasons to assign particular value scores to something, but the nature of value is tied to the systems that surround what is valued. If we are dealing with simple systems, those where there are clear cause-and-effect connections between the product or service under scrutiny and its ability to achieve its purpose, then valuation measurement makes sense. We can assert that X brand of laundry detergent is better than Y on the basis of Z. We can conduct experiments, trials and repeated measures that can compare across conditions.

It is also safe to make an assumption of value based on the product’s purpose that can be generalized. In other words, our reason for using the product is clear and relatively unambiguous (e.g., to clean clothes using the above example). There may be additional reasons for choosing X brand over Y, but most of those reasons can be also controlled for and understood discretely (e.g., scent, price, size, bottle shape etc..).

This kind of thinking breaks down in complex systems. And to make it even more complex, it breaks down imperfectly so we have simple systems interwoven within complex ones. We have humans using simple products and services that operate in new, innovative and complex conditions. Unfortunately, what comes with simple systems is simple thinking. Because they are — by their nature — simple, these system dynamics are easy to understand. Returning to our example of the economy, classical micro-economic models of supply and demand as illustrated below.

Relationships and the systems that surround them

supply_and_demand

Using this model, we can do a reasonable job of predicting influence, ascertaining value and hypothesizing relationships between both.

In complex systems, the value links are often in flux, dynamic, and relative requiring a form of adaptive evaluation like developmental evaluation. But that doesn’t happen as much as it should, mostly because of a failure to question the systems and their influence. Without questioning the values and value that systems create — the isms that were mentioned earlier — and their supposed connection to outcomes, we risk measuring things that have no clear connection to value and worse, we create systems that get designed around these ineffective measures.

What this manifests itself in is mindless bureaucracy, useless meetings, pompous and intelligible titles, and innovation-squashing regulations that get divorced from the purpose that they are meant to solve. And in doing so, this undermines the potential benefit that the original purpose of a bureaucracy (to document and create an organizational memory to guide decisions), meetings (to discuss and share ideas and solve problems), titles (to denote role and responsibility — although these aren’t nearly as useful as people think in the modern organization), and regulations (to provide a systems lens to constrain uncoordinated individual actions from creating systems problems like the Tragedy of the Commons).

More importantly, this line of thinking also focuses us on measuring the things that don’t count. And as often quoted and misquoted, the phrase that is apt is:

Not everything that counts can be counted, and not everything that can be counted counts.

Counting what counts

It is critical to be mindful of the purpose — or to reconnect, rediscover, reinvent and reflect upon the purposes we create lest we allow our work to be driven by isms. Evaluators and their program clients and partners need to stand back and ask themselves: What is the purpose of this system I am dealing with?

What do we measure and is that important enough to matter? 

Perhaps the most useful way of thinking about this is to ask yourself: what is this system being hired to do? 

Regular mindful check-ins as part of reflective practice at the individual, organizational and, where possible, systems level are a way to remind ourselves to check our values and practices and align and realign them with our goals. Just as a car’s wheels go out of alignment every so often and need re-balancing, so too do our systems.

In engaging in reflective practice and contemplating what we measure and what we mean by it we can better determine what part of what we do is the dog, what is the tail and what is being wagged and by whom.

Photo credit: Wagging tail by Quinn Dombrowski used under Creative Commons License via Flickr. Thanks Quinn for making your great work available to the world.

Economic model image credit from Resources for Teachers used under Creative Commons License. Check out their great stuff for helping teachers teach better.

Bullying, the market for education and the damaged quest for learning

Dark classroom, light minds

Dark classroom, light minds

A recent study found looked into the experience of cyberbullying by university professors at the hands of their students. This disturbing phenomenon points to much larger issues beyond mental health promotion and calls into question many of the assumptions we have about the systems we’ve designed to foster education and what it means to be a learner at university. 

The university is one of our oldest cultural institutions and its instructors are considered to have among societies most respected jobs, even if not always well compensated. In the past, students often approached their professors with a mixed sense of wonder, respect, curiosity and fear and that, in healthy situations, was reciprocated by faculty to create a space where people could explore ideas, learn, and challenge themselves and others to grow. That relationship has started to change as evidenced by the rise of cyberbullying in the classroom.

A recent article in Macleans Magazine looked at the changing state of the post-secondary classroom and the role of cyberbullying. Only this was not about student victims, but students as the perpetrators against their professors. The effects of cyberbullying are crippling and professors are bearing the burden of having hundreds of eyes watching them, writing about them and writing ‘consumer reviews’ about them in anonymous and sometimes unflattering, inflammatory and questionable terms on sites like RateMyProfessor.com .

Researchers at the University of California, Riverside found that as students age the incidence of face-to-face bullying decreases and cyberbullying increases, which might partly explain why we’re seeing this in university settings when face-to-face bullying goes subterranean. Yet, the notion that professors that are getting bullied by their students belies some other issues that require further investigation, namely those related to the nature of education and the role of students-as-consumers.

Consuming knowledge, producing expectations

If you pay for something, should you not expected to get something rather specific for that experience or product? Aside from some rare experiences of profane/profound personal challenge/punishment like Tough Mudder and its peers or dental work, there are few things we willingly pay for that we don’t derive pleasure from or achieve a very specific (anticipated) outcome.

Education is problematic because we might not know what we’ll get from it going in, what kind of experiences or ideas will emerge, and how our relationship to those experiences will change us. That is its great gift.

Many of us have had profound life changes because of something we experienced through our education and writing as one who has completed four different degree programs and a post-doc I can confidently say that I didn’t receive a lot of what I expected in any of those programs and I am a better person for it. Indeed, if I go to a specific learning event (aside from those focused on a specific technique or technology) I am disappointed if I actually come away with exactly what I expected.

That is part of the point. We don’t know what we don’t know.

But when you start viewing education as a thing that resembles any other market-driven product or services, you begin to focus on learning as a consumable good and your students as customers. In following this line of thought, it makes some sense to focus the delivery of this product on the desires of the consumer.

Increasingly, teachers (of various stripes) are being asked to consider a range of student-related variables in their education. Things like learning styles and preferences are now being woven into classroom instruction and students have come to learn to expect and are increasingly demanding to be taught in ways that match their unique learning preferences and styles. While there is reason to imagine that this approach is useful in stimulating engagement of students in the lessons, there is increasing evidence much of it does little to enhance actual learning. Many of the life lessons we’ve gained that shape what we do and who we are were not delivered in the manner of our choosing, conformed with our preferences and were not desired, expected or enjoyed in the moment. We risk confusing enjoyment with learning; they can be aligned but one isn’t necessary for the other to take place.

However, when we are viewing education from a consumer model, the specific outcomes become part of the contract. If I come to get a degree in X because I believe that the job market demands the skills and knowledge that X brings and I am paying tens of thousands of dollars and spending four or more years acquiring X then I feel entitled to expect all the benefits that X brings. Further, I expect that my journey to acquiring X will be enjoyable, because why would I spend more money than I’ve ever seen on something I don’t enjoy.

Particularly when that is money I don’t have.

A debt to pay

In Canada and the United States, student debt rates have dramatically increased. The Canadian Federation of Students note that Canadian’s attending post-secondary education now owe more than $15B to the Canadian federal government (PDF) as part of their student loan program, a number that doesn’t include debt accumulated from borrowing from banks, family, credit cards and other means. In Canada’s largest province, Ontario, the rate of graduate employment has decreased since 2001 and the overall youth unemployment rate continues to be the highest, despite the province having one of the most educated youth population in the country (and arguably, the world). And while Ontario universities continue to promote the fact that education is a better pathway to success, it is a hard pill for many students to swallow when many can’t apply what they trained for and paid for after they graduate.

Satirist John Oliver has an informative, humorous and distressing take on student debt and the state of consumer-oriented education for those who want to learn more.

None of these reasons are excuses for cyberbullying, but it does give a more complicated picture of those that might feel they are entitled to bully others and their reasoning behind it.

What we are seeing is a systems change in the way education is being produced, consumed and experienced. Even the mere fact that we can now reasonably use the language of consumerism to speak to something like education should give us pause and concern. I’ve been involved in post-secondary education for nearly 20 years and there has always been students who simply wanted the ‘piece of paper’ (degree) as a stepping stone to a job and little more than that from their time at school. They were willing to do the work — often the minimum possible — to graduate, but they knew they had to put the effort in to be successful. There was never an expectation that one was entitled to anything from going to school, although that might be changing.

Market identities and education systems

Belgian psychotherapist Paul Verhaeghe has explored the role of identity in market-based economies in his new book What About Me? In the book, Verhaeghe illustrates how we construct our identities as people drawing on the research that reflects (and often contradicts or obscures) the two major perspectives on personality and identity: the person-as-blank-slate and the person as a reflection of the environment. The former perspective assumes we come into the world as we are while the latter assumes the world makes us who we are and both have enormous amount of moral, cultural and evidentiary baggage attached to them.

What Verhaeghe does is point to the ways in which both have elements of truth to them, but that they are mediated by the manner in which we construct the very questions about who we are and what our purpose is. These questions are (for many cultural, historical, economic and political reasons that he elaborates on) frequently market-based. Thus, who we are is defined by what we do, what we own, what we produce, and how we use such things once out into the world and that the value that come with such ways of defining ourselves is considered self-evident. He makes a disturbing and convincing case when one stops to reflect on the way we think about how we think (metacognition + mindfulness) .

When viewed from the perspective of a market, knowledge and its products soon become the goal and not the journey. Indeed, I’ve even written about this in support of an argument for better research-to-action and knowledge translation. Much of the knowledge-to-action discourse is about viewing knowledge as a product even if the more progressive models also view this as part of a process and even more as part of a system. But it is the last part — the system — that we often give the shortest shrift to in our discussions. What Verhaeghe and others are doing is encouraging us to spend more time thinking about this and the potential outcomes that emerge from this line of thinking.

Unless we are willing to talk more about the systems we create to learn, explore and relate we will continue to support Verhaeghe’s thesis and uphold the conditions for the kind of education-as-a-product thinking that I suspect is contributing to students’ changing behaviour with their professors and creating a climate at universities that is toxic instead of inspiring.

Photo credit: Classroom by Esparta Palma used under Creative Commons License via Flickr. Check out Esparta’s remarkable work here.

Thinking systemically about systems thinking

Carnaby Street

The Whole and the Parts

Systems thinking is a class of theories, models and methods for understanding human and non-human interactions as seen as wholes instead of parts. This focus on interconnections and relationships is precisely what makes it challenging for many when it comes to systemically considering what systems thinking is all about and the implications of this are many. This post provides an introduction to certain ideas in systems thinking and points to what makes it different than other non-systems thinking approaches to understanding something. 

Perhaps the most popular aphorism about systems thinking is the statement that the whole is greater than the sum of its parts, something borrowed from Gestalt Psychology. That statement is intended to reflect system thinking’s principal focus on the system itself rather than on the actors and actions within it.

It’s a subtle difference, but a meaningful one. For example, psychology might look at why individuals make choices and act and what implications come from those actions. Systems thinking seeks to look at the combined interaction of these interactions as a unified whole.

Boundaries

Fundamental to this way of seeing things is the concept of boundaries. Boundaries are essentially where the differences that make a difference lie. In a closed system, everything that makes a difference is clearly contained and observed within a relatively solid set of boundary conditions. Mechanical systems often function this way, making them simple or complicated in that they have the potential to be understood clearly in terms of causal connections and relations. These systems are more amenable to things like “best practices” where we can reasonably expect similar outcomes from consistent actions.

This kind of systems thinking is not as useful when applied to human systems, because they are mostly characterized as open systems. Open systems are those where the boundaries require some form of negotiation and may actually be in flux.

A general shorthand rule for setting boundaries in this kind of environment is this:

If you find yourself lost over and again in trying to understand where the influences and relationships within the system are, then you’ve probably bound your system too loosely. If you are finding too many influences laying outside of your boundaries, you’ve probably bound it too tightly.

Perspective: Where you sit

Systems are all about where you sit in relation to them. For instance, let’s take the example of family and some of the boundary questions one might ask in understanding this social entity as a system.

  • Firstly, who is family? You could define family as blood relationships. But is that immediate blood relations? For example, If parents and children count, then how do we consider grandparents who are the parents of the parents? Do they count as family when you bound the system? Do great grandparents? Should we use genes and, if so, what level of genetic similarity do we share? Are we all family?
  • Can family be defined socially? For example, if people become family by marriage and that marriage breaks down, does it influence the family system as you define it? What if that marriage ends via someone passing away? What if they are not married at all, but common law?
  • What about the roles that people play? Does an “Uncle” or “Aunt” who are close, intimate friends of the family, but not of blood ties still get included in the family? How about a trusted lifelong neighbour who has been a part of someone’s life the entire time, but was never genealogically connected to anyone?
  • Can our neighbourhood be part of the family?

One can make a case for any of these conditions. In defining a system there is no ‘good’ or ‘bad’ way to do it, just perspectives that are more or less useful and more or less attentive to specific details.

Purposeful systems

The answers to the questions about boundaries also depend on what the purpose of the system is in the first place. Purpose is the means by which we determine the differences and how they make a difference. You can imagine that one could potentially answer “yes” to almost every one of the questions asked above depending on where someone sits in the system and what kind of purpose they see in that system.

Part of thinking systemically about systems is defining the purpose of the system and ascertaining a perspective. That means being strategic about what you wish your systems thinking to support. It is here that much of the use of systems thinking I’ve witnessed breaks down. Organizations seeking to employ systems thinking often jump in without doing the pre-work needed to ground their perspective into some sense of purpose and perspective. This requires a mindful, honest accounting of the perspectives being brought into the discussion and connecting those to the strategic intent of your enterprise.

Being mindful of what one values, what one seeks to accomplish, and what kind of activities your organization engages in (or wants to engage in), and where the reach of your organization extends is a key starting position to thinking more systemically about systems.

The Myth of Fast-tracking Learning

The time keeper / El guardián del tiempo

The time keeper

In a quest for getting more, faster we pursue strategies that aim to compress and challenge the physics of time. Education is one of these areas where the quest to learn more, faster and ‘better’ may actually be taking us away from knowledge and speeding us to folly. 

What would you say or do if your physician or attending nurse in the hospital told you that they attended a medical school that distilled all the key sources of knowledge into packages that allowed them to complete their training in half the time?

Would you be comfortable being treated by them?

What if you were seated on your next flight  and learned that the pilot of your aircraft was taught by a flight school that claimed it could train pilots without the thousands of flight hours by focusing on the essence of what it meant to fly and do that really well in a short period of time?

Would you still want to fly with them?

What if someone said that they had a formula for taking Ericcson’s near mythical 10,000 hour rule* on building expertise and could halve it to produce the exact same results?

Would you believe them? And would you follow them?

Packaged learning and the myths of efficiency

While we might say no to these, we say yes to a lot of other things that are perhaps just as hard to believe. One of these is the myth of online education. Major online learning platforms (MOOC’s) like EdX, Coursera, and Udacity along with global education pioneers Khan Academy are delivering educational content to millions along with universities and thousands of smaller or independent education providers with the promise of offering distance education, some with degrees attached to them.

There is a place for this type of learning, but as often happens, the enthusiasm for speed, efficiency and profit blind and blur. Correspondence classes and the earliest online or distance learning programs were designed to meet the educational needs of those who were geographically isolated from others where face-to-face learning was impractical. What had a practical idea to solving a specific set of problem existing in a particular set of constraint conditions it is suddenly morphing into a standard for everyone and that isn’t a good idea.

Look around and you will see more ‘packaging’ educational experiences so that they can be scaled and delivered efficiently to different audiences. This might be fine if the content is simple and can be matched with the educator, the learning space (physical or online), and the cognitive and emotional demands placed on the learner in the process of learning the material. Yet, frequently this isn’t the case. Now, we see efforts to create programs to teach complex, important topics in a weekend, a week or a short retreat with the idea that we can just get to the essence of what’s needed and the rest will take care of itself.

Doing the work, putting in the time

No better example of this is hyper-learning myth is found that with Timothy Ferris, author of the 4-hour workweek and other rapid-fire learning books. Ferris takes his readers through his journeys to be hyper-efficient and learn things in a compressed time along the way.

One example is how he became a champion in a martial arts tournament in a sport he knew nothing about before engaging in mere weeks of training before the event. This achievement was done through some clever exploitation of tournament rules and engaging in a near obscene dehydration plan that enabled him to lose weight prior to weigh ins to allow him to fight below his normally expected weight class. This doesn’t change the outcome, but it adds a very big asterisk to its notation in the record books. Ferris’ work is filled with these sleight of hand kind of efficiencies that might work for a one-off, who’s longer term is questionable**.

Ferris has made a career out of intense, hyper-condensed learning and, even if he does what he claims, his approach to learning is his job and life. For most people, learning is one of a great many things they have on the go. Further, the problems they are trying to solve might not be ones that have a clear answer or a way to circumvent using a close read of the rules, rather they may be the kind of protracted, complex, thorny and wicked problems that we see in healthcare, social policy, environmental action, and organizational development. These are spaces where sleights of hand aren’t well received.

Other sleights of hand

In professional circles it is the longer-term that matters. System change, social innovation, healthcare transformation and community or organizational development are all areas where learning needs to start and continue throughout a long process. It often involves consideration of complex scenarios, an understanding of theory, reflective practice and experimentation that simply take time to not only engage with, but to contemplate.

It is like the parable about the farmer who wakes up one morning to find all of his crops dead because his unknowing son spent the night pulling up every stalk of grain with the belief that he could make them grow faster.

We have not been able to circumvent time, no matter what we wish.

The sleight of hand is in making busywork and information disguised as active learning and knowledge. There are certainly ways we can improve teaching, learning, knowledge translation and exchange and knowledge integration in its effectiveness, reach and impact, but we won’t be finding the ‘killer app’ that gives us the ability to download knowledge to our heads like the Matrix. These are developmental problems and thus ought to be treated using developmental thinking.

But we still try. Apps are being developed that allow us to learn anything, anywhere, in real time, from our phone or change our behaviour with a couple simple clicks, except there is virtually no evidence that we actually learn, actually change or do anything other than buy more and worry more.

True learning innovation will come from being wide-eyed about what we mean by learning, what we seek to achieve through it and creating the developmental thinking around what it means to bring them together rather than subscribing to legends or quick-fixes that simply don’t work.

 

* Anders Ericsson’s research on deliberative practice (PDF), which shows that attentive, intentional learning over time is a key determinant in high performing individuals. Malcolm Gladwell’s book Outliers highlights this work in detail and has led to the popularization of what has been colloquially referred to as ‘The 10,000 hour rule’, which reflects the approximate number of hours of deliberative practice required to gain expert-level skill and knowledge in a field.

**Many of Ferris’ claims from learning languages in a few weeks to mastering other subjects are unverified.

Image Credit: The Time Keeper /El guardián del tiempo by Jesus Solano via Flickr used under Creative Commons License. Thanks Jesus for sharing your wonderful art with the world through Creative Commons.

 

 

 

The urban legends of learning (and other inconvenient truths)

Learning simulacrum, simulation or something else?

Learning simulacrum, simulation or something else?

Learning styles, technology-driven teaching, and self-direction are all concepts that anyone interested in education should be familiar with, yet the foundations for their adoption into the classroom, lab or boardroom are more suspect than you might think. Today we look at the three urban legends of learning and what that might mean for education, innovation and beyond. 

What kind of learner are you? Are you a visual learner perhaps, where you need information presented in a particular visual style to make sense of it? Maybe you need to problem-solve to learn because that’s the way you’ve been told is best for your education.

Perhaps you are a self-directed learner who is one that, when given the right encouragement and tools, will find your way through the muck to the answers and that others just need to get out of the way. With tools like the web and social media, you have the world’s knowledge at your disposal and have little need to be ‘taught’ that stuff, because its online.

And if you’re a digital native (PDF), this is all second nature to you because you’re able to use multiple technologies simultaneously to solve multiple problems together with ease if given the ability to do so. After all, you’ve had these tools your entire life.

A recent article by Paul Kirschner and Jeroen van Merriënboer published in the peer-reviewed journal Educational Psychologist challenges these ‘truths’ and many more, calling them urban legends:

An urban legend, urban myth, urban tale, or contemporary legend, is a form of modern folklore consisting of stories that may or may not have been believed by their tellers to be true.

The authors are quick to point out that there are differences in the way people approach material and prefer to learn, but they also illustrate that there is relatively little evidence to support much of the thinking that surrounds these practices, confusing learning preferences for learning outcomes. I’ve commented on this before, noting that too often learning is conflated with interest and enjoyment when they are different things and if we were really serious about it we might change the way we do a great deal many things in life.

In the paper, the authors debunk — or at least question — the evidence that supports the ‘legends’ of digital natives as a type of learner, the presence of specific learning styles and the need to customize learning to suit such styles of learning, and that of the lone self-educator. In each case, the authors present much evidence to challenge these ideas so as not to take them as truths, but hypotheses that have little support for them in practice.

Science and its inconvenient truths about learning

Science has a funny way of revealing truths that we may find uncomfortable or at least challenge our current orthodoxy.

This reminds me of a terrific quote from the movie Men in Black that illustrates the fragility of ideas in the presence and absence of evidence after one of the characters (played by Will Smith) uncovers that aliens were living on earth (in the film) and is consoled by his partner (played by Tommy Lee Jones) about what is known and unknown in the world:

Fifteen hundred years ago everybody knew the Earth was the center of the universe. Five hundred years ago, everybody knew the Earth was flat, and fifteen minutes ago, you knew that humans were alone on this planet. Imagine what you’ll know tomorrow.

One of the problems with learning is that there is a lot to learn and not all of it is the same in content, format and situational utility. Knowledge is not a ‘thing’ in the way that potatoes, shoes, patio furniture, orange juice, and pencils are things where you can have more or less of it and measure the increase, decrease and change in it over time. But we often treat it that way. Further, knowledge is also highly contextualized and combines elements that are stable, emergent, and transformative in new, complex arrangements simultaneously over time. It is a complex adaptive system.

Learning (in practice) resists simple truths.

It’s why we can be taught something over and again and not get it, while other things get picked up quickly within the same person even if the two ‘things’ seem alike. The conditions in which a person might learn are cultural (e.g., exposure to teaching styles at school, classroom designs, educational systems, availability and exposure to technology, life experiences, emphasis on reflective living/practice within society, time to reflect etc..) and psycho-social/biological (e.g., attention, intelligence, social proximity, literacy, cognitive capacity for information processing, ability to engage with others) so to reduce this complex phenomena to a series of statements about technology, preference and perception is highly problematic.

Science doesn’t have all the answers — far from it — but at least it can test out what is consistent and observable over time and build on that. In doing so, it exposes the responsibility we have as educators and learners.

With great power comes great responsibility…?

Underpinning the urban legends discussed by Kirschner and van Merriënboer and not discussed is the tendency for these legends to create a hands-off learning systems where workplaces, schools, and social systems are freed from the responsibility of shaping learning experiences and opportunities. It effectively reduces institutional knowledge, wisdom and experience to mere variables in a panoply of info-bites treated as all the same.

It also assumes that design doesn’t matter, which undermines the ability to create spaces and places that optimize learning options for people from diverse circumstances.

This mindset frees organizations from having to give time to learning, provide direction (i.e., do their own homework and set the conditions for effective learning and knowledge integration at the outset). It also frees us up from having to choose, to commit to certain ideas and theories, which means some form of discernment, priority setting, and strategy. That requires work up front and leadership and hard, critical, and time-consuming conversations about what is important, what we value in our work, and what we want to see.

When we assume everyone will just find their way we abdicate that responsibility.

Divesting resources and increasing distraction

In my home country of Canada, governments have been doing this with social investment for years where the federal government divests interest to the provinces who divest it to cities and towns who divest it to the public (and private) sector, which means our taxes never go up even if the demands on services do and we find that individual citizens are responsible for more of the process of generating collective benefit without the advantage of any scaled system to support resource allocation and deployment throughout society (which is why we have governments in the first place). It also means our services and supports — mostly — get smaller, lesser in quality, more spread thinly, and lose their impact because there isn’t the scaled allocation of resources to support them.

Learning is the same way. We divest our interests in it and before you know it, we learn less and do less with it because we haven’t the cultural capital, traditions or infrastructure to handle it. Universities turn campus life to an online experience. Secondary schools stop or reduce teaching physical education that involves actual physical activity.  Scholarly research is reduced to a Google search. Books are given up as learning vehicles because they take too long to read. It goes on.

It’s not that there are no advantages to some of these ideas in some bites, but that we are transforming the entire enterprise with next to no sense of the systems they are operating in, the mission they are to accomplish, a theory of change that is backed up by evidence, or the will to generate the evidence needed to advise and the resources to engage in the sensemaking needed to evaluate that evidence.

Science, systems and learning

It is time to start some serious conversations about systems, science and learning. It would help if we started getting serious about what we mean when we speak of learning, what theories we use to underpin that language and what evidence we have (or need) to understand what those theories mean in practice and for policy. This starts by asking better questions — and lots of them — about learning and its role in our lives and work.

Design thinking and systems thinking are two thinking tools that can help us find and frame these issues. Mindfulness and its ethics associated with non-judgement, open-mindedness, compassion and curiosity are also key tools. The less we judge, the more open we are to asking good questions about what we are seeing that can lead us to getting better answers rather than getting trapped by urban legends.

Doing this within a systems thinking frame also allows us to see how what we learn and where and how we learn is interconnected to better spot areas of leverage and problems in our assumptions.

This might allow us to make many of our urban legends obsolete instead of allowing them to grow like the alligators that live in the sewers of New York City. 

 

 

Developmental Evaluation: Questions and Qualities

Same thing, different colour or different thing?

Same thing, different colour or different thing?

Developmental evaluation, a form of real-time evaluation focused on innovation and complexity, is gaining interest and attention with funders, program developers, and social innovators. Yet, it’s popularity is revealing fundamental misunderstandings and misuse of the term that, if left unquestioned, may threaten the advancement of this important approach as a tool to support innovation and resilience. 

If you are operating in the social service, health promotion or innovation space it is quite possible that you’ve been hearing about developmental evaluation, an emerging approach to evaluation that is suited for programs operating in highly complex, dynamic conditions.

Developmental evaluation (DE) is an exciting advancement in evaluative and program design thinking because it links those two activities together and creates an ongoing conversation about innovation in real time to facilitate strategic learning about what programs do and how they can evolve wisely. Because it is rooted in both traditional program evaluation theory and methods as well as complexity science it takes a realist approach to evaluation making it fit with the thorny, complex, real-world situations that many programs find themselves inhabiting.

I ought to be excited at seeing DE brought up so often, yet I am often not. Why?

Building a better brand for developmental evaluation?

Alas, with rare exception, when I hear someone speak about the developmental evaluation they are involved in I fail to hear any of the indicator terms one would expect from such an evaluation. These include terms like:

  • Program adaptation
  • Complexity concepts like emergence, attractors, self-organization, boundaries,
  • Strategic learning
  • Surprise!
  • Co-development and design
  • Dialogue
  • System dynamics
  • Flexibility

DE is following the well-worn path laid by terms like systems thinking, which is getting less useful every day as it starts being referred as any mode of thought that focuses on the bigger context of a program (the system (?) — whatever that is, it’s never elaborated on) even if there is no structure, discipline, method or focus to that thinking that one would expect from true systems thinking. In other words, its thinking about a system without the effort of real systems thinking. Still, people see themselves as systems thinkers as a result.

I hear the term DE being used more frequently in this cavalier manner that I suspect reflects aspiration rather than reality.

This aspiration is likely about wanting to be seen (by themselves and others) as innovative, as adaptive, and participative and as being a true learning organization. DE has the potential to support all of this, but to accomplish these things requires an enormous amount of commitment. It is not for the faint of heart, the rigid and inflexible, the traditionalists, or those who have little tolerance for risk.

Doing DE requires that you set up a system for collecting, sharing, sensemaking, and designing-with data. It means being willing to — and competent enough to know how to — adapt your evaluation design and your programs themselves in measured, appropriate ways.

DE is about discipline, not precision. Too often, I see quests to get a beautiful, elegant design to fit the ‘social messes‘ that represent the programs under evaluation only to do what Russell Ackoff calls “the wrong things, righter” because they apply a standard, rigid method to a slippery, complex problem.

Maybe we need to build a better brand for DE.

Much ado about something

Why does this fuss about the way people use the term DE matter? Is this not some academic rant based on a sense of ‘preciousness’ of a term? Who cares what we call it?

This matters because the programs that use and can benefit from DE matter. If its just gathering some loose data, slapping it together and saying its an evaluation and knowing that nothing will ever be done with it, then maybe its OK (actually, that’s not OK either — but let’s pretend here for the sake of the point). When real program decisions are made, jobs are kept or lost, communities are strengthened or weakened, and the energy and creative talents of those involved is put to the test because of evaluation and its products, the details matter a great deal.

If DE promises a means to critically, mindfully and thoroughly support learning and innovation than it needs to keep that promise. But that promise can only be kept if what we call DE is not something else.

That ‘something else’ is often a form of utilization-focused evaluation, or maybe participatory evaluation or it might simply be a traditional evaluation model dressed up with words like ‘complexity’ and ‘innovation’ that have no real meaning. (When was the last time you heard someone openly question what someone meant by those terms?)

We take such terms as given and for granted and make enormous assumptions about what they mean that are not always supported). There is nothing wrong with any of these methods if they are appropriate, but too often I see mis-matches between the problem and the evaluative thinking and practice tools used to address them. DE is new, sexy and a sure sign of innovation to some, which is why it is often picked.

Yet, it’s like saying “I need a 3-D printer” when you’re looking to fix a pipe on your sink instead of a wrench, because that’s the latest tool innovation and wrenches are “last year’s” tool. It makes no sense. Yet, it’s done all the time.

Qualities and qualifications

There is something alluring about the mysterious. Innovation, design and systems thinking all have elements of mystery to them, which allows for obfuscation, confusion and well-intentioned errors in judgement depending on who and what is being discussed in relation to those terms.

I’ve started seeing recent university graduates claiming to be developmental evaluators who have almost no concept of complexity, service design, and have completed just a single course in program evaluation. I’m seeing traditional organizations recruit and hire for developmental evaluation without making any adjustments to their expectations, modes of operating, or timelines from the status quo and still expecting results that could only come from DE. It’s as I’ve written before and that Winston Churchill once said:

I am always ready to learn, but I don’t always like being taught

Many programs are not even primed to learn, let alone being taught.

So what should someone look for in DE and those who practice it? What are some questions those seeking DE support ask of themselves?

Of evaluators

  • What familiarity and experience do you have with complexity theory and science? What is your understanding of these domains?
  • What experience do you have with service design and design thinking?
  • What kind of evaluation methods and approaches have you used in the past? Are you comfortable with mixed-methods?
  • What is your understanding of the concepts of knowledge integration and sensemaking? And how have you supported others in using these concepts in your career?
  • What is your education, experience and professional qualifications in evaluation?
  • Do you have skills in group facilitation?
  • How open and willing are you to support learning, adapt, and change your own practice and evaluation designs to suit emerging patterns from the DE?

Of programs

  • Are you (we) prepared to alter our normal course of operations in support of the learning process that might emerge from a DE?
  • How comfortable are we with uncertainty? Unpredictability? Risk?
  • Are our timelines and boundaries we place on the DE flexible and negotiable?
  • What kind of experience do we have truly learning and are we prepared to create a culture around the evaluation that is open to learning? (This means tolerance of ambiguity, failure, surprise, and new perspectives?)
  • Do we have practices in place that allow us to be mindful and aware of what is going on regularly (as opposed to every 6-months to a year)?
  • How willing are we to work with the developmental evaluator to learn, adapt and design our programs?
  • Are our funders/partners/sponsors/stakeholders willing to come with us on our journey?

Of both evaluators and program stakeholders

  • Are we willing to be open about our fears, concerns, ideas and aspirations with ourselves and each other?
  • Are we willing to work through data that is potentially ambiguous, contradictory, confusing, time-sensitive, context-sensitive and incomplete in capturing the entire system?
  • Are we willing/able to bring others into the journey as we go?

DE is not a magic bullet, but it can be a very powerful ally to programs who are operating in domains of high complexity and require innovation to adapt, thrive and build resilience. It is an important job and a very formidable challenge with great potential benefits to those willing to dive into it competently. It is for these reasons that it is worth doing and doing well.

In order for us to get there this means taking DE seriously and the demands it puts on us, the requirements for all involved, and the need to be clear in our language lest we let the not-good-enough be the enemy of the great.

 

Photo credit: Highline Chairs by the author

Follow

Get every new post delivered to your Inbox.

Join 3,677 other followers

%d bloggers like this: