Tag: research

design thinkingpsychologyresearch

Elevating Design & Design Thinking

on October 25, 2017

 

ElevateDesignLoft2.jpgDesign thinking has brought the language of design into popular discourse across different fields, but it’s failings threaten to undermine the benefits it brings if they aren’t addressed. In this third post in a series, we look at how Design (and Design Thinking) can elevate themselves above their failings and match the hype with real impact. 

In two previous posts, I called out ‘design thinkers’ to get the practice out of it’s ‘bullshit’ phase, characterized by high levels of enthusiastic banter, hype, and promotion and low evidence, evaluation or systematic practice.

Despite the criticism, it’s time for Design Thinking (and the field of Design more specifically) to be elevated beyond its current station. I’ve been critical of Design Thinking for years: its popularity has been helpful in some ways, problematic in others.  Others have, too. Bill Storage, writing in 2012 (now unavailable), said:

Design Thinking is hopelessly contaminated. There’s too much sleaze in the field. Let’s bury it and get back to basics like good design.

Bruce Nussbaum, who helped popularize Design Thinking in the early 2000’s called it a ‘failed experiment’, seeking to promote the concept of Creative Intelligence instead. While many have called for Design Thinking to die, it’s not going to happen anytime soon. Since first publishing a piece on Design Thinking’s problems five years ago the practice has only grown. Design Thinking is going to continue to grow, despite its failings and that’s why it matters that we pay attention to it — and seek to make it better.

Lack of quality control, standardization or documentation of methods, and evidence of impact are among the biggest problems facing Design Thinking if it is to achieve anything substantive beyond generating money for those promoting it.

Giving design away, better

It’s hard to imagine that the concepts of personality, psychosis, motivation, and performance measurement from psychology were once unknown to most people. Yet, before the 1980’s, much of the public’s understanding of psychology was confined to largely distorted beliefs about Freudian psychoanalysis, mental illness, and rat mazes. Psychology is now firmly ensconced in business, education, marketing, public policy, and many other professions and fields. Daniel Kahneman, a psychologist, won the Nobel Prize in Economics in 2002 for his work applying psychological and cognitive science to economic decision making.

The reason for this has much to do with George Miller who, as President of the American Psychological Association, used his position to advocate that professional psychology ‘give away’ its knowledge to ensure its benefits were more widespread. This included creating better means of communicating psychological concepts to non-psychologists and generating the kind of evidence that could show its benefits.

Design Thinking is at a stage where we are seeing similar broad adoption beyond professional design to these same fields of business, education, the military and beyond. While there has been much debate about whether design thinking as practiced by non-designers (like MBA’s) is good for the field as a whole, there is little debate that its become popular just as psychology has.

What psychology did poorly is that it gave so much away that it failed to engage other disciplines enough to support quality adoption and promotion and, simultaneously, managed to weaken itself as newfound enthusiasts pursued training in these other disciplines. Now, some of the best psychological practice is done by social workers and the most relevant research comes from areas like organizational science and new ‘sub-disciplines’ like behavioural economics, for example.

Design Thinking is already being taught, promoted, and practiced by non-designers. What these non-designers often lack is the ‘crit’ and craft of design to elevate their designs. And what Design lacks is the evaluation, evidence, and transparency to elevate its work beyond itself.

So what next?

BalloonsEgypt.jpg

Elevating Design

As Design moves beyond its traditional realms of products and structures to services and systems (enabled partly by Design Thinking’s popularity) the implications are enormous — as are the dangers. Poorly thought-through designs have the potential to exacerbate problems rather than solve them.

Charles Eames knew this. He argued that innovation (which is what design is all about) should be a last resort and that it is the quality of the connections (ideas, people, disciplines and more) we create that determine what we produce and their impact on the world. Eames and his wife Ray deserve credit for contributing to the elevation of the practice of design through their myriad creations and their steadfast documentation of their work. The Eames’ did not allow themselves to be confined by labels such as product designer, interior designer, or artist. They stretched their profession by applying craft, learning with others, and practicing what they preached in terms of interdisciplinarity.

It’s now time for another elevation moment. Designers can no longer be satisfied with client approval as the key criteria for success. Sustainability, social impact, and learning and adaptation through behaviour change are now criteria that many designers will need to embrace if they are to operate beyond the fields’ traditional domains (as we are now seeing more often). This requires that designers know how to evaluate and study their work. They need to communicate with their clients better on these issues and they must make what they do more transparent. In short: designers need to give away design (and not just through a weekend design thinking seminar).

Not every designer must get a Ph.D. in behavioural science, but they will need to know something about that domain if they are to work on matters of social and service design, for example. Designers don’t have to become professional evaluators, but they will need to know how to document and measure what they do and what impact it has on those touched by their designs. Understanding research — that includes a basic understanding of statistics, quantitative and qualitative methods — is another area that requires shoring up.

Designers don’t need to become researchers, but they must have research or evaluation literacy. Just as it is becoming increasingly unacceptable that program designers from fields like public policy and administration, public health, social services, and medicine lack understanding of design principles, so is it no longer feasible for designers to be ignorant of proper research methods.

It’s not impossible. Clinical psychologists went from being mostly practitioners to scientist-practitioners. Professional social workers are now well-versed in research even if they typically focus on policy and practice. Elevating the field of Design means accepting that being an effective professional requires certain skills and research and evaluation are now part of that skill set.

DesignWindow2.jpeg

Designing for elevated design

This doesn’t have to fall on designers to take up research — it can come from the very people who are attracted to Design Thinking. Psychologists, physicians, and organizational scientists (among others) all can provide the means to support designers in building their literacy in this area.

Adding research courses that go beyond ethnography and observation to give design students exposure to survey methods, secondary data analysis, ‘big data’, Grounded Theory approaches, and blended models for data collection are all options. Bring the behavioural and data scientists into the curriculum (and get designers into the curriculum training those professionals).

Create opportunities for designers to do research, publish, and present their research using the same ‘crit’ that they bring to their designs. Just as behavioural scientists expose themselves to peer review of their research, designers can do the same with their research. This is a golden opportunity for an exchange of ideas and skills between the design community and those in the program evaluation and research domains.

This last point is what the Design Loft initiative has sought to do. Now in its second year, the Design Loft is a training program aimed at exposing professional evaluators to design methods and tools. It’s not to train them as designers, but to increase their literacy and confidence in engaging with Design. The Design Loft can do the same thing with designers, training them in the methods and tools of evaluation. It’s but one example.

In an age where interdisciplinarity is spoken of frequently this provides the means to practically do it and in a way that offers a chance to elevate design much like the Eames’ did, Milton Glaser did, and how George Miller did for psychology. The time is now.

If you are interested in learning more about the Design Loft initiative, connect with Cense. If you’re a professional evaluator attending the 2017 American Evaluation Association conference in Washington, the Design Loft will be held on Friday, November 10th. 

Image Credit: Author

art & designcomplexitysocial systemssystems thinking

A Beautiful Idea

on April 10, 2017

MakeSomethingBeautiful_Snapseed

Is what you do, where you work, or how you organize, beautiful? Among the many words used to describe our work lives the most neglected and maybe necessary might be described that one word: it’s time to take it seriously. 

For those working in design one of the biggest challenges is getting people to understand that good design isn’t just about making things pretty, but making them better, more useful, more responsive, sustainable, and impactful. Good design is too often seen as a ‘nice to have’ than a ‘must have’ and is thus invested in accordingly.

‘Beautiful’ as a concept has it even worse. In my entire working career I’ve never heard the word uttered even once on a matter of professional importance by others. That’s a shame and it speaks loudly to our present situation where innovation is hard to come by, organizations struggle to attract and retain good people, and the battle for attention — of the market and our workforce — is maybe the biggest one of them all.

But beauty is worth a look, particularly because it is, well, beautiful.

A beautiful term

What is beautiful? Consider the Oxford English Dictionary’s definition.

beautiful |ˈbyo͞odəfəl| adjective

pleasing the senses or mind aesthetically: beautiful poetry | a beautiful young woman | the mountains were calm and beautiful.

• of a very high standard; excellent: the house had been left in beautiful order | she spoke in beautiful English.

Note two key features of this definition: pleasing the senses or mind and high standards. The first part might sound a bit hedonistic (PDF), but when you consider what motivates us at the most base level of existence: it’s pleasure and pain. We are attracted to people, experiences, objects and environments that generate pleasure. In an environment described above when attracting talent, eyeballs — attention — is so hard to come by, why would we not amplify beauty?

The second term is high standards. It’s not enough to attract attention, we need to hold it and to inspire action, loyalty and persistence if we wish to succeed on most counts. Quality is a competitive advantage in many environments, particularly in human services where the complexity associated with poor quality decisions, processes and management are potentially catastrophic. (Enron, anyone?).

An associated term to this is aesthetics, which is defined as:

aesthetic |esˈTHedik| (also esthetic) adjective

concerned with beauty or the appreciation of beauty: the pictures give great aesthetic pleasure.

• giving or designed to give pleasure through beauty; of pleasing appearance.

Aesthetics is the more active appreciation of beauty — the application of it in the world. Organizational aesthetics is an emergent area of scholarship and practice that seeks to understand the role of beauty in the organization and its implications. Steven Taylor describes organizational aesthetics through storytelling, outlining the way he came to know something through connecting his work with his senses. His story points to different ways in which organizational aesthetics is experienced and understood, but ultimately how its sensed. It’s that attention to the senses that really sets this field apart, but also how practical it is.

Practical beauty

Organizational aesthetics are about practical realities of organizational life, brought to bear through our five senses, not just the mind. Strange that so much of what is produced in the literature and scholarship is so cognitive and devoid of discussion of any other sensory experiences. Yet, we are sensuous beings and most healthy when we are in touch (literally!) with our senses in our lives. Consider the cortical homunculus and you’ll know that we feel through a lot more than we often use in our work lives.

Organizational aesthetics is about using methods that tap into these senses and the qualities of physical, social, psychological spaces where they can be used more fully to contribute to more impactful, healthier and happier environments for humans to work and thrive. This approach is rooted in design and the hypothesis that, as human created (thus designed) constructs, the modern organization can design in beauty as much as it can design beauty away. Like design itself, organizational aesthetics is practical, above all.

Citing earlier work from Roozenberg & Eekles (1995) on the topic of design causality, Steven De Groot, from the Eindhoven University of Technology, points to the way in which design is a responsive means to helping an organization adapt.

By fulfilling functions a design satisfies needs, and gives people the possibility to realize one or more values. Transferring these fundamentals, the design of the organization needs to change as a consequence of changing roles and needs of the employees in this case.

Roozenberg and Eekles assert that form follows value and thus, as De Groot sought to explore, explicit value of beauty can produce beautiful organizations. The reasoning for this research comes from earlier studies that show that when organizations value and nurture beauty within them, employees are happier, their commitment increases and the organizational function is improved.

Dispelling beautiful myths

Despite the reams of research that has emerged from a variety of disciplines showing the connections between beauty and positive outcomes and experiences in organizations, there will be many who are still troubled by the idea of integrating the word ‘beautiful’ into the serious world of work. It may be tempted to rely on a few myths to deny its utility so let’s dispel those right away.

  1. Promotion of beauty is not denial of the ugly. Ugly is everywhere: in the news, on social media (spent time on Facebook lately?), and embedded in many of our global, social challenges. Embracing the beautiful is not about denying ugly, but drawing our focus to areas where we can create change. As I discussed in a previous post, good design is increasingly about reducing information overload and focusing on areas we can influence by creating positive attractors, not negative ones. It’s based on attention and human nature. We stop and remark on fresh cut flowers. We comment on a colleagues’ attractive new outfit or clothing item (“I love your new socks!”). We see something that is well designed and we admire it, covet it or just enjoy it. Beauty captures something of the most rarest of commodities in the modern age: attention. We won’t change the world by yelling louder, we’ll change it by speaking beautifully, better.
  2. There’s no single definition of beauty. Beauty is truly subjective. What I might find particularly beautiful is different than what someone else will, yet there is much evidence that there is also a shared sense of the beautiful. Pierre Bourdieu’s work on taste and taste-making (PDF) points to the social means in which we — fair or not — share perspectives to elevate ideas, concepts and artifacts. We are social and thus share social rules, tastes and ideas and that this might be done across cultures, within ‘tribes’ or tied to specific settings or groups, but there is always something shared.
  3. There are shared principles of beauty. What makes for a shared cultural experience is something that we refer to as simple rules in complexity studies. These are rules that may be explicit, unconscious or tacit that guide collective actions and shared experiences. It, combined with history (and something we call path dependence – a driver of stability and stasis in a system), is what allows us to have some collective appreciation of the beautiful. It’s why natural elements (e.g., plants) or use of certain colours can create a positive atmosphere and psychological experience within a setting even if those plants or colours are universally loved.
  4. There is plenty of evidence to support the case for making changes based on beauty. This ‘absence of evidence’ myth will take a while to dispel as people will see (or not see) what they want to. All I would suggest is that you take a long hard look at some of the research — in particular Steven de Groot’s doctoral work — and put that up against any other theory or program of research and explain how it’s less than — particularly given how young of a field it is. There is an entire academic journal devoted to this topic (and, like in any journal, not all the evidence is top-notch, but there’s good work in there and throughout the literature). Consider how management theory, a well-established area of scholarship, is already becoming ‘a compendium of dead ideas‘ given the paucity of solid research behind it and yet something like organizational aesthetics hasn’t taken hold? The battle is long, but adoption of some new, beautiful thinking is one that will pay off. I’ve not even started getting into the arguments for environmental and organizational psychology or design.

Change in a complex system is about creating, finding and amplifying positive attractors and dampening and eliminating negative ones (and in complex systems positive isn’t always good and negative bad, it’s about what the goals are in the system — what you wish to achieve within that system. In society, these are almost always socially negotiated, somewhat contested).

Attracting attention, ideas, and energy is one of our biggest social challenges at the moment and a huge barrier to change.

Everyone’s looking for a way to capture attention and hold it when there is a beautiful solution right under their noses.

“Everyone needs beauty as well as bread, places to play in and pray in where nature may heal and cheer and give strength to body and soul alike” – John Muir, 1869

Image Credit: Author

behaviour changecomplexitypublic healthsocial innovation

Confusing change-making with actual change

on May 3, 2016

658beggar_KeepCoinsChange

Change-making is the process of transformation and not to be confused with the transformed outcome that results from such a process. We confuse the two at our peril.

“We are changing the world” is a rallying cry from many individuals and organizations working in social innovation and entrepreneurship which is both a truth and untruth at the same time. Saying you’re changing the world is far easier than actually doing it. One is dramatic — the kind that make for great reality TV as we’ll discuss — and the other is rather dull, plodding and incremental. But it may be the latter that really wins the day.

Organizations like Ashoka (and others) promote themselves as a change-maker organization authoring blogs titled “everything you need to know about change-making”. That kind of language, while attractive and potentially inspiring to diverse audiences, points to a mindset that views social change in relatively simple, linear terms. This line of thinking suggests change is about having the right knowledge and the right plan and the ability to pull it together and execute.

This is a mindset that highlights great people and great acts supported by great plans and processes. I’m not here to dismiss the work that groups like Ashoka do, but to ask questions about whether the recipe approach is all that’s needed. Is it really that simple?

Lies like: “It’s calories in, calories out”

Too often social change is viewed with the same flawed perspective that weight loss is. Just stop eating so much food (and the right stuff) and exercise and you’ll be fine — calories in and out as the quote suggests — and you’re fine. The reality is, it isn’t that simple.

A heartbreaking and enlightening piece in the New York Times profiled the lives and struggles of past winners of the reality show The Biggest Loser (in parallel with a new study released on this group of people (PDF)) that showed that all but one of the contestants regained weight after the show as illustrated below:

BiggestLoser 2016-05-03 09.17.10

The original study, published in the journal Obesity, considers the role of metabolic adaptation that takes place with the authors suggesting that a person’s metabolism makes a proportional response to compensate for the wide fluctuations in weight to return contestants to their original pre-show weight.

Consider that during the show these contestants were constantly monitored, given world-class nutritional and exercise supports, had tens of thousands of people cheering them on and also had a cash prize to vie for. This was as good as it was going to get for anyone wanting to lose weight shy of surgical options (which have their own problems).

Besides being disheartening to everyone who is struggling with obesity, the paper illuminates the inner workings of our body and reveals it to be a complex adaptive system rather than the simple one that we commonly envision when embarking on a new diet or fitness regime. Might social change be the same?

We can do more and we often do

I’m fond of saying that we often do less than we think and more than we know.

That means we tend to expect that our intentions and efforts to make change produce the results that we seek directly and because of our involvement. In short, we treat social change as a straightforward process. While that is sometimes true, rare is it that programs aiming at social change coming close to achieving their stated systems goals (“changing the world”) or anything close to it.

This is likely the case for a number of reasons:

  • Funders often require clear goals and targets for programs in advance and fund based on promises to achieve these results;
  • These kind of results are also the ones that are attractive to outside audiences such as donors, partners, academics, and the public at large (X problem solved! Y number of people served! Z thousand actions taken!), but may not fully articulate the depth and context to which such actions produce real change;
  • Promising results to stakeholders and funders suggests that a program is operating in a simple or complicated system, rather than a complex one (which is rarely, if ever the case with social change);
  • Because program teams know these promised outcomes don’t fit with their system they cherry-pick the simplest measures that might be achievable, but may also be the least meaningful in terms of social change.
  • Programs will often further choose to emphasize those areas within the complex system that have embedded ordered (or simple) systems in them to show effect, rather than look at the bigger aims.

The process of change that comes from healthy change-making can be transformative for the change-maker themselves, yet not yield much in the way of tangible outcomes related to the initial charge. The reasons likely have to do with the compensatory behaviours of the system — akin to social metabolic adaptation — subduing the efforts we make and the initial gains we might experience.

Yet, we do more at the same time. Danny Cahill, one of the contestants profiled in the story for the New York Times, spoke about how the lesson learned from his post-show weight gain was that the original weight gain wasn’t his fault in the first place

“That shame that was on my shoulders went off”

What he’s doing is adapting his plan, his goals and working differently to rethink what he can do, what’s possible and what is yet to be discovered. This is the approach that we take when we use developmental evaluation; we adapt, evolve and re-design based on the evidence while continually exploring ways to get to where we want to go.

A marathon, not a sprint, in a laboratory

The Biggest Loser is a sprint: all of the change work compressed into a short period of time. It’s a lab experiment, but as we know what happens in a laboratory doesn’t always translate directly into the world outside its walls because the constraints have changed. As the show’s attending physician, Dr. Robert Huizenga, told the New York Times:

“Unfortunately, many contestants are unable to find or afford adequate ongoing support with exercise doctors, psychologists, sleep specialists, and trainers — and that’s something we all need to work hard to change”

This quote illustrates the fallacy of real-world change initiatives and exposes some of the problems we see with many of the organizations who claim to have the knowledge about how to change the world. Have these organizations or funders gone back to see what they’ve done or what’s left after all the initial funding and resources were pulled? This is not just a public, private or non-profit problem: it’s everywhere.

I have a colleague who spent much time working with someone who “was hired to clean up the messes that [large, internationally recognized social change & design firm] left behind” because the original, press-grabbing solution actually failed in the long run. And the failure wasn’t in the lack of success, but the lack of learning because that firm and the funders were off to another project. Without building local capacity for change and a sustained, long-term marathon mindset (vs. the sprint) we are setting ourselves up for failure. Without that mindset, lack of success may truly be a failure because there is no capacity to learn and act based on that learning. Otherwise, the learning is just a part of an experimental approach consistent with an innovation laboratory. The latter is a positive, the former, not so much.

Part of the laboratory approach to change is that labs — real research labs — focus on radical, expansive, long-term and persistent incrementalism. Now that might sound dull and unsexy (which is why few seem to follow it in the social innovation lab space), but it’s how change — big change — happens. The key is not in thinking small, but thinking long-term by linking small changes together persistently. To illustrate, consider the weight gain conundrum as posed by obesity researcher Dr. Michael Rosenbaum in speaking to the Times:

“We eat about 900,000 to a million calories a year, and burn them all except those annoying 3,000 to 5,000 calories that result in an average annual weight gain of about one to two pounds,” he said. “These very small differences between intake and output average out to only about 10 to 20 calories per day — less than one Starburst candy — but the cumulative consequences over time can be devastating.”

Building a marathon laboratory

Marathoners are guided by a strange combination of urgency, persistence and patience. When you run 26 miles (42 km) there’s no sprinting if you want to finish the same day you started. The urgency is what pushes runners to give just a little more at specific times to improve their standing and win. Persistence is the repetition of a small number of key things (simple rules in a complex system) that keep the gains coming and the adaptations consistent. Patience is knowing that there are few radical changes that will positively impact the race, just a lot of modifications and hard work over time.

Real laboratories seek to learn a lot, simply and consistently and apply the lessons from one experiment to the next to extend knowledge, confirm findings, and explore new territory.

Marathons aren’t as fun to watch as the 100m sprint in competitive athletics and lab work is far less sexy than the mythical ‘eureka’ moments of ‘discovery’ that get promoted, but that’s what changes the world. The key is to build organizations that support this. It means recognizing learning and that it comes from poor outcomes as well as positive ones. It encourages asking questions, being persistent and not resting on laurels. It also means avoiding getting drawn into being ‘sexy’ and ‘newsworthy’ and instead focusing on the small, but important things that make the news possible in the first place.

Doing that might not be as sweet as a Starburst candy, but it might avoid us having to eat it.

 

 

 

behaviour changecomplexitypsychologysocial innovationsocial systems

Decoding the change genome

on April 12, 2016

11722018066_c29148186b_k

Would we invest in something if we had little hard data to suggest what we could expect to gain from that investment? This is often the case with social programs, yet its a domain that has resisted the kind of data-driven approaches to investment that we’ve seen in other sectors and one theory is that we can approach change in the same way we code the genome, but: is that a good idea?

Jason Saul is a maverick in social impact work and dresses the part: he’s wearing a suit. That’s not typically the uniform of those working in the social sector railing against the system, but that’s one of the many things that gets people talking about what he and his colleagues at Mission Measurement are trying to do. That mission is clear: bring the same detailed analysis of the factors involved in contributing to real impact from the known evidence that we would do to nearly any other area of investment.

The way to achieving this mission is to take the thinking behind the Music Genome Project, the algorithms that power the music service Pandora, and apply it to social impact. This is a big task and done by coding the known literature on social impact from across the vast spectrum of research from different disciplines, methods, theories and modeling techniques. A short video from Mission Measurement on this approach nicely outlines the thinking behind this way of looking at evaluation, measurement, and social impact.

Saul presented his vision for measurement and evaluation to a rapt audience in Toronto at the MaRS Discovery District on April 11th as part of their Global Leaders series en route to the Skoll World Forum ; this is a synopsis of what came from that presentation and it’s implications for social impact measurement.

(Re) Producing change

Saul began his presentation by pointing to an uncomfortable truth in social impact: We spread money around with good intention and little insight into actual change. He claims (no reference provided) that 2000 studies are published per day on behaviour change, yet there remains an absence of common metrics and measures within evaluation to detect change. One of the reasons is that social scientists, program leaders, and community advocates resist standardization making the claim that context matters too much to allow aggregation.

Saul isn’t denying that there is truth to the importance of context, but argues that it’s often used as an unreasonable barrier to leading evaluations with evidence. To this end, he’s right. For example, the data from psychology alone shows a poor track record of reproducibility, and thus offers much less to social change initiatives than is needed. As a professional evaluator and social scientist, I’m not often keen to being told how to do what I do, (but sometimes I benefit from it). That can be a barrier, but also it points to a problem: if the data shows how poorly it is replicated, then is following it a good idea in the first place? 

Are we doing things righter than we think or wronger than we know?

To this end, Saul is advocating a meta-evaluative perspective: linking together the studies from across the field by breaking down its components into something akin to a genome. By looking at the combination of components (the thinking goes) like we do in genetics we can start to see certain expressions of particular behaviour and related outcomes. If we knew these things in advance, we could potentially invest our energy and funds into programs that were much more likely to succeed. We also could rapidly scale and replicate programs that are successful by understanding the features that contribute to their fundamental design for change.

The epigenetic nature of change

Genetics is a complex thing. Even on matters where there is reasonably strong data connecting certain genetic traits to biological expression, there are few examples of genes as ‘destiny’as they are too often portrayed. In other words, it almost always depends on a number of things. In recent years the concept of epigenetics has risen in prominence to provide explanations of how genes get expressed and it has as much to do with what environmental conditions are present as it is the gene combinations themselves . McGill scientist Moshe Szyf and his colleagues pioneered research into how genes are suppressed, expressed and transformed through engagement with the natural world and thus helped create the field of epigenetics. Where we once thought genes were prescriptions for certain outcomes, we now know that it’s not that simple.

By approaching change as a genome, there is a risk that the metaphor can lead to false conclusions about the complexity of change. This is not to dismiss the valid arguments being made around poor data standardization, sharing, and research replication, but it calls into question how far the genome model can go with respect to social programs without breaking down. For evaluators looking at social impact, the opportunity is that we can systematically look at the factors that consistently produce change if we have appropriate comparisons. (That is a big if.)

Saul outlined many of the challenges that beset evaluation of social impact research including the ‘file-drawer effect’ and related publication bias, differences in measurement tools, and lack of (documented) fidelity of programs. Speaking on the matter in response to Saul’s presentation, Cathy Taylor from the Ontario Non-Profit Network, raised the challenge that comes when much of what is known about a program is not documented, but embodied in program staff and shared through exchanges.  The matter of tacit knowledge  and practice-based evidence is one that bedevils efforts to compare programs and many social programs are rich in context — people, places, things, interactions — that remain un-captured in any systematic way and it is that kind of data capture that is needed if we wish to understand the epigenetic nature of change.

Unlike Moshe Szyf and his fellow scientists working in labs, we can’t isolate, observe and track everything our participants do in the world in the service of – or support to – their programs, because they aren’t rats in a cage.

Systems thinking about change

One of the other criticisms of the model that Saul and his colleagues have developed is that it is rather reductionist in its expression. While there is ample consideration of contextual factors in his presentation of the model, the social impact genome is fundamentally based on reductionist approaches to understanding change. A reductionist approach to explaining social change has been derided by many working in social innovation and environmental science as outdated and inappropriate for understanding how change happens in complex social systems.

What is needed is synthesis and adaptation and a meta-model process, not a singular one.

Saul’s approach is not in opposition to this, but it does get a little foggy how the recombination of parts into wholes gets realized. This is where the practical implications of using the genome model start to break down. However, this isn’t a reason to give up on it, but an invitation to ask more questions and to start testing the model out more fulsomely. It’s also a call for systems scientists to get involved, just like they did with the human genome project, which has given us great understanding of what influences our genes have and stressed the importance of the environment and how we create or design healthy systems for humans and the living world.

At present, the genomic approach to change is largely theoretical backed with ongoing development and experiments but little outcome data. There is great promise that bigger and better data, better coding, and a systemic approach to looking at social investment will lead to better outcomes, but there is little actual data on whether this approach works, for whom, and under what conditions. That is to come. In the meantime, we are left with questions and opportunities.

Among the most salient of the opportunities is to use this to inspire greater questions about the comparability and coordination of data. Evaluations as ‘one-off’ bespoke products are not efficient…unless they are the only thing that we have available. Wise, responsible evaluators know when to borrow or adapt from others and when to create something unique. Regardless of what design and tools we use however, this calls for evaluators to share what they learn and for programs to build the evaluative thinking and reflective capacity within their organizations.

The future of evaluation is going to include this kind of thinking and modeling. Evaluators, social change leaders, grant makers and the public alike ignore this at their peril, which includes losing opportunities to make evaluation and social impact development more accountable, more dynamic and impactful.

Photo credit (main): Genome by Quinn Dombrowski used under Creative Commons License via Flickr. Thanks for sharing Quinn!

About the author: Cameron Norman is the Principal of Cense Research + Design and assists organizations and networks in supporting learning and innovation in human services through design, program evaluation, behavioural science and system thinking. He is based in Toronto, Canada.

complexityinnovationsocial innovation

The Ecology of Innovation: Part 2 – Language

on October 2, 2015
Idea Factories or ecologies of innovation?

Idea Factories or ecologies of innovation?

Although Innovation is about producing value through doing something new or different than before, the concept is far from simple when applied in practice by individuals and institutions. This second in a series of articles on innovation ecology looks at the way we speak of innovation and how what we talk about new ideas and discovery shapes what we do about it. 

“Language can be a way of hiding your thoughts and preventing communication” – Abraham Maslow

Innovation is one of the few concepts that offers little benefit contemplated in the abstract. We innovate on specific things with an eye to application, maybe even scaling that idea broadly. Humans innovate because the status quo is no longer satisfying, is unacceptable or has changed so we strive to come up with new ways of doing things, novel processes and tools to make the current situation a preferred one.

Thus, we are designers seeking our client, customer and creation through innovation and we do this through our words and actions — our language. Indeed, if one agrees with Marty Neumeier‘s assertion that design is the discipline of innovation and Greg Van Alystne & Bob Logan’s definition of design as “creation for reproduction” then our language of innovation is critical to ensuring that we design products and services that have the potential to reproduce beyond an idea.

Language matters in innovation.

To illustrate, lets look at how language manifests itself in the communication of ideas using an example from public health. In a paper entitled Knowledge integration: Conceptualizing communications in cancer control systems I co-authored with my colleagues Allan Best and Bob Hiatt, we looked at the way language was used within a deep and broad field like cancer control in shaping communications. This was not merely an academic exercise, but served to illustrate the values, practices and structures that are put in place to support communicating concepts and serves to illustrate how innovations are communicated.

Innovation as product

What we found was that there are three generations of cancer communications defined by their language and the practices and policies that are manifested in or representative of that language. The first generation of terms were traced up to the 1990’s and were characterized by viewing knowledge as a product. Indeed, the term knowledge products can be traced back to this period. Other key characteristics of this period include:

  • The terminology used to describe communications included the terms diffusion, dissemination, knowledge transfer, and knowledge uptake.
  • Focus on the handoff between knowledge ‘producers’ and knowledge (or research) ‘users’. These two groups were distinct and separate from one another
  • The degree of use is a function of effective packaging and presentation presuming the content is of high quality.

The language of this first generation makes the assumption that the ideas are independent of the context in which they are to be used or where they were generated. The communication represented in this generation of models relies on expertise and recognition of this. But what happens when expertise is not recognized? Or where expertise isn’t even possible? This is a situation we are increasingly seeing as we face new, complex challenges that require mass collaboration and innovation, something the Drucker Forum suggests represents the end of expertise.

Innovation as a contextual process

From the early and mid-1990’s through to the present we’ve seen a major shift from viewing knowledge or innovation as a product to that of a dynamic process where expertise resides in multiple places and sources and networks are valued as much as institutions or individuals. Some of the characteristics of this generation are:

  • Knowledge and good ideas come from multiple sources, not just recognized experts or leaders
  • Social relationships media what is generated and how it is communicated (and to whom)
  • Innovation is highly context-dependent
  • The degree of use of ideas or knowledge is a function of having strong, effective relationships and processes.

What happens when the context is changing consistently? What happens when the networks are dynamic and often unknown?

Systems-embedded innovation

What the paper argues is that we are seeing a shift toward more systems-oriented approaches to communication and that is represented in the term knowledge integration. A systems-oriented model views the design of knowledge structures as an integral to the support of effective innovation by embedding the activities of innovation — learning, discovery, and communication — within systems like institutions, networks, cultures and policies. This model also recognizes the following:

  • Both explicit and implicit knowledge is recognized and must be made visible and woven into policy making and practice decisions
  • Relationships are mediated through a cycle of innovation and must be understood as a system
  • The degree of integration of policies, practices and processes within a system is what determines the degree of use of an idea or innovation.

The language of integration suggests there is some systems-level plan to take the diverse aspects within a set of activities and connect, coordinate and, to some degree, manage to ensure that knowledge is effectively used.

Talking innovation

What makes language such a critical key to understanding innovation ecologies is that the way in which we speak about something is an indication of what we believe about something and how we act. As the quote from psychologist Abraham Maslow suggests above, language can also be used to hide things.

One example of this is in the realm of social innovation, where ideas are meant to be generated through social means for social benefit. This process can be organized many different ways, but it is almost never exclusively top-down, expert-driven. Yet, when we look at the language used to discuss social innovation, we see terms like dissemination regularly used. Examples from research, practice and connecting the two to inform policy all illustrate that the language of one generation continues to be used as new ones dawn.  This is to be expected as the changes in language of one generation never fully supplants that of previous generations — at least not initially. Because of that, we need to be careful about what we say and how we say it to ensure that our intentions are reflected in our practice and our language. Without conscious awareness of what we say and what those words mean there is a risk that our quest to create true innovation ecosystems, ones where innovation is truly systems-embedded and knowledge is integrated we unwittingly create expectations and practices rooted in other models.

If we wish to walk the walk of innovation at a systems level, we need to talk the talk.

Tips and Tricks

Organizational mindfulness is a key quality and practice that embeds reflective practice and sensemaking into the organization. By cultivating practices that regularly check-in and examine the language and actions of an organization in reference to its goals, processes and outcomes. A recent article by Vogus and Sutcliffe (2012) (PDF) provides some guidance on how this can be understood.

Develop your sensemaking capacity by introducing space at regular meetings that bring together actors from different areas within an organization or network to introduce ideas, insights and observations and process what these mean with respect to what’s happened, what is happening and where its taking the group.

Some key references include: 

Best, A., Hiatt, R. A., & Norman, C. D. (2008). Knowledge integration: Conceptualizing communications in cancer control systems. Patient Education and Counseling, 71(3), 319–327. http://doi.org/10.1016/j.pec.2008.02.013

Best, A., Terpstra, J. L., Moor, G., Riley, B., Norman, C. D., & Glasgow, R. E. (2009). Building knowledge integration systems for evidence‐informed decisions. Journal of Health Organization and Management, 23(6), 627–641. http://doi.org/10.1108/14777260911001644

Vogus, T. J., & Sutcliffe, K. M. (2012). Organizational Mindfulness and Mindful Organizing: A Reconciliation and Path Forward. Academy of Management Learning & Education, 11(4), 722–735. http://doi.org/10.5465/amle.2011.0002C

Weick, K. E., Sutcliffe, K. M., & Obstfeld, D. (2005). Organizing and the Process of Sensemaking. Organization Science, 16(4), 409–421. http://doi.org/10.1287/orsc.1050.0133

*** If you’re interested in applying these principles to your organization and want assistance in designing a process to support that activity, contact Cense Research + Design.

education & learningresearchsystems thinking

The urban legends of learning (and other inconvenient truths)

on July 28, 2014
Learning simulacrum, simulation or something else?

Learning simulacrum, simulation or something else?

Learning styles, technology-driven teaching, and self-direction are all concepts that anyone interested in education should be familiar with, yet the foundations for their adoption into the classroom, lab or boardroom are more suspect than you might think. Today we look at the three urban legends of learning and what that might mean for education, innovation and beyond. 

What kind of learner are you? Are you a visual learner perhaps, where you need information presented in a particular visual style to make sense of it? Maybe you need to problem-solve to learn because that’s the way you’ve been told is best for your education.

Perhaps you are a self-directed learner who is one that, when given the right encouragement and tools, will find your way through the muck to the answers and that others just need to get out of the way. With tools like the web and social media, you have the world’s knowledge at your disposal and have little need to be ‘taught’ that stuff, because its online.

And if you’re a digital native (PDF), this is all second nature to you because you’re able to use multiple technologies simultaneously to solve multiple problems together with ease if given the ability to do so. After all, you’ve had these tools your entire life.

A recent article by Paul Kirschner and Jeroen van Merriënboer published in the peer-reviewed journal Educational Psychologist challenges these ‘truths’ and many more, calling them urban legends:

An urban legend, urban myth, urban tale, or contemporary legend, is a form of modern folklore consisting of stories that may or may not have been believed by their tellers to be true.

The authors are quick to point out that there are differences in the way people approach material and prefer to learn, but they also illustrate that there is relatively little evidence to support much of the thinking that surrounds these practices, confusing learning preferences for learning outcomes. I’ve commented on this before, noting that too often learning is conflated with interest and enjoyment when they are different things and if we were really serious about it we might change the way we do a great deal many things in life.

In the paper, the authors debunk — or at least question — the evidence that supports the ‘legends’ of digital natives as a type of learner, the presence of specific learning styles and the need to customize learning to suit such styles of learning, and that of the lone self-educator. In each case, the authors present much evidence to challenge these ideas so as not to take them as truths, but hypotheses that have little support for them in practice.

Science and its inconvenient truths about learning

Science has a funny way of revealing truths that we may find uncomfortable or at least challenge our current orthodoxy.

This reminds me of a terrific quote from the movie Men in Black that illustrates the fragility of ideas in the presence and absence of evidence after one of the characters (played by Will Smith) uncovers that aliens were living on earth (in the film) and is consoled by his partner (played by Tommy Lee Jones) about what is known and unknown in the world:

Fifteen hundred years ago everybody knew the Earth was the center of the universe. Five hundred years ago, everybody knew the Earth was flat, and fifteen minutes ago, you knew that humans were alone on this planet. Imagine what you’ll know tomorrow.

One of the problems with learning is that there is a lot to learn and not all of it is the same in content, format and situational utility. Knowledge is not a ‘thing’ in the way that potatoes, shoes, patio furniture, orange juice, and pencils are things where you can have more or less of it and measure the increase, decrease and change in it over time. But we often treat it that way. Further, knowledge is also highly contextualized and combines elements that are stable, emergent, and transformative in new, complex arrangements simultaneously over time. It is a complex adaptive system.

Learning (in practice) resists simple truths.

It’s why we can be taught something over and again and not get it, while other things get picked up quickly within the same person even if the two ‘things’ seem alike. The conditions in which a person might learn are cultural (e.g., exposure to teaching styles at school, classroom designs, educational systems, availability and exposure to technology, life experiences, emphasis on reflective living/practice within society, time to reflect etc..) and psycho-social/biological (e.g., attention, intelligence, social proximity, literacy, cognitive capacity for information processing, ability to engage with others) so to reduce this complex phenomena to a series of statements about technology, preference and perception is highly problematic.

Science doesn’t have all the answers — far from it — but at least it can test out what is consistent and observable over time and build on that. In doing so, it exposes the responsibility we have as educators and learners.

With great power comes great responsibility…?

Underpinning the urban legends discussed by Kirschner and van Merriënboer and not discussed is the tendency for these legends to create a hands-off learning systems where workplaces, schools, and social systems are freed from the responsibility of shaping learning experiences and opportunities. It effectively reduces institutional knowledge, wisdom and experience to mere variables in a panoply of info-bites treated as all the same.

It also assumes that design doesn’t matter, which undermines the ability to create spaces and places that optimize learning options for people from diverse circumstances.

This mindset frees organizations from having to give time to learning, provide direction (i.e., do their own homework and set the conditions for effective learning and knowledge integration at the outset). It also frees us up from having to choose, to commit to certain ideas and theories, which means some form of discernment, priority setting, and strategy. That requires work up front and leadership and hard, critical, and time-consuming conversations about what is important, what we value in our work, and what we want to see.

When we assume everyone will just find their way we abdicate that responsibility.

Divesting resources and increasing distraction

In my home country of Canada, governments have been doing this with social investment for years where the federal government divests interest to the provinces who divest it to cities and towns who divest it to the public (and private) sector, which means our taxes never go up even if the demands on services do and we find that individual citizens are responsible for more of the process of generating collective benefit without the advantage of any scaled system to support resource allocation and deployment throughout society (which is why we have governments in the first place). It also means our services and supports — mostly — get smaller, lesser in quality, more spread thinly, and lose their impact because there isn’t the scaled allocation of resources to support them.

Learning is the same way. We divest our interests in it and before you know it, we learn less and do less with it because we haven’t the cultural capital, traditions or infrastructure to handle it. Universities turn campus life to an online experience. Secondary schools stop or reduce teaching physical education that involves actual physical activity.  Scholarly research is reduced to a Google search. Books are given up as learning vehicles because they take too long to read. It goes on.

It’s not that there are no advantages to some of these ideas in some bites, but that we are transforming the entire enterprise with next to no sense of the systems they are operating in, the mission they are to accomplish, a theory of change that is backed up by evidence, or the will to generate the evidence needed to advise and the resources to engage in the sensemaking needed to evaluate that evidence.

Science, systems and learning

It is time to start some serious conversations about systems, science and learning. It would help if we started getting serious about what we mean when we speak of learning, what theories we use to underpin that language and what evidence we have (or need) to understand what those theories mean in practice and for policy. This starts by asking better questions — and lots of them — about learning and its role in our lives and work.

Design thinking and systems thinking are two thinking tools that can help us find and frame these issues. Mindfulness and its ethics associated with non-judgement, open-mindedness, compassion and curiosity are also key tools. The less we judge, the more open we are to asking good questions about what we are seeing that can lead us to getting better answers rather than getting trapped by urban legends.

Doing this within a systems thinking frame also allows us to see how what we learn and where and how we learn is interconnected to better spot areas of leverage and problems in our assumptions.

This might allow us to make many of our urban legends obsolete instead of allowing them to grow like the alligators that live in the sewers of New York City. 

 

 

eHealthinnovationpublic healthsocial innovationsocial media

Seeing the lights in research with our heads in the clouds

on July 11, 2014
Lights in the clouds

Lights in the clouds

Some fields stagnate because they fail to take the bold steps into the unknown by taking chances and proposing new ideas because the research isn’t there to guide it while social innovation has a different twist on the problem: it has plenty of ideas, but little research to support those ideas. Unless the ideas and research match up it is unlikely that either area will develop.

 

Social innovation is a space that doesn’t lack for dreamers and big ideas. That is a refreshing change of pace from the world of public policy and public health that are well-populated by those who feel chained down to what’s been done as the entry to doing something new (which is oxymoronic when you think about it).

Fields like public health and medicine are well-served by looking to the evidence for guidance on many issues, but an over-reliance on using past-practice and known facts as the means to guide present action seriously limits the capacity to innovate in spaces where evidence doesn’t exist and may not be forthcoming.

The example of eHealth, social media and healthcare

A good example of this is in the area of eHealth. While social media has been part of the online communication landscape for nearly a decade (or longer, depending on your definition of the term), there has been sparse use of these tools and approaches within the health domain by professionals until recently. Even today, the presence of professional voices on health matters is small within the larger discourse on health and wellbeing online.

One big reason for this — and there are many — is that health systems are not prepared for the complexity that social media introduces.  Julia Belluz’s series on social media and healthcare at Macleans provides among the best examples of the gaps that social media exposes and widens within the overlapping domains of health, medicine, media and the public good. Yet, such problems with social media do not change the fact that it is here, used by billions worldwide, and increasingly becoming a vehicle for discussing health matters from heart disease to weight management to smoking cessation.

Social innovation and research

Social innovation has the opposite problem. Vision, ideas, excitement and energy for new ideas abound within this world, yet the evidence generation to support it, improve upon it and foster further design innovations is notably absent (or invisible). Evaluation is not a word that is used much within this sphere nor is the term research applied — at least with the rigour we see in the health field.

In late May I participated in a one-day event in Vancouver on social innovation research in Vancouver organized by the folks at Simon Fraser University’s Public Square program and Nesta as part of the Social Innovation Week Canada events.Part of the rationale for the event can be explained by Nesta on its website promoting an earlier Social Frontiers event in the UK:

Despite thriving practitioner networks and a real commitment from policymakers and foundations to support social innovation, empirical and theoretical knowledge of social innovation remains uneven.

Not only is this research base uneven, it’s largely invisible. I choose to use the word invisible because it’s unclear how much research there is as it simply isn’t made visible. Part of the problem, clearly evident at the Vancouver event, is that social innovation appears to be still at a place where it’s busy showing people it exists. This is certainly an important first step, but as this was an event devoted to social innovation research it struck me that most attendees ought to have already been convinced of that.

Missing was language around t-scores, inter-relater reliability, theoretical saturation, cost-benefit analysis, systematic reviews and confidence intervals – the kind of terms you’d expect to hear at a research conference. Instead, words like “impact” and “scale” were thrown out with little data to back them up.

Bring us down to earth to better appreciate the stars

It seems that social innovation is a field that is still in the clouds with possibility and hasn’t turned the lights on bright enough to bring it back down to earth. That’s the unfortunate part of research: it can be a real buzz-kill. Research and evaluation can confirm what it means for something to ‘work’ and forces us to be clear on terms like ‘scale’ and ‘impact’ and this very often will mean that many of the high-profile, well-intentioned initiatives will prove to be less impactful than we hope for.

Yet, this attention to detail and increase in the quality and scope of research will also raise the overall profile of the field and the quality and scope of the social innovations themselves. That is real impact.

By bringing us down to earth with better quality and more sophisticated research presented and discussed in public and with each other we offer the best opportunity for social innovation to truly innovate and, in doing so, reach beyond the clouds and into the stars.

Photo credit: Lightbulb Clouds by MyCatkins used under Creative Commons License. Thanks Mike for sharing!

1 2 3 … 7