When journalist and book author Daniel Pink tweeted the above image the other day it provoked thinking about what real learning means and what it takes to achieve it. We produce enormous amounts of knowledge, yet struggle to put it into use, but we also teach much and learn little because the systems we’ve designed for education and experience don’t match our expressed interest and rhetoric around learning.
In my graduate course on behaviour change I would ask students on the first day why they were taking the class in the first place. Aside from the few students for whom the course was required everyone else was doing it by choice because there were many others to choose from. So why would they choose this one?
The answers would vary, but inevitably I’d hear over and again that students love learning and wanted to understand more about behaviour change, because they were interested in change and some would even say they were good at it and wanted to help others do it.
These are all well-meaning and said in a spirit that I think was honest and true. Except the reality is that it is likely a big, huge lie and one that we all share in its telling.
I would counter with two things:
To illustrate the first point I simply ask people to consider the last conference they went to where there were options on what sessions to attend. How many of the sessions did they attend that featured content that confirmed or gently extended what they already knew versus content that was new? If you’re a health promoter doing community engagement work, sessions on Bayesian modelling for epidemics might offer far more learning than a session on working with diversity in communities (particularly if that is what you already do). Even more, how often do people go to sessions from people they know or have already seen speak? Chances are, many.
One could argue that there are subtleties that a conference presentation might offer on a familiar topic that are worth attending and while I would say that has merit, most learning that has impact is uncomfortable at some level. It extends our thinking, challenges our beliefs, or re-arranges our worldview — in ways small and large.
Many people will say “I love change”, but that is usually in the context that everyone else is changing, not them. When I was the boss and said “things must change” it was very different than when my staff or my boss would say “things must change“. As a behaviour change educator and intervener, I need to be mindful of my own ironies and resistance to change. So should we all.
The same thing goes for knowledge. Academics are famous for ending studies with “more research is needed”. We never seem to have enough knowledge. There are two problems with this idea.
The first is that, in dynamic and evolving environments, we will never have perfect knowledge that fits like a glove, because the contexts are always novel. This isn’t to say that evidence isn’t useful, but ‘good enough’ knowledge might be a more reasonable demand than ‘best evidence’ in many of the situations where complexity is high and so is change. That’s why data gathering techniques like developmental evaluation aren’t attractive to those who need certainty.
But there is another problem with the knowledge quest and that is one of integration. In our efforts to seek more knowledge, are we integrating what we are learning from what we already have? Are we savouring the data we collect, the articles we read, the Tweets and blogs that get forwarded are way?
We quest for more, but should we quest for better?
A newly published paper synthesized research on event horizons on memory and found that shifts in activities around an event — boundaries — can prompt forgetting and recall. We remember transitions between activities, but they also prompt forgetting depending on the mindfulness associated with the act. When we are deluging ourselves with more data, more media, more everything, we are increasing the potential remember rate, but due to the volume of content, I would surmise that we are increasing the forget rate much more. Simply reflect on your high school or undergraduate education and ask yourself if you remember more than you forgot about what you learned.
We are so busy with our search for new knowledge that we interrupt opportunities to learn from what we have.
Returning to the tweet from Dan Pink, it’s worthwhile considering what it means to learn and the systems we have in place to facilitate learning. The tweet links to a discussion of how German companies give their employees five days of off-site continuing education each year. This concept of Bildungsurlaub is a leave designed to allow employees to stretch their thinking and integrate something new. Not only is off-site learning important, but the time associated with integrating material is critical.
A read of the literature on innovation and research shows consistently how time off, quiet time, slow time and down time all contribute to discovery. Robert Scott Root-Bernstein’s brilliant Discovering, Jonah Lehrer’s Imagine, or Steven Johnson’s Where Good Ideas Come From are all books that dive deep into creative production and show that great discoveries and innovations come from having time (with limits) to integrate material to learn. Freedom to create, explore and sit and mindfully reflect are all united concepts in the pursuit of good learning. Not everything requires this, but big concepts and bold ideas do from mathematics to science to social science and philosophy.
Yet, at an organizational and systems level, where is the support for this? Even university faculty (the tenured ones at least) who have generous vacations and sabbaticals are finding themselves crunched for time between the fight for one of the ever-fewer grants, increasing numbers of students and teaching demands, and the added push to ensure knowledge is translated. The image of faculty sitting and reading and thinking is truly an imagination. Most of my colleagues in academia do little of this, because they are out of time.
In the corporate and non-profit world this is worse. Every hour and day is to be accounted for. The idea of sending people off to learn and to think seems anathema to productivity, yet research shows incredible powers associated with taking a break and doing less and not more.
To illustrate the scope of the problem, the University of Toronto holds one of the finest academic library systems in the world and has over 11.5 million books and 5.7 million microform materials. It is one university (of many) in one city. Add in the local Toronto public library system, the network of universities and other libraries it is connected to, local and global bookstores and all the content freely available online that is not part of this system and I challenge anyone working in social innovation or public health to say with conviction that there is a lack of knowledge out there on any important topic. Yes, we don’t know it all, but we don’t do nearly enough with what we do know because there is so much.
We will not read it all nor can we hope to synthesize it all, but we can do much with what we have. Just looking at my own personal library of physical books (not including all I have in the digital realm between books and papers) it’s easy to see that I have more than enough knowledge to tackle most of what I am facing in my work. Most of us do. But do we have the wisdom to use it? Do we have the systems — organizations and personal — that allow us to take the time and soak this in, share our ideas with others, and be mindful of the world around us enough to learn, not just consume?
When we spend as much time creating those spaces, places and systems, then we can answer “yes” to the question of whether we’re serious about learning.
A recent study found looked into the experience of cyberbullying by university professors at the hands of their students. This disturbing phenomenon points to much larger issues beyond mental health promotion and calls into question many of the assumptions we have about the systems we’ve designed to foster education and what it means to be a learner at university.
The university is one of our oldest cultural institutions and its instructors are considered to have among societies most respected jobs, even if not always well compensated. In the past, students often approached their professors with a mixed sense of wonder, respect, curiosity and fear and that, in healthy situations, was reciprocated by faculty to create a space where people could explore ideas, learn, and challenge themselves and others to grow. That relationship has started to change as evidenced by the rise of cyberbullying in the classroom.
A recent article in Macleans Magazine looked at the changing state of the post-secondary classroom and the role of cyberbullying. Only this was not about student victims, but students as the perpetrators against their professors. The effects of cyberbullying are crippling and professors are bearing the burden of having hundreds of eyes watching them, writing about them and writing ‘consumer reviews’ about them in anonymous and sometimes unflattering, inflammatory and questionable terms on sites like RateMyProfessor.com .
Researchers at the University of California, Riverside found that as students age the incidence of face-to-face bullying decreases and cyberbullying increases, which might partly explain why we’re seeing this in university settings when face-to-face bullying goes subterranean. Yet, the notion that professors that are getting bullied by their students belies some other issues that require further investigation, namely those related to the nature of education and the role of students-as-consumers.
If you pay for something, should you not expected to get something rather specific for that experience or product? Aside from some rare experiences of profane/profound personal challenge/punishment like Tough Mudder and its peers or dental work, there are few things we willingly pay for that we don’t derive pleasure from or achieve a very specific (anticipated) outcome.
Education is problematic because we might not know what we’ll get from it going in, what kind of experiences or ideas will emerge, and how our relationship to those experiences will change us. That is its great gift.
Many of us have had profound life changes because of something we experienced through our education and writing as one who has completed four different degree programs and a post-doc I can confidently say that I didn’t receive a lot of what I expected in any of those programs and I am a better person for it. Indeed, if I go to a specific learning event (aside from those focused on a specific technique or technology) I am disappointed if I actually come away with exactly what I expected.
That is part of the point. We don’t know what we don’t know.
But when you start viewing education as a thing that resembles any other market-driven product or services, you begin to focus on learning as a consumable good and your students as customers. In following this line of thought, it makes some sense to focus the delivery of this product on the desires of the consumer.
Increasingly, teachers (of various stripes) are being asked to consider a range of student-related variables in their education. Things like learning styles and preferences are now being woven into classroom instruction and students have come to learn to expect and are increasingly demanding to be taught in ways that match their unique learning preferences and styles. While there is reason to imagine that this approach is useful in stimulating engagement of students in the lessons, there is increasing evidence much of it does little to enhance actual learning. Many of the life lessons we’ve gained that shape what we do and who we are were not delivered in the manner of our choosing, conformed with our preferences and were not desired, expected or enjoyed in the moment. We risk confusing enjoyment with learning; they can be aligned but one isn’t necessary for the other to take place.
However, when we are viewing education from a consumer model, the specific outcomes become part of the contract. If I come to get a degree in X because I believe that the job market demands the skills and knowledge that X brings and I am paying tens of thousands of dollars and spending four or more years acquiring X then I feel entitled to expect all the benefits that X brings. Further, I expect that my journey to acquiring X will be enjoyable, because why would I spend more money than I’ve ever seen on something I don’t enjoy.
Particularly when that is money I don’t have.
In Canada and the United States, student debt rates have dramatically increased. The Canadian Federation of Students note that Canadian’s attending post-secondary education now owe more than $15B to the Canadian federal government (PDF) as part of their student loan program, a number that doesn’t include debt accumulated from borrowing from banks, family, credit cards and other means. In Canada’s largest province, Ontario, the rate of graduate employment has decreased since 2001 and the overall youth unemployment rate continues to be the highest, despite the province having one of the most educated youth population in the country (and arguably, the world). And while Ontario universities continue to promote the fact that education is a better pathway to success, it is a hard pill for many students to swallow when many can’t apply what they trained for and paid for after they graduate.
Satirist John Oliver has an informative, humorous and distressing take on student debt and the state of consumer-oriented education for those who want to learn more.
None of these reasons are excuses for cyberbullying, but it does give a more complicated picture of those that might feel they are entitled to bully others and their reasoning behind it.
What we are seeing is a systems change in the way education is being produced, consumed and experienced. Even the mere fact that we can now reasonably use the language of consumerism to speak to something like education should give us pause and concern. I’ve been involved in post-secondary education for nearly 20 years and there has always been students who simply wanted the ‘piece of paper’ (degree) as a stepping stone to a job and little more than that from their time at school. They were willing to do the work — often the minimum possible — to graduate, but they knew they had to put the effort in to be successful. There was never an expectation that one was entitled to anything from going to school, although that might be changing.
Belgian psychotherapist Paul Verhaeghe has explored the role of identity in market-based economies in his new book What About Me? In the book, Verhaeghe illustrates how we construct our identities as people drawing on the research that reflects (and often contradicts or obscures) the two major perspectives on personality and identity: the person-as-blank-slate and the person as a reflection of the environment. The former perspective assumes we come into the world as we are while the latter assumes the world makes us who we are and both have enormous amount of moral, cultural and evidentiary baggage attached to them.
What Verhaeghe does is point to the ways in which both have elements of truth to them, but that they are mediated by the manner in which we construct the very questions about who we are and what our purpose is. These questions are (for many cultural, historical, economic and political reasons that he elaborates on) frequently market-based. Thus, who we are is defined by what we do, what we own, what we produce, and how we use such things once out into the world and that the value that come with such ways of defining ourselves is considered self-evident. He makes a disturbing and convincing case when one stops to reflect on the way we think about how we think (metacognition + mindfulness) .
When viewed from the perspective of a market, knowledge and its products soon become the goal and not the journey. Indeed, I’ve even written about this in support of an argument for better research-to-action and knowledge translation. Much of the knowledge-to-action discourse is about viewing knowledge as a product even if the more progressive models also view this as part of a process and even more as part of a system. But it is the last part — the system — that we often give the shortest shrift to in our discussions. What Verhaeghe and others are doing is encouraging us to spend more time thinking about this and the potential outcomes that emerge from this line of thinking.
Unless we are willing to talk more about the systems we create to learn, explore and relate we will continue to support Verhaeghe’s thesis and uphold the conditions for the kind of education-as-a-product thinking that I suspect is contributing to students’ changing behaviour with their professors and creating a climate at universities that is toxic instead of inspiring.
Learning styles, technology-driven teaching, and self-direction are all concepts that anyone interested in education should be familiar with, yet the foundations for their adoption into the classroom, lab or boardroom are more suspect than you might think. Today we look at the three urban legends of learning and what that might mean for education, innovation and beyond.
What kind of learner are you? Are you a visual learner perhaps, where you need information presented in a particular visual style to make sense of it? Maybe you need to problem-solve to learn because that’s the way you’ve been told is best for your education.
Perhaps you are a self-directed learner who is one that, when given the right encouragement and tools, will find your way through the muck to the answers and that others just need to get out of the way. With tools like the web and social media, you have the world’s knowledge at your disposal and have little need to be ‘taught’ that stuff, because its online.
And if you’re a digital native (PDF), this is all second nature to you because you’re able to use multiple technologies simultaneously to solve multiple problems together with ease if given the ability to do so. After all, you’ve had these tools your entire life.
An urban legend, urban myth, urban tale, or contemporary legend, is a form of modern folklore consisting of stories that may or may not have been believed by their tellers to be true.
The authors are quick to point out that there are differences in the way people approach material and prefer to learn, but they also illustrate that there is relatively little evidence to support much of the thinking that surrounds these practices, confusing learning preferences for learning outcomes. I’ve commented on this before, noting that too often learning is conflated with interest and enjoyment when they are different things and if we were really serious about it we might change the way we do a great deal many things in life.
In the paper, the authors debunk — or at least question — the evidence that supports the ‘legends’ of digital natives as a type of learner, the presence of specific learning styles and the need to customize learning to suit such styles of learning, and that of the lone self-educator. In each case, the authors present much evidence to challenge these ideas so as not to take them as truths, but hypotheses that have little support for them in practice.
Science has a funny way of revealing truths that we may find uncomfortable or at least challenge our current orthodoxy.
This reminds me of a terrific quote from the movie Men in Black that illustrates the fragility of ideas in the presence and absence of evidence after one of the characters (played by Will Smith) uncovers that aliens were living on earth (in the film) and is consoled by his partner (played by Tommy Lee Jones) about what is known and unknown in the world:
Fifteen hundred years ago everybody knew the Earth was the center of the universe. Five hundred years ago, everybody knew the Earth was flat, and fifteen minutes ago, you knew that humans were alone on this planet. Imagine what you’ll know tomorrow.
One of the problems with learning is that there is a lot to learn and not all of it is the same in content, format and situational utility. Knowledge is not a ‘thing’ in the way that potatoes, shoes, patio furniture, orange juice, and pencils are things where you can have more or less of it and measure the increase, decrease and change in it over time. But we often treat it that way. Further, knowledge is also highly contextualized and combines elements that are stable, emergent, and transformative in new, complex arrangements simultaneously over time. It is a complex adaptive system.
Learning (in practice) resists simple truths.
It’s why we can be taught something over and again and not get it, while other things get picked up quickly within the same person even if the two ‘things’ seem alike. The conditions in which a person might learn are cultural (e.g., exposure to teaching styles at school, classroom designs, educational systems, availability and exposure to technology, life experiences, emphasis on reflective living/practice within society, time to reflect etc..) and psycho-social/biological (e.g., attention, intelligence, social proximity, literacy, cognitive capacity for information processing, ability to engage with others) so to reduce this complex phenomena to a series of statements about technology, preference and perception is highly problematic.
Science doesn’t have all the answers — far from it — but at least it can test out what is consistent and observable over time and build on that. In doing so, it exposes the responsibility we have as educators and learners.
Underpinning the urban legends discussed by Kirschner and van Merriënboer and not discussed is the tendency for these legends to create a hands-off learning systems where workplaces, schools, and social systems are freed from the responsibility of shaping learning experiences and opportunities. It effectively reduces institutional knowledge, wisdom and experience to mere variables in a panoply of info-bites treated as all the same.
It also assumes that design doesn’t matter, which undermines the ability to create spaces and places that optimize learning options for people from diverse circumstances.
This mindset frees organizations from having to give time to learning, provide direction (i.e., do their own homework and set the conditions for effective learning and knowledge integration at the outset). It also frees us up from having to choose, to commit to certain ideas and theories, which means some form of discernment, priority setting, and strategy. That requires work up front and leadership and hard, critical, and time-consuming conversations about what is important, what we value in our work, and what we want to see.
When we assume everyone will just find their way we abdicate that responsibility.
In my home country of Canada, governments have been doing this with social investment for years where the federal government divests interest to the provinces who divest it to cities and towns who divest it to the public (and private) sector, which means our taxes never go up even if the demands on services do and we find that individual citizens are responsible for more of the process of generating collective benefit without the advantage of any scaled system to support resource allocation and deployment throughout society (which is why we have governments in the first place). It also means our services and supports — mostly — get smaller, lesser in quality, more spread thinly, and lose their impact because there isn’t the scaled allocation of resources to support them.
Learning is the same way. We divest our interests in it and before you know it, we learn less and do less with it because we haven’t the cultural capital, traditions or infrastructure to handle it. Universities turn campus life to an online experience. Secondary schools stop or reduce teaching physical education that involves actual physical activity. Scholarly research is reduced to a Google search. Books are given up as learning vehicles because they take too long to read. It goes on.
It’s not that there are no advantages to some of these ideas in some bites, but that we are transforming the entire enterprise with next to no sense of the systems they are operating in, the mission they are to accomplish, a theory of change that is backed up by evidence, or the will to generate the evidence needed to advise and the resources to engage in the sensemaking needed to evaluate that evidence.
It is time to start some serious conversations about systems, science and learning. It would help if we started getting serious about what we mean when we speak of learning, what theories we use to underpin that language and what evidence we have (or need) to understand what those theories mean in practice and for policy. This starts by asking better questions — and lots of them — about learning and its role in our lives and work.
Design thinking and systems thinking are two thinking tools that can help us find and frame these issues. Mindfulness and its ethics associated with non-judgement, open-mindedness, compassion and curiosity are also key tools. The less we judge, the more open we are to asking good questions about what we are seeing that can lead us to getting better answers rather than getting trapped by urban legends.
Doing this within a systems thinking frame also allows us to see how what we learn and where and how we learn is interconnected to better spot areas of leverage and problems in our assumptions.
This might allow us to make many of our urban legends obsolete instead of allowing them to grow like the alligators that live in the sewers of New York City.
Keith Sawyer’s latest post provides a terrific introduction to a new series coming on the design of learning that I’ll be doing on Censemaking in the coming weeks. The importance of the environments — social and physical — that support creativity cannot be understated and Dr Sawyer’s reflections, if taken seriously by educators and academic administrators, could transform the demands that educational institutions pose on their builders, their teachers and their students (who, by the way, are already asking for better spaces to learn). If you’re not familiar with Keith Sawyer’s work, look him up and consider reading some of his many outstanding texts on creativity and innovation; they are top-notch.
I’ve just spent two stimulating days with a small group of architects, university professors, and creativity researchers, at a beautiful old lakeside estate called Marigold Lodge, in Western Michigan. Our goal: To collect everything we know about how to design spaces that maximize learning and foster creativity. With funding from the Sloan Foundation and from the legendary furniture company Herman Miller (which now owns Marigold Lodge), our task is to write a report that will advise university administrations and architecture firms, to guide how new university buildings are designed.
The good news: Very quickly, we came to a consensus. Our group includes artists, furniture designers, architects, musicians, and psychologists. And even with all of that diversity, we agreed on the underlying features of creative learning spaces:
View original post 199 more words
Earlier this week I has the pleasure of attending talks from Bryan Boyer from the Helsinki Design Lab and learning about the remarkable work they are doing in applying design to government and community life in Finland. While the focus of the audience for the talks was on their application of design thinking, I found myself drawn to the issue of evaluation and the discussion around that when it came up.
One of the points raised was that design teams are often working with constraints that emphasize the designed product, rather than its extended outcome, making evaluation a challenge to adequately resource. Evaluation is not a term that frequents discussion on design, but as the moderator of one talk suggested, maybe it should.
I can’t agree more.
It has puzzled me to no end that we have these emergent fields of practice aimed at social good — social finance and social impact investing, social innovation, social benefit (PDF)– that have little built into their culture to assess what kind of influence they are having beyond the basics. Yet, social innovation is rarely about simple basics, it’s influence is likely far larger, for better or worse.
What is the impact being invested in? What is the new thing being created of value? and what is the benefit and for whom? What else happened because we intervened?
Evaluation is often the last thing to go into a program budget (along with knowledge translation and exchange activities) and the first thing to get cut (along with the aforementioned KTE work) when things go wrong or budgets get tightened. Regrettably, our desire to act supersedes our desire to understand the implication of those actions. It is based on a fundamental idea that we know what we are doing and can predict its outcomes.
Yet, with social innovation, we are often doing things for the first time, or combining known elements into an unknown corpus, or repurposing existing knowledge/skills/tools into new settings and situations. This is the innovation part. Novelty is pervasive and with that comes opportunities for learning as well as the potential for us to good as well as harm.
There are reasons beyond product quality and accountability that one should take evaluation and strategic design for social innovation seriously.
Design thinking involves embracing failure (e.g, fail often to succeed sooner is the mantra espoused by product design firm IDEO) as a means of testing ideas and prototyping possible outcomes to generate an ideal fit. This is ideal for ideas and products that can be isolated from their environment safely to measure the variables associated with outcomes, if considered. This works well with benign issues, but can get more problematic when such interventions are aimed at the social sphere.
Unlike technological failures in the lab, innovations involving people do have costs. Clinical intervention trials go through a series of phases — preclinical through five stages to post-testing — to test their impact, gradually and cautiously scaling up with detailed data collection and analysis accompanying each step and its still not perfect. Medical reporter Julia Belluz and I recently discussed this issue with students at the University of Toronto as part of a workshop on evidence and noted that as complexity increases with the subject matter, the ability to rely on controlled studies decreases.
Complexity is typically the space where much of social innovation inhabits.
As the social realm — our communities, organizations and even global enterprises — is our lab, our interventions impact people ‘out of the gate’ and because this occurs in an inherently a complex environment, I argue that the imperative to evaluate and share what is known about what we produce is critical if we are to innovate safely as well as effectively. Alas, we are far from that in social innovation.
There are a series of issues that permeate through the social innovation sector in its current form that require addressing if we are to better understand our impact.
Moving forward, the challenge for social innovation is to find ways to make what it does more accessible to those beyond its current field of practice. Evaluation is one way to do this, but in pursuing such a course, the field needs to create space for evaluation to take place. Interestingly, FSG and the Center for Evaluation Innovation in the U.S. recently delivered a webinar on evaluating social innovation with the principle focus being on developmental evaluation, something I’ve written about at length.
Developmental evaluation is one approach, but as noted in the webinar : an organization needs to be a learning organization for this approach to work.
The question that I am left with is: is social innovation serious about social impact? If it is, how will it know it achieved it without evaluation?
And to echo my previous post: if we believe learning is essential to strategic design we must ask: How serious are we about learning?
Tough questions, but the answers might illuminate the way forward to understanding social impact in social innovation.
* Photo credit from Deviant Art innovation_by_genlau.jpg used under Creative Commons Licence.
Developmental evaluation, a form of real-time evaluation focused on innovation and complexity, is gaining interest and attention with funders, program developers, and social innovators. Yet, it’s popularity is revealing fundamental misunderstandings and misuse of the term that, if left unquestioned, may threaten the advancement of this important approach as a tool to support innovation and resilience.
If you are operating in the social service, health promotion or innovation space it is quite possible that you’ve been hearing about developmental evaluation, an emerging approach to evaluation that is suited for programs operating in highly complex, dynamic conditions.
Developmental evaluation (DE) is an exciting advancement in evaluative and program design thinking because it links those two activities together and creates an ongoing conversation about innovation in real time to facilitate strategic learning about what programs do and how they can evolve wisely. Because it is rooted in both traditional program evaluation theory and methods as well as complexity science it takes a realist approach to evaluation making it fit with the thorny, complex, real-world situations that many programs find themselves inhabiting.
I ought to be excited at seeing DE brought up so often, yet I am often not. Why?
Alas, with rare exception, when I hear someone speak about the developmental evaluation they are involved in I fail to hear any of the indicator terms one would expect from such an evaluation. These include terms like:
DE is following the well-worn path laid by terms like systems thinking, which is getting less useful every day as it starts being referred as any mode of thought that focuses on the bigger context of a program (the system (?) — whatever that is, it’s never elaborated on) even if there is no structure, discipline, method or focus to that thinking that one would expect from true systems thinking. In other words, its thinking about a system without the effort of real systems thinking. Still, people see themselves as systems thinkers as a result.
I hear the term DE being used more frequently in this cavalier manner that I suspect reflects aspiration rather than reality.
This aspiration is likely about wanting to be seen (by themselves and others) as innovative, as adaptive, and participative and as being a true learning organization. DE has the potential to support all of this, but to accomplish these things requires an enormous amount of commitment. It is not for the faint of heart, the rigid and inflexible, the traditionalists, or those who have little tolerance for risk.
Doing DE requires that you set up a system for collecting, sharing, sensemaking, and designing-with data. It means being willing to — and competent enough to know how to — adapt your evaluation design and your programs themselves in measured, appropriate ways.
DE is about discipline, not precision. Too often, I see quests to get a beautiful, elegant design to fit the ‘social messes‘ that represent the programs under evaluation only to do what Russell Ackoff calls “the wrong things, righter” because they apply a standard, rigid method to a slippery, complex problem.
Maybe we need to build a better brand for DE.
Why does this fuss about the way people use the term DE matter? Is this not some academic rant based on a sense of ‘preciousness’ of a term? Who cares what we call it?
This matters because the programs that use and can benefit from DE matter. If its just gathering some loose data, slapping it together and saying its an evaluation and knowing that nothing will ever be done with it, then maybe its OK (actually, that’s not OK either — but let’s pretend here for the sake of the point). When real program decisions are made, jobs are kept or lost, communities are strengthened or weakened, and the energy and creative talents of those involved is put to the test because of evaluation and its products, the details matter a great deal.
If DE promises a means to critically, mindfully and thoroughly support learning and innovation than it needs to keep that promise. But that promise can only be kept if what we call DE is not something else.
That ‘something else’ is often a form of utilization-focused evaluation, or maybe participatory evaluation or it might simply be a traditional evaluation model dressed up with words like ‘complexity’ and ‘innovation’ that have no real meaning. (When was the last time you heard someone openly question what someone meant by those terms?)
We take such terms as given and for granted and make enormous assumptions about what they mean that are not always supported). There is nothing wrong with any of these methods if they are appropriate, but too often I see mis-matches between the problem and the evaluative thinking and practice tools used to address them. DE is new, sexy and a sure sign of innovation to some, which is why it is often picked.
Yet, it’s like saying “I need a 3-D printer” when you’re looking to fix a pipe on your sink instead of a wrench, because that’s the latest tool innovation and wrenches are “last year’s” tool. It makes no sense. Yet, it’s done all the time.
There is something alluring about the mysterious. Innovation, design and systems thinking all have elements of mystery to them, which allows for obfuscation, confusion and well-intentioned errors in judgement depending on who and what is being discussed in relation to those terms.
I’ve started seeing recent university graduates claiming to be developmental evaluators who have almost no concept of complexity, service design, and have completed just a single course in program evaluation. I’m seeing traditional organizations recruit and hire for developmental evaluation without making any adjustments to their expectations, modes of operating, or timelines from the status quo and still expecting results that could only come from DE. It’s as I’ve written before and that Winston Churchill once said:
I am always ready to learn, but I don’t always like being taught
Many programs are not even primed to learn, let alone being taught.
So what should someone look for in DE and those who practice it? What are some questions those seeking DE support ask of themselves?
Of both evaluators and program stakeholders
DE is not a magic bullet, but it can be a very powerful ally to programs who are operating in domains of high complexity and require innovation to adapt, thrive and build resilience. It is an important job and a very formidable challenge with great potential benefits to those willing to dive into it competently. It is for these reasons that it is worth doing and doing well.
In order for us to get there this means taking DE seriously and the demands it puts on us, the requirements for all involved, and the need to be clear in our language lest we let the not-good-enough be the enemy of the great.
Photo credit: Highline Chairs by the author
Innovators transform the world around them in big and small ways and while a successful effort can be lauded by pundits, politicians and the public there is a long road to making change happen. That road is also a lonely one and doing things different means more than just innovating and experiencing what it means to be resilient firsthand.
Clayton Christensen’s seminal book The Innovator’s Dilemma has been one of the leading sources of thinking-inspriation in business and social innovation. The book reflects the challenges with those seeking to introduce new ideas, products or services into established markets (or ecosystems) in the aim of addressing both people’s present and future needs.
These innovators — change-makers — risk disrupting the very markets they seek to influence bringing uncertainty for everyone. What innovators bet on is that the changes they introduce will have wide-ranging, positive benefits even if they don’t fully know what those are before setting out. Not surprisingly, these efforts are not always welcome at first and the road toward understanding and acceptance is a long one.
Innovation means doing something new and while we like to talk about new, many don’t actually like doing ‘new’ because that means questioning and changing things. Indeed, change — profound change — in thinking is often vigorously opposed as Albert Einstein pointed out in a quote that is paraphrased as:
Great spirits have always encountered violent opposition from mediocre minds
This opposition is a challenge for anyone, but the long slog towards innovation is not only hard on the spirit, it is often a lonely path.
To innovate means to lead through ideas and products. We live in a society that admires and elevates the innovators. No better or perhaps inspiring example is the 1997 advertisement from Apple as part of the Think Different campaign in the 1990’s.
What is missing from the platitudes, plaudits and celebrations is the quiet, often lonely, life away from the attention that successful innovations bring (nevermind those that are not deemed successful). To innovate is to lead and to lead is often to be lonely by definition because there are few leading and more following. This leadership by thought or action is often what makes leaders appear creative, innovative and — as Seth Godin affectionately calls being weird. A study discussed in the Harvard Business Review and dissected in Forbes pointed to high rates of loneliness among those at the CEO level, which is among those who “made it”. Consider those who haven’t yet “made it”, who haven’t had their idea “succeed” or take off and it might feel even more lonely.
At a recent workshop I conducted a participant expressed publicly a sense of gratitude for simply having the opportunity to connect with others who were simply open to seeing the world in the same way that they were. In hosting a learning workshop for social innovators a positive byproduct was that attendees who might have been isolated in their activities and thinking in one context could come together in another.
Innovation, because it is new, means that innovators have few peers available to directly commiserate with and may need to find ways to connect on idea, method, philosophy or role, but rarely something direct. That requires extra work in the search and more effort to connect in the finding, which takes time and energy — two things innovators are often short of.
But that doesn’t diminish the value and importance of time and energy and directing it towards efforts to reduce isolation.
Paul Born, Director of the Tamarack Institute for Community Engagement, recently published a book on creating deep community connections as a necessary means of fostering transformative change. Born offers four pillars to a deepening community are: 1) sharing stories, 2) taking the time to enjoy one another, 3) taking care of one another, and 4) working together towards a bigger social goal.
While there is little to argue with here, these pillars rest on the ability to locate, co-locate and create the space to share, enjoy, care and collaborate in the first place. For many innovators this is the hardest part. Where do we find the others like ourselves and how do begin to frame this journey?
There is a reason that innovators have flocked to tools like the Business Model Canvas and the Lean Startup method to help people define, refine and develop their products and mission. It’s easy to point to firms like Apple as examples of clear-focused innovators now, but 20 or 30 years ago it wasn’t so clear. Apple’s overall mission and vision are easy to see lived out in hindsight, not at the beginning. A read of Steve Jobs’ biography illustrates how often his way of approaching the world clashed with nearly everyone and everything and how difficult life was for him.
But Steve Jobs happened to be challenging the world in a place that would come to be known as Silicon Valley. For the last thirty years the San Francisco bay area has been a spark for creative thinking and innovation, one of many hotbeds of business and cultural transformation that Richard Florida documented as home of the Creative Class(es). But not all innovation takes place in these centres and even within such centres it might be hard to connect when an idea is ill-formed or new. We lose out when innovation is only done in certain places by certain people.
(Social) innovators are part of a diffuse and sometimes lost tribe.
If you look at the language that we frame innovation we reveal many of the problems with not only our ideas, but what we do with them. As mentioned in previous posts, we privilege terms like creativity, but often ignore craft. We aspire to be learners, but often don’t like real learning. We tout the role of failure in design and innovation, yet our overloaded cultural baggage attached to the term prevents us from really failing (or asking such tepid questions we don’t really stretch ourselves).
Having access to social media and electronic communities offer a lot and something we didn’t have before, but its very difficult to forge strong, connective bonds mediated through a technological interface. Technology is good at initiating superficial connections or maintaining deeper connections, but not so good at creating deep connections. Those deeper connections as Paul Born points out are the things that sustain us and allow us to do our best work.
The dilemma is how to allocate time and resources in cultivating uniqueness, depth and connecting to similar innovators when that pool is small or integrating more with those in the convention system. Of course innovators need to relate to both groups at some level because an innovation doesn’t grow if we only connect to ‘true believers’, but at different stages it matters how we’re allocating our time, energy and enthusiasm particularly along that journey up Mt. Isolation.
There is no ready answer for this problem. Indeed, the lonely path to being different, weird or constructively challenge the harmful or less effective parts of the status quo may be one of the most wicked ones innovators face.
For those interested in social innovation there are a few examples for those who want to find peers and connect:
Born, P. (2014). Deepening Community: Finding Joy Together in Chaotic Times (p. 216). San Francisco, CA: Berrett-Koehler Publishers.
Wheatley, M. (2006). Leadership and the New Science: Discovering Order in a Chaotic World (3rd. ed., p. 218). San Francisco, CA: Berrett-Koehler Publishers.
Wheatley, M. (2007). Finding Our Way: Leadership for an Uncertain Time (p. 300). San Francisco, CA: Berrett-Koehler Publishers.
Wheatley, M. (2010). Perseverance (p. 168). San Francisco, CA: Berrett-Koehler Publishers.
Building a mindful organization requires a sense of understanding where you are, where you came from and where you’re going. The first step is grounding your organization and learning about — and (re)creating your point of origin.
In 1806, what became the city of Detroit, Michigan was first designed and conceived. Unlike other cities, Detroit clearly defined itself by its geography explicitly and sought to build everything around a single point of origin. It is for this reason that there are roads such as 6 mile, 7 mile, and the popularized 8 mile to mark actual distances from this central point in the city. The marker and monument photographed above serves as a reminder of Detroit’s history and offers a place to gain perspective on its present and increasingly its future as it undergoes a serious remake.
As a city, Detroit is seeking to re-ground itself as it looks forward by looking to the past and present at the same time. It is a city looking to recapture the entrepreneurial spirit that made it into one of the world’s great manufacturing centres in the early to mid 1900’s in a way that is more socially integrative than it was before.
In a previous post, I highlighted eight stages to creating a mindful organization, one that is aware and conscious of itself and the systems its a part of. Grounding is the first step.
Below is a look at the different ways the term ground can be used. It’s important to note connection between the solidity of the terms, perception, place, and connection. Grounding means all of these.
At the heart of grounding is mindfulness: being fully aware of one’s self and setting in the context of the present moment. While mindfulness is about paying attention to the present moment, but not at the expense of the past, nor is it about inattention to intention moving forward. It is about being aware of the moment-by-moment connections between the person or organization and what is being experienced. Mindfulness has many paradoxical elements to it, which is one reason it works well with complexity, which has its own paradoxes as well.
Mindfulness is a means of establishing that connection to our ground — whatever that might be. Grounding might be in community, fields of practice, time, markets or populations of interest or engagement and most likely is some combination of these.
Do you know the ground your organization is stationed? In answering that question it is worth asking some key questions:
1. Do you have a detailed, articulated strategy for your work linked to some clear purpose? (In other words: do you know what the point of your work is and what you’re trying to accomplish?). The role of intention in mindful practice is enormous and being clear on what is intended from the work and being aware of that intention while the work is being performed is a key factor.
2. Do you have a means of matching your intentions and strategy to the work that you are doing? Many organizations have goals and visions, but no ongoing monitoring and evaluation methods to assess whether or not they are actually doing this work. This will be discussed in a future post, but it is worthwhile to ask early whether there is a means of assess what work you are actually doing?
3. Is your ground solid? This question looks at the logic of your enterprise. Whether you are a for-profit, not-for-profit, charity or some other enterprise there needs to be a solid connection between what you do, your products and services, and markets, partners, resources and income streams you have available. The Business Model Canvas is a tool that can help expose the logic — and the gaps in it — of your organization’s work. The canvas was an crowd-consulted, co-created initiative led by Alex Osterwalder that worked through a series of iterations to create a simple, easy-to-use framework for linking the various components of your organization’s mission together. By being aware of the logic of your business you’re able to be mindful of how those activities connect to your purpose, intentions and aspirations.
4. What will solidify your ground (i.e.: what has value?)? Dave Snowden from Cognitive Edge is critical of the approach of naming values as an organization. Snowden asserts that once values are named, they are summarily ignored. Value statements are useless unless they truly express some form of value, that is reflect where investment, decisions and actions of the organizations are placed. If one is acting with intention and a clear grounding, then these values become evident and the need to express them seems moot. A further problem is that the social pressures to name values that are acceptable (rather than consistent with practice) mean that we often find organizations with operationally meaningless values. If you don’t know what values like “inclusiveness”, “respect for diversity”, “participation” and “learning organizations” really mean in practice as well as intention, they don’t serve your enterprise.
5. How committed are you to standing your ground? Wherever your organization chooses to stand, that is the system that it will see. Once grounded, many path dependencies are set in motion, which will determine how aligned what you do, say, seek, and find in the future. In complex systems it is critical to have some flexibility in boundaries, however they must be set somewhere. Consider what it is that you value and whether you are committed as an organization to doing what you say and aspire towards. Visionary companies come from alignment between what the leaders say and what everyone does.
It’s never too late to ask yourself these questions whether you’re starting up or seeking to re-establish yourself or create a new path forward.
Just like Detroit, there is always a chance for re-birth. And just like Detroit, you’ll have that point to launch from and look back at to help you wayfind as you engage in complexity through your work. As we will see, knowing where you start from will help determine where you go.
For more information on this process of grounding and what it could look like for your organization contact CENSE Research + Design
Photo credits: Cameron Norman
In 2009 Censemaking was launched as a platform to explore issues in complexity and ways we can make sense of it to design for better futures and a sustainable world. After 300 posts it has become evident that there is much more to write as we see ever-new crises from complexity and ever-greater design opportunities to deal with it all.
As I was reflecting on what to write for my 300th post for Censemaking I found myself — as I often do — drawing some connections between disparate experiences as I started my daily reading and listening. Within moments of sitting at the table with materials, turning on the radio, and scanning online I found the following semi-related stories:
Within each of these categories is a reflection of some form of crisis — an unstable situation affecting many people — particularly the worlds of science, journalism, politics, publishing, policy, and design.
This motley collection of tidbits loosely connects science, design, public affairs, knowledge translation and communication, and the complexity that comes when they intersect. It seems fitting that this greeted me as I sat down to write post #300.
The Censemaking name is a riff on both the name of my social innovation consultancy (CENSE Research + Design) and the term sensemaking that is a trans-disciplinary field / practice of making meaning from complex, divergent data points and experience (which is what I help my clients, collaborators and students do). It has been a vehicle that has allowed me the freedom and pleasure to explore the knotty intersections of these disparate areas of practice and scholarship that don’t fall under any particular umbrella, yet are things that are wrestled with in health promotion, industry, publishing and media, social services, policymaking, the military and social enterprise (to speak of a few).
And as I often do, I find the strangest threads are often the most useful in understanding complexity and our world.
That I would even put those four words above together above might have already turned you off, but stick with me. While the Miley Cyrus reference in the above list of media notes might be the most disparate of them all, complexity science teaches us that there is often gold in looking at weak signals and Miley Cyrus might be the best example of that in this list.
In a week where the once Hannah Montana actor and singer has garnered enormous attention in the media for her moves, her behaviour and her attitude at last weekends’ MTV Video Music Awards, particularly her performance with singer Robin Thicke it seems there is little left to discuss. Or not.
Others jumped on the fact that it was Miley Cyrus who got all the flack for the acts performed while Robin Thicke, a married father, gets away with little public condemnation despite being the main performer of a song with a deeply sexist, near misogynistic lyrics, message and related video.
The Belle Jar Blog points to how Miley’s appropriation of black culture is a racist and patriarchal act that deserved the real condemnation as much as any sexual act that it was associated with, something that only adds to the slut-shaming says the Washington Post who nevertheless seek to question the fuss.
Reading and contemplating Miley’s performance could at once be seen as juvenile, offensive, and racist, while also represent shrewd marketing, behaviour not inconsistent with previous VMA awards and its time-honoured practice of female sexualization to draw eyeballs (and commentary) , and a situation reflective of a woman growing up at a time and place where the lines between activities rooted in a particular racial, ethnic, geographic, socio-demographic heritage are — no pun intended — quite blurred and may be genuinely obscured to her.
This is a rather banal, yet clear example of the way complexity and wicked problems rise up from an interconnected, multimedia, 24/7, global culture of communication that we’ve created for ourselves. Miley is at once a perpetrator, a victim and a bystander all at the same time. She is a social construction and a real person who is accountable for what she says and does (but to whom and for what?). That is complexity in the modern age of public engagement, expression and media.
It’s one example. We are facing similar thorny, hairy issues with vaccination, big data, chronic disease, community planning, social media, journalism’s independence and viability, educational policy and the structure of learning, private-public partnerships for social benefit and beyond. There is no simple answer or simple problem. Sensemaking is a way to understand complexity and then determine what it means.
When you know better you do better – Maya Angelou
Better knowing is the biggest step towards better doing. Sensemaking complexity means looking broadly and deeply, consulting widely and taking the time to reflect on what it means. Being mindful of our time, and its disruption, is critical.
What comes from that is the possibility not just to understand our world, but to shape it into something we deem to be better for us all. This motivation to shape is what makes us human. We are the one species that creates for enjoyment, expression, and practical need. We are makers and designers and often both at the same time.
Design is the conscious intent to shape things while design thinking is a means of engaging complexity to foster more effective designs. We cannot control complexity, but we can design for it (PDF) and work with the emergent patterns it produces. This process of design for emergence and developmental design, which brings together sensemaking, structured feedback through ongoing developmental evaluation, and foresight methods allows us to take account of complexity without letting it take hold of us. It helps us make the world we want, not just accept the world we get.
Thank you to all of my readers — the tens of thousands of people who have come to Censemaking since it started and the many of you who come regularly and share it with the world. In a world of attention scarcity, I am deeply appreciative of you spending some of your time with my work.
Create your own audiences. I am honoured to have been able to create the audience I have; thank you for being a part of it. I hope to continue to provide you with things to contemplate and help you make sense of.
I look forward to the next 300 posts and finding new ways to navigate and contemplate complexity and design for innovation.
Image: Thinkstock used under license & Cameron Norman
Critics of design thinking suggest that it neglects the craft of products while advocates suggest that it extends itself beyond the traditional constraints of design’s focus on the brief. What separates the two are the implications associated with making something and the question: can we be good designer thinkers without being good design makers?
A review of the literature and discussions on design thinking finds a great deal of debate on whether it is a fad, a source of innovation salvation, or whether it is a term that fails to take the practice of design seriously. While prototyping — and particularly rapid prototyping — is emphasized there is little attention to the manner in which that object is crafted. There are no standards of practice for design thinking and the myriad settings in which it could be applied — everything from business to education to the military to healthcare — indicate that there is unlikely to be a single model that fits. But should there be some form of standards?
While design thinking encourages prototyping there is remarkably little in the literature on the elements of design that focus on the made product. Unlike design where there is at least some sense of what makes a product good or not, there are no standards for what ought to emerge from design thinking. Dieter Rams, among the most vocal critics of the term design thinking, has written 10 principles for good design that can be applied to a designed product. These principles include a focus on innovation, sustainability, aesthetics, and usability.
These principles can be debated, but they at least offer something others can comment on or use as foil for critique. Design thinking lacks the same correlate. Is that a good (or necessary) thing?
Unlike design itself, design thinking is not tied to a particular product profile; it can be used to create physical products as easily as policies and programs. Design thinking is a process that is centred largely on complex, ambiguous problems where success has no pre-defined outcome and the journey has no set pathway. It is for this reason that concepts like best practices are inappropriate for use in design thinking and complex problem solving. Design thinking offers a great deal of conceptual freedom without the pressure to produce a specific outcome that might be proscribed by a design brief.
Yet, design thinking is not design. Certainly many designers draw on design thinking in their work, but there is no requirement to create products using that way of approaching design problems. Likewise, there is little demand for design thinking to produce products that would fit what Dieter Rams suggests are hallmark features of good design. Indeed, we can use design thinking to create many possible futures without a requirement to actually manifest any of them.
Design requires an outcome and one that can be judged by a client (or customer or user or donor) as satisfactory, exemplary or otherwise. While what is considered ‘good design’ might be debated, there is little debate that if a client does not like what is produced that product it is a failure on some level*. Yet, if design thinking produces a product (a design?), what is the source of the excellence or failure? And does it matter if anything is produced at all?
Herein lies a fundamental dilemma of design and design thinking: how do we know when we are doing good or great work?
Can we have good design thinking and poor design making?
Roger Martin, writing in Design Observer, highlighted how design thinking was being applied to the US Army through the adaptation of its Field Operations Manual. This new version was based on principles of complexity science and systems thinking, which encourage adaptive, responsive unit actions rather than relying solely on top-down directives. It was an innovative step and design thinking helped contribute to the development of this new Field Manual.
On discussing the process of developing the new manual (FM-05) Martin writes:
In the end, FM5-0 defines design as “a methodology for applying critical and creative thinking to understand, visualize, and describe complex, ill-structured problems and develop approaches to solve them” (Page 3.1), which is a pretty good definition of design. Ancker and Flynn go on to argue that design “underpins the exercise of battle command within the operations process, guiding the iterative and often cyclic application of understanding, visualizing, and describing” and that it should be “practiced continuously throughout the operations process.” (p. 15-16)
The manual’s development involved design thinking and the process in which it is enacted is based on applying design thinking to field operations. As unseemly as it may be to some, the US Army’s application of design thinking is notable and something that can be learned from. But what is the outcome?
Does a design thinking soldier become better at killing their enemy? Or does their empathy for the situation — their colleagues, opponents and neutral parties — increase their sensitivities to the multiplicities of combat and treat it as a wicked problem? What is the outcome in which design thinking is contributing to and how can that be evaluated in its myriad consequences intended or otherwise? In the case of the US Army it might not be so clear.
One of terms conspicuously absent from the dialogue on design thinking is craft. In a series of interviews with professionals doing design thinking it was noted that those trained as designers — makers — often referred to ‘craft’ and ‘materials’ in describing design thinking. Those who were not designers, did not**. No assessment can be made about the quality of the design thinking that each participant did (that was out of scope of the study), but it is interesting to note how concepts traditionally associated with making — craft and materials and studios — do not have much parallel discussion in design thinking.
One reason to consider craft is that it can be assessed with at least some independence. There is an ability to judge the quality of materials and the product integrity associated with a designed object according to some standards that can be applied somewhat consistently — if imperfectly — from reviewer to reviewer. For programs and policies, this could be done by looking at research evidence or through developmental evaluation of those products. Developmental design, an approach I’ve written about before, could be the means in which evaluation data, rapid prototyping, design excellence and evidence could come together to potentially create more robust design thinking products.
We have little correlates with design thinking assessment.
The danger with looking at evaluation and design thinking is falling into the trap of devising and applying rigid metrics, best practices and the like to domains of complexity (and where design thinking resides) where they tend to fail catastrophically. Yet, there is an equal danger that by not aspiring to vision what great design thinking looks like we produce results that not only fail (which is often a necessary and positive step in innovation if there is learning from it), but are true failures in the sense that they don’t produce excellent products. It is indeed possible to create highly collaborative, design thinking-inspired programs, policies and products that are dull, ineffective and uninspiring.
Where we go and how we get there is a problem for design and design thinking. Applying them both to each other might be a way to create the very products we seek.
* It is interesting to note that Finnish designer Alvar Aalto’s 1933 three-legged children’s stool has been considered both a design flop from a technical standpoint (it’s unstable given its three legs) and one of the biggest commercial successes for Artek, its manufacturer.
** The analysis of the findings of the project are still ongoing. Updates and results will be published on the Design Thinking Foundations project site in the coming months, where this post will be re-published.