Tag: design thinking

behaviour changecomplexityeducation & learningpsychologysystems science

Asking Better and More Beautiful Questions

Why__by_WhiteSpeed

Beautiful answers require beautiful (and better) questions and Warren Berger’s new book looks at this very phenomenon of inquiry and asks: What does it mean to ask better questions and what does that mean for the answers we seek and receive?  

Warren Berger recently published  A More Beautiful Question, a book looking at something we take for granted and yet is the foundational building block for all great designs and innovations: the question.

Perhaps more specifically, Berger is looking at hundreds of questions as he delves into the process of questioning, the kind of questions that lead to provocative and insightful answers, and the habits of good questioning that make for sustained innovation over time.  Berger is well suited to this inquiry having penned the book Glimmer, which profiled designer Bruce Mau and explored the concept of design thinking in great detail.

Asking good questions is perhaps the (often unstated, missed and neglected) foundation of what design thinking is all about and seeing that design is the foundation of innovation it therefore means that questioning is at that foundation, too. This is important stuff.

Finding the right problem by asking better questions

A look at any bookstore, blog roll, or journal dealing with the topic of innovation and you’ll inevitably find the word “creativity” used a lot. Creativity — the act and process of creating things — is highly correlated with the questions that spur the creation in the first place. Education professor J.W. Getzels did some of the earliest research on creativity and questioning (which is interestly absent from Berger’s book) and found that those who took more time to find the best problem to solve – and thus, asked better and deeper questions of their world and subject matter — came up with more creative ideas than those who dove quickly into solving the problem as they initially saw it.

The simple take-away is: 

At the root of an answer is a question – J.W. Getzels

The better the question, the better the answer.

In complexity terms, the questions asked often create the path dependencies that entrench practices that come after it. So by asking better or ‘more beautiful’ questions and giving that attention we are not only doing ourselves a service, but are acting more ethical as well. This ethical foundation is what underlies mindfulness practice. Jon Kabat Zinn has written extensively on the importance of grounding oneself to ask better questions of the world, something that I’ve done through CENSE Research + Design in developing a mindful organization model.

In his 2004 presidential address to the Canadian Psychological Association Pat O’Neill looked at how sub-fields like community psychology changed the nature of how many “problems” in psychology were framed at the outset. Issues like poverty, drug addiction and unemployment were often (and still are in many domains) framed as personal, moral failings or just bad choices. By asking different questions of these problems, community psychologists were able to see how social policies, neighbourhood structures, social networks, and historical social exclusion — all systems issues — factor in to frame and constrain individual’s choices and risk behaviours. Suddenly, what had been framed as a personal problem, became a shared one that we all had at least some stake in.

It is this thinking that has led to greater awareness of how social change is inextricably linked to systems change and why we need to understand systems at the individual, organizational, community and societal level if we wish to address many of our social problems. Asking systems questions is asking different, sometimes more beautiful questions that get at the root of problems and inspire social innovation.

Finding the beautiful question

In his book, Berger finds that those best equipped to solve or at least address these big wicked questions in business, philanthropy and social innovation are those that ask ‘beautiful questions’ and do it often. Berger cites studies that have shown a clear relationship between success in leadership and a propensity to ask good questions. Asking good questions however takes time and the willingness to take time to question, think and question some more is another stand-out feature of these successful leaders.

It is why good questioning is also a leadership issue. Effective leaders often take the time needed to fully process the most important decisions to form what Gary Hamel and C.K. Pralahad refer to as strategic intent. Psychologist Daniel Goleman recently summarized the research linking mindfulness to focus and leadership, showing how leaders are able to better focus on what they do by being mindful. This mindful attention clears away much of the cognitive clutter to enable better question finding and asking.

Berger shows that finding the question requires some persistence. Good questioners are able to live with not having an answer or even the right question for a while. They have great patience. That ability to stand back and think, see, reflect and think some more while prototyping questions is what separates those who ask the better questions from those who don’t.

Creative collisions also helps. By mixing up ideas and connections with others, good questioners give themselves the raw material to work with. However, many of the best questioners that Berger spoke to also advocated for the need for some solitude and time to process these ideas and questions on their own. This mix of collaboration, collision, and independence is a key factor in developing the beautiful idea.

Designing better question-making

What jumped out at me in this book was how little support most organizations offer themselves for asking better, beautiful questions. Berger noted that the need for ‘serial mastery’ and constant learning is a staple of the new work environment, which should lend itself to question asking. However, if organizations are unwilling or unable to provide time for reflection, training, knowledge integration and ongoing discovery through better questions how likely is it that the workforce is going to respond to this need for new skills?

Are organizations willing to invest in a culture of inquiry? Are organizations able to make the leap from knowing things to asking things? How many public sector, non-profit, social and health service organizations (let alone industry groups) would be willing to follow companies like Google who create space — literally and figuratively — for questioning? These are some of the questions I asked myself as I read Berger’s book.

These are design questions. Berger notes how Google’s founders, Larry Page and Sergey Brin, were both Montessori school graduates. The Montessori system of education is based on question asking and Google is run as an organization largely framed around questions (and queries as noted by the very notion of “googling” something). Google has been designed to support better questions in its literal architecture of its software, its hardware, its office space, and the ‘20 per cent time‘ they offer employees to explore questions they have and projects that are of personal importance to them.

True to the idea of questions being worthy of paying attention to, Warren Berger’s book is filled with them including some answers. I liked the book and believe that he has tapped into something very big. Whether or not organizations and leaders will be inspired to ask better questions from this or simply try to find better answers in the processes they have is perhaps the big question next.

On a related note, March 14th has been dubbed Question Day by Berger and his colleagues at the Right Question Institute, a non-profit organization that provides support for teachers and students to ask better questions in school as a foundation for a lifetime of learning. 

References:

Berger, W. (2009). Glimmer: How Design Can Transform Your Life, Your Business, and Maybe Even the World. Toronto, ON: Random House Canada.

Berger, W. (2014). A more beautiful question: The power of inquiry to spark breakthrough ideas. New York, N.Y.: Bloomsbury Publishing.

Chand, I., & Runco, M. A. (1993). Problem finding skills as components in the creative process. Personality and Individual Differences, 14(1), 155–162.

Getzels, J. W. (1979). Problem Finding: a Theoretical Note. Cognitive Science, 3(2), 167–172. doi:10.1207/s15516709cog0302_4

Getzels, J. W. (1980). Problem Finding and Human Thought. The Educational Forum, 44(2), 243–244.

Goleman, D. (2013). Focus: The Hidden Driver of Excellence. New York, N.Y.: Harper Collins.

O’Neill, P. (2005). The ethics of problem definition. Canadian Psychology, 46(13-22).

Photo credit: Why? by Whitespeed via DeviantArt

art & designdesign thinkingfood systemssocial systemssystems thinking

“If You Build It..”: A Reflection on A Social Innovation Story

If You Build it is documentary about a social innovation project aimed at cultivating design skills with youth to tackle education and social issues in a economically challenged community in North Carolina. The well-intentioned, well-developed story is not unfamiliar to those interested in social innovation, but while inspiring to some these stories mask bigger questions about the viability, opportunity and underlying systems issues that factor into the true impact of these initiatives. (Note: Spoiler alert > this essay will discuss the film and plotlines, yet hopefully won’t dissuade you from seeing a good film). 

Last week I had the opportunity to see Patrick Creadon‘s terrific new documentary “If You Build It” at the Bloor Hot Docs Cinema in Toronto as part of the monthly Doc Soup screening series. It was a great night of film, discussion and popcorn that inspired more than just commentary about the film, but the larger story of social innovation that the film contributes to.

If You Build It is the story of the Project H studio that was developed in in Bertie County (click here for a film outline) and run for two years by Emily Pilloton and her partner Matthew Miller. To learn more about the start of the story and the philosophy behind Project H, Emily’s TED talk is worth the watch:

It’s largely a good-news kind of story of how design can make a difference to the lives of young people and potentially do good for a community at the same time. While it made for a great doc and some inspiring moments, the film prompted thoughts about what goes on beyond the narrative posed by the characters, the community and those seeking to innovate through design and education.

Going beyond the story

Stories are often so enjoyable because they can be told in multiple ways with emphasis placed on different aspects of the plot, the characters and the outcomes. For this reason, they are both engaging and limiting as tools for making decisions and assessing impact of social interventions. It’s why ‘success stories’ are problematic when left on their own.

One of the notable points that was raised in the film is that the cost of the program was $150,000 (US), which was down from the original budget of $230,000 because Emily and Matthew (and later on, a close friend who helped out in the final few months) all didn’t take a salary. This was funded off of grants. Three trained designers and builders worked to teach students, build a farmers market, and administer the program for no cost at all.

The film mentions that the main characters — Matthew and Emily — live off credit, savings and grants (presumably additional ones?) to live off of. While this level of commitment to the idea of the Bertie County project is admirable, it’s also not a model that many can follow. Without knowing anything about their family support, savings or debt levels, the idea of coming out of school and working for free for two years is out of reach of most young, qualified designers of any discipline. It also — as we see in the film — not helpful to the larger cause as it allows Bertie County yo abdicates responsibility for the project and lessens their sense of ownership over the outcomes.

One segment of If You Build It looks back on Matthew’s earlier efforts to apply what he learned at school to provide a home for a family in Detroit, free of charge in 2007. Matthew built it himself and gave it to a family with the sole condition that they pay the utilities and electricity bills, which amounted to less than this family was paying in just rent at the time. That part of the story ends when Matthew returns to the home a few years later to find the entire inside gutted and deserted long after having to evict that original family 9 months after they took possession when they failed to pay even a single bill as agreed.

From Bertie County to Detroit and back

The Detroit housing experience is a sad story and there is a lot of context we don’t get in the film, but two lessons taught from that experience are repeated  in the story in Bertie County. In both cases, we see something offered that wasn’t necessarily asked for, with no up-front commitment of investment and the influence that the larger system has on the outcomes.

In Detroit, a family was offered a house, yet they were transplanted into a neighbourhood that is (like many in Detroit) sparsely populated, depressed, and without much infrastructure to enable a family to make the house a home easily. Detroit is still largely a city devoted to the automobile and there are wide swaths of the city where there is one usable home on every three or four lots. It’s hard to conceive of that as a neighbourhood. Images like the one taken below are still common in many parts of the city even though it is going through a notable re-energizing shift.

Rebirth of Detroit

Rebirth of Detroit

In the case of Bertie County, the same pattern repeats in a different form. The school district gets an entire program for free even to the point of refusing to pay for salaries for the staff (Emily and Matthew) over two years, after the initial year ended with the building of a brand-new farmers market pavillion that was fully funded by Project H and its grants.

The hypothesis ventured by Patrick Creadon when he spoke to the Doc Soup audience in Toronto was that there was some resentment at the project (having been initiated by a change-pushing school superintendent who was let go at the film’s start and was the one who brought Emily and Matthew to the community) and by some entrenched beliefs about education and the way things were done in that community.

Systems + Change = Systems Change (?)

There is a remarkably romantic view of how change happens in social systems. Bertie County received a great deal without providing much in the way of support. While the Studio H project had some community cheerleaders like the mayor and a few citizens, it appeared from the film that the community – and school board — was largely disengaged from the activities at Studio H. This invokes memories of Hart’s Ladder of Participation, (PDF) which is applied to youth, but works for communities, too. When there is a failure to truly collaborate, the ownership of the problem and solution are not shared.

At no time in the film do you get a sense of a shared ownership of the problem and solution between Studio H, the school board, and the community. While the ideas were rooted in design research, the community wasn’t invested — literally — in solving their problems (through design, at least). It represents a falsehood of design research that says you can understand a community’s needs and address it successfully through simple observation, interviews and data gathering.

Real, deep research is hard. It requires understanding not just the manifestations of the system, but the system itself.

Systems Iceberg

Systems Iceberg

Very often that kind of analysis is missing from these kinds of stories, which make for great film and books, but not for long-term success.

In a complex system, meaning and knowledge is gained through interactions, thus we need stories and data that reflect what kind of interactions take place and under what conditions. Looking at the systems iceberg model above, the tendency is to focus on the events (the Studio H’s), but often we need to look at the structures beneath.

To be sure, there is a lot to learn from Studio H now and from the story presented in If You Build It. The lesson is in the prototyping: Emily and Matthew provide a prototype that shows us it can be done and what kind of things we can learn from it. The mistake is trying to replicate Studio H as it is represented in the film, rather than seeing it as a prototype.

In the post-event Q & A with the audience, a well-intentioned gentleman working with school-building in Afghanistan asked Patrick Creadon how or whether he could get Emily and Matthew to come there and help (with pay) and Creadon rightly answered that there are Emilys and Matthews all over the place and that they are worth connecting to.

Creadon is half right. There are talented, enthusiastic people out there who can learn from the experience of the Studio H team, but probably far fewer who have the means to assume the risk that Emily and Matthew did. Those are the small details that separate out a good story from a sustainable, systemic intervention that really innovates in a way that changes the system. But its a start.

If You Build It is in theatres across North America.

businesscomplexityinnovation

What’s the big idea and how are you going to make it real?

What is your strategy?

What is your strategy?

Concepts like design thinking and developmental evaluation are best used when they help ask big questions before seeking answers. How we frame the problem is much more important than the solution we generate, but that way of thinking means going into an area that is much talked about and rarely delivered on: strategy.

Many companies and human service organizations are getting desperate for solutions to the vexing problems they face. However, it may be that the organizations are as stuck finding solutions because they are tackling the wrong problem.

Problem framing is among the most critical, yet often overlooked, steps in design and innovation and often leads to more solutions that fail than those that succeed. Asking better questions is a start and developing a strategy from that is where to go next.

The big idea is your problem, making it real is the strategy to solving it.

What is the big idea?

Herbert Simon wrote about problem forming, framing and solving as the central tenets of design. Albert Einstein, another Nobel laureate, was famously (mis?)quoted as saying this about the discovery process:

If I had only one hour to save the world, I would spend fifty-five minutes defining the problem, and only five minutes finding the solution.

Like so many of these ‘famous’ quotes, its origins are murky and the (hypothesized) original is much less poetic, but the spirit of the phrase is that problem finding and forming is enormously important for innovation. Case studies from design missions, innovation labs, and my own personal experience suggest that this ratio of 55 and 5 in resourcing is probably not far off from the truth.

Problem forming is also tied to a greater sense of mission, which is where a lot of organizations get it wrong. A clear, appropriately scoped mission provides the boundaries for creativity to flourish and innovation efforts to focus. Steve Jobs charged Apple with the mission of developing tools to enable people to create. That may have started with computers, but it soon grew to software with features that were design-forward and attractive, and then mobile devices and the ecosystems that powered them. When viewed from the mission of enabling creativity, the move to being a music and bookseller isn’t a leap from Apple’s roots as a maker of desktop computers.

Where are you going?

Strategy is about saying what you don’t do as much as it is about saying what you do. It also means saying what you do clearly and meaning it. Both of these have enormous implications for what a program focuses on and what feedback systems they develop to help them innovate and guide their strategy moving forward.

A good, simple resource on strategy is Howell J. Maltham Jr‘s recent book I Have a Strategy, No You Don’t. In the book the author illustrates the many ways in which we claim strategy when really it’s a wish. Malthan asserts that a strategy has:

  1. A purpose
  2. A plan
  3. A sequence of actions or tactics
  4. A distinct, measurable goal

However, most importantly according to Maltham is that this all needs a narrative – the story of what you do and how you do it. Too often we see the absence of narrative or a lack of connection to any of the four components above. Apple has famously developed a strong narrative for how it operates and realizes it mission.

Maltham’s four-point description of strategy works when you are dealing with simple and maybe slightly complicated systems; those with some measure of predictability and control. It doesn’t work well for complexity, which is where many human services are either immersed or shifting to. For that, we need some form of adaptive strategy that provides guidance, but also works with, rather than against complexity. Yet, it still requires a narrative.

Strategy for complex times

Like the above cartoon from Tom Fishburne, the tactics should not precede the strategy. It’s interesting to see how often the term tactic and strategy get confused and conflated. It’s easy to see why. Tactics are tangible. They — like 90% of meetings, answering email and phone messages — offer the illusion of productivity and impact. Getting hundreds or thousands of likes, followers, and re-tweets is a proxy for impact for a lot of people.

But if you’re looking to make real change, it doesn’t matter so much that you’re doing stuff, but rather whether you’re moving stuff.

It’s why adaptive strategy is difficult, because it means moving your ideas, your thinking, your relationships and your operations to constantly re-calibrate your focus. Just like looking at birds through binoculars or watching a football game from the stands, you need to constantly adjust your focus to maintain engagement. The same thing happens with strategy.

At the same time, difficult shouldn’t be the reason not to do something.

This is the new thinking that is needed to innovate and that is why many organizations seek to do the wrong thing righter by doubling down on trendiness to appear innovative without thinking deeply about what the big idea is and how it is supposed to become real. Whether static or adaptive, the narrative will tie that together. So what is your organization’s story and do you know how to tell it?

 

complexitydesign thinkingemergenceevaluationsystems science

Developmental Evaluation and Design

Creation for Reproduction

Creation for Reproduction

 

Innovation is about channeling new ideas into useful products and services, which is really about design. Thus, if developmental evaluation is about innovation, then it is also fundamental that those engaging in such work — on both evaluator and program ends — understand design. In this final post in this first series of Developmental Evaluation and.., we look at how design and design thinking fits with developmental evaluation and what the implications are for programs seeking to innovate.  

Design is a field of practice that encompasses professional domains, design thinking, and critical design approaches altogether. It is a big field, a creative one, but also a space where there is much richness in thinking, methods and tools that can aid program evaluators and program operators.

Defining design

In their excellent article on designing for emergence (PDF), OCAD University’s Greg Van Alstyne and Bob Logan introduce a definition they set out to be the shortest, most concise one they could envision:

Design is creation for reproduction

It may also be the best (among many — see Making CENSE blog for others) because it speaks to what design does, is intended to do and where it came from all at the same time. A quick historical look at design finds that the term didn’t really exist until the industrial revolution. It was not until we could produce things and replicate them on a wide scale that design actually mattered. Prior to that what we had was simply referred to as craft. One did not supplant the other, however as societies transformed through migration, technology development and adoption, shifted political and economic systems that increased collective actions and participation, we saw things — products, services, and ideas — primed for replication and distribution and thus, designed.

The products, services and ideas that succeeded tended to be better designed for such replication in that they struck a chord with an audience who wanted to further share and distribute that said object. (This is not to say that all things replicated are of high quality or ethical value, just that they find the right purchase with an audience and were better designed for provoking that).

In a complex system, emergence is the force that provokes the kind of replication that we see in Van Alstyne and Logan’s definition of design. With emergence, new patterns emerge from activity that coalesces around attractors and this is what produces novelty and new information for innovation.

A developmental evaluator is someone who creates mechanisms to capture data and channel it to program staff / clients who can then make sense of it and thus either choose to take actions that stabilize that new pattern of activity in whatever manner possible, amplify it or — if it is not helpful — make adjustments to dampen it.

But how do we do this if we are not designing?

Developmental evaluation as design

A quote from Nobel Laureate Herbert Simon is apt when considering why the term design is appropriate for developmental evaluation:

“Everyone designs who devises courses of action aimed at changing existing situations into preferred ones”.

Developmental evaluation is about modification, adaptation and evolution in innovation (poetically speaking) using data as a provocation and guide for programs. One of the key features that makes developmental evaluation (DE) different from other forms of evaluation is the heavy emphasis on use of evaluation findings. No use, no DE.

But further, what separates DE from ulitization-focused evaluation (PDF) is that the use of evaluation data is intended to foster development of the program, not just use. I’ve written about this in explaining what development looks like in other posts. No development, no DE.

Returning to Herb Simon’s quote we see that the goal of DE is to provoke some discussion of development and thus, change, so it could be argued that, at least at some level, DE it is about design. That is a tepid assertion. A more bold one is that design is actually integral to development and thus, developmental design is what we ought to be striving for through our DE work. Developmental design is not only about evaluative thinking, but design thinking as well. It brings together the spirit of experimentation working within complexity, the feedback systems of evaluation, with a design sensibility around how to sensemake, pay attention to, and transform that information into a new product evolution (innovation).

This sounds great, but if you don’t think about design then you’re not thinking about innovating and that means you’re really developing your program.

Ways of thinking about design and innovation

There are numerous examples of design processes and steps. A full coverage of all of this is beyond the scope of a single post and will be expounded on in future posts here and on the Making CENSE blog for tools. However, one approach to design (thinking) is highlighted below and is part of the constellation of approaches that we use at CENSE Research + Design:

The design and innovation cycle

The design and innovation cycle

Much of this process has been examined in the previous posts in this series, however it is worth looking at this again.

Herbert Simon wrote about design as a problem forming (finding), framing and solving activity (PDF). Other authors like IDEO’s Tim Brown and the Kelley brothers, have written about design further (for more references check out CENSEMaking’s library section), but essentially the three domains proposed by Simon hold up as ways to think about design at a very basic level.

What design does is make the process of stabilizing, amplifying or dampening the emergence of new information in an intentional manner. Without a sense of purpose — a mindful attention to process as well — and a sensemaking process put in place by DE it is difficult to know what is advantageous or not. Within the realm of complexity we run the risk of amplifying and dampening the wrong things…or ignoring them altogether. This has immense consequences as even staying still in a complex system is moving: change happens whether we want it or not.

The above diagram places evaluation near the end of the corkscrew process, however that is a bit misleading. It implies that DE-related activities come at the end. What is being argued here is that if the place isn’t set for this to happen at the beginning by asking the big questions at the beginning — the problem finding, forming and framing — then the efforts to ‘solve’ them are unlikely to succeed.

Without the means to understand how new information feeds into design of the program, we end up serving data to programs that know little about what to do with it and one of the dangers in complexity is having too much information that we cannot make sense of. In complex scenarios we want to find simplicity where we can, not add more complexity.

To do this and to foster change is to be a designer. We need to consider the program/product/service user, the purpose, the vision, the resources and the processes that are in place within the systems we are working to create and re-create the very thing we are evaluating while we are evaluating it. In that entire chain we see the reason why developmental evaluators might also want to put on their black turtlenecks and become designers as well.

No, designers don't all look like this.

No, designers don’t all look like this.

 

Photo Blueprint by Will Scullen used under Creative Commons License

Design and Innovation Process model by CENSE Research + Design

Lower image used under license from iStockphoto.

complexitydesign thinkingemergenceevaluationsystems thinking

Developmental Evaluation and Sensemaking

Sensing Patterns

Sensing Patterns, Seeing Pathways

Developmental evaluation is only as good as the sense that can be made from the data that is received. To assume that program staff and evaluators know how to do this might be one of the reasons developmental evaluations end up as something less than they promise. 

Developmental Evaluation (DE) is becoming a popular subject in the evaluation world. As we see greater recognition of complexity as a factor in program planning and operations and what it means for evaluations it is safe to assume that developmental evaluation will continue to attract interest from program staff and evaluation professionals alike.

Yet, developmental evaluation is as much a mindset as it is a toolset and skillset; all of which are needed to do it well. In this third in a series of posts on developmental evaluation we look at the concept of sensemaking and its role in understanding program data in a DE context.

The architecture of signals and sense

Sensemaking and developmental evaluation involve creating an architecture for knowledge,  framing the space for emergence and learning (boundary specification), extracting the shapes and patterns of what lies within that space, and then working to understand the meaning behind those patterns and their significance for the program under investigation. A developmental evaluation with a sensemaking component creates a plan for how to look at a program and learn from what kind of data is generated in light of what has been done and what is to be done next.

Patterns may be knowledge, behaviour, attitudes, policies, physical structures, organizational structures, networks, financial incentives or regulations. These are the kinds of activities that are likely to create or serve as attractors within a complex system.

To illustrate, architecture can be both a literal and figurative term. In a five-year evaluation and study of scientific collaboration at the University of British Columbia’s Life Sciences Institute, my colleagues Tim Huerta, Alison Buchan and Sharon Mortimer and I explored many of these multidimensional aspects of the program* / institution and published our findings in the American Journal of Evaluation and Research Evaluation journals. We looked a spatial configurations by doing proximity measurements that connected where people work to whom they work with and what they generated. Research has indicated that physical proximity makes a difference to collaboration (E.g.,: Kraut et al., 2002). There is relatively little concrete evaluation on the role of space in collaboration, mostly just inferences from network studies (which we also conducted). Few have actually gone into the physical areas and measured distance and people’s locations.

Why mention this? Because from a sense-making perspective those signals provided by the building itself had an enormous impact on the psychology of the collaborations, even if it was only a minor influence on the productivity. The architecture of the networks themselves was also a key variable that went beyond simple exchanges of information, but without seeing collaborations as networks it is possible that we would have never understood why certain activities produced outcomes and others did not.

The same thing exists with cognitive architecture: it is the spatial organization of thoughts, ideas, and social constructions. Organizational charts, culture, policies, and regulations all share in the creation of the cognitive architecture of a program.

Signals and noise

The key is to determine what kind of signals to pay attention to at the beginning. And as mentioned in a previous post, design and design thinking is a good precursor and adjunct to an evaluation process (and, as I’ve argued before and will elaborate on, is integral to effective developmental evaluation). Patterns could be in almost anything and made up of physical, psychological, social and ‘atmospheric’ (org and societal environmental) data.

This might sound a bit esoteric, but by viewing these different domains through an eye of curiousity, we can see patterns that permit evaluators to measure, monitor, observe and otherwise record to use as substance for programs to make decisions based on. This can be qualitative, quantitative, mixed-methods, archival and document-based or some combination. Complex programs are highly context-sensitive, so the sense-making process must include diverse stakeholders that reflect the very conditions in which the data is collected. Thus, if we are involving front-line worker data, then they need to be involved.

The manner in which this is done can be more or less participatory and involved depending on resources, constraints, values and so forth, but there needs to be some perspective taking from these diverse agents to truly know what to pay attention to and determine what is a signal and what is noise. Indeed, it is through this exchange of diverse perspectives that this can be ascertained. For example, a front line worker with a systems perspective may see a pattern in data that is unintelligible to a high-level manager if given the opportunity to look at it. That is what sensemaking can look like in the context of developmental evaluation.

“What does that even mean?” 

Sensemaking is essentially the meaning that people give to an experience. Evidence is a part of the sensemaking process, although the manner in which it is used is consistent with a realist approach to science, not a positivist one. Context is critical in the making of sense and the decisions used to act on information gathered from the evaluation. The specific details of the sensemaking process and its key methods are beyond the depth of this post, some key sources and scholars on this topic are listed below. Like developmental evaluation itself, sensemaking is an organic process that brings an element of design, design thinking, strategy and data analytics together in one space. It brings together analysis and synthesis.

From a DE perspective, sensemaking is about understanding what signals and patterns mean within the context of the program and its goals. Even if a program’s goals are broad, there must be some sense of what the program’s purpose is and thus, strategy is a key ingredient to the process of making sense of data. If there is no clearly articulated purpose for the program or a sense of its direction then sensemaking is not going to be a fruitful exercise. Thus, it is nearly impossible to disentangle sensemaking from strategy.

Understanding the system in which the strategy and ideas are to take place — framing — is also critical. An appropriate frame for the program means setting bounds for the system, connecting that to values, goals, desires and hypotheses about outcomes, and the current program context and resources.

Practical sensemaking takes place on a time scale that is appropriate to the complexity of information that sits before the participants in the process. If a sensemaking initiative is done with a complex program that has a rich history and many players involved that history, it is likely that multiple interactions and engagements with participants will be needed to undertake such a process. In part, because the sensemaking process is about surfacing assumptions, revisiting the stated objectives of the program, exploring data in light of those assumptions and goals, and then synthesizing it all to be able to create some means of guiding future action. In some ways, this is about using hindsight and present sight to generate foresight.

Sensemaking is not just about meaning-making, but also a key step in change making for future activities. Sensemaking realizes one of the key aspects of complex systems: that meaning is made in the interactions between things and less about the things themselves.

Building the plane while flying it

In some cases the sense made from data and experience can only be made in the moment. Developmental evaluation has been called “real time” evaluation by some to reflect the notion that evaluation data is made sense of as the program unfolds. To draw on a metaphor illustrated in the video below, sensemaking in developmental evaluation is somewhat like building the plane while flying it.

Like developmental evaluation as a whole, sensemaking isn’t a “one-off” event, rather it is an ongoing process that requires attention throughout the life-cycle of the evaluation. As the evaluator and evaluation team build capacity for sensemaking, the process gets easier and less involved each time its done as the program builds its connection both to its past and present context. However, such connections are tenuous without a larger focus on building in mindfulness to the program — whether organization or network — to ensure that reflections and attention is paid to the activities on an ongoing basis consistent with strategy, complexity and the evaluation itself.

We will look at the role of mindfulness in an upcoming post. Stay tuned.

* The Life Sciences Institute represented a highly complicated program evaluation because it was simultaneously bounded as a physical building, a corporal institution within a larger institution, and a set of collaborative structures that were further complicated by having investigator-led initiatives combined with institutional-level ones where individual investigators were both independent and collaborative. Taken together it was what was considered to be a ‘program’.
References & Further Reading:

Dervin, B. (1983). An overview of sense-making research: Concepts, methods and results to date. International Communication Association Meeting, 1–13.

Klein, G., & Moon, B. (2006). Making sense of sensemaking 1: Alternative perspectives. Intelligent Systems. Retrieved from http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1667957

Klein, G., Moon, B., & Hoffman, R. R. (2006). Making sense of sensemaking 2: A macrocognitive model. IEEE Intelligent Systems, 21(4),

Kolko, J. (2010a). Sensemaking and Framing : A Theoretical Reflection on Perspective in Design Synthesis. In Proceedings of the 2010 Design Research Society Montreal Conference on Design & Complexity. Montreal, QC.

Kolko, J. (2010b). Sensemaking and Framing : A Theoretical Reflection on Perspective in Design Synthesis Understanding Sensemaking and the Role of Perspective in Framing Jon Kolko » Interaction design and design synthesis . In 2010 Design Research Society (DRS) international conference: Design & Complexity (pp. 6–11). Montreal, QC.

Kraut, R., Fussell, S., Brennan, S., & Siegel, J. (2002). Understanding effects of proximity on collaboration: Implications for technologies to support remote collaborative work. Distributed work, 137–162. Retrieved from NCI.

Mills, J. H., Thurlow, A., & Mills, A. J. (2010). Making sense of sensemaking: the critical sensemaking approach. Qualitative Research in Organizations and Management An International Journal, 5(2), 182–195.

Rowe, A., & Hogarth, A. (2005). Use of complex adaptive systems metaphor to achieve professional and organizational change. Journal of advanced nursing, 51(4), 396–405.

Norman, C. D., Huerta, T. R., Mortimer, S., Best, A., & Buchan, A. (2011). Evaluating discovery in complex systems. American Journal of Evaluation32(1), 70–84.

Weick, K. E. (1995). The Nature of Sensemaking. In Sensemaking in Organizations (pp. 1–62). Sage Publications.

Weick, K. E., Sutcliffe, K. M., & Obstfeld, D. (2005). Organizing and the Process of Sensemaking. Organization Science, 16(4), 409–421.

Photo by the author of Happy Space: NORA, An interactive Study Model at the Museum of Finnish Architecture, Helsinki, FI.

behaviour changecomplexitydesign thinkinginnovationpsychology

The Organizational Zombie Resistance Kit

How to thwart a zombie

How to thwart a zombie

Zombies — unaware, semi-conscious, distracted individuals — are all around us and running many of the organizations we work in or with. And just like combatting real zombies there is a need to target the head.

There is much musing about what a zombie apocalypse might look like, but anyone paying attention to what is going on around them might not have to imagine what that looks like as they’d be forgiven for thinking it is already here. Whether its people glued to cellphones while walking/running/biking/driving, asking ‘dumb’ questions immediately following the answer, or scientists lazily allowing junk to pass peer review, we are surrounded by zombie-like behaviour.

As discussed in a previous post, the zombies are already here. A zombie in this context exhibits mindless attention in a manner that restricts awareness and appreciation of one’s immediate context and the larger system to which that behaviour occurs. Zombies are great fodder for horror movies, but lousy companions on the journey of life and even worse problem solvers. Building resistance to them involves more than just aiming for the head, it means aiming for the heart (of an organization). Thankfully, there are methods and tools that can do that and thus, CENSEMaking brings you the Zombie Resistance Kit.

Building resistance to zombies

I am a professional zombie hunter. I do this by helping organizations to be more mindful. A mindful organization is aware of where it sits in the systems it inhabits, connects the current context to its past, and from those places envisions paths to futures not yet realized; it is part psychology, part strategic foresight, and part research and evaluation. How it expresses this knowledge into value is design.

Building a mindful organization — one resistant to zombies — requires inoculation through awareness. There are eight broad areas of attention.

1. Grounding is a process of holding to where you are by first revealing to yourself where that is. It is about locating yourself within the system you are in and connecting to your history. Mindfulness is often seen as being focused on the present moment, but not at the expense of the past. Understanding the path you took to get to the present allows you to see path dependencies and habits and mindfully choose whether such pathways are beneficial and how they relate to the larger system. Surfacing assumptions and system mapping are key methods and tools to aid in the process of grounding an organization.

2.  Attunement is a means of syncing yourself to the environment, your role within it (after having been grounded) and increasing your receptor capacity for sensing and learning. It is about calibrating ones mission, vision, and strategy with the system purposefully and intentionally building your awareness for understanding how harmonious they are for your organization. When attuned to what is going on — literally being tuned into the signals around you — the potential to see and process both strong and weak signals is heightened, increasing sensemaking and sensing capability at the same time. The ability to see the system and understand what it means for who you are and what you do is a terrific means of combating zombie-like thinking.

3. Discovery: Encouraging curiosity and promoting a culture of inquiry is another key means of enhancing awareness. Kids are constantly amazed by the things they see and experience everyday. The world is no less amazing today than it is was when we were kids, but the pressures to act and ‘be’ particular ways can greatly inhibit the natural curiosity that we all have about what is going on around us. Encouraging discovery and asking critical questions about what we find is a means of enhancing overall engagement with the raw materials of our enterprise. It is risky because it might call into question some long-held assumptions that are no longer true, but if people are genuinely supported in asking these questions an organization increases the number of ‘sensors’ it has in it across conditions, roles and sectors generating new, context-ready knowledge that can seed innovation and enhance overall resiliency.

4. Creativity: Application of creative methods of problem finding, framing and solving via design thinking is a means of promoting engagement and seeing systems solutions. Design thinking can be a means of creative facilitation that guides mindful development, discovery, synthesis and solution proposals. Encouraging generation of ideas of all types, firsthand research, creation of prototypes, and the opportunity to test these prototypes in practice allow for individuals to claim legitimate ownership of the problem space and the solution space. This ownership is what creates true investment in the work and its outcomes, which is what zombies lack.

5. Strategic Foresight: By envisioning not only what a design can produce in the short-term, but see a future for what is created today into the years ahead, we build commitment to long-term goals. Strategic foresight brings together all of the preceding components to start envisioning what possible futures might look like so that an organization can better prepare for them or even create them. Strategic foresight is a structured means of visualizing possible futures based on current trends, data-driven projections, models and strategic priorities of the organization and connects the present activities to the past and projects possible futures from all of this giving the zombie a reason to stop its relentless blind pursuit of an unaware present goal.

6. Focus: While creative thinking is useful in enhancing divergent perspective taking and seeing new possibilities, focus allows for attendance to the critical path and refinement of strategy to fit the context, desires, capacity and intentions. Of the many futures that a strategic foresight process might produce, focusing the energy on those that are the most beneficial, congruent with goals and desires, and synchronous with the systems that an organization engages is another way to shock mindless thinking out of its zombie-like state. A focus provides a richer experience and something to strive for.

7. Knowledge integration. Introducing possibilities, building a creative culture, enhancing receptor capacity and building a focus is not sustainable if knowledge isn’t integrated throughout the process of moving forward; it is the knowledge practice behind developmental design.  Knowledge integration involves critically examining the organizational structure and culture to observe current knowledge practices. Do you have the right tools? The ability to use those tools effectively and make sense of the findings? Is the system understood and aligned to the purpose and resources available? When your system is aligned and the structures are put into place to work with that alignment knowledge is put to use.

8. Design Cycling: Developmental design is the means of engaging in ongoing evaluation and design simultaneously, while knowledge integration is taking the learning from those products and incorporating it into the DNA of the organization. Design cycling is the process by which this unfolds and iteratively repeats over cycles of innovation. Invariably, organizations tend to drift a little and by framing the innovation process as a cycle it acknowledges that even the best ideas will reach an ebb and flow and require renewal. This cyclical process encourages us to return to the first stage. This is an approach consistent with the Panarchy approach to life cycle development in complex systems. Everything runs its course.  This approach is consistent with a natural systems perspective and a pillar of the work on sustainable development in natural systems.

This model of development and organizational awareness provides balm against zombie-like behaviour. It gets people excited, it produces visible results that can be scrutinized in a transparent way, and it heightens engagement by bringing everyone in an organization into the role of problem framing, finding and solving. It enhances accountability for everyone who are now enlisted as creators, researchers, designers, and sensemakers.

By being more aware and alive we better engage brains rather than use that grey matter as food for zombies.

For more details on using this approach with your organization contact CENSE Research + Design.

Photo credit: From Zombie Walk 2012 SP collection by Gianluca Ramahlo Misiti used under Creative Commons Licence

behaviour changecomplexitypublic healthsocial systemssystems thinking

The New Zombie

Zombie stare

They are among us and hungry for brains

Zombies are attacking us; not for brains, but for attention. The consequences of this is that they are everywhere and sucking the intelligence out of human systems. 

Forget orange, zombie is the new black.

Zombies are hot. TV shows, books and films about zombies are more popular than ever, and this time of year the public’s attention to the undead is at its nadir. The CDC in the United States even got into the act by using zombies as a health promotion vehicle to support emergency preparedness. From zombie walks to art shows, the staggering brain-eating, brain-less are everywhere.

Yet, there is a new breed being formed that doesn’t eat brains and has them, but may not be using them well and they are all around us everywhere.

They walk among us

Look around and what do you see? People online, on the phone, texting and walking and driving, being everywhere except where they are. Examples of people walking into fountains or falling into a sinkhole while on the phone are often seen as comi-tragic, yet they belie a remarkably powerful trend towards disengagement from the world around them. Charlene deGuzman and Miles Crawford‘s beautiful and disheartening short film I Forgot My Phone plays this for further comic and sad effect as they portray a day in the life of someone paying attention to those not paying attention to anything away from their screen. The film highlights the modern paradox of being more connected than ever, yet overwhelmingly alone.

Emerging research is showing remarkable spikes in risks associated with mobile phone use and injury and mortality. We might laugh at people falling into holes or bumping into things, but only when it hurts the ego and not the body. This is serious stuff. Keep in mind that we don’t see non-reported injuries (e.g., someone bruising their head) and the many near misses between person and object — including cars, which have their own epidemic of problems with texting and attention.

Indeed, zombies embody paradox: a brainless being that is undead seeks brains to stay unalive. Whether they are alive or dead depends on where you stand and that is what makes them a complex character despite their surface-level simplicity.

Brains…need…more…(use of science) brains….

Zombie Science

Zombie Science?

While it might be easy to point to those on phones, zombie behaviour occurs elsewhere in places where the effects are far less comic and just as dangerous. The latest issue of The Economist features a cover story on the problems science is having with it credibility and quality control. Some of this is due to what I would call zombie-like behaviour: mindless attention in a manner that restricts awareness and appreciation of one’s immediate context and the larger system to which that behaviour occurs.

The recent expose by science journalist John Bohannon published in the journal Science exposes zombie-like thinking in how open-access science journals accept and reject papers. Bohannon’s inquiry was prompted by questions about the way fees were charged for open access journals (which is how they can remain open to the public) and the peer review require to advance publication. Presumably, an article has to pass review from peer professional scientists before it is accepted and then the fee is paid. No acceptance, no fee (except for perhaps a small application processing charge).

As profiled in an interview with the CBC radio show The Current

Bohannon wanted to find out whether fee-charging open access journals were actually keeping their promise to do peer review — a process in which scientists with some knowledge of a paper’s topic volunteer to check it for scientific flaws…

…In the end, what he concluded was that “a huge proportion” of the journals were not ensuring their papers were peer reviewed.

Even in cases where peer review happened, it didn’t always function correctly. For example, the Ottawa-based International Journal of Herbs and Medicinal Plants clearly sent the paper out to be reviewed by real scientists, who pointed out some flaws, Bohannon recalled. Even so, when Bohannon submitted a revised version of the paper without correcting any of the flaws, it was accepted.

Bohannon’s approach and findings are not without some problems of their own, but they don’t much change the conclusion that there are deep problems within the scientific enterprise.

Much of what Bohannon found can be attributed to greed, but a great deal of it is due to bad scientific practice. As a consultant who is also a publishing researcher and ‘recovering’ academic I know the enormous amount of energy that goes into publishing an academic article in a scholarly journal. As one who is sent between 4 to 5 manuscripts to review from legitimate journals per month I know the demands that are placed on reviews. We also publish far too much for the system to handle. Writing in the Chronicle of Higher EducationMark Bauerlein and colleagues look closely at the ‘avalanche’ of publishing and shed light on many reasons why the problems that the Economist and Science occur (Note: I’d strongly encourage you to read through the comments as it is as instructive as the article itself).

They are everywhere

To add to the examples of zombie culture I need only look to my own daily life outside of science and  mobile phones. Just the other day I witnessed the following example at a community meeting that was organized in part to discuss the expenditure of funds to make a better living area for people in a building:

Presenter: “…and I am pleased to conclude that the new furniture for the outdoor spaces is going to be made by a company that created the same products at [place] out of recycled materials. We will expect to have the new furniture here in 6 to 8 weeks. Any questions?”

First question: “I love the work you’ve done. Can you tell me when the furniture will be here?”

Sadly, I have many other stories that show that many people are not paying attention. They are sitting through workshops and not picking up basic concepts (even after having asked for it and having been given it multiple times over), asking for materials that were already shared on multiple occasions, suggesting ideas that were already discussed and agreed upon over because that person didn’t engage in the discussion and so on. This happens not because people are stupid, but because they are disengaged.

A simple search through statistics compilations finds enormous material on what kind of inputs we expose ourselves to and its impact on attention. There is more coming at us in quantity and context and that is undoubtedly influencing quality of processing and engagement. I can speak of this personally and through observation. The amount of times I find people not hearing what is said, processing it effectively, or even remembering something said is staggering.

It’s not surprising. We are alerted everywhere: a text message, a phone call, a Facebook message, an email, an app alert, someone coming by the office, external noise outside, and visual noise everywhere. The explicit and ambient signals we are exposed to in a day is staggering. Clay Shirky suggests it’s not that we have too much information, it’s that our filters are failing. I think it’s now both and one reinforces the other.

Coming back…a look at systems

While individuals are distracted, they are products of distracted systems. To look at one part of the science zombie situation, professors are now asked to publish more than ever, get grants from a dwindling pool, teach more students than ever and in more crowded conditions and with greater social needs, and to find ways to make their research more accessible to different audiences while engaging more with the communities of interest affected by that research. All of this takes time. Add to that the probability that the professor her/himself has to raise their own salary and that the only way to do this is to be very successful at the above-mentioned tasks and you get someone who is stressed and overtaxed.

Mindfulness-based approaches do not change any of that, but they can help strengthen the filter. By being more individually mindful, but more importantly create mindful organizations. Building resilient tribes of social innovators and the leadership communities to steward them is another. Granting ourselves the time to reflect, sensemake and listen to the systems we work in is also key. By listening better, we are better able to design systems that are innovative, responsive and humane by building them to human scale.

All well and good you might say, but how? That’s what’s to come in some future posts as we look at designing better systems and making them more attractive so people stay engaged.

Stay tuned….and watch out for zombies.

Photo credit: Zombie Walk 2012 SP by Gianluca Ramalho Misiti used under Creative Commons License