The PR Problem for Design, Evaluation,and Complexity

Posted: January 29, 2012 | Author: | Filed under: complexity, design thinking, evaluation | Tags: , , , , , , , , | 2 Comments »

I (heart) PR

Complex concepts like evaluation, design and even complexity itself provide insight, strategies and applications that provide usable solutions to real-world problems, but also suffer from widespread misunderstandings, confusion and even derision. If they are to take hold beyond their initial communities of interest, they need to address their PR problem head on. 

This past week was Design Week in Toronto. As one works extensively with design concepts and even has a health promotion-focused design studio, one couldn’t be faulted for thinking that this would be a big week for someone like me who lives and works in the city.  Well, it came and went and I didn’t attend a single thing. The reason was partly due to timing and my schedule, but largely because the focus of the week was not really on design writ large, but rather interior design. Sure, there were a few events that focused on social issues (what I am interested in) like the Design With Dialogue session on Designing a Future for our Future, but mostly it was focused on one area of a large field.

And thus, interior design was left to represent all of design.

So why does this matter? It matters a lot because when people hear the term design, most of what was presented this week fits with that perception. The problem is that design is so much more than that. It is about making things, creative thinking and problem tackling (design thinking), social innovation, and responsive planning for complex situations. Architects, business leaders, military strategists, social service agencies and health promoters all engage in design. Indeed, Herbert Simon‘s oft-quoted and often contested definition fits nicely here:

Everyone designs who devises courses of action aimed at changing existing situations into preferred ones

If one accepts that we are all designers and all of what we create and use for change is design, than a week devoted to the topic should offer much more than innovative concepts in furniture or flooring. Yet, this high-concept style showcase is what most people think about when they first hear design. Give people a choice between a Philippe Stark Juicy Salif citrus juicer and creating a trades-based, social change curriculum for low-income kids such as the work by Emily Pilloton as the example of design and they will probably guess think Stark over Pilloton, when both are equally valid examples.

Evaluation (another area I focus my work on) is equally fraught with perception problems. If you want to raise someone’s blood pressure or heart rate, tell them that either they or their work will be the focus of an evaluation. Evaluation may be the longest four-letter word in the English language. Yet, tell someone that you have a strategy that can enable people to learn about what they do, its impact, and provide intelligence on ways to improve, adapt and outperform their competitors and you’ll find an inspired audience for evaluation services.

Lastly, complexity presents the same challenge. It’s very name — complexity — can make people shy away from it. As humans, we crave the simple in most things as it is easier to understand, manage and control. Complexity offers none of these things and, if anything, reveals how little control we have. Entire fields of inquiry have been established around complexity science and its related theories and practices. Complexity can help us make sense of why things don’t work as we think they ought to and allow us to better navigate through unpredictable terrain with greater resilience than if we tried to tackle such problems as if they were linear in their cause and consequence.

In all of these cases — design, evaluation and complexity — there exists a PR problem. The advantages that they pose are tremendous, yet these concepts are frequently misunderstood, dismissed, or inappropriately used . When this happens, it creates even greater distance between the potential benefits these concepts offer and their real-world application.

This distance is partly an artefact of poorly articulated definitions and examples, but also by design (no pun intended). There are those who relish having these concepts appear opaque to those outside of their social cluster. Thus, we have the ‘superstar designer’ who seeks to create products and personas that are built upon their rarity, rather than accessibility. There are evaluators who exploit the fear that people have of evaluation and lack the understanding of the methods and practices of evaluation (vs concepts like research or innovation consulting) to gain contracts and social influence within their field. Complexity, with its foundations in physics and systems biology, can appear to the layperson as otherworldly, making its practitioners and scientists seem all the more powerful and smart. These tactics benefit a small ‘elite’(?) number of professionals, while robbing a far larger audience of the potential benefits.

In 1969, then president of the American Psychological Association, George Miller, implored members to “give psychology away“. His message was that psychology was too important to be left just to the professional, graduate-trained practitioners to use. If psychology was to confer social benefits, it was necessary to ensure that everyone had access to it — it’s theories, methods, models and treatments. It is perhaps no surprise that psychology remains one of the most popular undergraduate degree programs in the arts and social sciences and the focus of television shows, magazines and and an array of services. Miller was commenting on the need to change a field that he perceived was becoming elitist and not serving the needs of society.

The same might be true of design, evaluation and complexity if we let it. It’s not a surprise that these three concepts are intimately tied together, as those training to apply design thinking and strategic foresight learn. Perhaps its time to start giving these ideas away, but to do so we first need to rehab their image and apply some design thinking and brand development strategy to all three ideas. As practitioners in any or all of these fields, giving away what we do by educating, reinforcing, and ensuring that the work we do is of the highest quality is a way to lead by example. None of us is likely to change things by ourselves, but together we can do wonders.

For those interested in evaluation, I suggest catching up on the AEA365 blog sponsored by the American Evaluation Association, where evaluation bloggers and practitioners share ideas about how to practice evaluation, but also how to communicate it to others. For those interested in design, I would encourage you to look at places like the Design Thinkers LinkedIn group, where practitioners from around the world discuss innovations and way to promote and apply design thinking. A similar group, and opportunity, exists with the Systems Thinking LinkedIn group or by joining the Plexus Institute, which does considerable work to promote complexity and systems thinking in North America.

Photo: I (Heart) PR by The Silfwer used under Creative Commons Licence from Flickr.

Common Sense, Complexity and Leadership

Posted: January 23, 2012 | Author: | Filed under: complexity, social systems | Tags: , , , , , , , , , | Leave a comment »

Bye, Bye Common Sense

Great leaders are often ascribed traits that include ample common sense. But what passes for common sense is often a grab bag of miscellaneous, inconsistent ideas that are context dependent and less useful in the complex environments where leadership is called for most. 

common sense |ˌkɑmən ˈsɛns|

noun

good sense and sound judgment in practical matters: use your common sense | [ as modifier ] : a common-sense approach.

Today Research in Motion announced that its founder Mike Lazaridis and his co-CEO Jim Balsillie would be relinquishing their roles with the company. In their place, a ‘pragmatic, operational-type guy ‘was installed. Presumably, Thorsten Heins has the common sense to lead RIM after the founders lost theirs. Yet, the pragmatic, common sense that RIM is looking for might not be what they need given the complexity of the environment they are leading in.

Common sense is a false lure in complex systems. In his recent book, Everything is Obvious *Once You Know the Answer, social network researcher  and Yahoo! Research scientist Duncan Watts eloquently critiques the concept of common sense, illustrating dozens of times over how “common sense” doesn’t fare so well in decisions that go beyond the routine and into the complex. Indeed. the very definition of the term implies that the problems that common sense works towards addressing are relatively simple and pragmatic.

Certainly, navigating daily social conventions might lend itself well to what we might call common sense. Watts refers to sociologist Harry Collins’ term ‘collective tacit knowledge‘ that is encoded in social norms, customs and practices of a particular world to describe common sense. However, what becomes common is a byproduct of many small decisions, dynamic and flexible changes to perspective, an accumulation of knowledge gained from small experiments over time, and the application of all of this knowledge to particular, context-dependent, situations. This constellation of factors and its interdependent, contextual overlap is why artificial intelligence systems have such a difficult time mimicking human thought and action. It is this attention to context that is most worth noting for it is this context that keeps common sense from being anything but common:

Common sense…is not so much a worldview as a grab bag of logically inconsistent, often contradictory beliefs, each of which seems right at the time but carries no guarantee of being right any other time.

Watts goes on to argue:

Commonsense reasoning, therefore, does not suffer from a single overriding limitation but rather from a combination of limitations, all of which reinforce and even disguise one another. The net result is that common sense is wonderful at making sense of the world, but not necessarily at understanding it.

Thus, we often concoct a narrative about the way something happens that sounds plausible, rational and be completely wrong. Throughout the book, Watts shows how often mistakes are made based on this common sense approach to solving problems.

When it comes to RIM, some have pointed to the late Steve Jobs’ assertion that they would have difficulty catching up to firms like Apple given that the consumer market is not their strength, the enterprise market is. Yet, Steve Jobs didn’t let the fact that Apple was a computer company stop him from making music players (the iPod), mobile phones (the iPhone) or becoming book, music and movie vendors (iTunes). A read of Steve Jobs’ biography by Walter Isaacson reveals a man who was able to lead and be successful through what appeared to be common sense, yet was decidedly uncommon among media and technology leaders. That is why Apple is where it is and why so many other technology companies lag behind them or simply disappeared.

The reason is that common sense in leadership looks as simple in hindsight only, not in foresight or even in the present moment. This is one of the big points that Watts makes. He uses the example of Sony’s MiniDisc system that, when introduced, had all of the hallmark features of the innovations that Apple introduced (novel, high quality, portable, smaller, visible advantages over the alternatives), yet it was a spectacular failure. Canadian management consultant Michael Raynor has called this the strategy paradox. When qualities such as vision, bold leadership, and focused execution — all the commonsensical aspects of great leaders — are applied to organizations it can lead to great success (Steve Jobs and Apple) or resounding failures (RIM?).

Strategic flexibility, making small adjustments consistently, and imaging scenarios for the future in an ongoing manner are some of the potential ways to limit the damage from common sense (or use its advantages more fully). This requires feedback mechanisms and close monitoring of program activities, developmental evaluation, and a willingness to tweak programs and design on the go (what I call: developmental design) . It’s not a surprise that this incremental approach to development is consistent with the way change is best produced in a complex adaptive system.

By recognizing that common sense is less than common and is certainly not consistent, program designers, developers, evaluators and other professionals will be better positioned to provide true leadership that addresses challenges and complexity rather than adds to the complexity and creates more problems.

Photo: Goodbye to Common Sense Space by Amulet Dream from Deviant Art

The Complexity of Planning and Design in Social Innovation

Posted: January 7, 2012 | Author: | Filed under: complexity, design thinking, evaluation, Systems science, systems thinking | Tags: , , , , , , , , | 2 Comments »

The Architecture of Complex Plans

Planning works well for linear systems, but often runs into difficulty when we encounter complexity. How do we make use of plans without putting too much faith in their anticipated outcome and still design for change and can developmental design and developmental evaluation be a solution? 

It’s that time of year when most people are starting to feel the first pushback to their New Year’s Resolutions. That strict budget, the workout plan, the make-time-for-old-friends commitments are most likely encountering their first test. Part of the reasons is that most of us plan for linear activities, yet in reality most of these activities are complex and non-linear.

A couple interesting quotes about planning for complex environments:

No battle plan survives contact with the enemy – Colin Powell

In preparing for battle I have always found that plans are useless, but planning is indispensable – Dwight D. Eisenhower

Combat might be the quintessential complex system and both Gens Powell and Eisenhower knew about how to plan for it and what kind of limits planning had, yet it didn’t dissuade them from planning, acting and reacting. In war, the end result is what matters not whether the plan for battle went as outlined (although the costs and actions taken are not without scrutiny or concern). In human services, there is a disproportionate amount of concern about ‘getting it right’ and holding ourselves to account for how we got to our destination relative what happens at the destination itself.

Planning presents myriad challenges for those dealing with complex environments. Most of us, when we plan, expect things to go according to what we’ve set up. We develop programs to fit with this plan, set up evaluation models to assess the impact of this plan, and envisage entire strategies to support the delivery and full realization of this plan into action. For those working in social innovation, what is often realized falls short of what was outlined, which inevitably causes problems with funders and sponsors who expect a certain outcome.

Part of the problem is the mindset that shapes the planning process in the first place. Planning is designed largely around the cognitive rational approach to decision making (PDF), which is based on reductionist science and philosophy. Like the image above, a plan is often seen as a blueprint for laying out how a program or service is to unfold over time. Such models of outlining a strategy is quite suitable for building a physical structure like an office where everything from the materials to the machines used to put them together can be counted, measured and bound. This is much less relevant for services that involve interactions between autonomous agents who’s actions have influence on the outcome of that service and that result might vary from context to context as a consequence.

For evaluators, this is problematic because it reduces the control (and increases variance and ‘noise’) into models that are designed to reveal specific outcomes using particular tools. For program implementers, it is troublesome because rigid planning can drive actions away from where people are and for them into activities that might not be contextually appropriate due to some change in the system.

For this reason the twin concepts of developmental evaluation and developmental design require some attention. Developmental evaluation is a complexity-oriented approach to feedback generation and strategic learning that is intended for programs where there is a high degree of novelty and innovation. Programs where the evidence is low or non-existent, the context is shifting, and there are numerable strong and diverse influences are those where developmental evaluations are not only appropriate, but perhaps one of the only viable models of data collection and monitoring available.

Developmental design is a concept I’ve been working on as a reference to the need to incorporate ongoing design and re-design into programs even after they have been initially launched. Thus, a program evolves over time drawing in information from feedback gained through processes like evaluation to tweak its components to meet changing circumstances and needs. Rather than have a static program, a developmental design is one that systematically incorporates design thinking into the evolutionary fabric of the activities and decision making involved.

Both developmental design and evaluation work together to provide data required to allow program planners to constantly adapt their offerings to meet changing conditions, thus avoiding the problem of having outcomes becoming decoupled from program activities and working with complexity rather than against it. For example, developmental evaluation can determine what are the key attractors shaping program activities while developmental design can work with those attractors to amplify them or dampen them depending on the level of beneficial coherence they offer a program. In two joined processes we can acknowledge complexity while creating more realistic and responsive plans.

Such approaches to design and evaluation are not without contention to traditional practitioners, leaving questions about the integrity of the finished product (for design) and the robustness of the evaluation methods, but without alternative models that take complexity into account, we are simply left with bad planning instead of making it like Eisenhower wanted it to be: indispensable .

A Call to Evaluation Bloggers: Building A Better KT System

Posted: December 9, 2011 | Author: | Filed under: evaluation, knowledge translation | Tags: , , , , , , , , , , , | 1 Comment »

Time To Get Online...

Are you an evaluator and do you blog? If so, the American Evaluation Association wants to hear from you. This CENSEMaking post features an appeal to those who evaluate, blog and want to share their tips and tricks for helping create a better, stronger KT system. 

Build a better moustrap and the world will beat a path to your door — Attributed to Ralph Waldo Emerson

Knowledge translation in 2011 is a lot different than it was before we had social media, the Internet and direct-to-consumer publishing tools. We now have the opportunity to communicate directly to an audience and share our insights in ways that go beyond just technical reports and peer-reviewed publications, but closer to sharing our tacit knowledge. Blogs have become a powerful medium for doing this.

I’ve been blogging for a couple of years and quite enjoy it. As an evaluator, designer, researcher and health promoter I find it allows me to take different ideas and explore them in ways that more established media do not. I don’t need to have the idea perfect, or fully formed, or relevant to a narrow audience. I don’t need to worry about what my peers think or my editor, because I serve as the peer review, editor and publisher all at the same time.

I originally started blogging to share ideas with students and colleagues — just small things about the strange blend of topics I engage in that many don’t know about or understand or wanted to know more of. Concepts like complexity, design thinking, developmental evaluation, and health promotion can get kind of fuzzy or opaque for those outside of those various fields.

Blogs enable us to reach directly to an audience and provide a means of adaptive feedback on ideas that are novel. Using the comments, visit statistics, and direct messages sent to me from readers, I can gain some sense of what ideas are being taken up with people and which one’s resonate. That enables me to tailor my messages and amplify those parts that are of greater utility to a reader, thus increasing the likelihood that a message will be taken up. For CENSEMaking, the purpose is more self-motivated writing rather than trying to assess the “best” messages for the audience, however I have a series of other blogs that I use for projects as a KT tool. These are, in many cases, secured and by invitation only to the project team and stakeholders, but still look and feel like any normal blog.

WordPress (this site) and Posterous are my favorite blogging platforms.

As a KT tool, blogs are becoming more widely used. Sites like Research Blogging are large aggregations of blogs on research topics. Others, like this one, are designed for certain audiences and topics — even KT itself, like the KTExchange from the Research Into Action Action initiative at the University of Texas and MobilizeThis! from the Research Impact Knowledge Mobilization group at York University.

The American Evaluation Association has an interesting blog initiative led by AEA’s Executive Director Susan Kistler called AEA365, which is a tip-a-day blog for evaluators looking to learn more about who and what is happening in their field. A couple of years ago I contributed a post on using information technology and evaluation and was delighted at the response it received. So it reaches people. It’s for this reason that AEA is calling out to evaluation bloggers to contribute to the AEA365 blog with recommendations and examples for how blogging can be used for communications and KT. AEA365 aims to create small-bite pockets of information that are easily digestible by its audience.

If you are interested in contributing, the template for the blog is below, with my upcoming contribution to the AEA365 blog posted below that.

By embracing social media and the power to share ideas directly (and done so responsibly), we have a chance to come closer to realizing the KT dream of putting more effective, useful knowledge into the hands of those that can use it faster and engage those who are most interested and able to use that information more efficiently and humanely.

Interested in submitting a post to the AEA365 blog? Contact the AEA365 curators at aea365@eval.org.

Template for aea365 Blogger Posts (see below for an example)

[Introduce yourself by name, where you work, and the name of your blog]

Rad Resource – [your blog name here]: [describe your blog, explain its focus including the extent to which it is related to evaluation, and tell about how often new content is posted]

Hot Tips – favorite posts: [identify 3-5 posts that you believe highlighting your blogging, giving a direct link and a bit of detail for each (see example)]

  • [post 1]
  • [post 2]
  • Etc.

Lessons Learned – why I blog: [explain why you blog – what you find useful about it and the purpose for your blog and blogging. In particular, are you trying to inform stakeholders or clients? Get new clients? Provide a public service? Help students?]

Lessons Learned: [share at least one thing you have learned about blogging since you started]

Remember – stay under 450 words total please!

–

My potential contribution (with a title I just made up): Cameron Norman on Making Sense of Complexity, Design, Systems and Evaluation: CENSEMaking

Rad Resource – [CENSEMaking]: CENSEMaking is a play on the name of my research and design studio consultancy and on the concept of sensemaking, something evaluators help with all the time. CENSEMaking focuses on the interplay of systems and design thinking, health promotion and evaluation and weaves together ideas I find in current social issues, reflections on my practice as well as the evidence used to inform it. I aspire to post on CENSEMaking 2-3 times per week, although because it is done in a short-essay format, find the time can be a challenge.

Hot Tips – favorite posts:

  • What is Developmental Evaluation? This post came from a meeting of working group with Michael Quinn Patton and was fun to write because the original exercise that led to the content (described in the post) was so fun to do. It also provided an answer to a question I get asked all the time.
  • Visualizing Evaluation and Feedback. I believe that the better we can visualize complexity the more feedback we provide, the greater the opportunities we have for engaging others, and more evaluations will be utilized. This post was designed to provoke thinking about visualization and illustrate how its been creatively used to present complex data in interesting and accessible ways. My colleague and CENSE partner Andrea Yip has tried to do this with a visually oriented blog on health promoting design, which provides some other creative examples of ways to make ideas more appealing and data feel simpler.
  • Developmental Design and Human Services. Creating this post has sparked an entire line of inquiry for me on bridging DE and design that has since become a major focus for my work. This post became the first step in a larger journey.

Lessons Learned – why I blog: CENSEMaking originally served as an informal means of sharing my practice reflections with students and colleagues, but has since grown to serve as a tool for knowledge translation to a broader professional and lay audience. I aim to bridge the sometimes foggy world that things like evaluation inhabit  – particularly developmental evaluation – and the lived world of people whom evaluation serves.

Lessons Learned: Blogging is a fun way to explore your own thinking about evaluation and make friends along the way. I never expected to meet so many interesting people because they reached out after reading a blog post of mine or made a link to something I wrote. This has also led me to learn about so many other great bloggers, too. Give a little, get a lot in return and don’t try and make it perfect. Make it fun and authentic and that will do.

___

** Photo by digitalrob70 used under Creative Commons License from Flickr

Developmental Thinking and Evaluation

Posted: June 20, 2011 | Author: | Filed under: evaluation, innovation, research | Tags: , , , , | 1 Comment »

Think developmentally before evaluating developmentally

Developmental evaluation is difficult to initiate, largely because the thinking behind it is so foreign to normal program planning and reporting. It appears that developmental thinking needs to be in place before one can hope to implement a DE project successfully.

Over the next few days I will be meeting with colleagues working with the Social Innovation Generation Group, Michael Quinn Patton  and others who share an interest and wrestle with developmental evaluation (DE) in practice.

Over the course of the last year we have been meeting monthly to discuss our experiences, challenges and learning on the issue of developmental evaluation. Although our group members come from diverse fields — government, academia, non-profit and others — and are focused on projects that range in scope, we all share one common experience: frustration with implementing DE.

Reading through a case study the other night I couldn’t help be see something I’d seen before: the principal barrier to the implementation of DE is that the program, its partners, or the stakeholders associated with the program didn’t individually or collectively function in a manner that supported DE. Whether they actually bought into DE in the first place is also not known, but it seems to me that the two are related.

Developmental thinking about social issues has shown itself in my work to be a linchpin for any progress on developmental evaluation. Commiserating with colleagues in this area, it seems evident to me that assessing for DE is a critical step in the pre-work that needs to come before any evaluation takes place. Without developmental thinking, developmental actions and evaluation is hard to reasonably achieve.

If you do not see your program as one that evolves, but rather just gets bigger, better, stronger, weaker etc.., having real-time evaluation tools will be less useful or perhaps even harmful in the absence of a thinking framework to make sense of the data. Real-time, consultative evaluation, and its utilization-focused actions makes DE stand apart from other approaches to evaluation, even if the methods and tools are similar.

The implications for this assertion in practice are enormous. It means that a DE practitioner cannot be just an evaluator or at least must find others that can work with a program to educate, inspire and contemplate collaboratively about developmental thinking and what it means for a program. It also brings the evaluation function far closer to program planning than evaluators (and program planners) might be used to. It also means holding a willingness to think different, not just implement different thinking. To that end, knowledge of motivation and some sense of how one provokes or creates space for change is also important.

Taken together, we have ourselves a real challenge. The “core competencies” for DE already include qualities like people skills, knowledge of complexity, and communication skills (in addition to fundamental skills in evaluation methods and process implementation), but now we are adding additional ones. Systems thinking, behaviour change, program planning and design are all reasonable skills that would assist an evaluator in doing this work. Nice in theory, but how about in practice? Can we reasonably expect that there are enough people out there with these skills to do it well? Or is this a call for more of a team-science (or rather, team evaluation) approach to evaluation?

 

 

 

 

Grasping Developmental Evaluation

Posted: February 8, 2011 | Author: | Filed under: complexity, Education & Learning, evaluation, innovation | Tags: , , , | 2 Comments »

Is this what you ordered?

Developmental evaluation is gaining interest in the evaluation, service and policy communities, but do these newfound fans know what they are getting when they ask for it? Grasping the fundamentals of development as opposed to growth is one of the foundations to its utility.

I recently concluded four days of meetings with a peer group — a community of practice – on Developmental Evaluation. The CoP brings together a diverse group of professionals who are seeking to learn more about DE, apply it to their work, and to advance thinking and scholarship on the idea. We are the true-believers. At the same time, we are evaluators, policy makers, and program leaders who are pragmatic and see that DE isn’t for every situation and indeed, there are many situations where DE is not appropriate.

In his book on the subject, Michael Quinn Patton outlines a few questions that can help frame questions to ask when considering whether DE is a suitable approach — particularly with respect to when you hope to use the results:

  • What is being developed now?
  • What do the results of what has been developed mean now?
  • What are the next steps now? What is the next stage of development now?

These simple questions can help users of DE determine whether it is appropriate. In particular, the bigger question is: are you able to use information gleaned from DE now? If not, then it might be you are not wrestling with a developmental issue, but one more suited to formative or summative evaluation. You may want to know whether a program worked or how it worked, not necessarily how it is developing.

One of the analogies that I find fits DE is that of purchasing something on the Internet. Yes, you can read all about the thing, consider the specs on its dimensions, and see a picture, but when it arrives to your door there is often a surprise when you open the box and are actually holding it , seeing it before you. Sometimes that is a pleasant surprise, sometimes not, or maybe its just different, but quite often it is not the same as what you thought it was when you clicked “buy” online.

DE is truly suited to situations where you seek to develop a program, evolve it, and adapt it to changing circumstances. It requires a different mindframe around the problem context and the goals of the program than the ordinary one that is focused on growth, change over time and improvement. A development is an adaptation, while an improvement is a judgement based on pre-established criteria. Because DE is suited to programs operating in great complexity, establishing the context in advance (i.e., creating controls in traditional research parlance) is not possible.

For those looking to innovate, to evolve their programs operating in complex conditions, and to learn-as-you-go in a manner that enables on-the-fly decision making, then DE is for you. If you want evaluators to go off, bring the data, and give you results, you might want to try something else.

** Photo package! by uzbeckistan used under a Creative Commons License from Flickr

Systems Thinking and Design: A Case for Egypt?

Posted: January 30, 2011 | Author: | Filed under: art & design, complexity, design thinking, evaluation, Systems science, systems thinking | Tags: , , , , , , | 1 Comment »

Politics provides a great analogy for why systems thinking and design fit together and how effective “design” and systems thinking work so closely together. It’s time that our politicians and policy makers start considering the role of design and systems thinking a little more and Egypt provides a great example of what happens when those areas come together.

Designing (Building) a New Egypt?

Over the past month I’ve done a lot of reading on the role of design and the culture of designers. The reasons are many, but mostly because I see the challenges that we face as a society as ones of poor design and an inability to see systems, think about them clearly, and translate that into action. This last part is really where design comes into play.

Take the current situation in Egypt, that is in many ways a design problem. Many years ago, Egyptians were comfortable — if not always happy — to accept a government designed to be under the heavy influence of one person. Despite the flaws of that leadership, there was acceptance and general, if somewhat muted, support for that model for the last part of the 20th century and the early start of this century. The recent events in Tunisia showed Egyptians that there were alternatives to their current model of politics and that the people could design a new leadership. The past few days have seen a remarkable chain of events that represent the culmination of desire of Egypt’s people for change.

Design thinking provides a lens for viewing problems and developing contextual solutions — or new situations. The exploration of the problem and developing new solutions using design thinking involves a number of steps. This usually includes such steps as:

  • Define the scope, scale and context of the problem at hand;
  • Research the problem and determine causes, consequences, alternatives and opportunities related to that problem;
  • Start working on developing possible options — ideate — in ways that include the wild and outlandish to ensure that there is sufficient opportunity to build on ideas that might encompass the fullest possible perspective on the issue, even if such ideas may seem impractical;
  • Prototype some new options. In the case of political systems, perhaps trying some new ways to organize parts of the government, shift the leadership structure or conduct local experiments to try new models of governance in relatively safe environments;
  • Evaluate the implementation of the prototype and incorporate the findings into successive models and then re-implement them in the form of new prototypes. This rapid-cycle prototyping on small scale experiments enable a safe-fail culture to form rather than aim for the impractical fail-safe models that almost never work in complex systems;
  • Implement and repeat. Take the lessons learned from taking a utilization-focused evaluation (PDF) approach or a developmental evaluation approach (or a combination), implement the necessary changes and repeat.

The manner by which the outcomes and implementation of the new model are assessed can be viewed by taking a systems thinking perspective, largely because what is being designed is indeed a system. By considering the boundaries, the interconnections between civil society and government, and by articulating intention in guiding the change, it is possible to design a new political system for one of the world’s oldest societies in a manner that honours the past and creates a compelling, healthy future.

Change isn’t usually so straightforward, but that’s often because there isn’t the planning and process in place for change. Now is a perfect time to bring that about as Egyptians struggle along with their Northern African neighbours in Tunisia to find ways to bring their intention to bear on the way their country is governed and, in doing so, create one of the most significant opportunities for design and systems thinking in either of those nations’ histories.

The Problem With Grades

Posted: January 7, 2011 | Author: | Filed under: complexity, Education & Learning, public health | Tags: , , , , , , , , , | 6 Comments »

Good little factory workers (Photo CC Flicker Michael 1952)

Grading is the tyrrany of higher education and this week I had to face it full-force and get reminded why we value the appearance of education rather than true learning.

This week I submitted the final grades for my graduate course on health behaviour change. Submitting grades is always an emotional time for me. I’ve watched students do poorly not due to lack of understanding, but circumstance. I’ve also seen students turn things around after having started out slow and ending on a high note. In every case, I end up assigning a letter (sometimes with a + or – attached to it) to assess the quality of the work done, which is supposed to be a proxy for learning. The truth is far from that.

More than any other semester in my teaching history I found myself struggling with grading. Grades are holdovers from a system designed to produce good little factory workers who would have enough knowledge not to hurt themselves and do the job right, but not quite enough to truly challenge the system that said they had to work the way that they did. Unfortunately, old habits are hard to break (which is, ironically perhaps, the theme of the course I teach).

I am fortunate enough to have a room to teach with moveable chairs and tables, although there is really only a few designs open to me given that the room is literally filled to the capacity set by the fire marshall. It’s still better than the circumstance illustrated in the above photo, with students sitting in rows all looking the same. I am pleased that many of the students in my course call me “Cameron” and not just “Dr. Norman” or “Professor”. We’ve made a lot of progress, but at the same time there is much illusion about the nature of education today.

A wonderful illustration of the problems of education and its historical roots can be viewed as part of a TED talk summary by Sir Ken Robinson.

I teach up to 30 graduate students — both masters and doctoral level students — at a time. When you pile 30+ learners (including the TA’s, guest lecturers, guest students) into one room, the type of teaching and learning you are able to do is seriously limited by the size of the class, the room, the complexity of the material being presented, and the time you have available to explore that material. I do my best and the students do theirs, but it is limited. And yet, we call this graduate education. We call it education; period.

It is as if individuals have no prior experience of their own and couldn’t possibly add to the discussion in any meaningful way. As such, we set up a system of evaluation that suggest that I, as the implied smartest person in the room, can truly judge the worthiness of any idea with complete objectivity, precision, and efficiency and that is worth something. Well, when it comes to health behaviour change or systems thinking (the two courses I teach) I can confidently say that I have more codified, structured, academically acceptable knowledge than any one person in my classroom. But do I have more than the class combined? No way. I’m not even close.

So it would surmise that some method of tapping into that knowledge of the 10, 20, or 30 students is a good idea. But doing so means acknowledging that the professor — the said “smart person” in the room — might not have all the answers and maybe some of the students have those answers. Or in the case of complex and novel problems that we see in public health more often these days, maybe no one has the answer. Maybe the answer needs to be generated by collaboration, discussion and bringing diverse groups together.

But what does this mean for grading? If five people help derive a solution to a problem, who gets the grade? Some models might suggest the leader gets more credit than the others as we see in the academic peer-reviewed publication traditions . While others might have some form of negotiated hierarchy of authors. A systems thinking perspective might throw the whole authorship issue out altogether because it was the contribution of the team acting as such that generated the knowledge. Yes, some may have worked longer hours, taken bigger roles, but the entire product is the sum of the whole of the parts, therefore every component is considered vital. If we took this into account, we’d have to award the same grade to everyone if  producing knowledge that was useful was the goal of the activity.

That’s pretty heretical stuff where I come from. Yes, it is true we can have group grades, but this is speaking to a fundamental issue of contribution and acknowledging that not everyone will add the same value and that is OK, because in the end it is what people add in its totality that is most important the whole.

In complexity terms, grading is anathema. It suggests that we can know what is “right” and “wrong” and “effective” and “ineffective” in each circumstance. In simple systems, that might be true. When we have “best practice” that is reliable and valid and can be assessed consistently, then grades are perfectly reasonable. Yet, when we work in spaces where the context changes, the variables multiply and shift, and the outcomes can, at best, be anticipated but not predicted, the idea of assessing people based on concrete, objective standards seems silly at best, dangerous at worst. But that’s what we do all the time.

Complexity does endorse — and indeed, thrives on — feedback. Getting some form of assessment is great so long as it is  provides opportunities for adaptation. Without it, complex systems would become simple ones.

Imagine a system where we gave students feedback, allowed them to adapt, and to take the information they learn and apply it in ways that fit the context they are working in? Consider what that might look like in terms of grades and grading and how the absence of such almost arbitrary assessments could lead to knowledge that could truly advance the health and wellbeing of everyone, not just propose to do so.

This is not just systems thinking, but true systems change and that is what education is all about in my books.

The Science of Design & the Design of Science

Posted: December 21, 2010 | Author: | Filed under: behaviour change, design thinking, evaluation, research, Science & technology | Tags: , , , , , , | 2 Comments »

Glasgow Science Centre (by bruce89, used under Creative Commons Licence)

As the holidays approach I’ve been spending an increasing amount of time looking at a field that has become my passion: design. Design is relevant to my work in part because it frequently deals with the complex, requires excellent communication, and as Herbert Simon would suggest, is all about those interest in changing existing situations into preferred ones.

Yet for all the creativity, innovation and practicality that design has I find it lacking in a certain scientific rigour that it requires to gain the widespread acceptance it deserves.

This is not to say that designers do not employ rigorous methods or that there is no science informing design. For example, architecture, a field where design is embedded and entwined, employs high levels of both rigour and science in its practice. The issue isn’t that these two concepts aren’t applied, they just aren’t applied to each other. I was heartened this week to see Dexigner profile a new pamphlet on the science of design. Although true in spirit, it wasn’t what I expected to see as it largely profiled ways to assess the quality of design projects from the perspective of design.

What if we could assess the impact of design on a larger scale, a social and human scale?

Interaction designers speak of this need to connect to the human in design work. The emergent field of social design exemplified by groups like Design 21 who aim to produce better products for social good. All of this is important, but it’s important largely because we say it is so. Rhetorical arguments are fine, but at some point design needs to confront the problem of evidence.

Does “good” design lead to better products than “bad” design?

What components of design thinking are best suited to addressing certain kinds of problems? Or are there simply problems that design thinking is just better at addressing than other ways of approaching them?

What methods of learning produce effective design thinkers? And what is effective design thinking anyway? Does it exist?

What is the comparative advantage of a design-forward approach to addressing complex problems than one where design is less articulated or not at all?

These are just some of the many questions that there seems to be little evidence in support of. A scientific approach to design might be one of the first ways of addressing this. In doing so, a scientifically-grounded design field is far more likely to garner support of decision makers who are the ones who will approve and fund the kind of projects that can have wide-scale impact. Design is making serious in-roads to fields such as business, education, and health, but it represents a niche market when it has the potential to be much larger.

Roger Martin has argued that the reliance on scientific approaches to problem solving runs counter to much of design thinking. This assumes that science is applied in a very detached, prescriptive manner, which is common, but not the only way. Micheal Gibbons and colleagues have described two forms of science, which they call Mode I and Mode II science. The first Mode is the one that most people think of when they hear the term “scientist”. It is of the (usually) lone researcher working in a lab on problems that are driven by curiosity with the aim of generating discoveries. For this reason, it is often referred to as discovery-oriented research.

Mode 2 research is designed to be problem-centred and aimed at answering questions posed by practical issues and has a strong emphasis on knowledge translation. This is an area more accustomed to the designer.

Design presents the opportunity to transcend both of these Modes into something akin to Mode 3 research, which I surmise is a blend of the abductive reasoning inherent in Roger Martin’s view of design thinking and the discovery-oriented approach that goes beyond just the problem to create value beyond the contracted issue. A design-oriented approach to the science of design would involve leveraging the creative processes of designers with some of the tools and methods accustomed to researchers in Mode 1 and 2 science. Can we not do detailed ethnographic studies looking at the process of design itself? Is there any reason why we cannot, with limits acknowledged and in appropriate contexts, attempt to do randomized controlled trials looking at certain design thinking activities and situations?

If design is to make a leap beyond niche market situations, a new field must dawn within design + science and that is the science of design and the design of science.

More Design Thinking & Evaluation

Posted: November 15, 2010 | Author: | Filed under: design thinking, Education & Learning, evaluation, innovation | Tags: , , , , | Leave a comment »

Capturing Design in Evaluation (CameraNight by Dream Sky, Used under Creative Commons License)

On the last day of the American Evaluation Association conference, which wrapped up on Saturday, I participated in an interactive session on design thinking and evaluation by a group from the Savannah College of Art and Design.

One of the first things that was presented was some of the language of design thinking for those in the audience who are not accustomed to this way of approaching problems (which I suspect was most of those in attendance).

At the heart of this was the importance of praxis, which is the link between theory, design principles and practice (see below)

Design Thinking

As part of this perspective is the belief that design is less a field about the creation of things on their own, but rather a problem solving discipline.

Design is a problem solving discipline

When conceived of this way, it becomes easier to see why design thinking is so important and more than a passing fad. Inherent in this way of thinking are principles that demand inclusion of multiple perspectives on the problem, collaborative ideation, and purposeful wandering through a subject matter.

Another is to view design as serious play to support learning.

Imagine taking the perspective of design as serious play to support learning?

The presenters also introduced a quote from Bruce Mau, which I will inaccurately capture here, but is akin to this:

One of the revelations in the studio is that life is something that we create every single day. We create our space and place.

Within this approach is a shift from sympathy with others in the world, to empathy. It is less about evaluating the world, but rather engaging with it to come up with new insights that can inform its further development. This is really a nod (in my view) to developmental evaluation.

The audience was enthralled and engaged and, I hope, willing to take the concept of design and design thinking further in their work as evaluators. In doing so, I can only hope that evaluation becomes one of the homes for design thinking beyond the realm of business and industrial arts.

← Older Entries
Follow

Get every new post delivered to your Inbox.

Join 1,313 other followers