The Importance of Journalism to Public Health: 10 Years After SARS How Are We Doing?
Posted: April 4, 2013 Filed under: public health, social media, systems thinking | Tags: developmental design, developmental evaluation, evidence, health communications, health reporting, Helen Branswell, journalism, Julie Leask, media, public health, social media, systems thinking, Wayne MacPhail 2 Comments »
Risk communication in public health with Julie Leask
If a health scare manifested itself in the world and there were no journalists to cover the story, what would the impact on the public be?
That is a question that lingered with me throughout the start of the 2013 Ontario Public Health Convention (TOPHC) which began with a morning dedicated to improving public health communication. Opening up the conference was a series of linked keynote presentations from a risk communications researcher (Julie Leask); a former newspaper editor, journalism professor and social media advocate (Wayne MacPhail), and one of Canada’s leading health specialist reporters (Helen Branswell).
The Academic’s Perspective
Keynote speaker Julie Leask (pictured above) and her colleague Dr. Claire Hooker (a good friend of mine) have been looking at the ways journalists engage in risk communication with the public on matters of public health from immunization to SARS to understanding the health priorities of professionals. In 2010 they published a paper looking at how the media covers health topics and argued that the health professions need to be aware of how stories are made, communicated and to be an active partner with reporters if they are to have positive impact in moments of health scares.
“It’s too late when the crisis comes up” – Julie Leask speaking on the need for public health to get engaged with the public using social media
In a previous post I wrote about how journalism is the fourth estate of medicine and public health. Journalists are the storytellers that the public listen to and are charged with looking at a problem from many perspectives to develop that coherent narrative that speaks to their audience. These are qualities that most scientists and public health professionals don’t bring to their jobs, nor are they always expected to or even should. As such, journalists play an important role for this very reason.
Nonetheless, the health sector has an uneasy relationship with journalism. Health professionals – particularly researchers — poorly understand the world of journalists and sometimes view the profession with suspicion. Julie Leask and her colleagues have found this to be the case, but argue that it is no reason to shy away from engaging the public using the tools that are comfortable to journalists. She spoke to the invaluable role of specialist health journalists in acting not only as producers of high quality health content in the news, but also guardians against low quality content making into press. In speaking to her research, she pointed out that specialist health journalists help educate their peers and editors on health issues, which are often complex and require more than a passing understanding of context to communicate well, as key gatekeepers for quality in the health landscape.
The Editor’s Perspective
To this end, Wayne MacPhail, a former editor of the Hamilton Spectator, argued that public health has a near ethical imperative (my choice of term) to be in the social media space to not only promote good health, but counter and challenge myths and misinformation. This isn’t some naive pronouncement that we’ll eliminate the snake oil sales or quackery that proliferates in the public sphere and media, but rather a simple observation that we have no chance of making impact if we are not even engaged in the space at all.
Like Leask, MacPhail says that it’s too late to engage the public when a health crisis comes up and that public health needs to be in the conversation stream before that happens.
The Reporter’s Perspective
Reporting through SARS to today: Helen Branswell
Helen Branswell, a reporter from The Canadian Press, rounded out the panel and spoke frankly about the dwindling resources and rapidly changing landscape in journalism. She was on the front lines of reporting the 2003 SARS outbreak and showed a picture taken during that time of an empty newsroom and remarked how that the scene is the same now only for different reasons (limited budgets due to decreased ad revenue and the related shift to digital information on the web being two such reasons, among others).
Branswell paints a bleak picture of the present and future in many areas of health journalism. Stories are increasingly being covered by general reporters who may treat the story the same as they would a traffic incident, political story, or crime; journalists who are unlikely to know the context and details that are critical to communicating the nuances present in health matters. Interns are replacing some full time or veteran reporters in the newsroom and there are only a handful of specialists in practice.
Pressures from time, budget and competing interests in the newsroom are all contributing to an environment where quality health reporting is threatened.
What Next?
I asked the panel what they thought public health should do to ensure that the healthy stories are reported well and there was little answers. Helen Branswell said, truthfully and somewhat cheekily: “buy newspapers”. She reminded us that we should be paying for the quality content and supporting good journalism in practice if we want it to survive, which is hard to argue against.
But that alone will not do all the work needed to preserve good journalism. I spoke to another conference attendee, a formally trained journalist who is now working with a research firm, about the ways in which journalists have helped other organizations craft their messages and engaging the public citing the Calgary Police Service’s social media team as an example. This pointed to ways in which journalists can make a difference in matters of public health and social services.
Yet, what about investigative journalism? What about the potential conflicts that come from being paid to report on issues that might be critical of the organization who does the paying (e.g., Ministries of Health, Departments of Public Health, Universities and colleges etc..)? This model doesn’t solve that, but it is at least another option.
Yet, the examples from public health taking this challenge of working with journalists up are few. Many still believe that social media is another means of broadcasting, which misses the mark. Others still view social media, journalism, engaging with the public through the media, with suspicion on the grounds that much of the work out there is not evidence based.
But what evidence did we have when SARS hit us 10 years ago? We had lots of epidemiological data on infectious disease, but that was only part of the story. Many of the leading health scientists were adapting their models, creating new ones and only after the disease left did we really have a full sense of what happened. We learned as we went.
This is what social media is all about, too. The lessons from major health events — disasters, outbreaks, and pandemics — parallel social media. It is innovation space at its clearest and thus there is an imperative to view it as innovation space with the tools and lenses that best support movement within complex adaptive system. From a communications standpoint, social media and the tools of modern journalism (and the style of communication they employ) are one thing to consider. Developmental design and evaluation are also among these tools combined with systems thinking.
Linear thinking and action will not work in a complex system and as this panel pointed out, there is much reason to be concerned if we are not prepared to communicate and support those that communicate well in such times when — not if — they come back.
Ten years after SARS how better off are we? And if we are better, how are we communicating that to the public?
Evaluating Social Innovation For Social Impact
Posted: November 15, 2012 Filed under: complexity, design thinking, education & learning, emergence, evaluation, knowledge translation, systems thinking | Tags: design, design thinking, developmental design, developmental evaluation, evaluation, evidence, Helsinki Design Lab, knowledge exchange, knowledge translation, research design, research methods, social impact, social innovation 1 Comment »
How do the innovation letters line up?
Earlier this week I has the pleasure of attending talks from Bryan Boyer from the Helsinki Design Lab and learning about the remarkable work they are doing in applying design to government and community life in Finland. While the focus of the audience for the talks was on their application of design thinking, I found myself drawn to the issue of evaluation and the discussion around that when it came up.
One of the points raised was that design teams are often working with constraints that emphasize the designed product, rather than its extended outcome, making evaluation a challenge to adequately resource. Evaluation is not a term that frequents discussion on design, but as the moderator of one talk suggested, maybe it should.
I can’t agree more.
Design and Evaluation: A Natural Partnership
It has puzzled me to no end that we have these emergent fields of practice aimed at social good – social finance and social impact investing, social innovation, social benefit (PDF)– that have little built into their culture to assess what kind of influence they are having beyond the basics. Yet, social innovation is rarely about simple basics, it’s influence is likely far larger, for better or worse.
What is the impact being invested in? What is the new thing being created of value? and what is the benefit and for whom? What else happened because we intervened?
Evaluation is often the last thing to go into a program budget (along with knowledge translation and exchange activities) and the first thing to get cut (along with the aforementioned KTE work) when things go wrong or budgets get tightened. Regrettably, our desire to act supersedes our desire to understand the implication of those actions. It is based on a fundamental idea that we know what we are doing and can predict its outcomes.
Yet, with social innovation, we are often doing things for the first time, or combining known elements into an unknown corpus, or repurposing existing knowledge/skills/tools into new settings and situations. This is the innovation part. Novelty is pervasive and with that comes opportunities for learning as well as the potential for us to good as well as harm.
An Ethical Imperative?
There are reasons beyond product quality and accountability that one should take evaluation and strategic design for social innovation seriously.
Design thinking involves embracing failure (e.g, fail often to succeed sooner is the mantra espoused by product design firm IDEO) as a means of testing ideas and prototyping possible outcomes to generate an ideal fit. This is ideal for ideas and products that can be isolated from their environment safely to measure the variables associated with outcomes, if considered. This works well with benign issues, but can get more problematic when such interventions are aimed at the social sphere.
Unlike technological failures in the lab, innovations involving people do have costs. Clinical intervention trials go through a series of phases — preclinical through five stages to post-testing — to test their impact, gradually and cautiously scaling up with detailed data collection and analysis accompanying each step and its still not perfect. Medical reporter Julia Belluz and I recently discussed this issue with students at the University of Toronto as part of a workshop on evidence and noted that as complexity increases with the subject matter, the ability to rely on controlled studies decreases.
Complexity is typically the space where much of social innovation inhabits.
As the social realm — our communities, organizations and even global enterprises — is our lab, our interventions impact people ‘out of the gate’ and because this occurs in an inherently a complex environment, I argue that the imperative to evaluate and share what is known about what we produce is critical if we are to innovate safely as well as effectively. Alas, we are far from that in social innovation.
Barriers and Opportunities for Evaluation-powered Social Innovation
There are a series of issues that permeate through the social innovation sector in its current form that require addressing if we are to better understand our impact.
- Becoming more than “the ideas people”: I heard this phrased used at Bryan Boyer’s talk hosted by the Social Innovation Generation group at MaRS. The moderator for the talk commented on how she had wished she’d taken more interest in statistics in university because they would have helped in assessing some of the impact fo the work done in social innovation. There is a strong push for ideas in social innovation, but perhaps we should also include those that know how to make sense and evaluate those ideas in our stable of talent and required skillsets for design teams.
- Guiding Theories & Methods: Having good ideas is one thing, implementing them is another. But tying them both together is the role of theory and models. Theories are hypotheses about the way things happen based on evidence, experience, and imagination. Strategic designers and social innovators rarely refer to theory in their presentations or work. I have little doubt that there are some theories being used by these designers, but they are implicit, not explicit, thus remaining unevaluable and untestable or challenged by others. Some, like Frances Westley, have made theories guiding her work explicit, but this is a rarity. Social theory, behaviour change models and theories of discovery beyond just use of Rogers’ Diffusion of Innovation theory must be introduced to our work if we are to make better judgements about social innovation programs and assess their impact. Indeed, we need the kind of scholarship that applies theory and builds it as part of the culture of social innovation.
- Problem scope and methodological challenges with it. Scoping social innovation is immensely wide and complicated task requiring methods and tools that go beyond simple regression models or observational techniques. Evaluators working social innovation require a high-level understanding of diverse methods and I would argue cannot be comfortable in only one tradition of methods unless they are part of a diverse team of evaluation professionals, something that is costly and resource intensive. Those working in social innovation need to live the very credo of constant innovation in methods, tools and mindsets if they are to be effective at managing the changing conditions in social innovation and strategic design. This is not a field for the methodologically disinterested.
- Low attendance to rigor and documentation. When social innovators and strategic designers do assess impact, too often there is a low attention to methodological rigor. Ethnographies are presented with little attention to sampling and selection or data combination, statistics are used sparingly, and connections to theory or historical precedent are absent. Of course, there are exceptions, but this is hardly the rule. Building a culture of innovation within the field relies on the ability to take quality information from one context and apply it to another critically and if that information is absent, incomplete or of poor quality the possibility for effective communication between projects and settings diminishes.
- Knowledge translation in social innovation. There are few fora to share what we know in the kind of depth that is necessary to advance deep understanding of social innovation, regularly. There are a lot of one-off events, but few regular conferences or societies where social innovation is discussed and shared systematically. Design conferences tend towards the ‘sage on the stage’ model that favours high profile speakers and agencies, while academic conferences favour research that is less applied or action-oriented. Couple that with the problem of client-consultant work that is common in social innovation areas and we get knowledge that is protected, privileged or often there is little incentive to add a KT component to the budget.
- Poor cataloguing of research. To the last point, we have no formalized methods of determining the state-of-the-art in social innovation as research and practice is not catalogued. Groups like the Helsinki Design Lab and Social Innovation Generation with their vigorous attention to dissemination are the exception, not the rule. Complicating matters is the interdisciplinary nature of social innovation. Where does one search for social innovation knowledge? What are the keywords? Innovation is not a good one (too general), yet neither is the more specialized disciplinary terms like economics, psychology, geography, engineering, finance, enterprise, or health. Without a shared nomenclature and networks to develop such a project the knowledge that is made public is often left to the realm of unknown unknowns.
Moving forward, the challenge for social innovation is to find ways to make what it does more accessible to those beyond its current field of practice. Evaluation is one way to do this, but in pursuing such a course, the field needs to create space for evaluation to take place. Interestingly, FSG and the Center for Evaluation Innovation in the U.S. recently delivered a webinar on evaluating social innovation with the principle focus being on developmental evaluation, something I’ve written about at length.
Developmental evaluation is one approach, but as noted in the webinar : an organization needs to be a learning organization for this approach to work.
The question that I am left with is: is social innovation serious about social impact? If it is, how will it know it achieved it without evaluation?
And to echo my previous post: if we believe learning is essential to strategic design we must ask: How serious are we about learning?
Tough questions, but the answers might illuminate the way forward to understanding social impact in social innovation.
* Photo credit from Deviant Art innovation_by_genlau.jpg used under Creative Commons Licence.
The Ideology of Scaling Social Innovations
Posted: August 24, 2012 Filed under: complexity, design thinking, innovation, social systems, systems science, systems thinking | Tags: Charles and Ray Eames, complexity, design, developmental design, efficiency, evaluation, ideology, Jamer Hunt, Janice Gross Stein, Power of 10, scalar variance, six sigma, social innovation 1 Comment »
Does it scale? That question is central to the discussion of social innovation, yet the answer to it might lead us to questions about why it is so important to us in the first place and answers that could surprise us.
“Does it scale?” or “how to we take [idea, product, service] to scale?” are commonly heard questions in social innovation circles; so much so that they are left unquestioned. The thinking behind these questions is that if something works well at one level (or scale) then taking it another scale larger and achieving a wider reach must be better. Who wouldn’t want to see the benefits of something that serves the needs of one population, community or user extended outward and upward?
This is a laudable utilitarian goal, but it is a deceptively problematic one when we look a little closer at what scaling something actually means in practice.
Conceptualizing Scale
Jamer Hunt, the Director of the MFA program in Transdisciplinary Design at the New School in New York, speaking at last year’s DMI Fall Conference (which is available to view for DMI members), looked at the issue of design scaling through the lens of complexity and pointed to some of the problems with ‘scaling design’ in varied contexts. One of the examples he suggested is that of an ant compared with a human being taking a shower. For humans, the shower’s droplets of water are fine bodies of liquid that perform a particular task of facilitating cleaning, but for an ant those same droplets are enormous orbs of potential death. Water doesn’t scale the same for a human and an ant even though it is the same substance at both levels and the shower is identical in its structure.
In physics this is called scalar variance. What works ideally for humans is terrible for ants even though we are speaking of the same substance, same planet, same context. Water (most notably, a shower of it) doesn’t scale well in this case.
Yet, there is this insatiable desire among those working in social innovation to “scale things up” and “bring our innovations to scale” (even if we have little concept of how that would look or — as I will discuss — what that really means). The adherence to scaling as an ideology in social innovation (and applied social science in general) is bordering on “four legs good, two legs better” territory.
The Cult of Efficiency
International affairs scholar Janice Gross Stein attributes some of this fascination with scaling to a cult of efficiency, a political ideology that assumes that we can always rationalize human services optimally. What she found is that efficiency is used falsely as a stand-in for accountability, particularly in fields like education. Far from being against striving for optimal use of scarce resources, Stein nonetheless concludes that efficiency in human systems doesn’t always scale (my phrase, not hers) and that bigger and faster is often not better. Anyone who has taken a lecture with hundreds of others knows the difference of scale in learning between that and a seminar of five to ten people.
Taking Jamer Hunt’s argument: Bigger is just bigger…and whether its better or not is dependent on whether you’re an ant, a human and need to come into contact with water.
Designing for Systems and Scale: The Powers of 10
Designers and systems thinkers probably know the movie “The Powers of 10” by legendary designers Charles and Ray Eames. It’s a fascinating short film that looks at the universe moving out from a human being into the cosmos and inward towards what would now be quarks and everything in between. It is perhaps the best example of scaling ever produced. Beyond its educational and entertainment value, the Powers of 10 provide an illustrative example of where striving for scaling social innovations could be foolish and where it could have potential.
When traveling through the universe it is easy to see scales that are self-similar, thus they share properties that make them optimally relatable. These forms are often fractal in nature (thus, they share the same properties at different scales like that of a snowflake). Imperfectly, certain scales in the Powers of 10 are close to self-similarity where one scale looks and shows behaviour similar to those adjacent to it. These are spaces where it may be possible to transport an innovation from one to the other to good effect. Others scales look radically different from one another, suggesting a mis-fit in the scalar variance.
This is an idea, not an empirical point as we have little research on scalar variance in social innovation. Scaling innovation makes greater sense when the social systems have similar structures and ‘shapes’ and less when they do not. It is why in organizational science, certain models of management and decision making transport well from setting to setting and others do not. It’s why we’ve seen quality improvement processes like Six Sigma achieve great success in certain industries and firms and spectacularly fail in others.
Rather than adhere to an ideology that imposes scaling as a goal, social innovators need to generate the kinds of intelligence about the systems they are operating in (or seeking to operate or expand into) before making plans for scaling a promising intervention or product. As funders and policymakers this means setting performance targets that are appropriate or, perhaps better yet, working developmentally with innovators to co-create the outcomes of interest and the measures and metrics used to determine scalability and appropriateness early in the design and implementation cycle.
Without best evidence (which is almost always lacking in social innovation by its very nature), setting performance targets related to scale a priori is foolish. For innovators themselves, equally foolish is not gathering the kind of information about the systems they are operating in to know if they are the human or the ant and whether a shower is on the way.
The Forward Orientation Problem With Complexity
Posted: June 26, 2012 Filed under: complexity, evaluation, social systems, systems science | Tags: complexity, contemplative inquiry, developmental design, developmental evaluation, mindfulness, mindset, perception, program planning, science, social innovation, time 6 Comments »
The human body is oriented towards forward motion and so too are our institutions, yet while this helps us move linearly and efficiently from place to place, it may obscure opportunities and challenges that come from other directions such as those posed by complexity. Thinking about and re-orienting our perceptions of who we are and where we are going might be the key to understanding and dealing with complexity now and in the future.
When heading out into the turbulent waters that face us we humans tend to look straight ahead and press forward. Our entire physical being and that of all mammals is aimed at facing forward. We look forward, walk forward and this often means thinking forward.
Doing this predisposes us to seeing problems ahead of us or behind us, but is less useful when what challenges us is positioned elsewhere. For this reason, fish and birds, with their eyes on the side of their head, are able to adapt to challenges from nearly any direction quickly. It also allows them to fly/swim in flocks/swarms/schools and operate with high degrees of coordination on a large scale.
These are skills that are useful for handling the social problems that are complex in nature and require mass action to address. But, we don’t have eyes on the side of our head and we tend to look forward or backward to orient ourselves and our activities.
One way this expresses itself in our perceptions of time. Thor Muller, writing in Psychology Today online, highlighted how our perceptions of time influence the way we handle appointments and punctuality with modern technology. Citing the work of anthropologist Edward T. Hall (although mistakenly referring to Manhattan Project contributor Edward Teller), Muller points to the differences in perceived time across cultures and the way that plays out in our treatment of time and technology used to “manage” it and the complexity of everyday life. Monochronistic and polychronistic time orientations matter to whether you see time as a linear, quantifiable phenomenon or a more non-linear, contextual one. One allows you to “bank” time while the other perception deals more with the present moment, less dependent on forward-backward thinking.
Western society and the technologies developed within it are oriented primarily towards dealing with a monochronistic form of time. This works well when patterns, problems and situations have a linear, ordered set of circumstances to them. The cause-and-effect world of normal science fits within this worldview.
Complexity is non-linear and not easily defined in cause-and-effect terms and conditions. Two-dimensional space doesn’t capture complexity the way it can for linear situations. It also means thinking solely in forward and back terms is problematic.
An example of where this comes to conflict is in program planning and evaluation. Traditional evaluation methods and metrics are set up for looking at programs that are planned to start and end with impacts developed and detected in between. This implies a certain level of consistency in the conditions in which that program operates. This control and measure aspect of evaluation is part of the hallmark features of scientific inquiry.
For programs operating in environments of great change and flux, this is a faulty proposition. We cannot hold constant the environment for starters. Secondly, feedback gained from learning about the program as it proceeds is critical to ensuring adaptation and promoting resilience in the face of changing conditions. In these cases, failure to act and adapt on the go may result in a program failing catastrophically.
This is where developmental evaluation comes in. Developmental evaluation works with these conditions to generate data in a manner that programs can make sense of and use to facilitate strategic adaptation rather than simply reacting to changes. As the name suggests, it promotes development rather than improvement.Developmental design is the incorporation of this feedback into an ongoing program development and design process.
Both developmental design and evaluation require ways of seeing the world beyond forward/backward. This seeing comes from understanding where one’s position is in the first place and that requires methods of centring that take us into the world of polychronistic time. One example of a strategy that suits this approach is mindfulness programming. Mindfulness-based programs have shown remarkable efficacy in healing and health interventions aimed at stress reduction across conditions. Mindfulness techniques ranging from meditation to contemplative inquiry (video) brings focus to the present moment away from an orientation towards linear trajectories of time, thought and attention.
Some forms of martial arts promote attentive awareness to the present moment by training practitioners in strategies that are focused on simple rules of engagement, rather than just learning techniques for defence.
These approaches combine inward reflection — reflective practice — with an openness to the data that comes in around them without imposing an order on it a priori. The orientation is to the data and the lessons that come from it rather than its directionality or imposing values on what the data might mean at the start. It means slowing down, contemplating things, and acting on reflection not reacting based on protocol. This is a fundamental shift for many of our activities, but may be the most necessary thing we can focus on if we are to have any hope of understanding, dealing with, and adapting to complexity.
All the methods and tools at our disposal will not help if we cannot change our mindset and orientation — even in the temporary — to this reality when looking at complexity in our work. One of complexity’s biggest challenges right now is that it is seductive in accounting for the massive, dynamic sets of conditions we face every day, yet it lacks methods beyond evaluation to do things with it. The irony of mindfulness and contemplative approaches is that they are less about acting differently and more about seeing things in new ways, yet it is that orientation that is the key to making real change from talking about change. It is the design doing that comes with design thinking and the systems change from systems thinking.
Evaluation and Design For Changing Conditions
Posted: June 5, 2012 Filed under: complexity, design thinking, emergence, evaluation, innovation, systems thinking | Tags: complexity, developmental design, developmental evaluation, evaluation, IDEO, measurement, Michael Quinn Patton, social innovation, Tim Brown, utilization-focused evaluation 3 Comments »
Growth and Development
The days of creating programs, products and services and setting them loose on the world are coming to a close posing challenges to the models we use for designing and evaluation. Adding the term ‘developmental’ to both of these concepts with an accompanying shift in mindset can provide options moving forward in these times of great complexity.
We’re at the tail end of a revolution in product and service design that has generated some remarkable benefits for society (and its share of problems), creating the very objects that often define our work (e.g., computers). However, we are in an age of interconnectedness and ever-expanding complexity. Our disciplinary structures are modifying themselves, “wicked problems” are less rare
Developmental Thinking
At the root of the problem is the concept of developmental thought. A critical mistake made in comparative analysis — whether through data or rhetoric — is one that mistakenly views static things to moving things through the same lens. Take for example a tree and a table. Both are made of wood (maybe the same type of wood), yet their developmental trajectories are enormously different.
Wood > Tree
Wood > Table
Tables are relatively static. They may get scratched, painted, re-finished, or modified slightly, but their inherent form, structure and content is likely to remain constant over time. The tree is also made of wood, but will grow larger, may lose branches and gain others; it will interact with the environment providing homes for animals, hiding spaces or swings for small children; bear fruit (or pollen); change leaves; grow around things, yet also maintain some structural integrity that would allow a person to come back after 10 years and recognize that the tree looks similar.
It changes and it interacts with its environment. If it is a banyan tree or an oak, this interaction might take place very slowly, however if it is bamboo that same interaction might take place over a shorter time frame.
If you were to take the antique table shown above, take its measurements and record its qualities and come back 20 years later, you will likely see an object that looks remarkably similar to the one you lefty. The time of initial observation was minimally relevant to the when the second observation was made. The manner by which the table was used will have some effect on these observations, but to a matter of degree the fundamental look and structure is likely to remain consistent.
However, if we were to do the same with the tree, things could look wildly different. If the tree was a sapling, coming back 20 years might find an object that is 2,3,4 times larger in size. If the tree was 120 years old, the differences might be minimal. It’s species, growing conditions and context matters a great deal.
Design for Development / Developmental Design
In social systems and particularly ones operating with great complexity, models of creating programs, policies and products that simply release into the world like a table are becoming anachronistic. Tables work for simple tasks and sometimes complicated ones, but not complex ones (at least, consistently). It is in those areas that we need to consider the tree as a more appropriate model. However, in human systems these “trees” are designed — we create the social world, the policies, the programs and the products, thus design thinking is relevant and appropriate for those seeking to influence our world.
Yet, we need to go even further. Designing tables means creating a product and setting it loose. Designing for trees means constantly adapting and changing along the way. It is what I call developmental design. Tim Brown, the CEO of IDEO and one of the leading proponents of design thinking, has started to consider the role of design and complexity as well. Writing in the current issue of Rotman Magazine, Brown argues that designers should consider adapting their practice towards complexity. He poses six challenges:
- We should give up on the idea of designing objects and think instead about designing behaviours;
- We need to think more about how information flows;
- We must recognize that faster evolution is based on faster iteration;
- We must embrace selective emergence;
- We need to focus on fitness;
- We must accept the fact that design is never done.
Developmental Evaluation
Bringing Design and Evaluation Together
Image (Header) Growth by Rougeux
Image (Tree) Arbre en fleur by zigazou76
Image (Table) Table à ouvrage art nouveau (Musée des Beaux-Arts de Lyon) by dalbera
All used under licence.
Social Media and Health: Leaders(hip) and Followers(hip)
Posted: May 18, 2012 Filed under: behaviour change, eHealth, health promotion, innovation, public health, social media, systems thinking | Tags: complexity, creativity, design thinking, developmental design, developmental evaluation, eHealth, evaluation, Facebook, health, health promotion, healthcare, innovation, medicine, organizational change, organizational design, public health, social media, systems thinking 2 Comments »
Social media is finally catching on with healthcare, public health, and health promotion. With a few recent articles published in the academic literature to rest on, academic health sciences has finally (and I might argue, begrudgingly) conceded that 900+ million users and $100B valuations (Facebook), and thousands of messages exchanged every milisecond (microblogs like Twitter and Sina Wiebo) might have some value for the public beyond entertainment.
If you note how long it took the health sector to start using the telephone as a serious means of engaging their patients or the public, this is lightning-quick adoption. Still, the barriers to adoption are high and the approach to using the technology is scattered. Indeed, just like the start of Internet-delivered telehealth (or cybermedicine (PDF), which has now evolved into eHealth), there is a mad rush to get liked, followed or some other metrics that most health professionals barely understand.
And that is part of the problem.
Meaningful Social Media Metrics
What is a meaningful metric for social media and health? A recently published article in Health Promotion Practice suggested four metrics that are taken from social marketing and applied to social media. These Key Performance Indicators (KPI’s) are:
- Insights (consumer feedback)
- Exposure (media impressions, visits, views, etc..)
- Reach (# people who connect to the social media application)
- Engagement (level of interaction with the content)
These are reasonable, but to to the uninitiated I would suggest a few words of caution and commentary to this list.
Firstly, the insights suggested by Neiger and colleagues “can be derived from practices such as sentiment analysis or data mining that uses algorithms to extract consumer attitudes and other perspectives on a particular topic” (p.162). While not incorrect, this makes the job sound relatively simple and it is not. Qualitative analysis + quantitative metrics such as those derived from data mining are key. Context counts immeasurably in social media use. It’s only in situations where social media is used as a broadcasting tool that gross measures of likes and sentiment analysis work with little qualification.
Even that is problematic. Counts of ‘likes’, ‘visits’, ‘follows’ and such are highly problematic and can be easily gamed. I am ‘followed’ on Twitter by people who have tens of thousands of followers, yet virtually no presence online. Most often they are from marketing fields where the standard practice is to always follow back those who follow you. Do this enough and pretty quickly you, too can have 23,000 followers and follow 20,000 more. This is meaningless from the perspective of developing relationships.
Engagement is the most meaningful of these metrics and the hardest to fully apply. This category gets us to consider the difference between “OMG! AWESOME!” and “That last post made me think of this situation [described here] and I suggest you read [reference] here for more” as comments. Without understanding the context in which these are made within the post, between posts (temporally and sequentially), and in relation to a larger social and informational context, simple text analysis won’t do.
Social Media Evidence: Problems and More Problems
One of the objections to the use of social media by some is that it is not evidence-based. To that extent I would largely agree that this is the case, but then we’ve been jumping out of airplanes with parachutes despite any randomized controlled trial to prove their worth.
Another article in Health Promotion Practice in 2011 highlights potential applications for social media and behaviour change without drawing on specific examples from the literature, but rather on theoretical and rhetorical arguments. An article published in the latest issue of Perspectives on Psychological Science highlights the current state of research on Facebook, which is timely given that its IPO is set for today. That review by Wilson and colleagues illustrates the largely descriptive nature of the field and offers some insight on to the motivation of Facebook users and their online activities, but rather little in what Facebook does to promote active change in individuals and communities when they leave the platform.
The answer to whether social media like platforms such as Facebook ‘work’ as methods of promoting change is simply: we don’t know.
Does social media provide support to people? Yes. Does it inform them? Yes to that too. Does that information produce something other than passive activity on the topic? We don’t know.
In order to answer these questions, health sciences professionals, evaluators, and tech developers need to consider not just followership, but leadership. In this respect, it means creating changes to the way we gather evidence, the tools and methods we use to analyse data, and the organizational structures necessary to support the kind of real-time, rapid cycle evaluation and developmental design work necessary to make programs and evidence relevant to a changing context.
As Facebook launches into its new role as a public company it is almost assured to be introducing new innovations at a rapid pace to ensure that investor expectations (which are enormous) are met. This means that today’s Facebook will not be next month’s. Having funding mechanisms, review and approval mechanisms, a staff trained and oriented to rapid response research, and an overall organizational support system for innovation is the key.
Right now, we are a long way from that. Hospitals are very large, risk averse organizations; public health units are not much different. They both operate in a command-and-control environment suited for complicated, not complex informational and social environments. Social media is largely within the latter.
Systems thinking, design thinking, developmental evaluation, creativity, networks and innovation: these are the keywords for health in the coming years. They are as author Eric Topol calls the dawning of the creative destruction of medicine.
The public is already using social media for health and now the time has come for health (care, promotion and protection) systems to get on board and make the changes necessary to join them.
Common Sense, Complexity and Leadership
Posted: January 23, 2012 Filed under: complexity, social systems | Tags: common sense, complexity, decision making, developmental design, Duncan Watts, evaluation, leadership, social networks, strategy, systems thinking Comments Off
Bye, Bye Common Sense
Great leaders are often ascribed traits that include ample common sense. But what passes for common sense is often a grab bag of miscellaneous, inconsistent ideas that are context dependent and less useful in the complex environments where leadership is called for most.
common sense |ˌkɑmən ˈsɛns|
noun
good sense and sound judgment in practical matters: use your common sense | [ as modifier ] : a common-sense approach.
Today Research in Motion announced that its founder Mike Lazaridis and his co-CEO Jim Balsillie would be relinquishing their roles with the company. In their place, a ‘pragmatic, operational-type guy ‘was installed. Presumably, Thorsten Heins has the common sense to lead RIM after the founders lost theirs. Yet, the pragmatic, common sense that RIM is looking for might not be what they need given the complexity of the environment they are leading in.
Common sense is a false lure in complex systems. In his recent book, Everything is Obvious *Once You Know the Answer, social network researcher and Yahoo! Research scientist Duncan Watts eloquently critiques the concept of common sense, illustrating dozens of times over how “common sense” doesn’t fare so well in decisions that go beyond the routine and into the complex. Indeed. the very definition of the term implies that the problems that common sense works towards addressing are relatively simple and pragmatic.
Certainly, navigating daily social conventions might lend itself well to what we might call common sense. Watts refers to sociologist Harry Collins’ term ‘collective tacit knowledge‘ that is encoded in social norms, customs and practices of a particular world to describe common sense. However, what becomes common is a byproduct of many small decisions, dynamic and flexible changes to perspective, an accumulation of knowledge gained from small experiments over time, and the application of all of this knowledge to particular, context-dependent, situations. This constellation of factors and its interdependent, contextual overlap is why artificial intelligence systems have such a difficult time mimicking human thought and action. It is this attention to context that is most worth noting for it is this context that keeps common sense from being anything but common:
Common sense…is not so much a worldview as a grab bag of logically inconsistent, often contradictory beliefs, each of which seems right at the time but carries no guarantee of being right any other time.
Watts goes on to argue:
Commonsense reasoning, therefore, does not suffer from a single overriding limitation but rather from a combination of limitations, all of which reinforce and even disguise one another. The net result is that common sense is wonderful at making sense of the world, but not necessarily at understanding it.
Thus, we often concoct a narrative about the way something happens that sounds plausible, rational and be completely wrong. Throughout the book, Watts shows how often mistakes are made based on this common sense approach to solving problems.
When it comes to RIM, some have pointed to the late Steve Jobs’ assertion that they would have difficulty catching up to firms like Apple given that the consumer market is not their strength, the enterprise market is. Yet, Steve Jobs didn’t let the fact that Apple was a computer company stop him from making music players (the iPod), mobile phones (the iPhone) or becoming book, music and movie vendors (iTunes). A read of Steve Jobs’ biography by Walter Isaacson reveals a man who was able to lead and be successful through what appeared to be common sense, yet was decidedly uncommon among media and technology leaders. That is why Apple is where it is and why so many other technology companies lag behind them or simply disappeared.
The reason is that common sense in leadership looks as simple in hindsight only, not in foresight or even in the present moment. This is one of the big points that Watts makes. He uses the example of Sony’s MiniDisc system that, when introduced, had all of the hallmark features of the innovations that Apple introduced (novel, high quality, portable, smaller, visible advantages over the alternatives), yet it was a spectacular failure. Canadian management consultant Michael Raynor has called this the strategy paradox. When qualities such as vision, bold leadership, and focused execution — all the commonsensical aspects of great leaders — are applied to organizations it can lead to great success (Steve Jobs and Apple) or resounding failures (RIM?).
Strategic flexibility, making small adjustments consistently, and imaging scenarios for the future in an ongoing manner are some of the potential ways to limit the damage from common sense (or use its advantages more fully). This requires feedback mechanisms and close monitoring of program activities, developmental evaluation, and a willingness to tweak programs and design on the go (what I call: developmental design) . It’s not a surprise that this incremental approach to development is consistent with the way change is best produced in a complex adaptive system.
By recognizing that common sense is less than common and is certainly not consistent, program designers, developers, evaluators and other professionals will be better positioned to provide true leadership that addresses challenges and complexity rather than adds to the complexity and creates more problems.
Photo: Goodbye to Common Sense Space by Amulet Dream from Deviant Art
The Complexity of Planning and Design in Social Innovation
Posted: January 7, 2012 Filed under: complexity, design thinking, evaluation, systems science, systems thinking | Tags: complexity, design, developmental design, developmental evaluation, evaluation, health, human services, social media, systems thinking 2 Comments »
The Architecture of Complex Plans
Planning works well for linear systems, but often runs into difficulty when we encounter complexity. How do we make use of plans without putting too much faith in their anticipated outcome and still design for change and can developmental design and developmental evaluation be a solution?
It’s that time of year when most people are starting to feel the first pushback to their New Year’s Resolutions. That strict budget, the workout plan, the make-time-for-old-friends commitments are most likely encountering their first test. Part of the reasons is that most of us plan for linear activities, yet in reality most of these activities are complex and non-linear.
A couple interesting quotes about planning for complex environments:
No battle plan survives contact with the enemy – Colin Powell
In preparing for battle I have always found that plans are useless, but planning is indispensable – Dwight D. Eisenhower
Combat might be the quintessential complex system and both Gens Powell and Eisenhower knew about how to plan for it and what kind of limits planning had, yet it didn’t dissuade them from planning, acting and reacting. In war, the end result is what matters not whether the plan for battle went as outlined (although the costs and actions taken are not without scrutiny or concern). In human services, there is a disproportionate amount of concern about ‘getting it right’ and holding ourselves to account for how we got to our destination relative what happens at the destination itself.
Planning presents myriad challenges for those dealing with complex environments. Most of us, when we plan, expect things to go according to what we’ve set up. We develop programs to fit with this plan, set up evaluation models to assess the impact of this plan, and envisage entire strategies to support the delivery and full realization of this plan into action. For those working in social innovation, what is often realized falls short of what was outlined, which inevitably causes problems with funders and sponsors who expect a certain outcome.
Part of the problem is the mindset that shapes the planning process in the first place. Planning is designed largely around the cognitive rational approach to decision making (PDF), which is based on reductionist science and philosophy. Like the image above, a plan is often seen as a blueprint for laying out how a program or service is to unfold over time. Such models of outlining a strategy is quite suitable for building a physical structure like an office where everything from the materials to the machines used to put them together can be counted, measured and bound. This is much less relevant for services that involve interactions between autonomous agents who’s actions have influence on the outcome of that service and that result might vary from context to context as a consequence.
For evaluators, this is problematic because it reduces the control (and increases variance and ‘noise’) into models that are designed to reveal specific outcomes using particular tools. For program implementers, it is troublesome because rigid planning can drive actions away from where people are and for them into activities that might not be contextually appropriate due to some change in the system.
For this reason the twin concepts of developmental evaluation and developmental design require some attention. Developmental evaluation is a complexity-oriented approach to feedback generation and strategic learning that is intended for programs where there is a high degree of novelty and innovation. Programs where the evidence is low or non-existent, the context is shifting, and there are numerable strong and diverse influences are those where developmental evaluations are not only appropriate, but perhaps one of the only viable models of data collection and monitoring available.
Developmental design is a concept I’ve been working on as a reference to the need to incorporate ongoing design and re-design into programs even after they have been initially launched. Thus, a program evolves over time drawing in information from feedback gained through processes like evaluation to tweak its components to meet changing circumstances and needs. Rather than have a static program, a developmental design is one that systematically incorporates design thinking into the evolutionary fabric of the activities and decision making involved.
Both developmental design and evaluation work together to provide data required to allow program planners to constantly adapt their offerings to meet changing conditions, thus avoiding the problem of having outcomes becoming decoupled from program activities and working with complexity rather than against it. For example, developmental evaluation can determine what are the key attractors shaping program activities while developmental design can work with those attractors to amplify them or dampen them depending on the level of beneficial coherence they offer a program. In two joined processes we can acknowledge complexity while creating more realistic and responsive plans.
Such approaches to design and evaluation are not without contention to traditional practitioners, leaving questions about the integrity of the finished product (for design) and the robustness of the evaluation methods, but without alternative models that take complexity into account, we are simply left with bad planning instead of making it like Eisenhower wanted it to be: indispensable .
Systems Thinking and the Design of Empathy
Posted: September 22, 2011 Filed under: complexity, design thinking, systems thinking | Tags: Charles Leadbeater, design, developmental design, developmental evaluation, empathy, organizational change, systems thinking Comments Off
Leadbeater's Systems Thinking - Empathy Grid
Scalability is an issue that faces practitioners in systems and design. How do we design systems at scale and if so, what might they look like.
Charles Leadbeater has been on a mission to find ways to make large organizations — particularly those in the social sector – more innovative. Leadbeater, like many social innovators, is hard to pin down to a single title or role. He is at once a researcher, a designer, a systems thinker, and a urbanist. Like most innovators, all and none of these descriptors truly fit.
Leadbeater was in Toronto earlier this week to speak on the issue of innovation in cross-sector collaboration for public good at the MaRS Discovery District. If you’ve seen Leadbeater speak (consider the talks on TED here or elsewehere), you’ll know that you’re in for some English-style self-depricating humour alongside of much about the manner in which people engage in change actions within a system. You’ll also get a lesson in social design, the kind that Victor Papanek advocated for.
To my delight, Leadbeater did not disappoint. Unlike other talks, the value came less from a focused “take home message” and more in a way of conceiving social systems through the combined lens of systems and design thinking (my terms, not his). At the heart of his talk was the challenge we face with building systems and empathy at scale. When things interact, eventually they become understood within a set of boundary conditions and interact, thus making a system. The system in turn begins to establish rules (or rather, the rules determine the system). These emergent properties thus shape the way the system operates or, in social situations, governs or guides the actions of those within it.
The problem is that at certain scales the very factors that create positive social relations, that is those that yield tangible emotional, resource, or informational benefits for one or more parties, get warped under the changes in that scaling. Thus, we have the countless stories of a beloved small business grown in a confined community that becomes a multinational corporation and, in doing so, loses the intimacy and connection to its customers in the process. Companies do this, government organizations show this, and so do cities.
The more one designs for the humans within the system in ways that create meaningful engagement, the greater the empathy. Yet deep empathy is often founded upon intimacy, which is something that is difficult to scale. Leadbeater illustrates the various ways in which firms and cities have addressed this on the graph above. In each case, there are examples that fit. In business for example, there is Ryanair, which embodies a highly structured system with low empathy (top left corner). Opposed to that is the local farmer’s market where one gets to know their grower, experience high mutual empathy, but in a manner that is unique, idiosyncratic and non-systematic in most ways. The challenge is how to design organizations at scale from the cosy-ness of the Farmer’s Market without becoming a Ryanair.
It struck me that the food service industry might be one of the areas where the scalability can be achieved. For example, Starbucks is a gigantic corporation with shops worldwide, yet it still manages to create a very homey, local feel at each one. In the mornings I go to the gym I stop by a location to get a smoothie and my server always remembers my order. At the location near where I take classes, they gave me a free drink because they couldn’t get the computer to take off my 10 cent reusable cup discount. In each location, the benefits were not just in customer service, but in the chit-chat and relationships that I develop with the staff. It’s not like I am speaking to owner-operators at some of the great independent coffee shops around my city, but it is close.
The Starbucks experience was thought through, intentional and thus, by design. We can do this with other systems. The key is whether or not the systems themselves are aware enough to know when they have, indeed, become systems. Starbucks today could not be empathic in exactly the same way as it was when it was a one-shop place at Pikes Market in Seattle. But it can create something similar, which is parallel to Simon’s notion that design is about the science of the artificial.
I’ve been developing and advocating for an approach to creating scale — in time and scope — that I call developmental design. A developmental design approach means shifting and changing over time and designing things in a manner that adjust to the complexities associated with dynamic systems. It brings together complexity, systems, design and the detailed feedback mechanism that comes through developmental evaluation. Leadbeater’s grid helps add to this concept by giving a focus to the development, from one level of empathy to another and one systemic scale to another.
Through thinking in systems and acting through design, perhaps then we can create the kinds of services and organizations that respond to the challenges we face.
And designing for empathy will help us know when we’ve achieved it.
Design Thinking and the Metaphors of Science
Posted: August 28, 2011 Filed under: complexity, design thinking | Tags: Andrea Yip, chemistry, complexity, design, developmental design, learning, science 3 Comments »
The Chemistry of Creation
There is a certain way in which things come together to create a successful design (or relationship) that is often chalked up to “chemistry”. But design chemistry could mean something both literal and evolving just like biological organisms if we take the concept to its fullest.
Metaphors are commonly used in tackling complex problems. The uniqueness of the situation, the level of detail of the manner by which the influencing factors coalesce, and the multidisciplinary ways of seeing the problem in the first place all present a problem of language, thus using oblique comparators can often fill the gap.
Science and mathematics have the advantage of being closer to ‘universal’ languages than many of the other forms of communication we share as a species (Leibniz’s ideas notwithstanding). They are less (not completely) influenced by cultural variations and local differences and can be shared globally. It is for this reason that the the prospect for a means of communicating concepts like design through science has appeal. As Andrea Yip has pointed out, design itself can be transformed into chemistry using the periodic table as a guide to serve as a more universal metaphor for understanding the way design thinking is experienced and practiced.
Chemistry is the study and creation of the bonds of the universe. More specifically, it is:
the science of matter, especially its properties, structure, composition, behavior, reactions, interactions and the changes it undergoes.
As a metaphor for design thinking it works beautifully. Through the Periodic Table of Design Thinking we see an attempt to lay out the properties of design thinking, map out the structure and explain their composition. Through practice and reflection we will see how these compounds play out in the design process.
Another scientific metaphor that takes up the charge from where chemistry leaves off is from developmental biology:
the study of the process by which organisms grow and develop
In the case of this metaphor, design thinking is the organism. Just as an organism, made of chemical compounds interacting over time, evolves, so too does the design process and the thinking that comes with it. In this case, metaphors like those proposed by Ms Yip and the concept of developmental design fit harmoniously.
Designing for and with complexity requires attention to a dynamism that can be lost if one takes the approach that product development happens at only stage of its life cycle. For many products this might be appropriate, but it falls short when we describe social design issues such as creating policies or social programs such as those found in health and education. I’ve referred to this concept as developmental design. Developmental design, like developmental evaluation, implies an evolved, dynamic approach to generating knowledge or outcomes and while I only loosely conceived of it in a way that matched developmental biology, it may be time to revisit that more intently. Designing developmentally means working through the design process on an ongoing basis, like perpetual beta in the software industry. It means evolving strategies for adaptation rather than solving problems because true solutions to wicked problems are often more dream than reality.
Taking the chemistry metaphor, it means that the ingredients, dosage and combinatorial mixes change over time in the production of a new compound or design. They may require catalysts — such as the inclusion of new perspectives or a particular discipline — to provoke certain reactions and move ideas into new space. It may also involve the same type of intervention from the designer to bring these chemicals to life. The chemist is not removed from her creation.
All of these are metaphors, yet they provide us with a means of taking the messiness of the language, something discussed in previous posts, to a new place until we can find the language that is most appropriate. Until that time, science might offer one of the better means of conveying design, complexity and the creativity that comes when we apply them both to generating products and services.
Photo Chemistry! by matfred used under Creative Commons Licence from Flickr.
