Social Media and Health: Leaders(hip) and Followers(hip)
Posted: May 18, 2012 Filed under: behaviour change, eHealth, health promotion, innovation, public health, social media, systems thinking | Tags: complexity, creativity, design thinking, developmental design, developmental evaluation, eHealth, evaluation, Facebook, health, health promotion, healthcare, innovation, medicine, organizational change, organizational design, public health, social media, systems thinking 2 Comments »
Social media is finally catching on with healthcare, public health, and health promotion. With a few recent articles published in the academic literature to rest on, academic health sciences has finally (and I might argue, begrudgingly) conceded that 900+ million users and $100B valuations (Facebook), and thousands of messages exchanged every milisecond (microblogs like Twitter and Sina Wiebo) might have some value for the public beyond entertainment.
If you note how long it took the health sector to start using the telephone as a serious means of engaging their patients or the public, this is lightning-quick adoption. Still, the barriers to adoption are high and the approach to using the technology is scattered. Indeed, just like the start of Internet-delivered telehealth (or cybermedicine (PDF), which has now evolved into eHealth), there is a mad rush to get liked, followed or some other metrics that most health professionals barely understand.
And that is part of the problem.
Meaningful Social Media Metrics
What is a meaningful metric for social media and health? A recently published article in Health Promotion Practice suggested four metrics that are taken from social marketing and applied to social media. These Key Performance Indicators (KPI’s) are:
- Insights (consumer feedback)
- Exposure (media impressions, visits, views, etc..)
- Reach (# people who connect to the social media application)
- Engagement (level of interaction with the content)
These are reasonable, but to to the uninitiated I would suggest a few words of caution and commentary to this list.
Firstly, the insights suggested by Neiger and colleagues “can be derived from practices such as sentiment analysis or data mining that uses algorithms to extract consumer attitudes and other perspectives on a particular topic” (p.162). While not incorrect, this makes the job sound relatively simple and it is not. Qualitative analysis + quantitative metrics such as those derived from data mining are key. Context counts immeasurably in social media use. It’s only in situations where social media is used as a broadcasting tool that gross measures of likes and sentiment analysis work with little qualification.
Even that is problematic. Counts of ‘likes’, ‘visits’, ‘follows’ and such are highly problematic and can be easily gamed. I am ‘followed’ on Twitter by people who have tens of thousands of followers, yet virtually no presence online. Most often they are from marketing fields where the standard practice is to always follow back those who follow you. Do this enough and pretty quickly you, too can have 23,000 followers and follow 20,000 more. This is meaningless from the perspective of developing relationships.
Engagement is the most meaningful of these metrics and the hardest to fully apply. This category gets us to consider the difference between “OMG! AWESOME!” and “That last post made me think of this situation [described here] and I suggest you read [reference] here for more” as comments. Without understanding the context in which these are made within the post, between posts (temporally and sequentially), and in relation to a larger social and informational context, simple text analysis won’t do.
Social Media Evidence: Problems and More Problems
One of the objections to the use of social media by some is that it is not evidence-based. To that extent I would largely agree that this is the case, but then we’ve been jumping out of airplanes with parachutes despite any randomized controlled trial to prove their worth.
Another article in Health Promotion Practice in 2011 highlights potential applications for social media and behaviour change without drawing on specific examples from the literature, but rather on theoretical and rhetorical arguments. An article published in the latest issue of Perspectives on Psychological Science highlights the current state of research on Facebook, which is timely given that its IPO is set for today. That review by Wilson and colleagues illustrates the largely descriptive nature of the field and offers some insight on to the motivation of Facebook users and their online activities, but rather little in what Facebook does to promote active change in individuals and communities when they leave the platform.
The answer to whether social media like platforms such as Facebook ‘work’ as methods of promoting change is simply: we don’t know.
Does social media provide support to people? Yes. Does it inform them? Yes to that too. Does that information produce something other than passive activity on the topic? We don’t know.
In order to answer these questions, health sciences professionals, evaluators, and tech developers need to consider not just followership, but leadership. In this respect, it means creating changes to the way we gather evidence, the tools and methods we use to analyse data, and the organizational structures necessary to support the kind of real-time, rapid cycle evaluation and developmental design work necessary to make programs and evidence relevant to a changing context.
As Facebook launches into its new role as a public company it is almost assured to be introducing new innovations at a rapid pace to ensure that investor expectations (which are enormous) are met. This means that today’s Facebook will not be next month’s. Having funding mechanisms, review and approval mechanisms, a staff trained and oriented to rapid response research, and an overall organizational support system for innovation is the key.
Right now, we are a long way from that. Hospitals are very large, risk averse organizations; public health units are not much different. They both operate in a command-and-control environment suited for complicated, not complex informational and social environments. Social media is largely within the latter.
Systems thinking, design thinking, developmental evaluation, creativity, networks and innovation: these are the keywords for health in the coming years. They are as author Eric Topol calls the dawning of the creative destruction of medicine.
The public is already using social media for health and now the time has come for health (care, promotion and protection) systems to get on board and make the changes necessary to join them.
The Face-to-Face Complexity of eHealth & Knowledge Exchange
Posted: November 24, 2010 Filed under: behaviour change, complexity, eHealth, innovation, knowledge translation, public health, social media, systems science, systems thinking | Tags: community organizing, complexity, CoNEKTR model, eHealth, innovation, knowledge translation, public eHealth, public health, social media, systems thinking 2 Comments »The Public Health Agency of Canada‘s 2010 Knowledge Forum on Chronic Disease was held last night today in Ottawa with the focus on social media. The invitation-only affair was designed to bring together a diverse array of researchers, practitioners, policy developers, consultants and administrators who work with social media in some capacity. There were experts and non-experts alike gathered to learn about what the state of the art of social media is and how it can support public health. By state of the art, I refer not to the technological side of things, but rather the true art of public health, much like that discussed earlier this year at the University of Toronto.
Last night began with a presentation from Leanne Labelle that got us all thinking about how social media is radically different in the speed of its adoption and breadth of its social impact drawing inspiration from this video from Eric Qualman’s Socialnomics website.
Today we got down to business and started working through some of the issues that we face as a field when adopting social media. I would probably consider myself among the most experienced users in the audience, yet still gained so much from the day. Although I learned some things about how to use social media in new ways, what I learned most was how others use it and what struggles they have. This is always a useful reminder.
What stuck out was a presentation and related discussion from Christopher Wilson from the University of Ottawa’s Centre on Governance and a consultant on governance issues. In speaking about the challenges of doing collaboration, Christopher pointed to the problems of a ‘one-size fits all’ strategy using a diagram illustrating the fundamental differences between engagement at a small scale (under 25 people) and what is the mass collaboration that folks like Clay Shirky, Don Tapscott, and others write about. His diagram looks like this:
Technology Spectrum of Social Collaboration by Christopher Wilson
What Wilson stressed to the audience was the role that complexity plays in all of this. Specifically, he stated:
The more complex and interdependent things become, the more people need to be aware of the changing context and the changes in shared understanding.
As part of this, groups are required to engage in ways that enable them to deal with this complexity. In his experience, this can’t be done exclusively online. He further stated:
As complexity increases, the need for offline engagement increases.
I couldn’t agree more. In my work with community organizing and eHealth promotion, I’ve found the most effective means of fostering collaboration is to blend the two forms of knowledge generation and exchange together. The model that my research team and I developed is called the CoNEKTR (Complexity, Networks, EHealth, and Knowledge Translation Research Model).
This model combines both face-to-face methods of organizing and ideation, with a social media strategy that connects people together between events. The CoNEKTR model has been applied in many forms, but in each case the need to have ways to use the power of social media and rich media together with in-person dialogue has been front and centre. Using complexity science principles to guide the process and powered by social media and face-to-face engagement, the power to take what we know, contextualize it, and transform it into something we can act on seems to me the best way forward in dealing with problems of chronic disease that are so knotted and pervasive, yet demand rapid responses from public health.
Social Media / Social Activism Redux: Can we Learn from Behaviour Change Theories?
Posted: October 13, 2010 Filed under: behaviour change, psychology, social media | Tags: eHealth, health behaviour change, health communications, psychology, public health, social media, Twitter 3 Comments »What started as a simple column post a column on October 4th in the New Yorker has really turned into a firestorm of discussion over the last few days (reflecting a building crescendo of discontent and plaudits from those on each side of the debate). The latest volley in the debate has come from the founder of Twitter themselves in a piece in the Guardian. In that column, the chief Tweeters remark:
Williams said:
“It was a very well-constructed argument but it was kind of laughable.
“Anyone who’s claiming that sending a tweet by itself is activism, that’s ludicrous — but no one’s claiming that, at least no one that’s credible. If you can’t organise you can’t activate. I thought [the article] was entertaining but kind of pointless.”
They have a good point. But while Gladwell might be too dismissive of the power of tools like Twitter, it is easy to overstep and imply that information is power and having more of it networked leads to activation (something I discussed earlier this week).
Knowing and doing are very different and any analysis of major theories on behaviour change and the evidence, shows a relatively weak correlation between knowing more and doing more. It also shows an OK, but also not a strong correlation.
What does change people’s behaviour? Lots of things — and that’s the problem. The either/or thinking that permeates the discussion of social media is too often simplistic and driven by an interplay of ideas and values that are not always aligned with the evidence or personal experience.
People tend to change for the following reasons:
1. They have information that tells them there is a threat or a problem with the status quo;
2. Others believe that the behaviour should be changed;
3. The person changing actually cares what other people think;
4. That person has the skills and tools to be able to change;
5. The environment is supportive of change and facilitative (e.g. there are policies, procedures, access to resources — including time);
6. There are more pros to changing than cons (and there are more pros to the strategy of change than the cons);
7. A person actually wants to change (they are self-motivated and not doing things because everyone else thinks they should);
8. A person feels capable of making the change at all.
The more of these elements are present, the more likely the change is going to take place and stick. This is a big list and indicates that change isn’t always straightforward, and it certainly isn’t easy.
The revolution most likely will be Tweeted, but whether that is the cause or the consequence is why research on social media and social activism is needed. Otherwise, we will wind up with another chicken and egg problem.
Chicken, Eggs or Social Eggs?
Systems (Science): Sexy and Not So Much
Posted: September 9, 2010 Filed under: innovation, systems science | Tags: academia, complexity, design thinking, eHealth, gambling, innovation, knowledge translation, mhealth, public health, systems thinking 1 Comment »Recently I was discussing what I do with someone relatively unfamiliar to my research, yet in the same field and I described my interest in systems thinking, knowledge translation, and eHealth and how they go together. Somehow in the conversation the term “sexy” was used to describe these fields along with “hot” and “upcoming”. It’s nice to be at the forefront of a field — or three — but it also has some downsides.
One of the downsides is that rigor often gets displaced by enthusiasm. Even fields like knowledge translation, which first emerged in the mid 1990′s (and far earlier than that if you’re willing to consider different terms), is just now evolved to the point where it is widely accepted and supported as a legitimate area of research. There is still much work that calls itself KT that is really just dissemination with a different name, but the concept of KT at least has some respect.
So too, does the idea of systems thinking. With recent special issues in respected journals like the American Journal of Preventive Medicine and American Journal of Public Health and full monographs from the National Cancer Institute, the idea of transmitting systems science from the backwaters of public health to the forefront is close to reality.
eHealth was sexy too, but too much investment matched with too little patience for good evidence quickly burned through much of the potential that consumer-directed eHealth had for making transformative differences on a broad scale. No worry, mHealth is here and that is quickly proving to be as “hot” as eHealth was ten years ago.
The problem is that “hot” and “sexy” terms often presage their demise in respected discourse before too long. I was once told by a senior official with a large health NGO that he’d given up on eHealth because he knew it didn’t work. This was 2002. Most of the best evidence hadn’t been generated yet and already people had thrown in the towel.
Knowledge translation is having its problems too, because the evidence of a shortening from “evidence to effect” is hard to generate. Often, KT requires systems level changes and systems thinking to create the conditions to generate effective KT practice. That suddenly transforms something that is “sexy” into something difficult and much less so in the eyes of those who are responsible for implementation of KT plans.
Systems science as it is applied to health is in greater danger because the scale and scope of change required to generate good evidence is often at a scale that is prohibitive. Take gambling as one potential public health problem. Governments are now deriving enormous revenue from gambling, while the social costs seem to rise with it. So important is gambling to provincial government revenues in most Canadian provinces that the only way to really change is to change the system as a whole. Diabetes care, mental health, and public nutrition and food security are other issues that are complex and of a scope that requires a true systems-level intervention to effectively address. Suddenly, when you speak of connecting the private sector to the public sector, changing regulations, building true KT systems within these areas, supporting public health education and practice, and tackling the social inequalities that are propped up by a current system of organization, systems don’t seem as sexy.
Furthermore, in the case of systems, KT, or eHealth, acknowledging complexity in the way we handle things, and considering problems from a systems perspective, means hard work, different time horizons, and truly working collaboratively across disciplines and settings. That is hard stuff and it sure isn’t sexy.
There are lots of areas within the health system that are not sexy, but few are seen less interesting than issues that were once viewed as sexy, but now not so much. That’s the danger with systems, knowledge translation and eHealth.
The Lady Gaga – mHealth Connection
Posted: August 28, 2010 Filed under: design thinking, eHealth, public health | Tags: design, eHealth, innovation, Lady Gaga, marketing, mhealth, social media 4 Comments »
Lady Gaga: Calling Health?
It’s probably fair to say that Lady Gaga isn’t the first person you think of with Mobile Health (mHealth). Accessing patient records, behaviour change resources, and information on wellbeing are topics that reside closer on the spectrum of similarity than those most associated with the current reigning queen of pop.
But looking a little deeper, there is reason to consider that this image is a little more than a poker face (bad pun intended). Let me point to the comparisons.
1. One of central points of comparison is that, like mHealth opportunities, there is no escaping Lady Gaga. Her music is everywhere — in front, as background music, as a harbinger of taste (good and bad, depending on the audience), and something people are talking about. MHealth is in the same boat.
2. Consider that Lady Gaga herself is the subject of considerable attention. She’s on the cover and profiled in this month’s Vanity Fair and her image is strewn all over the Internet. She is what people are talking about. MHealth is likewise. If you want more Twitter followers, add #mhealth. If you want more readers, subscribers, and conversation, talk of mHealth. It is a very hot topic in the world of healthcare and mobile technologies.
3. Lady Gaga is a mystery. While we know her roots and her family name, there is surprisingly little that is known about the “real” Lady Gaga. How much is style and show and how much is her? That holds true for mHealth. Without a large field of research and evidence, the entire mHealth phenomenon is a bit of a mystery to many in and out of the industry. It is perhaps for that reason that many members of the public are not sold on the reasons for why they should need or want to access medical information like personal health records using electronic tools as was recently reported this week.
There are also a couple of areas where the comparison between her and mHealth should be strong, but isn’t.
4. Lady Gaga is enmeshed with social media. This past week, Lady Gaga surpassed Britney Spears to become the pop Queen of Twitter. Never to miss an opportunity to mark this regal occasion, Ms. Gaga went to YouTube, where she was, until recently upended by Justin Beiber, creator of the most watched video on that channel of all time. Surprisingly, mHealth hasn’t quite got there yet. There are many apps for health to be sure, and some of them are quite well put together, but most of them use a model of service that represents a push model of service, rather than a social model of conversation.
This issue of conversation was the topic that marketing thought leaders Mitch Joel and Joe Jaffe spoke about yesterday in their live-fed podcast discussion. That conversation centred on the idea that marketing is rarely about conversation per se, but trying to get information to people quickly with the hope that it will lead to something. True conversation requires relationships and time and many companies are not willing to do what it takes to get there. I would argue that the same holds true in the health sector and its related industries. There is too much money to be made quickly to slowly develop relationships, healthcare institutions are not (ironically) set up for relationship development, and health providers are rarely given the resources or incentives to spend the time with their patients in real time, let alone develop social media channels. Its therefore no surprise that mHealth and social media are struggling to find their way in their relationship.
5. Lady Gaga delivers. In conversation with colleagues and reading reviews of her concerts by even reluctant fans (I have not yet seen the spectacle that is the Monsters Ball) one story emerges: it is an amazing performance. And by performance, it means that she entertains and delivers something of value to her audience. From what I hear and read, even those who do not consider themselves as one of Gaga’s ‘Little Monsters‘ (i.e., fans), she is worth the price of admission to see. MHealth still isn’t there…yet. Indeed, for the reasons discussed above and in previous posts, there is a lot of questions about mHealth and what it can, will and should deliver. So far, its delivering on simple things like iPhone apps and push-model tools, but little on interactive, social media-based programs. The potential is to create environments of truly interactive, user-driven health content.
A project that I’m involved with is trying to do this. My research group and partners just lauched the Youth4Health website and, in the next few weeks, will have our multi-platform app distributed to youth with iPhones to provide mobile content as well. It’s a start.
6. Lastly, Lady Gaga is a 360 degree celebrity. She makes much of her own clothes, runs her own design shop, writes her songs, and produces many of her own work. She also has relationships with her distribution channels, including a sponsorship with Virgin Mobile. MHealth is nowhere near this. As an mHealth researcher, I can point to few peers who have relationships with developers, producers, the public, funders and distribution channels at the same time. It is for that reason that this work takes so long to build and why mHealth is either run by non-health professionals or run badly and in obscurity by health professionals.
Maybe mHealth needs to take a little more from the reigning Queen of Twitter and get a little more bold, stylish and out there. Then, and maybe then, will we see its promise unfold.
eHealth / mHealth And Privacy Will Never Coexist
Posted: August 21, 2010 Filed under: eHealth, public health, social media | Tags: eHealth, information technology, mhealth, mobile phones, privacy, security, social media 2 Comments »
Erasing Privacy?
There is some debate right now in the tech world about whether or not there is or will be two Internets: one that is the domain of computers and one that is the domain of mobile devices. If you’re like me, there is only one Internet that involves both.
Throughout the day I access information using a variety of devices from different places that combine wired and wireless Internet. Much of my Facebook updates are done on a Blackberry, while my Tweets might come equally from my laptop, iPod or that same Blackberry. I don’t think about what I am using when I engage the Internet world and that’s pretty common.
But this past week a joint statement by Google and U.S. mobile service provider Verizon pointed to the idea that these two ways of accessing the Internet are distinct and they are seeking to create dialogue about rules on how we engage each ‘Internet’. This statement, summarized and discussed by Elliot van Buskirk on Wired.com , basically points to a preference by both companies for some kind of restricted (controlled) form of wireless Internet. This is about net neutrality, the concept that all traffic on the Internet is treated equally, regardless of who you are and what content you are producing or consuming.
Google has responded to the criticism on their own blog and go to point out that their apparent repudiation of the idea of net neutrality on the wireless web is a myth.
Net neutrality is a big issue and one that is worth paying attention to even if you are not a ‘techie’. It is not my intention to discuss it here, but rather discuss another piece of media that got me to this issue and to a place of thinking that privacy and eHealth are an impossible pair.
How did I get there? The inspiration came from Mitch Joel’s Twist Image podcast, Six Pixels of Separation. If you’ve never listened to it before, it is well worth an hour to drop in** and hear someone who is quite articulate about social media issues explore in a casual, but engaging way the social mediasphere. The last episode (#215), was particularly good. Actually, scary might be a better term. Why? In one hour, Joel and his guests discussed, over sushi, a range of issues that, for me, cemented the fact that eHealth and privacy will never coexist. Ever.
They did this without actually using the term eHealth or mHealth (mobile health), but for those of you in this area of work you’ll immediately know why I came to this conclusion. Interestingly, none of the issues that they covered were new to me (aside from the Google-Verizon one). But I had never heard them all discussed in the same narrative in a manner that got me to imagine the future in the way I did when I listened to it. What it did was showcase the already loose controls we have on our “regular” online data, stored in website host servers and ISP provider databases and how these already poor provisions are nothing compared to wireless data, where our data providers and devices give up every little piece of information about us every time we engage in this “other” Internet — the one that has Verizon and Google so interested in controlling/influencing.
The example used to point this out was a health one — albeit a hypothetical one — and it shows just how vulnerable we can be, particularly if you imagine someone posting all of your search data online. This is a real threat. These things happen. Just look at what is going on with the current Wikileaks / Afghan war documents.
I have no suggestion for how to address this and, frankly, I don’t really see any solution that will work to address privacy in eHealth/mHealth; the problem is too embedded. Having some faceless corporation holding our confidential information or some government bureaucracy doing the same are two lousy options. This is the price we pay for convenience, for the power that information has in helping us keep well and for the tools that allow us to connect to others. It’s a big price and no doubt there will be some who listen to this podcast and view the topic through an eHealth light and start questioning whether its worth it. But unless we go back to paper with all we do, I don’t see an option for addressing it in any practical manner. We’ve paid the price of admission to the fair that is eHealth/mHealth and now we need to pay it off with the cost of running the show.
** I like Joel’s style and frankness about media issues, but I will say that his podcast — particularly those that are cross-labeled as ‘media hacks’ episodes — features occasional off-colour jokes among his guests in dialogue (often including fellow social media marketing leaders like Julien Smith, Chris Brogan, and Hugh McGuire) that smack of sexual inappropriateness that I think is unnecessary, beneath him and frankly add no value to the show or his brand. These are relatively mild, but still not worth it. To them I say: Swear all you like guys, but cut out the sexual joking around. It cheapens the whole experience and makes you smart guys look like sophomoric jocks when you do it.
Creating the Future Through Systems-Design Thinking
Posted: August 20, 2010 Filed under: complexity, design thinking, eHealth, innovation, science & technology, systems science, systems thinking | Tags: complexity science, design thinking, eHealth, ICT, innovation, organizational change, social innovation, systems thinking Comments Off
Back to the Future
Arturo Muente-Kunigami wrote in the World Bank’s Information and Communication Technology blog about the challenge of innovation and putting new information technology into practice in governments worldwide. Muente-Kunigami writes:
Most governments that introduce ICTs in their service delivery structure have basically applied technology to the exact same workflow they had before, replacing papers with emails and signatures with digital certificates. But ICTs in general – and broadband in particular – do not just improve the efficiency of governments. They have the potential to transform how governments work, redefining their relationship with citizens and expanding the array of services and transactions that could be provided and implemented.
This, however, is a very risky proposition for governments. And if most private companies rely on analytical thinking due to their overall aversion to risk, governments in most developing countries have a much less functional innovation system (in many cases, equivalent to a “copy-paste” function to be applied to “best practices” in other countries).
This is basically a ‘back-to-the-future’ problem: how to use the past to shape the future? How do we create best practices in areas where there are constant shifts, changes and altered contexts? Marty Neumaier would argue that we can’t. This is a design problem, not a knowledge transfer one. Muente-Kunigami also recognizes the potential for design thinking here and argues that governments need to follow their private sector peers in applying it to ICT and innovation:
So what is design thinking for governments anyway? It is not that much different than its private sector equivalent. It is about going back to the basics. And I mean the basics, trying to understand what citizens need from their governments (yes, that far back) and then answering the question: how could governments (hopefully, leveraging the new set of technologies and devices that exist today – and their spread among the general population) be able to satisfy these needs? Then, it is all about building prototypes, testing, trial and error, and of course a good set of evaluation and feedback mechanisms2.
This scary territory for a lot of organizations, particularly governments where decisions are not only shaped by history, but capital P politics. It’s also a language problem: Design gets equated with style instead of substance. Innovation is something done in business, not social and public services. Technology is something for wealthy nerds, not everyday citizens.
Marty Neumaier, Bruce Mau, Roger Martin and other design thinkers have been trying to shape this attitude, but it is an uphill battle. Language is one barrier, thinking differently is another. Both are challenges that I’ll address in future blogs, but the one I want to focus on here is the concept of best practices and the pull of the past on the present. Indeed, this is as good of an example of the power of an idea that you can find. Ideas may be the most powerful concept in human thinking as they shape the cognitive space that we inhabit by illustrating what is, what was, and what could be.
It is when what was becomes what could be that problems occur, particularly in the space of complex systems, which is where a great deal of government’s work is. Best practices is one of those ideas that is seductive because it reduces variation and provides a blueprint for how to handle problems. Indeed, best practices are pretty good when your problems are simple, or maybe even complicated at a very low level of abstraction, but lousy when you get into the realm of complexity.
Another point that Muente-Kunigami hints at is the systems problem; that is, the need to design systems to accommodate change. Implementing ICT-based strategies into a system straight-away is a recipe for failure. Technical systems do not enhance functionality without corresponding changes in social systems. An organizational shift in the way ICT is deployed is necessary if there is much chance of these tools and technologies living up to their potential. This, too, requires design thinking – in creating usable technologies and receptive social systems (including those that are literate enough to take advantage of them).
I would also argue that this approach requires an evaluation approach that supports incremental evaluation and rapid-response feedback like we see in developmental evaluation (PDF), which I discuss elsewhere.
Taken together, the future of government may well be in design, but to create this future we need both the systems and design thinking to make it one day be the past.
Developmental Evaluation: Problems and Opportunities with a Complex Concept
Posted: July 31, 2010 Filed under: complexity, evaluation, research, social systems, systems science, systems thinking | Tags: complexity, developmental evaluation, eHealth, evaluation, knowledge integration, knowledge translation, public health, systems thinking 11 Comments »
Everyone's Talking About Developmental Evaluation
“When it rains, it pours” so says the aphorism about how things tend to cluster. Albert Lazlo-Barabasi has found that pattern to be indicative of a larger complex phenomenon that he calls ‘bursts‘, something worth discussing in another post.
This week, that ‘thing’ seems to be developmental evaluation. I’ve had more conversations, emails and information nuggets placed in my consciousness this week than I have in a long time. It must be worth a post.
Developmental evaluation is a concept widely attributed to Michael Quinn Patton, a true leader in the field of evaluation and its influence on program development and planning. Patton first wrote about the concept in the early 1990′s, although the concept didn’t really take off until recently in parallel with the growing popularity of complexity science and systems thinking approaches to understanding health and human services.
At its root, Developmental Evaluation (DE) is about evaluating a program in ‘real time’ by looking at programs as evolving, complex adaptive systems operating in ecologies that share this same set of organizing principles. This means that there is no definitive manner to assess program impact in concrete terms, nor is any process that is documented through evaluation likely to reveal absolute truths about the manner in which a program will operate in the future or in another context. To traditional evaluators or scientists, this is pure folly, madness or both. When your business is coming up with the answer to a problem, any method that fails to give you ‘the’ answer is problematic.
But as American literary critic H.L. Mencken noted:
“There is always an easy solution to every human problem — neat, plausible and wrong”
Traditional evaluation methods work when problems are simple or even complicated, but rarely do they provide the insight necessary for programs with complex interactions. Most community-based social services fall into this realm as does much of the work done in public health, eHealth, and education. The reason is that there are few ways to standardize programs that are designed to adapt to changing contexts or operate in an environment where there is no stable benchmark to compare.
Public health operates well within the former situation. Disaster management, disease outbreaks, or wide-scale shifts in lifestyle patterns all produce contexts that shift — sometimes radically — so that the practice that works best today, might not be the one that works best tomorrow. We can see this problem demonstrated in the difficulty with ‘best practice’ models of public health and health promotion, which don’t really look like ‘best’ practices, but rather provide some examples of things that worked well in a complex environment. (It is for this reason that I don’t favour or use the term ‘best practice’ in public health, because I simply view too much of it as operating in the realm of the complex, which is something for which the term is not suited.)
eHealth provides an example of the latter. The idea that we can expect to develop, test and implement successful eHealth interventions and tools in a manner that fits with the normal research and evaluation cycle is impractical at best and dangerous at the worst. Three years ago Twitter didn’t exist except in the minds of a few thousand and now has a user population bigger than a large chunk of Europe. Geo-location services like Foursquare, Gowalla and Google Latitude are becoming popular and morphing so quickly that it is impossible to develop a clear standard to follow.
And that is OK, because that is the way things are, not the way evaluators want them to be.
DE seeks to bring some rigour, method and understanding to these problems by creating opportunities to learn from this constant change and use the science of systems to help make sense of what has happened, what is going on now, and to anticipate possible futures for a program. While it is impossible to fully predict what will happen in a complex system due to the myriad interacting variables, we can develop an understanding of a program in a manner that accounts for this complexity and creates useful means of understanding opportunities. This only really works if you embrace complexity rather than try and pretend that things are simple.
For example, evaluation in a complex system considers the program ecology as interactive, relationship-based (and often networked) and dynamic. Many of the traditional evaluation methods seek to understand programs as if they were static. That is, that the lessons of the past can predict the future. What isn’t mentioned, is that we evaluators can ‘game the system’ by developing strategies that can generate data that can fit well into a model, but if the questions are not suited to a dynamic context, the least important parts of the program will be highlighted and thus, the true impact of a program might be missed in the service of developing an acceptable evaluation. It is, what Russell Ackoff called: doing the wrong things righter.
DE also takes evaluation one step further and fits it with Patton’s Utlization-focused evaluation approach., which frames evaluation in a manner that focuses on actionable results. This approach to evaluation integrates the process of problem framing,data collection, analysis, interpretation and use together akin to the concept of knowledge integration. Knowledge integration is the process by which knowledge is generated and applied together, rather than independently, and reflects a systems-oriented approach for knowledge-to-action activities in health and other sciences, with an emphasis on communication.
So hopefully these conversations will continue and that DE will no longer be something that peaks on certain weeks, but rather infuses my colleagues conversations about evaluation and knowledge translation on a regular basis.
Mobile Health Promotion and Web 3.0
Posted: June 10, 2010 Filed under: eHealth, health promotion, innovation, public health, social media | Tags: design thinking, eHealth, health promotion, innovation, knowledge translation, mhealth, mobile phones, public health, Toronto 1 Comment »
Telephones of the Distant Future by Catmachine.
This week the MaRS Discovery District in Toronto is hosting NetChange Week, which describes itself as being “A week exploring tech for change” .
(follow comment on Twitter using #ncwk).
Yesterday’s focus was on mobile technologies and the ways in which they’ve been used to promote health and facilitate fundraising and knowledge development with non-profits. A series of innovations and novel forms of engagement were proposed, most notably in the area of sexual health.
Toronto Public Health presented work on a sexual health promotion program that uses proximity marketing through Bluetooth technologies. Health promoters with TPH go into the (mostly) gay community, particularly bars and clubs, wearing monitors that allow people to opt-in to receive Bluetooth-transported messages directly to their phones. The messages, contained in a GIF format so they can be viewed at a later time, provide a discrete way to deliver sexual health information specifically suited to the gay population.
Another similar program came from Black Cap, which has sought to engage the black community in Toronto through a variety of sexual health programs aimed at men who have sex with men and youth. The latter program involves a group of youth opinion leaders / health promoters who use text messages and their personal social networks to spread positive health messages in the community. Thus far, the program appears to be creating a buzz and leading to some action.
A third presentation from Lisa Campbell Salazar, a health promoter working with TakingITGlobal (among others), presented her research on youth and mobile technologies. Although the survey was not all focused on health issues, they certainly provided highlights (details of the survey can be found here).
One of the most salient findings from this survey was that mobile tools provide youth with a safe, accessible way to offer peer support to one another and connect in real time in situations where their health risk behaviour takes place. As TPH Health Promoter Michelle Hamilton-Page said in her presentation:
No one who is coming up to our booth is having sex at the moment, they need information for later when they are. Mobile phones provide a means to do that.
This is the bottom line for mobile technologies and health promotion. It provides support where people are — literally and figuratively — rather than where we wish them to be. Where we wish them to be are in places where we don’t have to work too hard to reach them (or are not complex): clinics, traditional media spaces, office buildings. Traditional media is usually passive, it can be crafted in boardrooms and office buildings, with little need to actually engage the community your trying to reach*. It is harder to do that with mobile messaging (although there are examples where this works in practice — TPH’s messages are crafted in advance, but the way they are delivered by an ambassador in the community adds that customized component that is part of the message. Black Cap’s youth opinion leaders custom craft their own messages on the fly using guidelines).
*- although even traditional media tries to solicit input before deploying things into the field.
Traditional, developer-designed, limited-authored websites (Web 1.0) allowed us the opportunity to broadcast messages in new ways to an enormous population. Social media enabled people to not only take part in a conversation, but initiate and re-create dialogical spaces and express themselves in ways that transcend text to pictures, video and other creative media (Web 2.0). Mobile technologies combine both of these earlier phases and enable conversations to take place where people are physically situated, freed of wired connections (Web 3.0). Here, the concept of ‘web’ is truly a network, a spiderweb of connections that are poised to promote health and engage the public in new ways.
It is here that the future of health promotion, and public health more broadly, lies.
A Web of Digital Health Promotion
