Tag: strategy

businesscomplexityevaluation

A mindset for developmental evaluation

on September 7, 2017

BoyWithGlasses.jpg

Developmental evaluation requires different ways of thinking about programs, people, contexts and the data that comes from all three. Without a change in how we think about these things, no method, tool, or approach will make an evaluation developmental or its results helpful to organizations seeking to innovate, adapt, grow, and sustain themselves. 

There is nothing particularly challenging about developmental evaluation (DE) from a technical standpoint: for the most part, a DE can be performed using the same methods for data collection as other evaluations. What stands DE apart from those other evaluations is less the methods and tools, but the thinking that goes into how those methods and tools are used. This includes the need to ensure that sensemaking is a part of the data analysis plan because it is almost certain that some if not the majority of the data collected will not have an obvious meaning or interpretation.

Without developmental thinking and sensemaking, a DE is just an evaluation with a different name

This is not a moot point, yet the failure of organizations to adopt a developmental mindset toward its programs and operations is (likely) the single-most reason for why DE often fails to live up to its promise in practice.

No child’s play

If you were to ask a five-year old what they want to be when they grow up you might hear answers like a firefighter, princess, train engineer, chef, zookeeper, or astronaut. Some kids will grow up and become such things (or marry accordingly for those few seeking to become princesses or they’ll work for Disney), but most will not. They will become things like sales account managers, marketing directors, restaurant servers, software programmers, accountants, groundskeepers and more. While this is partly about having the opportunity to pursue a career in a certain field, it’s also about changing interests.

A five-year old that wants to be a train engineer might seem pretty normal, but one that wants to be an accountant specializing in risk management in the environmental sector would be considered odd. Yet, it’s perfectly reasonable to speak to a 35-year-old and find them excited about being in such a role.

Did the 35-year-old that wanted to be a firefighter when they were five but became an accountant, fail? Are they a failed firefighter? Is the degree to which they fight fires in their present day occupation a reasonable indicator of career success?

It’s perfectly reasonable to plan to be a princess when you’re five, but not if you’re 35 or 45 or 55 years old unless you’re currently dating a prince or in reasonable proximity to one. What is developmentally appropriate for a five-year-old is not for someone seven times that age.

Further, is a 35-year-old a seven-times better five-year-old? When you’re ten are you twice the person you were when you were five? Why is it OK to praise a toddler for sharing, not biting or slapping their peers, and eating all their vegetables and weird to do it with someone in good mental health in their forties or fifties? It has to do with developmental thinking.

It has to do with a developmental mindset.

Charting evolutionary pathways

We know that as people develop through stages, ages and situations the knowledge, interests, and capacities that a person has will change. We might be the same person and also a different person than the one we were ten years ago. The reason is that we evolve and develop as a person based on a set of experiences, genetics, interests, and opportunities that we encounter. While there are forces that constrain these adaptations (e.g., economics, education, social mobility, availability of and access to local resources), we still evolve over time.

DE is about creating the data structures and processes to understand this evolution as it pertains to programs and services and help to guide meaningful designs for evolution. DE is a tool for charting evolutionary pathways and for documenting the changes over time. Just as putting marks on the wall to chart a child’s growth, taking pictures at school, or writing in a journal, a DE does much of the same thing (even with similar tools).

As anyone with kids will tell you, there are a handful of decisions that a parent can make that will have sure-fire, predictable outcomes when implemented. Many of them are created through trial-and-error and some that work when a child is four won’t work when the child is four and five months. Some decisions will yield outcomes that approximate an expected outcome and some will generate entirely unexpected outcomes (positive and negative). A good parent is one who pays attention to the rhythms, flows, and contexts that surround their child and themselves with the effort to be mindful, caring and compassionate along the way.

This results in no clear, specific prototype for a good parent that can reliably be matched to any kid, nor any highly specific, predictable means of determining who is going to be a successful, healthy person. Still, many of us manage to have kids we can proud of, careers we like, friendships we cherish and intimate relationships that bring joy despite no means of predicting how any of those will go with consistency. We do this all the time because we approach our lives and those of our kids with a developmental mindset.

Programs as living systems

DE is at its best a tool for designing for living systems. It is about discerning what is evolving (and at what rate/s) and what is static within a system and recognizing that the two conditions can co-exist. It’s the reason why many conventional evaluation methods still work within a DE context. It’s also the reason why conventional thinking about those methods often fails to support DE.

Living systems, particularly human systems, are often complex in their nature. They have multiple, overlapping streams of information that interact at different levels, time scales and to different effects inconsistently or at least to a pattern that is only partly ever knowable. This complexity may include simple relationships and more complicated ones, too. Just as a conservation biologist might see a landscape that changes, they can understand what changes are happening quickly, what isn’t, what certain relationships are made and what ones are less discernible.

As evaluators and innovators, we need to consider how our programs and services are living systems. Even something as straightforward as the restaurant industry where food is sought and ordered, prepared, delivered and consumed, then finished has elements of complexity to it. The dynamics of real-time ordering and tracking, delivery, shifting consumer demand, the presence of mobile competitors (e.g., food trucks), changing regulatory environment, novelty concepts (e.g., pop-ups!), and seasonality of food demand and supply has changed how the food preparation business is run.

A restaurant might not just be a bricks-and-mortar operation now, but a multi-faceted, dynamic food creation environment. The reason could be that even if they are good at what they did if everything around them is changing they could still deliver consistently great food and service and fail. They may need to change to stay the same.

This only can happen if we view our programs as living systems and create evaluation mechanisms and strategies that view them in that manner. That means adopting a developmental mindset within an organization because DE can’t exist without it.

If a developmental evaluation is what you need or you want to learn more about how it can serve your needs, contact Cense and inquire about how they can help you. 

Image Credit: Thinkstock used under license.

behaviour changeeducation & learningsystems thinking

Learning fails in bad systems

on August 28, 2017

2348137226_2d6536745e_o_Edits.jpgEnormous energy is spent on developing strategies to accomplish things with comparatively little paid to the systems that they are being deployed in. A good strategy works by design and that means designing systems that improve the likelihood of their success rather than fight against them and this is no truer than in the effort to learn on the job.

 

A simple search of the literature — gray or academic — will find an enormous volume of resources on how to design, implement and support learning for action in organizations. At an individual level, there are countless* articles on personal change, self-improvement, and performance ‘hacks’ that individuals can do to better themselves and supposedly achieve more in what they do.

Psychology and related behavioural sciences have spent inordinate time learning how individuals and organizations change by emphasizing specific behaviours, decision processes, and data that can support action. A close inspection will find that relatively few strategies produce consistent results and this has to do less with execution, skill or topic and more with the system in which these strategies are introduced.

To illustrate this, consider the role of learning in the organization and how our strategies to promote it ultimately fail when our systems are not designed to support it.

Knowledge integration: A case study

Consider the example of attending a conference as a means of learning and integrating knowledge into practice.

Surajit Bhattacharya published a primer for how to get value from conferences in medicine, pointing to tips and strategies that a medical practitioner can take such as arriving a day early (so you’re not groggy), planning out your day, and be social. These are all practical, logical suggestions, yet they are premised upon a number of things that we might call system variables. These include:

  • The amount of control you have over your schedule week-to-week.
  • The availability of transportation and accommodation options that suit your schedule, budget, and preferences.
  • The nature and type of work you do, including the amount of hours and intensity of the work you perform in a typical week. This will determine the amount of energy you have and the readiness to be attentive.
  • The volume of email and other digital communications (e.g., messages and updates via social media, chat, project management platforms) you receive on a daily basis and the nature of those kinds of messages (e.g.urgency and importance).
  • The amount and nature of travel required to both attend the event and the amount you had prior to attending the event.
  • The level of rest you’ve had. Sleep amount, timing, and quality all factor into how much rest you get. Add in the opportunity to engage in an activity like walking, exercise or stretching that one might do and we see a number of factors that could influence learning performance.
  • The setting. The lighting, air quality and air flow, seat arrangement, room acoustics, and access to some natural light are all factors in our ability to attend to and engage with a learning event.
  • The quality and format of the content and its delivery. Speaker quality, preparation, content and overall performance will all contribute to the ability to convey information and engage the audience.
  • Food and drink. Are you eating the kinds of foods and beverages that enable your body’s performance? Do you have access to these foods and drinks? Are they served at times that suit your body?
  • Your level of comfort and skill at engaging strangers. This matters if you’re more introverted, dislike small talk, or are not energized by others.

These are all platform issues: those in which motivation and energy can be channeled to focus on and engage with learning content. The fewer of these factors present the greater the energy expenditure needed on the part of the learner.

Learning within systems

W. Edwards Deming noted that most of the issues of performance in any organization were due to processes and systems (estimated to be up to 85% or more) rather than individual employees. While Deming was referring largely to manufacturing contexts, the same might be said for learning.

Consider our example from earlier about the conference. We’ve already outlined the factors that could contribute to learning at the conference itself, but let’s extend the case further to what happens after the conference. After all, a surgeon, engineer, computer programmer, law clerk, or carpenter isn’t going to practice her or his craft at the conference; they’ll do it when they return to regular work.

Now consider what our attendee encounters after they have made the trip home to apply this newfound learning:

  • A backlog of emails, phone messages and other correspondence that has either been left untouched, scantly attended to, or fully managed. In the first case, the backlog might be high and requires a considerable amount of time and energy to ‘catch up’ on upon return, however at least the learner was fully present to perform the many activities suggested byBhattacharya in the earlier article. In the second case, there is a higher than usual amount to attend to and the learner might have been selectively disengaged from the learning event. In the third, the learner returns to usual life without a backlog but may have sacrificed considerable attention toward the usual correspondence than actually learning.
  • A backlog of meetings. Scheduled meetings, calls or other events that require a co-presence (virtual or physical) that were put off due to travel are now picked up.
  • A backlog of administrative tasks. Submitting receipts and conference expenses, regular accounting or administrative tasks are all things that either was left untouched or, in the case of submitting expenses, unlikely or impossible to do until the trip has returned.
  • Fatigue. Sitting in a conference can be exhausting, particularly because of the conditions of the rooms, the volume of content and the break in the routine of every day (which can be energizing, too). Add in any travel issues that might arise and there is a reasonable chance that a person is not in an optimal state to take what they have been exposed to and apply it.
  • The usual organization processes and structures. Are there are opportunities to reflect upon, discuss, and process what has been learned with others and spaces to apply those lessons directly with appropriate feedback? How often have we been exposed to inspiring or practical content only to find few opportunities to apply it in practice upon our return in enough time before the details of the lessons fade?

It’s not reasonable to expect to have optimal conditions in our work much of the time, if ever. However, as you can see there are a lot of factors that contribute to our capacity to learn and the required energy needed to take what we’ve been exposed to and integrate it into our work. The fewer of these situations in place, the greater the likelihood that the investment in the learning experience will be lost.

An organization or individual requires a platform for learning that includes systems that allow for learners to be at their best and to provide a means for them to take what they learn and apply it — if it’s valuable. Otherwise, why invest in it?

This isn’t to say that no good can come from a conference, but if the main focus is on actual learning and the application of knowledge to the betterment of an organization and individual why would we not invest in the platform to make use of that rather than discarding it.

Rethinking our systems

When I was doing evaluation work in continuing medical education I was amazed to see how often learning events were held at 7 or 8 am. The rationale was that this was often tied to shift changes at hospitals and were the one time of day when most physicians were least likely to have other appointments. This was also the time when physicians were either highly fatigued from a night shift or having battled traffic on their commute to work or were planning the rest of their day ahead — all circumstances when they might be least focused on actually learning.

This choice of time was done for scheduling purposes, not for learning purposes. Yet, the stated purpose of continuing education was to promote learning and its various outcomes. Here, the strategy was to expose medical professionals to necessary, quality content to keep them informed and skilled and doing it at a time that appeared most convenient for all is an example of an idea that had logic to it, but ultimately failed in most regards.

How? If one looked at the evaluation data, typically the results suggested this strategy wasn’t so bad. Most often post-event surveys suggested that the overall ratings were consistently high. Yet a closer look at the data yields some questions.

For example, the questions asked to assess impact were things like: did the presenter speak clearly? or did the presenter provide the content they said they would? In most cases, participants were asked if the speaker arrived on time, presented what they said they would, were intelligible and whether there was a chance the learner might find useful what was presented. It had little to no bearing on whether the content was appropriate, impactful or applied in practice. This is because the system for evaluation was based on a model of knowledge transmission: content is delivered to a person and, assuming the content is good, the lesson is learned.

We know this to be among the weakest forms of moving knowledge to action and certainly not something suited to more complex situations or conditions, particularly in health systems. This is still what prevails.

Design for learning

If you’re seeking to promote learning and create a culture where individuals across an organization can adapt, develop, and grow learning requires much more than simply sending people to conferences, hosting seminars, providing books and other materials or watching some instructional videos. Without a means to integrate and promote that new knowledge as part of a praxis, organizations and individuals alike will continue to get frustrated, lag in their efforts to anticipate and respond to changing conditions and will ultimately fail to achieve anything close to their potential.

Designing for learning is as much about a curriculum as the context for how that curriculum is delivered and how learners are set up to engage with it all in their organizations and everyday lives.

*This is literally the case because the volume of new articles being published daily is so high.

If you’re looking to create learning systems in your organization, visit Cense to explore what it can do for you in shaping your strategy and evaluation to support sustainable, impactful learning for complex conditions. 

Image credit: “Platform” by Martin L is licensed under CC BY 2.0

 

innovationpsychologysocial systemssystems thinking

What’s in your baggage?

on April 19, 2017

IMG_0494

Social and emotional baggage is what we bring with us and, like the real thing, it’s all in what we pack, how we pack it and how aware we are of what we’re packing it for. And like the real thing, individuals and organizations can’t go on much of a journey without carrying some of it with us.

Social and emotional baggage is a concept that has been drawn from psychology to metaphorically represent the things — memories, expectations, experiences — that we bring from our past into our present context, usually in a negative, debilitating or otherwise limiting way. It’s something that is attributed to individuals, but may also fit organizations, too. But baggage is a part of any trip and looking at it as part of the journey might provide us better ways to use it, rather than ignore it.

Metaphors are useful ways to take what we cannot see or touch, but nonetheless are experienced as real and makes the concept more tangible. The metaphor of baggage is useful because we can look at it in many different ways.

Baggage is what we bring with us on our journey and, like real baggage, it matters what we bring, how much of it, and how amenable it is to being moved.

What’s in your carry-on?

Let’s consider what we might bring with us this journey of ours and consider what roles these items play in our lives and organizations. What’s going in our bag?

Clothing: Different outfits allow us to transform our appearance, to stand out or blend in, or express ourselves in creative ways (along with keeping us comfortable while avoiding the whole “naked in public” thing that’s a bit problematic in most cultures). Spare clothes provide us with the ability to envision ourselves in different forms. They represent our ability to adapt and to dream new futures. There’s a difference between having the latitude to transform and having so much stuff that either it’s not all that different or is so different, we lose our sense of who we are. When we’ve packed too much of anything, we get lost in choice and focus. If we don’t pack the right stuff, we lose our ability to adapt to changing conditions.

Toiletries: A basic toiletry set allows us to care for ourselves, maybe even make ourselves up a little. These are the things that repair the damage from day-to-day wear and tear on the body, heal, and protect ourselves from the wear to come and prevent future damage (hello, sunscreen!). We might also wish to make ourselves up a little sometimes, too (hello, lipstick or cologne!). If we pack too much of these things we can get vaingloriously trapped in what others might think of us and present a face that’s less authentic than our true selves. We might also be so focused on repair and prevention that we fail to recognize what’s in front of us in the present moment (the only moment when we can do anything to change the game).

Gifts: These are things we bring to others based on our experience and are shared best through acts of service, kindness, generosity and love. They may be souvenirs, stories, photos and keepsakes – things given and transported with care for others and might include sharing our knowledge (tacit and explicit) and experience with others through storytelling. Our fellow travellers benefit from our gifts, and so do we as they often bring joy to the giver through the giving. However, if we pack our bags with too many we may wind up looking more after others than ourselves. Our focus is on giving to others at the expense of caring for ourselves. Our bags only have so much room and gifts take up some of that room.

Memories & Experience: This is our past. This is the part that accumulates over time as we get older and experience the world. It builds on and continually adds to our carriage, meaning we need to consider how we pack it, what we choose to hold on to from this vast collection, and what we might want to discard. This is where wisdom resides. It’s also a seat of some of our biggest problems. If real healing — that integration of experience with understanding, reflection and growth through our social life — doesn’t take place, we might find ourselves with things in our bag that we’d thought we’d discarded, but didn’t. It’s like finding the scorpion that might have hid in your luggage from your tropical vacation as you go home. If we’re not careful and mindful about what we pack, we might let in things we thought we’d left behind.

Cargo: This last element is cartage from one space to another. This is the stuff we bring from one place to another that may or may not have any purpose. It’s ‘stuff’. It’s trivia, the news, pop culture, or the flotsam and jetsam of everyday life. It seems like this would be the easiest thing to disregard, but it’s not. It can take up a lot of room. Consider where you put your attention and what you consume in a day — your social media feeds, gossip at the office or the dog park, advertisements, broadcast media — and what you hang on to. This is the stuff that can make us lose sight of what we think is important, so it’s critical that we are mindful about what cartage we add to our baggage.

As my colleague Alex Jadad says:

Nothing really matters apart from what matters to us. Therefore, we must be very careful about what we choose to matter to us.

Packing what matters most

Our baggage represents a system. We create the boundaries by the shape and size of the bag (or bags) we choose to bring with us and all of what goes in the bag interacts (it all has to work together to fit). What happens when we pack piecemeal, we throw all of the stuff above into a bag and try and organize it. Sometimes we seek to bring more bags or getting a bigger suitcase; maybe that will work. But things don’t fit, it doesn’t go together. Or we find ourselves laden with luggage, slow to move, strained in the back and joints from carting it around…but at least we have our stuff, right? (if we can find it).

The problem with that ‘add more’ strategy is that, the more we have, the slower we are, the more encumbered, and the more confused. We add to complexity, rather than create simplicity. We need to design better.

The best packers are those who create extra space for things they’ll pick up along the way, put in things that go together (e.g., outfits that mix and match), and they determine their essentials ahead of time. They spend the time considering what is most important, most used, most necessary and organize around those things: they employ strategy (and they adapt their strategy along their journey). They know what they need, what they like, and what makes them comfortable, safe and happy on their journey. Why? They’ve paid attention and collected data to support that decision (e.g. through ongoing evaluation, reflective practice, mindfulness, personal therapy).

This is all about being mindful about our work and life. For organizations, there are things you can do to create mindfulness in the way you work to help understand the choices you make and their consequences. For individuals, it’s about doing self-development work and engaging in reflective practice — in work and life.

If we don’t know what’s in our bags, we might be surprised what comes out. The subconscious works that way: it will pop things out at times of its choosing when we are often not expecting it or desiring it. Subconscious processes work at the individual and collective levels — it’s not just a personal thing.

Mindful packing

We are contributors to the story of our lives, but not the sole authors (despite what many seem to think). What has happened to us because of others matters as much as what we create for ourselves. It’s not about labeling those experiences as ‘good’ things and ‘bad’ things, rather dealing with the consequences that those experiences bring to our life in the here and now and asking if they are helpful or unhelpful to living the life we want.

Were you under appreciated by your colleagues or family? Bullied? Neglected? Think it’s all in the past? Replicate that situation in the present and see how you feel — it might not be all in the past. Abuse and neglect are common experiences at home and work and how we integrate that into our lives — or whether we do at all — can be a key factor in determining how we relate to the experiences in the present.

What about that project at work that got everyone excited and failed to deliver the value that everyone expected or felt promised? Is that going to temper the willingness to try again, to innovate or risk something new again? Organizations and teams might be tempted to ‘lower the bar’ to avoid disappointment, despite suggestions that an organizations’ settling for ‘mediocrity’ drives quality people away from work. 

What about that romantic relationship that was perfect, but ended because it was too perfect? How is too perfect even possible, you ask? If you’re not accustomed to being loved and cared for you might find it very uncomfortable to get exactly what you want (and need) and find (invent?) reasons why the relationship won’t work and end it (or sabotage it so you don’t have to end it). The issue isn’t that you don’t aspire for this ‘perfect love’, it’s that you’re not used to it; maybe the only way you were loved before was through neglect, abuse, or simple disinterest and partner disengagement. When that changes, so does our narrative about what real love is all about and if we don’t ‘flip the script’ we’ll write the new story into the old one and that just won’t work. (And if you’ve not experienced this in romance, how about a job? A friendship? An opportunity at work? — self sabotage is very real and underexamined).

None of this is crazy-making: it’s just how some people deal with the intense sadness of not knowing how to be loved or to achieve real success.

Like anything in life: change challenges us and these experiences (fears, hopes, unrequited dreams, and victories) go into our luggage and often not by choice — or awareness. But knowing this can happen will help us understand our baggage and how our past and hopes for the future affect the present.

The temptation is to make some sort of judgement about baggage and assume its a problem. We bring what we need with us and that means that we each will have and will need baggage differently. What we wish is for it to serve us, not debilitate us or keep us from growing. Being mindful, reflective and careful about what we have already packed is another critical step.

The next is realizing that, like real baggage, we can re-pack. We can discard things, re-organize, re-prioritize what goes inside. It’s never too late, but it does require work. The best thing of all is that, when we reorganize our baggage we create more of what we want — what matters to us — and less of what we don’t want or others want for us. Our baggage is our asset if we allow it to be.

Reflective practice, healing (to be covered in a future post in more depth), compassion (because we all have things we wish weren’t packed for us in our bags – be good to ourselves), systems thinking (and design thinking — creatively considering how we pack, not just what we pack) and healthy social engagement are all ways to improve our relationship with baggage.

And that’s something to hang our hats on.

Happy, healthy travels.

IMG_0382

Image credits: Author

 

 

social innovationsocial systemssystems thinking

Lost together

on July 13, 2016
Lost and found

A post certainty world

Doing new things to create social value means going into the great unknown, yet our fear of being lost need not prevent us from innovating, wisely and sustainably. Instead of being lost alone, we can be lost together. 

I’ve heard it all so many times before

It’s all a dream to me now
A dream to me now
And if we’re lost
Then we are lost together

– Blue Rodeo (Lost Together)

Humans have real problems with uncertainty. Risk mitigation is an enormous field of work within business, government and politics and permeates decision making in our organizations. It’s partly this reason that our politicians too often speak so cryptically to the point of basically uttering nonsense – because they want to avoid the risk of saying something that will hurt them. The alternative perhaps is to spout so much untruth that it no longer matters what you say, because others will create messages about you.

Thankfully, we are still — and hopefully into the future — in a world where most of what organizations do is considered and evaluated with some care to the truth. ‘Truth’ or facts are much easier to deal with in those systems where we can generate the kind of evidence that enable us to make clear decisions based on replicable, verifiable and defensible research. Ordered systems where there is a cause-and-effect relationship that is (usually) clear, consistent and observable are those where we can design interventions to mitigate risk with confidence.

Risky options

There are four approaches to risk mitigation.

  1. Risk Acceptance involves awareness of what risks are present within the system and establishing strategy and an organizational culture where the nature, type and potential consequences of risks are (largely) known, accepted and lived with.
  2. Risk Avoidance takes the opposite approach and seeks to steer operations away from activities where risk is limited.
  3. Risk Limitation seeks to curtail and mitigate the effects of risk where possible and often involves contingency planning and balancing activities with higher levels of uncertainty with areas of greater confidence and certainty.
  4. Risk Transference involves finding ways to offload risk to a third party. An example can be found in many partnerships between organizations of different sizes or types where one is able to absorb certain risks while others cannot for various reasons and the activities allow for one partner to take lead on an activity that isn’t feasible for another to do so.

Within social innovation — those activities involving public engagement, new thinking and social benefit — there are few opportunities for #2, plenty for #1 and #3 and a growing number for #4.

Risk is a core part of innovation. To innovate requires risking time, energy, focus and other resources toward the attempt at something new to produce a valued alternative to the status quo. For many human service organizations and funders, these resources are so thinly spread and small in abundance that the idea of considering risk seems like a risk itself. Yet, the real problem comes in assuming that one can choose whether or not to engage risk. Unless you’re operating in a closed system that has relatively few changing elements to it, you’re exposed to risk by virtue of being in the system. To draw on one of my favourite quotes from the author Guiseppe di Lampedusa:

If we want things to stay as they are, things will have to change.

So even keeping things away from risk involves risk because if the world around you is changing the system changes with it and so, too does your position in it. If this makes you feel lost, you’re not alone. Many organizations (individuals, too) are struggling with figuring out where they fit in the world. If you want evidence of this consider the growing calls for skilled knowledge workers at the same time we are seeing a crisis among those with a PhD — those with the most knowledge (of certain sorts) — in the job market.

Community of flashlights

There is a parable of the drunkard who loses his keys on his way home at night and searches for them under the streetlight not because that’s where he lost them, but because it’s easier to see that spurred something called the Streetlight Effect. It’s about the tendency to draw on what we know and what we have at our disposal to frame problems and seek to solve them, even if they are the wrong tools — a form of observation bias in psychology. Streetlights are anchored, stable means of illuminating a street within a certain range – a risk zone, perhaps — but remain incomplete solutions.

Flashlights on the other hands have the same limitations (a beam of light), are less powerful, but are adaptive. You can port a flashlight or torch and aim it to where you want the light to shine. They are not as powerful as a streetlight in terms of luminosity, but are far more nimble. However, if you bring more than one flashlight together, all of a sudden the power of the light is extended. This is the principle behind many of the commercial LED systems that are in use. Small numbers of lights brought together, each using low energy, but collectively providing a powerful, adaptive lighting system

27242239234_91fb8d5b05_o

This same principle can apply to organizations seeking to make change. Like an LED flashlight, they need a housing to hold and focus the lights. This can be in the form of a backbone organization such as those advocated in collective impact strategies. It can also be a set of principles or simple rules that provide a set of guides for organizations operating independently to follow, which will stimulate a consistent pattern of activity when applied, allowing similar, focused action on the same target at a distance.

This latter approach differs from collective impact, which is a top-down and bottom-up approach simultaneously and is a good means of focusing on larger, macro issues such as poverty reduction, climate change and city-building. It is an approach that holds potential for working within these larger issues on smaller, more dynamic ones such as neighbourhood building, conservation actions within a specific region, and workplace health promotion. In both cases the light analogy can hold and they need not be done exclusive of one another.

Let there be light

A flashlight initiative requires a lot of things coming together. It can be led by individuals making connections between others, brokering relationships and building community. It requires a vision that others can buy into and an understanding of the system (it’s level of complexity, structure and history). This understanding is what serves as the foundation for the determination of the ‘rules’ of the system, those touch points, attractors, leverage points and areas of push and pull that engage energy within a system (stay tuned to a future post for more detailed examples).

Much of the open-source movement is based on this model. This is about creating that housing for ideas to build and form freely, but with constraints. It’s a model that can work when collective impact is at a scale too large for an organization (or individual) to adequately envision contribution, but an alternative to going alone or relying only on the streetlight to find your way.

You might be lost, but with a flashlight you’ll be lost together and may just find your way.

Image credits: Author (Cameron Norman)

complexityevaluationsystems thinking

Diversity / Complexity in Focus

on June 27, 2016
5901575754_a7ca159cd4_b

Diversity in focus?

 

As cities and regions worldwide celebrate Pride the role of diversity, understanding and unity has been brought to mind just as it has contemplating the British public’s choice to leave the EU with Brexit. Both events offer lessons in dealing with complexity and why diversity isn’t about either/or, but more both and neither and that we might learn something not just from the English, but their gardens, too. 

It’s a tad ironic that London has been celebrating its Pride festival this past week, a time when respect and celebration of diversity as well as unification of humanity was top-of-mind while the country voted to undo many of the policies that fostered political and economic union and could likely reduce cultural diversity with Europe. But these kind of ironies are not quirky, but real manifestations of what can happen when we reduce complexity into binaries.

This kind of simplistic, reductionist thinking approach can have enormously harmful and disrupting effects that ripple throughout a system as we are seeing with what’s happened so far in the United Kingdom, Europe and the world in the past week.

Complexity abhors dichotomies

There are two kinds of people in the world: those who believe there are two kinds of people and those who don’t

The above quote (which has many variations, including one attributed to author Tom Robbins that I like) makes light of the problem of lumping the complex mass of humanity into two simple categories. It abstracts variation to such a level that it becomes nearly meaningless. The Brexit vote is similar. Both are lessons in complexity lived in the world because they reflect a nuanced, mutli-faceted set of issues that are reduced into binary options that are clustered together.

It is no surprise that, in the days following the Brexit vote in the UK, that there is much talk of a divided, rather than a united kingdom.

Diversity is difficult to deal with and is often left unaddressed as a result. The benefits to having diversity expressed and channeled within a complex system are many and articulated in research and practice contexts across sectors and include protection from disruption, better quality information, a richer array of strategic options and, in social contexts, a more inclusive social community.

The risks are many, too, but different in their nature. Diversity can produce tension which can be used for creative purposes, liberation, insight as well as confusion and conflict, simultaneously. This as a lot do with humans uneasy relationship with change. For some, change is easier to deal with by avoiding it — which is what many in the Leave camp thought they could do by voting the way they did. The darker side of the Leave campaign featured change as an image of non-white immigrant/refugees flooding into Britain, presumably to stoke those uncomfortable with (or outwardly hostile) to others to fear the change that could come from staying in the European Union.

Staying the same requires change

The author Guiseppe de Lampedussa once wrote about the need to change even when desiring to keep things as they are, because even if we seek stability, everything around us is changing and thus the system (or systems) we are embedded in are in flux. That need to change to stay the same was something that many UK citizens voiced. What was to change and what was to stay the same was not something that could be captured by a “Leave” or “Remain” statement, yet that is what they were given.

It should come to no surprise that, when presented with a stark choice on a complex matter, that there would be deep dissatisfaction with the result no matter what happened. We are seeing the fallout from the vote in the myriad factions and splintering of both of the main political parties — Conservative and Labour — and a House of Commons that is now filled with rebellion. Is the UK better off? So far, no way.

This is not necessarily because of the Leave vote, but because of what has come from the entire process of mis-handling the campaigns and the lack of plan for moving forward (by both camps). Further complicating matters is that the very EU that Britain has voted to leave is now not the same place as it was when the Brexit vote was held just five days ago. It’s also faced with rising voices for reform and potential separation votes from other member states who saw their causes bolstered or hindered because of the UK referendum. This is complexity in action.

Tending the garden of complex systems

The English know more about complexity than they might realize. An English garden is an example of complexity in action and how it relates to the balance of order, disorder and unordered systems. A look at a typical English garden will find areas of managed beauty, wildness, and chaos all within metres of one another.

9556155847_5eaf2a7eee_h

What also makes a garden work is that it requires the right balance of effort, adaptive action, planning and compensating as well as the ability to let go all at the same time. Gardening requires constant attention to the weather, seasons, the mix of diversity within the system, the boundaries of the system itself (lest weeds or other species seek to invade from outside the garden or plants migrate out to a neighbours place) and how one works with all of it in real time.

Social systems are the same way. They need to be paid attention to and acted upon strategically, in their own time and way. This is why annual strategic planning retreats can be so poorly received. We take an organization with all it’s complexity and decide that once per year we’ll sit down and reflect on things and plan for the future. Complexity-informed planning requires a level of organizational mindfulness that engages the planning process dynamically and may involve the kind of full-scale, organization-wide strategy sessions more frequently or with specific groups than is normally done. Rather than use what is really arbitrary timelines — seen in annual retreats, 5-year plans and so forth — the organization takes a developmental approach, like a gardener, and tends to the organizations’ strategic needs in rhythms that fit the ecosystem in which it finds itself.

This kind of work requires: 1) viewing yourself as part of a system, 2) engaging in regular, sustained planning efforts that have 3) alignment with a developmental evaluation process that continually monitors and engages data collection to support strategic decision-making as part of 4) a structured, regular process of sensemaking so that an organization can see what is happening and make sense of it in real-time, not retrospectively because one can only act in the present, not the future or past.

Just as a garden doesn’t reduce complexity by either being on or off, neither should our social or political systems. Until we start realizing this and acting on it — by design — at the senior strategic level of an organization, community or nation, we may see Brexit-like conditions fostered in places well beyond the white cliffs of Dover into governments and organizations globally.

Photo Credits: The London Eye Lit Up for Pride London by David Jones and Hidcote Manor GardenHidcote Manor GardenHidcote Manor Garden by David Catchpole both used under Creative Commons License via Flickr. Thanks to the Davids for sharing their work.

evaluationsocial innovation

Benchmarking change

on April 4, 2016

The quest for excellence within social programs relies on knowing what excellence means and how programs compare against others. Benchmarks can enable us to compare one program to another if we have quality comparators and an evaluation culture to generate them – something we currently lack. 

5797496800_7876624709_b

A benchmark is something used by surveyors to provide a means of holding a levelling rod to determine some consistency in elevation measurement of a particular place that could be compared over time. A benchmark represents a fixed point for measurement to allow comparisons over time.

The term benchmark is often used in evaluation as a means of providing comparison between programs or practices, often taking one well-understood and high performing program as the ‘benchmark’ to which others are compared. Benchmarks in evaluation can be the standard to which other measures compare.

In a 2010 article for the World Bank (PDF), evaluators Azevedo, Newman and Pungilupp, articulate the value of benchmarking and provide examples for how it contributes to the understanding of both absolute and relative performance of development programs. Writing about the need for benchmarking, the authors conclude:

In most benchmarking exercises, it is useful to consider not only the nature of the changes in the indicator of interest but also the level. Focusing only on the relative performance in the change can cause the researcher to be overly optimistic. A district, state or country may be advancing comparatively rapidly, but it may have very far to go. Focusing only on the relative performance on the level can cause the researcher to be overly pessimistic, as it may not be sufficiently sensitive to pick up recent changes in efforts to improve.

Compared to what?

One of the challenges with benchmarking exercises is finding a comparator. This is easier for programs operating with relatively simple program systems and structures and less so for more complex ones. For example, in the service sector wait times are a common benchmark. In the province of Ontario in Canada, the government provides regularly updated wait times for Emergency Room visits via a website. In the case of healthcare, benchmarks are used in multiple ways. There is a target that is used as the benchmark, although, depending on the condition, this target might be on a combination of aspiration, evidence, as well as what the health system believes is reasonable, what the public demands (or expects) and what the hospital desires.

Part of the problem with benchmarks set in this manner is that they are easy to manipulate and thus raise the question of whether they are true benchmarks in the first place or just goals.

If I want to set a personal benchmark for good dietary behaviour of eating three meals a day, I might find myself performing exceptionally well as I’ve managed to do this nearly every day within the last three months. If the benchmark is consuming 2790 calories as is recommended for someone of my age, sex, activity levels, fitness goals and such that’s different. Add on that, within that range of calories, the aim is to have about 50% of those come from carbohydrates, 30% from fat and 20% from protein, and we a very different set of issues to consider when contemplating how performance relates to a standard.

One reason we can benchmark diet targets is that the data set we have to set that benchmark is enormous. Tools like MyFitnessPal and others operate to use benchmarks to provide personal data to its users to allow them to do fitness tracking using these exact benchmarks that are gleaned from having 10’s of thousands of users and hundreds of scientific articles and reports on diet and exercise from the past 50 years. From this it’s possible to generate reasonably appropriate recommendations for a specific age group and sex.

These benchmarks are also possible because we have internationally standardized the term calorie. We have further internationally recognized, but slightly less precise, measures for what it means to be a certain age and sex. Activity level gets a little more fuzzy, but we still have benchmarks for it. As the cluster of activities that define fitness and diet goals get clustered together we start to realize that it is a jumble of highly precise and somewhat loosely defined benchmarks.

The bigger challenge comes when we don’t have a scientifically validated standard or even a clear sense of what is being compared and that is what we have with social innovation.

Creating an evaluation culture within social innovation

Social innovation has a variety of definitions, however the common thread of these is that its about a social program aimed at address social problems using ideas, tools, policies and practices that differ from the status quo. Given the complexity of the environments that many social programs are operating, it’s safe to assume that social innovation** is happening all over the world because the contexts are so varied. The irony is that many in this sector are not learning from one another as much as they could, further complicating any initiative to build benchmarks for social programs.

Some groups like the Social Innovation Exchange (SIX) are trying to change that. However, they and others like them, face an uphill battle. Part of the reason is that social innovation has not established a culture of evaluation within it. There remains little in the way of common language, frameworks, or spaces to share and distribute knowledge about programs — both in description and evaluation — in a manner that is transparent and accessible to others.

Competition for funding, the desire to paint programs in a positive light, lack of expertise, not enough resources available for dissemination and translation, absence of a dedicated space for sharing results, and distrust or isolation from academia among certain sectors are some reasons that might contribute to this. For example, the Stanford Social Innovation Review is among the few venues dedicated to scholarship in social innovation aimed at a wide audience. It’s also a venue focused largely on international development and what I might call ‘big’ social innovation: the kind of works that attract large philanthropic resources. There’s lot of other types of social innovation and they don’t all fit into the model that SSIR promotes.

From my experiences, many small organizations or initiatives struggle to fund evaluation efforts sufficiently, let alone the dissemination of the work once it’s finished. Without good quality evaluations and the means to share their results — whether or not they cast a program in positive light or not — it’s difficult to build a culture where the sector can learn from one another. Without a culture of evaluation, we also don’t get the volume of data and access to comparators — appropriate comparators, not just the only things we can find — to develop true, useful benchmarks.

Culture’s feast on strategy

Building on the adage attributed to Peter Drucker that culture eats strategy for breakfast (or lunch) it might be time that we use that feasting to generate some energy for change. If the strategy is to be more evidence based, to learn more about what is happening in the social sector, and to compare across programs to aid that learning there needs to be a culture shift.

This requires some acknowledgement that evaluation, a disciplined means of providing structured feedback and monitoring of programs, is not something adjunct to social innovation, but a key part of it. This is not just in the sense that evaluation provides some of the raw materials (data) to make informed choices that can shape strategy, but that it is as much a part of the raw material for social change as enthusiasm, creativity, focus, and dissatisfaction with the status quo on any particular condition.

We are seeing a culture of shared ownership and collective impact forming, now it’s time to take that further and shape a culture of evaluation that builds on this so we can truly start sharing, building capacity and developing the real benchmarks to show how well social innovation is performing. In doing so, we make social innovation more respectable, more transparent, more comparable and more impactful.

Only by knowing what we are doing and have done can we really sense just how far we can go.

** For this article, I’m using the term social innovation broadly, which might encompass many types of social service programs, government or policy initiatives, and social entrepreneurship ventures that might not always be considered social innovation.

Photo credit: Redwood Benchmark by Hitchster used under Creative Commons License from Flickr.

About the author: Cameron Norman is the Principal of Cense Research + Design and works at assisting organizations and networks in supporting learning and innovation in human services through design, program evaluation, behavioural science and system thinking. He is based in Toronto, Canada.

systems thinking

Thinking systemically about systems thinking

on September 4, 2014
Carnaby Street

The Whole and the Parts

Systems thinking is a class of theories, models and methods for understanding human and non-human interactions as seen as wholes instead of parts. This focus on interconnections and relationships is precisely what makes it challenging for many when it comes to systemically considering what systems thinking is all about and the implications of this are many. This post provides an introduction to certain ideas in systems thinking and points to what makes it different than other non-systems thinking approaches to understanding something. 

Perhaps the most popular aphorism about systems thinking is the statement that the whole is greater than the sum of its parts, something borrowed from Gestalt Psychology. That statement is intended to reflect system thinking’s principal focus on the system itself rather than on the actors and actions within it.

It’s a subtle difference, but a meaningful one. For example, psychology might look at why individuals make choices and act and what implications come from those actions. Systems thinking seeks to look at the combined interaction of these interactions as a unified whole.

Boundaries

Fundamental to this way of seeing things is the concept of boundaries. Boundaries are essentially where the differences that make a difference lie. In a closed system, everything that makes a difference is clearly contained and observed within a relatively solid set of boundary conditions. Mechanical systems often function this way, making them simple or complicated in that they have the potential to be understood clearly in terms of causal connections and relations. These systems are more amenable to things like “best practices” where we can reasonably expect similar outcomes from consistent actions.

This kind of systems thinking is not as useful when applied to human systems, because they are mostly characterized as open systems. Open systems are those where the boundaries require some form of negotiation and may actually be in flux.

A general shorthand rule for setting boundaries in this kind of environment is this:

If you find yourself lost over and again in trying to understand where the influences and relationships within the system are, then you’ve probably bound your system too loosely. If you are finding too many influences laying outside of your boundaries, you’ve probably bound it too tightly.

Perspective: Where you sit

Systems are all about where you sit in relation to them. For instance, let’s take the example of family and some of the boundary questions one might ask in understanding this social entity as a system.

  • Firstly, who is family? You could define family as blood relationships. But is that immediate blood relations? For example, If parents and children count, then how do we consider grandparents who are the parents of the parents? Do they count as family when you bound the system? Do great grandparents? Should we use genes and, if so, what level of genetic similarity do we share? Are we all family?
  • Can family be defined socially? For example, if people become family by marriage and that marriage breaks down, does it influence the family system as you define it? What if that marriage ends via someone passing away? What if they are not married at all, but common law?
  • What about the roles that people play? Does an “Uncle” or “Aunt” who are close, intimate friends of the family, but not of blood ties still get included in the family? How about a trusted lifelong neighbour who has been a part of someone’s life the entire time, but was never genealogically connected to anyone?
  • Can our neighbourhood be part of the family?

One can make a case for any of these conditions. In defining a system there is no ‘good’ or ‘bad’ way to do it, just perspectives that are more or less useful and more or less attentive to specific details.

Purposeful systems

The answers to the questions about boundaries also depend on what the purpose of the system is in the first place. Purpose is the means by which we determine the differences and how they make a difference. You can imagine that one could potentially answer “yes” to almost every one of the questions asked above depending on where someone sits in the system and what kind of purpose they see in that system.

Part of thinking systemically about systems is defining the purpose of the system and ascertaining a perspective. That means being strategic about what you wish your systems thinking to support. It is here that much of the use of systems thinking I’ve witnessed breaks down. Organizations seeking to employ systems thinking often jump in without doing the pre-work needed to ground their perspective into some sense of purpose and perspective. This requires a mindful, honest accounting of the perspectives being brought into the discussion and connecting those to the strategic intent of your enterprise.

Being mindful of what one values, what one seeks to accomplish, and what kind of activities your organization engages in (or wants to engage in), and where the reach of your organization extends is a key starting position to thinking more systemically about systems.

1 2 3