Tag: ideology


Of tails, dogs and the wagging of both

Who's wagging whom?

Who’s wagging whom?

Evaluation is supposed to be driven by a program’s needs and activities, but that isn’t always the case. What happens when the need for numbers, metrics, ‘outcomes’ and data shape the very activities programs do and how that changes everything is something that is worth paying some attention to. 

Since the Second World War we’ve seen a gradual shift towards what has been called presence of neo-liberal values across social institutions, companies, government and society. This approach to the world is characterized, among other things, by its focus on personal and economic efficiency, freedom, and policies that support actions that encourage both. At certain levels of analysis, these policies have rather obvious benefits.

Who wouldn’t like to have more choice, more freedom, more perceived control and derive more value from their products, services and outputs? Not many I suspect. Certainly not me.

Yet, when these practices move to different levels and systems they start to produce enormous complications that are at odds with — and produce distortions of — the very values that they espouse. We’ve seen the same happen with other value systems that have produced social situations that are highly beneficial in some contexts and oppressive and toxic in others – capitalism and socialism both fit this bill.

Invisible tails and wags

What makes ‘isms’ so powerful is that they can become so prevalent that their purpose, value and opportunity stop being questioned at all. It is here that the tail starts to wag the dog.

Take our economy (or THE economy as it is somewhat referred to). An economy is intended to be a facilitator and product of activities used to create certain types of value in a society. We work and produce goods (or ideas), exchange and trade them for different things, and these allow us to fulfill certain human goals. It can take various shapes, be regulated more or less, and can operate at multiple scales, but it is a human construction — we invented it. Sometimes this gets forgotten and in times when we use the economy to justify behaviour we forget that it is our behaviour that is the economy.

We see over and again with neoliberalism (which is among the most dominant societal ‘ism’ of the past 50 years in the West and more reflected globally all the time) taken at the broadest level, the economy becomes the central feature of our social systems rather than a byproduct of what we do as social beings. Thus, things like goods, experiences, relations and so on we used to consider as having some type of inherent value suddenly become transformed into objects that judgements can be made.

The role of systems

This can make sense where there are purpose-driven reasons to assign particular value scores to something, but the nature of value is tied to the systems that surround what is valued. If we are dealing with simple systems, those where there are clear cause-and-effect connections between the product or service under scrutiny and its ability to achieve its purpose, then valuation measurement makes sense. We can assert that X brand of laundry detergent is better than Y on the basis of Z. We can conduct experiments, trials and repeated measures that can compare across conditions.

It is also safe to make an assumption of value based on the product’s purpose that can be generalized. In other words, our reason for using the product is clear and relatively unambiguous (e.g., to clean clothes using the above example). There may be additional reasons for choosing X brand over Y, but most of those reasons can be also controlled for and understood discretely (e.g., scent, price, size, bottle shape etc..).

This kind of thinking breaks down in complex systems. And to make it even more complex, it breaks down imperfectly so we have simple systems interwoven within complex ones. We have humans using simple products and services that operate in new, innovative and complex conditions. Unfortunately, what comes with simple systems is simple thinking. Because they are — by their nature — simple, these system dynamics are easy to understand. Returning to our example of the economy, classical micro-economic models of supply and demand as illustrated below.

Relationships and the systems that surround them


Using this model, we can do a reasonable job of predicting influence, ascertaining value and hypothesizing relationships between both.

In complex systems, the value links are often in flux, dynamic, and relative requiring a form of adaptive evaluation like developmental evaluation. But that doesn’t happen as much as it should, mostly because of a failure to question the systems and their influence. Without questioning the values and value that systems create — the isms that were mentioned earlier — and their supposed connection to outcomes, we risk measuring things that have no clear connection to value and worse, we create systems that get designed around these ineffective measures.

What this manifests itself in is mindless bureaucracy, useless meetings, pompous and intelligible titles, and innovation-squashing regulations that get divorced from the purpose that they are meant to solve. And in doing so, this undermines the potential benefit that the original purpose of a bureaucracy (to document and create an organizational memory to guide decisions), meetings (to discuss and share ideas and solve problems), titles (to denote role and responsibility — although these aren’t nearly as useful as people think in the modern organization), and regulations (to provide a systems lens to constrain uncoordinated individual actions from creating systems problems like the Tragedy of the Commons).

More importantly, this line of thinking also focuses us on measuring the things that don’t count. And as often quoted and misquoted, the phrase that is apt is:

Not everything that counts can be counted, and not everything that can be counted counts.

Counting what counts

It is critical to be mindful of the purpose — or to reconnect, rediscover, reinvent and reflect upon the purposes we create lest we allow our work to be driven by isms. Evaluators and their program clients and partners need to stand back and ask themselves: What is the purpose of this system I am dealing with?

What do we measure and is that important enough to matter? 

Perhaps the most useful way of thinking about this is to ask yourself: what is this system being hired to do? 

Regular mindful check-ins as part of reflective practice at the individual, organizational and, where possible, systems level are a way to remind ourselves to check our values and practices and align and realign them with our goals. Just as a car’s wheels go out of alignment every so often and need re-balancing, so too do our systems.

In engaging in reflective practice and contemplating what we measure and what we mean by it we can better determine what part of what we do is the dog, what is the tail and what is being wagged and by whom.

Photo credit: Wagging tail by Quinn Dombrowski used under Creative Commons License via Flickr. Thanks Quinn for making your great work available to the world.

Economic model image credit from Resources for Teachers used under Creative Commons License. Check out their great stuff for helping teachers teach better.

complexitydesign thinkinginnovationsocial systemssystems science

The Ideology of Scaling Social Innovations

Box scaling

Does it scale? That question is central to the discussion of social innovation, yet the answer to it might lead us to questions about why it is so important to us in the first place and answers that could surprise us. 

Does it scale?” or “how to we take [idea, product, service] to scale?” are commonly heard questions in social innovation circles; so much so that they are left unquestioned. The thinking behind these questions is that if something works well at one level (or scale) then taking it another scale larger and achieving a wider reach must be better. Who wouldn’t want to see the benefits of something that serves the needs of one population, community or user extended outward and upward?

This is a laudable utilitarian goal, but it is a deceptively problematic one when we look a little closer at what scaling something actually means in practice.

Conceptualizing Scale

Jamer Hunt, the Director of the MFA program in Transdisciplinary Design at the New School in New York, speaking at last year’s DMI Fall Conference (which is available to view for DMI members), looked at the issue of design scaling through the lens of complexity and pointed to some of the problems with ‘scaling design’ in varied contexts. One of the examples he suggested is that of an ant compared with a human being taking a shower. For humans, the shower’s droplets of water are fine bodies of liquid that perform a particular task of facilitating cleaning, but for an ant those same droplets are enormous orbs of potential death. Water doesn’t scale the same for a human and an ant even though it is the same substance at both levels and the shower is identical in its structure.

In physics this is called scalar variance. What works ideally for humans is terrible for ants even though we are speaking of the same substance, same planet, same context. Water (most notably, a shower of it) doesn’t scale well in this case.

Yet, there is this insatiable desire among those working in social innovation to “scale things up” and “bring our innovations to scale” (even if we have little concept of how that would look or — as I will discuss — what that really means). The adherence to scaling as an ideology in social innovation (and applied social science in general) is bordering on “four legs good, two legs better” territory.

The Cult of Efficiency

International affairs scholar Janice Gross Stein attributes some of this fascination with scaling to a cult of efficiency, a political ideology that assumes that we can always rationalize human services optimally. What she found is that efficiency is used falsely as a stand-in for accountability, particularly in fields like education. Far from being against striving for optimal use of scarce resources, Stein nonetheless concludes that efficiency in human systems doesn’t always scale (my phrase, not hers) and that bigger and faster is often not better. Anyone who has taken a lecture with hundreds of others knows the difference of scale in learning between that and a seminar of five to ten people.

Taking Jamer Hunt’s argument: Bigger is just bigger…and whether its better or not is dependent on whether you’re an ant, a human and need to come into contact with water.

Designing for Systems and Scale: The Powers of 10

Designers and systems thinkers probably know the movie “The Powers of 10” by legendary designers Charles and Ray Eames. It’s a fascinating short film that looks at the universe moving out from a human being into the cosmos and inward towards what would now be quarks and everything in between. It is perhaps the best example of scaling ever produced. Beyond its educational and entertainment value, the Powers of 10 provide an illustrative example of where striving for scaling social innovations could be foolish and where it could have potential.

When traveling through the universe it is easy to see scales that are self-similar, thus they share properties that make them optimally relatable. These forms are often fractal in nature (thus, they share the same properties at different scales like that of a snowflake). Imperfectly, certain scales in the Powers of 10 are close to self-similarity where one scale looks and shows behaviour similar to those adjacent to it. These are spaces where it may be possible to transport an innovation from one to the other to good effect. Others scales look radically different from one another, suggesting a mis-fit in the scalar variance.

This is an idea, not an empirical point as we have little research on scalar variance in social innovation. Scaling innovation makes greater sense when the social systems have similar structures and ‘shapes’ and less when they do not. It is why in organizational science, certain models of management and decision making transport well from setting to setting and others do not. It’s why we’ve seen quality improvement processes like Six Sigma achieve great success in certain industries and firms and spectacularly fail in others.

Rather than adhere to an ideology that imposes scaling as a goal, social innovators need to generate the kinds of intelligence about the systems they are operating in (or seeking to operate or expand into) before making plans for scaling a promising intervention or product. As funders and policymakers this means setting performance targets that are appropriate or, perhaps better yet, working developmentally with innovators to co-create the outcomes of interest and the measures and metrics used to determine scalability and appropriateness early in the design and implementation cycle.

Without best evidence (which is almost always lacking in social innovation by its very nature), setting performance targets related to scale a priori is foolish. For innovators themselves, equally foolish is not gathering the kind of information about the systems they are operating in to know if they are the human or the ant and whether a shower is on the way.