Evaluation and Design For Changing Conditions

1 Comment / design, evaluation, innovation, systems thinking / By
Growth and Development

The days of creating programs, products and services and setting them loose on the world are coming to a close posing challenges to the models we use for designing and evaluation. Adding the term ‘developmental’ to both of these concepts with an accompanying shift in mindset can provide options moving forward in these times of great complexity.

We’re at the tail end of a revolution in product and service design that has generated some remarkable benefits for society (and its share of problems), creating the very objects that often define our work (e.g., computers). However, we are in an age of interconnectedness and ever-expanding complexity. Our disciplinary structures are modifying themselves, “wicked problems” are less rare

Developmental Thinking

At the root of the problem is the concept of developmental thought. A critical mistake made in comparative analysis — whether through data or rhetoric — is one that mistakenly views static things to moving things through the same lens. Take for example a tree and a table. Both are made of wood (maybe the same type of wood), yet their developmental trajectories are enormously different.

Wood > Tree
Wood > Table

Tables are relatively static. They may get scratched, painted, re-finished, or modified slightly, but their inherent form, structure and content is likely to remain constant over time. The tree is also made of wood, but will grow larger, may lose branches and gain others; it will interact with the environment providing homes for animals, hiding spaces or swings for small children; bear fruit (or pollen); change leaves; grow around things, yet also maintain some structural integrity that would allow a person to come back after 10 years and recognize that the tree looks similar.

It changes and it interacts with its environment. If it is a banyan tree or an oak, this interaction might take place very slowly, however if it is bamboo that same interaction might take place over a shorter time frame.

If you were to take the antique table shown above, take its measurements and record its qualities and  come back 20 years later, you will likely see an object that looks remarkably similar to the one you lefty. The time of initial observation was minimally relevant to the when the second observation was made. The manner by which the table was used will have some effect on these observations, but to a matter of degree the fundamental look and structure is likely to remain consistent.

However, if we were to do the same with the tree, things could look wildly different. If the tree was a sapling, coming back 20 years might find an object that is 2,3,4 times larger in size. If the tree was 120 years old, the differences might be minimal. It’s species, growing conditions and context matters a great deal.

Design for Development / Developmental Design

In social systems and particularly ones operating with great complexity, models of creating programs, policies and products that simply release into the world like a table are becoming anachronistic. Tables work for simple tasks and sometimes complicated ones, but not complex ones (at least, consistently). It is in those areas that we need to consider the tree as a more appropriate model. However, in human systems these “trees” are designed — we create the social world, the policies, the programs and the products, thus design thinking is relevant and appropriate for those seeking to influence our world.

Yet, we need to go even further. Designing tables means creating a product and setting it loose. Designing for trees means constantly adapting and changing along the way. It is what I call developmental design. Tim Brown, the CEO of IDEO and one of the leading proponents of design thinking, has started to consider the role of design and complexity as well. Writing in the current issue of Rotman Magazine, Brown argues that designers should consider adapting their practice towards complexity. He poses six challenges:

  1. We should give up on the idea of designing objects and think instead about designing behaviours;
  2. We need to think more about how information flows;
  3. We must recognize that faster evolution is based on faster iteration;
  4. We must embrace selective emergence;
  5. We need to focus on fitness;
  6. We must accept the fact that design is never done.
That last point is what I argue is the critical feature of developmental design. To draw on another analogy, it is about tending gardens rather than building tables.

Developmental Evaluation

Brown also mentions information flows and emergence. Complex adaptive systems are the way they are because of the diversity and interaction of information. They are dynamic and evolving and thrive on feedback. Feedback can be random or structured and it is the opportunity and challenge of evaluators to provide the means of collecting and organizing this feedback to channel it to support strategic learning about the benefits, challenges, and unexpected consequences of our designs. Developmental evaluation is a method by which we do this.
Developmental evaluators work with their program teams to advise, co-create, and sense-make around the data generated from program activities. Ideally, a developmental evaluator is engaged with program implementation teams throughout the process. This is a different form of evaluation that builds on Michael Quinn Patton’s Utilization Focused-Evaluation (PDF) methods and can incorporate much of the work of action research and participatory evaluation and research models as well depending on the circumstance.

Bringing Design and Evaluation Together

To design developmentally and with complexity in mind, we need feedback systems in place. This is where developmental design and evaluation come together. If you are working in social innovation, your attention to changing conditions, adaptation, building resilience and (most likely) the need to show impact is familiar to you. Developmental design + developmental evaluation, which I argue are two sides of the same coin, are ways to conceive of the creation, implementation, evaluation, adaptation and evolution of initiatives working in complex environments.
This is not without challenge. Designers are not trained much in evaluation. Few evaluators have experience in design. Both areas are familiarizing themselves with complexity, but the level and depth of the knowledge base is still shallow (but growing). Efforts like those put forth by Social Innovation Generation initiative and the Tamarack Institute for Community Engagement in Canada are good examples of places to start. Books like Getting to Maybe,  M.Q. Patton’s Developmental Evaluation, and Tim Brown’s Change by Design are also primers for moving along.
However, these are start points and if we are serious about addressing the social, political, health and environmental challenges posed to us in this age of global complexity we need to launch from these start points into something more sophisticated that brings these areas further together. The cross training of designers and evaluators and innovators of all stripes is a next step. So, too, is building the scholarship and research base for this emergent field of inquiry and practice. Better theories, evidence and examples will make it easier for all of us to lift the many boats needed to traverse these seas.
It is my hope to contribute to some of that further movement and welcome your thoughts on ways to build developmental thinking in social innovation and social and health service work

Image (Header) Growth by Rougeux

Image (Tree) Arbre en fleur by zigazou76

Image (Table) Table à ouvrage art nouveau (Musée des Beaux-Arts de Lyon) by dalbera

All used under licence.

1 thought on “Evaluation and Design For Changing Conditions”

Comments are closed.

Scroll to Top
%d bloggers like this: