Developmental Evaluation is all about using data to support innovation. How does this look in practice (and how’s it done?)
Developmental Evaluation (DE) is all about supporting and developing innovation. It provides some of the raw feedback materials and a process to help innovators make decisions about what they are creating, adapting to changing conditions, and making the most of what they do while operating in complex situations.
DE is most useful for social innovation or innovations aimed at human services and systems — those environments where there are many unknowns, competing forces, and what we call ‘complexity‘ . I’ve described what Developmental Evaluation is elsewhere, including use of the metaphor of a road trip.
But what does doing DE look like?
Let’s take a quick look using the metaphor of sailing to guide us along this short journey of Developmental Evaluation.
Preparedness
“You can’t change the direction of the wind, but you can adjust
Jimmy Dean
your sails to always reach your destination.”
The Center for Evaluation Innovation has a great primer for preparing for DE, emphasizing the need to get out of your own way. Without preparedness and asking the right questions of yourself, you’re unlikely to realize much of the benefits from DE and many of the problems.
A list of statements and situations to explore if DE is really for you can be found here. The risk with DE is that we often think we are more ready to change than we are and confusion interest in the idea of change with the real motivation and systems required to facilitate change.
DE is about preparing to adjust your sails and course to reach your journey. Sailors know that you rarely can get where you need to go in a straight line. The currents, wind, and other barriers will direct the boat in certain ways, and like sailing, DE is about preparing to ‘go with the wind’ while not losing sight of the destination.
Evaluation Design
Developmental Evaluation is not a method, but an approach. The actual design of the evaluation might include many of the methods and tools that professional evaluation is accustomed to. This could include surveys, interviews, or observations as well as review of secondary sources of data.
What makes the design developmental, is the recognition of complexity and the willingness to make adaptations to the methods to suit the demands of the situation. Rigid adherence to a method that doesn’t yield useful data is unlikely to benefit and may indeed harm the evaluation. The design must also include within it some accounting for sensemaking and developmental design.
Sensemaking is a structured process of making sense of what it is that your data is telling you, because it is likely not going to be straightforward due to the complexity of the situation. Developmental design is what we do with the insights that emerge from this process.
Learning
For DE, the key criteria for a ‘good’ evaluation designs is data utility, not some external benchmark for evidence (although the data we collect should always be performed to the highest standard possible for the situation at hand). For innovators, it is about having data you can use to make decisions about what to do next. This might mean staying the course, modifying something, or adding or removing something altogether.
Learning is the only real guaranteed outcome of a DE, however that can only happen when we create systems to support that learning. These systems require we provide ourselves with time, a mindset for discovery, and a willingness to adapt and change based on new information. Learning is easy to speak about, harder to do. And contrary to much of what gets discussed about learning, it isn’t always fun.
(But it also can provide more joy than almost anything else you will do as an innovator).
As a sailor, this is about understanding how small adjustments in your sail or reading of the currents can yield benefits or contribute to problems and using that ongoing, rapid feedback to make course corrections on the fly. Sailing requires small adjustments, regularly and persistently and based on ongoing feedback. That feedback is where learning takes place.
Program (Re) Design
The final part of a DE is also the most neglected and least discussed in the literature: design. Once you have your data, you’ve learned about what is happening, and are ready to take your assessment of the situation and act, we get into design.
Design is a structured, systematic, and creative means of transforming your learning into those adjustments and using them to shape your course. Good design is about creating adjustments that enhance the value of your program to its users. It means applying what we know and translating it into changes to the design of our programs — not just improvements.
This is where your sailor’s compass comes into play, indicating what your true North is and allowing you to calibrate the adjustments to suit your desired destination. Principles (PDF) can serve as this true North in some cases where it is more important to live values and principles than achieve a specific goal.
Readers are encouraged to look at the work on Principles-focused evaluation (PDF) for more on this.
Planning Your Evaluation
Putting this altogether, a DE might look like this:
- Prepare your organization for DE in advance. Ask yourself the key questions and consider the resources requirements.
- Create a design that will maximize the opportunity to learn from what you’re doing. This includes methods that provide a richness of data (including multiple and mixed methods) from a variety of sources.
- Build in organizational supports for learning and design in your organization.
- Prepare your team and partners by fostering an innovator’s mindset for using evaluation data in creating your innovation, shaping it, and making decisions.
- Invest in the design of your program and re-design work. Professional human systems and service designers can help you to assess and enhance the usability, operations, and flow of your program with this new information in a way that doesn’t disrupt what you’re doing (as many of us don’t have the luxury to stop what we’re doing outright).
DE requires much work, but it also generates tremendous rewards and can be the innovator’s secret advantage. Building in structured ways to learn about your innovation while you are developing it will make for a better innovation and help justify the resources used in the process. It also increases your overall capacity to learn, which is perhaps the greatest benefit that a DE approach can bring to your organization.
Just like a sailor plans her journey and prepares for the joys and stresses along the way, so should those of us looking to learn and use DE as a means to support our innovation.
The destination is worth the effort.
Photo by Lance Asper on Unsplash and Noble Mitchell on Unsplash