Welcome to the Censemaking Methods Series dedicated to profiling tools, techniques, methods, and issues tied to strategic design for impact. Today we look at evaluative thinking, the bedrock of understanding our influence on the world.
Evaluative thinking is a disciplined approach to reflective practice, inquiry, and creative design that forms the backbone for any initiative that seeks to create intentional change in the world. Whether it’s products, services, or policies, evaluative thinking is what allows us to ask informed questions about what we’re doing and what influence we’re having through that work.
Evaluative Thinking sits at the heart of evaluation and can be fostered by anyone looking to understand their influence. To help strengthen those evaluative neural pathways, we offer some questions to aid you in developing your evaluative thinking skills.
To begin, let’s first look at what it means to engage in evaluative thinking.
Tom Grayson’s post on the AEA 365 Blog looked at this topic and provided a useful summary of definitions of evaluative thinking. In its simplest term: evaluative thinking is what we do when we think about things from an evaluation perspective, which is to say, a point of view that considers the merit, worth, and significance of something.
While we have many methods and tools that can aid us in the process of doing an evaluation, engaging in the evaluative thinking supporting it is among the most challenging aspects of the practice. But while it can be challenging, it can also be made simple. To help foster evaluative thinking I suggest asking three simple questions: What is going on? What’s new? and What does it mean?
What is going on?
This question is about paying attention and doing so with an understanding of perspective. Asking this question focuses on the many things that might be happening within a situation (program, service, etc..) and the context around it. It gets you to pay attention to the activities, actors, and relationships that exist between them through observation and listening. By asking this question you also can start to empathize with those engaged in the program.
Ask:
What is going on for [ ] person, group, or population?
What is going on in [ ] situation?
What is going on when I step back and look at it collectively as a system?
Inquiring about what is going on enlists one of the evaluator’s most powerful assets: curiosity.
By paying attention and asking questions about the smallest and most mundane activities through to those common threads across a situation, you will start to see things you never noticed before and took for granted. This opens up possibilities to see connections, relationships, and potential opportunities that were previously hidden.
What’s new?
Asking what is new? builds on the answers from the first question. In looking at what is new, we start to see what might be elements of movement and change. It allows us to identify where things are shifting and where the ‘action’ might be when it comes to the product, program or service of interest. Most of what we seek in our programs or new products is change — improvements in something, reductions in something else — and sometimes these aren’t obvious. Sometimes they are so small that we can’t perceive them unless we pause, look, and listen.
There are many evaluation methods that can detect change, however, asking the question about what’s new can help you to direct your evaluation toward methods best suited to capturing this change clearly. Asking this question also amplifies your attentive capacity, which is enormously important for evaluation in detecting large and small changes (because often small changes can have big effects in complex systems like those in human services).
What does it mean?
This last question is about sensemaking. It’s about understanding the bigger significance of something in relation to your enterprise. For example, there might be a lot happening and changing within a program, but it might not mean much to the overall enterprise. Conversely, there can be little to nothing happening, which can be enormously important for an organization by demonstrating poor effects of an intervention or program or, in the case of prevention-based programs, show success.
This question also returns us to empathy and encourages some perspective-taking by getting us to consider what something means for a particular person or audience. A system (like an organization or program) looks different from where you sit in relation to it. Managers will have a different perspective than that of front-line staff, which is different for clients and customers, and different yet from funders or investors. The concept of ‘success’ or ‘failure’ is judged from the perspective of the viewer and a program may be highly successful from one point of view (e.g., easy to administer for a manager) and a failure from another (e.g., relatively low return on a funding investment from a grantor’s perspective).
This question also affords an opportunity to get a little philosophical about the ‘big picture’. It allows program stakeholders to inquire about a program or service’s purpose. Many programs, once useful and effective, can lose their relevance over time due to new entrants to a market or environment, shifting conditions, or changes in the needs of the population served. By not asking this question, there is a risk that a program won’t realize it needs to adapt until it is too late.
By asking these three simple questions you can kick-start your evaluation and innovation work and better strengthen your capacity to think evaluatively.
References:
Buckley, J., Archibald, T., Hargraves, M., & Trochim, W. M. (2015). Defining and Teaching Evaluative Thinking: Insights From Research on Critical Thinking. American Journal of Evaluation, 36(3), 375–388. https://doi.org/10.1177/1098214015581706
Cole, M. J. (2023). Evaluative thinking. Evaluation Journal of Australasia, 23(2), 70–90. https://doi.org/10.1177/1035719X231163932
Paproth, H., Clinton, J.M., Aston, R. (2023). The Role of Evaluative Thinking in the Success of Schools as Community Hubs. In: Cleveland, B., Backhouse, S., Chandler, P., McShane, I., Clinton, J.M., Newton, C. (eds) Schools as Community Hubs. Springer, Singapore. https://doi.org/10.1007/978-981-19-9972-7_21
Image Credits: Jon Tyson on Unsplash Gary Butterfield on Unsplash, Tim Mossholder on Unsplash
If you’re interested in learning more about design and evaluation to elevate your impact, subscribe to the Design Loft – my innovative learning newsletter and course designed for change-makers.