This second in a series on Principles-Focused Evaluation (PFE) draws our attention to the concepts of systems and complexity and how can PFE help us navigate both.
Systems (thinking), complexity, and principles all go together. Principles-focused Evaluation (PFE) brings them together in meaningful ways, but to understand that, we first need to understand what we mean by systems and complexity.
For starters, PFE combines strategic and evaluative thinking, borrowing language from these areas of study and practice. Terms like adaptive strategy, developmental or utilization-focused evaluation and complexity are commonly brought to bear when speaking/writing on PFE. If you’re familiar with these terms, PFE language will not be problematic.
The Setting: Complexity
Before we do anything, let’s talk about the situations and settings where PFE is most helpful. To do that, we need to talk about systems and complexity.
Complexity presents itself when there are many overlapping interconnections between things (or “parts” of a system) — these can be people, actions, structures, or anything that contributes in some way to a purpose for that system — that occur at different times and scales.
In simple, straightforward systems, the relationships between the parts are stable, consistent, and both reliable and predictable. Think of a bicycle, your kitchen table, or a basketball and hoop.
In complex systems, the interconnections within them decouple or obscure the links between causes and consequences. Think about riding a bicycle through city streets, conversations over meals at a kitchen table, or a pick-up game of basketball.
This means we can’t reliably connect activities to outcomes, which is central to most forms of strategy and evaluation. This is where principles can help — which we’ll get to in a future post.
Complexity (the term) refers to a constellation of activities associated with certain systemic conditions. These conditions include:
- Dynamism (parts are in flux and subject to movement, combination, and transformation)
- Interconnection (mutually influencing relationships among parts)
- Temporal Flexibility (time and space affect parts and their relations differently)
- Flexible, Negotiated Boundaries (actions happen within a system, yet the boundaries of this system are malleable, will modify over time, and are affected by those who perceive the system)
The Setting: Systems
When I say systemic, I refer to a set of conditions, interactions, and situations that are bounded in some way. Systems Thinking is a term for a broad-based field of theories, models, methods, and approaches to understanding and working with systems. You might have heard of it.
When we speak of systems, what we are talking about are contexts for relationships. For example, a student might have a relationship with a school (both the building and the institution), a teacher, peers, a grade level, the subject matter, books and resources, and so on. Outside of these relationships, the term student is meaningless. We can’t learn about what it means to be a student if we don’t have these relationships.
This brings us to the importance of perspectives. It is a perspective that informs what we perceive as a system. To build on the example above, a learning system might be what the student perceives, a classroom environment might be a system a teacher perceives, an education system might be what society perceives, a school system might be what parents or administrators might perceive, and so on. These perspectives might include the same parts, but their relationships are framed by the perspectives we take on and how they relate to one another.
It’s why we see such different strategies in place when we refer to things like learner-centred approaches to teaching or outcomes-based education. The perspectives, purposes, and relationships between the parts and the whole system may vary widely.
Implications for Evaluation
What does this mean for learning, strategy and assessing outcomes and impact?
It means we cannot confidently predict or control what happens in complex systems. Most human systems are in some way complex.
It also means that evidence must be treated differently. We might have little control over what happens, but we can still learn. We can’t predict the future, but we can help shape it. For evaluation to be practical in these conditions, we need to shift our mindset, expand our toolset, and sharpen our skillset.
This means that terms like “best practice” are not helpful. In complex systems, we often deal with emergent or situational practice. This means learning from the situation and allowing an approach to emerge from the signals we sense within the situation. Sensemaking is worth adding to our evaluation approach. Principles-Focused Evaluation, much like Developmental Evaluation, can aid in this.
In future posts in this series, we’ll go further into these relationships.
Implications for Strategy
Evaluation data is a means of supporting learning and adaptive strategy for working in complex situations. Data is the raw material for making decisions and crafting useable strategies. With good data, we can anticipate possible outcomes and develop strategies considering a range of actions, outputs, and events.
The role of evidence, in this case, is to provide valuable suggestions for possible implications and choices. The implications of these choices can then be assessed in real-time after they are made and adjusted as needed. This means choosing the appropriate methods and tools (which we’ll look at in a future post).
But what if the evidence is thin and the data is inconclusive?
This is where principles come in. Principles-focused evaluation (PFE) provides guidance and accountability to programs and projects operating in complex conditions. A PFE offers ways of working with data and evidence and an additional layer of sensemaking to support adaptive strategy.
That’s systems and complexity for evaluation. In our next post, we will look at principles and how they work within systems and complexity to provide the guidance we’ve just discussed.
Thanks for reading. If this is the context you work in and need help in building an evaluation and strategy to meet it, let’s talk.
Image credit: All images from refargotohp on Unsplash
Comments are closed.