What is quality when we speak of learning? In this third post in series on education and evaluation metrics the issue of quality is within graduate and professional education is explored with more questions than answers about the very nature of learning itself.
But what does learning really mean and do we set the system up to adequately assess whether people do it or not and whether that has any positive impact on what they do in their practice.
What do you mean when you say learning?
The late psychologist Seymour Sarason asked the above question with the aim of provoking discussion and reflection on the nature and possible outcomes of educational reform. Far from being glib, Sarason felt this question exposed the slippery nature of the concept of learning as used in the context of educational programming and policy. It’s a worthwhile question when considering the value of university and professional education programming. What do we mean when we say learners are learning?
The answer to this question exposes the assumptions behind the efforts to provide quality educational experiences to those we call learners. To be a learner one must learn…something.
The Oxford English Dictionary defines learning this way:
the acquisition of knowledge or skills through experience, practice, or study, or by being taught: these children experienced difficulties in learning | [ as modifier ] : an important learning process.
• knowledge acquired in this way: I liked to parade my learning in front of my sisters.
This might sufficiently answer Dr Sarason except there is no sense of what the content is or whether that content is appropriate, sufficient, timely or well-supported with evidence (research or practice-based); the quality of learning.
Knowledge translation professionals know that learning through evidence is not achieved through a one-size-fits-all approach and that the match between what professionals need and what is available is rarely clean and simple (if it was, there would be little need for KT). The very premise of knowledge translation is that content itself is not enough and that sometimes it requires another process to help people learn from it. This content is also about what Larry Green argues: practice-based evidence is needed to get better evidence-based practice.
How do we know when learning is the answer (and what are the questions)?
If our metric of success in education is that those who engage in educational programming learn, how do we know whether what they have learned is of good quality? How do we know what is learned is sufficient or appropriately timed? Who determines what is appropriate and how is that tested? These are all questions pertaining to learning and the answers to them depend greatly on context. Yet, if context matters then the next question might be: what is the scope of this context and how are its parameters set?
Some might choose academic discipline as the boundary condition. To take learning itself as an example, how might we know if learning is a psychology problem or a sociology problem (or something else)? If it is a problem for the field of psychology, when does it become educational psychology, cognitive psychology, community psychology or one of the other subdisciplines looking at the brain, behaviour, or social organization? Successful learning through all of these lenses means something very different across conditions.
Yet, consider the last time you completed some form of assessment on your learning. Did you get asked about the context in which that learning took place? When you were asked questions about what you learned on your post-learning assessment:
- Did it take into account the learning context of delivery, reception, use, and possible ways to scaffold knowledge to other things?
- Did your learner evaluation form ask how you intended to use the material taught? Did you have an answer for that and might that answer change over time?
- Did it ask if your experience of the learning event matched what the teachers and organized expected you to gain and did you know what that really was?
- Did you know at the time of completing the evaluation whether what you were exposed to was relevant to the problems you needed to solve or would need to solve in the future?
- Did you get asked if you were interested in the material presented and did that even matter?
- Was there an assumption that the material you were exposed to could only be thought of in one way and did you know what that way was prior to the experience? If you didn’t think of the material in the way that the instructors intended did you just prove that the first of these two questions is problematic?
Years of work in post-secondary teaching and continuing professional education suggests to me that your answer to these questions was most likely “no”, except the very last one.
These many questions are not posed to antagonize educators (or “learners”, too) for there are no single or right answers to any of them. Rather, these are intended to extend Seymour Sarason’s question to the present day and put in the context of graduate and professional education at a time when both areas are being rethought and rationalized.
Learning to innovate (and being wrong)
A problem with the way much of our graduate and professional education is set up is that it presumes to have the answers to what learning is and seeks to deliver the content that fills a gap in knowledge within a very narrow interpretation. This is based on an assumption that what was relevant in the past is both still appropriate now and will be in the future unless we are speaking of a history lesson. However, innovation and discovery — and indeed learning itself — is based on failure, discomfort and not knowing the answers as much as building on what has come before us. There is no doubt that a certain base level of knowledge is required to do most professional and scientific work and that building a core is important, but it is far from sufficient.
The learning systems we’ve created for ourselves are based on a factory model of education, not for addressing complexity or dynamic systems like we find in most social worlds. We do not have a complex adaptive learning system in place, one that supports innovation (and the failures that produce new learning) because:
If you’re not prepared to be wrong, you’ll never come up with anything original. – Sir Ken Robinson, TED Talk 2006
The above quote comes from education advocate Sir Ken Robinson in a humorous and poignant TED talk delivered in 2006 and then built on further in a second talk in 2010. Robinson lays bare the assumptions behind much of our educational system and how it is structured. He also exposes the problem we face in advancing innovation (my choice of term) because we have designed a system that actively seeks to discourage wide swaths of learning that could support it, particularly with the arts.
Robinson points to the conditions of interdisciplinary learning and creativity that emerge when we free ourselves of the factory model of learning that much of our education is set up, “producing” educated people. If we are assessing learning and we go outside of our traditional disciplines how can we assess whether what we teach is “learned” if we have no standard to compare it to? Therein lies the rub with the current models and metrics.
If we are to innovate and create the evidence to support it we need to be wrong. That means creating educational experiences that allow students to be wrong and have that be right. If that is the case, then it means building an education system that draws on the past, but also creates possibilities for new knowledge and learning anchored in experimentation and transcends disciplines when necessary. It also means asking questions about what it means to learn and what quality means in the context of this experimental learning process.
If education is to transform itself and base that transformation on any form of evidence then getting the right metrics to evaluate these changes is imperative and quality of education might just need to be one of them.
There is much discussion about scaling social innovation – bringing small successes to a larger theatre — yet little is known about the properties that make something work at one level successful at another. When the “thing” to scale is relationships, such as the case with knowledge translation and design, is bigger better or even possible?
Last week the Design Management Institute held its annual North American conference themed: Design at Scale. The conference featured many prominent names in branding, market development, graphic design, and design management together to discuss the ways in which the creative process used in design can be leveraged from one level to another.
One of the best technical examples came from David Butler and Gerardo Garcia from Coca Cola who showed their modular design system being used to transform the way small local retailers in South America can create large or small displays with products that are regionally appropriate with ease. While it was interesting to see how one could create retail displays that could easily adapt and scale, I was left wondering whether the same system would permit the social variables associated with each of these 1 million vendors to do the same thing. Are these vendors likely to view the modular system in the same way that Coke does? Does it even make sense to them? Surely for some that will be a “yes”, but will it be as many as Coke thinks and does this system solve a problem that the retailers have as much as it aims to satisfy Coke’s goal of doubling its revenue in the next decade?
While the physical product generated from this system might scale, the relationships that surround its implementation might not.
Which got me to thinking about the other lessons that came from the conference. Perhaps the most intriguing ones were those presented by Jamer Hunt, the Director for Transdisciplinary Design at Parsons The New School for Design. Hunt drew on the work of design legends Charles and Ray Eames and their film the Powers of Ten as a means of illustrating scale and what it really means (which he wrote about in Fast Company article last year).
An aside: The Powers of Ten was shown on the first day of my first class in psychology when I was an undergraduate at the University of Regina and was used by my professor, the truly remarkable Paul Antrobus (PDF), to illustrate the realm that psychology could play in the universe. “This is the realm of psychology” he declared. It is something that has probably never been uttered in another class in psychology anywhere and probably should be everywhere. It changed the perspective I brought to my work and has changed my life in ways I can’t fully comprehend.
What The Powers of Ten does is illustrate scale at the macro and micro level by showing how great, yet relatively consistent, the differences are between different scales. Scalar changes happen at an order of magnitude that becomes difficult to grasp as one shifts up or down due to the massive, exponential change that, at small scales seem palpable, but at large scales seem incomprehensible. Jamer Hunt made this all the more concrete when he used the example of an ant taking a shower. No matter how intentional an ant might be about wanting a shower, the water molecules from a shower are too big and would crush him (or her). The water doesn’t scale.
Social innovation, social design and communications (particularly knowledge exchange and translation) is largely about relationships. Developing intimacy, expressing empathy, creating trust, and having authentic and meaningful conversations are the hallmark of healthy and strong human relationships. They also tend to cluster with good, effective practice in the above-mentioned areas. There are good reasons why (contrary to what Paris Hilton might suggest) we don’t have a lot of BFF‘s in our lives: we can’t maintain that level of closeness with a lot of people. It is precisely because we create a sense of intimacy with a few, that the relationship with the many is able to be maintained as it is. Relationships change, evolve, grow and whither, but the absolute number of close, personal relationships for people tend to remain relatively constant, even if that number differs between people.
The work by evolutionary psychologist and anthropologist Robin Dunbar has looked at these relationships and found that, by and large, we are not able to maintain meaningful relations (nevermind close relations) with more than about 150 (with a large standard deviation). While the variance in this number is large, the implications for scaling might be larger. Some of Dunbar’s original research with primates suggests that our brains are simply not evolved enough to handle the complexity of too many more relations.
It might also simply be less enjoyable. Meaning is something that requires attention to create and use and the more variables competing for attention in your life, the less meaningful things might be. If this is the case, can we design programs and initiatives that scale up from small to big? Or do we need to reframe the way we see scaling to something akin to a network, whereby there are a lot of small nodes connected together? Networking nodes seems to be a way to go big and go small.
If so, what does this mean for designing systems that scale? It might also mean that for those of us working to develop solutions that scale that we need to pay attention to the social and mathematical issues that come with scaling something. It means paying attention to psychosocial physics and dynamics and using research more intently to inform our designs and social innovations lest we scale in ways that create metaphorical water-droplets that are so big crush those we seek to shower.
Designers seek to put their best forward in their creations, but sometimes it is the dark rather than the light that provides an impetus for good design. Carl Jung’s look into our darker nature might provide a means of understanding the lighter side of what we produce.
My work and related inquiry into design has led me to Carl Jung’s doorstep on many occasions, and this week his concept of shadow was brought into focus through a series of conversations and reflections. In interviewing designers the past few weeks for the Design Thinking Foundations project an initial point of interest that has emerged from the data is the importance of the designer’s connection to the designed product, something I wrote about earlier.
If one is to consider design, the act of making something with intent, as something of an act of personal expression it follows that it be subjected to the same moral and ethical scrutiny that other such acts are put to. This becomes particularly important when one considers the potential impact that our designed creations can have on the world around us. The manner by which we, as designers, shape this artificial environment of human-made objects has profound implications and thus, the factors that shape the designers are important.
Jung’s shadow of the psyche mirrors the qualities of the darkness created by objects standing in the light. It is that part of a person that is often unrecognized, unspoken, or unconscious that reflects aspects of a person that may be perceived to be less desirable to others or to the person themselves.
As Jung states:
There can be no doubt that man (sic) is, on the whole, less good than he imagines himself or wants to be. Everyone carries a shadow, and the less it is embodied in the individual’s conscious life, the blacker and denser it is. If an inferiority is conscious, one always has a chance to correct it. Furthermore, it is constantly in contact with other interests, so that it is continually subjected to modifications. But if it is repressed and isolated from consciousness, it never gets corrected. Jung, C.G. (1938). Psychology and Religion”. In Psychology and Religion: West and East. P.131
Jung believed that shadow requires confrontation, or at the least acknowledgement. For the designer, this means being self-aware to ensure that their motives are clear when they approach a project and the people connected to it.
This is not just an issue for personal development or the protection of others, it is about doing the best work. For design, this often means doing work with others. For the designer, it means doing the work on themselves, which includes an obligation to learn, develop and grow.
There is a deep gulf between what a man is and what he represents, between what he is as an individual and what he is as a collective being. His function is developed at the expense of the individuality. Should he excel, he is merely identical with his collective function; but should he not, then, though he may be highly esteemed as a function in society, his individuality is wholly on the level of his inferior, undeveloped functions, and he is simply a barbarian, while in the former case he has happily deceived himself as to his actual barbarism. Jung, C.G. (1921) Psychological Types, P.III
Psychologists and Social Workers might be used to integrating deep self-work into their professional roles, but designers typically are not. Creation on its own is a scary subject that terrifies artists and designers alike. It takes courage to put one’s work “out there” for people to see, critique and explore. It exposes our potential weaknesses, our vulnerabilities and our aspirations in ways that few encounters can. When design is done with passion and integrity, not just intent, it means putting a piece of ourselves into the product.
What we might not be aware of is that the self that is reflected in our work might include both the light and the shadow. As a designer, the fear of being under the gaze of others is amplified by the fear that such inquiry will reveal parts of our shadow. To reveal one’s shadow, is to expose one’s truest self in its entirety, not just part of it.
To confront a person with his shadow is to show him his own light. Once one has experienced a few times what it is like to stand judgingly between the opposites, one begins to understand what is meant by the self. Anyone who perceives his shadow and his light simultaneously sees himself from two sides and thus gets in the middle. Jung, C.G. (1959) Good and Evil in Analytical Psychology”In CW Civilization in Transition. P.872
The shadow introduces what Jung calls “a moral problem” to the enterprise of design. The products of design are intimately tied to the designer. It is perhaps for this reason that the field is often known for having practitioners with large egos and star-like status. But if one is to consider the manner in which the shadow is — or can be — expressed through design, the possibility for a design process that is overwhelmingly ego-driven is lowered.
It also presents the opportunity for a more authentic, if risky, form of design.
The shadow is a moral problem that challenges the whole ego-personality, for no one can become conscious of the shadow without considerable moral effort. To become conscious of it involves recognizing the dark aspects of the personality as present and real. This act is the essential condition for any kind of self-knowledge. Jung, C.G. Aion (1951). CW 9, Part II: P.14
And isn’t design about taking risks? Perhaps to create the very best work in the light, we need to embrace the shadow’s that it helps create.