A brilliant and comprehensive new book has been launched that brings together the best scholars working in the area of systems thinking and complexity and applying it to health.
The book description can be found here along with a link to the abstract for a chapter I co-authored with Andrea Yip looking at the overlap between design thinking and systems science and complexity. This chapter takes a design lens on previous work developing the CoNEKTR model for engagement in complexity and health.
It’s a big book, but well worth a look if you’re wrestling with complexity and systems thinking in health and social innovation.
What is quality when we speak of learning? In this third post in series on education and evaluation metrics the issue of quality is within graduate and professional education is explored with more questions than answers about the very nature of learning itself.
But what does learning really mean and do we set the system up to adequately assess whether people do it or not and whether that has any positive impact on what they do in their practice.
What do you mean when you say learning?
The late psychologist Seymour Sarason asked the above question with the aim of provoking discussion and reflection on the nature and possible outcomes of educational reform. Far from being glib, Sarason felt this question exposed the slippery nature of the concept of learning as used in the context of educational programming and policy. It’s a worthwhile question when considering the value of university and professional education programming. What do we mean when we say learners are learning?
The answer to this question exposes the assumptions behind the efforts to provide quality educational experiences to those we call learners. To be a learner one must learn…something.
The Oxford English Dictionary defines learning this way:
the acquisition of knowledge or skills through experience, practice, or study, or by being taught: these children experienced difficulties in learning | [ as modifier ] : an important learning process.
• knowledge acquired in this way: I liked to parade my learning in front of my sisters.
This might sufficiently answer Dr Sarason except there is no sense of what the content is or whether that content is appropriate, sufficient, timely or well-supported with evidence (research or practice-based); the quality of learning.
Knowledge translation professionals know that learning through evidence is not achieved through a one-size-fits-all approach and that the match between what professionals need and what is available is rarely clean and simple (if it was, there would be little need for KT). The very premise of knowledge translation is that content itself is not enough and that sometimes it requires another process to help people learn from it. This content is also about what Larry Green argues: practice-based evidence is needed to get better evidence-based practice.
How do we know when learning is the answer (and what are the questions)?
If our metric of success in education is that those who engage in educational programming learn, how do we know whether what they have learned is of good quality? How do we know what is learned is sufficient or appropriately timed? Who determines what is appropriate and how is that tested? These are all questions pertaining to learning and the answers to them depend greatly on context. Yet, if context matters then the next question might be: what is the scope of this context and how are its parameters set?
Some might choose academic discipline as the boundary condition. To take learning itself as an example, how might we know if learning is a psychology problem or a sociology problem (or something else)? If it is a problem for the field of psychology, when does it become educational psychology, cognitive psychology, community psychology or one of the other subdisciplines looking at the brain, behaviour, or social organization? Successful learning through all of these lenses means something very different across conditions.
Yet, consider the last time you completed some form of assessment on your learning. Did you get asked about the context in which that learning took place? When you were asked questions about what you learned on your post-learning assessment:
- Did it take into account the learning context of delivery, reception, use, and possible ways to scaffold knowledge to other things?
- Did your learner evaluation form ask how you intended to use the material taught? Did you have an answer for that and might that answer change over time?
- Did it ask if your experience of the learning event matched what the teachers and organized expected you to gain and did you know what that really was?
- Did you know at the time of completing the evaluation whether what you were exposed to was relevant to the problems you needed to solve or would need to solve in the future?
- Did you get asked if you were interested in the material presented and did that even matter?
- Was there an assumption that the material you were exposed to could only be thought of in one way and did you know what that way was prior to the experience? If you didn’t think of the material in the way that the instructors intended did you just prove that the first of these two questions is problematic?
Years of work in post-secondary teaching and continuing professional education suggests to me that your answer to these questions was most likely “no”, except the very last one.
These many questions are not posed to antagonize educators (or “learners”, too) for there are no single or right answers to any of them. Rather, these are intended to extend Seymour Sarason’s question to the present day and put in the context of graduate and professional education at a time when both areas are being rethought and rationalized.
Learning to innovate (and being wrong)
A problem with the way much of our graduate and professional education is set up is that it presumes to have the answers to what learning is and seeks to deliver the content that fills a gap in knowledge within a very narrow interpretation. This is based on an assumption that what was relevant in the past is both still appropriate now and will be in the future unless we are speaking of a history lesson. However, innovation and discovery — and indeed learning itself — is based on failure, discomfort and not knowing the answers as much as building on what has come before us. There is no doubt that a certain base level of knowledge is required to do most professional and scientific work and that building a core is important, but it is far from sufficient.
The learning systems we’ve created for ourselves are based on a factory model of education, not for addressing complexity or dynamic systems like we find in most social worlds. We do not have a complex adaptive learning system in place, one that supports innovation (and the failures that produce new learning) because:
If you’re not prepared to be wrong, you’ll never come up with anything original. – Sir Ken Robinson, TED Talk 2006
The above quote comes from education advocate Sir Ken Robinson in a humorous and poignant TED talk delivered in 2006 and then built on further in a second talk in 2010. Robinson lays bare the assumptions behind much of our educational system and how it is structured. He also exposes the problem we face in advancing innovation (my choice of term) because we have designed a system that actively seeks to discourage wide swaths of learning that could support it, particularly with the arts.
Robinson points to the conditions of interdisciplinary learning and creativity that emerge when we free ourselves of the factory model of learning that much of our education is set up, “producing” educated people. If we are assessing learning and we go outside of our traditional disciplines how can we assess whether what we teach is “learned” if we have no standard to compare it to? Therein lies the rub with the current models and metrics.
If we are to innovate and create the evidence to support it we need to be wrong. That means creating educational experiences that allow students to be wrong and have that be right. If that is the case, then it means building an education system that draws on the past, but also creates possibilities for new knowledge and learning anchored in experimentation and transcends disciplines when necessary. It also means asking questions about what it means to learn and what quality means in the context of this experimental learning process.
If education is to transform itself and base that transformation on any form of evidence then getting the right metrics to evaluate these changes is imperative and quality of education might just need to be one of them.
Post-secondary and continuing education is continuing to be rationalized in ways that are transforming the very foundation of the enterprise. Funding is a major driver of change in this field: how much is available, when it flows, where it comes from, what is funded, and who gets the funding are questions on the minds of those running the academy.
At the centre of the focus of this funding issue is the job market. Training qualified professionals for the job market in various forms has been one of the roles a university has played for more than a century. Now that role has become central.
Let’s consider what that means and what it could do in shaping the various possible futures of the university. This second in a series looking at the post-secondary and continuing education focuses on the metrics of jobs.
“What are all these people going do?”
The employability of graduates is now the holy grail of education industry statistics. Earlier this year I was sitting on the stage at an academic convocation with a senior colleague staring out at a sea of soon-to-be-graduates when he leaned over and asked the question quoted above. Staring at a sea of masters and doctoral graduates numbered in the hundreds and knowing that this ceremony was held twice per year, the question stuck and remains without an answer.
Maybe there were enough jobs for that cohort, but this process gets repeated twice each year at universities around the world and each year that I’ve been a professor those numbers (of graduates) seem to go up. Some of our programs in the health sciences are admitting three times the number of students than they were just ten years ago. There is much demand for education (as judged by departmental applications), but are there jobs demanding this kind of education in its current form?
Yes, the Baby Boom is moving into an age of retirement and increasing needs for health services, but do we need to graduate 80+ Physical or Occupational Therapists to meet this need this year? Do we need a few dozen more epidemiologists or health promotion specialists to add to the pool? How about psychologists or social workers: how many of those do we need? The answer from my colleagues in these fields is: We don’t know.
Chasing the Wind
Jobs are a red herring. It’s one thing to have a job, but is it the job that you trained for? (And is having that job even a reasonable goal?) Being employed is not the same as building a career. What if you were trained perfectly for a job that no longer existed? Imagine a Blacksmith in the 20th century or a Bloodletter. These questions are not asked, nor is much asked about quality of education relative to the pressures of recruitment, cost-cutting and educational rationalization. Most of us don’t know what quality education is in real terms because we are measuring it (if we are measuring anything at all besides jobs) by standards set for the jobs of the past, not the future (or even the present?).
“Skate where the puck is going, not where it’s been.” – Wayne Gretzky
Jobs are living things and very few in 2013 will resemble what they did even 10 years ago. The citizens of the developing world are entering this rapidly changing job market ready for change (See also McKinsey Global Institute report on future of work in advanced economies) because they don’t have the old ways to rely on. They are primed for change and if professional education is to meet the needs of a changing world, it needs to change too. It means getting serious about learning.
If education is rationalizing itself to focus more on jobs, then it also needs to get serious about clarifying what jobs mean, defining what ‘success’ looks like for a graduate, and whether those jobs are designed for where the proverbial puck is now or for where it is going.
Disruptive Learning / Disturbed Education
“The Only Thing That Is Constant Is Change -” ― Heraclitus
I’ve pointed out that learners have an uneasy relationship with learning principally because it means disrupting things. This is a topic I’ll be covering in greater depth in a future post, but if one considers how our social, economic, and environmental systems are changing it is not unreasonable to call this the age of disruption .
Change in complex systems is often logarithmic, not linear. It may be massively punctuated like a Lévy Flight or it could be closer to a random walk. In environments with a change coefficient that is large the level of attention must be more fine-grained than 5-year reviews. It requires developmental evaluation methods and learning organizations, not just conventional approaches to generating and assessing feedback. It requires mindful attention and contemplative inquiry to guide a regular reflective practice if one is to pay attention to the subtleties in change that could have enormous impact.
For example, if journalists and news media waited every five years to assess the state of their profession, they would have missed out on Twitter and come late to blogging, two of their (now) powerful sources of competition and tools of the trade. Some have waited, which is why they are no longer around. Metrics for journalism education today might consider the amount of exposure and proficiency in social media use, digital photography, use of handheld tools for communication, and real-time reporting skills. Metrics of the past might focus on newspapers and radio broadcasting. Which mindset, skillset and toolset would you rather be trained in today?
Questions for educators, learners (and evaluators):
Whether health sciences, journalism, human services or any field, what might some questions be that can help determine the role of job training in professional education? Here are five starters:
1. What is the state of your profession right now and are you training people for existing in this state? Are you preparing people for the next evolution?
2. Where is your field of practice going? What are the possible futures for your profession in the next 5, 10, and 20 years? Will it still exist? Are you a blacksmith looking for more horses in the automobile age or Steve Jobs waiting to attract people to a new graphical user interface?
3. Is your mindset, skillset or toolset in need of re-consideration? Does it still do the job you’ve hired it to do?
4. What do people need that your skills can help with? What unfilled needs and expectations are there in the world that your mindset, skillset and toolset could solve?
5. What would happen if your field of practice disappeared? How else could you apply what you know to making the contribution you wish to make and earn a living? What other skills, tools and ways of thinking would you need to adapt?
Design thinking can greatly help shape the way that one conceives of a problem, works through possible options, and develops prototypes to address the needs of the present and the future. Foresight methods help lay additional context for design and systems thinking by providing ways to anticipate possible futures for any given field. Lastly, knowing what the state of things are now and how they got to where they are now can help determine the path dependencies that education may have fallen into.
We can’t change what we don’t see and better foresight, hindsight and present sight is critical to better ensuring that education outcomes are not imagined, but based on something that can actually improve learning.
Since R. Martin and others hijacked the term 'designthinking', there is an ongoing dispute. Two thought worlds exist and possibly these can be united by laying bare the essential characteristics of a 'design thinker'.
Design thinking frames the verb 'design' as a specific cognitive activity in order to solve problems and is discerned from other ways of thinking such as decision making.
Jonah Lehrer is/was as big as it gets in science writing and two weeks ago proved the adage that the higher one climbs the farther the fall after admitting to some false content in his stories. This is bad news for him, but may be much worse for all of us interested in making science and innovation knowledge accessible for reasons that have as much to do with the audience as it does the message and messenger.
Jonah Lehrer was one of our most prolific and widely read science writers until he admitted fudging some quotes about Bob Dylan in his new book, Imagine, which looks at the process of discovery, creativity and innovation. The discovery by fellow journalist (and fervent Bob Dylan fan) Michael Moynihan set off a wave of reflections and investigations of Lehrer’s work revealing passages in the book (and other pieces) that had been reused from his other writings without proper self-attribution and sparking questions about the integrity of the author’s entire body of work. The “fall of Jonah Lehrer” was big news at a time when the London Olympics were dominating most of the media’s attention.
This case is a testament to the wide appeal that Lehrer’s work had beyond the usual ‘science geeks’ while illustrating the power of the internet to enable the kind of curation and investigation to support on and offline fact checking. But what it spoke to most for me is the role
The Writer and his Craft
Much digital type has been spent on the Lehrer incident. Search Google and you’ll find dozens of commentaries looking at how things transpired and how Lehrer ironically succumbed to the cognitive biases he wrote about.
Roxane Gay, writing in Salon, took a gendered approach to the issue and questioned whether our fascination is less with the science and more about the ‘young male genius’. Lehrer’s youth was something she saw as critical to amplifying the fascination with his work. She writes:
When young people display remarkable intelligence or creativity, we are instantly enamored. We want or need geniuses to show us the power and potential of the human mind and we’re so eager to find new people to bestow this title upon that the term and the concept have become quite diluted.
I agree with her on the point about our desire to over-inflate the accomplishments of youth (as if we are *amazed* that any of them could possibly do anything brilliant, which is as offensive to them and it is to older people), although a careful look at Lehrer’s articles and much of the press around his work suggests that he was much less a focus of the attention than his ideas.
Call it “Gladwellization.” It’s not just lucrative, but powerful: your ideas (or rather, the ideas you’ve turned into compelling anecdotes for a popular audience) can influence everything from editorial choices across the publishing world to corporate management and branding strategies.
But with this comes mounting demands to produce, and to recycle. You have to be prolific, churning out longer pieces that give your insights some ballast, and brilliant, bite-sized items. And yet you can’t be too new either: people want to hear what you’re already famous for. In this cauldron of congratulation and pressure for more and more, it’s not hard to see how standards might erode, how the “ideas” might become more important than doing the necessary due diligence to make sure they sync with reality.
‘Snappy Science’ and Synthesis
Innovation is about ‘new’ and there are good reasons why its a challenge to get the message out that this ‘new’ can be adapted, small, and unsexy and still make a large difference in the long run instead of big, bold and transformative right away. We are in an age of selling “snappy science” and it says more about the media and audiences than the authors and scientists producing the original work.
This snappy, bite-sized science might sell books and make for great TED talks, but it is a misrepresentation of what we actually know and do as scientists. Rarely does a single finding lead to a solution, rather it is an amalgam of discoveries small and large brought together that gets us to closer to answers. Synthesis is the driver of change and synthesis is what journalists do particularly well. Malcolm Gladwell, Steven Johnson and Jonah Lehrer are among the best synthesizers out there and I would imagine (no pun intended) that they contribute to more to public and professional understanding of social innovation than all of the original-sourced scientific knowledge on the subject combined.
When I hear Malcolm Gladwell cited as an original source in serious discussions with colleagues on scientific matters, I realize we have a problem…and an opportunity. Gladwell’s writings popularized the concept of tipping points, but his work is based on a wealth of scientific data on complex systems. They are not his original ideas, but they are his syntheses and (sometimes) his interpretations. This is important work and I am not taking anything from anyone who makes science data digestible and accessible, but it is not the original science.
That Jonah Lehrer is as well known as he is tells me that there is an appetite for science and I’ll freely admit to using his work (and that of the other authors I’ve mentioned) to inform what I do in a general sense. It is good work, however I also acknowledge that I have the scientific training to know how to go beyond the initial articles to critically appraise the information, place it in context, and I have the resources to go to the original sources in academic journals. Most people (professionals and lay people) do not. This access is going to decrease as resources shrink.
It is for this reason that synthetic work is so important. My Twitter feed often is filled with references to such synthetic work, rather than original works of research because I aim to fill role that is somewhere between journalism and the science of design, systems and psychology. I am not a pure science blogger, nor am I speaking to the lay public, but rather other professionals seeking to enrich their knowledge base. That is a role I’ve created for myself, largely because there is a high demand and low supply.
We have a need for synthesis and a demand for it, but little acknowledgement of the value of this role in professional scientific circles. Yet, when we leave journalists to do the work for us, we allow a different system to take charge. John McQuaid ended his article with this caution:
Book publishers don’t do fact-checks, so there’s no fail-safe, just the conscience of the writer. Reach that point, and all is lost.
Filling the gap, meeting a need and shooting the messenger
Journalists like Johnson, Gladwell and Lehrer fill a gap, which is why I am saddened by the loss of one of them and angry at what has transpired. While there is no doubt that Lehrer made mistakes, they were of a rather minor nature in the grand scheme of things. Synthetic work is designed to provide a big picture overview, not guide microscopic decisions. I would like people to read Lehrer and learn about the creative process and the role of neuroscience in making our lives better, to appreciate systems thinking and decision making because of Malcolm Gladwell, and see innovation, emergence and discovery in new ways because of writers like Steven Johnson.
Yet, when we seek more and more from these authors, we might get less and less. This is what happened to Jonah Lehrer. As more people found themselves drawn to his work, the pressure grew for doing more, faster and getting that ‘snappy science’ out the door. GOOD magazine in the ‘tyranny of the big idea‘ goes further:
The problem is that it’s unreasonable to expect that every new piece of media should upend conventional wisdom or deliver a profound new insight. To think that Jonah Lehrer could expose an amazing new facet of human psychology every week, in 1,000-odd words no less, is ludicrous. There are only so many compelling, counterintuitive, true ideas out there.
But the demand for them doesn’t abate. That’s why you see so many science writers talking about the same handful of studies (the Stanford prison experiment, the rubber hand illusion, Dunbar’s number, the marshmallow test) over and over. That’s why you see pop economists who should know better creating flimsy and irresponsible contrarian arguments about climate change for shock value. That’s why you get influential bloggers confessing they’re only 30 percent convinced of their own arguments but “you gotta write something.” That’s why the#slatepitches meme hits home.
Search Censemaking and you’ll find many of these topics not just because they are punchy, but because they are useful.
I hope we haven’t lost Jonah Lehrer as a voice just as I hope more people stop putting writers like him on a pedestal, where they don’t belong (nor do the scientists who produce the research). Synthesis is about bringing ideas together to produce innovative insights that often lead to bigger conversations about how to socially innovate. Synthesis is bigger than science, but dependent on it. It means paying attention to parts and wholes together and is the epitome of systems thinking in knowledge work.
It also means taking responsibility as knowledge producers and consumers and be wary of shooting the messengers while asking more from the messages they deliver.
Unless we are prepared to give people time to search, appraise and synthesize research on their own — and train them to make informed choices — the role of synthesizers – professional, journalistic, or otherwise – will become more important than ever.
Photo from Wikimedia Commons and is used under licence.