Category: education & learning

Education and learning

complexitydesign thinkingeducation & learningsystems thinking

Integrative Thinking And Empathy in Systems

Seeing What You're Reaching For And With

Seeing What You’re Reaching For And With

Award-winning Canadian author and University of Toronto professor David Gilmour came under social/media fire for comments made about his stance of only including male, middle-aged writers in his list of readings for his undergraduate English courses because that is the experience he resonates with most. Drawing on what you know is both wise and foolish when looking at it from the perspective of systems change and by looking within and beyond our own boundaries we can see how. 

Richard Katz knows what it is like to be an outsider and see the world from deep within and far from outside a culture. Katz, a former professor and elder with the First Nations University of Canada and Harvard-trained anthropologist, was one of the first non-native individuals to be welcomed into the lives of the Kalahari Ju|’hoansi peoples of central Africa. The Ju|’hoansi are known to Westerners as ‘the Bushmen‘ and were the ‘stars’ of the film The Gods Must Be Crazy. His journey and decades-long experience with these peoples are chronicled in two remarkable books on healing and culture.

Dr Katz worked closely with my undergraduate advisor and mentor, Dr. Mary Hampton, a remarkable community psychologist and her husband (and elder) Dr. Eber Hampton, and would occasionally come to meet and speak with us eager students and the healing communities in Regina, where I studied. In life, but particularly in working with affairs of the heart and soul (which is the stuff of healing and community), Katz would say:

Talk only of what you know

I didn’t fully understand the meaning of this when I heard it until much later in life. As one interested in the science as well as the art of healing I struggled to understand how we couldn’t speak of things unknown if we were seeking discovery — which is about making the unknown, known. Over time I came to ‘know’ more about what Katz meant:  that our perspective is one of many in a system and it is one, that if contemplated and welcomed with an open mind and heart, is valid and true while also being apart and unique. While we hold a stance those around you have their own perspective and stance that is both the same and different and in this lies the heart of healing.

Katz was trying to warn students and other researchers against the idea that we can just go into some place and ‘know it’ without being in it and that even in immersing ourselves in the worlds of others we are still but a traveller, just as they are in ours. He also suggested that we can’t know other systems without knowing our own (my words, not his).

It is the paradox that we can connect on a fundamental human level and still hold an independent, personal account. Being at one and apart at the same time. This is a hallmark feature of a complex system. It is also what makes integrative thinking and empathy so critical in such systems.

Knowing me, knowing you

This brings us back to professor Gilmour. Speaking to the online culture magazine Hazlitt, David Gilmour said that he doesn’t teach books written by women, just men. This has caused a predictable uproar in the social/ media (see Storify link below).

In an interview with the Toronto Star, Gilmour tried to clarify his comments:

“My only point is that I tend to teach people whose lives are close to my own,” said Gilmour, who has taught at the university for seven years. “I’m an old guy and I understand about old guys.”

On the surface, Gilmour is doing just what Dick Katz implored us all to do: speak of what you know. Gilmour knows ‘old guys’ (who are White and straight) and not women or other ‘groups’. He is being authentic and true to his experience.

What Gilmour is missing on this topic is the empathy that is so important in working with complexity. Teaching English to undergraduates might not be an obvious example of systems thinking and complexity, but it can be. As Gilmour points out, English is about a point of view, which is another way to say its about where you stand. The writing of the ‘old guys’ Gilmour includes in his courses are able telling a narrative from a point of view. That makes for good literature.

Yet, it is the reader’s ability to adopt, interpret, experience and critique the point of view of a story character that makes a literary work compelling. That is in large part about empathy. Great writers make empathy easy. By being empathic, we see a setting or context — a system — that might be unfamiliar to us in ways that seem familiar by bringing us momentarily into the world of the other. This familiarity allows us to draw on the experience we have in other settings and contexts and apply them to the new one.

To the degree this has harmony and congruence with the narrative being told is the measure of fit between data from one context to another.

This is what we do in systems work. For Gilmour, the complexity in his system comes not from his perspective, but that of his students. They are women, maybe GLBT, most certainly from other age and cultural groups and geographic contexts. Gilmour is asking his students to empathize with his ‘old guy’ narrative while forgetting that he can empathize with the narrative of someone who is Asian, queer, or speaks Catalan in drawing narratives that can be welcomed into the classroom without it being the perspective he’s most familiar with. Indeed, it is when we extend ourselves beyond the most familiar narratives to finding something resonant in other narratives that we learn, discover and innovate.

Integrative (Design) Thinking

Integrative thinking is a concept that Roger Martin, also from the University of Toronto, has made popular and integrated into the teaching at the Rotman School of Business. (Indeed, Rotman’s marketing material brands itself as providing “a new way to think”). This style of thinking, which Martin has written about extensively through his research on CEO decision making, has been closely linked with design thinking, which is also tied closely to thinking about systems. It is about holding different ideas together at the same time and building models of reality through the exploration of these opposable thoughts.

It is a vehicle for empathy to flow through connecting feelings and observations with thoughts and prototyping actions. This is ultimately what we do when we design for engagement in complex systems. We aim to place ourselves in the system we seek to influence, learn where we are in relation to the boundaries we see, set those boundaries (maintaining flexibility throughout) and then build mechanisms to get feedback and probe the culture we are a part of — organizationally, individually and so on — to enable us to take some action. This continues in an iterative manner throughout our engagement with the system.

Integrative thinking combined with empathy allows us to engage human systems we don’t fully know in a meaningful way that recognizes our limits — speaking to Katz’s point about ‘talking about what we know’ — while opening up possibilities for communion on issues of shared concern.

This means that we can know others, but also that we can only know them as ourselves. It also means that the systems change we seek in our social world is both an intensely personal journey and one that shares our common humanity, regardless of whether we are looking at shifting an organization, a community or a global culture.

Perhaps by taking a bigger view, professor Gilmour might find the same passion in literature that is from a different perspective and ultimately find how its also very much the same.

education & learningevaluationsystems thinkingUncategorized

Scaling Education: The Absurd Case of the MOOC

Theatre at the Temple of Apollo

Theatre at the Temple of Apollo

The Chronicle of Higher Education (online) recently reported results of a survey looking at faculty teaching on MOOC’s (massive open online course) and found much interest and expectation around this new format.

The survey, conducted by The Chronicle, attempted to reach every professor who has taught a MOOC. The online questionnaire was sent to 184 professors in late February, and 103 of them responded.

Hype around these new free online courses has grown louder and louder since a few professors at Stanford University drew hundreds of thousands of students to online computer-science courses in 2011. Since then MOOCs, which charge no tuition and are open to anybody with Internet access, have been touted by reformers as a way to transform higher education and expand college access. Many professors teaching MOOCs had a similarly positive outlook: Asked whether they believe MOOCs “are worth the hype,” 79 percent said yes.

The survey of professors was not scientific, particularly because it was of those who are already teaching MOOC’s, but it paints a picture of enthusiasm among those — many who were initially reticent about the potential of online education at that scale – engaged with the medium.

Global Potential vs Global Hype

NY Times columnist Thomas Friedman, never one to resist enthusiasm for global movements, speaks of the MOOC as a revolution. Friedman suggests how a MOOC-driven education system could potentially change foreign aid given its promise to do good things.

Anant Agarwal, the former director of M.I.T.’s artificial intelligence lab, is now president of edX, a nonprofit MOOC that M.I.T. and Harvard are jointly building. Agarwal told me that since May, some 155,000 students from around the world have taken edX’s first course: an M.I.T. intro class on circuits. “That is greater than the total number of M.I.T. alumni in its 150-year history,” he said.

Yes, only a small percentage complete all the work, and even they still tend to be from the middle and upper classes of their societies, but I am convinced that within five years these platforms will reach a much broader demographic. Imagine how this might change U.S. foreign aid. For relatively little money, the U.S. could rent space in an Egyptian village, install two dozen computers and high-speed satellite Internet access, hire a local teacher as a facilitator, and invite in any Egyptian who wanted to take online courses with the best professors in the world, subtitled in Arabic.

For Friedman and many of those writing on MOOCs, the potential for this new format to bring the world’s best higher education to anyone, anywhere, in any country is enormous. It is an example of taking an innovation to scale on perhaps the most extreme. With the click of a mouse the entire world can learn together easily.

Except, it is a lie.

Writing as a guest on the Worldwise blog as part of the Chronicle of Higher Education, professor of writing and rhetoric Ghanashyam Sharma  puts truth to the lie of the modern online education movement’s hype. In an article called A MOOC Delusion: Why Visions to Educate the World Are Absurd, Sharma illustrates the faulty thinking that underpins much of the MOOC enthusiasm for transforming global education. His critique is less about MOOC’s as an educational vehicle in itself, but the global, culture-free scale at which they seek to operate.

There is a dire need for some healthy skepticism among educators about the idea that MOOCs are a wonderful means to go global in order to do good. For our desire to educate the whole world from the convenience of our laptops to be translated into any meaningful effect, we need more research about how students learn in massive open online platforms, and a better understanding of how students from different academic, cultural, social, and national backgrounds fare in such spaces.

Education at Scale

Drawing on his own personal experience with teaching in different contexts, beginning with Nepal and then moving to the University of Louisville and now SUNY Stony Brook, Sharma points to the subtleties in cultural learning styles that did not translate from space to space. He speaks of enormous challenges and, fortunately, the opportunity to meet these with resources to aid him in adapting his teaching from context to context. In doing so, he points to the myth that the global classroom will be filled with people all learning the same way, from a compatible perspective, and using the same language. Even simple differences in the way he presents as a teacher can change the manner in which students learn – and those differences are rooted in culture.

As a teacher myself, I know much of what he writes. Even a ‘simple’ face-to-face graduate course presents considerable pedagogical challenges for me. As an instructor  in public health I need to consider things like:

  • Disciplinary background. Each discipline has communication ‘sub-cultures’ and traditions that differ. A student with a sociology background might be used to rhetoric while one from the basic sciences may be used to communicating through technical reports. Each discipline also uses language in different ways, with terms unique to that discipline.
  • Cultural backgrounds. Students from around the world will present differently in the way they approach the material, the presentation of arguments, and the level of participation in class. Even within a narrow band of cultural contexts that present within my classroom (usually there is perhaps 10-20 per cent of students are international) I am always amazed at the nuances that play out in the classroom. Race, gender and language all add additional complex layers that would require volumes to unpack here.
  • Personality and motivation. Students that are more reserved in class will experience each lesson differently than those who are outgoing. Personality and motivation change the type of discussions learners engage in and shape their interactions with other students. This is not to say that one personality style is better than the other, but whether you’re introverted or extraverted, confident, clear spoken or highly social shapes the classroom. Face-to-face encounters allows for some modulation of these effects to encourage participation with those of different needs.
  • Literacy. Whether it prose, numeracy, or discipline-specific language use, the literacy of students is tested when they have to take in material — whether through lecture, small groups, video, audio or text — and convey arguments.

These are constraints (and opportunities) in most modern universities, whether the course is delivered face-to-face or online. The amount of effort to engage a room full of eager learners is enormous if I attend to these various issues. As an educator, it is effort that I believe makes for a good experience for everyone and is a joyous part of the job. But I am speaking to a room of about 25 graduate students at one of the best universities in the world who are all present and sharing the same environmental context even if it is as a visiting student.

What happens when we are teaching to a ‘room’ of 150,000 people from all over the world?

Understanding scale

I do believe that we can learn much from online courses and that much can be conveyed via the MOOC that brings content to the world in ways that are appropriate and useful for a general audience. I am not against the MOOC, but I do think we need to carefully consider what it means to take education to scale and ask some deeper questions about what the learning experience is intended to achieve.

We need to design our educational experience using the same principles or design thinking we would apply to any other service of value.

The lie is in believing true education of a global audience in the rich way that a university or college intends is not only possible, but appropriate. What is being lost in the effort to make a common experience among a global classroom? Are we just sending out information or are we creating a learning environment? To what degree does the MOOC serve this purpose well. That is what a designer might start with as they move to asking how might we bring learning from one context to another.

Ghanashyam Sharma points to the folly in those who think the jump from culture to culture can be easily made when teaching the world at one time. The MOOC offers enormous potential to re-shape the way people learn and promoting access to content and expertise that many in this world could have only dreamed of years ago. But the naivete that such learning can be done in an monocultural way without losing something special about the context in which people learn and use what they learn might lead to some expensive lessons.

Sharma is right to ask for more research. Thomas Friedman is right to inspire us to think of creative ways to bring the wealth of the west’s educational treasures to the globe. The key is to figure out what parts of this global vision are real or possible, which are illusions, and which are delusions.

Unlike the auditorium at the Temple of Apollo at Delphi (pictured above) the audience for what we teach now are not from one place, time and shared cultural perspective; they represent the world. Without an understanding of what scale means in education we might be producing more ignorance than knowledge.

Photo: Cameron Norman

complexityeducation & learningsystems sciencesystems thinking

A toolkit for toolsets, skillsets and a mindset

Image

For the last few years Censemaking has been a forum for exploring ideas around complexity, systems, design and social innovation. It has been a space for ideas and considering some of social and evaluative ramifications of complexity as it plays out in human systems.

This long-form blog has allowed for in-depth reflection on the issues that influence innovation and health in human systems.

Today I am launching a sister blog on the CENSE Research + Design site dedicated to exploring the methods and tools that can help us understand how to impact these systems. This new blog will feature short-form, practice-oriented articles that are aimed at building or augmenting the toolkit of the social innovation practitioner.

Censemaking will continue to explore issues in depth and will occasionally refer readers to the CENSE toolkit blog as a means of building the mindset, skillset and toolset of those readers interested in navigating complexity and designing innovation. The focus will be on systems science methods, design and design thinking tools, strategy, behavioural and applied social science techniques, and program evaluation.

I look forward to engaging readers through both venues and learning from you and with you.

Thanks for reading.

Photo Toolkit by Nick Farnhill used under Creative Commons Licence from Flickr.

education & learninginnovationjournalismknowledge translation

Is Knowledge Translation In Health Too Important to Leave to Health Professionals?

Storytelling

Knowledge translation — and its affiliated terms knowledge exchange, knowledge integration and knowledge mobilization — was coined to describe a process of taking what is known into what is done in health across the spectrum of science, practice, policy and  the public’s health. As health issues become more complex due to the intertwining of demographics, technology, science, and cultural transformations the need to better understand evidence and its impact on health has never been higher. Questions remain: has demand met supply? How are the health professions dealing with this equation?

Translating knowledge

The Canadian Institutes of Health Research (CIHR), one of the earliest champions of the concept of knowledge translation in research, define it as:

a dynamic and iterative process that includes synthesisdisseminationexchange and ethically-sound application of knowledge to improve the health of Canadians (sic), provide more effective health services and products and strengthen the health care system.

These ideas are expanded below:

Synthesis – Synthesis, in this context, means the contextualization and integration of research findings of individual research studies within the larger body of knowledge on the topic. A synthesis must be reproducible and transparent in its methods, using quantitative and/or qualitative methods. It could take the form of a systematic review, follow the methods developed by the Cochrane Collaboration, result from a consensus conference or expert panel or synthesize qualitative or quantitative results. Realist syntheses, narrative syntheses, meta-analyses, meta-syntheses and practice guidelines are all forms of synthesis. Resources related to synthesis are available.

Dissemination – Dissemination involves identifying the appropriate audience and tailoring the message and medium to the audience. Dissemination activities can include such things as summaries for / briefings to stakeholders, educational sessions with patients, practitioners and/or policy makers, engaging knowledge users in developing and executing dissemination/implementation plan, tools creation, and media engagement.

Exchange – The exchange of knowledge refers to the interaction between the knowledge user and the researcher, resulting in mutual learning. According to the Canadian Health Services Research Foundation (CHSRF), the definition of knowledge exchange is “collaborative problem-solving between researchers and decision makers that happens through linkage and exchange. Effective knowledge exchange involves interaction between knowledge users and researchers and results in mutual learning through the process of planning, producing, disseminating, and applying existing or new research in decision-making.”

Ethically-sound application of knowledge – Ethically-sound KT activities for improved health are those that are consistent with ethical principles and norms, social values, as well as legal and other regulatory frameworks – while keeping in mind that principles, values and laws can compete among and between each other at any given point in time. The term application is used to refer to the iterative process by which knowledge is put into practice.

In short, knowledge translation is about taking what we learn and know from evidence, sharing that knowledge with others and assisting them to make useful health choices in practice and policy through KT.

This often involves communicating across contexts, disciplines, and roles between and from scientists, clinicians, policy makers and to the public alike. In a health environment that is increasingly becoming complex, the ability to communicate across boundaries is no longer an advantage, it’s an essential skill. While we may not always have the right language, we can translate meaning through stories.

But if stories are to be effective they need to be valued.

The value of storytelling

I’ve seen health professionals — scientists and clinicians — roll their eyes when you mention storytelling in a work context. It is as if the only legitimate role for stories is to communicate with children (which University of Alberta researchers are exploring as a tool for sharing health knowledge with parents). Yet, it is through stories that most people share what they know in every other context; why would it be different in health?

Perhaps it is the connotation that stories are ‘made up’ like children’s bedtime tales, but one need only look to journalism to find that we’ve been making ‘stories’ a central part of our life every day. We listen to drive-time radio for stories about the traffic conditions, we watch, download and listen to news stories filed by professional journalists and citizen bloggers alike on mainstream media, Twitter, YouTube, Facebook along with myriad sources across the web. Last week we were glued to various sources to learn storiessome of them false — and create stories about the events of the Boston Marathon bombings.

Stories are what conveys multiple information threads and puts it in a coherent context.

Stories are coherence engines.

Valuing knowledge translation

If knowledge translation is important then it should be reflected in research priorities and evidence for its impact on the system across different disciplines. Dr Shannon Scott and her U of A team recently conducted a systematic review of knowledge translation strategies in the allied health professions and found that the field was full of low quality studies that made it impossible to make firm statements on which methods were best among them . That team has recently proposed a systematic review looking at how the arts and visual methods can further contribute to KT in practice, although it likely the same issue with methodological quality might come into play here, too.

What she and her team are doing is looking at the process of sharing stories and, from a research perspective, sharing stories appears to not have been worth investing in scientifically. At least, not enough to generate a lot of studies and good evidence.

One could argue that knowledge translation is still new and that it takes time to generate such evidence. That is partly true, but it is also an easy prop for those who want to avoid the messiness that comes with communication (and its problematic research context), learning from others, and creating more equitable information spaces, which is what knowledge translation ultimately does. Knowledge translation has also been in use for almost 20 years so in that time — even with the most dismal assessment of the length of time it takes to put knowledge into practice — we should be seeing some decent research published.

KT is fundamentally about sharing. Journalists’ are rewarded for sharing — the more they share and the more people who they share with (as measured by readers, listeners, viewers etc..) the more successful they are in their work. Teachers are rewarded for sharing because that means that they are teaching people. Librarians are rewarded for sharing because that means people are checking out books and using the resources in their library.

We don’t apply the same standard to academic research, even though we have some crude metrics to measure reach and impact,  and there is roughly no metric for the degree to which clinicians share among themselves. Maybe this needs to change.

I have scientific colleagues who are fierce in the face of their most strident academic critics and have delivered keynotes to auditoriums filled with researchers that are nearly paralyzed in the face of speaking to the public. This is not fear of public speaking, its fear of speaking to the public.

Should they be? I don’t think speaking to the public should be expected to be enjoyable for everyone, but neither are doing statistical calculations, completing ethics applications, or presenting posters at conferences, but we still expect scientists to do that. We still expect nurses, doctors, psychologists, medical technicians and social workers to traverse complex social problems to talk to their patients in an open and honest way.

Why is it when scientists are speaking to policy makers, clinicians to scientists, policy makers to the public, or any professional to another from another discipline, speciality or division we decide its not critical for them to make the effort?

Why don’t we do the research to support it? 

Why is it OK not to do KT because its uncomfortable, awkward, difficult or confusing?

Declining interest, rising demand

It is perhaps for reasons like this that knowledge translation is so poorly understood and taken up as a focus for research. Looking at Google NGram data (which tracks mention of specific topics in books and publications) we see a steady rise in citations until about 2003 followed by a levelling off. Keep in mind that the leveling begins before social media became known. In the years after Twitter, Facebook and YouTube — arguably the most powerful communications media we have for doing knowledge translation widely (but perhaps not deeply) — there is roughly no sharp increase.

Below are the citations for the terms knowledge translation, knowledge exchange, and knowledge integration  from 1996 (when the Web first started gaining wide use beyond academia and the military) and 2008, the latest year for which there is available data. Note that the numbers reflect general mentions as a percentage of overall terms, so they are relative, not absolute values.

Figure 1: Google NGram Data for KT, KE & KI: 1996-2008

Knowledge Translation, Exchange & Integration NGram

Is there so much other stuff to talk about in 2013 that the relative importance of knowledge translation is diminished?

A look at Google Trend data using the same terms finds that not only are these concepts not growing, their mention is actually shrinking.

Looking at the three terms we see that all three concepts have declined over time. During these years — 2004-2013 — we saw not only the birth of social media, but the rise of Internet-enabled handheld devices to allow knowledge to be shared anywhere there is a data signal. We now have apps and nearly all of the Internets resources in our pockets and we are seeing a decline in the use of these terms.

Figure 2: Google Trend Data for KT, KE & KI: 1996-2013

Knowledge Term Trends

Where to?

So to review: We have a body of evidence in KT that is problematic and incomplete at the same time we have a decrease in use of the terms, while at the very same time we have a sharp rise in available tools and technologies to share information quickly and a continued, steady demand for more information to make decisions for health providers, patients, policy makers and insurers.

Yes, the data presented here are not perfect. But does it not make sense that there should at least be some trend upward if knowledge translation is valued? Should we not see some shift to more research, better research evidence, and greater interest given the tools and scope of communications we have through social media?

This begs the question: is knowledge translation in health too important to leave to health professionals? 

In future posts this question will be looked at in greater depth. Stay tuned.

* Blog has been updated since original post

design thinkingeducation & learning

Hacking the Classroom: Beyond Design Thinking

A nice summation of what Design Thinking is and how its been applied elsewhere with an eye towards education. This is shared from the User Generated Education blog.

User Generated Education

Design Thinking is trending is some educational circles.  Edutopia recently ran a design thinking for educators workshop and I attended two great workshops at SXSWedu 2013 on Design Thinking:

Design Thinking is a great skill for students to acquire as part of their education.  But it is one process like the problem-solving model or the scientific method.   As a step-by-step process, it becomes type of box.  Sometimes we need to go beyond that box; step outside of the box.  This post provides an overview of design thinking, the problems with design thinking, and suggestions to hacking the world to go beyond design thinking.

Design Thinking

Design thinking is an approach to learning that includes considering real-world problems, research, analysis, conceiving original ideas, lots of experimentation, and sometimes building things by hand (http://blogs.kqed.org/mindshift/2013/03/what-does-design-thinking-look-like-in-school). The following graphic…

View original post 1,182 more words

education & learning

Much Ado About MOOC

Leading? Learning? Both? Neither?

It’s fair to imagine that one of the 2013 ‘words of the year‘ will be MOOC (which is not really a word, but an acronym that stands for Massive Open Online Course). It seems that everywhere you look in the higher education and professional development space we are seeing MOOC’s talked about and debated.

HBR editor Eric Hellwig, writing for the HBR blog, reported on a recent panel on MOOC’s held at the recent World Economic Forum in Davos with leaders like Bill Gates, Peter Thiel and Larry Summers. His report reflects the exuberance of the MOOC and the techno-deterministic spirit of much of the discourse on these tools:

The advent of massively open online classes (MOOCs) is the single most important technological development of the millennium so far. I say this for two main reasons. First, for the enormously transformative impact MOOCs can have on literally billions of people in the world. Second, for the equally disruptive effect MOOCs will inevitably have on the global education industry.

One of the panelists was Stanford professor Daphne Coller, the co-founder of Coursera, one of the largest MOOC providers offering more than 200 courses to millions of students worldwide. Coller has convinced top faculty at leading universities to provide high quality digital courses through Coursera for free and the result has surprised her.

We’re at 2.4 million students now. The biggest lesson I’ve learned on this is I underestimated the amount of impact this would have around the world. I really didn’t envision this scale and this impact this quickly.

Disrupting Education

Of these panelists, Peter Thiel may be the most controversial. He has spoken at length about the need to revamp education and sees technology and platforms like Facebook as a means to do it. (It’s worth noting that Theil was also an early investor in Facebook). He points to the multiple roles that education plays well beyond learning and suggests that when we go beyond that goal we start creating false economies of value within higher education:

You have to ask yourself, ‘What is the nature of education as a good?’ Ideally you want it to be learning. But it also functions as insurance. Parents will pay a lot of money for insurance against cracks in our society. Education as insurance has something to be said because it connects to the economy. You know computer science, you can get a job. But education also functions as a tournament. You do well if you go to a top school but for everyone else the diploma is a dunce hat in disguise. People need to understand what they’re trying to do? Is it insurance? A tournament? Learning?

Among Thiel’s biggest concerns is with the current educational system’s ability to support the kind of innovative thinking needed to make technological and scientific breakthroughs. So steadfast is he in the belief that some of the best minds are rotting in traditional classrooms that he founded the Thiel Fellowship, a scholarship fund to support promising young entrepreneurs in dropping out of school to pursue their ambitions of making social impact.

Thiel is disrupting education by taking learning away from the educational institutions charged with providing it. MOOC providers are seeking to develop a business model that puts them in the drivers seat of education and learning, drawing potential revenues away from traditional educational institutions. This will no doubt add to the pressures that universities and colleges are already facing as they rationalize ever more of what they do.

Education For All, Learning For Whom?

Free online learning of the calibre provided by Stanford University, Caltech, Harvard University, University of Toronto, MIT, and the Santa Fe Institute  for anyone, anywhere sounds like a dream come true.

In some ways it is. In others, it’s an illusion.

This comes back to the metrics we use in evaluating the impact of education and asking what its point is in the first place. What do we mean by learning and are we serious about it?

It’s been suggested that less than 10 per cent of those enrolled in a MOOC complete it. And of this 10 per cent, it isn’t clear what they learn, how well they learn it, and what kind of application (if any) that content is made to issues away from the course. Online courses with video tutorials, self-organized learning and largely uni-directional teaching bring together the best of former teaching methods like instructional TV, self-help, and classroom lectures.

The problem is that this ‘best’ isn’t particularly effective. A 2000 meta-analysis of distance instructional methods found:

There does not appear to be a difference in achievement between distance and traditional learners. Of the ten instructional features that were analyzed, only three had an impact on student achievement. These three features were type of interaction available during a broadcast, type of course, and type of remote site. There was an insufficient number of studies to ascertain whether or not the education level of the distance learners effected their achievement in the course (Machtmes & Asher, 2000).

While this review was done before widespread Internet use, the methods included reflected the same list above with one- or two-way audio and video. The studies were also done on programs that were designed for credit, not voluntary non-credit courses. Research on motivation will show that optional programs are far less likely to engender behavioural shifts than those that are mandated.

So who then is benefitting from MOOC’s? We don’t yet know, but it is likely those with time to attend to the content, high levels of intrinsic motivation (< PDF), the technological tools to succeed, and the environment that is ready to support integration of content into practice. That’s a tall order.

We are in the early days of MOOC’s and its too soon to tell how successful they will be. However, theoretically there is relatively little reason to expect that they will produce the kind of results worthy of hyperbole — at east not with those already accustomed to alternatives. To offer a MOOC from a world-class university to a learner somewhere in the world where education is but a distant dream achieves a great deal. But to transfer MOOC’s to replace more interactive and engaging methods — usually face-to-face — and expect great learning is a bit implausible.

Yet, with what we are offering now to students in the form of large classes, disconnected curricula, and didactic instruction MOOC’s offer an attractive option. What it loses is the experience of learning that is not packaged in a class. This means a change to campus life, the informal and serendipitous learning that comes from being in the same physical space interacting with each other, and may seriously limit the use of thinking and creative tools that design thinking and applied creativity demand. (for a detailed look at MOOC’s and the modern university check out Nathan Harden’s essay in the American Interest).

There is much ado about MOOC’s, but is this a Shakespearian tragedy in the making for learners?

Photo credit: iStockPhoto used under license.

education & learning

The Quality Metric in Education

Image

What goes on the pedestal of learning?

What is quality when we speak of learning? In this third post in series on education and evaluation metrics the issue of quality is within graduate and professional education is explored with more questions than answers about the very nature of learning itself.

But what does learning really mean and do we set the system up to adequately assess whether people do it or not and whether that has any positive impact on what they do in their practice.

What do you mean when you say learning?

The late psychologist Seymour Sarason asked the above question with the aim of provoking discussion and reflection on the nature and possible outcomes of educational reform. Far from being glib, Sarason felt this question exposed the slippery nature of the concept of learning as used in the context of educational programming and policy. It’s a worthwhile question when considering the value of university and professional education programming. What do we mean when we say learners are learning?

The answer to this question exposes the assumptions behind the efforts to provide quality educational experiences to those we call learners. To be a learner one must learn…something.

The Oxford English Dictionary defines learning this way:

learning |ˈlərniNG|

noun

the acquisition of knowledge or skills through experience, practice, or study, or by being taught: these children experienced difficulties in learning | [ as modifier ] : an important learning process.

• knowledge acquired in this way: I liked to parade my learning in front of my sisters.

ORIGIN Old English leornung (see learn,-ing1) .

This might sufficiently answer Dr Sarason except there is no sense of what the content is or whether that content is appropriate, sufficient, timely or well-supported with evidence (research or practice-based); the quality of learning.

Knowledge translation professionals know that learning through evidence is not achieved through a one-size-fits-all approach and that the match between what professionals need and what is available is rarely clean and simple (if it was, there would be little need for KT). The very premise of knowledge translation is that content itself is not enough and that sometimes it requires another process to help people learn from it. This content is also about what Larry Green argues: practice-based evidence is needed to get better evidence-based practice.

How do we know when learning is the answer (and what are the questions)?

If our metric of success in education is that those who engage in educational programming learn, how do we know whether what they have learned is of good quality? How do we know what is learned is sufficient or appropriately timed? Who determines what is appropriate and how is that tested? These are all questions pertaining to learning and the answers to them depend greatly on context. Yet, if context matters then the next question might be: what is the scope of this context and how are its parameters set?

Some might choose academic discipline as the boundary condition. To take learning itself as an example, how might we know if learning is a psychology problem or a sociology problem (or something else)? If it is a problem for the field of psychology, when does it become educational psychology, cognitive psychology, community psychology or one of the other subdisciplines looking at the brain, behaviour, or social organization? Successful learning through all of these lenses means something very different across conditions.

Yet, consider the last time you completed some form of assessment on your learning. Did you get asked about the context in which that learning took place? When you were asked questions about what you learned on your post-learning assessment:

  • Did it take into account the learning context of delivery, reception, use, and possible ways to scaffold knowledge to other things?
  • Did your learner evaluation form ask how you intended to use the material taught? Did you have an answer for that and might that answer change over time?
  • Did it ask if your experience of the learning event matched what the teachers and organized expected you to gain and did you know what that really was?
  • Did you know at the time of completing the evaluation whether what you were exposed to was relevant to the problems you needed to solve or would need to solve in the future?
  • Did you get asked if you were interested in the material presented and did that even matter?
  • Was there an assumption that the material you were exposed to could only be thought of in one way and did you know what that way was prior to the experience? If you didn’t think of the material in the way that the instructors intended did you just prove that the first of these two questions is problematic?

Years of work in post-secondary teaching and continuing professional education suggests to me that your answer to these questions was most likely “no”, except the very last one.

These many questions are not posed to antagonize educators (or “learners”, too) for there are no single or right answers to any of them. Rather, these are intended to extend Seymour Sarason’s question to the present day and put in the context of graduate and professional education at a time when both areas are being rethought and rationalized.

Learning to innovate (and being wrong)

A problem with the way much of our graduate and professional education is set up is that it presumes to have the answers to what learning is and seeks to deliver the content that fills a gap in knowledge within a very narrow interpretation. This is based on an assumption that what was relevant in the past is both still appropriate now and will be in the future unless we are speaking of a history lesson. However, innovation and discovery — and indeed learning itself — is based on failure, discomfort and not knowing the answers as much as building on what has come before us. There is no doubt that a certain base level of knowledge is required to do most professional and scientific work and that building a core is important, but it is far from sufficient.

The learning systems we’ve created for ourselves are based on a factory model of education, not for addressing complexity or dynamic systems like we find in most social worlds. We do not have a complex adaptive learning system in place, one that supports innovation (and the failures that produce new learning) because:

If you’re not prepared to be wrong, you’ll never come up with anything original. – Sir Ken Robinson, TED Talk 2006

The above quote comes from education advocate Sir Ken Robinson in a humorous and poignant TED talk delivered in 2006 and then built on further in a second talk in 2010. Robinson lays bare the assumptions behind much of our educational system and how it is structured. He also exposes the problem we face in advancing innovation (my choice of term) because we have designed a system that actively seeks to discourage wide swaths of learning that could support it, particularly with the arts.

Robinson points to the conditions of interdisciplinary learning and creativity that emerge when we free ourselves of the factory model of learning that much of our education is set up, “producing” educated people. If we are assessing learning and we go outside of our traditional disciplines how can we assess whether what we teach is “learned” if we have no standard to compare it to? Therein lies the rub with the current models and metrics.

If we are to innovate and create the evidence to support it we need to be wrong. That means creating educational experiences that allow students to be wrong and have that be right. If that is the case, then it means building an education system that draws on the past, but also creates possibilities for new knowledge and learning anchored in experimentation and transcends disciplines when necessary. It also means asking questions about what it means to learn and what quality means in the context of this experimental learning process.

If education is to transform itself and base that transformation on any form of evidence then getting the right metrics to evaluate these changes is imperative and quality of education might just need to be one of them.

Image: Shutterstock