Tag: teaching

education & learningknowledge translationpsychologysystems thinking

Bullying, the market for education and the damaged quest for learning

Dark classroom, light minds

Dark classroom, light minds

A recent study found looked into the experience of cyberbullying by university professors at the hands of their students. This disturbing phenomenon points to much larger issues beyond mental health promotion and calls into question many of the assumptions we have about the systems we’ve designed to foster education and what it means to be a learner at university. 

The university is one of our oldest cultural institutions and its instructors are considered to have among societies most respected jobs, even if not always well compensated. In the past, students often approached their professors with a mixed sense of wonder, respect, curiosity and fear and that, in healthy situations, was reciprocated by faculty to create a space where people could explore ideas, learn, and challenge themselves and others to grow. That relationship has started to change as evidenced by the rise of cyberbullying in the classroom.

A recent article in Macleans Magazine looked at the changing state of the post-secondary classroom and the role of cyberbullying. Only this was not about student victims, but students as the perpetrators against their professors. The effects of cyberbullying are crippling and professors are bearing the burden of having hundreds of eyes watching them, writing about them and writing ‘consumer reviews’ about them in anonymous and sometimes unflattering, inflammatory and questionable terms on sites like RateMyProfessor.com .

Researchers at the University of California, Riverside found that as students age the incidence of face-to-face bullying decreases and cyberbullying increases, which might partly explain why we’re seeing this in university settings when face-to-face bullying goes subterranean. Yet, the notion that professors that are getting bullied by their students belies some other issues that require further investigation, namely those related to the nature of education and the role of students-as-consumers.

Consuming knowledge, producing expectations

If you pay for something, should you not expected to get something rather specific for that experience or product? Aside from some rare experiences of profane/profound personal challenge/punishment like Tough Mudder and its peers or dental work, there are few things we willingly pay for that we don’t derive pleasure from or achieve a very specific (anticipated) outcome.

Education is problematic because we might not know what we’ll get from it going in, what kind of experiences or ideas will emerge, and how our relationship to those experiences will change us. That is its great gift.

Many of us have had profound life changes because of something we experienced through our education and writing as one who has completed four different degree programs and a post-doc I can confidently say that I didn’t receive a lot of what I expected in any of those programs and I am a better person for it. Indeed, if I go to a specific learning event (aside from those focused on a specific technique or technology) I am disappointed if I actually come away with exactly what I expected.

That is part of the point. We don’t know what we don’t know.

But when you start viewing education as a thing that resembles any other market-driven product or services, you begin to focus on learning as a consumable good and your students as customers. In following this line of thought, it makes some sense to focus the delivery of this product on the desires of the consumer.

Increasingly, teachers (of various stripes) are being asked to consider a range of student-related variables in their education. Things like learning styles and preferences are now being woven into classroom instruction and students have come to learn to expect and are increasingly demanding to be taught in ways that match their unique learning preferences and styles. While there is reason to imagine that this approach is useful in stimulating engagement of students in the lessons, there is increasing evidence much of it does little to enhance actual learning. Many of the life lessons we’ve gained that shape what we do and who we are were not delivered in the manner of our choosing, conformed with our preferences and were not desired, expected or enjoyed in the moment. We risk confusing enjoyment with learning; they can be aligned but one isn’t necessary for the other to take place.

However, when we are viewing education from a consumer model, the specific outcomes become part of the contract. If I come to get a degree in X because I believe that the job market demands the skills and knowledge that X brings and I am paying tens of thousands of dollars and spending four or more years acquiring X then I feel entitled to expect all the benefits that X brings. Further, I expect that my journey to acquiring X will be enjoyable, because why would I spend more money than I’ve ever seen on something I don’t enjoy.

Particularly when that is money I don’t have.

A debt to pay

In Canada and the United States, student debt rates have dramatically increased. The Canadian Federation of Students note that Canadian’s attending post-secondary education now owe more than $15B to the Canadian federal government (PDF) as part of their student loan program, a number that doesn’t include debt accumulated from borrowing from banks, family, credit cards and other means. In Canada’s largest province, Ontario, the rate of graduate employment has decreased since 2001 and the overall youth unemployment rate continues to be the highest, despite the province having one of the most educated youth population in the country (and arguably, the world). And while Ontario universities continue to promote the fact that education is a better pathway to success, it is a hard pill for many students to swallow when many can’t apply what they trained for and paid for after they graduate.

Satirist John Oliver has an informative, humorous and distressing take on student debt and the state of consumer-oriented education for those who want to learn more.

None of these reasons are excuses for cyberbullying, but it does give a more complicated picture of those that might feel they are entitled to bully others and their reasoning behind it.

What we are seeing is a systems change in the way education is being produced, consumed and experienced. Even the mere fact that we can now reasonably use the language of consumerism to speak to something like education should give us pause and concern. I’ve been involved in post-secondary education for nearly 20 years and there has always been students who simply wanted the ‘piece of paper’ (degree) as a stepping stone to a job and little more than that from their time at school. They were willing to do the work — often the minimum possible — to graduate, but they knew they had to put the effort in to be successful. There was never an expectation that one was entitled to anything from going to school, although that might be changing.

Market identities and education systems

Belgian psychotherapist Paul Verhaeghe has explored the role of identity in market-based economies in his new book What About Me? In the book, Verhaeghe illustrates how we construct our identities as people drawing on the research that reflects (and often contradicts or obscures) the two major perspectives on personality and identity: the person-as-blank-slate and the person as a reflection of the environment. The former perspective assumes we come into the world as we are while the latter assumes the world makes us who we are and both have enormous amount of moral, cultural and evidentiary baggage attached to them.

What Verhaeghe does is point to the ways in which both have elements of truth to them, but that they are mediated by the manner in which we construct the very questions about who we are and what our purpose is. These questions are (for many cultural, historical, economic and political reasons that he elaborates on) frequently market-based. Thus, who we are is defined by what we do, what we own, what we produce, and how we use such things once out into the world and that the value that come with such ways of defining ourselves is considered self-evident. He makes a disturbing and convincing case when one stops to reflect on the way we think about how we think (metacognition + mindfulness) .

When viewed from the perspective of a market, knowledge and its products soon become the goal and not the journey. Indeed, I’ve even written about this in support of an argument for better research-to-action and knowledge translation. Much of the knowledge-to-action discourse is about viewing knowledge as a product even if the more progressive models also view this as part of a process and even more as part of a system. But it is the last part — the system — that we often give the shortest shrift to in our discussions. What Verhaeghe and others are doing is encouraging us to spend more time thinking about this and the potential outcomes that emerge from this line of thinking.

Unless we are willing to talk more about the systems we create to learn, explore and relate we will continue to support Verhaeghe’s thesis and uphold the conditions for the kind of education-as-a-product thinking that I suspect is contributing to students’ changing behaviour with their professors and creating a climate at universities that is toxic instead of inspiring.

Photo credit: Classroom by Esparta Palma used under Creative Commons License via Flickr. Check out Esparta’s remarkable work here.

complexityeducation & learningevaluationsystems science

Evaluation, Evidence and Moving Beyond the Tyranny of ‘I Think’

I think, you think

The concrete evidence for ‘I think’

Good evidence provides a foundation for decision-making in programs that is dispassionate, comparable and open to debate and view, yet often it is ignored in favour of opinion. Practice-based evidence allows that expert opinion in, however the way to get it into the discussion is through the very means we use to generate traditional evidence. 

Picture a meeting of educators or health practitioners or social workers discussing a case or an issue about a program. Envision those people presenting evidence for a particular decision based on what they know and can find. If they are operating in a complex, dynamic field the chances are that the evidence will be incomplete, but there might be some. Once these studies and cases are presented, invariably the switch comes when someone says “I think...” and offers their opinion.

Incomplete evidence is not useful and complex systems require a very different use of evidence. As Ray Pawson illustrates in his most recent book, science in the realm of complexity requires a type of sensemaking and deliberations that differ greatly from extrapolating findings from simple or even complicated program data.

Practice-based evidence: The education case

Larry Green, a thought leader in health promotion, has been advocating to the health services community that if we want more evidence based practice we need more practice based evidence (video). Green argues that systems science has much to offer by drawing in connections between the program as a system and the systems that the program operates in. He further argues that we are not truly creating evidence-based programs without adding in the practice-based knowledge to the equation.

Yet, practice-based evidence can quickly devolve into “I think” statements that are opinion based on un-reflective bias, personal prejudice, convenience and lack of information. To illustrate, I need only consider a curriculum decision-making process at universities (and likely many primary and secondary schools too). Having been a part of training programs at different institutions — in-house degree programs and multi-centre networks between universities — I can say I’ve rarely seen evidence come into play in decisions about what to teach, how to teach, what is learned and what the purpose of the programs are, yet always see “I think”.

Part of the reason for this is that there is little useful data for designing programs. In post-secondary education we use remarkably crude metrics to assess student learning and progress. Most often, we use some imperfect time-series data like assignments that are aggregated together to form a grade. If this is undergraduate education, most likely we are using multiple-choice exams because they are easier to distribute and grade and use within the resource constraints. For graduate students, we still use exams, but perhaps we use papers as well. But rarely, do we have any process data to make decisions on.

Yet, we recruit students based on quotas and rarely look to where the intellectual and career ‘markets’ are going to set our programs. Instead, we use opinion. “I think we should be teaching X” or “I think we should teach X this way” with little evidence for why these decisions are made. It is remarkable how learning — whether formal or informal — in organizations is left without a clear sense of what the point is. Asking why we teach something, why people need to learn something, and what they are expected to do with that knowledge is a question that is well beyond a luxury. To get a sense of this absurdity, NYC principal Scott Conti’s talk at TEDX Dumbo is worth a watch. He points to the mis-match between what we seek to teach and what we expect from students in their lives.

Going small to go big

There is, as Green points out, a need for a systems approach to understanding the problem of taking these anecdotes and opinion and making them useful. Many of the issues with education have to do with resources and policy directions made at levels well beyond the classroom, which is why a systems approach to evaluation is important.Evaluation applied across the system using a systems approach that takes into account the structures and complexity in that system can be an enormous asset. But how does this work for teachers or anyone who operates at the front line of their profession?

Evaluation can provide the raw materials for discussion in a program. By systematically collecting data on the way we form decisions, design our programs, and make changes we create a layer of transparency and an opportunity to better integrate practice-based evidence more fully. The term “system” in evaluation or programming often invokes a sense of despair due to a perception of size. Yet, systems happen at multiple scales and a classroom and the teaching within it are systems.

One of the best ways to cultivate practice-based evidence is to design evaluations that take into account the way people learn and make decisions. It requires initially using a design-oriented approach to paying attention to how people operate in the classroom — both as teachers and learners. From there, we can match those activities to the goals of the classroom — the larger goals, the ones that ask “what’s the point of people being here” and also what the metrics for assessment are within the culture of the school. Next, consider ways to collect data on a smaller scale through things like reflective practice journals, recorded videos of teaching, observational notes, and markers of significance such as moments of insight, heightened emotion, or decisions.

By capturing the small decisions it is possible to generate practice-based evidence that goes beyond “I think”. It also allows others in. Rather than ideas being formed exclusively in one’s own head, we can illustrate where people’s knowledge comes from and permit greater learning from those around that person. Too often, the talents and tools of great leaders and teachers are accessible only in formal settings — like lectures or discussions — and not evident to others in the fire of everyday practice.

What are you doing to support evaluation at a small scale and allowing others to access your practice-based knowledge to create that practice-based evidence?

References:

Green, L. W. (2006). Public health asks of systems science: to advance our evidence-based practice, can you help us get more practice-based evidence? American Journal of Public Health, 96(3), 406–9.

Green, L. W. (2008). Making research relevant: if it is an evidence-based practice, where’s the practice-based evidence? Family practice, 25 Suppl 1(suppl_1), i20–4.

Pawson, R. (2013). The science of evaluation: A realist manifesto. London, UK: Sage Publications.

complexitydesign thinkingeducation & learningsystems thinking

Integrative Thinking And Empathy in Systems

Seeing What You're Reaching For And With

Seeing What You’re Reaching For And With

Award-winning Canadian author and University of Toronto professor David Gilmour came under social/media fire for comments made about his stance of only including male, middle-aged writers in his list of readings for his undergraduate English courses because that is the experience he resonates with most. Drawing on what you know is both wise and foolish when looking at it from the perspective of systems change and by looking within and beyond our own boundaries we can see how. 

Richard Katz knows what it is like to be an outsider and see the world from deep within and far from outside a culture. Katz, a former professor and elder with the First Nations University of Canada and Harvard-trained anthropologist, was one of the first non-native individuals to be welcomed into the lives of the Kalahari Ju|’hoansi peoples of central Africa. The Ju|’hoansi are known to Westerners as ‘the Bushmen‘ and were the ‘stars’ of the film The Gods Must Be Crazy. His journey and decades-long experience with these peoples are chronicled in two remarkable books on healing and culture.

Dr Katz worked closely with my undergraduate advisor and mentor, Dr. Mary Hampton, a remarkable community psychologist and her husband (and elder) Dr. Eber Hampton, and would occasionally come to meet and speak with us eager students and the healing communities in Regina, where I studied. In life, but particularly in working with affairs of the heart and soul (which is the stuff of healing and community), Katz would say:

Talk only of what you know

I didn’t fully understand the meaning of this when I heard it until much later in life. As one interested in the science as well as the art of healing I struggled to understand how we couldn’t speak of things unknown if we were seeking discovery — which is about making the unknown, known. Over time I came to ‘know’ more about what Katz meant:  that our perspective is one of many in a system and it is one, that if contemplated and welcomed with an open mind and heart, is valid and true while also being apart and unique. While we hold a stance those around you have their own perspective and stance that is both the same and different and in this lies the heart of healing.

Katz was trying to warn students and other researchers against the idea that we can just go into some place and ‘know it’ without being in it and that even in immersing ourselves in the worlds of others we are still but a traveller, just as they are in ours. He also suggested that we can’t know other systems without knowing our own (my words, not his).

It is the paradox that we can connect on a fundamental human level and still hold an independent, personal account. Being at one and apart at the same time. This is a hallmark feature of a complex system. It is also what makes integrative thinking and empathy so critical in such systems.

Knowing me, knowing you

This brings us back to professor Gilmour. Speaking to the online culture magazine Hazlitt, David Gilmour said that he doesn’t teach books written by women, just men. This has caused a predictable uproar in the social/ media (see Storify link below).

In an interview with the Toronto Star, Gilmour tried to clarify his comments:

“My only point is that I tend to teach people whose lives are close to my own,” said Gilmour, who has taught at the university for seven years. “I’m an old guy and I understand about old guys.”

On the surface, Gilmour is doing just what Dick Katz implored us all to do: speak of what you know. Gilmour knows ‘old guys’ (who are White and straight) and not women or other ‘groups’. He is being authentic and true to his experience.

What Gilmour is missing on this topic is the empathy that is so important in working with complexity. Teaching English to undergraduates might not be an obvious example of systems thinking and complexity, but it can be. As Gilmour points out, English is about a point of view, which is another way to say its about where you stand. The writing of the ‘old guys’ Gilmour includes in his courses are able telling a narrative from a point of view. That makes for good literature.

Yet, it is the reader’s ability to adopt, interpret, experience and critique the point of view of a story character that makes a literary work compelling. That is in large part about empathy. Great writers make empathy easy. By being empathic, we see a setting or context — a system — that might be unfamiliar to us in ways that seem familiar by bringing us momentarily into the world of the other. This familiarity allows us to draw on the experience we have in other settings and contexts and apply them to the new one.

To the degree this has harmony and congruence with the narrative being told is the measure of fit between data from one context to another.

This is what we do in systems work. For Gilmour, the complexity in his system comes not from his perspective, but that of his students. They are women, maybe GLBT, most certainly from other age and cultural groups and geographic contexts. Gilmour is asking his students to empathize with his ‘old guy’ narrative while forgetting that he can empathize with the narrative of someone who is Asian, queer, or speaks Catalan in drawing narratives that can be welcomed into the classroom without it being the perspective he’s most familiar with. Indeed, it is when we extend ourselves beyond the most familiar narratives to finding something resonant in other narratives that we learn, discover and innovate.

Integrative (Design) Thinking

Integrative thinking is a concept that Roger Martin, also from the University of Toronto, has made popular and integrated into the teaching at the Rotman School of Business. (Indeed, Rotman’s marketing material brands itself as providing “a new way to think”). This style of thinking, which Martin has written about extensively through his research on CEO decision making, has been closely linked with design thinking, which is also tied closely to thinking about systems. It is about holding different ideas together at the same time and building models of reality through the exploration of these opposable thoughts.

It is a vehicle for empathy to flow through connecting feelings and observations with thoughts and prototyping actions. This is ultimately what we do when we design for engagement in complex systems. We aim to place ourselves in the system we seek to influence, learn where we are in relation to the boundaries we see, set those boundaries (maintaining flexibility throughout) and then build mechanisms to get feedback and probe the culture we are a part of — organizationally, individually and so on — to enable us to take some action. This continues in an iterative manner throughout our engagement with the system.

Integrative thinking combined with empathy allows us to engage human systems we don’t fully know in a meaningful way that recognizes our limits — speaking to Katz’s point about ‘talking about what we know’ — while opening up possibilities for communion on issues of shared concern.

This means that we can know others, but also that we can only know them as ourselves. It also means that the systems change we seek in our social world is both an intensely personal journey and one that shares our common humanity, regardless of whether we are looking at shifting an organization, a community or a global culture.

Perhaps by taking a bigger view, professor Gilmour might find the same passion in literature that is from a different perspective and ultimately find how its also very much the same.

education & learningevaluationsystems thinkingUncategorized

Scaling Education: The Absurd Case of the MOOC

Theatre at the Temple of Apollo

Theatre at the Temple of Apollo

The Chronicle of Higher Education (online) recently reported results of a survey looking at faculty teaching on MOOC’s (massive open online course) and found much interest and expectation around this new format.

The survey, conducted by The Chronicle, attempted to reach every professor who has taught a MOOC. The online questionnaire was sent to 184 professors in late February, and 103 of them responded.

Hype around these new free online courses has grown louder and louder since a few professors at Stanford University drew hundreds of thousands of students to online computer-science courses in 2011. Since then MOOCs, which charge no tuition and are open to anybody with Internet access, have been touted by reformers as a way to transform higher education and expand college access. Many professors teaching MOOCs had a similarly positive outlook: Asked whether they believe MOOCs “are worth the hype,” 79 percent said yes.

The survey of professors was not scientific, particularly because it was of those who are already teaching MOOC’s, but it paints a picture of enthusiasm among those — many who were initially reticent about the potential of online education at that scale – engaged with the medium.

Global Potential vs Global Hype

NY Times columnist Thomas Friedman, never one to resist enthusiasm for global movements, speaks of the MOOC as a revolution. Friedman suggests how a MOOC-driven education system could potentially change foreign aid given its promise to do good things.

Anant Agarwal, the former director of M.I.T.’s artificial intelligence lab, is now president of edX, a nonprofit MOOC that M.I.T. and Harvard are jointly building. Agarwal told me that since May, some 155,000 students from around the world have taken edX’s first course: an M.I.T. intro class on circuits. “That is greater than the total number of M.I.T. alumni in its 150-year history,” he said.

Yes, only a small percentage complete all the work, and even they still tend to be from the middle and upper classes of their societies, but I am convinced that within five years these platforms will reach a much broader demographic. Imagine how this might change U.S. foreign aid. For relatively little money, the U.S. could rent space in an Egyptian village, install two dozen computers and high-speed satellite Internet access, hire a local teacher as a facilitator, and invite in any Egyptian who wanted to take online courses with the best professors in the world, subtitled in Arabic.

For Friedman and many of those writing on MOOCs, the potential for this new format to bring the world’s best higher education to anyone, anywhere, in any country is enormous. It is an example of taking an innovation to scale on perhaps the most extreme. With the click of a mouse the entire world can learn together easily.

Except, it is a lie.

Writing as a guest on the Worldwise blog as part of the Chronicle of Higher Education, professor of writing and rhetoric Ghanashyam Sharma  puts truth to the lie of the modern online education movement’s hype. In an article called A MOOC Delusion: Why Visions to Educate the World Are Absurd, Sharma illustrates the faulty thinking that underpins much of the MOOC enthusiasm for transforming global education. His critique is less about MOOC’s as an educational vehicle in itself, but the global, culture-free scale at which they seek to operate.

There is a dire need for some healthy skepticism among educators about the idea that MOOCs are a wonderful means to go global in order to do good. For our desire to educate the whole world from the convenience of our laptops to be translated into any meaningful effect, we need more research about how students learn in massive open online platforms, and a better understanding of how students from different academic, cultural, social, and national backgrounds fare in such spaces.

Education at Scale

Drawing on his own personal experience with teaching in different contexts, beginning with Nepal and then moving to the University of Louisville and now SUNY Stony Brook, Sharma points to the subtleties in cultural learning styles that did not translate from space to space. He speaks of enormous challenges and, fortunately, the opportunity to meet these with resources to aid him in adapting his teaching from context to context. In doing so, he points to the myth that the global classroom will be filled with people all learning the same way, from a compatible perspective, and using the same language. Even simple differences in the way he presents as a teacher can change the manner in which students learn – and those differences are rooted in culture.

As a teacher myself, I know much of what he writes. Even a ‘simple’ face-to-face graduate course presents considerable pedagogical challenges for me. As an instructor  in public health I need to consider things like:

  • Disciplinary background. Each discipline has communication ‘sub-cultures’ and traditions that differ. A student with a sociology background might be used to rhetoric while one from the basic sciences may be used to communicating through technical reports. Each discipline also uses language in different ways, with terms unique to that discipline.
  • Cultural backgrounds. Students from around the world will present differently in the way they approach the material, the presentation of arguments, and the level of participation in class. Even within a narrow band of cultural contexts that present within my classroom (usually there is perhaps 10-20 per cent of students are international) I am always amazed at the nuances that play out in the classroom. Race, gender and language all add additional complex layers that would require volumes to unpack here.
  • Personality and motivation. Students that are more reserved in class will experience each lesson differently than those who are outgoing. Personality and motivation change the type of discussions learners engage in and shape their interactions with other students. This is not to say that one personality style is better than the other, but whether you’re introverted or extraverted, confident, clear spoken or highly social shapes the classroom. Face-to-face encounters allows for some modulation of these effects to encourage participation with those of different needs.
  • Literacy. Whether it prose, numeracy, or discipline-specific language use, the literacy of students is tested when they have to take in material — whether through lecture, small groups, video, audio or text — and convey arguments.

These are constraints (and opportunities) in most modern universities, whether the course is delivered face-to-face or online. The amount of effort to engage a room full of eager learners is enormous if I attend to these various issues. As an educator, it is effort that I believe makes for a good experience for everyone and is a joyous part of the job. But I am speaking to a room of about 25 graduate students at one of the best universities in the world who are all present and sharing the same environmental context even if it is as a visiting student.

What happens when we are teaching to a ‘room’ of 150,000 people from all over the world?

Understanding scale

I do believe that we can learn much from online courses and that much can be conveyed via the MOOC that brings content to the world in ways that are appropriate and useful for a general audience. I am not against the MOOC, but I do think we need to carefully consider what it means to take education to scale and ask some deeper questions about what the learning experience is intended to achieve.

We need to design our educational experience using the same principles or design thinking we would apply to any other service of value.

The lie is in believing true education of a global audience in the rich way that a university or college intends is not only possible, but appropriate. What is being lost in the effort to make a common experience among a global classroom? Are we just sending out information or are we creating a learning environment? To what degree does the MOOC serve this purpose well. That is what a designer might start with as they move to asking how might we bring learning from one context to another.

Ghanashyam Sharma points to the folly in those who think the jump from culture to culture can be easily made when teaching the world at one time. The MOOC offers enormous potential to re-shape the way people learn and promoting access to content and expertise that many in this world could have only dreamed of years ago. But the naivete that such learning can be done in an monocultural way without losing something special about the context in which people learn and use what they learn might lead to some expensive lessons.

Sharma is right to ask for more research. Thomas Friedman is right to inspire us to think of creative ways to bring the wealth of the west’s educational treasures to the globe. The key is to figure out what parts of this global vision are real or possible, which are illusions, and which are delusions.

Unlike the auditorium at the Temple of Apollo at Delphi (pictured above) the audience for what we teach now are not from one place, time and shared cultural perspective; they represent the world. Without an understanding of what scale means in education we might be producing more ignorance than knowledge.

Photo: Cameron Norman

education & learning

Much Ado About MOOC

Leading? Learning? Both? Neither?

It’s fair to imagine that one of the 2013 ‘words of the year‘ will be MOOC (which is not really a word, but an acronym that stands for Massive Open Online Course). It seems that everywhere you look in the higher education and professional development space we are seeing MOOC’s talked about and debated.

HBR editor Eric Hellwig, writing for the HBR blog, reported on a recent panel on MOOC’s held at the recent World Economic Forum in Davos with leaders like Bill Gates, Peter Thiel and Larry Summers. His report reflects the exuberance of the MOOC and the techno-deterministic spirit of much of the discourse on these tools:

The advent of massively open online classes (MOOCs) is the single most important technological development of the millennium so far. I say this for two main reasons. First, for the enormously transformative impact MOOCs can have on literally billions of people in the world. Second, for the equally disruptive effect MOOCs will inevitably have on the global education industry.

One of the panelists was Stanford professor Daphne Coller, the co-founder of Coursera, one of the largest MOOC providers offering more than 200 courses to millions of students worldwide. Coller has convinced top faculty at leading universities to provide high quality digital courses through Coursera for free and the result has surprised her.

We’re at 2.4 million students now. The biggest lesson I’ve learned on this is I underestimated the amount of impact this would have around the world. I really didn’t envision this scale and this impact this quickly.

Disrupting Education

Of these panelists, Peter Thiel may be the most controversial. He has spoken at length about the need to revamp education and sees technology and platforms like Facebook as a means to do it. (It’s worth noting that Theil was also an early investor in Facebook). He points to the multiple roles that education plays well beyond learning and suggests that when we go beyond that goal we start creating false economies of value within higher education:

You have to ask yourself, ‘What is the nature of education as a good?’ Ideally you want it to be learning. But it also functions as insurance. Parents will pay a lot of money for insurance against cracks in our society. Education as insurance has something to be said because it connects to the economy. You know computer science, you can get a job. But education also functions as a tournament. You do well if you go to a top school but for everyone else the diploma is a dunce hat in disguise. People need to understand what they’re trying to do? Is it insurance? A tournament? Learning?

Among Thiel’s biggest concerns is with the current educational system’s ability to support the kind of innovative thinking needed to make technological and scientific breakthroughs. So steadfast is he in the belief that some of the best minds are rotting in traditional classrooms that he founded the Thiel Fellowship, a scholarship fund to support promising young entrepreneurs in dropping out of school to pursue their ambitions of making social impact.

Thiel is disrupting education by taking learning away from the educational institutions charged with providing it. MOOC providers are seeking to develop a business model that puts them in the drivers seat of education and learning, drawing potential revenues away from traditional educational institutions. This will no doubt add to the pressures that universities and colleges are already facing as they rationalize ever more of what they do.

Education For All, Learning For Whom?

Free online learning of the calibre provided by Stanford University, Caltech, Harvard University, University of Toronto, MIT, and the Santa Fe Institute  for anyone, anywhere sounds like a dream come true.

In some ways it is. In others, it’s an illusion.

This comes back to the metrics we use in evaluating the impact of education and asking what its point is in the first place. What do we mean by learning and are we serious about it?

It’s been suggested that less than 10 per cent of those enrolled in a MOOC complete it. And of this 10 per cent, it isn’t clear what they learn, how well they learn it, and what kind of application (if any) that content is made to issues away from the course. Online courses with video tutorials, self-organized learning and largely uni-directional teaching bring together the best of former teaching methods like instructional TV, self-help, and classroom lectures.

The problem is that this ‘best’ isn’t particularly effective. A 2000 meta-analysis of distance instructional methods found:

There does not appear to be a difference in achievement between distance and traditional learners. Of the ten instructional features that were analyzed, only three had an impact on student achievement. These three features were type of interaction available during a broadcast, type of course, and type of remote site. There was an insufficient number of studies to ascertain whether or not the education level of the distance learners effected their achievement in the course (Machtmes & Asher, 2000).

While this review was done before widespread Internet use, the methods included reflected the same list above with one- or two-way audio and video. The studies were also done on programs that were designed for credit, not voluntary non-credit courses. Research on motivation will show that optional programs are far less likely to engender behavioural shifts than those that are mandated.

So who then is benefitting from MOOC’s? We don’t yet know, but it is likely those with time to attend to the content, high levels of intrinsic motivation (< PDF), the technological tools to succeed, and the environment that is ready to support integration of content into practice. That’s a tall order.

We are in the early days of MOOC’s and its too soon to tell how successful they will be. However, theoretically there is relatively little reason to expect that they will produce the kind of results worthy of hyperbole — at east not with those already accustomed to alternatives. To offer a MOOC from a world-class university to a learner somewhere in the world where education is but a distant dream achieves a great deal. But to transfer MOOC’s to replace more interactive and engaging methods — usually face-to-face — and expect great learning is a bit implausible.

Yet, with what we are offering now to students in the form of large classes, disconnected curricula, and didactic instruction MOOC’s offer an attractive option. What it loses is the experience of learning that is not packaged in a class. This means a change to campus life, the informal and serendipitous learning that comes from being in the same physical space interacting with each other, and may seriously limit the use of thinking and creative tools that design thinking and applied creativity demand. (for a detailed look at MOOC’s and the modern university check out Nathan Harden’s essay in the American Interest).

There is much ado about MOOC’s, but is this a Shakespearian tragedy in the making for learners?

Photo credit: iStockPhoto used under license.

education & learning

The Quality Metric in Education

Image

What goes on the pedestal of learning?

What is quality when we speak of learning? In this third post in series on education and evaluation metrics the issue of quality is within graduate and professional education is explored with more questions than answers about the very nature of learning itself.

But what does learning really mean and do we set the system up to adequately assess whether people do it or not and whether that has any positive impact on what they do in their practice.

What do you mean when you say learning?

The late psychologist Seymour Sarason asked the above question with the aim of provoking discussion and reflection on the nature and possible outcomes of educational reform. Far from being glib, Sarason felt this question exposed the slippery nature of the concept of learning as used in the context of educational programming and policy. It’s a worthwhile question when considering the value of university and professional education programming. What do we mean when we say learners are learning?

The answer to this question exposes the assumptions behind the efforts to provide quality educational experiences to those we call learners. To be a learner one must learn…something.

The Oxford English Dictionary defines learning this way:

learning |ˈlərniNG|

noun

the acquisition of knowledge or skills through experience, practice, or study, or by being taught: these children experienced difficulties in learning | [ as modifier ] : an important learning process.

• knowledge acquired in this way: I liked to parade my learning in front of my sisters.

ORIGIN Old English leornung (see learn,-ing1) .

This might sufficiently answer Dr Sarason except there is no sense of what the content is or whether that content is appropriate, sufficient, timely or well-supported with evidence (research or practice-based); the quality of learning.

Knowledge translation professionals know that learning through evidence is not achieved through a one-size-fits-all approach and that the match between what professionals need and what is available is rarely clean and simple (if it was, there would be little need for KT). The very premise of knowledge translation is that content itself is not enough and that sometimes it requires another process to help people learn from it. This content is also about what Larry Green argues: practice-based evidence is needed to get better evidence-based practice.

How do we know when learning is the answer (and what are the questions)?

If our metric of success in education is that those who engage in educational programming learn, how do we know whether what they have learned is of good quality? How do we know what is learned is sufficient or appropriately timed? Who determines what is appropriate and how is that tested? These are all questions pertaining to learning and the answers to them depend greatly on context. Yet, if context matters then the next question might be: what is the scope of this context and how are its parameters set?

Some might choose academic discipline as the boundary condition. To take learning itself as an example, how might we know if learning is a psychology problem or a sociology problem (or something else)? If it is a problem for the field of psychology, when does it become educational psychology, cognitive psychology, community psychology or one of the other subdisciplines looking at the brain, behaviour, or social organization? Successful learning through all of these lenses means something very different across conditions.

Yet, consider the last time you completed some form of assessment on your learning. Did you get asked about the context in which that learning took place? When you were asked questions about what you learned on your post-learning assessment:

  • Did it take into account the learning context of delivery, reception, use, and possible ways to scaffold knowledge to other things?
  • Did your learner evaluation form ask how you intended to use the material taught? Did you have an answer for that and might that answer change over time?
  • Did it ask if your experience of the learning event matched what the teachers and organized expected you to gain and did you know what that really was?
  • Did you know at the time of completing the evaluation whether what you were exposed to was relevant to the problems you needed to solve or would need to solve in the future?
  • Did you get asked if you were interested in the material presented and did that even matter?
  • Was there an assumption that the material you were exposed to could only be thought of in one way and did you know what that way was prior to the experience? If you didn’t think of the material in the way that the instructors intended did you just prove that the first of these two questions is problematic?

Years of work in post-secondary teaching and continuing professional education suggests to me that your answer to these questions was most likely “no”, except the very last one.

These many questions are not posed to antagonize educators (or “learners”, too) for there are no single or right answers to any of them. Rather, these are intended to extend Seymour Sarason’s question to the present day and put in the context of graduate and professional education at a time when both areas are being rethought and rationalized.

Learning to innovate (and being wrong)

A problem with the way much of our graduate and professional education is set up is that it presumes to have the answers to what learning is and seeks to deliver the content that fills a gap in knowledge within a very narrow interpretation. This is based on an assumption that what was relevant in the past is both still appropriate now and will be in the future unless we are speaking of a history lesson. However, innovation and discovery — and indeed learning itself — is based on failure, discomfort and not knowing the answers as much as building on what has come before us. There is no doubt that a certain base level of knowledge is required to do most professional and scientific work and that building a core is important, but it is far from sufficient.

The learning systems we’ve created for ourselves are based on a factory model of education, not for addressing complexity or dynamic systems like we find in most social worlds. We do not have a complex adaptive learning system in place, one that supports innovation (and the failures that produce new learning) because:

If you’re not prepared to be wrong, you’ll never come up with anything original. – Sir Ken Robinson, TED Talk 2006

The above quote comes from education advocate Sir Ken Robinson in a humorous and poignant TED talk delivered in 2006 and then built on further in a second talk in 2010. Robinson lays bare the assumptions behind much of our educational system and how it is structured. He also exposes the problem we face in advancing innovation (my choice of term) because we have designed a system that actively seeks to discourage wide swaths of learning that could support it, particularly with the arts.

Robinson points to the conditions of interdisciplinary learning and creativity that emerge when we free ourselves of the factory model of learning that much of our education is set up, “producing” educated people. If we are assessing learning and we go outside of our traditional disciplines how can we assess whether what we teach is “learned” if we have no standard to compare it to? Therein lies the rub with the current models and metrics.

If we are to innovate and create the evidence to support it we need to be wrong. That means creating educational experiences that allow students to be wrong and have that be right. If that is the case, then it means building an education system that draws on the past, but also creates possibilities for new knowledge and learning anchored in experimentation and transcends disciplines when necessary. It also means asking questions about what it means to learn and what quality means in the context of this experimental learning process.

If education is to transform itself and base that transformation on any form of evidence then getting the right metrics to evaluate these changes is imperative and quality of education might just need to be one of them.

Image: Shutterstock

education & learningevaluationinnovationscience & technologysystems thinking

The Knowledge Metric in Education

EducationHead

Higher education is asking itself some big questions and making substantive changes to the way it sees itself and produces value for society. Education is increasingly being rationalized, which calls into question the metrics that are being used to judge how resources should be allocated. In a previous post, I looked at the jobs metric. Now, it’s time to look at the knowledge metric.

Just the facts

Education writer and teacher Will Richardson‘s TED Book Why School is a provocative read for those connected to teaching or just interested in schooling. While it focuses largely on grade school, the issues are the same for universities and colleges particularly as the primary and secondary students of today are tomorrow’s graduate and professional learners. Richardson questions the role of the school as institution in its current form suggesting that if the status quo — one characterized an information delivery warehouse — is maintained there is little need for schools to exist at all. Yet, if the education within schools is focused on asking better questions and learning when to apply knowledge, not just what knowledge to apply, there is hope.

The current trend in school reform is towards Common Core Standards, which emphasizes specific forms of knowledge, ‘facts’ and asks that students be able to recall such content when required. Under this model, the role of the teacher is one of content manager and facilitator rather than guide or mentor and students are prepped for the tests of their knowledge (memory) rather than be asked to demonstrate its application to anything outside of the test.  It is this model that many proponents of online education embrace, because the Internet is a fabulous content delivery system and education can be literally programmed and delivered to students directly without the ‘noise’ that teachers introduce to the signal. Under this model, educational content can be delivered cheaply and widely to support uniform intended effects among learners.

Richardson argues for reforming schools to something closer to the alternative model that was advanced by educational reformer and philosopher John Dewey. Richardson writes:

“In this version of reform, schools and classrooms are seen as nodes in a much larger learning network that expands far beyond local walls. Students are encouraged to connect with others, and to collaborate and create with them on a global scale. It’s not “do your own work,” so much as “do work with others, and make it work that matters.” To paraphrase Tony Wagner, assessments focus less on what students know, and more on what they can do with what they know. And, as Dewey espoused, school is “real life,” not simply a place to take courses, earn grades, amass credits, and compete against others for recognition. There lies the tension.

This second path is simply not as easy to quantify as the first. Developing creativity, persistence, and the skills for patient problem solving, B.S.-detecting, and collaborating may now be more important than knowing the key dates and battles of the Civil War (after all, those answers are just a few taps on our phones away), but they’re all much more difficult to assign a score to. I’m not saying that a foundation of content knowledge isn’t still important. To communicate, function, and reason in the world, students need effective reading and writing skills, as well as a solid foundation in math, science, history, and more. But I’m convinced we must revise the overreaching coursework requirements we place on students — requirements created at a time of scarcity, by the way. And we desperately need to revisit the thinking we’ve developed around assessment that, as Harvard researcher Justin Reich says, “optimizes the measurable at the risk of neglecting the immeasurable.””

Facts vs Problems

The knowledge metric is flawed because it assumes that content solves problems. It also presumes that the curriculum teaches the right knowledge for the right problems and that those problems can be known in advance. Let’s look at these.

One need only look to cigarette smoking as an example of how knowledge alone doesn’t always solve or prevent problems. One would be hard pressed to find anyone over the age of five who doesn’t know that sticking a lit tube of anything in their mouth and sucking on it isn’t at least somewhat unhealthy (and most know it is very unhealthy). An individual’s knowledge of smoking’s effects on physical health may not be complete, but it is often sufficient to inform the decision to quit or not start the unhealthy habit. And yet, citizens in highly educated countries like the United States, Canada and the U.K. smoke more than 1000 cigarettes per year per capita (and over 2700 per capita in places like Russia). These are not countries lacking in information on tobacco and health.

Using students’ ability to recall content makes the presumption that what is contained in a curriculum is what they need to know when they leave their program of study (at least as a start). While it may be somewhat true for students in the humanities and languages, it becomes highly problematic for those in dynamic fields or emergent areas of practice, which is becoming more normal than rare. There is no doubt that a corpus of key concepts, skills and ‘facts’ is useful, but the manner in which this knowledge can and may be applied is changing dramatically. For example, social media has upended communications in ways that very few health professionals are trained for. Journalists are particularly aware of the role that Twitter and related tools have had on their profession.

It also presumes that the content itself is relatively static. Certainly, curriculum renewal is something that most learning institutions engage in, but the primacy of content itself as the driver of education also assumes that the foundation for that knowledge is solid and can be applied today in the manner it was applied yesterday. In dynamic conditions, that isn’t often true. Further, the relevance of knowledge is framed by the problems to which that knowledge is applied. Genetic information, for example, can be incredibly useful when framed against tests that have high confidence, predictability and value to people, yet without such a context it is largely useless to those non-scientists who have it.

Areas of social innovation — which are expanding dramatically in number and scope — illustrate the problem of changing context well. This is a field characterized by problems, problem solving and novelty (which is what innovation is all about). Standard approaches don’t apply easily or at all when we are faced with high levels of novelty. Thinking and re-thinking the problem frame, knowing what to find, where to find it, and the skills to integrate relevant knowledge together is something that is not captured in the knowledge metric. Yet, it is those skills that will lead innovation. Knowledge translation professionals know this and so do knowledge brokers.

Are we designing our educational programming to advance on the kind of design issues of problem framing, finding and solving that our world is facing? Or are we simply taking content that can be obtained through books, the Internet and other materials, repackaging it and creating expensive warehouses of information that take learners out of the world and out of context in the process?

I don’t suggest that universities and continuing education programs stop delivering content, but if knowledge is the metric by which they are judging their success then it behooves educational administrators and funders to justify why they can do it better than other tools. What made sense when content was a rare commodity makes little today when it is overflowing in abundance for little or no cost. Universities and post-graduate training programs have an opportunity to re-imagine education and have the tools to do it in a way that makes learning more powerful and relevant for the 21st century should they choose to change their metrics of success.

Designing education

How might we take the enormous talent trust that exists among university faculty (and their students) who co-locate (physically, virtually or in some combination) in a school and develop the skills to not only address problems of today, but prepare everyone for possible challenges in the future?

How might we integrate what we know, identify the knowledge we need, and create systems to take advantage of the talent and creativity of individuals to make universities, colleges, and post-professional training venues for innovation and inspiration rather than just content delivery vehicles?

What kind of metrics do we need to evaluate this kind of education should we choose to develop it?

These are questions whose answers might yield more learning than those focused on what knowledge students have when they graduate.

___

Image source: Shutterstock.