More MOOC-iness than necessary

I’ll get to my update on my experiences in World Music and it’s interesting twists and turns after the break.  First I wanted to comment on a few important MOOC-oriented presentations I came across this week.  First, there are these two TED talks, the first from Daphne Koller, who is the cofounder of Coursera:

The second is by Peter Norvig, who taught a MOOC on artificial intelligence at Stanford in 2011:

Last and far from least is a talk that George Siemens gave at EDUCAUSE recently called “MOOCs:  Open Online Courses as Levers for Change in Higher Education.”  The slides are below, but this link will take you to Siemens’ site and a link to his actual talk.

I don’t have a recommended order for looking at these talks and I also realize that watching them all is going to take more than an hour, but if you’re interested in the whole MOOC thing, I’d encourage spending the time.  For me, these three talks– and really the TED talks vs. Siemens– cover a lot of the possibilities, perils, and frustrations of the current “MOOC-olution” that’s going on in higher education right now.  A few highlights for me:

  • Koller begins by talking about the issues of access to higher education all over the world and relates the story of a stampede of people trying to get into the University of Johannesburg, an event that the New York Times reported as embodying the “broad crisis in South Africa’s overstretched higher education system.”  Coursera, Koller argues, is part of the solution.  That’s a noble sentiment and it went over well during Koller’s TED talk, I suspect because the audience is made up of a lot of people who could have also been characters in that South Park “Smug Alert” episode. (For those who don’t remember and/or non-fans:  this is where Kyle’s family has to move to San Francisco after his father buys a hybrid car; while in SanFran, the Broflovski’s befriend similar high-n-mighty smug folks who also enjoy the smell of their own farts).
  • And it’s also worth noting that Siemens mentions in his talk the encouraging signs he saw first hand of MOOCs being used by students in India.   But I have to wonder:  do those thousands of largely poor South Africans have the level of computer and internet access to take advantage of MOOCs?  And given the larger problems for poor blacks in South Africa (the NYTimes article mentions that the jobless rate among poor youths is 70%, for example), isn’t this a bit of a “let them eat cake” type of proposal?
  • The thing I find most surprising and even irritating about both Koller’s and Norvig’s videos is they make it sound as if they have “discovered” online teaching.  For example, Norvig makes a big deal about how it turns out that one of the ways to help students succeed in online classes is to have deadlines.  Koller makes a big deal out of one way to deal with the huge numbers in Coursera courses is to have peer evaluations.  (And more about the troubles with peer evaluation below).  It’s maddening, and as you see in the beginning of Siemens’ talk, he feels the same way.  To paraphrase/more or less quote him, “It’s as if they are discovering North America all over again.” Coursera et al, Siemens argues, have spent a lot of money on hiring a lot of programmers to get the infrastructure up and running, but they have clearly not paid a lot of attention to the well-developed thinking about online teaching.
  • And just to be clear:  online teaching is a) not particularly new, and b) it is not only  happening in proprietary schools like University of Phoenix or Kaplan.  I’m far from a “pioneer” in the field, but I’ve been teaching online for seven years, and I’ve always had online classes that included these radical pedagogical innovations called “deadlines” with students all working together on projects (sometimes even collaborating!) and “peer review” where students comment on each others’ writing drafts.  I’m not alone in this.  As Siemens points out, about one third of U.S. and Canadian college students have taken at least one class online, and these are at “traditional” universities.  That’s hardly undiscovered territory.
  • I think Siemens is correct in assessing why MOOCs have all of a sudden become a big deal:  money.  He say’s in the last 8 months, there’s been around $100 million invested in MOOCs by venture capitalists.  You put that much money into anything and there’s going to be at least a ripple.  You put that much money into higher education, which runs on the cheap as it is and which has been all about slashing budgets every which way to stop the rising costs of tuition and fees, and you’ve got more than a ripple.
  • Finally, I think Siemens draws a good contrast between his MOOC experiments and what Coursera (et al) are doing with MOOCs.  Siemens and his colleagues were all about demonstrating how knowledge creation is messy and social, about using relatively low-powered and open-source tools, about emphasizing the development of social relationships that can be fostered after the class, and Siemens describes how he ran his MOOCs “off the side of my desk.”  Coursera is all about delivering knowledge in a rather conventional “sage on the stage” lecture format, about using a mix of proprietary tools (like its course shell software) and more open media tools (like YouTube), also about social relationships inside and outside of the class, and Coursera is ultimately about a hope/dream for making a lot of money.  Again, Coursera’s founders stated goal of giving the world access to “the best” higher education is noble, but I guarantee that the investors who have put up the money have other goals.

Anyway, on to a few specifics about “World Music” after the break:

Week two was about Paul Simon’s album Graceland, which I can imagine is a topic close to Professor Carol Muller’s heart since she’s from South Africa and she was in college/graduate school in the 1980s just before the apartheid era ended, more or less.  Once again, the delivery format is bad public access TV-styled videos.  I don’t want to be too hard on Muller because she comes across as personable and engaging despite the non-existent production values, but clearly she could have used a little practice with the technology.  A couple of times, while trying to point at the slides behind her (probably on a green screen of some sort) but obviously in a monitor in front of her, she gets demonstrably confused as to which way she’s pointing– is that my right or my left?  Besides, with a topic like world music, is it too much to ask for some examples that are a bit more sophisticated than YouTube clips?

There’s also the argument she’s making about Graceland I don’t entirely agree with, where she more or less says Simon didn’t give his South African musicians enough credit.  That’s just a legitimate difference of opinion though.  There is once again the frankly insufferable group of graduate students “modeling” a conversation about the class by just talking with each other when, of course, the “discussion” all takes place with words in a row in discussion forums.

Speaking of which: I thought the discussion was a little more useful this time, though that might be more because I know more about Graceland than I do about Gregorian chant and the Rorogwela Lullaby (that was week 1).  The number of participants still makes anything approaching a “discussion” impossible.  Last week, I said it felt like watching a cooking show really intensely.  This week, it occurred to me this time that the discussion forums are a lot like following a trending topic on Twitter.  As I type this, one of the top topics is about the Curiosity Rover that landed last night.  There are some interesting tweets here and there, some pointless ones, and some funny ones, but it’s of course not a discussion.  How could it be?  In the time I typed those two sentences, 80 more tweets with #Curiosity appeared.

One thought/suggestion I have for the Coursera people (assuming they’re interested in my advice) is I’d divide the discussion up into groups of about 150-200 or so.  That’s still a lot of people of course and I’m not sure about what would be the best way to do that, but if the group size was more manageable, the level of discussion might be deeper.

But the big thing that I thought was interestingly problematic from week 2 was the peer review process on the short writing assignments.  As I mentioned in my previous update, there are short– two or three paragraph– writing assignments every week, and the way these are being rated/ranked is through a process of peer assessment based on a rudimentary rubric (PDF).

I handed my short writing in on time at the end of week one and then I logged in at the beginning of week two to see if I could figure out how this assessment was going to work.  I was given links to student responses and the rubric was easy enough to fill out, but there wasn’t any guidelines on how many of our fellow students we were supposed to review or when those reviews were due.  So I did two Monday morning and then figured I’d get back to it later in the week.  Well, first thing Tuesday morning, I found out that I would have gotten a 10 out of 10 score on my first short writing but because I didn’t finish the peer reviews on time, I was docked 20%.  WTF?!?! And then it turned out that a) I wasn’t the only one who was annoyed/confused at the deduction since this was never explained anywhere, and b) there was some sort of technical glitch in the Coursera software that caused this to happen in the first place.

In the spirit of treating this like a “real class” (sort of), I figured I’d complain to somebody about this to get it straightened out.  But the problem was there’s no contact information readily available for the professor or the teaching assistants– certainly not any email addresses, other than to “Listening to World Music Course Staff.”  I emailed that anonymous email address and posted in some of the discussion forums complaining about these glitches, and Professor Muller (who does post once in a while) did say she’d point the problem out to the technical people.  Notably, this week’s peer review guidelines are much clearer.

I don’t want to make too much of a big deal out of this because I know these things are all experimental, not really for credit, free, new, etc.  And I have nothing at all against the premise of peer evaluation for these projects, though in a “real class” setting I suspect there might be some FERPA objections, a lot of students expressed some anxiety at the idea of grading their peers, and I’d like to see a much more robust rubric/evaluation technology– something like ELI or what a couple of textbook publishers have been working on for a while now.

But I think this little bump points to one of the problems of scaling classes to tens of thousands.  A lot of people were kind of irritated by this in a class that is completely free and where the stakes are very low; imagine what would happen if students enrolled had to score a certain number to be potentially eligible to take a CLEP-styled test to get real credit.  Worse yet, imagine if someone had actually paid real money for this!

Putting up a bunch of content for tens of thousands of people is easy– that’s just a web site nowadays, or a textbook both in the present and in the not so distant past.  But scaling class size has its human interaction limits, which is why places like the University of Michigan or the University of Pennsylvania don’t have lecture hall classes with  10,000 students in them  (I assume).  Oh, part of the limitation is just brick-n-mortar, but that could be solved easily through close-circuit TV in a variety of different rooms or having students meet in the basketball arena or something.  No, the real limitation is even when a professor has a small staff of teaching assistants and even when the format is a “sage on the stage” lecture, it’s really hard for that professor to manage all the human interactions of that big of a class.

You can pass some of that work off to participants in the form of peer review, but you’re still always going to have various student successes and problems that will require a level of human interaction that just does not scale very well.  This is why class size has always been important, why teaching load has always been important, regardless of the delivery format.  It’s not about content; it’s about the interaction.

BTW, week three is “throat singing.”  Wish me luck.

8 thoughts on “More MOOC-iness than necessary”

  1. Canadian distance ed consultant/thinker Tony Bates has a really good post raising some important critiques of Koller’s TED talk, critiques similar to mine and Siemens. Here’s a quote:

    Third, and this is the most enraging part of the presentation for me, Daphne Koller talks as if she invented online learning, and that nothing was known beforehand about works and doesn’t work in online learning. So she has discovered that students learn better if they are active, so there are lots of tests and activities in the courses. It is better to break up monolithic one hour lectures into smaller, more digestible chunks. Both these strategies in fact date back to the UK Open University print packages forty years ago and it has been standard practice to incorporate such strategies in most online learning since it began on a serious scale 20 years ago.

  2. Steve, it is so useful to learn about what is going on in other Coursera MOOCs – I can see a lot of overlap between the problems in the World Music course with the problems in the Fantasy-Scifi course (the peer evaluation technical glitches were a real disaster last week for us also, and apparently there were serious technical glitches both last week and this week that prevented students from turning in their little essays)… I wrote up some notes here about the communication problems, very serious ones, that I see with the class in general: http://courserafantasy.blogspot.com/2012/08/thinking-about-communication.html
    I was motivated to take the class by the peer feedback process because it sounded very intriguing but based on the first week, my impression is that it is being badly implemented (not very good writing prompt to start with, no guidelines for feedback, no real focus on just how to improve people’s writing skills AND feedback skills – the assumption seems to be that we all have all the skills we need, which based on the first week seems clearly not to be true). I was going to wait another couple of weeks before formulating what I see as the good/bad of the peer review system but it was interesting to learn that it works somewhat different in the different classes. Anyway, thanks for your detailed notes!
    Is Coursera gathering any feedback from students at all in your class? In this class, no data is being collected at all – nothing about our backgrounds, our goals for the course. Without that data, I don’t see how they are going to be able to interpret the dropout rates … and surely there is going to be a very high dropout rate for this MOOC as for others. Because of a glitch (or maybe they don’t care) in the URLs for our essays, it was possible to see how many people turned in essays last week by poking around with the URL… if they have left the URL open to manipulation like that again, I will be curious to see how many essays are turned in this week (last week it was appx. 5000).

  3. “It’s not about content; it’s about the interaction.”

    That line sums it up nicely, Steve. And that is what I have been telling people (faculty, who are interested in teaching online as well as those strongly opposed to anything taught at a distance; administrators, who are looking for the great cost-effective savior for higher education; and students and parents who ask me my opinion on college X or university Y).

    There is wonderful teaching occurring at two- and four-year schools in face-to-face, hybrid and online courses. Just as there are horrible examples of teaching in face-to-face, hybrid and online courses. One common element in those which succeed is … wait for it … interaction!

    Interaction between students and instructor …
    Interaction between students and peers …
    Interaction between students and content …
    Interaction between students and interface (where appropriate) …

    This all begins with an involved/engaged/interested instructor.

  4. Great post, Steve, and as others have commented, it is really valuable to have publicly available the experience of people doing different MOOCs. Many thanks for doing this.

    I have one comment to add to yours on peer review. Once again, this is not new. There has been a good deal of research on what works and what doesn’t in peer review. From your report, it looks like the Coursera people haven’t done their homework on this – or more likely have decided to ignore the research that doesn’t support peer review without some form of direct monitoring by an experienced instructor.
    I have used peer review in online courses, mainly for project work, to identify students who coast and rely on the others in the group to do the work. For that it worked well, but the high correlation between peer review and instructor grading depends heavily on students knowing that the instructor is checking, otherwise you tend to get general grade inflation.

    I hope though that you continue to enjoy the course!

    1. That is true– that is, there’s no kind of “experienced instructor” interaction, and as Koller argues in her TED Talk, this kind of peer review is their solution to dealing with the huge numbers. She sites some kind of stat about how students are nearly as accurate as instructors in their evaluations, but I suspect those studies presume the interaction of a teacher of some sort and in a course of less than 10,000 people.

      But I do think that there are ways in which this “peer only” review work can work with minimal instructor involvement. They aren’t doing this, but there are ways to do some training and/or “norming” with students on their evaluations that are a little more robust than what’s here, and there are even some software tools that help evaluation the quality of evaluations– this is what I meant with my reference to ELI (I know those people) and some of the work that’s going on with textbook publishers in the world of composition and rhetoric.

      I’m liking the course, but this week’s post is going to be about relating to my own students’ experiences of getting easily behind!

Leave a Reply to Tony Bates Cancel reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.

This site uses Akismet to reduce spam. Learn how your comment data is processed.