MOOC Week five, and the peer review turns

I’m wrapping up week five of the World Music MOOC, and I have to say it’s starting to drag a little.  This week was about Australian Aboriginal music, though it was another week that had very little to do with music and more to deal with the politics of oppression against indigenous peoples.  I understand the obvious relevance for this being a part of the discussion of world music, but it’s all starting to feel more and more like I went to a music class and an anthropology/sociology class decided to barge in and taket things over.

I continue to be less than blown away by the quality of the presentation of class materials.   Just a simple example of what I mean about the lectures:  in the introduction to this week’s unit, Carol Muller gets the dates of when Australia was first “discovered” by Cook mixed up– that is, she says 1788, which was the year the British set up a penal colony in Australia, and not 1770, which is when Cook first landed in Australia.  The video is interrupted and the correction is clumsily inserted, and there was even a quiz question about the error.  Now, it’s not a problem per se that Muller misspoke.  Lord knows I say lots of wrong stuff to my students.  But isn’t that a reason why these ought to be rehearsed and organized for the screen and not just a rehash of a in-class lecture?  Isn’t this one of the benefits of recorded materials in the first place?

So I’m kind of getting bored here.  If I weren’t doing this thing for other academic purposes and future writing projects, I’d probably “drop out.”  This brings me to this Chronicle of Higher Ed commentary from Kevin Carey, “The MOOC-Led Meritocracy.”  Carey argues that the enormous drop-out rate in MOOCs is not only not a problem, but rather it allows MOOCs to operate as a meritocracy.  A quote:

That meritocracy will serve as a powerful mechanism for signaling quality to an uncertain labor market. Traditional colleges rely mostly on generalized institutional reputations and, in a minority of cases, admissions selectivity to demonstrate what graduates know and can do. The opacity of most collegiate learning processes (see again, lack of standards) and the eroding force of grade inflation have left little other useful information.

MOOC credentials, by contrast, will signal achievement selectivity. Instead of running a tournament to decide who gets to take the class and very likely get an A-minus or A, they’re running tournaments to decide who did best in the class. That’s why people are already resorting to plagiarism in MOOC courses. That’s troublesome, although perhaps not distinctly so, given that the antiplagiarism software that will presumably be deployed in defense was developed in response to widespread cheating in traditional higher ed.

In a sense, this was what college was like when I was a student nearly 30 years ago.  Most people who were in college 20 or more years ago can recall some kind of moment where the high drop-out rate was touted as a sign of rigor and that the earned college degree separated you from those drop-outs.  This is the classic “look to your left, look to your right, because one of you won’t be here at the end of this first year” spiel.  I don’t recall hearing that speech directly, but the University of Iowa at the time did have a reputation for being a fairly easy school to get into but not that easy to graduate from.

This has all changed dramatically and now one of the key markers of a successful and “good” university is its graduation rate.  There are lots of reasons for this change, but one of the reasons is a direct response to cost:  if you’re spending $40K plus a year for attending some quasi-fancy school (and I am aware that dollar figure is actually low for many fancy schools), you’d damn well better be able to graduate in a reasonable amount of time.

So in theory, Carey has a point:  who cares if the drop-out rates from MOOCs are super-high if that means that the best and brightest are able to make it through these free courses, separating them from the many drop-outs in their wake?  Maybe that could be something that employers could look at as a sign of success.  But right now in practice, there are no standards governing this meritocracy and I don’t think some crappy plagiarism software is going to make these problems go away.

And that brings me to peer review.

As I mentioned in my previous post, I decided to take a more slip-shot/half-assed approach to the writing assignment as an experiment to see how the peer review would go.  The first paragraph of my short essay was earnest and connected to the point of the assignment.  The second paragraph began with a somewhat related tangent about Disney and “free culture,” but then I dropped in three or so sentences from this blog post— that is, something completely out of nowhere, sort of plagiarized, though I am plagiarizing myself.  I concluded with a paragraph that was back on topic but pretty hastily thrown-together.  In short, it’s a pretty bad, semi-plagiarized, dubious at best essay.

My grade?  8.5 out of 10, which isn’t too bad considering that I received some 8s for because I didn’t do the peer review.

Here is the optional written peer feedback I received, and I think this speaks volumes about the “diversity” of earnestness, seriousness, and just general abilities of my fellow students:

student1 → Well written, forceful, clearly articulated essay. I loved your first paragraph–it’s a very reasonable dissent on the definition of free culture, substantiated with specific examples. Your second paragraph was, however, highly problematic: it seemed like you’re missing some citations here. Your introduction of Koller and Norvig’s videos not only came out of nowhere (in the middle of what I thought was going to be a great discussion on cultural appropriation by the Disney company), but there was no citation for them, so I couldn’t really see what you were referring to. You also make a reference to the troubles of peer evaluations “below” and to a Siemens video, but there was no context or follow-up for either. Because of this disjointed incoherence, I did a quick google search of those sentences that didn’t seem appropriate and found http://stevendkrause.com/2012/08/06/more-moocines/ . I’ll give you the benefit of the doubt that this is your website, but those sentences just don’t belong in this essay. Finally, I thought your third paragraph was a bit lacking. You make two huge claims (on sampling and individual ownership), without any support for either of them. For instance, I disagree that pygmies don’t have an understanding of individual ownership; they may be more egalitarian, but to say that they don’t understand this fundamental concept does require more support than you provide. In either case, this should have been the meat of your essay–tying in free culture with what you’ve learned this past week, but you fall short. I still gave you a pretty high score, only marking off for “relevance to larger course themes,” but I thought you could have taken this essay further than you did.

student3 → Good work. Some chill out music for you http://youtu.be/ccqY-R_M2nY

On the one hand, student1 is completely earnest and spot-on in the review, even going so far as to do the work of finding where that “out of nowhere” quote came from.  I think my last paragraph was better than this reviewer thinks, and I also would have given me a pretty low score on my essay.  But beyond that, I’m guessing that student1 is another teacher/grad student/professor signed up for the class.  It seems like this isn’t the first piece of student writing they’ve ever seen.

Student3, on the other hand, obviously doesn’t have a lot invested here.  This person just wants to share a tune from a techno/dance/remix one person band who goes by “Bomb the Bass.”  I appreciate student3’s contribution because I had not previously heard of Bomb the Bass, but it isn’t exactly, um, constructive in terms of my writing.

So we see a couple of the problems here. The different levels of interaction are way out of whack, one student hardly engaged and another far too engaged.  And for all I know, student3 thought he/she was doing everything expected in the peer review process and sharing some “chill out music” with me was something this person thought was key.  It’s all anonymous, which can work if there is some accountability, but without that– again, I go back to the need of a “review of reviewers” function– it can’t.  What motivation does student3 or anyone else have for working too hard at these critiques?  Even student1– who obviously put a lot of work and thought into the process– still ended up giving me a good grade.

I don’t have this handy right now, but somewhere in Clay Shirky’s Here Comes Everybody he talks about how the Los Angeles Times tried to have op-ed wiki but it had to be shut down almost immediately because it was just full of vandals.  As Shirky explains, there was no culture or motivation to stop it.  I don’t know if this quite the same, but right now, there really is no motivation for my classmates and me to give each other overly critical grades.  I mean, what difference does it make to me if anonymous student1 or anonymous student3 gets a 10 or a 7 on their project?

Again, content scales, but instruction and interaction doesn’t.

This entry was posted in MOOCs, Teaching, Technology. Bookmark the permalink.