April is always the cruelest month in academia because it’s near (or, at EMU, is) the end of the semester, which means there are all kinds of last meetings, end of the school year celebrations and recognitions, planning for spring/summer teaching, etc., etc. So I’ve fallen behind in the English Composition I MOOC, though I did manage to throw together write an essay for peer review. Here’s an update on some of what’s been going on in the class, at least for me. It rambles on quite a bit in part because this post (and other posts, of course) are as much notes for future MOOC writing as they are anything else.
The first real assignment for the class was a critical review of Daniel Coyle’s “The Sweet Spot,” which is the first chapter of his book The Talent Code. More about this reading in a moment when I get to some stories of surfing the discussions, but my take on it is it is an okay, accessible, and fairly low-brow take on where expertise and talent comes from. It’s kind of an ironic choice in a way because part of Coyle’s point is that talent/expertise comes from “deep practice,” which isn’t exactly the kind of thing that is promoted in a MOOC.
One of the things that happened was a Google Hangout with Daniel Coyle talking about his book– here’s a link to the recording. To me, this is a good example of the things that are possible with MOOCs, but also the problems of scale and of “production values.” If I were to teach this selection from Coyle’s book in a version of first year writing at EMU and I asked Coyle to respond to student questions, he would certainly ignore me. But here, with 70,000 students (or however many– the number really doesn’t matter other than it is large) tuning in, Coyle is both flattered to show up to speak and he might even sell some books. Again, that’s one of the things that is potentially possible with a MOOC: like a textbook, it can give a large and diverse audience a communal experience.
But the problem here is this video is almost unwatchable. The questions asked seemed to have been on a first come/first served basis, and the results (at least from what I can tell about skipping around the video) are some frankly bad questions and a lot of rambling between Coyle and the questioners. It might have been better for Denise Comer to just sit down and interview Coyle herself in a studio someplace, maybe asking the questions that students asked.
Anyway, the videos for week two and three continued on, with more textbook-like lectures featuring the usual writing class suspects: writing a draft, integrating evidence, avoiding plagiarism, citing sources, giving feedback to others, etc., etc. Like I said before, solid and traditional first year composition advice, and, like I said before, kind of boring with the talking head thing. I’m not entirely sure what I would have done differently, but I think it’s evidence that small group pedagogy doesn’t graft well onto a massive group, and, as I said before, this might be an example of how a text that students can reference as they work might be more useful than a video. It’s not as if Denise hasn’t tried though. For example, week three features several workshops (again, via Google Hangouts) to demonstrate what a peer review group in the “real world” is supposed to be like. A good idea, but the problem is watching a group of people talk about their writing (as opposed to my writing or at least the writing of other people in my group) is just not very interesting to me.
Besides, I was super-duper busy, and this wasn’t exactly my first writing rodeo, so I went ahead and wrote my draft about Coyle and posted it for feedback without thinking about it a lot. The actual posting of my essay wasn’t all that easy to do either, and this is another thing that I’m noticing about this MOOC: I think Comer et al are pushing the capabilities of the Coursera interface, and that’s making it more difficult for me to find my way around the site. But I did manage to post my essay, eventually.
I procrastinated and/or was otherwise delayed (have I mentioned I’ve been busy busy busy?) with evaluating my peers, but I got around to it. I won’t say much about the peers whose work I reviewed because my take on the privacy issues for my blogging about my MOOCing has been that anything anyone could see if they signed up for the class or any feedback I receive is fair game, but it would be uncool for me to share the peers’ work I have been assigned to review. But I will say there were a lot of characteristics pretty typical of freshmen comp kind of writing with a heavy dose of what appears to me to non-native language issues. I rushed through the reviews in part because of the problems of MOOCs and peer evaluation– that is, I don’t know these people, there’s no quality control over my comments, I mostly just wanted to finish it, etc. But as I show below with the responses from my peers, I think that the questions and guidelines for the peer review were reasonably clear– certainly more detailed and clear than what I experienced in “Listening to World Music” last fall.
While I waited for responses from my peer evaluators and while I was trying to avoid my real work (you know you have too much to do and too little motivation/energy to do it when you use something like MOOC reading as a procrastination tool), I did take another stroll through the discussion forums. For the most part, still a waste of time, but I did notice a couple of things. First, there were a number of discussions in the forums where folks were incredibly dismissive of Coyle and his pop-journalism fluffiness. I even found the nearly 1000 page Cambridge Handbook of Expertise and Expert Performance recommended a couple of times as an alternative and better reading in a couple of places. I can only imagine how that would go over in a real freshman comp class, though I am kind of curious to see if I can check this out from the library for myself. And I came across what can only be called a “flame war” and/or troll-fest in the discussion in week three with the title “In deep shock.” Go check it out/search for it yourself if you’re interested, but there are about 50 comments from one guy who hates hates HATES Coyle. It’s not that this guy doesn’t have a point, but these critiques of Coyle are interesting to me because I think they might say something about these commentators’ assumptions about expertise and writing. Maybe folks are disagreeing with Coyle as much as they are because they flat-out reject his premise and they assume some people can write, others can’t, and that’s that. I’m not sure, but the idea that writing can be learned is one that a lot of my writing students have a hard time accepting.
Anyway, here is a “cut and paste” from the “overall evaluation/feedback” section from the peer review on my essay; the questions were part of the prompts, and the responses are peer 1, peer 2, and peer 3:
Where in the essay does the author show summary and understanding of Coyle’s chapter? Is that sufficient to convey Coyle’s main argument to readers who may not have read Coyle?
peer 1 → in the first few paragraphs author shows that understanding and is sufficient to explain.
peer 2 → In the third paragraph the author clearly shows an understanding of Coyle’s chapter. I think it is stated simply and it is also sufficient to convey Coyle’s main argument.
peer 3 → The author shows understanding of deep practice. However, the draft lacks of many other details. For instance, the concept of the “sweet spot” is not explained at all and could mislead the reader. It does not suffice to just train, according to Coyles argument, one has to train at the sweet spot in order to progress.Where does the author demonstrate understanding of key definitions of Coyle’s text, such as “deep practice,” “scaffolding,” or “chicken wire Harvards”?
peer 1 → “deep practice” is explained well, perhaps “scaffolding,” and “chicken wire Harvards” is not so clear to me.
peer 2 → The autor mentions “deep practice” in the third paragraph. He/She also mentions the importance of struggling (making mistakes) and practice in order to learn a skill.
peer 3 → The author only touches “deep practice” in the text but omits all the other concepts that were discussed. The discussion in the draft is quite superficial.Where does the author go beyond summary of the text to pose a question about Coyle’s text, raise a limitation about Coyle’s argument, or make some other point about Coyle’s article?
peer 1 → In last two paragraphs the author raises a limitation in coyle’s argument.
peer 2 → The author makes some questions at the end of the summary (last paragraph) I couldnt find any questions that implies a limitation to Coyle’s argument.
peer 3 → In the last paragraph.Summarize in a sentence or two what the writer is arguing, if you can. If you cannot, say what the writer might do to make the argument more clear.
peer 1 → writer is arguing that “deep pracitce” is important and how learning from small mistakes is possible.
peer 2 → The writer makes a relationship between Coyle’s argument and the assumption there is behind education. If people were naturally talented at anything then what would the point be of having teachers and courses.
peer 3 → The author /thinks/ this interesting and well-written essay discusses a core concept that is found in education in general.What evidence does the writer draw on to support or explicate his or her argument? Has the writer effectively integrated and cited quotes or evidence? If not, say what the writer might do to integrate and cite quotes or evidence more effectively.
peer 1 → writer has used quotes and integrated them well.
peer 2 → I think the writer has effectively integrated and cited quotes. He/she uses Coyle’s examples at some points. In the last paragraph the writer evaluates Coyle’s argument and also states his/her opinion.
peer 3 → None. The lack of evidence / arguing is a weakness in the current state of the draft.Do you see unconventional features in the writing (spelling, sentence structure, vocabulary, and so on) that interfere with reading? Identify in particular one of these features so the writer can focus on it for his or her revision.
peer 1 → no
peer 2 → Overall the text is very clear. I think he/she could add more ideas or evidence about what he/she argues. For instance he mentions something related to education and the assumption behind. He/she could explain a bit more about this idea.
peer 3 → An academic dispute should not about what you /think/ (cf. second paragraph). You should rather provide logical arguments supporting, questioning, or falsifying certain aspects. This is not about giving an opinion on the text. Also avoid informal writing, e.g., “seems quite clear”. This is something you can say to your friends but not write in such a text.What did you like best about this essay?
peer 1 → it is very clearly written and also challenges Coyles’ argument at the end.
peer 2 → I liked the fact that it was clear to read and the way the writer quoted some parts of the book without making it too long
peer 3 → It’s easy to read / follow.What did you learn about your own writing / your own project based on responding to this writer’s project?
peer 1 → i need to critically analyse the arguments which I have not done so myself.
peer 2 → There is always room for improvement =)
peer 3 → Nothing.
So, I think there are at least two patterns/things worth mentioning here. First, these are pretty negative reviews to me. I’m not going to claim that my essay was great or anything, but I kind of feel like all three of these reviewers– particularly peer 3– dumped on me pretty hard. It’s okay, I can take it, but this criticism is a lot more “direct” than what I typically see in my sections of first year writing. Maybe it’s because my peers seem to be beyond typical freshmen in their abilities and/or maybe it’s because they are so anonymous, I’m not sure.
Second, we have here the classic problem of peers offering conflicting advice. I demonstrate understanding, and yet I lack details in many other places. I explain “deep practice” well, or I don’t. I incorporate evidence well, or I don’t. I am apparently making an error in academic writing because I am not supposed to “think” but rather to use logical arguments, and I certainly should avoid “informal” writing with phrases like “seems quite clear.”
The problem of conflicting advice happens all the time, but there are at least two significant problems in this particular situation. First, I think my peers’ expectations for what should be in these projects are far beyond the level of what I think this MOOC is supposed to be, which is (more or less) first year composition. Maybe I’ve just taught too much freshman comp, but some of the comments– particularly from peer 3– seem to me to be geared toward a more advanced student. This is probably not that surprising because the students/participants in this MOOC are all over the place, but it is a huge problem of peer evaluation. I’ve seen this in my own teaching. I’ve taught sections of first year writing in the past that have included seniors who somehow managed to “forget” to take composition in their freshmen year, and that mix of freshmen and seniors is often volatile.
Second, while this kind of conflicting and/or otherwise not useful feedback is common and a “teachable moment” in a normal class, it’s hard to sort through in a class of thousands. Actually, this is a place where a Comer video might help, a sort of “general advice” lecture on how to sort through the advice of peers. Not surprisingly, the forums reveal a mixed reaction to the feedback, too: some posts praise the feedback, others are complaining about it, and others are asking what to do with it– that is, how to actually take the peer comments and apply them to a revision.
Anyway, I’ll press on. This week, the class shifts to an assignment on reading “visual texts” and I am hoping to get a little less busy as my own teaching starts to wrap up for the term.
Jessica Winck liked this on Facebook.
I don’t think you got dumped on Steve. Part of the problem w/ online text is the lack of audio/visual cushioning, but I didn’t read most of the peer comments as harsh. Clearly Peer 3 is professor or a scientist. Who else uses “falsification” correctly? And, just a clearly Peer 1 is ESL and not as experienced at writing academic English.
One aspect of your feedback that I also see in our online writing courses: peers answer the question but don’t talk to each other. F2F, conflicting comments would get a “waddaya mean he didn’t do [whatever]?” There would be a discussion of whether you did & where you did it. The issue would be clarified if not settled, revealing the set of values that the disparity between #1 & #3 peer suggests, and teaching everyone, not just you. That doesn’t seem to happen online very easily.
I’m not sure how I feel about all the “Where does” questions. I’m glad they aren’t binary “Does he” questions and not “What” questions, but they really want to be “how” and “why” questions. “Where does” points toward “how” & “why,” but it also allows unhelpful evasion as “in the last paragraph,” which should be followed up by quoting you and summarizing why the quote answers the “where does” question.
I’m looking forward to more blog on this subject.
The clarifying function in a f2f peer review is a good point, and actually, that’s something that could happen here too, it seems to me. So imagine a peer review process that begins anonymously like this but then it becomes a little group of us, so the writer could ask the reviewers follow-up questions and also the other reviewers could compare comments. That seems like something that could be facilitated electronically, doesn’t it?
Yes, Steve, it could. It has been in many situations. I haven’t seen what the peer review tool (is it a tool?) looks like in a MOOC, but I’m guessing from what you say that it ain’t possible to see each other’s comments or comments on the comments?
Thank you for posting this, Steve.
Ben Reynolds liked this on Facebook.
Kathryn Northcut liked this on Facebook.
Bonnie Lenore Kyburz liked this on Facebook.
sigh. Eli really does do some things to fix a lot of this. I guess I won’t go on about it any more here.