I had a conversation with a student the other day that continues to bother me. I think for what are obvious reasons, I am not going to go into specific detail, but in summary, this person was unhappy with the earned grade on a project. Very unhappy. So I of course scheduled a meeting with this person, assuming/hoping we were going to mostly discuss strategies for a revision. Instead, this student essentially yelled at me. I was told the assignment was not clear and that the student (in this person’s opinion, of course) had indeed fulfilled the assignment, regardless of my judgment and grade.
Furthermore, I as admonished for not being able to describe how many points certain portions of the assignment were worth and for not having (what this student had in mind as) a clear grading rubric. How, this student wondered aloud and angrily, how could I possibly hope to teach anything absent a rubric that clearly describes what parts of the assignment are worth what points?
A few thoughts:
- This student was not without a point– that is, this person did suggest valid ways in which the assignment description was lacking. My main problem is that this assignment is relatively new and some of the problems I am experiencing this term– this is not the only student who missed part of the point of the project– were things I had not anticipated and which were not a problem previously. I know, kind of vague. Let’s just say I will have to add some language to the project that I thought was somewhat obvious, but you know what they say when you assume. Of course, there was a vigorous discussion on the class site (it’s an online class) about what exactly the assignment was about and I gave numerous examples of possibilities, and there was nothing to stop this student from asking for clarification or from running an idea past me. Students– particularly college students, particularly college students who are juniors and seniors– have a certain obligation and responsibility to ask questions about assignments they are unclear about. But that is perhaps beside the point.
- Another point this student had that I think is valid and that I see all the time is a discrepancy between peer review comments and my comments– in other words, in the peer review process, students say the draft is great, but my comments and grade disagree with that. This is because too often student comments are not detailed and not constructively critical (e.g. “Great job! I learned a lot from your essay! This is perfect in every way!”), and this is generally because students often don’t spend enough time in peer review, and/or also because students are way WAY too nice to each other, even in comparably anonymous online environments. I think I need to do more to get students to question these overly rosy comments (Derek had some good points about this, btw), but I am completely convinced that the main reason students are not as constructively critical as they could be is because they don’t want to offend anyone or make anyone sad. I think that if the peer review process was anonymous, then students would give more critical advice. But that’s an experiment/change for next term.
- Several years ago, a student who came into my office itchin’ for an argument from me would have gotten it. Just a few years ago, a student coming into my office to bully me into changing a grade (and I am not accusing this student of that–not exactly– and I am quite sure this student has a very different interpretation of things) probably would have been successful simply because I would have decided it was not worth it. Nowadays, when students come into my office and yell at me (not that this happens that often), I realize that the problem here is probably not mine.
- What I object to most is the appeal to grading rubrics and points, a strategy I have quite frankly seen from a number of students who are studying to be K-12 teachers who have had just enough exposure to an undergraduate methods course to be vaguely familiar with the terms “rubric” and “assessment.” As I told this student the other day, the use of rubrics is complex and debatable, the assignment and peer review process constitute a “rubric” of sorts, and I would be happy to debate the use of rubrics in writing courses– particularly formal rubrics, and particularly in advanced writing courses– with any one of my colleagues in Education and/or assessment. The same goes for points. Assigning points (or percentages, of course) to projects or parts of projects in a class focused on something as leaky, fluid and non-discrete as the writing class is a convenient fiction at best. Or at least they are convenient fictions when the points/rubrics are not forced upon educators from some sort of outside assessment force. The kinds of institutional/external assessments that happen in higher education (accreditation, for example) and the sort of fetishized testing and rubrics and assessments forced upon folks in elementary and secondary education are entirely different kind of convenient fictions.
- On the other hand, it never ceases to amaze me how magically powerful assigned points and rubrics and the like are. If I had only accompanied my grade and comments on the final project with a chart listing discreet elements worth a certain number of points, no matter how “made up” and dubious that chart might be, I am quite sure I would not have had the unpleasant conversation I had today. After all, how could rubrics and points possibly lie?
- I do have a simple point system that helps me keep track of grades and that helps students to see where they’re at in a course. The class as a whole is worth 1000 points, participation is worth 200 points, each major project is worth 150 points, etc. The number doesn’t matter, of course because fundamentally, it’s just a percentage system, and in my view, what really matters is the dialog I usually have with students about their grades, about revising, about improvement, etc.. I have literally taught courses in the past worth 1,000,000 points, and if I were better with numbers, I’d teach a course worth one point. The points don’t matter because the process (and the percentages, for that matter) doesn’t change.
- And yet, the power of the point in even my simplistic system is indisputable. I have conversations with students who are seven points away from a “B” for the term, and when I say to a student in a conference “see, the problem is you are seven points short, so there’s really nothing I can do here, you’re going to get a ‘C+,'” that student inevitably nods and says “yeah, I see you’re point.” Never once have I had a student stop to think a moment and speak the truth to me: “Yeah, but you can pretty much give me whatever grade you think you can justify, right?”
Making comment anonymous might help improve peer review, but I don’t think it’s the only way and as an experiment where that alone is the change, it may not be enough.
For the Website to this post, I’ve put up a link to a document on responding that gets at some things in peer review I find important, especially on balancing peer comments with my instructor comments.
But briefly:
I see the writing of comments has something that has to be learned. It takes most teachers lots of practice to find their stride in writing effective comments, and we do it for a career and do it way more often than students. So naturally students will need more time how to do it well.
If we consider a comment to be a micro-essay, then we should consider it revisable, something the writer can be asked to revise. So “Great Job” would become a detailed description, via revision, of why the reader thought the passage was great, how they’re defining great, and what about the passage really worked.
But doing that takes time. It means having students read closely; it means reading their peer review comments; it means asking them to write fewer comments to start (so it’s more possible to review their reviews), and it means teaching them how to revise, what to look for, how to articulate things.
It’s a shift in focus and work, but not one I mind making because the skills involved: careful reading, descriptive commenting that lead more readily to grounds for analysis and even judgment or a prescriptive suggestion (if that’s what you want to see), revising a comment, taking care to explain to a writer in a way that’s useful what is intended, writing for two or more audiences — the writer, the instructor, and perhaps other reviewers — is valuable and rhetorically situated stuff. It’s writing.
I don’t want to suggest that doing anonymous commenting is a cure-all for the problems of peer review, and I agree with pretty much everything you’re saying here, Nick. It’s an interesting argument to make the commenting/peer review “count” as a grade of some sort in and of itself– that is, assign a certain number of points (or whatever– I’m trying to problematize the point here) to the peer review process itself apart from the grade on the essay. At least I think that’s what you’re saying.
The problem– well, not problem, but issue/consideration/potential downside– is the labor issue in that it already takes me long enough to read/comment on everyone’s essays; now I have to read/review everyone’s comments on everyone’s essays too? I dunno, that’s a lot of work potentially. This labor issue is one of the things that makes the stuff that the WIDE folks were talking about at C&W at Purdue about their system of rating ratings so potentially interesting.
Having said that, I am still willing to stick to my claim that I think peer review would be better with anonymous comments. I haven’t quite figured out yet how I would qualitatively or quantitatively prove that, but I think I may have started to figure out how I might be able to make this work with Google docs.
And this also makes me think of taking this a step further and trying to “crowd source” the whole process of not just peer review but grading. But that’s more complicated than I am liable to do anytime soon.
I have seen an increasing request for rubrics lately as well. I don’t mind using formal rubrics for some assignments, but, as you note, sometimes they just don’t work, or they arbitrarily slice up assignments that are better viewed more holistically. As to peer review, I have the same problem. Comments are too nice or sparse, partly out of a desire not to offend, partly, I suspect, out of laziness, and partly, perhaps, because they don’t really know how to do peer review, or don’t feel authoritative enough to do serious review. Perhaps a combination of these issues, in the form of a rubric for peer review, might work. Tell them exactly what they should be looking at and reviewing, and how, and grade them on it (although I agree this might be too labor intensive).
Hello,
I am a bit of a lurker who feels compelled to respond to this post (are you sure you weren’t hanging out in my office last week? :-)).
These issues have come up in my FYC courses, among others, and I’ve tried a few things that sometimes help:
1) Partnered peer review: two people partner up to review another student’s work. The pair writes then writes the critique.
2) Peer review as a more-or-less formal *letter*: the change in genre sometimes allows students to get to substantive criticism (formerly perceived as “negative”) because they begin with a kind salutation, “Dear Friend,” which softens (?) the more negative message. Or course, I have to show them what a formal letter looks like.
3) Writer’s judgment of most helpful peer critique comments: when the writer turns in her/his final draft, s/he must accompany it with a reflection on the peer comment(s) that moved the draft forward to where it is now. Sometimes, I have had students submit these responses to their peers first, just so good feedback folks can find out that their comments mean something to someone else.
4) Student-generated rubrics: I’ve used this method with some good success. The conversation about “what counts” then gets right out in the open – with a writing class, this conversation matters to me. I shape the conversation, of course, because “Looks pretty” isn’t always worth as much as “Complex idea” to me . . . or to most people interested in college-level analysis and expression. We revise the rubric after the midterm portfolio.
5) Students judge their own work against a visual “rubric” (basically a ranking of their work against a chart with criteria or against their earlier work or against the work of their peers [anonymously distributed]). Something about the visual ranking/assessment seems to resonate with students.
None of these works all the time, with all classes or courses. But I keep trying to minimize the kinds of conversations you talk about above and to maximize the conversations about purpose/expression.
Finally, the hands-down most successful strategy for handling these kinds of students is having a peer-tutor assigned to a class. This student has taken my courses before, and he can express the *exact same* thing that I said, but he’s heard more often by some students. I have not had a peer tutor in an online course, but my colleagues have (and I’m willing to try it).
FWIW,
Julie
All of these are good ideas and thanks for sharing them, Julie. But one thing that I am coming back to as the semester wraps up (and also after I had other students in the same class tell me how great it was– whether that’s true or not, makes me feel good anyway) is that there really was nothing that could have been done to satisfy this person. In fact, the more I think about it, the more I think that none of all of the right rubric-y sorts of things and peer reviewers and everything else in the world would have mattered. Can never please everyone.