E-Learning and Digital Cultures ends with a Meh #edcmooc

Back in January, before the E-learning and Digital Cultures MOOC got started, there was a bit of talk on the WPA-L and Tech-Rhet mailing lists about a bunch of folks from those communities signing up for the course. I know several did. I wonder how many others there finished; anyone?

Anyway, I finished it.  In brief, I was disappointed and it will definitely figure into my part of the roundtable about MOOCs I’m on for the Association of Teachers of Technical Writing. The title of my contribution for that is “MOOCs: the M is for ‘meh.'”

The fourth week of the course continued the post/transhumanism themes again with a mix of videos that were potentially useful conversation starters in a class like first year composition to essays from folks like Nick Bostrom and Katherine Hayles.  Kind of interesting– the discussion on Bostrom was especially good in the graduate class I’m teaching where I required students to take this MOOC– but I think all of us were ready to move away from the MOOC and back into our own class that actually is about “E-Learning.”

And I guess that speaks to my first main disappointment with #edcmooc:  It wasn’t really about “E-Learning” but about “digital cultures,” and a fairly simplistic view of digital culture at that (Alex Reid has a good post about some of those problems here).  If I had known that up front– that the class was going to have little to do with (to quote from the Coursera description of the course) “the ways in which we conduct education online” and “an invitation to view online educational practices” through popular culture– I might not have assigned this at all. Mind you, there were some interesting readings and videos; just not about what they had said the course was going to be about.

The second problem that really frustrated me by the end was an absence of teaching and leadership. Now, this was intentional on the part of the team teaching this course, clearly: they did not want to have a series of “talking head”/”sage on the stage” lectures because, as their manifesto makes clear, they are trying to question that idea of online education as just being delivered content from an expert to students. I get that. But when you have a “talking head” lecturer, at least then you have a common or “center” to grab on to as the discussions unfolds.  Here there was really not a there there.

In face to face classes, student-centered pedagogy works best (perhaps only) in small groups– say 100 tops, and certainly not in the thousands– and it still requires a teacher to be a facilitator, a guide, and a mediator. And “student-centeredness” works differently online.  In a normal face-to-face class, if the students take up a topic of discussion and are able to have a conversation with the teacher just moderating and not saying anything, then that’s usually a good teaching moment.  But in an online class, if the teacher doesn’t say anything at all, then that teacher is effectively “absent” and the students think that the teacher must not care about what’s going on in the class one way or the other.

The E-Learning and Digital Culture facilitators did participate in the discussion forums a fair amount and they did hold some synchronous meetings on Google Hangouts where the teaching team spoke and listeners posted questions on Twitter. But that wasn’t enough. As was the case with “Listening to World Music,” the discussion forums were white noise cacophony of tangents, repetitions, and irrelevant observations, with an occasional thoughtful comment thrown into the mix. Some of my grad students tried harder than others to participate in the discussion forums, but they all agreed that they were ultimately pretty useless.

Interestingly, the relative absence of teachers meant for me that “E-Learning and Digital Cultures” was less a “course” than it was a “collection” of readings and media on the topic, and a problematically diverse collection at that. Each week’s content included several short videos that perfectly appropriate for generating discussion in a freshmen level class, but there were also several complex essays and video lectures on technological determinism and trans/post-humanism that stumped my graduate students. That mix of accessibility of texts might be interesting as an anthology but not for a course much in the same way that we tend not to have college courses that include both freshmen and graduate students.

My students and I aren’t the only ones who thought this.  Via the EDC MOOC News blog I came across this post by Sandra K. Milligan, “Better than a Tarantino movie: raw peer assessment in #edcmooc.” After discussing the peer assessment process (which I’ll get to in a second) and the class as a whole, she writes:

In retrospect, it might have been better to market  the course as an ‘online extended conference’ or ‘a discussion group’ or, as Hamish Macleod, one of the team members wryly put it in the first hangout, a good old 1970s-style ‘happening’. As a course it was a great happening.  How to assess a happening, or how to structure a happening into a course are the questions I would turn to for next time.

An online conference, a discussion group, a happening, etc.: that’s great.  These are all good and valuable things to be sure, and maybe that’s all that we really need from something like #edcmooc.

But as I’ve said before about such learning opportunities, so what?  Maybe the MOOC professors can get some cool sneakers out of the deal, but that ain’t education.

Which brings me to the last week of the class was all about the “Digital Artefact” assignment.  We knew this was coming from the beginning of the class: students were asked to make “a digital artefact which expresses, for you, something important about one or more of the themes we have covered during the course. This artefact should be published somewhere on the web which is publicly accessible.”  The tools they suggested using were all readily available open source web 2.0 things like Prezi, Storify, Wordle, and Xtranormal.

Here’s a link to what I did, a Wordle about the discussion about #edcmooc that took place in my graduate course.  Most of my students posted a link to their projects in this discussion on the class web site or on their blogs.

For me, this assignment has two basic problems.  First, the bar they have set for these projects is very very low– a few Wordle images, some image altered to highlight the transhumanism thing, a short video, etc., etc.  That’s it. It would have been pretty easy to skim the titles and descriptions of the readings and videos for the course and then successfully complete the assignment all in a total of about 10 or 15 minutes.  I suppose that’s fine and it’s good to be inclusive and all, but keep in mind that the ultimate goal of Coursera MOOCs like this one is to give actual college credit.  Can you imagine actually getting credit for a course at EMU– even a 1 credit course– for completing this course and this assignment? If I was teaching sixth grade I would ask more from my students in five weeks.

Second, the assessment guidelines and process were very very thin.  Granted, there was more description and discussion of the assessment criteria here than there was in “Listening to World Music,” but that’s not saying much.  Here are the criteria we were supposed to consider in our evaluation:

The artefact addresses one or more themes for the course
The artefact suggests that the author understands at least one key concept from the course
The artefact has something to say about digital education
The choice of media is appropriate for the message
The artefact stimulates a reaction in you, as its audience, e.g. emotion, thinking, action

And then the grade was either a 0 (“does not achieve this, or achieve it only minimally”), 1 (“achieves this in part”), or 2 (“achieves this fully or almost fully”).  One grade for all five criteria, which actually begs a problem of pronouns here: what is “this” in the grading language in relation to the five different criteria? 

That is actually a problem that comes up in my peer’s response to my artefact:

peer 1 → Your artefact it’s a very nice selection for concepts for the course. But i can’t see a relationship with “Utopias & Dystopias” or “Being Human”. It’s a nice composition about Digital education, but you only construct a text. Images, videos or anymore media resources it’s absent. The artefect no stimulate my reaction o reasoning.
peer 2 → You addressed the theme of digital education and gave your opinion on what you think is the value of digital education. You creation of a woordle from a discussion board about digital education I thought was a good idea. You message regarding digital education i agreed with

So note here that “peer 1” doesn’t think I fulfilled the assignment because the “this” I didn’t talk about included the utopia/dystopia/transhuman stuff, while “peer 2” seems to have no problem with “this.”

peer 3 →

peer 4 → 1_ The artefact adresses to all the different points worked in the course. It’s a summarizing in words. 2_ The artefact suggests that the author has understood the contents and he has done a reflecyion writing some thoughs. 3_ The artefact is a summarizing of the digital cultures and e-learining, it’s done by key words, so they summarize the concenpots worked. 4_ A Wordle is perfect for showing the most important words of a content, and teh author has included the best keywords. 5_ The artefact stimulates a reaction in me becasue evryword has soemthing to say about e.learning and digital cultures. It’s the best way to cerate a debate. The author has done it withhis students in the blog created parallel to the MOOC.

And here we see “peer 4” has gone through each of the criteria and said how my project achieves all of “this.”  Then there was an “overall impression” response section:

peer 1 → Your artefact it’s a very nice selection for concepts for the course. But i can’t see a relationship with “Utopias & Dystopias” or “Being Human”. It’s a nice composition about Digital education, but you only construct a text. Images, videos or anymore media resources it’s absent. The artefect no stimulate my reaction o reasoning.
peer 2 → You have made an interesting digital artifact by using the comments from a previous discussion page to create an woordle. Your thoughts on that the mooc or moocs in general are not credit worthy I would agree with, however i feel you could have provide a suggestion into how moocs can be turned into a credit worthy course or discussed the recent mooc credit accreditation that has happened and what your thoughts are on this.
peer 3 → The artefact addressed many topics of the course, it has something to say about digital education. The artefact stimulates a reaction, which is adequate to the course content. It is very nice picture and I like it.
peer 4 → I think that teh author has used the copurse to create reactions in his students. SO, he has learnt about teh course and about teh discussions in his blog http://engl516.stevendkrause.com/2013/02/25/edcmooc-and-the-digital-artefact-and-beginning-to-debrief-on-the-mooc/#more-1292 whre you can see all the comments.

By the way, I earned a 1.5, I think because that jerk “peer 1” probably dinged me on the grade with a 0 or some such thing.

To sum up: some interesting readings and ideas, but no real feeling for me or my students of a “course” or even a “community” here, useless discussion forums, and a joke of an assignment with a kind of sketchy evaluation process. Meh.

Not that anyone from #edcmooc or Coursera are reading this, but I’ll mention four things that I think could make these courses happenings a lot more useful:

  • Break participants into groups.  As people sign up for the MOOC, put them in groups of some size– say 100, but the actual optimal number is something that would take some research and would certainly depend on the course.  Make it so participants aren’t locked into that group– that is, they can see the discussion going on in other places if that’s what they want to do– but give participants a “home” where they have a chance to actually connect with others. Surly doing this is an easy to solve software issue.
  • Make participation “count.”  Count posts, count words, and let other peers rate the quality of those posts.  Have a certain score necessary for certification.
  • Have an actual and ongoing discussion about assessment.  I think this would be useful for even MOOCs that are more test-driven, but they’re essential for things like #edcmooc.  I mean, if students don’t have a chance to react to the assessment on this artifact with a revision or a new project, there’s not much of a chance for the student to do anything differently–aka “learn” something.
  • Hold evaluators accountable.  This is where software like Eli from my friends and colleagues at Michigan State comes in.  I’m not saying this is the only solution and I’m not trying to sell any software here, but something that makes the act of peer evaluation the mark of success (or not) in the process would make this much more credible.

46 Replies to “E-Learning and Digital Cultures ends with a Meh #edcmooc”

  1. Steve, I think these are excellent points. You are inspiring me to take a MOOC. I have taken traditional online courses and found them useful.

    My take on MOOCs has been, “How can it work at such a massive scale? Is it really just a book but online?”

    Some of what you say lead me to conclude that MOOCs are best thought of as content repositories vs. interaction environments. Any reactions?

    1. I’ve thought for a while now that so far, what MOOCs are more or less quasi-interactive textbooks, and George Siemens and others have talked about MOOCs as a “platform” for interaction. I like the description of it as a “happening” too. These things are all fine and good and clearly have an inherent value. You can learn stuff and they can be fun for sure.

      But to say that a platform or a happening or a textbook is equal to a college experience that leads to some kind of credit akin to what happens in the courses that we teach is a problem. I think the “Massive” is part of the problem, though not the only one.

      1. I’ve been involved in online happenings, and my take is that the “mass” part can work well for improving content quality (but it needs editing oversight). Like you, I’m pretty unconvinced of the approach for course credit. I think a real test would be to ask the advocates if they would turn over what they do to someone with pure MOOC training or with a large proportion of their academic experience coming from MOOC training.

        Things like Khan Academy I have a lot more comfort with because the model is built around using the videos essentially as textbooks with problem solving to occur in class. Well executed, it can mirror the ideal of the case teaching method.

  2. First, you make an excellent point about the discussion boards. I think some careful thought needs to be given to the purpose and intent of the discussion boards. I’m not convinced, however, that tracking participation in discussion boards is necessarily a proxy for learning. We probably would have ended up with more bogged down repetition in the discussion boards if that was the case. I like the group idea (perhaps assigning everyone a discussion board?). Or, some discussion boards may need to be closed to new participants? Personally, I got the most out of twitter and the blog feed as points of interaction. So, what I enjoyed about the course was the ability to pick and choose outlets for interactions with folks and content. My guess is that for many people, certification was not the point of this particular course. However, it would be for other courses. So the question for me is whether a MOOC typology is emerging with different assessment models depending on the MOOC?

    1. First, I think you are raising a good point about the typology of MOOCs. Maybe what will emerge is that there is some material/subjects that lend themselves to MOOCs that grant credit, and so far, those seem to be in fields that are scientific/technical in nature, fields where the conventional pedagogy is based on lecture delivery and quantitative testing. I’m thinking here of some of the courses that ACE has “recommended” for credit, courses in genetics, calculus, and something called “bioelectricity.” MOOCs like “Digital Culture” or “Listening to World Music” might stay in the realm of “learning for the sake of learning.”

      And by the way, maybe the “learning for the sake of learning” thing is enough. My parents do these Road Scholar trips about once or so a year. They have a great time, they learn stuff, and they are in no way interested in making those classes “count” toward a degree.

      1. Grade is probably the most problematic issue about the MOOC, but MOOC is not only about grade. On the other hand, “Learning for the sake of learning” is a good point. But in the professional world, certification is a issue that matters. How to improve it ? Aside your recomendation, i think teachers must evaluate a sample as a control, participants must do an autoevaluation, and peers must do an extra evaluation of the calification given by other students. With that information, i think an algorithm could be created which would give a better grade to the student.

        About the groups and the chaos, i disagree with you and Margret. I think we have to learn to form our own groups (if wanted) in the digital world of MOOC. In the real world, we are not aware of all the discussion, doubts, documents that our classroom fellows make or share. Why do we want to control them in the virtual world?

        I have made some comments about #edcmooc (in Spanish): http://kmbalach.blogspot.com/2013/03/edcmooc.html

    1. Hi, it’s with a great interest that i read your post,
      Please let me present my view; fisrtly MOOC is a great opportunity to learn in an organize environnement, i understand by your artefact the digital is not your first skill but if learn means get new ideas and use them in our interest, it is sure that i learned too much in this course.
      About grading, i think peers did a great job to asses the artefact and if the mean is used, it’s for to minimize the possibles error in judment.

  3. hi Steve – I found your observations about edcmooc really interesting. We’re in the process of thinking about the next session, and are grateful for all such feedback and recommendations.

    One of the things the EDCMOOC team is particularly keen to do is to find more ways to help people self-organise into meaningful groupings. The Coursera platform doesn’t support this at present, so the challenges are significant, but it’s a priority.

    There have been a number of suggestions about giving people more focused practice with peer assessment – it’s a good idea, though not necessarily that easy to accomplish in such a short course. This is something we want to discuss further. Coursera know that we are keen to have a ‘feedback on feedback’ mechanism – we are hopeful they will implement this as the platform develops.

    I’d like to respond to one of your critiques of the MOOC – that the work required for the assignment was not worth university credit. (which is a different issue from that of the quality of many of the assignments – which is outstanding.) That may be a valid criticism of *some* perspectives on MOOC purpose, but I should stress that accreditability was not something we set out to achieve with EDCMOOC. We are seeking ways to spark and support engagement at MOOC scale, in our subject area, and that is our primary goal at this point – not how to accredit the course.

    1. Thanks for the comment, Jen. Three quick thoughts for now:

      * I think you could build more practice regarding peer assessment into the course from the very beginning of the course, even one where it is only five weeks long. If I were doing this, I’d have examples, I might ask “everyone” to evaluate the same artefacts to get some discussion going on what constitutes “success,” etc. And I sure as heck would have a scale a little more robust than a “1” or a “2.”

      * The fact that the Coursera platform doesn’t support a way to divide students int smaller groups nor a feedback on feedback system (and possibly not a more robust grading rubric) strikes me as problematic. I have heard from others that the “behind the scenes” programming of Coursera is kind of a mess, so maybe it isn’t surprising. Coursera is supposed to be made up of these “whiz kids” and this is what we get? Huh.

      * I appreciate your point about the intention of edcmooc not being for college credit and it being “learning for the sake of learning” as it were. But the heavy push Coursera is making– certainly in the US– is for credit, is for these courses to count for something. So while you and your colleagues who organized/lead this course had the goals of sparking interest and engagement with the topic and the online format, that is not the goals of the company that hosted your class.

      1. hi Steve – thanks for the peer assessment suggestion.

        My take is that the team at Coursera is eager to learn about what MOOCs in a range of disciplinary areas need, and wants to develop the platform so that it works for as many MOOC approaches as possible. It’s still early days, though.

        Coursera may have a vision for the MOOC, and I’d be the last person to claim that the politics of the platform aren’t important – I believe they are. But the University of Edinburgh (and presumably many of the other institutions who use the platform) is in this for its own reasons, which include building on its strong culture of digital education, exploring new teaching and learning environments, and expanding its reach. Those are the goals that the EDCMOOC team were asked to support when we were invited to make our MOOC.

        I fully agree that what Coursera thinks it is doing is relevant to ongoing MOOC debates, and as a company it may be an increasingly influential voice in policy discussions in the US. Educators are very rightly wary of what that might mean (we would count ourselves among the wary). But it seems to me that there are still a lot of ways this might play out, and for that reason it’s worth analysing each MOOC on its own terms and for what it is attempting (including, where necessary, criticising it for attempting the wrong things). I definitely see your analysis here as taking EDCMOOC on its own terms (though I disagree with some of your conclusions!), except for the point about accreditation.

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.

This site uses Akismet to reduce spam. Learn how your comment data is processed.