my life 2007

A look at the data.

From time to time, instructors meet and exchange ideas. At the last such meeting, one of my colleagues said she has her students assess their work when they turn in papers. I thought it sounded interesting, so I devised a quick four-question "hotwash." (A hotwash is a military term similar to "debrief"; at the end of a mission, the team has a frank discussion about what went right, what went wrong, etc.) Here are the questions:

1. Tell me what you did well on this paper. (This can include focus, organization, clear sentences, strong transitions, research, effective time management....)

2. Tell me what challenged you most on this paper.

3. Tell me what lessons you learned about writing in the course of this assignment that you can apply to improve your performance on your next paper.

4. Based upon the paper itself, what grade do you believe it deserves? (Disregard how much time and effort you put into the paper, because I don't know; I grade your paper upon its merits alone.)

I took up the self-assessments with the papers. To avoid undue influence, I pointedly avoided reading the self-assessments until I had finished grading the papers. (Also, reading the self-assessments was a nice reward for finishing the grading pile. :) ) In addition to the utility of the exercise for the students themselves, their observations of their own performances were most enlightening for me. A few of my lessons learned follow.

NOTE: These statistics were drawn from 34 students. For various reasons, I have incomplete data for the other four.

First, I was struck by the fact that the vast majority of my students can predict their grades for a given assignment within 4 points (usually less). That is, most of them, when forced to be honest with themselves, know what grades their work deserves. (38% gave themselves a lower grade than I did.)

12 students made 76 or below. Only one of these thought he'd earned less than he did (and he was off by .5%; he guessed 72.5 and made a 73). The rest guessed, on average, that they made 16.7% higher than they actually did.

The remaining students (who made 77 and above on this assignment), predicted their grades, on average, within 1.1%. (I had a handful of anomalies: a couple of students supposed they made 12 and 13 points less than they actually did, and one student in this range thought he made 9 points more than he did.)

A point which skews the data a bit is this: many students clearly didn't read the directions closely and thus overlooked/disregarded my injunction to include at least 5 cited sources, at least 3 of which would be bonafide scholarly articles, so in their ignorance, they couldn't predict how many points I'd take off the top for their failure to comply. These are almost entirely the students at the lower end of the spectrum: the 76 and below students. If I adjust for this, they could predict their own grades within 7.4 percentage points--not 16.7.

Generally, of course, the low-range students thought they did better than they did, and the high-range students thought they did worse. The most accurate predictions fell in the 85-92 range.

Interesting! I just found an article about a Cornell psychologist who is fascinated with this phenomenon. Here's a blurb:

...The least competent performers inflate their abilities the most [and] the reason for the overinflation seems to be ignorance, not arrogance; and...chronic self-beliefs, however inaccurate, underlie...over and underestimations of how well they're doing.

Apparently, this is a very common phenomenon.

Other than remaining on the lookout for ways to motivate students in general, I'm not overly concerned about those students who predict fairly what their grades will be or even under-predict what their grades will be. These students have made a choice to settle for the grade they will get. The ones that really concern me are those in the less academically gifted group who think they're doing better than they are.

The ability for a student to accurately (or under-) assess his own performance is, I'm convinced, key to success. Once the student has a clear idea of how he's performing on any given assignment, he can make the choice to exert the effort to improve his performance. Ignorance of his true performance level, however, results in no choice and no effort. After all, you don't pull the lawnmower out until you know the grass has grown. Once you see the grass needs mowing, though, you can then make the choice to fetch the mower (or bring out the weed slinger, depending upon your assessment of how tall the grass is)...or not.

My first question is this: What can I do to correct my low-scoring students' self-assessments? What causes over-rated self-performances? The psychologists say it's ignorance, but ignorance of what, exactly? We can rule out "ignorance of my expectations" because this is the third paper they've written for me, so they've had two papers to develop a very clear idea of my standards. Ignorance of the assignment requirements themselves, I think, is part of it, but I don't know how to fix that. I already provide in-depth written and oft-repeated spoken instructions concerning format, length, subject, number and type of sources. On this one, I even provided a completely anal outline with thesis, roadmap, and topic sentence requirements (i.e., you will repeat in each topic sentence what and where your social issue is, your proposal, and the argument for why your proposal is the best; ex: "Another reason quarantine is the best way to mitigate the AIDS epidemic in South Africa is its simplicity of application."); my outline went on to explain that this would be followed by two to three specific facts to support the argument, each with a citation, and they all would be followed with an analysis of the data that cements the paragraph's argument. The outline was embarrassingly anal, but for those students who followed it, amazingly effective (and not clunky or mechanical at all, surprisingly). I spent at least one class working through examples of ways to make arguments, brainstorming what sort of data to look for during the research phase, and tying this back to this paper.

While I think some of my students were still unclear on what my expectations were for this paper, I confess I have no idea how to mitigate that problem.

What reasons can you think of, and how can I improve?

d