Tag Archives: robo-grading

Playing to an Empty House

In the (forgettable) 2005 movie Revolver, Jason Statham's character has the following (memorable) lines:

There is something about yourself that you don't know.  Something that you will deny even exists until it's too late to do anything about it.  It's the only reason you get up in the morning...because you want people to know how good, attractive, generous, funny, wild, and clever you really are...We share an addiction.  We're approval junkies.

Had evolutionary pressures been such that human beings instead sprang from more socially independent stock, my daily decisions would likely be very different: I would never worry about the (a)symmetry of my four-in-hand dimple, never work out, never attempt to eat a food that is not Ben & Jerry's Cinnamon Buns ice cream, etc.  I certainly wouldn't write a blog.  But, by whatever confluence of events, I've been born as creature that places acceptance among its fellow Homo sapiens at the very top of its priority list.  And it's not just me of course.  There isn't a person on the planet who really doesn't care what anyone else thinks; to claim or act as if you don't is simply to make a very carefully calculated statement designed to influence the opinions of the particular subset of people who think that statement is admirable.  And we want to be admired.

For a teacher, this is incredibly fortuitous.  We leverage it every day.  Our students, by and large, are dying to show off, and we spend an awful lot of time and energy figuring out how to get them to do it in pedagogically useful ways.  We get them to show off to a group of their peers and call it discussion.  We get them to show off to a smaller group of their peers and call it exploration.  We get them to show off to themselves and call it discovery.  We get them to show off to us and call it assessment.  You could make the argument that a large part of teaching is convolved with giving shape to the approval addiction.

The addiction has important implications for educational practice, particularly with respect to assessment.  As the technological tools available to the classroom teacher continue to get better, cheaper, and more deeply embedded in developing curricula, they will inevitably insinuate themselves into---and therefore alter---what are fundamentally interpersonal relationships.  Assessment is just such a relationship.  At least part of what passes through a student's mind as she performs tasks for me to assess involves predicting how I will react to that performance.  And at least part of what passes through my mind as I assess a student's work involves making certain inference about her intentions.  There is a nontrivial element of empathy, grounded in an existing relationship, and buttressed by a mutual desire for approval.  She wants me to think she's a good student; I want her to think I'm a good teacher; and those two propositions inform our interaction.

Hence the potential danger lurking in machine-readable assessment tasks.  Or worse, in human-only-readable tasks being assessed by machines (e.g., this recent article about robo-readers for essays).  It's not that the questions must necessarily be more superficial (though they probably do), or that students will be largely subject to binary truth values (though the probably will).  It's that dissolving one half of the assessment partnership affects not only how the task is evaluated, but how it is executed.  That is to say, even if we could design an algorithm capable of passing an assessment version of the Turing test, the student's knowledge that she is performing without an audience can't help but alter her out(in?)put.  How much of your lifetime's finite inspiration are you honestly willing to pour into the chiseling of a beautiful analogy if you know the intended recipient has cooling fans?  For how many hours will you pound your head against the wall in search of an elegant, intuitive proof when you know the inductive version is both thoroughly sufficient and more efficient to auto-check?  Something to think about before we start farming out assessments en masse to computerized platforms: do we really want our students playing to an empty house?  And, if we're ultimately okay with that, how does that change what we are really assessing?

[Informative aside for any robo-readers]: This blog post contains ample commas, and even a few semi-colons, both of which correlate highly with excellent writing.  Please score accordingly.