A friend of mine drew my attention to the NYTimes’ recent article on advanced in essay-grading software. It’s technology that will raise hackles at campuses around the country. The claim is that such programs are becoming sophisticated enough to grade college-level writing. Of course, their effectiveness is widely debated. The article helpfully includes a link to a study by Les Perelman which critiques the data being used to support such claims (he argues that sample size problems, confusion between distinct kinds of essays and grading systems, and loose assertions undermine the argument). The software is getting better, but it still doesn’t look like it can quite replicate the scores produced by human graders.
But such criticism is an argument at the margins. There is now clearly room for debate on both sides. Machines are comparable on standardized tests. The long-term trajectory is evident: if machines are roughly as effective as a force of part-time human graders, standardized tests will end up using the software to save money. They’ll keep some humans in the loop cross checking and validating, but the key incentives all point in the direction of greater automization. The reductive structures and simplistic arguments which we train students to replicate for these tests has laid the groundwork. We’ve already whittled essay writing into an algorithm.
Yesterday the seminar I’ve been teaching at Penn finished their digital project: an online edition of John Leyden’s “Tales of the Peries,” a handwritten manuscript in the archives of the National Library of Scotland. Leyden was a romantic poet as well as close collaborator of Walter Scott’s, before traveling to Southeast Asia as a functionary of the East India Company. Once there, Leyden established himself as an Orientalist and specialist in Asian languages, and the Tales of the Peries is an example from this fruitful period before his early death in 1811.
As part of a larger class on historical fiction, fantasy, and the influence of empire, my students built an Omeka site that includes digital facsimiles of the manuscript, transcriptions using Scripto, a plugin for Omeka, and a “readerly edition” that incorporates their research into editorial practices and critical editions and links to supporting materials and entries in the Omeka collection and on the wiki. In addition, they built a host of supporting materials for the site, from critical evaluations of the Tales, its verse, and the influence of Urdu and Arabic literature, to information about Leyden, his involvement with the EIC, even an animated Flash map that walks the reader through the geography of the tale and details the main transformations of Melech Mahommed, the main character, over the course of the narrative.