A Cautionary Tale About Remote Education

The NY Times carries a cautionary tale about remote education. Excerpts below:

Accused of Cheating by an Algorithm, and a Professor She Had Never Met: An unsettling glimpse at the digitization of education

Kashmir Hill, May 27, 2022 

A Florida teenager taking a biology class at a community college got an upsetting note this year. A start-up called Honorlock had flagged her as acting suspiciously during an exam in February. She was, she said in an email to The New York Times, a Black woman who had been “wrongfully accused of academic dishonesty by an algorithm.”

What happened, however, was more complicated than a simple algorithmic mistake. It involved several humans, academic bureaucracy and an automated facial detection tool from Amazon called Rekognition. Despite extensive data collection, including a recording of the girl, 17, and her screen while she took the test, the accusation of cheating was ultimately a human judgment call: Did looking away from the screen mean she was cheating? ...

Honorlock, based in Boca Raton, Fla., was founded by a couple of business school graduates who were frustrated by classmates they believed were gaming tests. The start-up administered nine million exams in 2021, charging about $5 per test or $10 per student to cover all the tests in the course. Honorlock has raised $40 million from investors, the vast majority of it since the pandemic began.

Keeping test takers honest has become a multimillion-dollar industry, but Honorlock and its competitors, including ExamSoft, ProctorU and Proctorio, have faced major blowback along the way: widespread activism, media reports on the technology’s problems and even a Senate inquiry. Some surveilled test takers have been frustrated by the software’s invasiveness, glitches, false allegations of cheating and failure to work equally well for all types of people...

The Florida teenager’s biology professor, Jonelle Orridge, ... distant, her interactions with students taking place by email, as she assigned readings and YouTube videos. The exam this past February was the second the teenager had taken in the class. She set up her laptop in her living room in North Lauderdale making sure to follow a long list of rules set out in the class syllabus and in an Honorlock drop-down menu: Do not eat or drink, use a phone, have others in the room, look offscreen to read notes, and so on.

The student had to pose in front of her laptop camera for a photo, show her student ID, and then pick her laptop up and use its camera to provide a 360-degree scan of the room to prove she didn’t have any contraband material. She didn’t mind any of this, she said, because she hoped the measures would prevent others from cheating.

“You were flagged by Honorlock,” Dr. Orridge wrote. “After review of your video, you were observed frequently looking down and away from the screen before answering questions.” She was receiving a zero on the exam, and the matter was being referred to the dean of student affairs. “If you are found responsible for academic dishonesty the grade of zero will remain,” Dr. Orridge wrote. “This must be a mistake,” the student replied in an email. “I was not being academically dishonest. Looking down does not indicate academic dishonesty.” ...

The New York Times has reviewed the video. Honorlock recordings of several other students are visible briefly in the screen capture, before the teenager’s video is played.

The student and her screen are visible, as is a partial log of time stamps, including at least one red flag, which is meant to indicate highly suspicious behavior, just a minute into her test. As the student begins the exam, at 8:29 a.m., she scrolls through four questions, appearing to look down after reading each one, once for as long as 10 seconds. She shifts slightly. She does not answer any of the questions during the 50-second clip. It’s impossible to say with certainty what is happening in the video. What the artificial intelligence technology got right is that she looked down. But to do what? She could be staring at the table, a smartphone or notes. The video is ambiguous...

The Times analyzed images from the student’s Honorlock video with Amazon Rekognition. It was 99.9 percent confident that a face was present and that it was sad, and 59 percent confident that the student was a man...

The teenager graduated from Broward College this month. She remains distraught at being labeled a cheater and fears it could happen again. “I try to become like a mannequin during tests now,” she said.

Full story at https://www.nytimes.com/2022/05/27/technology/college-students-cheating-software-honorlock.html.

Of course, there are partial solutions to the problem identified in the article. Courses that require something other than a closed-book exam do not require artificial proctoring. Courses that do require such exams could require students who had been taking the course remotely to come to a central location to take the exam (a solution that may not be available in pandemics). We have noted in past posts that watching a "Sunrise Semester" college course on TV and then coming to a central place for exams is a concept that goes back to the 1950s:

http://uclafacultyassociation.blogspot.com/2020/03/return-of-straight-lecture.html.

Nonetheless, we are stuck with the reality that online education can deliver "content" and even some interaction. But it is not a perfect substitute for in-person instruction.