Three thousand official reports and a general pardon: this is how the Proctorio examinations at Tilburg University have gone so far

The last exam period of yet another eventful academic year has arrived. Many exams are already taking place on campus, but a significant number are still being conducted online. Despite fierce criticism, the controversial proctoring program Proctorio has not been abandoned. Time for an evaluation.

The introduction of proctoring at Tilburg University was not without its struggles. After the Executive Board announced (Dutch only) that Proctorio would be used for proctoring of online exams, an avalanche of critical reactions arose. Not only did the student parties Front and SAM express their doubts, but a petition was set up that was signed over 5,000 times. In addition, critical open letters were written by both students and lecturers at Tilburg University.

In the end, it did happen—albeit in a limited way. Instead of using proctoring for all exams, it was decided to use it only for “recognized key courses with knowledge-based exams.” Which exams were included in this—as is typical of the decentralized organizational structure of this university—was up to the Schools to determine.

Except for the Tilburg School of Catholic Theology (TST), every School decided to use proctoring. However, the extent and manner in which it is used varies per School, says Nicola Jägers, chair of the Examination Board at Tilburg Law School (TLS). “In general, the Schools have been very cautious in applying Proctorio. For each online exam, careful consideration has been given to whether proctoring is necessary.”

“The need for (online) proctoring depends in part on how easily one can cheat on an exam. A math exam is more prone to this than a legal argument or essay, for example. Another decisive factor is whether a good alternative form of assessment is available in which proctoring can be dispensed with. Despite the fact that these alternatives have been developed on a large scale, not every assessment lends itself to this.”

By this, Jägers is referring mainly to exams where factual knowledge is examined by means of multiple-choice questions, for example. This type of examination is—due to the considerable size of some courses—more common in a large School such as TiSEM. This leads to relatively more proctoring at this School compared to other Schools.

Opt-out possibility

In principle, every student has the option of taking an online proctored exam on campus. Reasons for this may be that the student experiences technical difficulties at home in the form of a poor WiFi connection or a noisy house, or that the student objects to the method of online proctoring for privacy reasons. The first reason was given 888 times since the beginning of the corona crisis, whereas the privacy objection was used 538 times.

There is also an opt-out possibility the other way around, which was used 11,960 times in the December-January exam block alone. It allows students who meet conditions for exceptions to take on-campusexams online. These conditions are mainly related to a potential risk of infection, making it better to take the exam online for safety reasons.

However, the largest group taking advantage of this exception is foreign students. With lectures taking place entirely online and travel restrictions being introduced between different countries, this group has moved back to their home countries in droves. For them, an online exam, therefore, offers a solution.

In addition, until April 29, there was a third party claiming this opt-out possibility. These were students who simply preferred an online exam to an offline one. Students now no longer have this free choice, as the government has relaxed the measures and, therefore, there is no longer a valid reason—apart from the other grounds for exceptions—for not coming to campus.

Delay

During a Proctorio exam, students are monitored by a form of artificial intelligence. This registers, among other things via the webcam camera, whether there is a so-called “irregularity.” This could be the fact that certain tabs are open, the registration of an unknown object in the room or on the desk, or that the identity card that a student has to show at the beginning of the exam is not visible.

The irregularity is then assessed by the human eye of an invigilator. If it appears to be a triviality, the invigilator waves it away. But if it appears to be more than that, the invigilator makes an official report. After a student has had the opportunity to respond to this, the entire file is sent to the School’s Examination Board. This Board decides whether the student can get away with a warning or that the examination will be declared invalid.

The above process did not always run smoothly in the past year, as can be seen by the high number of official reports issued since the introduction of proctoring. The 3,425 official reports issued university-wide in 2020 stand in stark contrast to the 408 reports in all of 2019. While, certainly, not all of them resulted in the invalidation of the exam, they did cause many delays in grading the exams.

This was most severe within Tilburg School of Economics and Management (TiSEM), where in the exam block around the turn of the year, congestion was so great that it caused the Examination Board to decide to impose a general pardon. Examinations of students who had still not received their results after a certain date were declared valid.

“Because so many reports came in and each case had to be assessed individually, it could sometimes take a month before a student was informed that an official report had been made against him or her,” says Jac Kragt, chair of the TiSEM Examination Board.

Therefore, students were often immediately advised to register for the resit, just to be on the safe side. “This created an unpleasant situation because students did not know where they stood.”

Necessary evil

The considerable number of official reports is clearly related to the use of online proctoring and is, therefore, also a major disadvantage of the system. Jägers: “In the vast majority of cases, it is a case of false alarm. For example, sunlight falls on the identity card so that it cannot be properly detected by the camera. However, not being able to show an ID card is potentially a serious form of fraud. Therefore, we cannot afford to become more lax with individual checks.”

However, something can be done about the sensitivity of the proctoring system so that it is less likely to detect innocent anomalies, Kragt says. “There are certain buttons we can turn such as light and sound settings. During the first exam period of this semester, for example, we made sure that certain sounds were picked up less easily. Judging by the numbers, that paid off as well.”

In addition to the increased likelihood of a proctored exam, students experience significantly more stress while taking a proctored exam. This is because a student has to do many extra things that do not have to be done during an on-campus exam.

“Because there are so many additional rules, the possibility of doing something wrong increased. This causes stress that comes on top of the already present tension before the exam,” Kragt says.

It should be noted that Proctorio did not cause only doom and gloom. On the contrary, Jägers asserts. “Yes, there has been a lot of complaining, but in the end, proctoring has ensured that diplomas have retained their value and that students have not been delayed. The fact that one of my students was able to take the exam in his pajamas in Mexico City, I consider a big win.”

Proctoring is thus a necessary evil; no one wants it, yet it must be done. And as long as exams are taken online, the proctoring program will continue to be used. No one dares to say how long that will be. Jägers does not venture into future scenarios either. “If we’ve learned anything from this crisis so far, it’s that we have to be careful about predicting the future.”

_Advertentie.