De Ruijter: always the same people in PhD committees

The investigation into doctoral dissertation fraud and embezzlement by former dean Arie de Ruijter glitches. The dissertations involved are harshly criticized and even ‘questionable research practices’ are mentioned. But no tough measures have been taken so far. Also striking: a small group of scientists was very often involved in the supervision process of the criticized dissertations, while they did not always have the right expertise.

During his deanery period from 2007 to December 31st 2014, Arie de Ruijter has supervised twenty-two doctoral theses at his faculty, according to Research Portal. According to our most recent information, De Ruijter has even supervised 53 dissertations during his entire period at Tilburg University at different faculties. In addition to the research on financial fraud, the university has also been conducting an investigation into the scientific integrity during the writing and realisation process of these dissertations.

A typical matter to be referred to Tilburg University’s Scientific Integrity Committee (CWI). But in that case the College Board would have to decide on the report, says spokesman Tineke Bennema. Establishing an external ad hoc committee would not require that. Bennema: “It is an independent investigation. By setting up an external ad hoc commission, maximum neutrality and independence are guaranteed.” Remarkably, the chairman of the external committee (Professor of Utrecht University Ton Hol) is also the chairman of Tilburg University’s own CWI.Arie de Ruijter, former dean of the Tilburg School of Humanities, is accused of financial fraud and embezzlement. In his function as a dean (2007 to December 31st 2014) he is said to have issued doctorate bonuses to companies of family members. A total of one million euros might have been diverted and siphoned off. The investigation in the case ‘De Ruijter’ is still ongoing.The external committee of inquiry had to review the dissertations and worked in secrecy. Hence they didn’t speak to interested parties and didn’t indicate which and how many dissertations they had read. It remained an indication: ‘a large number, very thorough’.

Critical findings

The Commission Hol is critical with regard to the level of the doctoral theses and comes to firm conclusions. The dissertations have been written by non-university professionals (external doctoral candidates), often at the end of their career. Their own experience has been over-emphasized. Case studies make up for a large part of the theses, while it remains unclear why. The associated risk of bias is insufficiently recognized. It is difficult to generalize the conclusions of the dissertations. Consequently, there was limited or even no scientific added value at all, according to the Commission Hol.The large number of dissertations (32) has been completed in a very short period of time, while they often failed to clearly fall within the expertise of the supervisor(s). The dissertations eluded from internal and external quality care. As a whole the dissertations were of poor or even insufficient quality for one research group or faculty chair.

There are ’too many disputable cases’ concerned here, which makes it very questionable whether ‘a doctorate policy that derives from this, is in line with standards of scientific quality.’ There are also facts that indicate the possibility of partiality and bias. For example, only a limited number of professors and graduated PhDs acted as evaluators on multiple doctorate theses. In addition financial transactions were made for guidance, sometimes paid to own companies.

Mild judgements

Based on these findings, the committee had to answer three questions:

1. Was the composition and method of the doctorate assessment commissions so that the assessment was carried out according to the usual scientific standards?

2. Could the assessment committees reasonably conclude that the dissertations were suitable and fit for defence?

3. Are there financial interests that give cause for doubts regarding the scientific integrity during the assessment?

Then something remarkable happens. Not even once does the committee conclude that something went haywire. The composition of assessment commissions? Nothing wrong with that. Admitting those dissertations to defence? Some thought will surely have been given to that, or should have been given. Also there are insufficient indications that doctorates have been wrongfully granted. The financial interests? No connection with the quality of the theses.

dr-ballonnetjeIn response to this third question, the committee does not rule out the fact that perverse financial incentives may have led to a “promotional culture, with quantity appearing to be prioritized over quality.” And the committee also states, that “although there may have been reasons for (eventually) granting doctorates (…), there is a practice here that should be avoided.”

General conclusion drawn by the Hol committee: There are no or at least insufficient indications that the respective assessment commissions have acted in violation with the scientific principles of independence and impartiality, at least not in such way that one could speak of an infringement of the standards of scientific integrity.

But: ‘In the future it should be prevented that the practice of having a relatively large quantity of scientifically weak dissertations evaluated by only a limited number of people (…), occurs again’. According to the committee this is a case of questionable research practices.On the verge

The report causes many raised eyebrows. After all, the extensive criticism is merely followed by very mild conclusions. Lex Bouter, Professor of Methodology and Integrity at the Vrije Universiteit (VU), does have the impression that the Hol committee has worked meticulously. According to Tilburg University professor methodology Jelte Wicherts, the commission didn’t have an easy job. “Investigating a whole range of research within a limited time frame? At Stapel, such a thing took a whole year, in full collaboration with three universities.” Bouter and Wicherts both note that the committee was not present when the dissertations were assessed to be sufficient for a PhD. Bouter: “Judging differently afterwards is only possible if the dissertation is evidently below level.” Wicherts: “An Integrity Commission won’t be inclined to call a thesis insufficient. If I were an external doctorate candidate and the commission states my previously approved thesis to be incorrect, due to which I miss out on a big assignment, I would be able to sue that commission.”

There is plenty more to say about the report. Wicherts thinks it is quite a harsh finding that the quality of the dissertations was generally low. “If all mentioned criticism is found in one dissertation, then that’s simply a bad dissertation.” However, it is difficult to determine the point where a dissertation becomes inadequate, Bouters notes. “There is a whole spectrum, from brilliant to deficient dissertations. Sometimes one committee will accept a thesis and the other won’t, but that’s not a bad thing in itself. What seems to be the case here is that all dissertations were of marginal or low quality. The Hol commission mentions that none of the individual dissertations was actually exposed as insufficient, but the problem is that there were so many of them right on the verge.”

Damage

According to Bouter, real fraud is of course far worse, but the doubtful practices found here are likely to cause more damage because they occur far more frequently. “The inadequate guidance of students and PhD candidates is seen by experts as a major problem, because it is relatively common. Supervising and guiding students is one of the most responsible tasks a scientist has.”The qualification of dubious practices seems to be non-binding, as there is no effect on the individual scientist at all, unlike in the case of fraud. According to Bouter and Wicherts, individual scientists should not be punished for such practices. Instead universities should think about procedures, such as the compilation of assessment commissions.

No expertise

According to the Hol committee, the supervisors did not have clear expertise in the field of the dissertation subjects. “That shouldn’t happen,” says Bouter. “You should supervise PhD students only within the area of ​​your own expertise. Otherwise you need to contract other people for those chapters or articles that you’re not able to supervise yourself.” Wicherts: “It wouldn’t be right if I were, for example, attending the defence of research done with a particle accelerator next week. That’s simple malpractice.”

“You shouldn’t ask the same group for commissions each time”

The fact that De Ruijter often guided doctorate theses in coordination with the same group of supervisors isn’t that unusual, Bouter reckons. But each assessment commission should definitely be well composed. “Colleagues who’ve had nothing to do with the product itself should check the work. It must be a diverse and independent group of people which contains at least a professor from another university.These are the checks and balances of the system.” Wicherts: “You shouldn’t ask the same group for those assessment commissions each time.”

Univers conducted a study of the various assessment committee compositions. This study reveals that only a small group of scientists was very frequently involved in the granting of multiple doctorates. According to Bouter, that could be explained if there had been only one research area: in that case you’re practically forced to always ask the same people because there aren’t that many unrelated experts. But the subjects and content of the dissertations differed widely. In addition, social sciences are not known to have a shortage of professors who can evaluate dissertations. However, Bouter doesn’t think that this small club of returning scientists has put any pressure on the other reviewers. “The assessment usually takes place in writing, commissions do not meet. Commission members do not see each other’s review, or only after having submitted their own judgement.”

Substantiated judgement

dr-ballonnetje-2Reflecting on all of this, one may be inclined to come to conclusions other than those of the Commission Hol. Such as: the composition and methods of the commissions were defective, the assessment committees should not have approved all these moderate to bad dissertations, and financial interests have facilitated malpractices. How can it all be just fine if it is so harshly criticized by even the Commission Hol itself?

“These are very understandable questions,” says Ton Hol. “Everyone may have his or her opinion about it. But the task of the committee was to make an informed decision. Suspicions or mere opinions are insufficient.” The committee saw that a limited group of scientists always returned to the assessment commissions. “That creates the appearance of partiality and bias, but that won’t make me say that this is a clear case of non-integrity.” Also, to compose the commissions, De Ruijter could only fish in a limited pool of scientists with sufficient expert knowledge. “The main point is: were these committees completely staged so that they’re all buddies and the whole thing was rigged? That wasn’t the case… at least, we could not prove that. We do state: this is conspicuous, it is a point of attention, but it can be explained.”

Insufficient

The actual level of the dissertations was rather poor. “We were confronted with dissertations that I was not deeply impressed with,” says Hol. “We can certainly assume there has been discussion about them, but that doesn’t mean the level was indeed insufficient.” Even dissertations without clear scientific added value can lead to a doctorate: “This happens more often; you start a research and eventually you’re dissatisfied with the results. Then there is still the aptitude test: were you at least able to conduct independent research?”

In addition, according to Hol, a committee can hardly reject dissertations afterwards. “If you do that, you’re taking the seat of the supervisors; in that case you have to be absolutely sure that the dissertation is deficient and faulty.” Hol would in fact reject a dissertation with the level of only a master’s thesis, with spelling errors and all. The practice the committee encountered wasn’t good at all, according to the report. “But we’ve also seen the serious work done by the doctoral students.” Also the financial transactions towards universities of (co-)supervisors weren’t so substantial that they could have influenced the scientists, says Hol.

Obvious mistakes

The Hol commission did not have access to all assessment reports written by the commissions back then. Nowadays these reports are archived by universities, but not a few years back, says Hol. Shouldn’t this at least have been noticed at that time? It is up to the vice-dean to supervise the research and up to deans themselves to monitor the quality of assessment commissions. “Perhaps the problem here is the fact that the dean was also the supervisor.”

“We weren’t encountered with obvious mistakes,” concludes Hol. Nevertheless, the committee concludes repeatedly and in bold terms that this practice can’t go on and must be prevented. “In individual cases, one could have valid reasons to approve these dissertations. But if you put it all in a row, I hope this won’t happen again.”

This article is a translation of ‘De kleine wereld van De Ruijter’, published in Univers 12.John Rijsman: “I was involved, but also totally independent”

“The dissertations were not suitable for publishing in journals of abstract science, but I would not say they were of poor quality. Like with all theses, I have seen good ones and weaker ones. I’ve had my doubts, and I’ve expressed them. But with Arie I always knew: this makes sense. Sometimes the theoretical reflection could be improved or certain literary perspectives could be taken into account. Then you’re dealing with tastes and preferences. I’ve never seen something that was totally inacceptable. And you can’t take the seat of commission members, who I think have given their judgement with full integrity.

Especially if you wish to have substantial, transferable societal knowledge, you need exactly this type of dissertations. I have often used them in college. They were fairly clear-structured and had one big advantage: they had a real subject, a point. They didn’t contain models or abstractions you could just as well think up yourself. It’s impossible to fool around when you write about business, about the real world, where things really happened.

A university is not only meant for pure science. I’ve been in assessment commissions with about twelve hundred other people. Hence I’ve seen many different approaches and visions. And I can say there are many kinds of science. If a university claims it wishes to gain a better understanding of society, I think it can’t go without conducting this kind of science research.

Arie de Ruijter has been my dean twice. He was well aware of my diverse and substantial international experience. It’s not so that Arie was able to choose from many people and could assemble completely different commissions every time. If you’re talking about dissertations on societal phenomena, most professors refuse to take place in commissions because they don’t find themselves fit. I was involved, but also totally independent of Arie, just as I am with regard to hundreds of other colleagues.”

John Rijsman was a member of De Ruijter’s assessment commissions 19 times out of 32 dissertations.

Bekijk meer recent nieuws

Schrijf je in voor onze nieuwsbrief

Blijf op de hoogte. Meld je aan voor de nieuwsbrief van Univers.