In a new study in the Proceedings of the National Academy of Sciences, my co-authors and I document bias in the peer review process that precedes the publication of research articles in scientific journals. Even if they are of equal quality, articles authored by prominent researchers get better ratings than articles authored by less well-known researchers. Implementing an idea by Jürgen Huber (University of Innsbruck), leader of our research team, we conducted a simple experiment: Vernon Smith (Professor at Chapman University and Nobel Memorial Prize Laureate in Economics 2002) and Sabiou Inoua (junior post-doc researcher at Chapman University), who were both members of the research team, jointly authored a research article and submitted it to the Journal of Behavioral and Experimental Finance for review. In my role as the editor of this journal and also a member of the research team, I sent invitations to review the article to a total of 3300 experts in the field. A total of 534 accepted the invitation to review and submitted a review report. While all reviewers received the same article for their evaluation, they received different information about who had authored the article. One group was informed that Nobel prize laureate Vernon Smith was one of the authors. Another group was informed that junior researcher Sabiou Inoua was one of the authors. A third group did not receive any information about the authors. Reviewers with different types of information about the article’s authors evaluated the quality of the research article markedly differently. Of the reviewers who had received no information about the article’s authors (“AA” in the figure below), 48.2% recommended against publishing the article. This proportion was even higher among the reviewers who had been informed that one of the authors was the relatively unknown junior researcher (“AL”); here 65.3% recommended against publication. Yet of the reviewers who had been informed that one of the authors was Nobel prize laureate Vernon Smith (“AH”), only 22.5% recommended against publication. Rudolf Kerschbamer (University of Innsbruck), another member of our research team, traces the source of the different evaluations to the “halo-effect”. This term from social psychology describes the phenomenon that we tend to evaluate the actions and work of someone more favorably when we are favorably disposed towards this person. Christian König genannt Kersting (University of Innsbruck), also a member of our research team, considers the results to constitute an important trigger to start rethinking the scientific review process: “As researchers, we are constantly working on improving our methods and processes. Our results have met great interest from the academic community and many editors of scientific journals are already testing new methods for evaluating and ensuring the quality of research findings.” Link to the paper (open access)
Leave a Reply