Together with my colleagues in Graz, Andrea Schertler and Erik Theissen, and another 340 authors around the globe, I recently participated in a large (obviously), cooperative, international research project to study “non-standard errors”. As a member of one of more than 160 one- or two-researcher teams, I analyzed the same dataset of financial market transactions with the aim of answering the same six questions. Our research shows that there is large variation in the results (i.e., the answers on the six research questions). The “non-standard errors” are similar in magnitude as the (mean) standard error and even looking only at a sub-sample of “highest quality results” does not change the picture much. In other words, we find that if you ask different expert researchers to study a question using the same data, you may still get different answers. Furthermore, the researchers themselves underestimate the variation in the answers that different researchers or research teams provide. Read more about this somewhat depressing but nevertheless exciting (at least to me) and definitely relevant research under the following links: Link to the project website | Link to the working paper Oh, and if you want a more humorous take on the issue, watch the below interview with Albert Menkveld, one of the lead authors on the paper.
Leave a Reply