The Journal of the Canadian Economics Association
La revue de l'association canadienne d'économique

Advice to Referees

Do I need to check the algebra or the empirical analysis?

Only rarely would a referee be in a position to check empirical results. Often the referee has neither sufficient time nor access to the necessary data to replicate the empirical analysis in a paper. Usually the best you can do is to convince yourself that the results seem plausible, given your knowledge of the area, and are internally consistent. The editors are, however, encouraging authors to make their data and (where relevant) estimation algorithms available to referees and potentially to other interested parties. Checking algebra is much more feasible than checking empirical analysis, but even so it may often not be necessary. In particular, if you are recommending rejection for some reason other than concerns about the correctness of the analysis, there is no need to check the algebraic details of the paper. Even if you are favourably disposed toward a paper, it will often be impossible to check everything. One useful approach is to check some items, particularly the initial model development, and then to make sure that you can at least understand why the other results make sense. If it is very hard to follow the algebraic development in a paper, that is in itself a serious flaw in the paper.