Skip to article frontmatterSkip to article content

Why Have We Developed the REMARK Standard?

As a discipline, economics has fallen behind other technical fields in the adoption of modern software development and reproducibility practices. In good part, this is because immediate professional rewards accrue to whoever publishes a particular computational result first; few additional rewards flow to the scholar who works hard to make sure that their code is robustly and easily replicable by other scholars. The result is that economists often write quick-and-dirty code that gets the job done for their particular problem, but is often difficult or impossible for others to use to tackle related problems.

Anyone who works in the field knows the frustration of downloading the zip file of a paper that claims to have solved an important problem, and finding that it can take weeks or months to get code written on, say, Windows 8 with Matlab x.y.z and a host of dependencies to work on any computer different from the one on which it ran when the zip file was created.

A first step in improving this situation is simply to make replication of results much easier. To that end, the REMARK standard defines a set of practices and standards whose aim is to guarantee that the claimed results of the paper can be reproduced on any computer, present or future, at the touch of a button (or rather, by means of running a required reproduce.sh script).

The REMARK standard defines a minimal set of other requirements that, if adopted widely, would allow economics research to proceed, as it does in other fields, by standing on the shoulders of scholars who have made important advances, rather than by having to reinvent their wheel before you can improve it.