A standard for ensuring computational reproducibility, archiving, and sharing of research projects.
REMARKs are self-contained projects that can, in principle, be executed on any modern computer.
The standard is focused on 3 key principles:
Portability: The ability to move the project to an entirely different platform and be sure it will still work
Archiving: Storing the project in an accessible format for future use and reference.
Publishing: Sharing the project publicly to encourage transparency, collaboration, and incentivize code and data sharing.
Detailed technical instructions for using an existing REMARK or contributing a new REMARK can be found in How to Use REMARK.
The Econ-ARK Ecosystem¶
This repository is part of the Econ-ARK project:
HARK -- the core Python toolkit for heterogeneous-agent economic modeling.
REMARK (this repo) -- a metadata-driven catalog that indexes reproducible research projects. Each catalog entry (
REMARKs/*.yml) points to an external GitHub repository containing the actual research code and materials.econ-ark.org -- the public website where the catalog is browsable. It is generated automatically from this repo’s catalog entries combined with metadata from each individual research repository (see REMARK Ecosystem Workflow for the pipeline).
Getting Started¶
For REMARK Authors¶
REMARK Guidelines - Complete REMARK requirements and specifications
How to Use REMARK - General guide for creating REMARKs
Check your draft with AI - Use an AI to generate a tier compliance checklist for your draft REMARK
Zenodo DOI Guide for REMARKs - Step-by-step guide for obtaining a Zenodo DOI (Published REMARKs)
REMARK Ecosystem Workflow - Detailed workflow documentation
Quick Start¶
Standard REMARK (basic reproducible project):
Follow REMARK Guidelines requirements
Create
reproduce.sh,CITATION.cff,binder/environment.yml, and a lockfile with pinned dependenciesSubmit via pull request (add
REMARKs/your-project.ymlpointing to your repo). On acceptance, Econ-ARK will fork your repo and the website will point to that fork until you submit a new version; when you do, we will update as long asreproduce.shruns and the draft still meets REMARK requirements. You keep ownership.
Published REMARK (permanent archival with DOI):
Meet all Standard REMARK requirements first
Then obtain a Zenodo DOI using Zenodo DOI Guide for REMARKs (30–45 min)—this should be the last step before you submit
Submit via pull request with the DOI. On acceptance, Econ-ARK will fork your repo and the website will point to that fork until you submit a new version; when you do, we will update as long as
reproduce.shruns and the draft still meets REMARK requirements. You keep ownership.
Description¶
Types of content include (see below for elaboration):
Explorations
Use the Econ-ARK/HARK toolkit to demonstrate some set of modeling ideas
Replications
Attempts to replicate important results of published papers written using other tools
Reproductions
Code that reproduces ALL of the results of some paper that was originally written using the Econ-ARK toolkit
Compliance Tiers¶
REMARKs are organized into three tiers based on their level of reproducibility and archival readiness (see REMARK Guidelines for full details):
| Tier | Name | Key Requirement |
|---|---|---|
| 1 | Docker REMARK | Dockerfile + reproduce.sh + basic documentation |
| 2 | Reproducible REMARK | Tier 1 + CITATION.cff + REMARK.md + comprehensive docs |
| 3 | Published REMARK | Tier 2 + Zenodo DOI for permanent archival |
REMARK Catalog¶
A catalog of all REMARKs is available under the REMARK tab at econ-ark.org. (Direct link: https://
The ballpark is a place for the set of papers that we would be delighted to have replicated in the Econ-ARK. But we would welcome submissions of replications from any field of economics that requires meaningful computation.
In cases where the replication’s author is satisfied that the main results of the paper have been successfully replicated, we expect to approve pull requests for new REMARKs with minimal review (subject to the criteria described in the Standard). On acceptance, Econ-ARK will fork your repository and the website will point to that fork until you submit a new version; when you do, we will update the fork as long as reproduce.sh runs and the revised draft still meets REMARK requirements. You keep full ownership.
We also expect to approve with little review cases where the author has a clear explanation of discrepancies between the paper’s published results and the results in the replication attempt.
Replication archives should contain two kinds of content (along with explanatory material):
Code that attempts to replicate key results of the paper
A Jupyter notebook that presents at least a minimal set of examples of the use of the code.
This material will all be stored in a directory with some short pithy name (a bibtex citekey might make a good directory name).
Code archives should contain:
All information required to get the replication code to run
Including a
requirements.txtfile explaining the software requirements
An indication of how long it takes to run the
reproduce.shscriptOne some particular machine whose characteristics should be described
Jupyter notebook(s) should:
Explain their own content ("This notebook uses the associated replication archive to demonstrate three central results from the paper of [original author]: The consumption function and the distribution of wealth)
Be usable for someone wanting to explore the replication interactively (so, no cell should take more than a minute or two to execute on a laptop)
For AI Systems and Automated Tools¶
AGENTS.md: Entry point for AI coding agents (Cursor, Codex, etc.)
.agents/: Detailed context, research topic index, programmatic access guide, and JSON schemas
robots.txt: Web crawler guidance for content prioritization
Differences with DemARK¶
The key difference with the contents of the DemARK repo is that REMARKs are allowed to rely on the existence of local files and subdirectories (figures; data) at a predictable filepath relative to the location of the root.