A REMARK refers to an open git repository that is indexed in this repository with appropriate metadata.
REMARKs are organized into three tiers based on their level of reproducibility and archival readiness:
Tier 1 (Docker REMARK): Minimal reproducibility via Docker containerization
Tier 2 (Reproducible REMARK): Enhanced reproducibility with comprehensive documentation
Tier 3 (Published REMARK): Publication-ready with permanent archival (Zenodo DOI)
Preparing your Repository¶
Base Requirements (All Tiers)¶
All REMARKs must meet these fundamental requirements:
Have a tagged release.
If you make changes to your work, please release a new version/tag.
Commonly, tags will follow semantic versioning having a 3 part numeric identifier
v{major}.{minor}.{patch}where:major version changes represent large refactors, new features, and/or new analyses.
minor version changes represent small refactoring efforts and small added functionality.
patch changes represent small fixes made in the codebase, but no change in overall functionality.
Required Files (All Tiers):
Dockerfile: Must be present in the repository root. This enables containerized execution and ensures maximum portability across different computing environments. The Dockerfile should be compatible with repo2docker or follow standard Docker practices.reproduce.sh: A script that runs and reproduces all the results. If this script fails, your reproduction is assumed to not have worked.README.md: A comprehensive README file documenting your REMARK. See tier-specific requirements for minimum line counts.LICENSE: A license file specifying the terms under which your code and content are distributed.binder/environment.yml: A conda environment file that can be used non-interactively to create an environment in whichreproduce.shruns successfully. For compatibility with the REMARK tooling (which currently builds environments withconda env update -f binder/environment.yml), this file MUST exist, but it may be either:a fully specified conda environment with exact version pins for every dependency (satisfying requirement 3 on its own); or
a minimal conda environment that installs your chosen environment manager (for example,
uv,poetry, orpip-tools) and any tools it needs to install dependencies from a separate lockfile (see requirement 3).You are encouraged to maintain your primary environment configuration in the format of your chosen tool (for example,
pyproject.toml/uv.lock,pyproject.toml/poetry.lock, orrequirements.in/requirements.txt). In this case,binder/environment.ymlserves as an adapter that ensures your tool is available inside the Binder/REMARK execution environment. See “Recommended:uvfor New Projects” below for a concrete example.Optionally, you may include any other files as part of the binder configuration specification
Pinned Dependencies (All Tiers):
All software dependencies must be pinned to specific versions so that future package upgrades cannot break the REMARK. Unpinned or loosely constrained dependencies (e.g.
numpy>=1.21) are not acceptable because a later release may change behaviour or break compatibility.You must commit a lockfile or equivalent that records exact resolved versions. Acceptable formats include:
uv.lock(recommended -- see below)poetry.lockA compiled
requirements.txtwith==pins for every package (produced bypip-compileoruv pip compile)A fully specified
binder/environment.ymlwith exact version pins for every dependency and sub-dependencyA
conda-lock.ymlgenerated by conda-lock
Whichever approach you choose,
binder/environment.ymlmust still exist as described above (it may simply be an adapter that installs your chosen tool).Optional Files:
reproduce_min.sh: You may include this file if your fullreproduce.shtakes ≥5 minutes to run on your local machine. This provides a quick validation path for testing.
Recommended: uv for New Projects¶
For new REMARKs we recommend uv as the
environment manager because:
uv.lockis a cross-platform lockfile that captures every resolved version, satisfying the pinning requirement above.uv sync --lockedguarantees bit-for-bit reproducible installs from the lockfile.Speed:
uvresolves and installs 10--100x faster thanpip, which makes iterating on environment changes painless.
A typical uv-based REMARK keeps:
| File | Role |
|---|---|
pyproject.toml | Primary dependency specification (human-editable) |
uv.lock | Lockfile with exact resolved versions (committed to repo) |
binder/environment.yml | Adapter that installs uv and runs uv sync --locked |
Example binder/environment.yml adapter for uv:
name: my-remark
channels:
- conda-forge
dependencies:
- python=3.12
- pip
- pip:
- uvAnd in reproduce.sh:
#!/bin/bash
uv sync --locked # install exact pinned versions
uv run python code/main.pyYou are free to use any tool (poetry, pip-tools, conda-lock, or a
fully pinned conda environment) provided it produces a committed lockfile
that satisfies requirement 3 above.
Minimal Repository Structure¶
.
|-- Dockerfile
|-- reproduce.sh
|-- reproduce_min.sh? # Optional
|-- README.md
|-- LICENSE
|-- CITATION.cff? # Required for Tier 2 and 3
|-- REMARK.md? # Required for Tier 2 and 3
|-- pyproject.toml? # If using uv/poetry (recommended)
|-- uv.lock? # Lockfile (or poetry.lock, requirements.txt, etc.)
`-- binder/
|-- environment.yml
# |-- {optional binder files}?
# ? indicates tier-specific or optional requirementsTier-Specific Requirements¶
Tier 1: Docker REMARK¶
Purpose: Minimal reproducibility via Docker containerization.
Requirements:
All base requirements listed above
README.md: Must be at least 50 lines (non-empty lines)CITATION.cff: Optional but recommended
Use Case: Suitable for exploratory work or early-stage projects that need basic containerization.
Tier 2: Reproducible REMARK¶
Purpose: Enhanced reproducibility with comprehensive documentation and metadata.
Requirements:
All Tier 1 requirements
README.md: Must be at least 100 lines (non-empty lines) with comprehensive documentationREMARK.md: Required metadata file that should specify the REMARK tier and include additional project metadataCITATION.cff: Required. A valid CITATION.cff document with bibliographic metadata for the repository. To create a CITATION.cff file, you may use the citation file format initializer
Use Case: Suitable for research projects that are ready for sharing and replication but not yet published.
Tier 3: Published REMARK¶
Purpose: Publication-ready with permanent archival and citation support.
Requirements:
All Tier 2 requirements
README.md: Must be at least 100 lines (non-empty lines) with comprehensive documentationREMARK.md: Must specifytier: 3in the metadataCITATION.cff: Must include citation information. DOI is recommended but not strictly required for Tier 3 LCD (Lowest Common Denominator) certification.Zenodo DOI: Obtain a permanent Zenodo DOI for your repository
Specific Git Tag: The Zenodo archive must correspond to a specific git tag (e.g.,
v1.0.0)Version Verification: The tag provides cryptographic proof that the econ-ark fork matches your Zenodo archive exactly
Complete Step-by-Step Guide: See Zenodo DOI Guide for REMARKs for detailed instructions with troubleshooting
When to obtain the Zenodo DOI: Do this only after your REMARK fully meets all Tier 2 requirements (reproduce.sh works, documentation complete, tagged release, etc.). Obtaining the Zenodo DOI should be the very last thing you do before submitting your REMARK for incorporation into the Econ-ARK ecosystem—so the archived version matches what you submit.
Quick Process Summary:
Create a git tag for the version to archive:
git tag -a v1.0.0 -m "Published version"Push the tag to GitHub:
git push origin v1.0.0Enable Zenodo-GitHub integration (see Zenodo DOI Guide for REMARKs for detailed setup)
Create a GitHub release from that tag, which triggers Zenodo archival
Obtain the Zenodo DOI from the archived version
Add the DOI to your CITATION.cff:
doi: 10.5281/zenodo.XXXXXXWhen submitting to econ-ark/REMARK, specify the exact git tag that was archived
Why This Matters:
The git tag creates an immutable snapshot at a specific commit
The commit SHA-1 hash provides cryptographic verification
Anyone can verify versions match by comparing commit hashes
This guarantees: econ-ark fork = Zenodo archive = your tagged version
Timeline: 30-45 minutes following the Zenodo DOI Guide for REMARKs checklist
Use Case: Suitable for research that is being submitted for publication or has been published, requiring permanent archival and citation support.
The kind Field¶
The catalog can list repositories that are not REMARKs. A repository says what kind of thing it is with the optional kind field in the front matter of its REMARK.md:
---
kind: canonical-model
---remarkis the kind this document describes: the base requirements and tiers above are what aremarkmust meet. A repository that declares no kind, or has noREMARK.md, is aremark, so an existing REMARK needs no change.Any other kind is defined by the project whose repositories are of that kind, not by this document. That project names the kind and provides the reader that reports on such a repository.
What the field changes: only how the catalog’s status table reports on the repository. A remark’s row gives the highest tier it meets; a row of another kind gives what that kind’s reader found. If no reader for a declared kind is installed, the row says so. See the Status Table section of the CLI guide for how a project provides a reader.
What it does not change: the kind field belongs in the repository’s REMARK.md, not in its catalog entry, and the pull, lint, build and execute commands treat an entry the same way whatever its kind.
Naming a kind: the name is matched exactly, capital letters included. Use lowercase words joined by hyphens, as in canonical-model.
Submitting a REMARK¶
What you do¶
Fork this repository (the REMARK catalog: github
.com /econ -ark /REMARK). Add a catalog entry for your REMARK: create or update a file under
REMARKs/(e.g.REMARKs/your-project-name.yml) that points to your repository. The entry stays minimal, for example:name: your-project-name remote: https://github.com/your-username/your-repo title: Your Project TitleUse your own GitHub (or organization) URL—the REMARK content stays under your account.
Open a Pull Request against the REMARK repository with that single change (the new or updated
.ymlfile).
Tier 3 (Published REMARK) authors: Complete all Tier 3 requirements and obtain your Zenodo DOI before opening the pull request. Getting the Zenodo DOI should be the last step before you submit—once you have the DOI and have added it to your CITATION.cff, you are ready to submit.
You do not transfer your repository to Econ-ARK. You do not need to fork your own repo into the econ-ark organization. You only add a pointer (the catalog entry) in the REMARK catalog.
What happens on acceptance¶
Once your Pull Request is accepted and merged:
The Econ-ARK team will create a fork of your repository under the econ-ark organization. That fork preserves the state of your repo at the point where it was tested and verified to work.
The catalog (and thus the website at econ
-ark .org /materials) will point to the fork, not your original repo, until there is a newly released version. That way users always see the tested, verified snapshot. You keep full ownership of your repository and can continue to develop it, fix bugs, or add features.
If you release an updated version (e.g. a new tag), submit a follow-up PR to the REMARK catalog. Econ-ARK will update the fork (and the catalog will continue to point to it) as long as
reproduce.shruns successfully and the revised draft still meets the requirements for your target tier.
So: you notify the REMARK team when you have a REMARK ready for inclusion by submitting the catalog PR; if it is assessed as meeting the requirements for your target tier, acceptance means it is listed, forked by Econ-ARK, and the website points to the fork until you submit a new version—at which point we will update as long as reproduction and compliance still hold.
Review and testing¶
When you open a Pull Request, the Econ-ARK team will review your submission and ensure that your reproduce.sh script runs successfully in the provided environment. If you would like to test this process on your own, you may clone this repository, add a temporary catalog entry for your REMARK, and build/execute it using the provided cli.py script.