Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

REMARK Guidelines

A REMARK refers to an open git repository that is indexed in this repository with appropriate metadata.

REMARKs are organized into three tiers based on their level of reproducibility and archival readiness:

Preparing your Repository

Base Requirements (All Tiers)

All REMARKs must meet these fundamental requirements:

  1. Have a tagged release.

    • If you make changes to your work, please release a new version/tag.

    • Commonly, tags will follow semantic versioning having a 3 part numeric identifier v{major}.{minor}.{patch} where:

      • major version changes represent large refactors, new features, and/or new analyses.

      • minor version changes represent small refactoring efforts and small added functionality.

      • patch changes represent small fixes made in the codebase, but no change in overall functionality.

  2. Required Files (All Tiers):

    • Dockerfile: Must be present in the repository root. This enables containerized execution and ensures maximum portability across different computing environments. The Dockerfile should be compatible with repo2docker or follow standard Docker practices.

    • reproduce.sh: A script that runs and reproduces all the results. If this script fails, your reproduction is assumed to not have worked.

    • README.md: A comprehensive README file documenting your REMARK. See tier-specific requirements for minimum line counts.

    • LICENSE: A license file specifying the terms under which your code and content are distributed.

    • binder/environment.yml: A conda environment file that can be used non-interactively to create an environment in which reproduce.sh runs successfully. For compatibility with the REMARK tooling (which currently builds environments with conda env update -f binder/environment.yml), this file MUST exist, but it may be either:

      • a fully specified conda environment with exact version pins for every dependency (satisfying requirement 3 on its own); or

      • a minimal conda environment that installs your chosen environment manager (for example, uv, poetry, or pip-tools) and any tools it needs to install dependencies from a separate lockfile (see requirement 3).

      • You are encouraged to maintain your primary environment configuration in the format of your chosen tool (for example, pyproject.toml/uv.lock, pyproject.toml/poetry.lock, or requirements.in/requirements.txt). In this case, binder/environment.yml serves as an adapter that ensures your tool is available inside the Binder/REMARK execution environment. See “Recommended: uv for New Projects” below for a concrete example.

      • Optionally, you may include any other files as part of the binder configuration specification

  3. Pinned Dependencies (All Tiers):

    All software dependencies must be pinned to specific versions so that future package upgrades cannot break the REMARK. Unpinned or loosely constrained dependencies (e.g. numpy>=1.21) are not acceptable because a later release may change behaviour or break compatibility.

    You must commit a lockfile or equivalent that records exact resolved versions. Acceptable formats include:

    • uv.lock (recommended -- see below)

    • poetry.lock

    • A compiled requirements.txt with == pins for every package (produced by pip-compile or uv pip compile)

    • A fully specified binder/environment.yml with exact version pins for every dependency and sub-dependency

    • A conda-lock.yml generated by conda-lock

    Whichever approach you choose, binder/environment.yml must still exist as described above (it may simply be an adapter that installs your chosen tool).

  4. Optional Files:

    • reproduce_min.sh: You may include this file if your full reproduce.sh takes ≥5 minutes to run on your local machine. This provides a quick validation path for testing.

For new REMARKs we recommend uv as the environment manager because:

A typical uv-based REMARK keeps:

FileRole
pyproject.tomlPrimary dependency specification (human-editable)
uv.lockLockfile with exact resolved versions (committed to repo)
binder/environment.ymlAdapter that installs uv and runs uv sync --locked

Example binder/environment.yml adapter for uv:

name: my-remark
channels:
  - conda-forge
dependencies:
  - python=3.12
  - pip
  - pip:
    - uv

And in reproduce.sh:

#!/bin/bash
uv sync --locked          # install exact pinned versions
uv run python code/main.py

You are free to use any tool (poetry, pip-tools, conda-lock, or a fully pinned conda environment) provided it produces a committed lockfile that satisfies requirement 3 above.

Minimal Repository Structure

.
|-- Dockerfile
|-- reproduce.sh
|-- reproduce_min.sh?      # Optional
|-- README.md
|-- LICENSE
|-- CITATION.cff?          # Required for Tier 2 and 3
|-- REMARK.md?             # Required for Tier 2 and 3
|-- pyproject.toml?        # If using uv/poetry (recommended)
|-- uv.lock?               # Lockfile (or poetry.lock, requirements.txt, etc.)
`-- binder/
    |-- environment.yml
#   |-- {optional binder files}?

# ? indicates tier-specific or optional requirements

Tier-Specific Requirements

Tier 1: Docker REMARK

Purpose: Minimal reproducibility via Docker containerization.

Requirements:

Use Case: Suitable for exploratory work or early-stage projects that need basic containerization.


Tier 2: Reproducible REMARK

Purpose: Enhanced reproducibility with comprehensive documentation and metadata.

Requirements:

Use Case: Suitable for research projects that are ready for sharing and replication but not yet published.


Tier 3: Published REMARK

Purpose: Publication-ready with permanent archival and citation support.

Requirements:

Complete Step-by-Step Guide: See Zenodo DOI Guide for REMARKs for detailed instructions with troubleshooting

When to obtain the Zenodo DOI: Do this only after your REMARK fully meets all Tier 2 requirements (reproduce.sh works, documentation complete, tagged release, etc.). Obtaining the Zenodo DOI should be the very last thing you do before submitting your REMARK for incorporation into the Econ-ARK ecosystem—so the archived version matches what you submit.

Quick Process Summary:

  1. Create a git tag for the version to archive: git tag -a v1.0.0 -m "Published version"

  2. Push the tag to GitHub: git push origin v1.0.0

  3. Enable Zenodo-GitHub integration (see Zenodo DOI Guide for REMARKs for detailed setup)

  4. Create a GitHub release from that tag, which triggers Zenodo archival

  5. Obtain the Zenodo DOI from the archived version

  6. Add the DOI to your CITATION.cff: doi: 10.5281/zenodo.XXXXXX

  7. When submitting to econ-ark/REMARK, specify the exact git tag that was archived

Why This Matters:

Timeline: 30-45 minutes following the Zenodo DOI Guide for REMARKs checklist

Use Case: Suitable for research that is being submitted for publication or has been published, requiring permanent archival and citation support.

The kind Field

The catalog can list repositories that are not REMARKs. A repository says what kind of thing it is with the optional kind field in the front matter of its REMARK.md:

---
kind: canonical-model
---

What the field changes: only how the catalog’s status table reports on the repository. A remark’s row gives the highest tier it meets; a row of another kind gives what that kind’s reader found. If no reader for a declared kind is installed, the row says so. See the Status Table section of the CLI guide for how a project provides a reader.

What it does not change: the kind field belongs in the repository’s REMARK.md, not in its catalog entry, and the pull, lint, build and execute commands treat an entry the same way whatever its kind.

Naming a kind: the name is matched exactly, capital letters included. Use lowercase words joined by hyphens, as in canonical-model.

Submitting a REMARK

What you do

  1. Fork this repository (the REMARK catalog: github.com/econ-ark/REMARK).

  2. Add a catalog entry for your REMARK: create or update a file under REMARKs/ (e.g. REMARKs/your-project-name.yml) that points to your repository. The entry stays minimal, for example:

    name: your-project-name
    remote: https://github.com/your-username/your-repo
    title: Your Project Title

    Use your own GitHub (or organization) URL—the REMARK content stays under your account.

  3. Open a Pull Request against the REMARK repository with that single change (the new or updated .yml file).

Tier 3 (Published REMARK) authors: Complete all Tier 3 requirements and obtain your Zenodo DOI before opening the pull request. Getting the Zenodo DOI should be the last step before you submit—once you have the DOI and have added it to your CITATION.cff, you are ready to submit.

You do not transfer your repository to Econ-ARK. You do not need to fork your own repo into the econ-ark organization. You only add a pointer (the catalog entry) in the REMARK catalog.

What happens on acceptance

Once your Pull Request is accepted and merged:

So: you notify the REMARK team when you have a REMARK ready for inclusion by submitting the catalog PR; if it is assessed as meeting the requirements for your target tier, acceptance means it is listed, forked by Econ-ARK, and the website points to the fork until you submit a new version—at which point we will update as long as reproduction and compliance still hold.

Review and testing

When you open a Pull Request, the Econ-ARK team will review your submission and ensure that your reproduce.sh script runs successfully in the provided environment. If you would like to test this process on your own, you may clone this repository, add a temporary catalog entry for your REMARK, and build/execute it using the provided cli.py script.