Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

⚠️ IMPORTANT: The REMARK CLI tool (cli.py) is for REMARK VALIDATION AND TESTING, not website generation. The website (econ-ark.org) is generated by a separate system (populate_remarks.py).

🔧 CLI Tool vs 🌐 Website Generation

This CLI tool is used for:

This CLI tool is NOT used for:

The REMARK CLI tool (cli.py) provides automated operations for cloning, validating, building, and executing REMARKs. This tool is essential for maintainers and anyone working with multiple REMARKs.

Installation Requirements

# From REMARK repository root
pip install -r requirements.txt

Required dependencies:

As a package

cli.py is a thin launcher for the remark package beside it, so python cli.py ... works from a checkout with nothing installed. Installing the package adds a remark command, which acts on the catalog in the current directory (or --root <checkout>), and makes each step importable:

pip install .                                                   # from a checkout
pip install "econ-ark-remark @ git+https://github.com/econ-ark/REMARK@<tag>"   # pinned, elsewhere
remark --root /path/to/REMARK lint --tier 1 --all
from remark.catalog import load_catalog     # REMARKs/*.yml -> Metadata
from remark.git import pull_entry           # clone or update at the catalogued tag
from remark.lint import lint_repo           # (errors, warnings) for a tier
from remark.actions import build, execute   # each step, with its record
from remark.records import validate         # check a record against the schema

A project’s own runner can call execute for its jobs. Beyond the command line’s options it takes the script’s arguments, an environment directory, and the settings of a run that keeps its outputs apart: target (where the checkout is mounted), read_only, mounts (such as {'/out': job_folder}), tmpfs, variables and user ('host' for the invoking user).

The tests (pip install '.[test]', then pytest) run in CI on every pull request that touches the package; the container tests run wherever a docker daemon answers and are skipped elsewhere.

Core Commands

1. Pull/Clone REMARKs

Download REMARK repositories locally:

# Clone all REMARKs
python cli.py pull --all

# Clone specific REMARKs
python cli.py pull REMARKs/BufferStockTheory.yml REMARKs/beyond-the-streetlight.yml

# Pull updates for already cloned REMARKs
python cli.py pull --all  # Updates existing repos

What it does:

2. Lint/Validate REMARKs

Check REMARK compliance against standards:

# Check all REMARKs
python cli.py lint --all

# Check specific REMARKs
python cli.py lint REMARKs/BufferStockTheory.yml

# Include optional files in validation
python cli.py lint --all --include-optional

What it does:

🚨 CRITICAL: Lint results show REMARK COMPLIANCE (reproducibility), NOT website functionality:

Example output:

------------------ REMARKs/BufferStockTheory.yml ------------------
/path/to/_REMARK/repos/BufferStockTheory
- missing reproduce_min.sh
- missing binder/environment.yml

3. Build Environments

Create reproducible execution environments:

# Build conda environments for all REMARKs
python cli.py build conda --all

# Build docker images for specific REMARKs
python cli.py build docker REMARKs/BufferStockTheory.yml

# Use parallel processing (default: 4 jobs)
python cli.py build conda --all --jobs 8

Conda builds:

Docker builds:

Both kinds of build write an execution record.

An environment in a directory of its own (docker only). A repository can keep further environments beside its own, for instance one per solver, each in its own directory:

# Build, then run in, the environment kept in envs/solver of the REMARK
python cli.py build --environment envs/solver docker REMARKs/BufferStockTheory.yml
python cli.py execute --environment envs/solver docker REMARKs/BufferStockTheory.yml

4. Execute REMARKs

Run reproduction scripts in built environments:

# Execute all REMARKs using conda
python cli.py execute conda --all

# Execute specific REMARKs using docker
python cli.py execute docker REMARKs/BufferStockTheory.yml

# Force use of reproduce.sh (ignore reproduce_min.sh)
python cli.py execute conda --all --no-min

# Parallel execution
python cli.py execute conda --all --jobs 2

# Limit each container (docker only): no network, 2 CPUs, 8 GB, 2 BLAS/OpenMP threads, one hour
python cli.py execute --no-network --cpus 2 --memory 8g --threads 2 --timeout 3600 docker REMARKs/BufferStockTheory.yml

Put options before docker or conda: the parser does not accept an option between the type and the REMARK paths.

Script selection:

  1. Uses reproduce_min.sh if available (faster)

  2. Falls back to reproduce.sh

  3. --no-min forces reproduce.sh

How a docker execution runs:

Output:

5. View Logs

Check results from build/execute operations:

# Show all recent results
python cli.py logs

# Show results for specific REMARKs
python cli.py logs REMARKs/BufferStockTheory.yml

Example output:

build
BufferStockTheory         = 0
beyond-the-streetlight    = 1

execute  
BufferStockTheory         = 0
beyond-the-streetlight    = 0

Return codes: 0 = success, non-zero = failure

6. Clean Environments

Remove built environments and data:

# Remove conda environments
python cli.py clean conda --all

# Remove docker images: every image built for each REMARK, whatever its tag
python cli.py clean docker --all

# Remove cloned repositories
python cli.py clean repo --all

Execution Records

Every build and every execute writes a record of what it did, with its log beside it, in _REMARK/records/<name>/:

_REMARK/records/Aiyagari1994QJE/
├── 20261111T181502Z_build.json
├── 20261111T181502Z_build.log
├── 20261111T182240Z_execution.json
└── 20261111T182240Z_execution.log

A record holds facts read from git, the machine and the process, and nothing inferred:

The format is remark/schemas/execution-record.schema.json, and a record that does not match it is never written. A project’s own runner can write the same format and add fields of its own under extensions. Two limits to know:

A second execute starts from a checkout the first one wrote into, so its record says dirty; for a run of record, start from a fresh clone (clean repo, then pull).

Status Table

One command reads every entry and writes a table of what each repository holds. Nothing is run, and nothing in the table is kept by hand:

python cli.py status --all                 # every public entry
python cli.py status --all --private       # private entries too, where this machine can reach them
python cli.py status REMARKs/BufferStockTheory.yml

What it does:

What a row says comes from a reader chosen by the kind of unit, which a repository declares in the front matter of its REMARK.md:

---
kind: comparison
---
[project.entry-points."remark.status"]
comparison = "some_package.status:read"

A run that read a private entry says so at the top of status.md, in status.json (private_entries_read) and on the terminal: those files hold what a private repository contains and are not for publication.

Advanced Usage

Working with Specific Versions

If a REMARK metadata file specifies a tag field:

name: BufferStockTheory
remote: https://github.com/econ-ark/BufferStockTheory
title: Buffer Stock Theory
tag: v1.0.0

The CLI will automatically checkout that specific version.

Parallel Processing

Most operations support parallel execution:

# Build 8 REMARKs simultaneously
python cli.py build conda --all -J 8

# Execute with 2 parallel jobs (safer for memory-intensive REMARKs)
python cli.py execute docker --all --jobs 2

Environment Management

The CLI creates isolated environments:

_REMARK/
├── repos/                    # Cloned repositories
│   ├── BufferStockTheory/
│   │   ├── .condaenv/       # Conda environment (if built)
│   │   └── ...
│   └── beyond-the-streetlight/
├── status/                   # status.md and status.json, from `status`
├── logs/                     # The latest log and return code of each step
│   ├── build/
│   │   ├── BufferStockTheory_conda.log
│   │   └── BufferStockTheory_conda_rc.log
│   └── execute/
│       ├── BufferStockTheory_conda.log
│       └── BufferStockTheory_conda_rc.log
└── records/                  # A record and a log for every build and execution
    └── BufferStockTheory/

Error Handling

Common Issues

Missing files: Use lint to identify missing required files

python cli.py lint --all

Build failures: Check build logs

python cli.py logs
cat _REMARK/logs/build/ProjectName_conda.log

Docker issues: Ensure Docker daemon is running

docker --version
systemctl status docker  # On Linux

Conda issues: Verify conda installation

conda --version

Integration with GitHub Workflows

The CLI is designed to integrate with GitHub Actions:

- name: Validate REMARKs
  run: python cli.py lint --all

- name: Test REMARKs
  run: |
    python cli.py build conda --all
    python cli.py execute conda --all

Maintainer Workflows

Adding New REMARKs

  1. Validate compliance: python cli.py lint REMARKs/new-remark.yml

  2. Test build: python cli.py build conda REMARKs/new-remark.yml

  3. Test execution: python cli.py execute conda REMARKs/new-remark.yml

  4. Check logs: python cli.py logs REMARKs/new-remark.yml

Bulk Operations

# Full validation pipeline
python cli.py pull --all
python cli.py lint --all
python cli.py build conda --all
python cli.py execute conda --all
python cli.py logs

Performance Considerations


This CLI tool enables efficient management of the entire REMARK ecosystem, from validation through execution, with comprehensive logging and error handling.