CLI#

The ref command-line interface (CLI) is the primary way to interact with the Climate-REF framework. This CLI tool is installed as part of the climate-ref package and provides commands for managing configurations, datasets, diagnostics, and more.

app#

A CLI for the Assessment Fast Track Rapid Evaluation Framework

This CLI provides a number of commands for managing and executing diagnostics.

Usage#

app [OPTIONS] COMMAND [ARGS]...

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--configuration-directory`	Configuration directory	No	-
`-v, --verbose`	Set the log level to DEBUG	No	-
`-q, --quiet`	Set the log level to WARNING	No	-
`--log-level`	Set the level of logging information to display	No	-
`--version`	Print the version and exit	No	-
`--install-completion`	Install completion for the current shell.	No	-
`--show-completion`	Show completion for the current shell, to copy it or customize the installation.	No	-
`--help`	Show this message and exit.	No	-

Commands#

Name	Description
`solve`	Solve for executions that require...
`doctor`	Check this deployment for data and...
`config`	View and update the REF configuration
`datasets`	View and ingest input datasets
`db`	Database management commands
`diagnostics`	View diagnostic metadata
`executions`	View execution groups and their results
`providers`	Manage the REF providers.
`test-cases`	Test data management commands for...
`celery`	Managing remote celery workers

Subcommands#

`app solve`#

Solve for executions that require recalculation

This may trigger a number of additional calculations depending on what data has been ingested since the last solve. This command will block until all executions have been solved or the timeout is reached.

Filters can be applied to limit the diagnostics and providers that are considered, see the options --diagnostic and --provider for more information.

Usage#

app solve [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--dry-run / --no-dry-run`	Do not execute any diagnostics [default: no-dry-run]	No	-
`--execute / --no-execute`	Solve the newly identified executions [default: execute]	No	-
`--timeout`	Timeout in seconds for waiting on executions to complete. Defaults to 6 hours. Set to 0 (or a negative value) to wait with no time limit. Ignored when --no-wait is used. [default: 21600]	No	-
`--wait / --no-wait`	Wait for executions to complete before exiting. Use --no-wait to queue executions and exit immediately. [default: wait]	No	-
`--one-per-provider / --no-one-per-provider`	Limit to one execution per provider. This is useful for testing [default: no-one-per-provider]	No	-
`--one-per-diagnostic / --no-one-per-diagnostic`	Limit to one execution per diagnostic. This is useful for testing [default: no-one-per-diagnostic]	No	-
`--diagnostic`	Filters executions by the diagnostic slug. Diagnostics will be included if any of the filters match a case-insensitive subset of the diagnostic slug. Multiple values can be provided	No	-
`--provider`	Filters executions by provider slug. Providers will be included if any of the filters match a case-insensitive subset of the provider slug. Multiple values can be provided	No	-
`--exclude-diagnostic`	Excludes diagnostics by slug or provider/diagnostic pair. Matching is exact and case-insensitive, unlike --diagnostic. For example, --exclude-diagnostic ilamb/thetao-woa2023-surface. Multiple values can be provided	No	-
`--dataset-filter`	Filter input datasets by facet values using key=value syntax. For example, --dataset-filter source_id=ACCESS-CM2 --dataset-filter variable_id=tas. Multiple values for the same facet are ORed (include any match), different facets are ANDed (must match all). Multiple values can be provided	No	-
`--limit`	Maximum number of executions to run. If not set, all executions are run.	No	-
`--rerun-failed / --no-rerun-failed`	Re-run all previously failed executions, even if the execution group is not dirty. By default, failed executions are only retried if explicitly flagged dirty. [default: no-rerun-failed]	No	-
`--check-health / --no-check-health`	Health check each provider before solving and skip the diagnostics of unhealthy ones. The command then exits non-zero. Only applies to executors that run diagnostics on this host. [default: check-health]	No	-
`--help`	Show this message and exit.	No	-

`app doctor`#

Check this deployment for data and configuration problems.

Use --format markdown to produce a report, including a description of this deployment that can be pasted into a bug report.

Exits non-zero if any error is found, or any warning when --strict is used.

Usage#

app doctor [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--format`	Output format: 'text' (default), 'markdown' to paste into an issue, or 'json'. [default: text]	No	-
`--verbose / --quiet`	Include the explanation and remedy for each finding. [default: verbose]	No	-
`--strict / --no-strict`	Exit non-zero for warnings as well as errors. [default: no-strict]	No	-
`--environment / --no-environment`	Include a description of this deployment. On by default for the markdown and json formats.	No	-
`--list`	List the available checks and where they came from, then exit.	No	-
`--help`	Show this message and exit.	No	-

`app config`#

View and update the REF configuration

Usage#

app config [OPTIONS] COMMAND [ARGS]...

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

Subcommands#

`app config list`#

Print the current climate_ref configuration

If a configuration directory is provided, the configuration will attempt to load from the specified directory.

Usage#

app config list [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--format`	Output format: 'toml' (default) or machine-readable 'json'. [default: toml]	No	-
`--help`	Show this message and exit.	No	-

`app config init`#

Create a ref.toml configuration file for onboarding.

By default this writes a complete template with defaults included. --force is destructive: it replaces any existing configuration file with a fresh template.

Usage#

app config init [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--force`	Overwrite an existing ref.toml with a fresh template.	No	-
`--no-defaults`	Write a minimal template that omits default values.	No	-
`--help`	Show this message and exit.	No	-

`app config get`#

Print one effective configuration value.

Environment variables are already applied, so this shows the value the REF will use at runtime. Scalar values are printed alone on stdout for scripts; structured sections are printed as TOML.

Usage#

app config get [OPTIONS] {key}

Arguments#

Name	Description	Required
`key`	Dotted configuration key	Yes

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

`app config set`#

Set one scalar configuration value and persist it to ref.toml.

The value is written to the file-backed configuration, even when environment variables shadow the effective runtime value.

Usage#

app config set [OPTIONS] {key} {value}

Arguments#

Name	Description	Required
`key`	Dotted scalar configuration key	Yes
`value`	New value	Yes

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

`app config unset`#

Reset one scalar configuration value to its default and persist it to ref.toml.

Structured values such as provider lists and executor config dictionaries must be edited directly.

Usage#

app config unset [OPTIONS] {key}

Arguments#

Name	Description	Required
`key`	Dotted scalar configuration key	Yes

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

`app config edit`#

Open ref.toml in the user's editor and warn if the result is invalid.

The editor is selected by Click from $VISUAL / $EDITOR. If no config file exists yet, run ref config init first.

Usage#

app config edit [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

`app config validate`#

Validate that ref.toml parses and matches the REF configuration schema.

The exit code is the CI contract: 0 means valid and 1 means invalid. JSON output includes a Terraform-style valid flag, error count and diagnostics list.

Usage#

app config validate [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--format`	Output format: 'text' (default) or machine-readable 'json'. [default: text]	No	-
`--help`	Show this message and exit.	No	-

`app datasets`#

View and ingest input datasets

The metadata from these datasets are stored in the database so that they can be used to determine which executions are required for a given diagnostic without having to re-parse the datasets.

Usage#

app datasets [OPTIONS] COMMAND [ARGS]...

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

Subcommands#

`app datasets list`#

List the datasets that have been ingested

The data catalog is sorted by the date that the dataset was ingested (first = newest).

By default only the latest, non-retracted version of each dataset is listed, matching what the solver would select. Pass --include-retracted to also see retracted datasets (see the retracted_at column). This also disables latest-only deduplication so every version is listed, since otherwise a retracted newer version could outrank and hide a live older one.

Usage#

app datasets list [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--source-type`	Type of source dataset [default: cmip6]	No	-
`--include-files / --no-include-files`	Include files in the output [default: no-include-files]	No	-
`--include-retracted / --no-include-retracted`	Include retracted datasets in the output. Off by default, so the listing matches solve-time eligibility, and a retracted dataset is never shown masking the live version that replaced it. Passing this flag also disables latest-only deduplication, so every version of every dataset is listed. Keeping the deduplication on would let a retracted newer version outrank and hide a live older one. [default: no-include-retracted]	No	-
`--limit`	Limit the number of datasets (or files when using --include-files) to display to this number. [default: 100; x>=0]	No	-
`--dataset-filter`	Filter datasets by facet values using key=value syntax. For example, --dataset-filter source_id=ACCESS-CM2 --dataset-filter variable_id=tas. Multiple values for the same facet are ORed (include any match), different facets are ANDed (must match all). Multiple values can be provided	No	-
`--format`	Output format: 'table' (default) or machine-readable 'json'. [default: table]	No	-
`--help`	Show this message and exit.	No	-

`app datasets list-columns`#

List the available columns in the data catalog for the given source type.

Usage#

app datasets list-columns [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--source-type`	Type of source dataset [default: cmip6]	No	-
`--include-files / --no-include-files`	Include files in the output [default: no-include-files]	No	-
`--help`	Show this message and exit.	No	-

`app datasets retract`#

Retract a dataset so it is excluded from future solve-time selection.

The dataset's row and files are left intact for provenance and historical execution inspection. Only its eligibility to satisfy a diagnostic's data requirements changes. There is no hard-delete path (a dataset that has ever run cannot be removed without either violating a foreign key or destroying execution provenance), so this is the only way to stop a superseded dataset (e.g. an old obs4MIPs row re-ingested under obs4REF) from continuing to match diagnostics.

Retracting an already-retracted dataset is a no-op.

Usage#

app datasets retract [OPTIONS] {slug}

Arguments#

Name	Description	Required
`slug`	[required]	No

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

`app datasets unretract`#

Restore a retracted dataset to solve-eligibility.

The inverse of retract. Unretracting a dataset that is not currently retracted is a no-op.

Usage#

app datasets unretract [OPTIONS] {slug}

Arguments#

Name	Description	Required
`slug`	[required]	No

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

`app datasets ingest`#

Ingest a directory of datasets into the database

Each dataset will be loaded and validated using the specified dataset adapter. This will extract metadata from the datasets and store it in the database.

A table of the datasets will be printed to the console at the end of the operation.

Usage#

app datasets ingest [OPTIONS] {file_or_directory}...

Arguments#

Name	Description	Required
`file_or_directory...`	[required]	No

Options#

Name	Description	Required	Default
`--source-type`	Type of source dataset	Yes	-
`--solve / --no-solve`	Solve for new diagnostic executions after ingestion [default: no-solve]	No	-
`--dry-run / --no-dry-run`	Do not ingest datasets into the database [default: no-dry-run]	No	-
`--n-jobs`	Number of jobs to run in parallel. Defaults to the `n_jobs` configuration value	No	-
`--skip-invalid / --no-skip-invalid`	Ignore (but log) any datasets that don't pass validation [default: skip-invalid]	No	-
`--chunk-size`	Stream the catalog in chunks of this many files instead of loading the whole directory at once. Bounds peak memory for large archives. Only supported by adapters that implement iter_local_datasets (all built-in adapters).	No	-
`--check-health / --no-check-health`	Health check providers before solving, as `ref solve --check-health` does [default: check-health]	No	-
`--skip-existing / --no-skip-existing`	Skip files that are already in the database without parsing them. Assumes files are immutable, so run a full ingest periodically to pick up changes. [default: no-skip-existing]	No	-
`--help`	Show this message and exit.	No	-

`app datasets stats`#

Show summary statistics for datasets.

Displays counts of datasets grouped by dataset type, with finalisation status breakdown and file counts. Optionally expand by a facet using --group-by (e.g., --group-by source_id).

Usage#

app datasets stats [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--source-type`	Filter by dataset type	No	-
`--group-by`	Group results by a dataset facet. Allowed values: source_id, variable_id. Requires --source-type to be specified.	No	-
`--help`	Show this message and exit.	No	-

`app datasets fetch-sample-data`#

Fetch the sample data for the given version.

These data will be written into the test data directory. This operation may fail if the test data directory does not exist, as is the case for non-source-based installations.

Usage#

app datasets fetch-sample-data [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--force-cleanup / --no-force-cleanup`	If True, remove any existing files [default: no-force-cleanup]	No	-
`--symlink / --no-symlink`	If True, symlink files into the output directory, otherwise perform a copy [default: no-symlink]	No	-
`--help`	Show this message and exit.	No	-

`app datasets fetch-data`#

Fetch REF-specific datasets

These datasets have been verified to have open licenses and are in the process of being added to Obs4MIPs.

Usage#

app datasets fetch-data [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--registry`	Name of the data registry to use	Yes	-
`--output-directory`	Output directory where files will be saved	No	-
`--force-cleanup / --no-force-cleanup`	If True, remove any existing files [default: no-force-cleanup]	No	-
`--symlink / --no-symlink`	If True, symlink files into the output directory, otherwise perform a copy [default: no-symlink]	No	-
`--verify / --no-verify`	Verify the checksums of the fetched files [default: verify]	No	-
`--help`	Show this message and exit.	No	-

`app db`#

Database management commands

Usage#

app db [OPTIONS] COMMAND [ARGS]...

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

Subcommands#

`app db migrate`#

Run database migrations to bring the schema up to date.

This applies any pending Alembic migrations. A backup is created before migrating (SQLite only).

Usage#

app db migrate [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

`app db status`#

Check if the database schema is up to date.

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

`app db sql`#

Execute an arbitrary SQL query against the database.

SELECT queries display results as a table (default limit: 100 rows). Other statements report the number of rows affected.

Usage#

app db sql [OPTIONS] {query}

Arguments#

Name	Description	Required
`query`	SQL query to execute	Yes

Options#

Name	Description	Required	Default
`-l, --limit`	Maximum number of rows to display [default: 100]	No	-
`--help`	Show this message and exit.	No	-

`app db tables`#

List all tables in the database.

Usage#

app db tables [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

`app diagnostics`#

View diagnostic metadata

Usage#

app diagnostics [OPTIONS] COMMAND [ARGS]...

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

Subcommands#

`app diagnostics list`#

List the registered diagnostics.

Columns:

provider: the provider slug.
diagnostic: the diagnostic slug.
name: the human-readable diagnostic name.
promoted_version: the currently promoted diagnostic version.
group_count: execution groups at the promoted version.
successful_group_count: groups whose latest execution succeeded.
failed_group_count: groups whose latest execution failed.
running_group_count: groups whose latest execution is still running.
not_started_group_count: groups with no execution yet.
dirty_group_count: groups marked to be rerun.
execution_count: number of executions at the promoted version, counting every rerun.

Usage#

app diagnostics list [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--provider`	Filter by provider slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--diagnostic`	Filter by diagnostic slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--column`	Only include specified columns in the output	No	-
`--limit`	Limit the number of rows to display [default: 100; x>=0]	No	-
`--format`	Output format: 'table' (default) or machine-readable 'json'. [default: table]	No	-
`--help`	Show this message and exit.	No	-

`app executions`#

View execution groups and their results

Usage#

app executions [OPTIONS] COMMAND [ARGS]...

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

Subcommands#

`app executions list-groups`#

List the diagnostic execution groups that have been identified

Groups are sorted by id, so the oldest comes first. Only groups at each diagnostic's promoted version are listed, and a group whose newest execution is retracted is left out. With --since, only groups that ran since then are listed, judged by their newest run in that time.

By default the table shows a subset of columns with the verbose selectors and updated_at fields omitted. Use --column to choose exactly which columns to display, or --format json to emit the full record for scripting.

Filters can be combined using AND logic across filter types and OR logic within a filter type.

The output is a table by default, or machine-readable JSON with --format json.

Usage#

app executions list-groups [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--column`	Only include specified columns in the output	No	-
`--limit`	Limit the number of rows to display [default: 100; x>=0]	No	-
`--diagnostic`	Filter by diagnostic slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--provider`	Filter by provider slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--since`	Only consider executions newer than this age (90d, 12h, 4w) or ISO date.	No	-
`--filter`	Filter by group selector key=value pairs (exact match). Multiple filters can be provided.	No	-
`--successful / --not-successful`	Filter by successful or unsuccessful executions.	No	-
`--dirty / --not-dirty`	Filter to include only dirty or clean execution groups.These execution groups will be re-computed on the next run.	No	-
`--failure-reason`	Filter to groups whose latest execution failed for this reason. Multiple values can be provided.	No	-
`--format`	Output format: 'table' (default) or machine-readable 'json'. [default: table]	No	-
`--help`	Show this message and exit.	No	-

`app executions delete-groups`#

Delete execution groups matching the specified filters.

This command will delete execution groups and their associated executions. Use filters to specify which groups to delete. At least one filter must be provided to prevent accidental deletion of all groups.

Only groups for each diagnostic's promoted version are deleted unless --include-superseded is given. Groups whose latest execution is retracted are included.

Filters can be combined using AND logic across filter types and OR logic within a filter type.

Usage#

app executions delete-groups [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--diagnostic`	Filter by diagnostic slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--provider`	Filter by provider slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--filter`	Filter by group selector key=value pairs (exact match). Multiple filters can be provided.	No	-
`--successful / --not-successful`	Filter by successful or unsuccessful executions.	No	-
`--dirty / --not-dirty`	Filter to include only dirty or clean execution groups.These execution groups will be re-computed on the next run.	No	-
`--include-superseded`	Also delete groups for diagnostic versions other than the promoted one.	No	-
`--remove-outputs`	Also remove output directories from the filesystem	No	-
`--dry-run`	Show what would be deleted without making changes.	No	-
`--force / --no-force`	Skip confirmation prompt [default: no-force]	No	-
`--help`	Show this message and exit.	No	-

`app executions inspect`#

Inspect a specific execution group by its ID

This will display the execution details, datasets, results directory, and logs if available. The details are for the group's newest execution, even when it is retracted.

Usage#

app executions inspect [OPTIONS] {group_id}

Arguments#

Name	Description	Required
`group_id`	Execution group ID to inspect.	Yes

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

`app executions values`#

Show computed metric values for an execution group.

By default this shows the scalar values from the latest execution in the group. Use --kind series for 1-d series, --execution-id to target a specific execution, and --dimension key=value to filter by controlled-vocabulary dimensions.

Values are read through the shared climate_ref.results layer, so the same filtering is applied here as in the API.

Usage#

app executions values [OPTIONS] {group_id}

Arguments#

Name	Description	Required
`group_id`	Execution group ID to show metric values for.	Yes

Options#

Name	Description	Required	Default
`--kind`	Which kind of value to show: scalar (default) or series. [default: scalar]	No	-
`--execution-id`	Show values for a specific execution instead of the latest one in the group.	No	-
`-d, --dimension`	Filter by CV dimension key=value (exact match). Repeat for AND across keys; repeat the same key for OR within a key.	No	-
`--outliers / --no-outliers`	Run source-id-aware IQR outlier detection (scalar values only). [default: no-outliers]	No	-
`--include-unverified`	Include values flagged as outliers (only meaningful with --outliers).	No	-
`--limit`	Maximum number of values to display. [default: 100; x>=0]	No	-
`--offset`	Number of values to skip before displaying. [default: 0; x>=0]	No	-
`--format`	Output format: table (default) or json. [default: table]	No	-
`--help`	Show this message and exit.	No	-

`app executions fail-running`#

Mark running executions as failed.

Running executions (those with no success/failure status) block their execution group from being requeued. Use this command as an escape hatch to fail stuck executions so they can be retried on the next solve.

An optional age threshold can be provided with --older-than to only fail executions that have been running for longer than the specified number of hours.

Usage#

app executions fail-running [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--older-than`	Only fail executions older than this many hours. If not specified, all running executions are failed.	No	-
`--diagnostic`	Filter by diagnostic slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--provider`	Filter by provider slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--force / --no-force`	Skip confirmation prompt [default: no-force]	No	-
`--help`	Show this message and exit.	No	-

`app executions stats`#

Show how many execution groups each diagnostic has, by the status of their newest execution.

The statuses are running, failed, successful and not started, plus a dirty count that overlaps them. Only groups at each diagnostic's promoted version are counted, and retracted executions are skipped. Diagnostics with no groups are listed with zero counts, and each provider gets a (total) row. With --since, only groups that ran since then are counted, by their newest run in that time.

Usage#

app executions stats [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--diagnostic`	Filter by diagnostic slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--provider`	Filter by provider slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--since`	Only consider executions newer than this age (90d, 12h, 4w) or ISO date.	No	-
`--help`	Show this message and exit.	No	-

`app executions resources`#

Show measured resource usage and a recommended worker size.

One row per diagnostic by default, sorted by recommended memory descending. Use --by provider for the roll-up a worker actually has to fit, which takes the largest memory over the provider's diagnostics.

Three columns qualify the recommendation and should be read with it. The OVER flag marks a p95 peak that has outgrown the recorded memory limit. The fail column counts failed executions, which record nothing when they are killed, so a high count means the numbers here are lower than the truth. The conf column reports how many samples the percentiles rest on.

Usage#

app executions resources [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--diagnostic`	Filter by diagnostic slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--provider`	Filter by provider slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--by`	Aggregate per diagnostic (default) or roll up per provider. [default: diagnostic]	No	-
`--since`	Only consider executions newer than this age (90d, 12h, 4w) or ISO date.	No	-
`--safety`	Multiplier applied to the p95 peak when recommending a memory size. [default: 1.3]	No	-
`--include-shared`	Also count cgroup readings taken while other executions shared the worker. Those readings cover the whole worker, not the execution.	No	-
`--format`	Output format: table (default) or json. [default: table]	No	-
`--help`	Show this message and exit.	No	-

`app executions reingest`#

Reingest existing executions without re-running diagnostics.

Re-runs build_execution_result() on existing output files and re-ingests the results into the database. Useful when new series definitions or metadata extraction logic has been added.

A new Execution record is always created under the same ExecutionGroup, leaving the original execution untouched. Results are treated as immutable.

The dirty flag is never modified by this command.

Usage#

app executions reingest [OPTIONS][group_ids]...

Arguments#

Name	Description	Required
`group_ids...`	Execution group IDs to reingest. If omitted, uses filters.	No

Options#

Name	Description	Required	Default
`--diagnostic`	Filter by diagnostic slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--provider`	Filter by provider slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--include-failed`	Also attempt reingest on failed executions.	No	-
`--dry-run`	Show what would be reingested without making changes.	No	-
`--force / --no-force`	Skip confirmation prompt [default: no-force]	No	-
`--help`	Show this message and exit.	No	-

`app executions flag-dirty`#

Flag an execution group for recomputation

Usage#

app executions flag-dirty [OPTIONS] {execution_id}

Arguments#

Name	Description	Required
`execution_id`	[required]	No

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

`app executions requeue`#

Flag failed execution groups as dirty so the next solve reruns them.

Only groups whose latest execution failed and that are not already dirty are selected. Use --failure-reason to requeue only one kind of failure, e.g. --failure-reason system_error after fixing a broken environment. Use --since to requeue only groups whose latest execution ran since then, e.g. --since 2d.

Usage#

app executions requeue [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--failure-reason`	Only requeue groups whose latest execution failed for this reason. Multiple values can be provided.	No	-
`--diagnostic`	Filter by diagnostic slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--provider`	Filter by provider slug (substring match, case-insensitive). Multiple values can be provided.	No	-
`--since`	Only consider executions newer than this age (90d, 12h, 4w) or ISO date.	No	-
`--filter`	Filter by group selector key=value pairs (exact match). Multiple filters can be provided.	No	-
`--dry-run`	Show what would be requeued without making changes.	No	-
`--force / --no-force`	Skip confirmation prompt [default: no-force]	No	-
`--help`	Show this message and exit.	No	-

`app providers`#

Manage the REF providers.

Usage#

app providers [OPTIONS] COMMAND [ARGS]...

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

Subcommands#

`app providers list`#

Print the available providers.

Usage#

app providers list [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--format`	Output format: 'table' (default) or machine-readable 'json'. [default: table]	No	-
`--help`	Show this message and exit.	No	-

`app providers show`#

Show diagnostics and data requirements for a provider.

Usage#

app providers show [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--provider`	Slug of the provider to show diagnostics for.	Yes	-
`--format`	Output format: 'list' for detailed per-diagnostic output, 'table' for a compact table. [default: list]	No	-
`--columns`	Columns to include in table output (e.g. --columns diagnostic --columns variables).	No	-
`--help`	Show this message and exit.	No	-

`app providers create-env`#

Create a conda environment containing the provider software.

.. deprecated:: Use ref providers setup instead, which handles both environment creation and data fetching in a single command.

If no provider is specified, all providers will be installed. If the provider is up to date or does not use a conda environment, it will be skipped.

Usage#

app providers create-env [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--provider`	Only install the environment for the named provider.	No	-
`--help`	Show this message and exit.	No	-

`app providers setup`#

Run provider setup for offline execution.

This command prepares all providers for offline execution by:

Creating conda environments (if applicable)
Fetching required reference datasets to pooch cache
Ingesting provider-specific datasets into the database

All operations are idempotent and safe to run multiple times. Run this on a login node with internet access before solving on compute nodes.

Usage#

app providers setup [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--provider`	Only run setup for the named provider.	No	-
`--skip-env / --no-skip-env`	Skip environment setup (e.g., conda). [default: no-skip-env]	No	-
`--skip-data / --no-skip-data`	Skip data fetching and ingestion. [default: no-skip-data]	No	-
`--skip-validate / --no-skip-validate`	Skip validation. [default: no-skip-validate]	No	-
`--validate-only / --no-validate-only`	Only validate setup, don't run it. [default: no-validate-only]	No	-
`--help`	Show this message and exit.	No	-

`app providers check-health`#

Check that each provider can launch a diagnostic on this host.

This is the check ref solve runs before solving. It is cheaper than ref providers setup --validate-only, which also verifies reference data. Exits non-zero when any provider is unhealthy.

Usage#

app providers check-health [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--provider`	Only check providers whose slug contains this value. Multiple values can be provided.	No	-
`--help`	Show this message and exit.	No	-

`app test-cases`#

Test data management commands for diagnostic development.

These commands are intended for diagnostic developers and require a source checkout of the project with test data directories available.

Usage#

app test-cases [OPTIONS] COMMAND [ARGS]...

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

Subcommands#

`app test-cases fetch`#

Fetch test data from ESGF for running diagnostic tests.

Downloads full-resolution ESGF data based on diagnostic test_data_spec. Use --provider or --diagnostic to limit scope.

A test case that already has a catalog is fetched by the datasets that catalog records, so it keeps the inputs its regression baseline was built from. Use --regen to resolve the requests from their declared facets instead.

Examples#

ref test-cases fetch                   # Fetch all test data
ref test-cases fetch --provider ilamb  # Fetch ILAMB test data only
ref test-cases fetch --diagnostic ecs  # Fetch ECS diagnostic data
ref test-cases fetch --only-missing    # Skip test cases with existing catalogs
ref test-cases fetch --regen           # Re-resolve datasets from the declared facets

Usage#

app test-cases fetch [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--provider`	Specific provider to fetch data for (e.g., 'esmvaltool', 'ilamb')	No	-
`--diagnostic`	Specific diagnostic slug to fetch data for	No	-
`--test-case`	Specific test case name to fetch data for	No	-
`--dry-run / --no-dry-run`	Show what would be fetched without downloading [default: no-dry-run]	No	-
`--only-missing / --no-only-missing`	Only fetch data for test cases without existing catalogs [default: no-only-missing]	No	-
`--force / --no-force`	Force overwrite catalog even if unchanged [default: no-force]	No	-
`--strict / --no-strict`	Exit with an error if any requested test case cannot be fetched [default: no-strict]	No	-
`--regen / --no-regen`	Resolve the requests from their declared facets instead of the datasets an existing catalog records. This may change which datasets the tests use. [default: no-regen]	No	-
`--help`	Show this message and exit.	No	-

`app test-cases list`#

List test cases for all diagnostics.

Shows which test cases are defined for each diagnostic and their descriptions. Also shows whether catalog and regression data exist for each test case.

Usage#

app test-cases list [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--provider`	Filter by provider	No	-
`-v, --verbose`	Show test case descriptions	No	-
`--help`	Show this message and exit.	No	-

`app test-cases run`#

Run test cases for diagnostics.

Executes diagnostics using pre-defined datasets from the test_data_spec and optionally compares against regression baselines.

Use --provider to select which provider's diagnostics to run (required). Use --diagnostic and --test-case to further narrow the scope.

With --from-slot the diagnostic is not executed. The native already materialised in output/<label>/ by a previous run, replay or mint is reused, and only the committed bundle is regenerated. This is how a bundle is refreshed after an extraction-code change. It cannot be combined with the options that only make sense when executing.

Examples#

ref test-cases run --provider ilamb              # Run all ILAMB test cases
ref test-cases run --provider example --diagnostic global-mean-timeseries
ref test-cases run --provider ilamb --test-case default --fetch
ref test-cases run --provider pmp --only-missing # Skip test cases with regression data
ref test-cases run --provider pmp --if-changed   # Only run if catalog changed
ref test-cases run --provider example --from-slot --label before --force-regen

Usage#

app test-cases run [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--provider`	Provider slug (required, e.g., 'example', 'ilamb')	Yes	-
`--diagnostic`	Specific diagnostic slug to run (e.g., 'global-mean-timeseries')	No	-
`--test-case`	Specific test case name to run (e.g., 'default')	No	-
`--output-directory`	Scratch directory for the diagnostic execution results. The regression workflow also writes the gitignored output/ slot.	No	-
`--force-regen / --no-force-regen`	Force regeneration of regression baselines [default: no-force-regen]	No	-
`--fetch / --no-fetch`	Fetch test data from ESGF before running [default: no-fetch]	No	-
`--size-threshold`	Flag files larger than this size in MB (default: 1.0) [default: 1.0]	No	-
`--dry-run / --no-dry-run`	Show what would be run without executing [default: no-dry-run]	No	-
`--only-missing / --no-only-missing`	Only run test cases without existing regression data [default: no-only-missing]	No	-
`--if-changed / --no-if-changed`	Only run if catalog has changed since regression data was generated [default: no-if-changed]	No	-
`--clean / --no-clean`	Delete existing output directory before running [default: no-clean]	No	-
`--label`	Output slot name under output/ (default: latest) [default: latest]	No	-
`--from-slot`	Rebuild the committed bundle from the native already in the output slot	No	-
`--help`	Show this message and exit.	No	-

`app test-cases replay`#

Replay committed baselines from native blobs and compare to the committed bundle.

Materialises the committed manifest's native blobs (public, credential-free) into a fresh output directory at their stored relative paths, re-runs build_execution_result, and compares the regenerated committed bundle to the in-repo copy using the tolerant content comparator.

Exits non-zero on drift.

Examples#

ref test-cases replay --provider example
ref test-cases replay --provider example --diagnostic global-mean-timeseries

Usage#

app test-cases replay [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--provider`	Provider slug (required, e.g., 'example', 'ilamb')	Yes	-
`--diagnostic`	Specific diagnostic slug to replay	No	-
`--test-case`	Specific test case name to replay	No	-
`--label`	Output slot name under output/ (default: latest) [default: latest]	No	-
`--help`	Show this message and exit.	No	-

`app test-cases mint`#

Mint canonical native baselines

Runs each test case, stores its native snapshot in the writable store, and authors the committed manifest.json's native block.

The test_case_version is bumped when the committed bundle changed versus the previous manifest, or always with --bump-version.

This requires write credentials and is generally run by the CI.

Examples#

ref test-cases mint --provider example
ref test-cases mint --provider example --bump-version
ref test-cases mint --provider example --dry-run

Usage#

app test-cases mint [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--provider`	Provider slug (required, e.g., 'example', 'ilamb')	Yes	-
`--diagnostic`	Specific diagnostic slug to mint	No	-
`--test-case`	Specific test case name to mint	No	-
`--label`	Output slot name under output/ (default: latest) [default: latest]	No	-
`--from-replay`	Author from a replay of the stored native instead of re-running the diagnostic	No	-
`--bump-version / --no-bump-version`	Always increment test_case_version (else only when the committed bundle changes) [default: no-bump-version]	No	-
`--dry-run / --no-dry-run`	Preflight the store and list what would be minted, without running or uploading [default: no-dry-run]	No	-
`--help`	Show this message and exit.	No	-

`app test-cases ci-gate`#

Decide how CI should verify each test case's regression baseline.

Compares each committed manifest.json to its counterpart on the base branch and reports the action CI should take per case: replay (cheap, against the cached native baseline), skip (nothing relevant changed, or nothing to replay against), or fail (an unauthorised baseline change). Exits non-zero if any case is gated fail.

The --json output is intended for CI to dispatch replay jobs.

Examples#

ref test-cases ci-gate                       # Gate all cases against origin/main
ref test-cases ci-gate --base origin/develop
ref test-cases ci-gate --provider example --json

Usage#

app test-cases ci-gate [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--base`	Git ref to compare against (the PR base branch) [default: origin/main]	No	-
`--provider`	Limit the gate to a single provider slug	No	-
`--diagnostic`	Limit the gate to a single diagnostic slug	No	-
`--test-case`	Limit the gate to a single test case name	No	-
`--json`	Emit the per-case decisions as JSON on stdout	No	-
`--help`	Show this message and exit.	No	-

`app test-cases diff`#

Render an HTML report of the regression baselines changed on this branch.

Compares every committed manifest.json to its counterpart on --base and writes one page per changed test case.

Exits 0 whether or not anything changed.

Examples#

ref test-cases diff --html-dir build/baseline-diff
ref test-cases diff --base origin/develop --html-dir build/baseline-diff
ref test-cases diff --html-dir build/baseline-diff --no-fetch
ref test-cases diff --html-dir build/baseline-diff --upload 912/0c7e1d4abc12

Usage#

app test-cases diff [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--html-dir`	Directory to write the HTML report into	Yes	-
`--base`	Git ref to compare against (the PR base branch) [default: origin/main]	No	-
`--no-fetch`	Skip blob downloads and report sizes only	No	-
`--upload`	Upload the report under this key prefix, e.g. 912/0c7e1d4abc12	No	-
`--comment-output`	Write the pull request comment markdown here	No	-
`--help`	Show this message and exit.	No	-

`app celery`#

Managing remote celery workers

This module is used to manage remote execution workers for the Climate REF project. It is added to the ref command line interface if the climate-ref-celery package is installed.

A celery worker should be run for each diagnostic provider.

Usage#

app celery [OPTIONS] COMMAND [ARGS]...

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

Subcommands#

`app celery start-worker`#

Start a Celery worker for the given provider.

A celery worker enables the execution of tasks in the background on multiple different nodes. This worker will register a celery task for each diagnostic in the provider. The worker tasks can be executed by sending a celery task with the name '{package_slug}_{diagnostic_slug}'.

Providers must be registered as entry points in the pyproject.toml file of the package. The entry point should be defined under the group climate-ref.providers (See import_provider for details).

Usage#

app celery start-worker [OPTIONS][extra_args]...

Arguments#

Name	Description	Required
`extra_args...`	Additional arguments for the worker	No

Options#

Name	Description	Required	Default
`--loglevel`	Log level for the worker [default: info]	No	-
`--provider`	Name of the provider to start a worker for. This argument may be supplied multiple times. If no provider is given, the worker will consume the default queue.	No	-
`--package`	Deprecated. Use provider instead	No	-
`--help`	Show this message and exit.	No	-

`app celery list-config`#

List the celery configuration

Usage#

app celery list-config [OPTIONS]

Arguments#

No arguments available

Options#

Name	Description	Required	Default
`--help`	Show this message and exit.	No	-

CLI#

app#

Usage#

Arguments#

Options#

Commands#

Subcommands#

app solve#

Usage#

Arguments#

Options#

app doctor#

Usage#

Arguments#

Options#

app config#

Usage#

Arguments#

Options#

Subcommands#

app config list#

Usage#

Arguments#

Options#

app config init#

Usage#

Arguments#

Options#

app config get#

Usage#

Arguments#

Options#

app config set#

Usage#

Arguments#

Options#

app config unset#

Usage#

Arguments#

Options#

app config edit#

Usage#

Arguments#

Options#

app config validate#

Usage#

Arguments#

Options#

app datasets#

Usage#

Arguments#

Options#

Subcommands#

app datasets list#

Usage#

Arguments#

Options#

app datasets list-columns#

Usage#

Arguments#

Options#

app datasets retract#

Usage#

Arguments#

Options#

app datasets unretract#

Usage#

Arguments#

Options#

app datasets ingest#

Usage#

Arguments#

Options#

app datasets stats#

Usage#

Arguments#

Options#

app datasets fetch-sample-data#

Usage#

Arguments#

`app solve`#

`app doctor`#

`app config`#

`app config list`#

`app config init`#

`app config get`#

`app config set`#

`app config unset`#

`app config edit`#

`app config validate`#

`app datasets`#

`app datasets list`#

`app datasets list-columns`#

`app datasets retract`#

`app datasets unretract`#

`app datasets ingest`#

`app datasets stats`#

`app datasets fetch-sample-data`#

`app datasets fetch-data`#

`app db`#

`app db migrate`#

`app db status`#

`app db heads`#

`app db history`#

`app db backup`#

`app db sql`#

`app db tables`#

`app diagnostics`#

`app diagnostics list`#

`app executions`#

`app executions list-groups`#

`app executions delete-groups`#

`app executions inspect`#

`app executions values`#

`app executions fail-running`#