Skip to content

Diagnose a deployment#

ref doctor looks for the problems that a solve hides rather than reports.

  • Reference data is missing, so its diagnostics quietly plan no executions.
  • Data is ingested under a source type no diagnostic requires or declares as a fallback, so nothing selects it.
  • obs4REF data is ingested as obs4mips, so the catalog cannot say where it came from.
  • An obs4REF dataset has since been published to obs4MIPs, so the registry copy is no longer used.
  • A dataset's files cover the same period twice, so a diagnostic reads that period more than once.
  • A diagnostic cannot be solved at all by the ingested data. The finding names the requirement that goes unmet.
ref doctor

Findings are grouped by the check that produced them, under the remedy they have in common, so a deployment missing twenty reference datasets reads as one instruction and twenty names:

3 findings from 7 checks: 3 warnings

missing-reference-data 3 warnings
  Fetch these, then ingest the directory they land in.
  ref datasets fetch-data --registry obs4ref --output-directory <dir>

  WOA-23 (obs4mips) is not ingested, so 2 diagnostics will not run
    Needed for so, thetao by ilamb/so-woa2023-surface, ilamb/thetao-woa2023-surface.
  ...

The command exits non-zero when it finds an error, or when it finds a warning and --strict is used, so it can gate a run:

ref doctor --strict && ref solve

To see which checks would run, and where each came from:

ref doctor --list

Reporting a problem#

--format markdown produces a report that can be pasted into an issue. Alongside the findings it describes the environment: package versions, platform, configuration, paths, enabled providers, what is ingested, and the REF_*, DASK_* and ESMVALTOOL_* environment variables that are set. This environment is excluded with the --no-environment option.

ref doctor --format markdown

--format json produces the same content for scripting.

From Python#

diagnose runs the checks and returns everything needed to report on them: the findings worst first, how many checks ran, and optionally a description of the deployment.

from climate_ref.doctor import DoctorContext, Severity, diagnose

report = diagnose(DoctorContext(config=config, database=database), environment=True)
if report.worst_severity == Severity.ERROR:
    ...

Adding a check#

A check is a function that takes a DoctorContext and returns a list of Findings, declared with climate_ref.doctor.check. The context loads providers and catalogs lazily, so a check pays only for what it reads, and a check that raises becomes a finding rather than stopping the rest of the run.

import os

from climate_ref.doctor import DoctorContext, Finding, Severity, check


@check("scratch-writable", "The scratch directory can be written to")
def check_scratch_writable(context: DoctorContext) -> list[Finding]:
    if context.config is None or os.access(context.config.paths.scratch, os.W_OK):
        return []
    return [
        Finding(
            severity=Severity.ERROR,
            summary=f"The scratch directory {context.config.paths.scratch} is not writable",
            detail="Executions write their working files here, so every execution will fail.",
            remedy="Grant the user running the REF write access to it.",
        )
    ]

detail explains one finding, and remedy says what to do about it. Findings sharing a remedy are reported under it once, so keep that wording free of anything specific to a single finding. A command that carries out the remedy goes in its own field, where it is printed unwrapped and stays pasteable.

The check does not name itself in its findings: the runner stamps the slug from the registration, so the two cannot drift apart. A check must tolerate a context that has nothing ingested for a given source type, and one built without a database (DoctorContext.from_catalogs).

Checks that ship with the REF live in climate_ref.doctor.checks. A package outside climate_ref contributes its own by pointing an entry point at the module that declares them:

[project.entry-points."climate-ref.doctor-checks"]
my_provider = "my_package.doctor_checks"

ref doctor imports that module for its @check declarations. A module that cannot be imported is reported as an error finding rather than taking the command down, because a check that never ran must not look like a check that passed.