Skip to content

climate_ref.results #

Data access layer for REF results.

This package is the single source of truth for how result data is filtered and queried. It serves notebooks / local users (via pandas-friendly, detached value objects) and, in future, the ref-app API (which can reuse the shared Select builders directly).

Layers:

  • _query -- MetricValueFilter + select_* builders (return a Select; the source of truth for filter/join logic) + count_values + latest_execution_for_group.
  • frames -- *_to_frame + collect_facets.
  • outliers -- source-id-aware IQR outlier detection.
  • values -- typed DTOs + collections + the Reader facade.
  • executions -- execution-group / execution DTOs + collection + the ExecutionsReader facade.
  • datasets -- dataset DTOs + collection + the DatasetsReader facade, querying the polymorphic Dataset hierarchy directly.
  • diagnostics -- diagnostic DTOs + collection + the DiagnosticsReader facade, querying Diagnostic -> Provider directly.
  • artifacts -- the ArtifactsReader facade, resolving execution output fragments into filesystem paths under a results root.

The typical notebook entry point::

from climate_ref.config import Config
from climate_ref.database import Database
from climate_ref.results import Reader

with Database.from_config(Config.default(), read_only=True) as db:
    df = Reader(db).values.scalar_values().to_pandas()
Public surface convention

The top level exports only what a caller names to make a call, so the namespace stays small as domains are added:

  • the Reader entry point,
  • filter objects you construct and pass in (MetricValueFilter, ExecutionGroupFilter, DatasetFilter, DiagnosticFilter),
  • value objects you pass in (OutlierPolicy).

Everything the package returns -- DTOs, collections, views -- and the sub-reader classes reached via reader.values etc. live in their domain submodule (climate_ref.results.values, ...). Import them from there on the rare occasion you need to name one, e.g. from climate_ref.results.values import ScalarValue. The Select builders and other plumbing stay in climate_ref.results._query and are not part of the public surface.

DatasetFilter #

Declarative filter over datasets.

source_type is required. It selects which which facet columns the query can target. This limits our filtering to a single source type at a time to ensure that the files can be collapsed into a dataframe.

Every other field is optional with None meaning "do not constrain on this axis".

Source code in packages/climate-ref/src/climate_ref/models/dataset_query.py 
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
@attrs.frozen(kw_only=True)
class DatasetFilter:
    """
    Declarative filter over datasets.

    ``source_type`` is required.
    It selects which which facet columns the query can target.
    This limits our filtering to a single source type at a time to ensure that the files can be
    collapsed into a dataframe.

    Every other field is optional with ``None`` meaning "do not constrain on this axis".
    """

    source_type: SourceDatasetType
    facets: Mapping[str, tuple[str, ...]] | None = attrs.field(default=None, converter=_as_facets)
    finalised: bool | None = None
    execution_id: int | None = None
    diagnostic_slug: str | None = None
    latest_only: bool = True

DiagnosticFilter #

Declarative filter over diagnostics.

Every field is optional; None means "do not constrain on this axis". diagnostic_contains/provider_contains are case-insensitive substring matches (OR-combined within each field), matching the semantics used by ExecutionGroupFilter.

Source code in packages/climate-ref/src/climate_ref/results/diagnostics.py 
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
@attrs.frozen(kw_only=True)
class DiagnosticFilter:
    """
    Declarative filter over diagnostics.

    Every field is optional; ``None`` means "do not constrain on this axis".
    ``diagnostic_contains``/``provider_contains`` are case-insensitive substring matches
    (OR-combined within each field), matching the semantics used by
    [ExecutionGroupFilter][climate_ref.results.executions.ExecutionGroupFilter].
    """

    provider_contains: tuple[str, ...] | None = attrs.field(default=None, converter=_as_str_tuple)
    """Case-insensitive substring matches on provider slug (OR-combined)."""

    diagnostic_contains: tuple[str, ...] | None = attrs.field(default=None, converter=_as_str_tuple)
    """Case-insensitive substring matches on diagnostic slug (OR-combined)."""

diagnostic_contains = attrs.field(default=None, converter=_as_str_tuple) class-attribute instance-attribute #

Case-insensitive substring matches on diagnostic slug (OR-combined).

provider_contains = attrs.field(default=None, converter=_as_str_tuple) class-attribute instance-attribute #

Case-insensitive substring matches on provider slug (OR-combined).

ExecutionGroupFilter #

Declarative filter over execution groups.

Every field is optional; None means "do not constrain on this axis". This mirrors exactly what get_execution_group_and_latest_filtered supports -- diagnostic_contains/provider_contains are case-insensitive substring matches (OR-combined), facets is the selector-facet map consumed by the helper's Python-side matching, and successful vs latest_successful expose the helper's post-rank / pre-rank success filters.

Exact-match diagnostic_slug/provider_slug are a documented follow-up, as the underlying helper only does substring matching today, so adding exact match means extending the helper first.

Source code in packages/climate-ref/src/climate_ref/results/executions.py 
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
@attrs.frozen(kw_only=True)
class ExecutionGroupFilter:
    """
    Declarative filter over execution groups.

    Every field is optional; ``None`` means "do not constrain on this axis".
    This mirrors exactly what [get_execution_group_and_latest_filtered]
    [climate_ref.models.execution.get_execution_group_and_latest_filtered] supports --
    ``diagnostic_contains``/``provider_contains`` are case-insensitive substring matches (OR-combined),
    ``facets`` is the selector-facet map consumed by the helper's Python-side matching,
    and ``successful`` vs ``latest_successful`` expose the helper's post-rank / pre-rank success filters.

    Exact-match ``diagnostic_slug``/``provider_slug`` are a documented follow-up,
    as the underlying helper only does substring matching today,
    so adding exact match means extending the helper first.
    """

    diagnostic_contains: tuple[str, ...] | None = attrs.field(default=None, converter=_as_str_tuple)
    """Case-insensitive substring matches on diagnostic slug (OR-combined)."""

    provider_contains: tuple[str, ...] | None = attrs.field(default=None, converter=_as_str_tuple)
    """Case-insensitive substring matches on provider slug (OR-combined)."""

    dirty: bool | None = None
    """Constrain on the group's ``dirty`` flag."""

    successful: bool | None = None
    """
    Post-rank filter on the *winning* execution: keep a group only if its latest execution matches.

    ``True`` keeps groups whose latest execution succeeded.
    ``False`` keeps groups whose latest execution failed, is in progress, or does not exist yet.
    This does not change which execution is considered latest -- contrast ``latest_successful``.
    """

    latest_successful: bool | None = None
    """
    Pre-rank population filter: change which execution is chosen as "latest" before ranking.

    ``True`` ranks only over successful executions,
    so a group's ``latest`` becomes its most recent *successful* run
    (surfacing an earlier success even when a later run failed).
    ``False`` ranks only over unsuccessful / in-progress runs.

    ``None`` (default) ranks over all executions.
    Composes with ``successful`` but answers a different question.
    """

    facets: Mapping[str, tuple[str, ...]] | None = attrs.field(default=None, converter=_as_facets)
    """Selector-facet map matched against each group's ``selectors``."""

    include_superseded: bool = False
    """Include execution groups whose ``diagnostic_version`` is not the diagnostic's promoted version."""

diagnostic_contains = attrs.field(default=None, converter=_as_str_tuple) class-attribute instance-attribute #

Case-insensitive substring matches on diagnostic slug (OR-combined).

dirty = None class-attribute instance-attribute #

Constrain on the group's dirty flag.

facets = attrs.field(default=None, converter=_as_facets) class-attribute instance-attribute #

Selector-facet map matched against each group's selectors.

include_superseded = False class-attribute instance-attribute #

Include execution groups whose diagnostic_version is not the diagnostic's promoted version.

latest_successful = None class-attribute instance-attribute #

Pre-rank population filter: change which execution is chosen as "latest" before ranking.

True ranks only over successful executions, so a group's latest becomes its most recent successful run (surfacing an earlier success even when a later run failed). False ranks only over unsuccessful / in-progress runs.

None (default) ranks over all executions. Composes with successful but answers a different question.

provider_contains = attrs.field(default=None, converter=_as_str_tuple) class-attribute instance-attribute #

Case-insensitive substring matches on provider slug (OR-combined).

successful = None class-attribute instance-attribute #

Post-rank filter on the winning execution: keep a group only if its latest execution matches.

True keeps groups whose latest execution succeeded. False keeps groups whose latest execution failed, is in progress, or does not exist yet. This does not change which execution is considered latest -- contrast latest_successful.

MetricValueFilter #

Declarative filter over metric values.

Every field is optional; None/empty means "do not constrain on this axis". The same object drives scalar and series queries, frame conversion and facet collection, so a filter is written once and reused across output shapes.

Two styles of diagnostic/provider filtering are offered deliberately:

  • diagnostic_slug / provider_slug are exact matches, for API path-scoped queries (/diagnostics/{provider_slug}/{diagnostic_slug}/values).
  • diagnostic_contains / provider_contains are case-insensitive substring matches (OR-combined), for CLI-style search.

Keeping them separate avoids the Diagnostic.slug vs Diagnostic.name divergence that exists today between the CLI and the ref-app API.

Source code in packages/climate-ref/src/climate_ref/results/_query.py 
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
@attrs.frozen(kw_only=True)
class MetricValueFilter:
    """
    Declarative filter over metric values.

    Every field is optional; ``None``/empty means "do not constrain on this axis". The same
    object drives scalar and series queries, frame conversion and facet collection, so a filter
    is written once and reused across output shapes.

    Two styles of diagnostic/provider filtering are offered deliberately:

    * ``diagnostic_slug`` / ``provider_slug`` are **exact** matches, for API path-scoped queries
      (``/diagnostics/{provider_slug}/{diagnostic_slug}/values``).
    * ``diagnostic_contains`` / ``provider_contains`` are case-insensitive **substring** matches
      (OR-combined), for CLI-style search.

    Keeping them separate avoids the ``Diagnostic.slug`` vs ``Diagnostic.name`` divergence that
    exists today between the CLI and the ref-app API.
    """

    execution_ids: Sequence[int] | None = None
    """Restrict to values produced by these executions."""

    execution_group_ids: Sequence[int] | None = None
    """Restrict to values produced by executions belonging to these groups."""

    diagnostic_slug: str | None = None
    """Exact-match diagnostic slug, for API path-scoped queries."""

    provider_slug: str | None = None
    """Exact-match provider slug, for API path-scoped queries."""

    diagnostic_contains: Sequence[str] | None = attrs.field(default=None, converter=_as_str_tuple)
    """Case-insensitive substring matches on diagnostic slug (OR-combined), for CLI-style search."""

    provider_contains: Sequence[str] | None = attrs.field(default=None, converter=_as_str_tuple)
    """Case-insensitive substring matches on provider slug (OR-combined), for CLI-style search."""

    dimensions: Mapping[str, str | Sequence[str]] | None = None
    """CV dimension filters keyed by registered dimension name; a string is equality, a sequence is IN."""

    isolate_ids: Sequence[int] | None = None
    """Restrict to exactly these value ids; takes precedence over ``exclude_ids``, matching the API."""

    exclude_ids: Sequence[int] | None = None
    """Exclude these value ids; ignored when ``isolate_ids`` is set."""

    reference_only: bool | None = None
    """Series-only: ``True`` for observation/reference series, ``False`` for model series."""

    promoted_only: bool = True
    """Restrict to execution groups at the diagnostic's currently promoted version."""

    include_retracted: bool = False
    """Include values produced by retracted executions."""

    def dimension_clauses(self, entity: type[MetricValue]) -> list[Any]:
        """
        Build validated SQLAlchemy clauses for the dynamic CV dimension columns.

        Raises
        ------
        KeyError
            If a key is not a registered CV dimension on ``entity``.
        """
        clauses: list[Any] = []
        for key, value in (self.dimensions or {}).items():
            if key not in entity._cv_dimensions:
                raise KeyError(f"Unknown dimension column {key!r}")
            col = getattr(entity, key)
            if isinstance(value, str) or not isinstance(value, Sequence):
                clauses.append(col == value)
            else:
                vals = list(value)
                clauses.append(col == vals[0] if len(vals) == 1 else col.in_(vals))
        return clauses

diagnostic_contains = attrs.field(default=None, converter=_as_str_tuple) class-attribute instance-attribute #

Case-insensitive substring matches on diagnostic slug (OR-combined), for CLI-style search.

diagnostic_slug = None class-attribute instance-attribute #

Exact-match diagnostic slug, for API path-scoped queries.

dimensions = None class-attribute instance-attribute #

CV dimension filters keyed by registered dimension name; a string is equality, a sequence is IN.

exclude_ids = None class-attribute instance-attribute #

Exclude these value ids; ignored when isolate_ids is set.

execution_group_ids = None class-attribute instance-attribute #

Restrict to values produced by executions belonging to these groups.

execution_ids = None class-attribute instance-attribute #

Restrict to values produced by these executions.

include_retracted = False class-attribute instance-attribute #

Include values produced by retracted executions.

isolate_ids = None class-attribute instance-attribute #

Restrict to exactly these value ids; takes precedence over exclude_ids, matching the API.

promoted_only = True class-attribute instance-attribute #

Restrict to execution groups at the diagnostic's currently promoted version.

provider_contains = attrs.field(default=None, converter=_as_str_tuple) class-attribute instance-attribute #

Case-insensitive substring matches on provider slug (OR-combined), for CLI-style search.

provider_slug = None class-attribute instance-attribute #

Exact-match provider slug, for API path-scoped queries.

reference_only = None class-attribute instance-attribute #

Series-only: True for observation/reference series, False for model series.

dimension_clauses(entity) #

Build validated SQLAlchemy clauses for the dynamic CV dimension columns.

Raises:

Type Description
KeyError

If a key is not a registered CV dimension on entity.
Source code in packages/climate-ref/src/climate_ref/results/_query.py 
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
def dimension_clauses(self, entity: type[MetricValue]) -> list[Any]:
    """
    Build validated SQLAlchemy clauses for the dynamic CV dimension columns.

    Raises
    ------
    KeyError
        If a key is not a registered CV dimension on ``entity``.
    """
    clauses: list[Any] = []
    for key, value in (self.dimensions or {}).items():
        if key not in entity._cv_dimensions:
            raise KeyError(f"Unknown dimension column {key!r}")
        col = getattr(entity, key)
        if isinstance(value, str) or not isinstance(value, Sequence):
            clauses.append(col == value)
        else:
            vals = list(value)
            clauses.append(col == vals[0] if len(vals) == 1 else col.in_(vals))
    return clauses

OutlierPolicy #

Configuration for scalar outlier detection.

The defaults reproduce the behaviour users currently see through the ref-app API (factor=10.0, source-id-aware IQR grouped by ("statistic", "metric")).

Source code in packages/climate-ref/src/climate_ref/results/outliers.py 
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
@attrs.frozen(kw_only=True)
class OutlierPolicy:
    """
    Configuration for scalar outlier detection.

    The defaults reproduce the behaviour users currently see through the ref-app API
    (``factor=10.0``, source-id-aware IQR grouped by ``("statistic", "metric")``).
    """

    method: str = attrs.field(default="iqr", validator=attrs.validators.in_(_SUPPORTED_METHODS))
    """Detection method. ``"off"`` disables detection; ``"iqr"`` enables it."""

    factor: float = 10.0
    """Multiplier on the IQR to set the outlier bounds (``Q1 - factor*IQR``, ``Q3 + factor*IQR``)."""

    min_n: int = attrs.field(default=4, validator=attrs.validators.ge(2))
    """
    Minimum sample size required to run detection.

    On the source-id-aware path this counts distinct non-reference ``source_id`` with a finite mean.
    On the fallback path it counts finite values.
    Non-finite values (NaN/inf) are excluded from the count
    and are flagged unconditionally regardless of whether detection runs.
    """

    group_by: tuple[str, ...] = ("statistic", "metric")
    """CV dimensions to group by before computing bounds. Missing dimensions are ignored."""

    @property
    def enabled(self) -> bool:
        """Whether detection should run."""
        return self.method == "iqr"

enabled property #

Whether detection should run.

factor = 10.0 class-attribute instance-attribute #

Multiplier on the IQR to set the outlier bounds (Q1 - factor*IQR, Q3 + factor*IQR).

group_by = ('statistic', 'metric') class-attribute instance-attribute #

CV dimensions to group by before computing bounds. Missing dimensions are ignored.

method = attrs.field(default='iqr', validator=(attrs.validators.in_(_SUPPORTED_METHODS))) class-attribute instance-attribute #

Detection method. "off" disables detection; "iqr" enables it.

min_n = attrs.field(default=4, validator=(attrs.validators.ge(2))) class-attribute instance-attribute #

Minimum sample size required to run detection.

On the source-id-aware path this counts distinct non-reference source_id with a finite mean. On the fallback path it counts finite values. Non-finite values (NaN/inf) are excluded from the count and are flagged unconditionally regardless of whether detection runs.

Reader #

Typed entry point to REF query results.

Constructed from a Database, which owns the session and the read-only story. This is a thin entry point: it exposes per-domain sub-readers as properties, values for metric-value reads, executions for execution-group reads, datasets for dataset reads, diagnostics for diagnostic reads, and artifacts for output path resolution (only available when a results root is supplied).

Source code in packages/climate-ref/src/climate_ref/results/values.py 
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
class Reader:
    """
    Typed entry point to REF query results.

    Constructed from a [Database][climate_ref.database.Database], which owns the session and the
    read-only story. This is a thin entry point: it exposes per-domain sub-readers as properties,
    [values][climate_ref.results.values.Reader.values] for metric-value reads,
    [executions][climate_ref.results.values.Reader.executions] for execution-group reads,
    [datasets][climate_ref.results.values.Reader.datasets] for dataset reads,
    [diagnostics][climate_ref.results.values.Reader.diagnostics] for diagnostic reads, and
    [artifacts][climate_ref.results.values.Reader.artifacts] for output path resolution
    (only available when a ``results`` root is supplied).
    """

    def __init__(self, database: Database, results: Path | None = None) -> None:
        self._db = database
        self._results = results

    @property
    def session(self) -> Session:
        """The underlying database session."""
        return self._db.session

    @functools.cached_property
    def values(self) -> ValuesReader:
        """Metric-value reads (scalar/series/facets)."""
        return ValuesReader(self._db)

    @functools.cached_property
    def executions(self) -> ExecutionsReader:
        """Execution-group and execution reads."""
        return ExecutionsReader(self._db)

    @functools.cached_property
    def datasets(self) -> "DatasetsReader":
        """Dataset reads."""
        from climate_ref.results.datasets import DatasetsReader  # noqa: PLC0415

        return DatasetsReader(self._db)

    @functools.cached_property
    def diagnostics(self) -> "DiagnosticsReader":
        """Diagnostic reads."""
        from climate_ref.results.diagnostics import DiagnosticsReader  # noqa: PLC0415

        return DiagnosticsReader(self._db)

    @functools.cached_property
    def artifacts(self) -> "ArtifactsReader":
        """
        Output path resolution.

        Raises ``ValueError`` when no ``results`` root was supplied to the constructor.
        """
        if self._results is None:
            raise ValueError(
                "reader.artifacts requires a results root; construct "
                "Reader(database, results=config.paths.results)."
            )
        from climate_ref.results.artifacts import ArtifactsReader  # noqa: PLC0415

        return ArtifactsReader(self._results)

artifacts cached property #

Output path resolution.

Raises ValueError when no results root was supplied to the constructor.

datasets cached property #

Dataset reads.

diagnostics cached property #

Diagnostic reads.

executions cached property #

Execution-group and execution reads.

session property #

The underlying database session.

values cached property #

Metric-value reads (scalar/series/facets).

sub-packages#

Sub-package Description
_converters Shared attrs field converters for the results filter DTOs.
_query Filter and Select builders for metric-value queries.
_stats Shared execution-statistics mapping for the results read layer.
artifacts Path resolution for execution output artifacts.
datasets Typed, detached read surface for datasets.
diagnostics Typed, detached read surface for diagnostic metadata.
executions Typed, ORM-free surface for execution-group and execution results.
frames DataFrame conversion and facet collection for metric values.
outliers Outlier detection for scalar metric values.
values Typed, ORM-free surface for metric-value results.