climate_ref.baseline_report.analyse

Turn a collected report into everything the templates need.

Text blobs are fetched from the native store and diffed here.

AnalysedCase

One test case, with its files analysed and tallied.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

@frozen
class AnalysedCase:
    """One test case, with its files analysed and tallied."""

    change: CaseChange
    """The underlying change."""

    files: tuple[AnalysedFile, ...]
    """Every analysed file, in the order collection produced them."""

    counts: tuple[KindCounts, ...]
    """One entry per kind, in report column order."""

    images: tuple[AnalysedFile, ...]
    """The image files, which render as a two-up comparison."""

    texts: tuple[AnalysedFile, ...]
    """The text files, which render as a diff."""

    netcdfs: tuple[AnalysedFile, ...]
    """The NetCDF files, which render as a header diff and a table of variable statistics."""

    others: tuple[AnalysedFile, ...]
    """Every remaining file, which renders as a table row."""

    committed: tuple[AnalysedCommitted, ...]
    """The committed artefacts that moved, in name order."""

    tree: tuple[TreeNode, ...]
    """The captured baseline's native files, as a flattened folder listing."""

    back_link: str
    """Relative link from this case's page back to the index, one ``..`` per label segment."""

back_link instance-attribute

Relative link from this case's page back to the index, one .. per label segment.

change instance-attribute

The underlying change.

committed instance-attribute

The committed artefacts that moved, in name order.

counts instance-attribute

One entry per kind, in report column order.

files instance-attribute

Every analysed file, in the order collection produced them.

images instance-attribute

The image files, which render as a two-up comparison.

netcdfs instance-attribute

The NetCDF files, which render as a header diff and a table of variable statistics.

others instance-attribute

Every remaining file, which renders as a table row.

texts instance-attribute

The text files, which render as a diff.

tree instance-attribute

The captured baseline's native files, as a flattened folder listing.

AnalysedCommitted

One committed regression artefact, with its diff.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

@frozen
class AnalysedCommitted:
    """One committed regression artefact, with its diff."""

    change: CommittedChange
    """The underlying change."""

    text: TextDiff
    """The diff between the two committed versions."""

change instance-attribute

The underlying change.

text instance-attribute

The diff between the two committed versions.

AnalysedFile

One native file, with its blob URLs and whatever its kind is analysed into.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

@frozen
class AnalysedFile:
    """One native file, with its blob URLs and whatever its kind is analysed into."""

    change: FileChange
    """The underlying change."""

    old_url: str | None
    """Store URL of the base blob, or ``None`` when the file is new."""

    new_url: str | None
    """Store URL of the head blob, or ``None`` when the file was removed."""

    text: TextDiff | None
    """The diff, set only for :attr:`~climate_ref.baseline_report.collect.FileKind.TEXT` files."""

    netcdf: NetcdfDiff | None
    """The header and stats diff, set only for
    :attr:`~climate_ref.baseline_report.collect.FileKind.NETCDF` files."""

    size_delta: int | None
    """Signed byte change, or ``None`` when the file exists on only one side."""

change instance-attribute

The underlying change.

netcdf instance-attribute

The header and stats diff, set only for :attr:~climate_ref.baseline_report.collect.FileKind.NETCDF files.

new_url instance-attribute

Store URL of the head blob, or None when the file was removed.

old_url instance-attribute

Store URL of the base blob, or None when the file is new.

size_delta instance-attribute

Signed byte change, or None when the file exists on only one side.

text instance-attribute

The diff, set only for :attr:~climate_ref.baseline_report.collect.FileKind.TEXT files.

AnalysedReport

A whole report, ready to render.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

@frozen
class AnalysedReport:
    """A whole report, ready to render."""

    report: Report
    """The underlying report."""

    store_url: str
    """Base location blobs are served from, matching what :func:`blob_url` builds."""

    cases: tuple[AnalysedCase, ...]
    """The analysed cases, in the order collection produced them."""

    kinds: tuple[str, ...]
    """The count column headers, matching the order of every case's ``counts``."""

cases instance-attribute

The analysed cases, in the order collection produced them.

kinds instance-attribute

The count column headers, matching the order of every case's counts.

report instance-attribute

The underlying report.

store_url instance-attribute

Base location blobs are served from, matching what :func:blob_url builds.

DiffLine

One line of a unified diff, tagged so a template can style it.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

@frozen
class DiffLine:
    """One line of a unified diff, tagged so a template can style it."""

    kind: str
    """One of ``add``, ``remove``, ``context``, ``hunk`` or ``header``."""

    text: str
    """The line itself, without a trailing newline."""

kind instance-attribute

One of add, remove, context, hunk or header.

text instance-attribute

The line itself, without a trailing newline.

KindCounts

How many files of one kind were added, changed and removed.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

@frozen
class KindCounts:
    """How many files of one kind were added, changed and removed."""

    label: str
    """The kind's name, as the report column header."""

    added: int
    """Files present only on HEAD."""

    changed: int
    """Files present on both sides with a different digest."""

    removed: int
    """Files present only on the base ref."""

added instance-attribute

Files present only on HEAD.

changed instance-attribute

Files present on both sides with a different digest.

label instance-attribute

The kind's name, as the report column header.

removed instance-attribute

Files present only on the base ref.

NetcdfDiff

What changed inside one NetCDF file, or the reason nothing could be read.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

@frozen
class NetcdfDiff:
    """What changed inside one NetCDF file, or the reason nothing could be read."""

    header: tuple[DiffLine, ...]
    """
    The two ncdump-style headers merged into one tagged listing.

    Every line is kept rather than only the changed hunks, because the header is what tells a
    reader what the file holds.
    """

    header_old: tuple[str, ...]
    """The base ref's header, for reading one side on its own."""

    header_new: tuple[str, ...]
    """HEAD's header, for reading one side on its own."""

    rows: tuple[StatRow, ...]
    """One row per data variable, in name order."""

    note: str | None
    """Why the file could not be analysed, or ``None`` when it was."""

    @property
    def header_changed(self) -> bool:
        """
        Whether the headers differ.

        Returns
        -------
        :
            ``True`` when any header line was added or removed.
        """
        return any(line.kind != "context" for line in self.header)

header instance-attribute

The two ncdump-style headers merged into one tagged listing.

Every line is kept rather than only the changed hunks, because the header is what tells a reader what the file holds.

header_changed property

Whether the headers differ.

Returns:

Type	Description
bool	True when any header line was added or removed.

header_new instance-attribute

HEAD's header, for reading one side on its own.

header_old instance-attribute

The base ref's header, for reading one side on its own.

note instance-attribute

Why the file could not be analysed, or None when it was.

rows instance-attribute

One row per data variable, in name order.

Pair

One statistic on each side of the change.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

@frozen
class Pair[T]:
    """One statistic on each side of the change."""

    old: T | None
    """The value on the base ref, or ``None`` when it could not be computed."""

    new: T | None
    """The value on HEAD, or ``None`` when it could not be computed."""

    atol: float = 0.0
    """How far a numeric pair may move before it reads as changed. Zero compares exactly."""

    @property
    def changed(self) -> bool:
        """
        Whether the two sides differ by more than :attr:`atol`.

        Returns
        -------
        :
            ``True`` when the value moved further than :attr:`atol`, which is what
            emphasises it in the table.
        """
        if self.old == self.new:
            return False
        if isinstance(self.old, float) and isinstance(self.new, float):
            return not abs(self.new - self.old) <= self.atol
        return True

atol = 0.0 class-attribute instance-attribute

How far a numeric pair may move before it reads as changed. Zero compares exactly.

changed property

Whether the two sides differ by more than :attr:atol.

Returns:

Type	Description
bool	True when the value moved further than :attr:atol, which is what emphasises it in the table.

new instance-attribute

The value on HEAD, or None when it could not be computed.

old instance-attribute

The value on the base ref, or None when it could not be computed.

StatRow

Whole-array statistics for one data variable, on each side of the change.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

@frozen
class StatRow:
    """Whole-array statistics for one data variable, on each side of the change."""

    name: str
    """The variable's name."""

    shape: Pair[str]
    """Dimensions on each side, as ``180x360``, or ``scalar``."""

    minimum: Pair[float]
    """Minimum on each side, ignoring NaN."""

    maximum: Pair[float]
    """Maximum on each side, ignoring NaN."""

    mean: Pair[float]
    """Mean on each side, ignoring NaN."""

    nan: Pair[int]
    """NaN cells on each side, ``None`` when the variable is absent or not numeric."""

    max_abs_diff: float | None
    """Largest absolute change, or ``None`` when the shapes differ or a side is absent."""

    max_rel_diff: float | None
    """:attr:`max_abs_diff` over the largest magnitude on the base ref."""

    cells_differ: int | None
    """Cells that changed, counting NaN as equal to NaN."""

    atol: float
    """
    How far a value may move before it counts as changed.

    :data:`NOISE_RTOL` scaled by the base ref's own magnitude, so one number governs
    both the row's verdict and which of its statistics are emphasised.
    """

    moved: bool
    """Whether anything about this variable changed, however small the change."""

    @property
    def severity(self) -> Literal["same", "noise", "changed"]:
        """
        How much weight the row deserves.

        A run reproduced on different hardware moves the last bits of nearly every cell, so a
        row that only moved that far is called out as noise rather than as a change.

        Returns
        -------
        :
            ``changed`` when the move is larger than :attr:`atol` or cannot be measured,
            ``noise`` when it is smaller, and ``same`` when nothing moved.
        """
        if not self.moved:
            return "same"
        if self.max_abs_diff is None or self.max_abs_diff > self.atol:
            return "changed"
        return "noise"

    @property
    def differs(self) -> bool:
        """
        Whether the cell by cell comparison found a change worth reading.

        Returns
        -------
        :
            ``True`` when at least one cell moved further than :attr:`atol`, which is
            what emphasises the diff columns.
        """
        return bool(self.cells_differ) and self.severity == "changed"

atol instance-attribute

How far a value may move before it counts as changed.

:data:NOISE_RTOL scaled by the base ref's own magnitude, so one number governs both the row's verdict and which of its statistics are emphasised.

cells_differ instance-attribute

Cells that changed, counting NaN as equal to NaN.

differs property

Whether the cell by cell comparison found a change worth reading.

Returns:

Type	Description
bool	True when at least one cell moved further than :attr:atol, which is what emphasises the diff columns.

max_abs_diff instance-attribute

Largest absolute change, or None when the shapes differ or a side is absent.

max_rel_diff instance-attribute

:attr:max_abs_diff over the largest magnitude on the base ref.

maximum instance-attribute

Maximum on each side, ignoring NaN.

mean instance-attribute

Mean on each side, ignoring NaN.

minimum instance-attribute

Minimum on each side, ignoring NaN.

moved instance-attribute

Whether anything about this variable changed, however small the change.

name instance-attribute

The variable's name.

nan instance-attribute

NaN cells on each side, None when the variable is absent or not numeric.

severity property

How much weight the row deserves.

A run reproduced on different hardware moves the last bits of nearly every cell, so a row that only moved that far is called out as noise rather than as a change.

Returns:

Type	Description
Literal['same', 'noise', 'changed']	changed when the move is larger than :attr:atol or cannot be measured, noise when it is smaller, and same when nothing moved.

shape instance-attribute

Dimensions on each side, as 180x360, or scalar.

TextDiff

The unified diff of one text output, or the reason there is not one.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

@frozen
class TextDiff:
    """The unified diff of one text output, or the reason there is not one."""

    lines: tuple[DiffLine, ...]
    """The diff lines, empty when :attr:`note` is set."""

    note: str | None
    """Why no diff could be produced, or ``None`` when :attr:`lines` carries one."""

    elided: int
    """Lines dropped past :data:`MAX_DIFF_LINES`."""

elided instance-attribute

Lines dropped past :data:MAX_DIFF_LINES.

lines instance-attribute

The diff lines, empty when :attr:note is set.

note instance-attribute

Why no diff could be produced, or None when :attr:lines carries one.

TreeNode

One row of the captured baseline's folder listing.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

@frozen
class TreeNode:
    """One row of the captured baseline's folder listing."""

    name: str
    """The directory or file name at this level."""

    depth: int
    """How deep the entry sits, so a template can indent it."""

    is_dir: bool
    """Whether the row is a directory rather than a file."""

    size: int | None
    """The file's size on HEAD, or on the base ref when it was removed.

    ``None`` for a directory and for a committed artefact, whose manifest entry has no size.
    """

    status: str | None
    """``added``, ``changed`` or ``removed``, or ``None`` when the file did not move."""

depth instance-attribute

How deep the entry sits, so a template can indent it.

is_dir instance-attribute

Whether the row is a directory rather than a file.

name instance-attribute

The directory or file name at this level.

size instance-attribute

The file's size on HEAD, or on the base ref when it was removed.

None for a directory and for a committed artefact, whose manifest entry has no size.

status instance-attribute

added, changed or removed, or None when the file did not move.

analyse(report, store, *, fetch, workdir)

Build everything the templates need from a collected report.

A blob that cannot be fetched becomes a note on its file rather than an exception, because one unreachable object should not cost the whole report.

Parameters:

Name	Type	Description	Default
report	Report	The collected report.	required
store	NativeStore	The store to read blobs from.	required
fetch	bool	Whether blobs may be downloaded. With False every text file carries a note instead of a diff and the store is never called.	required
workdir	Path	Directory fetched blobs are written into.	required

Returns:

Type	Description
AnalysedReport	The analysed report.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

def analyse(report: Report, store: NativeStore, *, fetch: bool, workdir: Path) -> AnalysedReport:
    """
    Build everything the templates need from a collected report.

    A blob that cannot be fetched becomes a note on its file rather than an exception, because
    one unreachable object should not cost the whole report.

    Parameters
    ----------
    report
        The collected report.
    store
        The store to read blobs from.
    fetch
        Whether blobs may be downloaded. With ``False`` every text file carries a note instead
        of a diff and the store is never called.
    workdir
        Directory fetched blobs are written into.

    Returns
    -------
    :
        The analysed report.
    """
    store_url = store.root.absolute().as_uri() if store.root is not None else store.url.rstrip("/")
    cases = []
    for case in report.cases:
        files = tuple(_analyse_file(change, store, fetch=fetch, workdir=workdir) for change in case.files)
        depth = len(PurePosixPath(case.label).parts)
        cases.append(
            AnalysedCase(
                change=case,
                files=files,
                counts=_counts(files),
                images=_of_kind(files, FileKind.IMAGE),
                texts=_of_kind(files, FileKind.TEXT),
                netcdfs=_of_kind(files, FileKind.NETCDF),
                others=_of_kind(files, FileKind.OTHER),
                committed=tuple(
                    AnalysedCommitted(change=change, text=committed_diff(change)) for change in case.committed
                ),
                tree=baseline_tree(case),
                back_link="/".join([*[".."] * depth, "index.html"]),
            )
        )
    return AnalysedReport(
        report=report,
        store_url=store_url,
        cases=tuple(cases),
        kinds=tuple(kind.value for kind in FileKind),
    )

baseline_tree(case)

Flatten the captured baseline's files into an indented folder listing.

Every captured file is listed, not only the ones that moved, because the listing is what tells a reviewer what the baseline actually holds.

Parameters:

Name	Type	Description	Default
case	CaseChange	The collected case.	required

Returns:

Type	Description
tuple[TreeNode, ...]	One row per directory and file, in path order.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

def baseline_tree(case: CaseChange) -> tuple[TreeNode, ...]:
    """
    Flatten the captured baseline's files into an indented folder listing.

    Every captured file is listed, not only the ones that moved, because the listing is what
    tells a reviewer what the baseline actually holds.

    Parameters
    ----------
    case
        The collected case.

    Returns
    -------
    :
        One row per directory and file, in path order.
    """
    entries = _baseline_entries(case)
    rows: list[TreeNode] = []
    previous: tuple[str, ...] = ()
    for path in sorted(entries):
        *directories, filename = PurePosixPath(path).parts
        shared = 0
        while shared < min(len(previous), len(directories)) and previous[shared] == directories[shared]:
            shared += 1
        for depth in range(shared, len(directories)):
            rows.append(TreeNode(name=directories[depth], depth=depth, is_dir=True, size=None, status=None))
        previous = tuple(directories)
        size, status = entries[path]
        rows.append(
            TreeNode(
                name=filename,
                depth=len(directories),
                is_dir=False,
                size=size,
                status=status,
            )
        )
    return tuple(rows)

blob_url(store, digest)

Build the URL a blob is served from.

A local store fans its blobs out by the first two digest characters, so a flat URL under its root would point at nothing.

Parameters:

Name	Type	Description	Default
store	NativeStore	The store the blob lives in.	required
digest	str	The blob's sha256 hex digest.	required

Returns:

Type	Description
str	An absolute URL a browser can open.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

def blob_url(store: NativeStore, digest: str) -> str:
    """
    Build the URL a blob is served from.

    A local store fans its blobs out by the first two digest characters, so a flat URL under
    its root would point at nothing.

    Parameters
    ----------
    store
        The store the blob lives in.
    digest
        The blob's sha256 hex digest.

    Returns
    -------
    :
        An absolute URL a browser can open.
    """
    if store.root is not None:
        return (store.root / digest[:2] / digest).absolute().as_uri()
    return f"{store.url.rstrip('/')}/{digest}"

committed_diff(change)

Build the unified diff of one committed regression artefact.

Parameters:

Name	Type	Description	Default
change	CommittedChange	The artefact that moved, carrying both sides' content.	required

Returns:

Type	Description
TextDiff	The diff, or a note explaining why there is not one.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

def committed_diff(change: CommittedChange) -> TextDiff:
    """
    Build the unified diff of one committed regression artefact.

    Parameters
    ----------
    change
        The artefact that moved, carrying both sides' content.

    Returns
    -------
    :
        The diff, or a note explaining why there is not one.
    """
    if change.old is not None and change.old_text is None:
        return TextDiff(lines=(), note="could not read the base version from git", elided=0)
    if change.new is not None and change.new_text is None:
        return TextDiff(lines=(), note="could not read the working tree version", elided=0)
    return _build_text_diff(
        _text_lines(change.old_text, change.name),
        _text_lines(change.new_text, change.name),
        has_old=change.old is not None,
        has_new=change.new is not None,
    )

netcdf_diff(old, new)

Diff the headers and whole-array statistics of two NetCDF blobs.

Decoding is turned off so a non-standard calendar or unit cannot fail the report.

Parameters:

Name	Type	Description	Default
old	Path | None	The base blob, or None when the file is new.	required
new	Path | None	The head blob, or None when the file was removed.	required

Returns:

Type	Description
NetcdfDiff	The header diff and one row per data variable, or a note when a file could not be opened.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

def netcdf_diff(old: Path | None, new: Path | None) -> NetcdfDiff:
    """
    Diff the headers and whole-array statistics of two NetCDF blobs.

    Decoding is turned off so a non-standard calendar or unit cannot fail the report.

    Parameters
    ----------
    old
        The base blob, or ``None`` when the file is new.
    new
        The head blob, or ``None`` when the file was removed.

    Returns
    -------
    :
        The header diff and one row per data variable, or a note when a file could not be opened.
    """
    try:
        with ExitStack() as stack:
            old_ds = (
                stack.enter_context(xr.open_dataset(old, decode_times=False, decode_cf=False))
                if old is not None
                else None
            )
            new_ds = (
                stack.enter_context(xr.open_dataset(new, decode_times=False, decode_cf=False))
                if new is not None
                else None
            )
            for dataset in (old_ds, new_ds):
                if dataset is not None and dataset.nbytes > MAX_DECODED_BYTES:
                    return NetcdfDiff(
                        header=(),
                        header_old=(),
                        header_new=(),
                        rows=(),
                        note=f"decodes to too much to analyse ({dataset.nbytes:,} B)",
                    )
            header_old = _header(old_ds)
            header_new = _header(new_ds)
            header = _header_diff(header_old, header_new)
            names = sorted(
                {
                    str(name)
                    for dataset in (old_ds, new_ds)
                    if dataset is not None
                    for name in dataset.data_vars
                }
            )
            rows = tuple(_stat_row(old_ds, new_ds, name) for name in names)
    except (OSError, ValueError, KeyError) as exc:
        return NetcdfDiff(header=(), header_old=(), header_new=(), rows=(), note=f"could not open: {exc}")
    return NetcdfDiff(
        header=header,
        header_old=tuple(header_old),
        header_new=tuple(header_new),
        rows=rows,
        note=None,
    )

text_diff(old, new, name)

Build the unified diff between two fetched blobs.

Parameters:

Name	Type	Description	Default
old	Path | None	The base blob, or None when the file is new.	required
new	Path | None	The head blob, or None when the file was removed.	required
name	str	The file's name, used in the diff header and to detect JSON.	required

Returns:

Type	Description
TextDiff	The diff, or a note explaining why there is not one.

Source code in packages/climate-ref/src/climate_ref/baseline_report/analyse.py

def text_diff(old: Path | None, new: Path | None, name: str) -> TextDiff:
    """
    Build the unified diff between two fetched blobs.

    Parameters
    ----------
    old
        The base blob, or ``None`` when the file is new.
    new
        The head blob, or ``None`` when the file was removed.
    name
        The file's name, used in the diff header and to detect JSON.

    Returns
    -------
    :
        The diff, or a note explaining why there is not one.
    """
    return _build_text_diff(
        _text_lines(_decode(old), name),
        _text_lines(_decode(new), name),
        has_old=old is not None,
        has_new=new is not None,
    )