Using pre-computed results¶
Results computed by the CMIP7 Assessment Fast Track Rapid Evaluation Framework are available from the website: https://dashboard.climate-ref.org and the associated API: https://api.climate-ref.org. This API provides an OpenAPI schema that documents what queries are available. The API documentation can be viewed at: https://api.climate-ref.org/docs.
This Jupyter notebook shows how to use this API to download pre-computed results and use those to do your own analyses.
Generate and install¶
We start by generating and installing a Python package for interacting with the API from the OpenAPI-compatible schema.
We pin the generated package name via an openapi-python-client config override so
that the Python import names below stay stable even if the API's info.title is
renamed upstream.
!printf 'package_name_override: climate_ref_client\nproject_name_override: climate-ref-client\n' > /tmp/openapi-client-config.yaml
!uvx --quiet --from openapi-python-client openapi-python-client generate --url https://api.climate-ref.org/api/v1/openapi.json --config /tmp/openapi-client-config.yaml --meta setup --output-path climate_ref_client --overwrite
Generating climate_ref_client
%pip install --quiet ./climate_ref_client
Note: you may need to restart the kernel to use updated packages.
from pathlib import Path
import cartopy.crs
import matplotlib.pyplot as plt
import pandas as pd
import requests
import seaborn as sns
import xarray as xr
from climate_ref_client import Client
from climate_ref_client.api.diagnostics import (
diagnostics_list,
diagnostics_list_metric_values,
)
from climate_ref_client.api.executions import executions_get
from climate_ref_client.models.metric_value_type import (
MetricValueType,
)
from IPython.display import Markdown
from pandas_indexing import formatlevel
%load_ext rich
View the available diagnostics¶
We start by setting up a client for interacting with the server:
client = Client("https://api.climate-ref.org")
Retrieve the available diagnostics from the server, and inspect the first one:
diagnostics = diagnostics_list.sync(client=client).data
diagnostics[0]
DiagnosticSummary( id=1, provider=ProviderSummary(slug='esmvaltool', name='ESMValTool', additional_properties={}), slug='climate-at-global-warming-levels', name='Climate at Global Warming Levels', description='Calculate climate variables at global warming levels (e.g. 1.5C, 2C, 3C, 4C above pre-industrial).', execution_groups=[1, 2, 3, 4], has_metric_values=True, has_scalar_values=True, has_series_values=False, execution_count=19, successful_execution_count=12, execution_group_count=4, successful_execution_group_count=3, group_by=[ GroupBy(source_type='cmip6', group_by=['experiment_id'], additional_properties={}), GroupBy(source_type='cmip7', group_by=['experiment_id'], additional_properties={}) ], aft_link=AFTDiagnosticDetail( id='5.3', name='Evaluation of key climate variables at global warming levels', theme='Impacts and Adaptation', version_control='version 1 - 24-11-04 REF launch', reference_dataset='GPCP-SG-2-3, HadCRUT5-0-2-0', endorser='CMIP Model Benchmarking Task Team', provider_link='https://docs.esmvaltool.org/en/latest/recipes/recipe_calculate_gwl_exceedance_stats.html', description='This recipe calculates years of Global Warming Level (GWL) as described in Swaminathan et al (2022). Time series of the anomalies in annual global mean surface air temperature (GSAT) are calculated with respect to the 1850-1900 time-mean of each individual time series. To limit the influence of short-term variability, a 21-year centered running mean is applied to the time series. Once the years of exceedance are calculated, the time averaged global mean and standard deviation for the multimodel ensemble over the 21-year period around the year of exceedance are plotted. By selecting specific scenarios, the multimodel mean and spread for a single future scenario can be plotted. Multimodel mean and standard deviation global maps of precipitation and temperature anomalies are plotted for each available global warming level.', short_description='This diagnostic calculates years of Global Warming Level (GWL) exceedances in CMIP models as described in Swaminathan et al (2022). Time series of the anomalies in annual global mean surface air temperature (GSAT) are calculated with respect to the 1850-1900 time-mean of each individual time series. To limit the influence of short-term variability, a 21-year centered running mean is applied to the time series. The year at which the time series exceeds warming levels or temperatures such as 1.5C is then recorded for the specific model ensemble member and future scenario. Once the years of exceedance are calculated, the time averaged global mean and standard deviation for the multimodel ensemble over the 21-year period around the year of exceedance are plotted.', diagnostics=[ RefDiagnosticLink( provider_slug='esmvaltool', diagnostic_slug='climate-at-global-warming-levels', additional_properties={} ) ], additional_properties={} ), reference_datasets=[], tags=['scenarios', 'warming-levels'], additional_properties={} )
To get an idea of what is available, we create a list of all diagnostics with short descriptions (a full overview is available in Appendix C of Hoffman et al., 2025):
txt = ""
for diagnostic in sorted(diagnostics, key=lambda diagnostic: diagnostic.name):
title = f"### {diagnostic.name} - {diagnostic.slug}"
description = diagnostic.description.strip()
if not description.endswith("."):
description += "."
if diagnostic.aft_link:
description += f" {diagnostic.aft_link.short_description.strip()}"
if not description.endswith("."):
description += "."
if not description.endswith("."):
description += "."
txt += f"{title}\n{description}\n\n"
Markdown(txt)
Annual Cycle Analysis - annual-cycle¶
Calculate the annual cycle for a dataset. Variables provided include surface temperature (ts), precipitation (pr), winds (uas, vas), sea level pressure (psl), air temperature (ta), zonal/meridional wind (ua, va), geopotential height (zg), and radiation fields (rlds, rlus, rsds, rsdt, rsus, rlut, rsut) including clear-sky variants. Maps of seasonal and annual climatology are generated for the reference datasets and model output on a target grid (2.5x2.5 deg), then calculate diverse metrics including bias, RMSE, spatial pattern correlation, and standard deviation.
Arctic and Antarctic Sea Ice Area Seasonal Cycle - sea-ice-area-basic¶
Calculate the seasonal cycle and time series of Northern Hemisphere and Southern Hemisphere sea ice area. Seasonal cycle of Arctic (NH) and Antarctic (SH) sea ice area, time series of Arctic September (NH) and Antarctic February (SH) sea ice area.
Atlantic Meridional Overturning Circulation (RAPID) - amoc-rapid¶
Apply the standard ILAMB analysis of the Atlantic Meridional Overturning Circulation (AMOC) with respect to the RAPID array reference dataset. Comparison of the model Atlantic meridional ocean circulation (AMOC) circulation strength with reference data from RAPID-v2023-1.
Burnt Fraction (GFED) - burntfractionall-gfed¶
Apply the standard ILAMB analysis of burnt area fraction (burntFractionAll) with respect to the GFED reference dataset. The diagnostic relies on the processing of fire climate drivers through the ConFire model and is based on Jones et al. (2024). The diagnostic computes the burnt fraction for each grid cell based on a number of drivers. Additionally, the respective controls due to fire weather and fuel load/continuity are computed. The stochastic control corresponds to the unmodelled processed influencing to fire occurrence.
Climate Drivers for Fire - climate-drivers-for-fire¶
Calculate diagnostics regarding climate drivers for fire, including temperature, precipitation, and humidity conditions that influence fire risk. The diagnostic relies on the processing of fire climate drivers through the ConFire model and is based on Jones et al. (2024). The diagnostic computes the burnt fraction for each grid cell based on a number of drivers. Additionally, the respective controls due to fire weather and fuel load/continuity are computed. The stochastic control corresponds to the unmodelled processed influencing to fire occurrence.
Climate at Global Warming Levels - climate-at-global-warming-levels¶
Calculate climate variables at global warming levels (e.g. 1.5C, 2C, 3C, 4C above pre-industrial). This diagnostic calculates years of Global Warming Level (GWL) exceedances in CMIP models as described in Swaminathan et al (2022). Time series of the anomalies in annual global mean surface air temperature (GSAT) are calculated with respect to the 1850-1900 time-mean of each individual time series. To limit the influence of short-term variability, a 21-year centered running mean is applied to the time series. The year at which the time series exceeds warming levels or temperatures such as 1.5C is then recorded for the specific model ensemble member and future scenario. Once the years of exceedance are calculated, the time averaged global mean and standard deviation for the multimodel ensemble over the 21-year period around the year of exceedance are plotted.
Cloud Radiative Effects - cloud-radiative-effects¶
Plot climatologies and zonal mean profiles of cloud radiative effects (shortwave + longwave) for a dataset. Maps and zonal means of longwave and shortwave cloud radiative effect.
Cloud Scatterplots for Reference dataset - cloud-scatterplots-reference¶
Reference scatterplots of two cloud-relevant variables from observational datasets. These provide the observational baseline for comparison with model cloud scatterplot diagnostics. 2D histograms with focus on clouds.
Cloud-Precipitation Scatterplots (clwvi vs pr) - cloud-scatterplots-clwvi-pr¶
Scatterplot of condensed water path (clwvi) vs precipitation (pr). 2D histograms with focus on clouds.
Cloud-Radiation Scatterplots (clivi vs lwcre) - cloud-scatterplots-clivi-lwcre¶
Scatterplot of ice water path (clivi) vs longwave cloud radiative effect (lwcre). 2D histograms with focus on clouds.
Cloud-Radiation Scatterplots (clt vs swcre) - cloud-scatterplots-clt-swcre¶
Scatterplot of total cloud fraction (clt) vs shortwave cloud radiative effect (swcre). 2D histograms with focus on clouds.
Cloud-Temperature Scatterplots (cli vs ta) - cloud-scatterplots-cli-ta¶
Scatterplot of ice water content of cloud (cli) vs air temperature (ta). Observational scatterplots for reference datasets are provided in the separate "Cloud Scatterplots for Reference dataset" diagnostic. 2D histograms with focus on clouds.
ENSO Basic Climatology - enso-basic-climatology¶
Calculate the ENSO CLIVAR metrics for background climatology, assessing the mean state of the tropical Pacific upon which ENSO events develop.
ENSO Characteristics - enso-characteristics¶
Calculate the ENSO CLIVAR metrics for basic ENSO characteristics, evaluating the amplitude, frequency, and spatial structure of ENSO events.
ENSO Processes - enso_proc¶
Calculate the ENSO process-oriented metrics for a dataset. These metrics evaluate the physical processes that drive ENSO, including ocean-atmosphere coupling, thermocline feedbacks, and energy balance.
ENSO Teleconnections - enso_tel¶
Calculate the ENSO teleconnection metrics for a dataset. Teleconnection metrics assess how well a model captures the remote climate impacts of ENSO events on temperature and precipitation patterns worldwide.
Equilibrium Climate Sensitivity (ECS) - equilibrium-climate-sensitivity¶
Calculate the global mean equilibrium climate sensitivity for a dataset. ECS is defined as the long-term global mean surface temperature change in response to a doubling of atmospheric CO2 concentration, estimated using the Gregory regression method from abrupt-4xCO2 experiments. Equilibrium climate sensitivity is defined as the change in global mean temperature as a result of a doubling of the atmospheric CO2 concentration compared to pre-industrial times after the climate system has reached a new equilibrium. This diagnostic uses a regression method based on Gregory et al. (2004).
Global Mean Timeseries - global-mean-timeseries¶
Calculate the annual mean global mean timeseries for a dataset.
Gross Primary Productivity (FLUXNET2015) - gpp-fluxnet2015¶
Apply the standard ILAMB analysis of gross primary productivity (gpp) with respect to the FLUXNET2015 tower-based reference dataset. Apply the ILAMB Methodology* to compare the model gross primary production (GPP) with reference data from FLUXNET2015-1-0 and WECANN-1-0.
Gross Primary Productivity (WECANN) - gpp-wecann¶
Apply the standard ILAMB analysis of gross primary productivity (gpp) with respect to the WECANN reference dataset. Apply the ILAMB Methodology* to compare the model gross primary production (GPP) with reference data from FLUXNET2015-1-0 and WECANN-1-0.
Leaf Area Index (AVH15C1) - lai-avh15c1¶
Apply the standard ILAMB analysis of leaf area index (lai) with respect to the AVH15C1 (AVHRR) reference dataset. Apply the ILAMB Methodology* to compare the model leaf area index (LAI) with reference data from NOAA-NCEI-LAI-5-0, and LAI4g-1-2.
Net Biome Productivity (Hoffman) - nbp-hoffman¶
Apply the standard ILAMB analysis of net biome productivity (nbp) with respect to the Hoffman reference dataset. Comparison of the model integrated land net ecosystem carbon balance with reference data from HOFFMAN-1-0.
North Atlantic Oscillation (NAO) - extratropical-modes-of-variability-nao¶
Calculate the extratropical modes of variability for the North Atlantic Oscillation (NAO), a large-scale pattern of sea level pressure variability between the Icelandic Low and Azores High. For extratropical modes of variability, maps of variability mode pattern and their principal component time series are generated from the reference dataset and model output. Then maps are compared to calculate bias, RMSE, and spatial pattern correlation, and time series are compared to calculate the ratio from their standard deviations.
North Pacific Gyre Oscillation (NPGO) - extratropical-modes-of-variability-npgo¶
Calculate the extratropical modes of variability for the North Pacific Gyre Oscillation (NPGO), which tracks changes in the strength of the North Pacific subtropical and subpolar gyres. For extratropical modes of variability, maps of variability mode pattern and their principal component time series are generated from the reference dataset and model output. Then maps are compared to calculate bias, RMSE, and spatial pattern correlation, and time series are compared to calculate the ratio from their standard deviations.
North Pacific Oscillation (NPO) - extratropical-modes-of-variability-npo¶
Calculate the extratropical modes of variability for the North Pacific Oscillation (NPO), defined by sea level pressure variability over the North Pacific. For extratropical modes of variability, maps of variability mode pattern and their principal component time series are generated from the reference dataset and model output. Then maps are compared to calculate bias, RMSE, and spatial pattern correlation, and time series are compared to calculate the ratio from their standard deviations.
Northern Annular Mode (NAM) - extratropical-modes-of-variability-nam¶
Calculate the extratropical modes of variability for the Northern Annular Mode (NAM), the dominant mode of sea level pressure variability in the Northern Hemisphere extratropics. For extratropical modes of variability, maps of variability mode pattern and their principal component time series are generated from the reference dataset and model output. Then maps are compared to calculate bias, RMSE, and spatial pattern correlation, and time series are compared to calculate the ratio from their standard deviations.
Ozone Diagnostics - ozone-annual-cycle¶
Calculate the ozone diagnostics - zonal mean total column ozone vs. annual cycle plot.
Ozone Diagnostics - ozone-lat-time¶
Calculate the ozone diagnostics - zonal mean total column ozone vs. time.
Ozone Diagnostics - ozone-nh-mar¶
Calculate the ozone diagnostics - March NH polar mean (60N-80N) time series.
Ozone Diagnostics - ozone-sh-oct¶
Calculate the ozone diagnostics - October SH polar mean (60S-85S) time series.
Pacific Decadal Oscillation (PDO) - extratropical-modes-of-variability-pdo¶
Calculate the extratropical modes of variability for the Pacific Decadal Oscillation (PDO), a pattern of Pacific climate variability characterized by SST anomalies in the North Pacific. For extratropical modes of variability, maps of variability mode pattern and their principal component time series are generated from the reference dataset and model output. Then maps are compared to calculate bias, RMSE, and spatial pattern correlation, and time series are compared to calculate the ratio from their standard deviations.
Pacific North American Pattern (PNA) - extratropical-modes-of-variability-pna¶
Calculate the extratropical modes of variability for the Pacific North American (PNA) pattern, a prominent mode of atmospheric variability influencing weather over the Pacific and North America. For extratropical modes of variability, maps of variability mode pattern and their principal component time series are generated from the reference dataset and model output. Then maps are compared to calculate bias, RMSE, and spatial pattern correlation, and time series are compared to calculate the ratio from their standard deviations.
Regional Historical Annual Cycle - regional-historical-annual-cycle¶
Plot the regional historical annual cycle of climate variables including near-surface air temperature (tas), precipitation (pr), sea level pressure (psl), zonal wind (ua), and specific humidity (hus). Regions are defined using IPCC AR6 reference regions as described by Iturbide et al. (2020). Time series, linear trend, and annual cycle for IPCC regions.
Regional Historical Timeseries - regional-historical-timeseries¶
Plot the regional historical mean and anomaly timeseries of climate variables including near-surface air temperature (tas), precipitation (pr), sea level pressure (psl), zonal wind (ua), and specific humidity (hus). Regions are defined using IPCC AR6 reference regions as described by Iturbide et al. (2020). Time series, linear trend, and annual cycle for IPCC regions.
Regional Historical Trends - regional-historical-trend¶
Plot the regional historical trend of climate variables including near-surface air temperature (tas), precipitation (pr), sea level pressure (psl), zonal wind (ua), and specific humidity (hus). Regions are defined using IPCC AR6 reference regions as described by Iturbide et al. (2020). Time series, linear trend, and annual cycle for IPCC regions.
Runoff (LORA) - mrro-lora¶
Apply the standard ILAMB analysis of total runoff (mrro) with respect to the LORA reference dataset. Apply the ILAMB Methodology* to compare the model surface runoff with reference data from LORA-1-1.
Sea Ice Sensitivity - sea-ice-sensitivity¶
Calculate sea ice sensitivity, quantifying the relationship between sea ice area (siconc) and global temperature change (tas). This is an inter-model comparison diagnostic with no observational datasets ingested. Rate of sea ice area loss per degree of warming, as in plot 1d of Notz et al. and figure 3e of Roach et al.
Sea Water Salinity (WOA2023 Surface) - so-woa2023-surface¶
Apply the standard ILAMB analysis of sea water salinity (so) at the surface with respect to the WOA2023 reference dataset. Apply the ILAMB Methodology* to compare the model sea surface temperature (SST) and sea surface salinity (SSS) with reference data from WOA2023.
Sea Water Temperature (WOA2023 Surface) - thetao-woa2023-surface¶
Apply the standard ILAMB analysis of sea water potential temperature (thetao) at the surface with respect to the WOA2023 reference dataset. Apply the ILAMB Methodology* to compare the model sea surface temperature (SST) and sea surface salinity (SSS) with reference data from WOA2023.
Snow Cover (ESA CCI) - snc-esacci¶
Apply the standard ILAMB analysis of snow cover fraction (snc) with respect to the ESA CCI CryoClim fractional snow cover reference dataset. Apply the ILAMB Methodology* to compare the model snow cover with reference data from CCI-CryoClim-FSC-1.
Soil Carbon (HWSD2) - csoil-hwsd2¶
Apply the standard ILAMB analysis of carbon mass in soil pool (cSoil) with respect to the HWSD2 reference dataset. Apply the ILAMB Methodology* to compare the model soil carbon with reference data from HWSD-2-0 and NCSCD-2-2.
Southern Annular Mode (SAM) - extratropical-modes-of-variability-sam¶
Calculate the extratropical modes of variability for the Southern Annular Mode (SAM), the dominant mode of sea level pressure variability in the Southern Hemisphere extratropics. For extratropical modes of variability, maps of variability mode pattern and their principal component time series are generated from the reference dataset and model output. Then maps are compared to calculate bias, RMSE, and spatial pattern correlation, and time series are compared to calculate the ratio from their standard deviations.
Surface Soil Moisture (WangMao) - mrsos-wangmao¶
Apply the standard ILAMB analysis of moisture in upper portion of soil column (mrsos) with respect to the WangMao reference dataset. Apply the ILAMB Methodology* to compare the model surface soil moisture with reference data from OLC-ORS-V0.
Transient Climate Response (TCR) - transient-climate-response¶
Calculate the global mean transient climate response for a dataset. TCR is defined as the global mean surface temperature change at the time of CO2 doubling in a 1pctCO2 experiment (1% per year CO2 increase). The transient climate response (TCR) is defined as the global and annual mean surface air temperature anomaly in the 1pctCO2 scenario (1% CO2 increase per year) for a 20 year period centered at the time of CO2 doubling, i.e. using the years 61 to 80 after the start of the simulation. We calculate the temperature anomaly by subtracting a linear fit of the piControl run for all 140 years of the 1pctCO2 experiment prior to the TCR calculation (see Gregory and Forster, 2008).
Transient Climate Response to Emissions (TCRE) - transient-climate-response-emissions¶
Calculate the global mean Transient Climate Response to Cumulative CO2 Emissions. TCRE quantifies the relationship between global warming and total cumulative carbon emissions, a key metric for carbon budget estimation. This diagnostic calculates the linear trend in global mean surface temperature warming (since the preindustrial) to respective cumulative anthropogenic CO2 emissions, usually presented in units of degrees C per GtCO2. This relationship is usually calculated from historical followed by future scenarios simulations.
Zero Emission Commitment (ZEC) - zero-emission-commitment¶
Calculate the global mean Zero Emission Commitment (ZEC) temperature. ZEC measures the additional warming (or cooling) that occurs after all CO2 emissions cease, reflecting the committed warming from past emissions. The Zero Emissions Commitment (ZEC) quantifies the change in global mean temperature expected to occur after net carbon dioxide (CO2) emissions cease. ZEC is therefore important to consider when estimating the remaining carbon budget. Calculation of ZEC requires dedicated simulations with emissions set to zero, branching off a base simulation with emissions. In CMIP6 the simulations were part of ZECMIP, with the simulations called esm-1pct-brch-xPgC branching off the 1pctCO2 simulation when emissions reach x PgC. The default x was 1000PgC, with additional simulations for 750PgC and 2000PgC. In CMIP7, ZEC simulations (esm-flat10-zec) are part of the fast track and branch off (esm-flat10) with constant emissions of 10GtC/yr at year 100 (Sanderson 2024).
emp-GLEAMGPCP2.3 - emp-gleamgpcp2.3¶
Apply the standard ILAMB analysis with respect to a given reference dataset.
Metrics¶
Many of the diagnostics provide "metric" values, single values that describe some property of a model. Here we show how to access these values and create a plot.
# Select the "Atlantic Meridional Overturning Circulation (RAPID)"
# diagnostic as an example
diagnostic_name = "Atlantic Meridional Overturning Circulation (RAPID)"
diagnostic = next(d for d in diagnostics if d.name == diagnostic_name)
# Inspect an example value.
diagnostics_list_metric_values.sync(
diagnostic.provider.slug,
diagnostic.slug,
value_type=MetricValueType.SCALAR,
client=client,
).data[0]
ScalarValue( dimensions=ScalarValueDimensions( additional_properties={ 'source_id': 'CAS-ESM2-0', 'experiment_id': 'historical', 'grid_label': 'gn', 'member_id': 'r1i1p1f1', 'metric': 'timeseries', 'region': 'None', 'statistic': 'Bias Score' } ), value=0.9245334852043176, id=89320, execution_group_id=3539, execution_id=3539, attributes=None, is_outlier=False, verification_status=<ScalarValueVerificationStatusType0.VERIFIED: 'verified'>, additional_properties={} )
Read the metric values into a Pandas DataFrame:
df = (
pd.DataFrame(
metric.dimensions.additional_properties | {"value": metric.value}
for metric in diagnostics_list_metric_values.sync(
diagnostic.provider.slug,
diagnostic.slug,
value_type=MetricValueType.SCALAR,
client=client,
).data
)
.replace("None", pd.NA)
.drop_duplicates()
)
# Drop a few columns that appear to be the same for all entries of
# particular diagnostic.
df.drop(columns=["experiment_id", "metric", "region"], inplace=True)
# Use the columns that do not contain the metric value for indexing
df.set_index([c for c in df.columns if c != "value"], inplace=True)
df
| value | ||||
|---|---|---|---|---|
| source_id | grid_label | member_id | statistic | |
| CAS-ESM2-0 | gn | r1i1p1f1 | Bias Score | 0.924533 |
| CESM2 | gn | r1i1p1f1 | Bias Score | 0.895831 |
| CESM2-WACCM | gn | r1i1p1f1 | Bias Score | 0.889744 |
| CanESM5 | gn | r1i1p1f1 | Bias Score | 0.765565 |
| CanESM5-1 | gn | r1i1p1f1 | Bias Score | 0.765364 |
| FGOALS-g3 | gn | r1i1p1f1 | Bias Score | 0.435503 |
| CanESM5-CanOE | gn | r1i1p2f1 | Bias Score | 0.770994 |
| E3SM-1-1 | gr | r1i1p1f1 | Bias Score | 0.851708 |
| E3SM-1-1-ECA | gr | r1i1p1f1 | Bias Score | 0.864512 |
| MIROC-ES2L | gr | r1i1p1f2 | Bias Score | 0.838881 |
| MIROC6 | gr | r1i1p1f1 | Bias Score | 0.844038 |
| INM-CM5-0 | gr1 | r1i1p1f1 | Bias Score | 0.858195 |
| MRI-ESM2-0 | gr2z | r1i1p1f1 | Bias Score | 0.950370 |
| NorCPM1 | grz | r1i1p1f1 | Bias Score | 0.396918 |
| ACCESS-CM2 | gn | r1i1p1f1 | Bias Score | 0.891568 |
| GISS-E2-2-G | gn | r1i1p1f1 | Bias Score | 0.712438 |
| CESM2-WACCM-FV2 | gn | r1i1p1f1 | Bias Score | 0.886149 |
| ACCESS-ESM1-5 | gn | r1i1p1f1 | Bias Score | 0.874716 |
| CESM2-FV2 | gn | r1i1p1f1 | Bias Score | 0.875639 |
| ICON-ESM-LR | gn | r1i1p1f1 | Bias Score | 0.939255 |
| MPI-ESM-1-2-HAM | gn | r1i1p1f1 | Bias Score | 0.835892 |
| MPI-ESM1-2-LR | gn | r1i1p1f1 | Bias Score | 0.965773 |
| MPI-ESM1-2-HR | gn | r1i1p1f1 | Bias Score | 0.972627 |
| NorESM2-MM | grz | r1i1p1f1 | Bias Score | 0.765709 |
| NorESM2-LM | grz | r1i1p4f1 | Bias Score | 0.816985 |
| SAM0-UNICON | gn | r1i1p1f1 | Bias Score | 0.824277 |
| CAS-ESM2-0 | gn | r1i1p1f1 | Correlation | 0.268867 |
| CESM2 | gn | r1i1p1f1 | Correlation | 0.235022 |
| CESM2-WACCM | gn | r1i1p1f1 | Correlation | 0.220829 |
| CanESM5 | gn | r1i1p1f1 | Correlation | 0.130241 |
| CanESM5-1 | gn | r1i1p1f1 | Correlation | 0.218495 |
| FGOALS-g3 | gn | r1i1p1f1 | Correlation | 0.187015 |
| CanESM5-CanOE | gn | r1i1p2f1 | Correlation | 0.102872 |
| E3SM-1-1 | gr | r1i1p1f1 | Correlation | 0.058833 |
| E3SM-1-1-ECA | gr | r1i1p1f1 | Correlation | 0.213765 |
| MIROC-ES2L | gr | r1i1p1f2 | Correlation | -0.089479 |
| MIROC6 | gr | r1i1p1f1 | Correlation | 0.131496 |
| INM-CM5-0 | gr1 | r1i1p1f1 | Correlation | 0.300088 |
| MRI-ESM2-0 | gr2z | r1i1p1f1 | Correlation | 0.109972 |
| NorCPM1 | grz | r1i1p1f1 | Correlation | 0.093051 |
| ACCESS-CM2 | gn | r1i1p1f1 | Correlation | 0.285341 |
| GISS-E2-2-G | gn | r1i1p1f1 | Correlation | -0.018144 |
| CESM2-WACCM-FV2 | gn | r1i1p1f1 | Correlation | 0.219790 |
| ACCESS-ESM1-5 | gn | r1i1p1f1 | Correlation | -0.001145 |
| CESM2-FV2 | gn | r1i1p1f1 | Correlation | 0.275772 |
| ICON-ESM-LR | gn | r1i1p1f1 | Correlation | -0.102087 |
| MPI-ESM-1-2-HAM | gn | r1i1p1f1 | Correlation | 0.268063 |
| MPI-ESM1-2-LR | gn | r1i1p1f1 | Correlation | 0.404317 |
and create a portrait diagram:
# Use the median metric value for models with multiple ensemble
# members to keep the figure readable.
df = df.groupby(level=["source_id", "grid_label", "statistic"]).median()
# Convert df to a "2D" dataframe for use with the seaborn heatmap plot
df_2D = (
formatlevel(df, model="{source_id}.{grid_label}", drop=True)
.reset_index()
.pivot(columns="statistic", index="model", values="value")
)
figure, ax = plt.subplots(figsize=(5, 8))
_ = sns.heatmap(
df_2D / df_2D.median(),
annot=df_2D,
cmap="viridis",
linewidths=0.5,
ax=ax,
cbar_kws={"label": "Color indicates value relative to the median"},
)
Series¶
Many of the diagnostics provide "series" values, a range of values along with an index that describe some property of a model. Here we show how to access these values and create a plot.
# Select the "Sea Ice Area Basic Metrics" diagnostic as an example
diagnostic_slug = "sea-ice-area-basic"
diagnostic = next(d for d in diagnostics if d.slug == diagnostic_slug)
# Inspect an example series value:
diagnostics_list_metric_values.sync(
diagnostic.provider.slug,
diagnostic_slug=diagnostic_slug,
value_type=MetricValueType.SERIES,
client=client,
).data[0]
SeriesValue( id=29680, dimensions=SeriesValueDimensions( additional_properties={ 'source_id': 'ACCESS-CM2', 'grid_label': 'gn', 'member_id': 'r1i1p1f1', 'region': 'Northern Hemisphere', 'statistic': 'September sea ice area' } ), values=[ None, 5.704837322235107, None, 6.520887851715088, 5.828301429748535, None, None, 5.755328178405762, 7.1296868324279785, None, 6.804515838623047, None, None, 6.367095947265625, 5.717348575592041, None, 6.017303466796875, 6.5888590812683105, 6.297767639160156, 6.413061141967773, 5.822549819946289, 6.876488208770752, 6.782400131225586, 5.891390323638916, 5.606761932373047, 5.219797134399414, 4.819399356842041, 5.333940505981445, 5.910629749298096, 5.041290760040283, 5.444366931915283, 4.584399700164795, 5.286465167999268, 4.972649574279785, 5.357391357421875, 5.821165084838867, 5.6602044105529785, 5.049866199493408, 5.476050853729248, 5.586740016937256, 4.472897052764893, 4.012683391571045, 4.192861080169678, 4.2967119216918945 ], execution_group_id=2076, execution_id=2076, index=[ '1979-09-15T12:00:00', '1979-09-16T00:00:00', '1980-09-15T12:00:00', '1980-09-16T00:00:00', '1981-09-16T00:00:00', '1981-09-16T12:00:00', '1982-09-15T12:00:00', '1982-09-16T00:00:00', '1983-09-16T00:00:00', '1983-09-16T12:00:00', '1984-09-16T00:00:00', '1984-09-16T12:00:00', '1985-09-15T12:00:00', '1985-09-16T00:00:00', '1986-09-16T00:00:00', '1986-09-16T12:00:00', '1987-09-16T00:00:00', '1988-09-16T00:00:00', '1989-09-16T00:00:00', '1990-09-16T00:00:00', '1991-09-16T00:00:00', '1992-09-16T00:00:00', '1993-09-16T00:00:00', '1994-09-16T00:00:00', '1995-09-16T00:00:00', '1996-09-16T00:00:00', '1997-09-16T00:00:00', '1998-09-16T00:00:00', '1999-09-16T00:00:00', '2000-09-16T00:00:00', '2001-09-16T00:00:00', '2002-09-16T00:00:00', '2003-09-16T00:00:00', '2004-09-16T00:00:00', '2005-09-16T00:00:00', '2006-09-16T00:00:00', '2007-09-16T00:00:00', '2008-09-16T00:00:00', '2009-09-16T00:00:00', '2010-09-16T00:00:00', '2011-09-16T00:00:00', '2012-09-16T00:00:00', '2013-09-16T00:00:00', '2014-09-16T00:00:00' ], index_name='time', attributes=SeriesValueAttributesType0( additional_properties={ 'calendar': 'standard', 'caption': 'Time series of NH September/SH February sea ice area in million km2 from ACCESS-CM2.r1i1p1f1.gn compared with OSI-450.', 'index_long_name': 'time', 'index_name': 'time', 'index_standard_name': 'time', 'value_long_name': 'Sea-Ice Area Percentage (Ocean Grid)', 'value_units': '1e6 km2', 'values_name': 'siconc' } ), additional_properties={} )
Read the metric values into a Pandas DataFrame:
statistic_name = "20-year average seasonal cycle"
value_name = "sea ice area (1e6 km2)"
df = pd.DataFrame(
metric.dimensions.additional_properties | {value_name: value, "month": int(month)}
for metric in diagnostics_list_metric_values.sync(
diagnostic.provider.slug,
diagnostic.slug,
value_type=MetricValueType.SERIES,
client=client,
).data
if metric.dimensions.additional_properties["statistic"].startswith(statistic_name)
for value, month in zip(metric.values, metric.index)
if value < 1e10 # Ignore some invalid values.
)
df
| source_id | grid_label | member_id | region | statistic | sea ice area (1e6 km2) | month | reference_source_id | |
|---|---|---|---|---|---|---|---|---|
| 0 | ACCESS-CM2 | gn | r1i1p1f1 | Northern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 13.886829 | 1 | NaN |
| 1 | ACCESS-CM2 | gn | r1i1p1f1 | Northern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 15.101398 | 2 | NaN |
| 2 | ACCESS-CM2 | gn | r1i1p1f1 | Northern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 15.561163 | 3 | NaN |
| 3 | ACCESS-CM2 | gn | r1i1p1f1 | Northern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 15.138600 | 4 | NaN |
| 4 | ACCESS-CM2 | gn | r1i1p1f1 | Northern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 13.367761 | 5 | NaN |
| ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 283 | CNRM-ESM2-1 | gn | r1i1p1f2 | Southern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 15.852147 | 8 | OSI-450-sh |
| 284 | CNRM-ESM2-1 | gn | r1i1p1f2 | Southern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 16.454039 | 9 | OSI-450-sh |
| 285 | CNRM-ESM2-1 | gn | r1i1p1f2 | Southern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 15.870502 | 10 | OSI-450-sh |
| 286 | CNRM-ESM2-1 | gn | r1i1p1f2 | Southern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 13.134278 | 11 | OSI-450-sh |
| 287 | CNRM-ESM2-1 | gn | r1i1p1f2 | Southern Hemisphere | 20-year average seasonal cycle of the sea ice ... | 7.807084 | 12 | OSI-450-sh |
288 rows × 8 columns
and create a plot:
_ = sns.relplot(
data=df.sort_values("source_id"),
x="month",
y=value_name,
col="region",
hue="source_id",
kind="line",
)
Files¶
Many of the diagnostics produce NetCDF files that can be used for further analysis or custom plotting. We will look at the global warming levels diagnostic and create our own figure using the available data.
Each diagnostic can be run (executed) multiple times with different input data. The global warmings levels diagnostic has been executed several times, leading to multiple "execution groups":
diagnostic_name = "Climate at Global Warming Levels"
diagnostic = next(d for d in diagnostics if d.name == diagnostic_name)
[executions_get.sync(g, client=client).key for g in diagnostic.execution_groups]
['cmip6_ssp126', 'cmip6_ssp245', 'cmip6_ssp370', 'cmip6_ssp585']
Let's select the "ssp585" scenario and look at the output files that were produced:
for group in diagnostic.execution_groups:
execution = executions_get.sync(group, client=client)
if execution.key.endswith("ssp585"):
ssp585_outputs = execution.latest_execution.outputs
break
else:
msg = "Failed to find the ssp585 execution group"
raise ValueError(msg)
[o.filename for o in ssp585_outputs]
[ 'executions/recipe_20260305_234556/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_1.5.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_2.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_3.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_4.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_5.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_1.5.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_2.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_3.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_4.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_5.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_1.5.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_2.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_3.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_4.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_5.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_1.5.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_2.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_3.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_4.0.png', 'executions/recipe_20260305_234556/plots/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_5.0.png', 'executions/recipe_20260305_234556/work/calculate_gwl_exceedance_years/gwl_exceedance_calculation/GWL_exceedance_years.csv', 'executions/recipe_20260305_234556/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_1.5.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_2.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_3.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_4.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_mean_5.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_1.5.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_2.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_3.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_4.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_pr/plot_gwl_stats/CMIP6_mm_stdev_5.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_1.5.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_2.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_3.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_4.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_5.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_1.5.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_2.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_3.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_4.0.nc', 'executions/recipe_20260305_234556/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_stdev_5.0.nc', 'executions/recipe_20260305_234556/index.html' ]
Select one of the output files and inspect it:
filename = "tas/plot_gwl_stats/CMIP6_mm_mean_2.0.nc"
file = next(f for f in ssp585_outputs if f.filename.endswith(filename))
file
ExecutionOutput( id=269512, execution_id=9206, output_type=<ResultOutputType.DATA: 'data'>, filename='executions/recipe_20260305_234556/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_2.0.nc', short_name='executions/recipe_20260305_234556/work/gwl_mean_plots_tas/plot_gwl_stats/CMIP6_mm_mean_2.0.nc', long_name='Multimodel mean of Temperature at 2.0 $^\\circ$ C', description='', created_at=datetime.datetime(2026, 3, 6, 0, 12, 26), updated_at=datetime.datetime(2026, 3, 6, 0, 12, 26), url='https://api.climate-ref.org/api/v1/results/269512', additional_properties={} )
Download the file and open it with xarray:
local_file = Path("/tmp") / Path(file.filename).name # noqa: S108
local_file.write_bytes(requests.get(file.url, timeout=120).content)
ds = xr.open_dataset(local_file).drop_vars("cube_label")
ds
<xarray.Dataset> Size: 272kB
Dimensions: (lat: 180, lon: 360, bnds: 2)
Coordinates:
* lat (lat) float64 1kB -89.5 -88.5 -87.5 -86.5 ... 86.5 87.5 88.5 89.5
* lon (lon) float64 3kB 0.5 1.5 2.5 3.5 4.5 ... 356.5 357.5 358.5 359.5
time datetime64[ns] 8B ...
Dimensions without coordinates: bnds
Data variables:
tas (lat, lon) float32 259kB ...
lat_bnds (lat, bnds) float64 3kB ...
lon_bnds (lon, bnds) float64 6kB ...
time_bnds (bnds) datetime64[ns] 16B ...
Attributes:
Conventions: CF-1.7
software: Created with ESMValTool v2.14.0rc1
caption: Multimodel mean of Temperature at 2.0 $^\circ$ CCreate our own plot:
plot = ds.tas.plot.contourf(
cmap="viridis",
vmin=-30,
vmax=30,
levels=11,
figsize=(12, 5),
transform=cartopy.crs.PlateCarree(),
subplot_kws={
"projection": cartopy.crs.Orthographic(
central_longitude=-100,
central_latitude=40,
),
},
)
_ = plot.axes.coastlines()
/home/docs/checkouts/readthedocs.org/user_builds/climate-ref/checkouts/latest/.venv/lib/python3.11/site-packages/cartopy/io/__init__.py:242: DownloadWarning: Downloading: https://naturalearth.s3.amazonaws.com/110m_physical/ne_110m_coastline.zip
warnings.warn(f'Downloading: {url}', DownloadWarning)