Testing diagnostic providers locally¶
Diagnostic providers can be run locally without requiring the rest of the REF infrastructure. This is useful for testing and debugging diagnostics.
Running a diagnostic locally requires that the target REF diagnostics package, e.g. climate_ref_example,
and its dependencies are installed in the current Python environment.
This guide will walk you through how to run a diagnostic provider locally.
In [2]:
Copied!
provider = climate_ref_example.provider
provider
provider = climate_ref_example.provider
provider
Out[2]:
DiagnosticProvider(name='Example', version='0.14.4')
We select a diagnostic which simply calculates the annual mean, global mean timeseries of a dataset.
The data requirements of this diagnostic filter out all variables except tas and rsut.
The group_by specification ensures that each execution has a unique combination of
source_id, variant_id, variable_id and experiment_id values.
In [3]:
Copied!
diagnostic = provider.get("global-mean-timeseries")
diagnostic = provider.get("global-mean-timeseries")
In [4]:
Copied!
prettyprinter.pprint(diagnostic.data_requirements[0])
prettyprinter.pprint(diagnostic.data_requirements[0])
(
climate_ref_core.diagnostics.DataRequirement(
source_type=climate_ref_core.source_types.SourceDatasetType.CMIP6,
filters=(
climate_ref_core.datasets.FacetFilter(
facets={'variable_id': ('tas', 'rsut')}
),
),
group_by=(
'source_id',
'variable_id',
'experiment_id',
'variant_label'
),
constraints=(
climate_ref_core.constraints.RequireContiguousTimerange(
group_by=('instance_id',)
),
climate_ref_core.constraints.AddSupplementaryDataset(
supplementary_facets={'variable_id': 'areacella'},
matching_facets=('source_id', 'grid_label'),
optional_matching_facets=(
'table_id',
'experiment_id',
'member_id',
'version'
)
)
)
),
)
In [5]:
Copied!
config = Config.default()
provider.configure(config)
db = Database.from_config(config)
config = Config.default()
provider.configure(config)
db = Database.from_config(config)
2026-06-05 05:54:33.876 | DEBUG | climate_ref.config:default:554 - Loading default configuration from /home/docs/.config/climate_ref/ref.toml
2026-06-05 05:54:33.892 | DEBUG | climate_ref_core.providers:configure:82 - Configuring provider example using ignore_datasets_file /home/docs/.cache/climate_ref/default_ignore_datasets.yaml
2026-06-05 05:54:33.908 | INFO | climate_ref.database:__init__:254 - Connecting to database at sqlite:////home/docs/.config/climate_ref/db/climate_ref.db
2026-06-05 05:54:33.916 | DEBUG | climate_ref.database:migrate:364 - Current database revision: b1c2d3e4f5a6
2026-06-05 05:54:33.917 | INFO | climate_ref.database:_create_backup:137 - Creating backup of database at /home/docs/.config/climate_ref/db/backups/climate_ref_20260605_055433.db
2026-06-05 05:54:33.926 | DEBUG | env_py:<module>:21 - Running alembic env
2026-06-05 05:54:33.957 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: mip_id
2026-06-05 05:54:33.958 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: source_id
2026-06-05 05:54:33.959 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: reference_source_id
2026-06-05 05:54:33.960 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: experiment_id
2026-06-05 05:54:33.961 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: variable_id
2026-06-05 05:54:33.962 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: reference_variable_id
2026-06-05 05:54:33.963 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: grid_label
2026-06-05 05:54:33.964 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: member_id
2026-06-05 05:54:33.965 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: variant_label
2026-06-05 05:54:33.966 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: table_id
2026-06-05 05:54:33.967 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: global_warming_level
2026-06-05 05:54:33.968 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: metric
2026-06-05 05:54:33.969 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: region
2026-06-05 05:54:33.970 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: season
2026-06-05 05:54:33.971 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: mode
2026-06-05 05:54:33.972 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: method
2026-06-05 05:54:33.973 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered MetricValue dimension: statistic
2026-06-05 05:54:33.974 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: mip_id
2026-06-05 05:54:33.975 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: source_id
2026-06-05 05:54:33.976 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: reference_source_id
2026-06-05 05:54:33.976 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: experiment_id
2026-06-05 05:54:33.977 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: variable_id
2026-06-05 05:54:33.978 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: reference_variable_id
2026-06-05 05:54:33.979 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: grid_label
2026-06-05 05:54:33.980 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: member_id
2026-06-05 05:54:33.981 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: variant_label
2026-06-05 05:54:33.982 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: table_id
2026-06-05 05:54:33.982 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: global_warming_level
2026-06-05 05:54:33.983 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: metric
2026-06-05 05:54:33.984 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: region
2026-06-05 05:54:33.985 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: season
2026-06-05 05:54:33.986 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: mode
2026-06-05 05:54:33.987 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: method
2026-06-05 05:54:33.988 | DEBUG | climate_ref.models.mixins:register_cv_dimensions:117 - Registered ExecutionOutput dimension: statistic
Load the data catalog containing the CMIP6 datasets.
This contains the datasets that have been ingested into the REF database.
You could also use the find_local_datasets function to find local datasets on disk,
thereby bypassing the need for a database.
In [6]:
Copied!
# Load the data catalog containing the
data_catalog = get_dataset_adapter("cmip6").load_catalog(db)
data_catalog.head()
# Load the data catalog containing the
data_catalog = get_dataset_adapter("cmip6").load_catalog(db)
data_catalog.head()
2026-06-05 05:54:33.993 | DEBUG | climate_ref.config:default:554 - Loading default configuration from /home/docs/.config/climate_ref/ref.toml
Out[6]:
| start_time | end_time | path | activity_id | branch_method | branch_time_in_child | branch_time_in_parent | experiment | experiment_id | frequency | ... | member_id | vertical_levels | version | standard_name | long_name | units | time_units | calendar | finalised | instance_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 85 | None | None | /home/docs/checkouts/readthedocs.org/user_buil... | DAMIP | standard | 0.0 | 29220.0 | historical well-mixed GHG-only run | hist-GHG | fx | ... | r2i1p1f1 | 1 | v20200615 | cell_area | Grid-Cell Area for Atmospheric Grid Variables | m2 | NaN | NaN | True | CMIP6.DAMIP.CSIRO.ACCESS-ESM1-5.hist-GHG.r2i1p... |
| 86 | 2015-01-16 12:00:00 | 2100-12-16 12:00:00 | /home/docs/checkouts/readthedocs.org/user_buil... | ScenarioMIP | standard | 60265.0 | 60265.0 | update of RCP2.6 based on SSP1 | ssp126 | mon | ... | r1i1p1f1 | 1 | v20210318 | precipitation_flux | Precipitation | kg m-2 s-1 | days since 1850-01-01 | proleptic_gregorian | True | CMIP6.ScenarioMIP.CSIRO.ACCESS-ESM1-5.ssp126.r... |
| 87 | 2015-01-16 12:00:00 | 2025-12-16 12:00:00 | /home/docs/checkouts/readthedocs.org/user_buil... | ScenarioMIP | standard | 60265.0 | 60265.0 | update of RCP2.6 based on SSP1 | ssp126 | mon | ... | r1i1p1f1 | 1 | v20210318 | toa_incoming_shortwave_flux | TOA Incident Shortwave Radiation | W m-2 | days since 1850-01-01 | proleptic_gregorian | True | CMIP6.ScenarioMIP.CSIRO.ACCESS-ESM1-5.ssp126.r... |
| 88 | 2015-01-16 12:00:00 | 2025-12-16 12:00:00 | /home/docs/checkouts/readthedocs.org/user_buil... | ScenarioMIP | standard | 60265.0 | 60265.0 | update of RCP2.6 based on SSP1 | ssp126 | mon | ... | r1i1p1f1 | 1 | v20210318 | toa_outgoing_shortwave_flux | TOA Outgoing Shortwave Radiation | W m-2 | days since 1850-01-01 | proleptic_gregorian | True | CMIP6.ScenarioMIP.CSIRO.ACCESS-ESM1-5.ssp126.r... |
| 89 | 2015-01-16 12:00:00 | 2100-12-16 12:00:00 | /home/docs/checkouts/readthedocs.org/user_buil... | ScenarioMIP | standard | 60265.0 | 60265.0 | update of RCP2.6 based on SSP1 | ssp126 | mon | ... | r1i1p1f1 | 1 | v20210318 | air_temperature | Near-Surface Air Temperature | K | days since 1850-01-01 | proleptic_gregorian | True | CMIP6.ScenarioMIP.CSIRO.ACCESS-ESM1-5.ssp126.r... |
5 rows × 38 columns
Below, the unique combinations of the metadata values that apply to the groupby are shown:
In [7]:
Copied!
data_catalog[["source_id", "variant_label", "variable_id", "experiment_id"]].drop_duplicates()
data_catalog[["source_id", "variant_label", "variable_id", "experiment_id"]].drop_duplicates()
Out[7]:
| source_id | variant_label | variable_id | experiment_id | |
|---|---|---|---|---|
| 85 | ACCESS-ESM1-5 | r2i1p1f1 | areacella | hist-GHG |
| 86 | ACCESS-ESM1-5 | r1i1p1f1 | pr | ssp126 |
| 87 | ACCESS-ESM1-5 | r1i1p1f1 | rsdt | ssp126 |
| 88 | ACCESS-ESM1-5 | r1i1p1f1 | rsut | ssp126 |
| 89 | ACCESS-ESM1-5 | r1i1p1f1 | tas | ssp126 |
| ... | ... | ... | ... | ... |
| 10 | CNRM-ESM2-1 | r1i1p1f2 | o3 | historical |
| 11 | ACCESS-ESM1-5 | r1i1p1f1 | tas | 1pctCO2 |
| 12 | ACCESS-ESM1-5 | r1i1p1f1 | areacella | 1pctCO2 |
| 13 | ACCESS-ESM1-5 | r1i1p1f1 | rlut | abrupt-4xCO2 |
| 14 | ACCESS-ESM1-5 | r1i1p1f1 | rsdt | abrupt-4xCO2 |
89 rows × 4 columns
Executions¶
An execution is a combination of a diagnostic, a provider, and a group of datasets.
The DiagnosticSolver is used to determine which executions are required given a set of requirements
and the currently available datasets.
This doesn't require the use of the REF database.
In [8]:
Copied!
executions_generator = solve_executions(
data_catalog={
SourceDatasetType.CMIP6: data_catalog,
},
diagnostic=diagnostic,
provider=provider,
)
# Convert from a generator to a list to inspect the complete set of executions
executions = list(executions_generator)
prettyprinter.pprint(executions)
executions_generator = solve_executions(
data_catalog={
SourceDatasetType.CMIP6: data_catalog,
},
diagnostic=diagnostic,
provider=provider,
)
# Convert from a generator to a list to inspect the complete set of executions
executions = list(executions_generator)
prettyprinter.pprint(executions)
2026-06-05 05:54:34.407 | DEBUG | climate_ref.solver:_solve_from_data_requirements:320 - No data catalog for source type SourceDatasetType.CMIP7 of example diagnostic global-mean-timeseries
[
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
15 0101-01-16 12:00:00 0250-12-16 12:00:00
17 NaN NaN
path activity_id \
15 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
17 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
branch_method branch_time_in_child branch_time_in_parent \
15 standard 0.0 0.0
17 standard 0.0 0.0
experiment experiment_id frequency ... member_id \
15 abrupt quadrupling of CO2 abrupt-4xCO2 mon ... r1i1p1f1
17 abrupt quadrupling of CO2 abrupt-4xCO2 fx ... r1i1p1f1
vertical_levels version standard_name \
15 1 v20191115 toa_outgoing_shortwave_flux
17 1 v20191115 cell_area
long_name units \
15 TOA Outgoing Shortwave Radiation W m-2
17 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
15 days since 0101-01-01 proleptic_gregorian True
17 NaN NaN True
instance_id
15 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.abrupt-4xCO2.r1...
17 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.abrupt-4xCO2.r1...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'abrupt-4xCO2'), ('source_id', 'ACCESS-ESM1-5'), ('variable_id', 'rsut'), ('variant_label', 'r1i1p1f1')))})
),
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
25 1996-01-16 12:00:00 2014-12-16 12:00:00
49 NaN NaN
path activity_id \
25 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
49 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
branch_method branch_time_in_child branch_time_in_parent \
25 standard 0.0 21915.0
49 standard 0.0 21915.0
experiment experiment_id frequency ... \
25 all-forcing simulation of the recent past historical mon ...
49 all-forcing simulation of the recent past historical fx ...
member_id vertical_levels version standard_name \
25 r1i1p1f1 1 v20191115 toa_outgoing_shortwave_flux
49 r1i1p1f1 1 v20191115 cell_area
long_name units \
25 TOA Outgoing Shortwave Radiation W m-2
49 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
25 days since 1850-01-01 proleptic_gregorian True
49 NaN NaN True
instance_id
25 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.historical.r1i1...
49 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.historical.r1i1...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'historical'), ('source_id', 'ACCESS-ESM1-5'), ('variable_id', 'rsut'), ('variant_label', 'r1i1p1f1')))})
),
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
56 0101-01-16 12:00:00 0250-12-16 12:00:00
58 NaN NaN
path activity_id \
56 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
58 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
branch_method branch_time_in_child branch_time_in_parent \
56 standard 0.0 36524.0
58 standard 0.0 36524.0
experiment experiment_id frequency ... member_id \
56 pre-industrial control piControl mon ... r1i1p1f1
58 pre-industrial control piControl fx ... r1i1p1f1
vertical_levels version standard_name \
56 1 v20210316 toa_outgoing_shortwave_flux
58 1 v20210316 cell_area
long_name units \
56 TOA Outgoing Shortwave Radiation W m-2
58 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
56 days since 0101-01-01 proleptic_gregorian True
58 NaN NaN True
instance_id
56 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.piControl.r1i1p...
58 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.piControl.r1i1p...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'piControl'), ('source_id', 'ACCESS-ESM1-5'), ('variable_id', 'rsut'), ('variant_label', 'r1i1p1f1')))})
),
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
88 2015-01-16 12:00:00 2025-12-16 12:00:00
91 NaN NaN
path activity_id \
88 /home/docs/checkouts/readthedocs.org/user_buil... ScenarioMIP
91 /home/docs/checkouts/readthedocs.org/user_buil... ScenarioMIP
branch_method branch_time_in_child branch_time_in_parent \
88 standard 60265.0 60265.0
91 standard 60265.0 60265.0
experiment experiment_id frequency ... member_id \
88 update of RCP2.6 based on SSP1 ssp126 mon ... r1i1p1f1
91 update of RCP2.6 based on SSP1 ssp126 fx ... r1i1p1f1
vertical_levels version standard_name \
88 1 v20210318 toa_outgoing_shortwave_flux
91 1 v20210318 cell_area
long_name units \
88 TOA Outgoing Shortwave Radiation W m-2
91 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
88 days since 1850-01-01 proleptic_gregorian True
91 NaN NaN True
instance_id
88 CMIP6.ScenarioMIP.CSIRO.ACCESS-ESM1-5.ssp126.r...
91 CMIP6.ScenarioMIP.CSIRO.ACCESS-ESM1-5.ssp126.r...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'ssp126'), ('source_id', 'ACCESS-ESM1-5'), ('variable_id', 'rsut'), ('variant_label', 'r1i1p1f1')))})
),
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
11 0101-01-16 12:00:00 0250-12-16 12:00:00
12 NaN NaN
path activity_id \
11 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
12 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
branch_method branch_time_in_child branch_time_in_parent \
11 standard 0.0 0.0
12 standard 0.0 0.0
experiment experiment_id frequency ... member_id \
11 1 percent per year increase in CO2 1pctCO2 mon ... r1i1p1f1
12 1 percent per year increase in CO2 1pctCO2 fx ... r1i1p1f1
vertical_levels version standard_name \
11 1 v20191115 air_temperature
12 1 v20191115 cell_area
long_name units \
11 Near-Surface Air Temperature K
12 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
11 days since 0101-01-01 proleptic_gregorian True
12 NaN NaN True
instance_id
11 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.1pctCO2.r1i1p1f...
12 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.1pctCO2.r1i1p1f...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', '1pctCO2'), ('source_id', 'ACCESS-ESM1-5'), ('variable_id', 'tas'), ('variant_label', 'r1i1p1f1')))})
),
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
16 0101-01-16 12:00:00 0250-12-16 12:00:00
17 NaN NaN
path activity_id \
16 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
17 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
branch_method branch_time_in_child branch_time_in_parent \
16 standard 0.0 0.0
17 standard 0.0 0.0
experiment experiment_id frequency ... member_id \
16 abrupt quadrupling of CO2 abrupt-4xCO2 mon ... r1i1p1f1
17 abrupt quadrupling of CO2 abrupt-4xCO2 fx ... r1i1p1f1
vertical_levels version standard_name \
16 1 v20191115 air_temperature
17 1 v20191115 cell_area
long_name units \
16 Near-Surface Air Temperature K
17 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
16 days since 0101-01-01 proleptic_gregorian True
17 NaN NaN True
instance_id
16 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.abrupt-4xCO2.r1...
17 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.abrupt-4xCO2.r1...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'abrupt-4xCO2'), ('source_id', 'ACCESS-ESM1-5'), ('variable_id', 'tas'), ('variant_label', 'r1i1p1f1')))})
),
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
1 0168-01-16 12:00:00 0268-12-16 12:00:00
2 NaN NaN
path activity_id \
1 /home/docs/checkouts/readthedocs.org/user_buil... C4MIP CDRMIP
2 /home/docs/checkouts/readthedocs.org/user_buil... C4MIP CDRMIP
branch_method branch_time_in_child branch_time_in_parent \
1 standard 24471.0 24471.0
2 standard 24471.0 24471.0
experiment experiment_id \
1 zero emissions simulation branched from 1% run... esm-1pct-brch-1000PgC
2 zero emissions simulation branched from 1% run... esm-1pct-brch-1000PgC
frequency ... member_id vertical_levels version standard_name \
1 mon ... r1i1p1f1 1 v20191206 air_temperature
2 fx ... r1i1p1f1 1 v20191206 cell_area
long_name units time_units \
1 Near-Surface Air Temperature K days since 0101-01-01
2 Grid-Cell Area for Atmospheric Grid Variables m2 NaN
calendar finalised \
1 proleptic_gregorian True
2 NaN True
instance_id
1 CMIP6.C4MIP CDRMIP.CSIRO.ACCESS-ESM1-5.esm-1pc...
2 CMIP6.C4MIP CDRMIP.CSIRO.ACCESS-ESM1-5.esm-1pc...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'esm-1pct-brch-1000PgC'), ('source_id', 'ACCESS-ESM1-5'), ('variable_id', 'tas'), ('variant_label', 'r1i1p1f1')))})
),
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
27 1850-01-16 12:00:00 2014-12-16 12:00:00
49 NaN NaN
path activity_id \
27 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
49 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
branch_method branch_time_in_child branch_time_in_parent \
27 standard 0.0 21915.0
49 standard 0.0 21915.0
experiment experiment_id frequency ... \
27 all-forcing simulation of the recent past historical mon ...
49 all-forcing simulation of the recent past historical fx ...
member_id vertical_levels version standard_name \
27 r1i1p1f1 1 v20191115 air_temperature
49 r1i1p1f1 1 v20191115 cell_area
long_name units \
27 Near-Surface Air Temperature K
49 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
27 days since 1850-01-01 proleptic_gregorian True
49 NaN NaN True
instance_id
27 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.historical.r1i1...
49 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.historical.r1i1...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'historical'), ('source_id', 'ACCESS-ESM1-5'), ('variable_id', 'tas'), ('variant_label', 'r1i1p1f1')))})
),
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
57 0101-01-16 12:00:00 0250-12-16 12:00:00
58 NaN NaN
path activity_id \
57 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
58 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
branch_method branch_time_in_child branch_time_in_parent \
57 standard 0.0 36524.0
58 standard 0.0 36524.0
experiment experiment_id frequency ... member_id \
57 pre-industrial control piControl mon ... r1i1p1f1
58 pre-industrial control piControl fx ... r1i1p1f1
vertical_levels version standard_name \
57 1 v20210316 air_temperature
58 1 v20210316 cell_area
long_name units \
57 Near-Surface Air Temperature K
58 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
57 days since 0101-01-01 proleptic_gregorian True
58 NaN NaN True
instance_id
57 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.piControl.r1i1p...
58 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.piControl.r1i1p...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'piControl'), ('source_id', 'ACCESS-ESM1-5'), ('variable_id', 'tas'), ('variant_label', 'r1i1p1f1')))})
),
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
89 2015-01-16 12:00:00 2100-12-16 12:00:00
91 NaN NaN
path activity_id \
89 /home/docs/checkouts/readthedocs.org/user_buil... ScenarioMIP
91 /home/docs/checkouts/readthedocs.org/user_buil... ScenarioMIP
branch_method branch_time_in_child branch_time_in_parent \
89 standard 60265.0 60265.0
91 standard 60265.0 60265.0
experiment experiment_id frequency ... member_id \
89 update of RCP2.6 based on SSP1 ssp126 mon ... r1i1p1f1
91 update of RCP2.6 based on SSP1 ssp126 fx ... r1i1p1f1
vertical_levels version standard_name \
89 1 v20210318 air_temperature
91 1 v20210318 cell_area
long_name units \
89 Near-Surface Air Temperature K
91 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
89 days since 1850-01-01 proleptic_gregorian True
91 NaN NaN True
instance_id
89 CMIP6.ScenarioMIP.CSIRO.ACCESS-ESM1-5.ssp126.r...
91 CMIP6.ScenarioMIP.CSIRO.ACCESS-ESM1-5.ssp126.r...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'ssp126'), ('source_id', 'ACCESS-ESM1-5'), ('variable_id', 'tas'), ('variant_label', 'r1i1p1f1')))})
),
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
72 1996-01-15 12:00:00 2014-12-15 12:00:00
79 NaN NaN
path activity_id \
72 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
79 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
branch_method branch_time_in_child branch_time_in_parent \
72 standard 674885.0 219000.0
79 standard 674885.0 219000.0
experiment experiment_id frequency ... \
72 all-forcing simulation of the recent past historical mon ...
79 all-forcing simulation of the recent past historical fx ...
member_id vertical_levels version standard_name \
72 r1i1p1f1 1 v20190308 toa_outgoing_shortwave_flux
79 r1i1p1f1 1 v20190308 cell_area
long_name units \
72 TOA Outgoing Shortwave Radiation W m-2
79 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
72 days since 0001-01-01 noleap True
79 NaN NaN True
instance_id
72 CMIP6.CMIP.NCAR.CESM2.historical.r1i1p1f1.Amon...
79 CMIP6.CMIP.NCAR.CESM2.historical.r1i1p1f1.fx.a...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'historical'), ('source_id', 'CESM2'), ('variable_id', 'rsut'), ('variant_label', 'r1i1p1f1')))})
),
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
5 1979-01-16 12:00:00 2014-12-16 12:00:00
8 NaN NaN
path activity_id \
5 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
8 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
branch_method branch_time_in_child branch_time_in_parent \
5 Spin-up documentation 0.0 1223115.0
8 Spin-up documentation 0.0 1223115.0
experiment experiment_id frequency ... \
5 all-forcing simulation of the recent past historical mon ...
8 all-forcing simulation of the recent past historical fx ...
member_id vertical_levels version standard_name \
5 r1i1p1f1 1 v20190429 air_temperature
8 r1i1p1f1 1 v20190429 cell_area
long_name units time_units \
5 Near-Surface Air Temperature K days since 1850-01-01
8 Grid-Cell Area for Atmospheric Grid Variables m2 NaN
calendar finalised instance_id
5 365_day True CMIP6.CMIP.CCCma.CanESM5.historical.r1i1p1f1.A...
8 NaN True CMIP6.CMIP.CCCma.CanESM5.historical.r1i1p1f1.f...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'historical'), ('source_id', 'CanESM5'), ('variable_id', 'tas'), ('variant_label', 'r1i1p1f1')))})
),
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
59 1979-01-16 00:00:00 2014-12-16 00:00:00
62 NaN NaN
path activity_id \
59 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
62 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
branch_method branch_time_in_child branch_time_in_parent \
59 standard 0.0 0.0
62 standard 0.0 267840.0
experiment experiment_id frequency ... \
59 all-forcing simulation of the recent past historical mon ...
62 pre-industrial control piControl fx ...
member_id vertical_levels version standard_name \
59 r1i1p1f3 1 v20190624 air_temperature
62 r1i1p1f1 1 v20190709 cell_area
long_name units \
59 Near-Surface Air Temperature K
62 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
59 days since 1850-01-01 360_day True
62 NaN NaN True
instance_id
59 CMIP6.CMIP.MOHC.HadGEM3-GC31-LL.historical.r1i...
62 CMIP6.CMIP.MOHC.HadGEM3-GC31-LL.piControl.r1i1...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'historical'), ('source_id', 'HadGEM3-GC31-LL'), ('variable_id', 'tas'), ('variant_label', 'r1i1p1f3')))})
),
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
4 1910-01-16 12:00:00 1914-12-16 12:00:00
4 1890-01-16 12:00:00 1909-12-16 12:00:00
4 1870-01-16 12:00:00 1889-12-16 12:00:00
4 1850-01-16 12:00:00 1869-12-16 12:00:00
64 NaN NaN
path activity_id \
4 /home/docs/checkouts/readthedocs.org/user_buil... C4MIP CDRMIP
4 /home/docs/checkouts/readthedocs.org/user_buil... C4MIP CDRMIP
4 /home/docs/checkouts/readthedocs.org/user_buil... C4MIP CDRMIP
4 /home/docs/checkouts/readthedocs.org/user_buil... C4MIP CDRMIP
64 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
branch_method branch_time_in_child branch_time_in_parent \
4 standard 0.0 0.0
4 standard 0.0 0.0
4 standard 0.0 0.0
4 standard 0.0 0.0
64 standard 0.0 146097.0
experiment experiment_id \
4 emissions driven 1% run esm-1pctCO2
4 emissions driven 1% run esm-1pctCO2
4 emissions driven 1% run esm-1pctCO2
4 emissions driven 1% run esm-1pctCO2
64 pre-industrial control simulation with CO2 con... esm-piControl
frequency ... member_id vertical_levels version standard_name \
4 mon ... r1i1p1f1 1 v20190815 air_temperature
4 mon ... r1i1p1f1 1 v20190815 air_temperature
4 mon ... r1i1p1f1 1 v20190815 air_temperature
4 mon ... r1i1p1f1 1 v20190815 air_temperature
64 fx ... r1i1p1f1 1 v20190815 cell_area
long_name units \
4 Near-Surface Air Temperature K
4 Near-Surface Air Temperature K
4 Near-Surface Air Temperature K
4 Near-Surface Air Temperature K
64 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
4 days since 1850-01-01 proleptic_gregorian True
4 days since 1850-01-01 proleptic_gregorian True
4 days since 1850-01-01 proleptic_gregorian True
4 days since 1850-01-01 proleptic_gregorian True
64 NaN NaN True
instance_id
4 CMIP6.C4MIP CDRMIP.MPI-M.MPI-ESM1-2-LR.esm-1pc...
4 CMIP6.C4MIP CDRMIP.MPI-M.MPI-ESM1-2-LR.esm-1pc...
4 CMIP6.C4MIP CDRMIP.MPI-M.MPI-ESM1-2-LR.esm-1pc...
4 CMIP6.C4MIP CDRMIP.MPI-M.MPI-ESM1-2-LR.esm-1pc...
64 CMIP6.CMIP.MPI-M.MPI-ESM1-2-LR.esm-piControl.r...
[5 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'esm-1pctCO2'), ('source_id', 'MPI-ESM1-2-LR'), ('variable_id', 'tas'), ('variant_label', 'r1i1p1f1')))})
),
climate_ref.solver.DiagnosticExecution(
provider=DiagnosticProvider(name='Example', version='0.14.4'),
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
63 1850-01-16 12:00:00 1869-12-16 12:00:00
63 1890-01-16 12:00:00 1909-12-16 12:00:00
63 1870-01-16 12:00:00 1889-12-16 12:00:00
63 1910-01-16 12:00:00 1915-12-16 12:00:00
64 NaN NaN
path activity_id \
63 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
63 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
63 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
63 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
64 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
branch_method branch_time_in_child branch_time_in_parent \
63 standard 0.0 146097.0
63 standard 0.0 146097.0
63 standard 0.0 146097.0
63 standard 0.0 146097.0
64 standard 0.0 146097.0
experiment experiment_id \
63 pre-industrial control simulation with CO2 con... esm-piControl
63 pre-industrial control simulation with CO2 con... esm-piControl
63 pre-industrial control simulation with CO2 con... esm-piControl
63 pre-industrial control simulation with CO2 con... esm-piControl
64 pre-industrial control simulation with CO2 con... esm-piControl
frequency ... member_id vertical_levels version standard_name \
63 mon ... r1i1p1f1 1 v20190815 air_temperature
63 mon ... r1i1p1f1 1 v20190815 air_temperature
63 mon ... r1i1p1f1 1 v20190815 air_temperature
63 mon ... r1i1p1f1 1 v20190815 air_temperature
64 fx ... r1i1p1f1 1 v20190815 cell_area
long_name units \
63 Near-Surface Air Temperature K
63 Near-Surface Air Temperature K
63 Near-Surface Air Temperature K
63 Near-Surface Air Temperature K
64 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
63 days since 1850-01-01 proleptic_gregorian True
63 days since 1850-01-01 proleptic_gregorian True
63 days since 1850-01-01 proleptic_gregorian True
63 days since 1850-01-01 proleptic_gregorian True
64 NaN NaN True
instance_id
63 CMIP6.CMIP.MPI-M.MPI-ESM1-2-LR.esm-piControl.r...
63 CMIP6.CMIP.MPI-M.MPI-ESM1-2-LR.esm-piControl.r...
63 CMIP6.CMIP.MPI-M.MPI-ESM1-2-LR.esm-piControl.r...
63 CMIP6.CMIP.MPI-M.MPI-ESM1-2-LR.esm-piControl.r...
64 CMIP6.CMIP.MPI-M.MPI-ESM1-2-LR.esm-piControl.r...
[5 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'esm-piControl'), ('source_id', 'MPI-ESM1-2-LR'), ('variable_id', 'tas'), ('variant_label', 'r1i1p1f1')))})
)
]
We get multiple proposed executions.
In [9]:
Copied!
pd.concat(execution.datasets["cmip6"] for execution in executions)[
["experiment_id", "variable_id"]
].drop_duplicates()
pd.concat(execution.datasets["cmip6"] for execution in executions)[
["experiment_id", "variable_id"]
].drop_duplicates()
Out[9]:
| experiment_id | variable_id | |
|---|---|---|
| 15 | abrupt-4xCO2 | rsut |
| 17 | abrupt-4xCO2 | areacella |
| 25 | historical | rsut |
| 49 | historical | areacella |
| 56 | piControl | rsut |
| 58 | piControl | areacella |
| 88 | ssp126 | rsut |
| 91 | ssp126 | areacella |
| 11 | 1pctCO2 | tas |
| 12 | 1pctCO2 | areacella |
| 16 | abrupt-4xCO2 | tas |
| 1 | esm-1pct-brch-1000PgC | tas |
| 2 | esm-1pct-brch-1000PgC | areacella |
| 27 | historical | tas |
| 57 | piControl | tas |
| 89 | ssp126 | tas |
| 4 | esm-1pctCO2 | tas |
| 64 | esm-piControl | areacella |
| 63 | esm-piControl | tas |
Each execution contains a single unique dataset because of the groupby definition. The data catalog for the diagnostic execution may contain more than one row as a dataset may contain multiple files.
In [10]:
Copied!
executions[0].datasets["cmip6"].instance_id.unique().tolist()
executions[0].datasets["cmip6"].instance_id.unique().tolist()
Out[10]:
['CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.abrupt-4xCO2.r1i1p1f1.Amon.rsut.gn.v20191115', 'CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.abrupt-4xCO2.r1i1p1f1.fx.areacella.gn.v20191115']
In [11]:
Copied!
executions[0].datasets["cmip6"]
executions[0].datasets["cmip6"]
Out[11]:
| start_time | end_time | path | activity_id | branch_method | branch_time_in_child | branch_time_in_parent | experiment | experiment_id | frequency | ... | member_id | vertical_levels | version | standard_name | long_name | units | time_units | calendar | finalised | instance_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 15 | 0101-01-16 12:00:00 | 0250-12-16 12:00:00 | /home/docs/checkouts/readthedocs.org/user_buil... | CMIP | standard | 0.0 | 0.0 | abrupt quadrupling of CO2 | abrupt-4xCO2 | mon | ... | r1i1p1f1 | 1 | v20191115 | toa_outgoing_shortwave_flux | TOA Outgoing Shortwave Radiation | W m-2 | days since 0101-01-01 | proleptic_gregorian | True | CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.abrupt-4xCO2.r1... |
| 17 | NaN | NaN | /home/docs/checkouts/readthedocs.org/user_buil... | CMIP | standard | 0.0 | 0.0 | abrupt quadrupling of CO2 | abrupt-4xCO2 | fx | ... | r1i1p1f1 | 1 | v20191115 | cell_area | Grid-Cell Area for Atmospheric Grid Variables | m2 | NaN | NaN | True | CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.abrupt-4xCO2.r1... |
2 rows × 38 columns
Diagnostic Definitions¶
Each diagnostic execution requires a ExecutionDefinition object.
This object contains the information about where data should be stored
and which datasets should be used for the diagnostic calculation.
In [12]:
Copied!
output_directory = Path("../out")
definition = executions[0].build_execution_definition(output_directory)
prettyprinter.pprint(definition)
output_directory = Path("../out")
definition = executions[0].build_execution_definition(output_directory)
prettyprinter.pprint(definition)
climate_ref_core.diagnostics.ExecutionDefinition(
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
key='cmip6_abrupt-4xCO2_ACCESS-ESM1-5_rsut_r1i1p1f1',
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
15 0101-01-16 12:00:00 0250-12-16 12:00:00
17 NaN NaN
path activity_id \
15 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
17 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
branch_method branch_time_in_child branch_time_in_parent \
15 standard 0.0 0.0
17 standard 0.0 0.0
experiment experiment_id frequency ... member_id \
15 abrupt quadrupling of CO2 abrupt-4xCO2 mon ... r1i1p1f1
17 abrupt quadrupling of CO2 abrupt-4xCO2 fx ... r1i1p1f1
vertical_levels version standard_name \
15 1 v20191115 toa_outgoing_shortwave_flux
17 1 v20191115 cell_area
long_name units \
15 TOA Outgoing Shortwave Radiation W m-2
17 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
15 days since 0101-01-01 proleptic_gregorian True
17 NaN NaN True
instance_id
15 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.abrupt-4xCO2.r1...
17 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.abrupt-4xCO2.r1...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'abrupt-4xCO2'), ('source_id', 'ACCESS-ESM1-5'), ('variable_id', 'rsut'), ('variant_label', 'r1i1p1f1')))}),
output_directory=pathlib.PosixPath(
'/home/docs/checkouts/readthedocs.org/user_builds/climate-ref/'
'checkouts/out/example/global-mean-timeseries/_pending'
),
_root_directory=pathlib.PosixPath(
'/home/docs/checkouts/readthedocs.org/user_builds/climate-ref/'
'checkouts/out'
)
)
Running directly locally¶
A diagnostic can be run directly if you want to run a calculation synchronously without any additional infrastructure.
This will not perform and validation/verification of the output executions.
In [13]:
Copied!
definition.output_directory.mkdir(parents=True, exist_ok=True)
direct_result = diagnostic.run(definition=definition)
assert direct_result.successful
prettyprinter.pprint(direct_result)
definition.output_directory.mkdir(parents=True, exist_ok=True)
direct_result = diagnostic.run(definition=definition)
assert direct_result.successful
prettyprinter.pprint(direct_result)
/home/docs/checkouts/readthedocs.org/user_builds/climate-ref/checkouts/latest/packages/climate-ref-example/src/climate_ref_example/example.py:41: FutureWarning: In a future version of xarray the default value for compat will change from compat='no_conflicts' to compat='override'. This is likely to lead to different results when combining overlapping variables with the same name. To opt in to new defaults and get rid of these warnings now use `set_options(use_new_combine_kwarg_defaults=True) or set compat explicitly. xr_ds = xr.open_mfdataset(input_files, combine="by_coords", chunks=None, decode_times=time_coder) /home/docs/checkouts/readthedocs.org/user_builds/climate-ref/checkouts/latest/packages/climate-ref-example/src/climate_ref_example/example.py:41: FutureWarning: In a future version of xarray the default value for compat will change from compat='no_conflicts' to compat='override'. This is likely to lead to different results when combining overlapping variables with the same name. To opt in to new defaults and get rid of these warnings now use `set_options(use_new_combine_kwarg_defaults=True) or set compat explicitly. xr_ds = xr.open_mfdataset(input_files, combine="by_coords", chunks=None, decode_times=time_coder)
climate_ref_core.diagnostics.ExecutionResult(
definition=climate_ref_core.diagnostics.ExecutionDefinition(
diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'),
key='cmip6_abrupt-4xCO2_ACCESS-ESM1-5_rsut_r1i1p1f1',
datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \
15 0101-01-16 12:00:00 0250-12-16 12:00:00
17 NaN NaN
path activity_id \
15 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
17 /home/docs/checkouts/readthedocs.org/user_buil... CMIP
branch_method branch_time_in_child branch_time_in_parent \
15 standard 0.0 0.0
17 standard 0.0 0.0
experiment experiment_id frequency ... member_id \
15 abrupt quadrupling of CO2 abrupt-4xCO2 mon ... r1i1p1f1
17 abrupt quadrupling of CO2 abrupt-4xCO2 fx ... r1i1p1f1
vertical_levels version standard_name \
15 1 v20191115 toa_outgoing_shortwave_flux
17 1 v20191115 cell_area
long_name units \
15 TOA Outgoing Shortwave Radiation W m-2
17 Grid-Cell Area for Atmospheric Grid Variables m2
time_units calendar finalised \
15 days since 0101-01-01 proleptic_gregorian True
17 NaN NaN True
instance_id
15 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.abrupt-4xCO2.r1...
17 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.abrupt-4xCO2.r1...
[2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'abrupt-4xCO2'), ('source_id', 'ACCESS-ESM1-5'), ('variable_id', 'rsut'), ('variant_label', 'r1i1p1f1')))}),
output_directory=pathlib.PosixPath(
'/home/docs/checkouts/readthedocs.org/user_builds/climate-ref/'
'checkouts/out/example/global-mean-timeseries/_pending'
),
_root_directory=pathlib.PosixPath(
'/home/docs/checkouts/readthedocs.org/user_builds/climate-ref/'
'checkouts/out'
)
),
output_bundle_filename=pathlib.PosixPath('output.json'),
metric_bundle_filename=pathlib.PosixPath('diagnostic.json'),
successful=True,
series_filename=pathlib.PosixPath('series.json')
)
Diagnostic calculations¶
Diagnostic calculations are typically run using an Executor which provides an abstraction to enable diagnostics to be run in multiple different ways. These executors can run diagnostics locally, on a cluster, or on a remote service
The default executor is the [LocalExecutor][climate_ref.executor.LocalExecutor] This executor runs executions in parallel using a process pool. Another option is the [SynchronousExecutor][climate_ref.executor.SynchronousExecutor]. This executor runs the execution in the main process which can be useful for debugging purposes.
The executor can be specified in the configuration, or
using the REF_EXECUTOR environment variable which takes precedence.
The [LocalExecutor][climate_ref.executor.LocalExecutor] is the default executor,
if no other configuration is provided.
In [14]:
Copied!
# In this example a [SynchronousExecutor][climate_ref.executor.SynchronousExecutor] is used to run
# the diagnostic.
# This executor runs the execution in the main process which can be useful for debugging purposes.
# In this example a [SynchronousExecutor][climate_ref.executor.SynchronousExecutor] is used to run
# the diagnostic.
# This executor runs the execution in the main process which can be useful for debugging purposes.
In [15]:
Copied!
executor = SynchronousExecutor(config=config, database=db)
diagnostic = provider.get("global-mean-timeseries")
executor.run(definition)
executor = SynchronousExecutor(config=config, database=db)
diagnostic = provider.get("global-mean-timeseries")
executor.run(definition)
2026-06-05 05:54:39.414 | INFO | climate_ref_core.executor:execute_locally:54 - Executing 'example/global-mean-timeseries/cmip6_abrupt-4xCO2_ACCESS-ESM1-5_rsut_r1i1p1f1'
2026-06-05 05:54:39.415 | WARNING | climate_ref_core.executor:execute_locally:58 - Output directory /home/docs/checkouts/readthedocs.org/user_builds/climate-ref/checkouts/out/example/global-mean-timeseries/_pending already exists. Removing the existing directory.
2026-06-05 05:54:39.426 | INFO | climate_ref_core.logging:redirect_logs:200 - Running definition ExecutionDefinition( diagnostic=GlobalMeanTimeseries(name='Global Mean Timeseries'), key='cmip6_abrupt-4xCO2_ACCESS-ESM1-5_rsut_r1i1p1f1', datasets=ExecutionDatasetCollection({<SourceDatasetType.CMIP6: 'cmip6'>: DatasetCollection(datasets= start_time end_time \ 15 0101-01-16 12:00:00 0250-12-16 12:00:00 17 NaN NaN path activity_id \ 15 /home/docs/checkouts/readthedocs.org/user_buil... CMIP 17 /home/docs/checkouts/readthedocs.org/user_buil... CMIP branch_method branch_time_in_child branch_time_in_parent \ 15 standard 0.0 0.0 17 standard 0.0 0.0 experiment experiment_id frequency ... member_id \ 15 abrupt quadrupling of CO2 abrupt-4xCO2 mon ... r1i1p1f1 17 abrupt quadrupling of CO2 abrupt-4xCO2 fx ... r1i1p1f1 vertical_levels version standard_name \ 15 1 v20191115 toa_outgoing_shortwave_flux 17 1 v20191115 cell_area long_name units \ 15 TOA Outgoing Shortwave Radiation W m-2 17 Grid-Cell Area for Atmospheric Grid Variables m2 time_units calendar finalised \ 15 days since 0101-01-01 proleptic_gregorian True 17 NaN NaN True instance_id 15 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.abrupt-4xCO2.r1... 17 CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.abrupt-4xCO2.r1... [2 rows x 38 columns], slug_column='instance_id', selector=(('experiment_id', 'abrupt-4xCO2'), ('source_id', 'ACCESS-ESM1-5'), ('variable_id', 'rsut'), ('variant_label', 'r1i1p1f1')))}), output_directory=PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/climate-ref/checkouts/out/example/global-mean-timeseries/_pending'), _root_directory=PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/climate-ref/checkouts/out') )
/home/docs/checkouts/readthedocs.org/user_builds/climate-ref/checkouts/latest/packages/climate-ref-example/src/climate_ref_example/example.py:41: FutureWarning: In a future version of xarray the default value for compat will change from compat='no_conflicts' to compat='override'. This is likely to lead to different results when combining overlapping variables with the same name. To opt in to new defaults and get rid of these warnings now use `set_options(use_new_combine_kwarg_defaults=True) or set compat explicitly. xr_ds = xr.open_mfdataset(input_files, combine="by_coords", chunks=None, decode_times=time_coder) /home/docs/checkouts/readthedocs.org/user_builds/climate-ref/checkouts/latest/packages/climate-ref-example/src/climate_ref_example/example.py:41: FutureWarning: In a future version of xarray the default value for compat will change from compat='no_conflicts' to compat='override'. This is likely to lead to different results when combining overlapping variables with the same name. To opt in to new defaults and get rid of these warnings now use `set_options(use_new_combine_kwarg_defaults=True) or set compat explicitly. xr_ds = xr.open_mfdataset(input_files, combine="by_coords", chunks=None, decode_times=time_coder)
2026-06-05 05:54:43.925 | INFO | climate_ref_core.logging:redirect_logs:207 - Diagnostic execution complete. Results available in example/global-mean-timeseries/_pending
In [16]:
Copied!
output_file = definition.to_output_path("output.json")
with open(output_file) as fh:
# Load the output bundle and pretty print
loaded_result = json.loads(fh.read())
print(json.dumps(loaded_result, indent=2))
output_file = definition.to_output_path("output.json")
with open(output_file) as fh:
# Load the output bundle and pretty print
loaded_result = json.loads(fh.read())
print(json.dumps(loaded_result, indent=2))
{
"index": "index.html",
"provenance": {
"environment": {},
"modeldata": [],
"obsdata": {},
"log": "cmec_output.log"
},
"data": {},
"plots": {},
"html": {},
"metrics": null,
"diagnostics": {}
}
In [ ]:
Copied!