Skip to content

climate_ref.provider_registry #

Registry of the currently active providers in the REF

This module provides a registry for the currently active providers. Often, we can't directly import a provider and it's diagnostics as each provider maintains its own virtual environment to avoid dependency conflicts.

For remote providers, a proxy is used to access the metadata associated with the diagnostics. These diagnostics cannot be run locally, but can be executed using other executors.

ProviderRegistry #

Registry for the currently active providers

In some cases we can't directly import a provider and it's diagnostics, in this case we need to proxy the diagnostics.

Source code in packages/climate-ref/src/climate_ref/provider_registry.py 
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
@frozen
class ProviderRegistry:
    """
    Registry for the currently active providers

    In some cases we can't directly import a provider and it's diagnostics,
    in this case we need to proxy the diagnostics.
    """

    providers: list[DiagnosticProvider] = field(factory=list)

    def get(self, slug: str) -> DiagnosticProvider:
        """
        Retrieve a provider by name

        Parameters
        ----------
        slug
            Slug of the provider of interest

        Raises
        ------
        KeyError
            A provider with the matching slug has not been registered

        Returns
        -------
            The requested provider
        """
        for p in self.providers:
            if p.slug == slug:
                return p

        raise KeyError(f"No provider with slug matching: {slug}")

    def get_metric(self, provider_slug: str, diagnostic_slug: str) -> "Diagnostic":
        """
        Retrieve a diagnostic by name

        This is a convenience method to retrieve a diagnostic from a provider

        Parameters
        ----------
        provider_slug :
            Slug of the provider of interest
        diagnostic_slug
            Slug of the diagnostic of interest

        Raises
        ------
        KeyError
            If the provider/diagnostic with the given slugs is not found.

        Returns
        -------
            The requested diagnostic.
        """
        return self.get(provider_slug).get(diagnostic_slug)

    @staticmethod
    def build_from_config(
        config: Config,
        db: Database,
        *,
        configure: bool = True,
        register: bool = True,
    ) -> "ProviderRegistry":
        """
        Create a ProviderRegistry instance using information from the database.

        Parameters
        ----------
        config
            Configuration object.
        db
            Database instance.
        configure
            Whether to call each provider's ``configure`` hook.
            Metadata-only commands can disable this to avoid setup side effects such as conda bootstrap.
        register
            Whether to upsert provider and diagnostic rows into the database.

            Registration is the only step here that writes to the database.
            Read-only consumers (e.g. the API serving a database mounted read-only)
            must set this to ``False``; the provider and diagnostic rows they need
            are written by the CLI (``ref providers setup``) and the workers.

        Returns
        -------
        :
            A new ProviderRegistry instance.
        """
        providers = []
        for provider_info in config.diagnostic_providers:
            provider = import_provider(provider_info.provider)
            if configure:
                provider.configure(config)
            providers.append(provider)

        if register:
            with db.session.begin():
                for provider in providers:
                    _register_provider(db, provider)

        return ProviderRegistry(providers=providers)

build_from_config(config, db, *, configure=True, register=True) staticmethod #

Create a ProviderRegistry instance using information from the database.

Parameters:

Name Type Description Default
config Config

Configuration object.

 required
db Database

Database instance.

 required
configure bool

Whether to call each provider's configure hook. Metadata-only commands can disable this to avoid setup side effects such as conda bootstrap.

 True
register bool

Whether to upsert provider and diagnostic rows into the database.

Registration is the only step here that writes to the database. Read-only consumers (e.g. the API serving a database mounted read-only) must set this to False; the provider and diagnostic rows they need are written by the CLI (ref providers setup) and the workers.

 True

Returns:

Type Description
ProviderRegistry

A new ProviderRegistry instance.
Source code in packages/climate-ref/src/climate_ref/provider_registry.py 
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
@staticmethod
def build_from_config(
    config: Config,
    db: Database,
    *,
    configure: bool = True,
    register: bool = True,
) -> "ProviderRegistry":
    """
    Create a ProviderRegistry instance using information from the database.

    Parameters
    ----------
    config
        Configuration object.
    db
        Database instance.
    configure
        Whether to call each provider's ``configure`` hook.
        Metadata-only commands can disable this to avoid setup side effects such as conda bootstrap.
    register
        Whether to upsert provider and diagnostic rows into the database.

        Registration is the only step here that writes to the database.
        Read-only consumers (e.g. the API serving a database mounted read-only)
        must set this to ``False``; the provider and diagnostic rows they need
        are written by the CLI (``ref providers setup``) and the workers.

    Returns
    -------
    :
        A new ProviderRegistry instance.
    """
    providers = []
    for provider_info in config.diagnostic_providers:
        provider = import_provider(provider_info.provider)
        if configure:
            provider.configure(config)
        providers.append(provider)

    if register:
        with db.session.begin():
            for provider in providers:
                _register_provider(db, provider)

    return ProviderRegistry(providers=providers)

get(slug) #

Retrieve a provider by name

Parameters:

Name Type Description Default
slug str

Slug of the provider of interest

 required

Raises:

Type Description
KeyError

A provider with the matching slug has not been registered

Returns:

Type Description
The requested provider
Source code in packages/climate-ref/src/climate_ref/provider_registry.py 
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
def get(self, slug: str) -> DiagnosticProvider:
    """
    Retrieve a provider by name

    Parameters
    ----------
    slug
        Slug of the provider of interest

    Raises
    ------
    KeyError
        A provider with the matching slug has not been registered

    Returns
    -------
        The requested provider
    """
    for p in self.providers:
        if p.slug == slug:
            return p

    raise KeyError(f"No provider with slug matching: {slug}")

get_metric(provider_slug, diagnostic_slug) #

Retrieve a diagnostic by name

This is a convenience method to retrieve a diagnostic from a provider

Parameters:

Name Type Description Default
provider_slug str

Slug of the provider of interest

 required
diagnostic_slug str

Slug of the diagnostic of interest

 required

Raises:

Type Description
KeyError

If the provider/diagnostic with the given slugs is not found.

Returns:

Type Description
The requested diagnostic.
Source code in packages/climate-ref/src/climate_ref/provider_registry.py 
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
def get_metric(self, provider_slug: str, diagnostic_slug: str) -> "Diagnostic":
    """
    Retrieve a diagnostic by name

    This is a convenience method to retrieve a diagnostic from a provider

    Parameters
    ----------
    provider_slug :
        Slug of the provider of interest
    diagnostic_slug
        Slug of the diagnostic of interest

    Raises
    ------
    KeyError
        If the provider/diagnostic with the given slugs is not found.

    Returns
    -------
        The requested diagnostic.
    """
    return self.get(provider_slug).get(diagnostic_slug)