Skip to content

Add CountryProcessor#592

Open
dc-almeida wants to merge 18 commits into
IAMconsortium:mainfrom
dc-almeida:feature/country-processor
Open

Add CountryProcessor#592
dc-almeida wants to merge 18 commits into
IAMconsortium:mainfrom
dc-almeida:feature/country-processor

Conversation

@dc-almeida

@dc-almeida dc-almeida commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Closes #587. Adds a CountryProcessor to aggregate country-level data into regional levels (R5, R9, R10).
Currently implemented with importing common-definitions as a dependency.

@dc-almeida dc-almeida self-assigned this Jun 15, 2026
@dc-almeida dc-almeida added the enhancement New feature or request label Jun 15, 2026

@danielhuppmann danielhuppmann left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks nice, a first round of suggestions and comments below.

Comment thread docs/api/countryprocessor.rst Outdated
Comment thread nomenclature/processor/countries.py Outdated
Comment thread nomenclature/processor/countries.py Outdated
Comment thread nomenclature/processor/countries.py Outdated
Comment thread tests/test_country_aggregation.py Outdated
@danielhuppmann

Copy link
Copy Markdown
Member

Is this ready for review yet?

@dc-almeida

Copy link
Copy Markdown
Contributor Author

Yes, I'll open it for final review. I left it as a draft because I was going to look into the minimum countries for aggregation question, but meanwhile looked at other tasks so best to close this one and tackle that a bit later.

@dc-almeida
dc-almeida marked this pull request as ready for review June 30, 2026 07:24
@dc-almeida
dc-almeida requested review from Copilot, danielhuppmann and phackstock and removed request for Copilot June 30, 2026 07:24

@danielhuppmann danielhuppmann left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments inline, and one general question about the architecture - wouldn't it make sense to have a constructor that builds a RegionProcessor class and then applies that class?

Maybe add a RegionProcessor.from_codelist(hierarchy) constructor?

Comment thread docs/api/countryprocessor.rst Outdated
Comment thread docs/api/countryprocessor.rst Outdated
Comment thread nomenclature/processor/countries.py Outdated
Comment thread nomenclature/processor/countries.py Outdated
Comment thread nomenclature/processor/countries.py Outdated
Comment thread nomenclature/processor/countries.py Outdated
@phackstock

Copy link
Copy Markdown
Contributor

A few comments inline, and one general question about the architecture - wouldn't it make sense to have a constructor that builds a RegionProcessor class and then applies that class?

Maybe add a RegionProcessor.from_codelist(hierarchy) constructor?

I'd agree with that approach. The current implementation produces quite a lot of code duplication which increases the chance of failure and makes future maintenance harder.
I'd suggest to either go with something along the lines of RegionProcessor.create_country_processor_from_codelist or making a CountryProcessor class that inherits from RegionProcessor to get all of the common functionality. As part of this, I'd suggest to directly create RegionAggregationMappings for the models that are specified in the config.

@phackstock phackstock left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In terms for functionality, I'd say this PR is where we want it to be with the exception of checking a minimum set of countries.
Implementation wise, I want to echo what @danielhuppmann said earlier in the discussion in this PR, CountryProcessor should either be implemented as a subclass of RegionProcessor or generated through a dedicated constructor as part of the RegionProcessor class. The RegionCodeList and model information should be used to generate model mappings on-the-fly that are then applied to incoming data.

Comment thread nomenclature/processor/countries.py Outdated
Comment thread nomenclature/processor/countries.py Outdated
Comment thread nomenclature/core.py Outdated
Comment thread nomenclature/core.py
@dc-almeida

dc-almeida commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Applied review suggestions and:

  • added a RegionProcessor.from_country_codelist factory method that creates a processor/aggregator for a list of hierarchies with country attributes
  • added a create_country_processor function as a specific use case of the method above, for R5/R9/R10 regions
  • initially added by_hierarchy RegionCodeList method but prioritised use of already existing filter CodeList method
  • updated docs to have the country processor guide in RegionProcessor

@danielhuppmann danielhuppmann left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, tests and implementation look good, but as detailed below, I would go for the proper subclass-strategy instead of a utility function.

Comment thread docs/api/regionprocessor.rst Outdated
Comment thread docs/api/regionprocessor.rst Outdated
Comment thread nomenclature/processor/region.py Outdated
Comment thread nomenclature/processor/country.py Outdated
Comment thread nomenclature/processor/region.py Outdated

@danielhuppmann danielhuppmann left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two minor comments, good from my side to merge after a review by @phackstock

Comment thread tests/test_country_aggregation.py Outdated
Comment thread tests/test_country_aggregation.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add CountryProcessor

3 participants