feat: add export for REMIND/mrindustry by tobiasdiez · Pull Request #160 · pik-piam/remind-mfa

tobiasdiez · 2026-07-24T08:20:24Z

Add an option to export data that should then be used in mrindustry.

Each output variable is written as a seperate csv file under data/<material>/output/export/remind_input.

A few ponits for consideration:

Proposal for a better naming than "remind_input"?
Is the naming of the variables okay?
And most importantly, is the correct data exported?

for more information, see https://pre-commit.ci

tobiasdiez · 2026-07-24T08:21:36Z

@leonieschweiger @bennet21 could I please get your quick first reactions, especially concerning the points mentioned above in the PR description? Thanks!

JakobBD · 2026-07-24T08:28:29Z

+
+        def steel_scrap(mfa: fd.MFASystem) -> fd.FlodymArray:
+            """Total scrap available after trade and before the model diverts any surplus to excess scrap."""
+            return mfa.stocks["in_use"].outflow.sum_to(("t", "r", "g"))


Should be after collection.

Right, fixed.

JakobBD · 2026-07-24T08:29:23Z

Looks great! See my one comment.

folder could be named mrindustry_input, but remind_input is also fine.

bennet21 · 2026-07-24T09:27:31Z

Thanks @tobiasdiez. For cement production, I'd say you should include losses as well (prod_cement => sysenv). Is there a specific reason you call it raw production?
Another cement-related input provided to REMIND via mrindustry is clinker-to-cement ratio. That could be overwritten by the one from mrmfa and is given in mfa.parameters["clinker_ratio"]. Feel free to add that :)

for more information, see https://pre-commit.ci

tobiasdiez · 2026-07-24T10:32:40Z

For cement production, I'd say you should include losses as well (prod_cement => sysenv).

Okay, done!

Is there a specific reason you call it raw production?

No not really, removed the raw prefix.

Another cement-related input provided to REMIND via mrindustry is clinker-to-cement ratio. That could be overwritten by the one from mrmfa and is given in mfa.parameters["clinker_ratio"]. Feel free to add that :)

Added!

leonieschweiger

Looks great, thanks!
With regards to the exported variables: In REMIND, we only model the production of high value chemicals (HVC), the precursor of plastic production. This is why we need the HVC input into plastic production here (see my suggestions) - sorry for not specifying this earlier!

leonieschweiger · 2026-07-24T13:11:06Z

+    def get_remind_input_variables(self) -> list[RemindInputVariable]:
+        def plastics_production(mfa: fd.MFASystem) -> fd.FlodymArray:
+            """Plastics output after polymerization losses and before trade"""
+            return mfa.flows["polymerization => primary_market"].sum_to(("t", "r"))


Suggested change

def get_remind_input_variables(self) -> list[RemindInputVariable]:

def plastics_production(mfa: fd.MFASystem) -> fd.FlodymArray:

"""Plastics output after polymerization losses and before trade"""

return mfa.flows["polymerization => primary_market"].sum_to(("t", "r"))

def get_remind_input_variables(self) -> list[RemindInputVariable]:

def hvc_input(mfa: fd.MFASystem) -> fd.FlodymArray:

"""HVC input into plastics production"""

return mfa.flows["HVC_input => polymerization"].sum_to(("t", "r"))

leonieschweiger · 2026-07-24T13:11:44Z

+        return [
+            RemindInputVariable(
+                name="plastics_production",
+                calculation_function=plastics_production,
+                unit="t/yr",
+            ),


Suggested change

return [

RemindInputVariable(

name="plastics_production",

calculation_function=plastics_production,

unit="t/yr",

),

return [

RemindInputVariable(

name="HVC_input",

calculation_function=hvc_input,

unit="t/yr",

),

Co-authored-by: Leonie Schweiger <leonie.schweiger@pik-potsdam.de>

tobiasdiez · 2026-07-24T19:17:31Z

folder could be named mrindustry_input, but remind_input is also fine.

Thanks for the suggestion. I now went with the even simpler mrindustry (that it is the input for mrindustry should be clear from the context). What do you think?

tobiasdiez · 2026-07-24T19:17:55Z

Looks great, thanks! With regards to the exported variables: In REMIND, we only model the production of high value chemicals (HVC), the precursor of plastic production. This is why we need the HVC input into plastic production here (see my suggestions) - sorry for not specifying this earlier!

Thanks Leonie! I've now applied your suggestions.

tobiasdiez · 2026-07-24T19:21:17Z

Thanks a lot! This is now officially ready for review.

JakobBD · 2026-07-27T06:04:40Z

-                calculation_function=lambda mfa: mfa.stocks["in_use"].outflow.sum_to(
-                    ("t", "r", "g")
-                ),
+                calculation_function=SteelDataExporter._steel_scrap,


I am afraid this change is not intended, though it would be nice.

PRISMA's definition is pre-collection, what we need for REMIND is post-collection.
You could call this third function _eol or _scrap_pre_collection (and the other one _scrap_post_collection) or so.

Additional info:
In the long run, I would like to have the IAMC export to include both (and be comprehensive in general), possibly with the CIRCOMOD nomenclature or our own.
The piamInterfaces package can then map these onto project-specific nomenclatures.

I'm not sure if I follow completely. At the moment, this is only a refactoring to extract and reuse this code. It shouldn't have changed anything in the IAMC export. Do you mean that because in the future these two will be different, it doesn't make sense now to use the same function? Or do you mean that I need to export something else for mrindustry (and if so, what)?

The confusion was my bad, this comment was misleading and I didn't check your fix.

IAMC export and mrindustry export need two different quantities for steel scrap. The one for IAMC export needs to remain as is.

However, what I meant with the comment linked above is that the scrap quantity for mrindustry should be something else.

I'll just post the proposed end result here for simplicity.

For IAMC:

def _eol_scrap_potential(mfa: fd.MFASystem) -> fd.FlodymArray: """End-of-life scrap available before collection and trade""" return mfa.stocks["in_use"].outflow.sum_to(("t", "r", "g"))

For mrindustry:

def _total_available_scrap(mfa: fd.MFASystem) -> fd.FlodymArray: """Home and new scrap, plus old scrap after collection and trade, but before the model diverts any surplus to excess scrap""" return mfa.flows["forming => scrap_market"] + mfa.flows["fabrication => scrap_market"] + mfa.flows["recycling => scrap_market"]

JakobBD

Great stuff!

Apart from the one comment, good to merge.

NB:
I originally would have hard-coded some functionality into this output function, where the files are directly put into the sources folder on the cluster and a new version sub-folder is created.
But I now too prefer to move this step to a stand-alone script (as you probably intended). It is cleaner that way, and much less dangerous.

bennet21

Great stuff!

tobiasdiez · 2026-07-27T14:15:20Z

NB: I originally would have hard-coded some functionality into this output function, where the files are directly put into the sources folder on the cluster and a new version sub-folder is created. But I now too prefer to move this step to a stand-alone script (as you probably intended). It is cleaner that way, and much less dangerous.

When I started working on this, it was still unclear how to integrate it with mrindustry, and so I went with the most straightforward implementation. But now that Falk confirmed that a separate orchestration script is the way to go, I agree that renaming & moving it to the source folder on the cluster is best done there.

JakobBD · 2026-07-28T08:14:30Z

+        """Cement output after production losses and before trade"""
+        return (
+            mfa.flows["prod_cement => market_cement"] + mfa.flows["prod_cement => sysenv"]
+        ).sum_to(("t", "r"))


The documentation and function seem inconsistent. If it's after losses, why add losses?
Which one should it be @bennet21?

When #164 is merged, this should be

Suggested change

"""Cement output after production losses and before trade"""

return (

mfa.flows["prod_cement => market_cement"] + mfa.flows["prod_cement => sysenv"]

).sum_to(("t", "r"))

"""Cement output before trade and construction losses"""

return mfa.flows["prod_cement => market_cement"].sum_to(("t", "r"))

Co-authored-by: Jakob Duerrwaechter <jakob.duerrwaechter@pik-potsdam.de>

tobiasdiez and others added 2 commits July 24, 2026 10:14

feat: add export for REMIND

642b8ae

[pre-commit.ci] auto fixes from pre-commit.com hooks

6ff33ed

for more information, see https://pre-commit.ci

JakobBD reviewed Jul 24, 2026

View reviewed changes

correct comment for steel_scrap

6f9ebab

tobiasdiez and others added 3 commits July 24, 2026 12:05

Merge remote-tracking branch 'origin/main' into export_to_remind

dfa84e3

update cement export, reduce code duplication in steel and plastics

0971f93

[pre-commit.ci] auto fixes from pre-commit.com hooks

f7126e8

for more information, see https://pre-commit.ci

leonieschweiger approved these changes Jul 24, 2026

View reviewed changes

use HVC input for plastics export

649ec95

Co-authored-by: Leonie Schweiger <leonie.schweiger@pik-potsdam.de>

rename remind_input to mrindustry

8d7a58d

tobiasdiez requested review from JakobBD and bennet21 July 24, 2026 19:19

tobiasdiez marked this pull request as ready for review July 24, 2026 19:19

tobiasdiez changed the title ~~feat: add export for REMIND~~ feat: add export for REMIND/mrindustry Jul 24, 2026

JakobBD reviewed Jul 27, 2026

View reviewed changes

JakobBD approved these changes Jul 27, 2026

View reviewed changes

bennet21 approved these changes Jul 27, 2026

View reviewed changes

Comment thread remind_mfa/cement/cement_export.py

JakobBD mentioned this pull request Jul 27, 2026

feat: add notebook for comparing mrindustry and MFA steel data #163

Open

JakobBD reviewed Jul 28, 2026

View reviewed changes

tobiasdiez and others added 2 commits July 29, 2026 16:07

Merge remote-tracking branch 'origin/main' into export_to_remind

137803c

Update steel export

b95a5bc

Co-authored-by: Jakob Duerrwaechter <jakob.duerrwaechter@pik-potsdam.de>

Uh oh!

Conversation

tobiasdiez commented Jul 24, 2026

Uh oh!

tobiasdiez commented Jul 24, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JakobBD commented Jul 24, 2026

Uh oh!

bennet21 commented Jul 24, 2026

Uh oh!

tobiasdiez commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

leonieschweiger left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tobiasdiez commented Jul 24, 2026

Uh oh!

tobiasdiez commented Jul 24, 2026

Uh oh!

tobiasdiez commented Jul 24, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JakobBD left a comment

Choose a reason for hiding this comment

Uh oh!

bennet21 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tobiasdiez commented Jul 27, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tobiasdiez commented Jul 24, 2026 •

edited

Loading