Interpolation function overwriting changes to same file from base case

[robbieorvis]

When using setInterpolationFunction for a policy scenario, if a file is being modified that is also modified in the base case, the base case changes are overwritten, probably because pygcam is starting from a fresh copy of the file in the project space instead of in the sandbox.

For example, I have a coal ban in the base case for certain regions and I am testing increasing the share-weights for gas in policy scenarios, but when doing so the coal ban is no longer carried through.

[rjplevin]

Yes, this is why in later code, I've moved toward using a config file tag to identify the file. As you noted, this creates a chicken & egg problem in that the generated config.xml must exist so that the tag can be used to lookup the name of the relevant xml file. It's a bit brittle currently and I haven't figured out how best to handle this.

One possibility is to make the config file tag an optional argument; if not present, the reference file is used, which is what you'd do in a baseline, though you'd still have to ensure that this was the first manipulation of that file, which is still too brittle.

If a file tag were provided--policy scenarios only, since this would reference the baseline's config.xml--this would be used to identify the file to operate on.

We might want to discuss use patterns to see what's most convenient / user-friendly.

[robbieorvis]

Hm okay. That makes sense and sounds familiar though I'm forgetting a bit about the chicken and egg issue.

Just running through some ideas here to see if I understand correctly.

For the base case, I don't think there is an issue, right? Because it should always modify the reference files. And the scenarios don't work unless the setup for the base case has already been completed.

For the policy case, the issue is that pygcam doesn't know whether or not to change the reference XML file or files modified in the base. By default it changes the reference files. There's probably a reason why this won't work, but is there a reason why pygcam can't start with the base config file and use that to find the right file to modify? After the files are modified and copied to the sandbox, it could then delete the config file and create a new one referencing the files in the sandbox? I'm probably missing something here, though...