Hercules v2 checklist

[paulf81]

Hercules v2: https://github.com/NREL/hercules/tree/v2 is now online as a branch of Hercules. I think it's getting close to ready and so wanted to have a place to track steps toward release. Maybe with some light tagging (@misi9170 and @genevievestarke tagging up here to have you review all)

Issue and branch clean up

• Clear hercules issues that are taken care of by v2
• Add issues of things we want to include in v2 that don't block it's release
• Remove any stale branches
• All PRs to main after creation of long-run are in v2 in some way

Examples

• Do we still need the separate shell runscript? I think we could relax to just the python runscript to simplify things. Could be useful for kestrel though,
• Remove all bash scripts
• Update docs and scripts to not assume a specific package manager

Solar

• Remove simplesolar (@brookeslawski hope that is ok with you)
• Split solar_pysam into 2, one for pysam and one for pvwatts @brookeslawski if you could weigh in
• After @brookeslawski considers which pvwatts inputs always/with-default/hard-coded, implement the changes here

Update the demo

• Remove bash scripts from demo
• Make sure all scenarios work following updates

Naming

• @ZackTully , what do you think of lib -> li_battery
• better name for wind_sim_long_term

Tools/Utilities

• Consolidate tools and utilities to gether

File Organization

• Reorganize Example_case_folders → examples
• Remove numbers from folders (add ordering to docs instead) (PF: I like the numbers but willing to be persuaded)
• Delete scripts/ folder (no longer needed)

Version Updates

• Update version number in pyproject.toml
• Update version in CITATION.cff
• Add test to confirm they are the same

Controller Dependencies

• Remove controller_standin and use in place definitions for simple controllers (as in demo) instead

Solar updates

• Making solar and wind timing test
• Also add a new test the curtailment works as expected for all wind and solar
• Make change from target to adjustment as means of curtailment
• Add documentation of controllers
• All tests pass
• check new timing
• Make input file reader work with multiple column name options

Wind

• Rename derating to setpoint on wind side for consistency
• Move to running FLORIS on regular interval

Make sure all issues Brooke raised below are cleared

• Missing files

Emulator Issues

• Fix progress bar overwriting (prints on new lines instead) [Maybe this isn't an issue for everyone? Perhaps to do with packages]
• Move logging dict before main loop (remove first_iteration logic)
• Review __del__ and close() methods - determine if both needed (PF: This fixes some warnings about improper file closures in tests)
• Improve rounding precision (consider 3 sig figs vs 3 decimal places)
• Search and review "py sims" references

HybridPlant Improvements

• Replace if statements with ModelMap dict (see FLORIS for an example)
• Convert self.component_objects from dict to list
• Simplify step() method implementation

Electrolyzer

• Update electrolyzer to v2
• Ensure working tests
• Make sure there is a working example
• Update docs to include electrolyzer

Fixes to go

• Example 05 (wind_solar_storage) needs to be updated to v2 style, or possibly simply removed

Hidden functions

• Sweep code, where functions can be appropriately made hidden do so

Speed and memory

• Add a hercules data type and use in code to reduce memory
• Use consistent data structures (likely numpy arrays) in the run loop
• Ensure all memory is declared prior to the run loop to a) make using consistent data types easier and b) avoid memory leaks in long simulations

Decide on whether to fix now or make issues for later

• Do we want to keep the name py_sim or simplify the structure?
• Do we want to keep h_dict as dict, or something inherited
• Need to remove rated_power from 1DOF model (@abhineet-gupta ), see code, this should now be directly inferred from FLORIS, can follow filter model example (see 1DOF model does not include region 3 and can not determine rated power #157)

[paulf81]

@misi9170 just making a quick note on missed merges into main, I think checking the git history this can be ticked but just putting my logic here. You can see the last time v2 merged from develop here:

After that the other commits I think are just develop into main except for #153

And checking the edits output keys are no longer in v2 anyway. So ticking the box but wanted to provide justification

[brookeslawski]

Looking at https://github.com/NREL/hercules/blob/40049004e2641b5d401e7de7c548f66290e57c7b/hercules/plant_components/solar_pysam_base.py#L81, do we want to support the weather_data_input .csv file at all? Right now, when I try to run it with weather_data_input: ../07_floris_standin_and_solar_pysam/inputs/Flatirons_solar_data_sunset.csv, I get this error:

$ python hercules_runscript.py Traceback (most recent call last): File "/Users/bstanisl/hercules-pysam/hercules/example_case_folders/03_wind_and_solar/hercules_runscript.py", line 41, in <module> hybrid_plant = HybridPlant(h_dict) ^^^^^^^^^^^^^^^^^^^ File "/Users/bstanisl/hercules-pysam/hercules/hercules/hybrid_plant.py", line 57, in __init__ self.component_objects[component_name] = self.get_plant_component( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bstanisl/hercules-pysam/hercules/hercules/hybrid_plant.py", line 102, in get_plant_component return SolarPySAMPVWatts(h_dict) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bstanisl/hercules-pysam/hercules/hercules/plant_components/solar_pysam_pvwatts.py", line 21, in __init__ super().__init__(h_dict) File "/Users/bstanisl/hercules-pysam/hercules/hercules/plant_components/solar_pysam_base.py", line 48, in __init__ self._load_solar_data(h_dict) File "/Users/bstanisl/hercules-pysam/hercules/hercules/plant_components/solar_pysam_base.py", line 100, in _load_solar_data df_solar = pd.DataFrame.from_dict(h_dict[self.component_name]["weather_data_input"]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bstanisl/miniforge3/envs/hercules-pysam6-whoc/lib/python3.11/site-packages/pandas/core/frame.py", line 1813, in from_dict return cls(data, index=index, columns=columns, dtype=dtype) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bstanisl/miniforge3/envs/hercules-pysam6-whoc/lib/python3.11/site-packages/pandas/core/frame.py", line 841, in __init__ raise ValueError("DataFrame constructor not properly called!") ValueError: DataFrame constructor not properly called!

[paulf81]

i think you're right here, this is probably an unnecessary and confusing backwards compatibility, since the file is ultimately the same information, going to refactor it out now

[paulf81]

done

[brookeslawski]

Looking at https://github.com/NREL/hercules/blob/ee848b59f5b36ad9b7c4d3c3001cab60266cc476/hercules/plant_components/solar_pysam_pvwatts.py#L45

I believe the target_system_capacity and target_dc_ac_ratio are now inherited in the SolarPySAMBase class, so do we want to use those values instead of re-reading them in from h_dict?

[paulf81]

I'm going to look at this now, but could I ask @brookeslawski , isn't target_system_capacity a kind of vague term? Like, in my mind once a system is built, shouldn't the capacity be fixed, this feels like this with this number we're more impacting the power_setpoint or similar, @misi9170 , also would like your input here

On the point of the comment I think you're right, this feels like it should just happen once in for all in base if the same for the two types of solar

[paulf81]

ok I consolidated this to the base, but I think I would like to check what you think of this understanding @brookeslawski:

target_dc_ac_ratio: this is a fixed design parameter that is unaffected by controls so it's appropriate this is set once at start of sim and held fixed

target_system_capacity: in pysam, this is supposed to also be a fixed parameter, but we're using it to allow derating/curailment, hence the slightly awkward naming

[brookeslawski]

oh, I didn't realize that target_system_capacity was being used for curtailment. this name originally came from the PySAM developer who implemented dynamic sizing in Hercules, where "dynamic sizing" refers just to being able to specify the PV system capacity in the input file rather than generating a .json for the system as a pre-processing step. so in that regard, I think the variable name in the input file should be system_capacity, not target_system_capacity. It sounds like target_system_capacity should be a separate variable used in the simulation for the curtailment target

[brookeslawski]

It looks like we still use some files from the previous Ex 07 (floris standin and solar):

• 07_floris_standin_and_solar_pysam/inputs/Flatirons_solar_data_sunset.csv (to generate solar_input.p)
• 07_floris_standin_and_solar_pysam/inputs/100MW_1axis_pvsamv1.json (to run the detailed pv model (component_type: SolarPySAMPVSam))
  Will these files be removed when 07_floris_standin_and_solar_pysam is removed? If so, I'd suggest we put them in 03_wind_and_solar/inputs/

[paulf81]

that's a good catch @brookeslawski , I'll move these back soon

[paulf81]

ok think this is done now

[brookeslawski]

I've reviewed the solar files (python scripts and documentation) and made some minor updates. I've removed solar_pysam.py since that functionality was split out into the 3 new files. Overall, this new structure looks great! It's so fast and simple.

[paulf81]

Thanks very much!

[paulf81]

@brookeslawski and @misi9170 one other solar refactor question, should we move to insisting that the solar input file data, has column names that match the expected simpler names within Hercules, like dni, instead of "SRRL BMS Direct Normal Irradiance (W/m²_irr)". It made sense when were were starting out but now I think for clarity and consistency with wind, good to just make these the same, rather than one thing in the input file and another in the code

[misi9170]

Perhaps those names are so that PySAM files can be used directly (PySAM expects those column names)? @brookeslawski could you confirm?

[paulf81]

maybe the right solution is to allow either with a dictionary that maps a few possible column heads to dni or something like it, then we could have unrecognized keys etc., shouldn't take long to work up, what do you think @misi9170 and @brookeslawski ?

[brookeslawski]

Yeah, those column names (like "SRRL BMS Direct Normal Irradiance (W/m²_irr)") are expected by PySAM. So yes, maybe this solution that maps a few possible columns to dni or something could work

[misi9170]

Sure!

[paulf81]

ok added to todo list above

[paulf81]

@genevievestarke and @misi9170 I updated the list above per our discussion today

[misi9170]

Thanks @paulf81 ! I added a couple more notes to the "Speed and memory" list

[abhineet-gupta]

Couple of observations:

1. Running example 00 or 01 does not seem to generate the feather files (and so, unable to plot the output) for the v2 branch.
2. I optimized the 1DOF model to increase its speed and it now runs at around 75% of the speed of the filter model. It adds around 3-5 seconds (independent of run duration) to the initialization time.
   This makes it vaible for some long term demos showcasing a more dynamic simulation if needed.

[paulf81]

Thanks @abhineet-gupta , this is really cool, thanks for working on the speed! I have to tinker a little more the output file, writing all the time was too slow, waiting to write to memory intense, and the current writing in chunks is a little kludgy. Hope to have a fix in today and that should fix the issue. Sorry for that!