Reorganize logging in Hercules

docs/battery.md

@@ -32,6 +32,7 @@ Battery parameters are defined in the hercules input yaml file used to initializ
 - `usage_calc_interval`: Interval for usage calculations in seconds (BatterySimple only)
 - `usage_lifetime`: Battery lifetime in years for time-based degradation (BatterySimple only)
 - `usage_cycles`: Number of cycles until replacement for cycle-based degradation (BatterySimple only)
+- `log_channels`: List of output channels to log (see [Logging Configuration](#logging-configuration) below)
 
 
 Once initialized, the battery is only interacted with using the `step` method.
@@ -55,12 +56,41 @@ Outputs are returned as a dict containing the following values:
 - `power`: Actual battery power in kW 
 - `reject`: Rejected power due to constraints in kW (positive when power cannot be absorbed, negative when required power unavailable)
 - `soc`: Battery state of charge (0-1)
+- `power_setpoint`: Requested power setpoint in kW
 
 #### Additional Outputs (BatterySimple only when track_usage=True)
 - `usage_in_time`: Time-based usage percentage
 - `usage_in_cycles`: Cycle-based usage percentage  
 - `total_cycles`: Total equivalent cycles completed
 
+### Logging Configuration
+
+The `log_channels` parameter controls which outputs are written to the HDF5 output file. This is a list of channel names. The `power` channel is always logged, even if not explicitly specified.
+
+**Available Channels:**
+- `power`: Actual battery power output in kW (always logged)
+- `soc`: State of charge (0-1)
+- `power_setpoint`: Requested power setpoint in kW
+
+**Example:**
+```yaml
+battery:
+  component_type: BatterySimple
+  energy_capacity: 100.0  # kWh
+  charge_rate: 50.0  # kW
+  discharge_rate: 50.0  # kW
+  max_SOC: 0.9
+  min_SOC: 0.1
+  log_channels:
+    - power
+    - soc
+    - power_setpoint
+  initial_conditions:
+    SOC: 0.5
+```
+
+If `log_channels` is not specified, only `power` will be logged.
+
 
 ## `BatterySimple`
 

docs/h_dict.md

@@ -31,12 +31,13 @@ The `h_dict` is a Python dictionary that contains all the configurations for eac
 | **Hybrid Plant Components** |
 
 ### Wind Farm (`wind_farm`)
-| `component_type` | str | Must be "Wind_MesoToPower" |
+| `component_type` | str | Must be "Wind_MesoToPower" or "Wind_MesoToPowerPrecomFloris" |
 | `floris_input_file` | str | FLORIS input file path |
 | `wind_input_filename` | str | Wind data input file |
 | `turbine_file_name` | str | Turbine configuration file |
 | `log_file_name` | str | Wind farm log file path |
-| `logging_option` | str | Logging level: "base", "turb_subset", or "all" |
+| `log_channels` | list | List of channels to log (e.g., ["power", "wind_speed_mean_background", "turbine_powers"]) |
+| `floris_update_time_s` | float | How often to update FLORIS wake calculations in seconds |
 
 ### Solar Farm (`solar_farm`)
 | `component_type` | str | "SolarPySAMPVWatts" |
@@ -48,6 +49,7 @@ The `h_dict` is a Python dictionary that contains all the configurations for eac
 | `lat` | float | Latitude |
 | `lon` | float | Longitude |
 | `elev` | float | Elevation in meters |
+| `log_channels` | list | List of channels to log (e.g., ["power", "dni", "poa", "aoi"]) |
 | `initial_conditions` | dict | Initial power, DNI, POA |
 
 ### Battery (`battery`)
@@ -61,6 +63,7 @@ The `h_dict` is a Python dictionary that contains all the configurations for eac
 | `min_SOC` | float | Minimum state of charge (0-1) | Required |
 | `initial_conditions` | dict | Contains initial SOC | Required |
 | `allow_grid_power_consumption` | bool | Allow grid power consumption | False |
+| `log_channels` | list | List of channels to log (e.g., ["power", "soc", "power_setpoint"]) | ["power"] |
 | `roundtrip_efficiency` | float | Roundtrip efficiency (BatterySimple only) | 1.0 |
 | `self_discharge_time_constant` | float | Self-discharge time constant in seconds (BatterySimple only) | inf |
 | `track_usage` | bool | Enable usage tracking (BatterySimple only) | False |

docs/hercules_input.md

@@ -48,7 +48,11 @@ wind_farm:
   wind_input_filename: inputs/wind_input.csv
   turbine_file_name: inputs/turbine_filter_model.yaml
   log_file_name: outputs/log_wind_sim.log
-  logging_option: all
+  log_channels:
+    - power
+    - wind_speed_mean_background
+    - wind_speed_mean_withwakes
+    - wind_direction_mean
   floris_update_time_s: 30.0
 
 solar_farm:
@@ -59,6 +63,11 @@ solar_farm:
   elev: 1829
   system_capacity: 10000  # kW (10 MW)
   tilt: 0  # degrees
+  log_channels:
+    - power
+    - dni
+    - poa
+    - aoi
   initial_conditions:
     power: 2000  # kW
     dni: 1000
@@ -71,6 +80,10 @@ battery:
   discharge_rate: 50.0  # MW
   max_SOC: 0.95
   min_SOC: 0.05
+  log_channels:
+    - power
+    - soc
+    - power_setpoint
   initial_conditions:
     SOC: 0.5
 
@@ -126,7 +139,11 @@ wind_farm:
   floris_input_file: inputs/floris_input.yaml
   wind_input_filename: inputs/wind_input.csv
   turbine_file_name: inputs/turbine_filter_model.yaml
-  logging_option: all
+  log_channels:
+    - power
+    - wind_speed_mean_background
+    - wind_speed_mean_withwakes
+    - wind_direction_mean
   floris_update_time_s: 30.0
 
 controller:
@@ -141,6 +158,6 @@ The `load_hercules_input()` function performs strict validation on input files t
 - Numeric validation for timing and power parameters
 - File existence checks for referenced input files
 - Output configuration validation (`log_every_n` must be a positive integer)
-- Component-specific validation (e.g., wind farm `logging_option` must be "base", "turb_subset", or "all")
+- Component-specific validation (e.g., `log_channels` must be a list of valid channel names)
 
 Invalid configurations will raise descriptive `ValueError` exceptions to help with debugging.

docs/output_files.md

@@ -18,10 +18,18 @@ hercules_output.h5
 │   ├── plant_power             # Total plant power output
 │   ├── plant_locally_generated_power  # Locally generated power
 │   ├── components/
-│   │   ├── wind_farm.power     # Wind farm power output
-│   │   ├── wind_farm.wind_speed # Wind speed at hub height
-│   │   ├── solar_farm.power    # Solar farm power output
-│   │   └── ...                 # Other component outputs
+│   │   ├── wind_farm.power                   # Wind farm power output
+│   │   ├── wind_farm.wind_speed_mean_background # Farm-average background wind speed
+│   │   ├── wind_farm.wind_speed_mean_withwakes   # Farm-average with-wakes wind speed
+│   │   ├── wind_farm.wind_direction_mean     # Farm-average wind direction
+│   │   ├── wind_farm.turbine_powers.000      # Turbine 0 power (if logged)
+│   │   ├── wind_farm.turbine_powers.001      # Turbine 1 power (if logged)
+│   │   ├── solar_farm.power                  # Solar farm power output
+│   │   ├── solar_farm.dni                    # Direct normal irradiance (if logged)
+│   │   ├── solar_farm.poa                    # Plane-of-array irradiance (if logged)
+│   │   ├── battery.power                     # Battery power (if present)
+│   │   ├── battery.soc                       # Battery state of charge (if logged)
+│   │   └── ...                               # Other component outputs
 │   └── external_signals/
 │       └── ...                 # Other external signals
 └── metadata/
@@ -60,7 +68,7 @@ from hercules.utilities import read_hercules_hdf5_subset
 # Read specific columns
 df_subset = read_hercules_hdf5_subset(
     "outputs/hercules_output.h5",
-    columns=["wind_farm.power", "solar_farm.power", "external_signals.wind_speed"]
+    columns=["wind_farm.power", "wind_farm.wind_speed_mean_background", "solar_farm.power"]
 )
 
 # Read specific time range (seconds)

docs/solar_pv.md

@@ -35,7 +35,30 @@ The PVWatts model is configured with the following hardcoded parameters for util
 
 The array tilt angle must be specified in the input configuration file.
 
-When `log_extra_outputs` is set to `True` in the input .yaml file, the solar modules also output plane-of-array irradiance (`poa`) in W/m^2, direct normal irradiance (`dni`) in W/m^2, and the angle of incidence (`aoi`) in degrees.
+### Logging Configuration
+
+The `log_channels` parameter controls which outputs are written to the HDF5 output file. This is a list of channel names. The `power` channel is always logged, even if not explicitly specified.
+
+**Available Channels:**
+- `power`: DC power output in kW (always logged)
+- `poa`: Plane-of-array irradiance in W/m²
+- `dni`: Direct normal irradiance in W/m²
+- `aoi`: Angle of incidence in degrees
+
+**Example:**
+```yaml
+solar_farm:
+  component_type: SolarPySAMPVWatts
+  solar_input_filename: inputs/solar_input.csv
+  log_channels:
+    - power
+    - dni
+    - poa
+    - aoi
+  # ... other parameters
+```
+
+If `log_channels` is not specified, only `power` will be logged.
 
 ### Efficiency and Loss Parameters
 

docs/wind.md

@@ -37,21 +37,13 @@ Required parameters for both components in [h_dict](h_dict.md) (see [timing](tim
 
 Required parameters for Wind_MesoToPower:
 - `floris_update_time_s`: How often to update FLORIS (the last `floris_update_time_s` seconds are averaged as input)
-
-Required parameters for Wind_MesoToPower:
-- `logging_option`: Logging level. Options are:
-  - `"base"`: Log basic outputs (power, wind_speed, wind_direction, wind_speed_waked)
-  - `"turb_subset"`: Base outputs plus 3 random turbines' waked_velocities, turbine_powers, and turbine_power_setpoints
-  - `"all"`: All available outputs including floris_wind_speed, floris_wind_direction, floris_ti, unwaked_velocities, waked_velocities, turbine_powers, turbine_power_setpoints
+- `log_channels`: List of output channels to log. See [Logging Configuration](#logging-configuration) section below for details.
 
 ### Wind_MesoToPowerPrecomFloris Specific Parameters
 
 Required parameters for Wind_MesoToPowerPrecomFloris:
 - `floris_update_time_s`: Determines the cadence of wake precomputation. At each cadence tick, the last `floris_update_time_s` seconds are averaged and used to evaluate FLORIS. The computed wake deficits are then applied until the next cadence tick.
-- `logging_option`: Logging level. Options are:
-  - `"base"`: Log basic outputs (power, wind_speed, wind_direction, wind_speed_waked)
-  - `"turb_subset"`: Base outputs plus 3 random turbines' waked_velocities, turbine_powers, and turbine_power_setpoints
-  - `"all"`: All available outputs including floris_wind_speed, floris_wind_direction, floris_ti, unwaked_velocities, waked_velocities, turbine_powers, turbine_power_setpoints
+- `log_channels`: List of output channels to log. See [Logging Configuration](#logging-configuration) section below for details.
 
 ## Turbine Models
 
@@ -65,33 +57,81 @@ Advanced model with rotor dynamics, pitch control, and generator torque control.
 
 ### Common Outputs
 
-Both components provide these outputs:
-- `power`: Total wind farm power
-- `turbine_powers`: Individual turbine power outputs  
-- `turbine_power_setpoints`: Current power setpoint values
-- `wind_speed`, `wind_direction`: Farm-level wind conditions
-
-### Logging Options
-
-The logging behavior depends on the `logging_option` setting:
-
-#### Base Logging (`logging_option: "base"`)
-- `power`: Total wind farm power
-- `wind_speed`, `wind_direction`: Farm-level wind conditions
-- `wind_speed_waked`: Average waked wind speed across the farm
-
-#### Turbine Subset Logging (`logging_option: "turb_subset"`)
-Includes all base outputs plus:
-- `waked_velocities_turb_XXX`: Waked velocities for 3 randomly selected turbines
-- `turbine_powers_turb_XXX`: Power outputs for 3 randomly selected turbines  
-- `turbine_power_setpoints_turb_XXX`: Power setpoints for 3 randomly selected turbines
-
-#### Full Logging (`logging_option: "all"`)
-Includes all base outputs plus:
-- `turbine_powers`: Individual turbine power outputs
-- `turbine_power_setpoints`: Current power setpoint values
-- `floris_wind_speed`: Wind speed used in FLORIS calculations
-- `floris_wind_direction`: Wind direction used in FLORIS calculations
-- `floris_ti`: Turbulence intensity values
-- `unwaked_velocities`: Wind speeds without wake effects
-- `waked_velocities`: Wind speeds with wake effects applied
+Both components provide these outputs in the h_dict at each simulation step:
+- `power`: Total wind farm power (kW)
+- `turbine_powers`: Individual turbine power outputs (array, kW)
+- `turbine_power_setpoints`: Current power setpoint values (array, kW)
+- `wind_speed_mean_background`: Farm-average background wind speed (m/s)
+- `wind_speed_mean_withwakes`: Farm-average with-wakes wind speed (m/s)
+- `wind_direction_mean`: Farm-average wind direction (degrees)
+- `wind_speeds_background`: Per-turbine background wind speeds (array, m/s)
+- `wind_speeds_withwakes`: Per-turbine with-wakes wind speeds (array, m/s)
+
+## Logging Configuration
+
+The `log_channels` parameter controls which outputs are written to the HDF5 output file. This is a list of channel names. The `power` channel is always logged, even if not explicitly specified.
+
+### Available Channels
+
+**Scalar Channels:**
+- `power`: Total wind farm power output (kW)
+- `wind_speed_mean_background`: Farm-average background wind speed (m/s)
+- `wind_speed_mean_withwakes`: Farm-average with-wakes wind speed (m/s)  
+- `wind_direction_mean`: Farm-average wind direction (degrees)
+
+**Array Channels:**
+- `turbine_powers`: Power output for all turbines (creates datasets like `wind_farm.turbine_powers.000`, `wind_farm.turbine_powers.001`, etc.)
+- `turbine_power_setpoints`: Power setpoints for all turbines
+- `wind_speeds_background`: Background wind speeds for all turbines
+- `wind_speeds_withwakes`: With-wakes wind speeds for all turbines
+
+### Selective Array Element Logging
+
+For large wind farms, logging all turbine data can significantly increase file size and slow down the simulation. You can log specific turbine indices by appending a 3-digit turbine index to the channel name:
+
+```yaml
+# Log only turbines 0, 5, and 10
+log_channels:
+  - power
+  - wind_speed_mean_background
+  - wind_speed_mean_withwakes
+  - wind_direction_mean
+  - turbine_powers.000
+  - turbine_powers.005
+  - turbine_powers.010
+```
+
+### Example Configurations
+
+**Minimal Logging:**
+```yaml
+log_channels:
+  - power
+  - wind_speed_mean_background
+  - wind_speed_mean_withwakes
+  - wind_direction_mean
+```
+
+**Detailed Logging (all turbines):**
+```yaml
+log_channels:
+  - power
+  - wind_speed_mean_background
+  - wind_speed_mean_withwakes
+  - wind_direction_mean
+  - turbine_powers
+  - wind_speeds_withwakes
+```
+
+**Selected Turbine Logging:**
+```yaml
+# Log first 3 turbines only
+log_channels:
+  - power
+  - wind_speed_mean_background
+  - wind_speed_mean_withwakes
+  - wind_direction_mean
+  - turbine_powers.000
+  - turbine_powers.001
+  - turbine_powers.002
+```

examples/00_wind_farm_only/hercules_input.yaml

@@ -5,7 +5,7 @@ name: example_00
 
 ###
 # Describe this emulator setup
-description: Wind Farm Only
+description: Wind Farm Only, Logging All Turbine Data
 
 dt: 1.0
 starttime: 0.0
@@ -22,7 +22,14 @@ wind_farm:
   wind_input_filename: ../inputs/wind_input_small.ftr
   turbine_file_name: ../inputs/turbine_filter_model.yaml
   log_file_name: outputs/log_wind_sim.log
-  logging_option: all
+  log_channels:
+    - power
+    - wind_speed_mean_background
+    - wind_speed_mean_withwakes
+    - wind_direction_mean
+    - turbine_powers
+    - wind_speeds_withwakes
+    - wind_speeds_background
   floris_update_time_s: 30.0 # Update wakes every 30 seconds