Skip to content

ENH: Add Monte Carlo outputs export to CSV/JSON (#242) #912

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
GMinoruy wants to merge 6 commits into
base: develop
Choose a base branch
from
Open

ENH: Add Monte Carlo outputs export to CSV/JSON (#242) #912

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e51b221
ENH: trying to add Monte Carlo output to csv and json see issue #242
GMinoruy Dec 4, 2025
fe7c69e
ENH: corrects errors identified during testing
GMinoruy Dec 6, 2025
1c1c77a
ENH: adds example on how to use the export_results function
GMinoruy Dec 6, 2025
ee4bab5
ENH: adds visual feedback when exporting in csv and json
GMinoruy Dec 6, 2025
d0cc223
ENH: adds Docstring to the function
GMinoruy Dec 7, 2025
6503273
ENH: adds unit test of the function export_results
GMinoruy Dec 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions docs/notebooks/monte_carlo_analysis/monte_carlo_class_usage.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -800,6 +800,34 @@
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also export our Monte Carlo results into .csv and .json files using the method `MonteCarlo.export_results()`.\n",
"\n",
"Choose a name for the output file and select a format to output."
Comment on lines +807 to +809
Copy link

Copilot AI Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The documentation could be more informative. Consider adding:

  1. Mention that the file extension will be added automatically
  2. Note that valid formats are "csv" and "json" (case-insensitive)
  3. Clarify that the method reads from the previously generated .outputs.txt file

Example improvement:

We can also export our Monte Carlo results to CSV or JSON files using the `MonteCarlo.export_results()` method.

Provide the output filename (without extension) and the desired format ("csv" or "json"). The appropriate file extension will be added automatically. The method reads data from the `.outputs.txt` file generated during simulation.

Copilot uses AI. Check for mistakes.
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Results saved to monte_carlo_analysis_outputs/monte_carlo_csv_output_example as .csv file\n",
"Results saved to monte_carlo_analysis_outputs/monte_carlo_json_output_example as .json file\n"
]
}
],
"source": [
"test_dispersion.export_results(\"monte_carlo_analysis_outputs/monte_carlo_csv_output_example\", \"csv\")\n",
"test_dispersion.export_results(\"monte_carlo_analysis_outputs/monte_carlo_json_output_example\", \"json\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
Expand Down
36 changes: 36 additions & 0 deletions rocketpy/simulation/monte_carlo.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
"""

import json
import csv
import os
import traceback
import warnings
Expand Down Expand Up @@ -588,6 +589,41 @@ def __evaluate_flight_outputs(self, flight, sim_idx):
json.dumps(outputs_dict, cls=RocketPyEncoder, **self._export_config) + "\n"
)

def export_results(self, output_filename, output_format):
"""Converts the default Monte Carlo .txt output to .cvs or .json file
Copy link

Copilot AI Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo in docstring: ".cvs" should be ".csv"

Suggested change
"""Converts the default Monte Carlo .txt output to .cvs or .json file
"""Converts the default Monte Carlo .txt output to .csv or .json file

Copilot uses AI. Check for mistakes.
depending on the user's choice

Parameters
----------
output_filename : str
Name of the file in which the converted data will be saved
output_format : str
Format of the output file

Returns
-------
None
Comment on lines +593 to +605
Copy link

Copilot AI Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docstring is missing important details. It should:

  1. Specify that output_filename can be a string or Path object
  2. Document the valid values for output_format ("csv" or "json")
  3. Explain what happens if the output file already exists (will it be overwritten?)
  4. Mention that the method reads from self.filename.outputs.txt
  5. Follow NumPy docstring format with a proper description section before Parameters

Example improvement:

\"\"\"Export Monte Carlo simulation results to CSV or JSON format.

This method reads simulation output data from the `.outputs.txt` file
and converts it to either CSV or JSON format based on user selection.

Parameters
----------
output_filename : str or Path
    Path and name of the output file (without extension). The appropriate
    extension (.csv or .json) will be added automatically.
output_format : str
    Format of the output file. Must be either "csv" or "json" (case-insensitive).

Returns
-------
None

Raises
------
FileNotFoundError
    If the `.outputs.txt` file does not exist.
ValueError
    If `output_format` is not "csv" or "json".
\"\"\"
Suggested change
"""Converts the default Monte Carlo .txt output to .cvs or .json file
depending on the user's choice
Parameters
----------
output_filename : str
Name of the file in which the converted data will be saved
output_format : str
Format of the output file
Returns
-------
None
"""Export Monte Carlo simulation results to CSV or JSON format.
This method reads simulation output data from the `.outputs.txt` file
(specifically, from `self.filename.outputs.txt`) and converts it to either
CSV or JSON format based on user selection. The output file will be
overwritten if it already exists.
Parameters
----------
output_filename : str or Path
Path and name of the output file (without extension). The appropriate
extension (.csv or .json) will be added automatically.
output_format : str
Format of the output file. Must be either "csv" or "json"
(case-insensitive).
Returns
-------
None
Raises
------
FileNotFoundError
If the `.outputs.txt` file does not exist.
ValueError
If `output_format` is not "csv" or "json".
Examples
--------
>>> mc.export_results("results", "csv")
>>> mc.export_results(Path("results_folder/results"), "json")

Copilot uses AI. Check for mistakes.
"""
txt_data = []
with open(f"{self.filename}.outputs.txt", "r", encoding="utf-8") as f:
for line in f:
line = line.strip()
data = json.loads(line)
txt_data.append(data)
Comment on lines +608 to +612
Copy link

Copilot AI Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing error handling for file I/O operations. The method should handle FileNotFoundError when the .outputs.txt file doesn't exist and provide a clear error message to the user. Additionally, consider handling json.JSONDecodeError if the file contains malformed JSON data.

Suggested change
with open(f"{self.filename}.outputs.txt", "r", encoding="utf-8") as f:
for line in f:
line = line.strip()
data = json.loads(line)
txt_data.append(data)
try:
with open(f"{self.filename}.outputs.txt", "r", encoding="utf-8") as f:
for line_number, line in enumerate(f, start=1):
line = line.strip()
try:
data = json.loads(line)
txt_data.append(data)
except json.JSONDecodeError as e:
_SimMonitor.reprint(
f"Error decoding JSON on line {line_number} of '{self.filename}.outputs.txt': {e}\n"
f"Line content: {line}\n"
"Please check the file for malformed JSON data."
)
raise
except FileNotFoundError:
_SimMonitor.reprint(
f"File '{self.filename}.outputs.txt' not found. "
"Please ensure that the Monte Carlo simulation has been run and the output file exists."
)
raise

Copilot uses AI. Check for mistakes.

output_format = output_format.strip().lower()
if output_format == "json":
with open(f"{output_filename}.json", "w", encoding="utf-8") as f:
json.dump(txt_data, f, indent=4)
_SimMonitor.reprint(f"Results saved to {Path(output_filename)} as .json file")
elif output_format == "csv":
output_csv_header = txt_data[0].keys()
Copy link

Copilot AI Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential IndexError if txt_data is empty. If the .outputs.txt file is empty or contains only blank lines, txt_data[0] will raise an IndexError. Add validation to check that txt_data is not empty before attempting to access its first element.

Copilot uses AI. Check for mistakes.
with open(f"{output_filename}.csv", "w", newline= "") as f:
Copy link

Copilot AI Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra space before empty string in newline= "" parameter. Should be newline="" (no space before the equals sign) to follow Python style conventions (PEP 8).

Suggested change
with open(f"{output_filename}.csv", "w", newline= "") as f:
with open(f"{output_filename}.csv", "w", newline="") as f:

Copilot uses AI. Check for mistakes.
output_writer = csv.DictWriter(f, fieldnames=output_csv_header)
output_writer.writeheader()
output_writer.writerows(txt_data)
_SimMonitor.reprint(f"Results saved to {Path(output_filename)} as .csv file")
Comment on lines +614 to +625
Copy link

Copilot AI Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing input validation for output_format parameter. The method should validate that output_format is either "csv" or "json" and raise a ValueError with a helpful message if an unsupported format is provided. Currently, if an unsupported format is passed, the method silently does nothing, which could confuse users.

Copilot uses AI. Check for mistakes.

def __terminate_simulation(self):
"""
Terminates the simulation, closes the files and prints the results.
Expand Down
30 changes: 30 additions & 0 deletions tests/unit/simulation/test_monte_carlo.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
import matplotlib as plt
import numpy as np
import pytest
import json
import os

from rocketpy.simulation import MonteCarlo

Expand Down Expand Up @@ -185,3 +187,31 @@ def test_estimate_confidence_interval_raises_type_error_for_invalid_statistic():

with pytest.raises(TypeError):
mc.estimate_confidence_interval("apogee", statistic="not_a_function")


def test_monte_carlo_export_results_to_csv_and_json_files(monte_carlo_calisto, tmp_path):
Copy link

Copilot AI Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test function name could be more descriptive and follow the pattern test_methodname_expectedbehaviour. Consider renaming to test_export_results_creates_csv_and_json_files to better indicate what method is being tested and what behavior is expected.

Suggested change
def test_monte_carlo_export_results_to_csv_and_json_files(monte_carlo_calisto, tmp_path):
def test_export_results_creates_csv_and_json_files(monte_carlo_calisto, tmp_path):

Copilot uses AI. Check for mistakes.
"""Checks that the export_results create .csv and .json files

Parameters
----------
monte_carlo_calisto : MonteCarlo
Fixture that has the .txt files necessary for the export_results
"""
try:
mc = monte_carlo_calisto
mc.filename = tmp_path / "mock_output"
mock_data = {"apogee": 100, "max_velocity": 255}
with open(tmp_path / "mock_output.outputs.txt", "w") as f:
f.write(json.dumps(mock_data) + "\n")

mc.export_results(tmp_path / "mock_outputs_in_csv", "csv")
expected_file_in_csv = tmp_path / f"{"mock_outputs_in_csv"}.csv"
assert expected_file_in_csv.exists()

mc.export_results(tmp_path / "mock_output_in_json", "json")
expected_file_in_json = tmp_path / f"{"mock_output_in_json"}.json"
Copy link

Copilot AI Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary f-string usage. The expression f"{"mock_output_in_json"}" is redundant and can be simplified to just "mock_output_in_json".

Suggested change
expected_file_in_json = tmp_path / f"{"mock_output_in_json"}.json"
expected_file_in_json = tmp_path / "mock_output_in_json.json"

Copilot uses AI. Check for mistakes.
Comment on lines +208 to +212
Copy link

Copilot AI Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary f-string usage. The expression f"{"mock_outputs_in_csv"}" is redundant and can be simplified to just "mock_outputs_in_csv". The same applies to line 212.

Suggested change
expected_file_in_csv = tmp_path / f"{"mock_outputs_in_csv"}.csv"
assert expected_file_in_csv.exists()
mc.export_results(tmp_path / "mock_output_in_json", "json")
expected_file_in_json = tmp_path / f"{"mock_output_in_json"}.json"
expected_file_in_csv = tmp_path / "mock_outputs_in_csv.csv"
assert expected_file_in_csv.exists()
mc.export_results(tmp_path / "mock_output_in_json", "json")
expected_file_in_json = tmp_path / "mock_output_in_json.json"

Copilot uses AI. Check for mistakes.
assert expected_file_in_json.exists()
Comment on lines +207 to +213
Copy link

Copilot AI Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test only verifies that files are created but doesn't validate their contents. Consider adding assertions to:

  1. Verify the CSV/JSON files contain the expected mock data (apogee: 100, max_velocity: 255)
  2. Check CSV header format
  3. Verify JSON structure and indentation

Example:

# For CSV
import csv
with open(expected_file_in_csv, "r") as f:
    reader = csv.DictReader(f)
    rows = list(reader)
    assert len(rows) == 1
    assert rows[0]["apogee"] == "100"
    assert rows[0]["max_velocity"] == "255"

# For JSON  
with open(expected_file_in_json, "r") as f:
    data = json.load(f)
    assert len(data) == 1
    assert data[0]["apogee"] == 100
    assert data[0]["max_velocity"] == 255

Copilot uses AI. Check for mistakes.
finally:
os.remove("monte_carlo_test.errors.txt")
os.remove("monte_carlo_test.inputs.txt")
os.remove("monte_carlo_test.outputs.txt")
Comment on lines +192 to +217
Copy link

Copilot AI Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test doesn't cover important edge cases and error conditions:

  1. Invalid output_format parameter (e.g., "xml", "pdf")
  2. Non-existent .outputs.txt file
  3. Empty .outputs.txt file
  4. Malformed JSON in .outputs.txt file
  5. Case insensitivity of output_format (e.g., "CSV", "Json")

Consider adding separate test cases for these scenarios to ensure the method handles errors gracefully.

Copilot uses AI. Check for mistakes.
Comment on lines +215 to +217
Copy link

Copilot AI Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

File cleanup in the finally block will fail if the files don't exist. This can happen if the fixture fails to create these files or if the test is run in isolation. Use os.path.exists() checks or use a helper function similar to _post_test_file_cleanup() found in integration tests (see tests/integration/simulation/test_monte_carlo.py:12-25).

Suggested fix:

finally:
    for filepath in ["monte_carlo_test.errors.txt", "monte_carlo_test.inputs.txt", "monte_carlo_test.outputs.txt"]:
        if os.path.exists(filepath):
            os.remove(filepath)
Suggested change
os.remove("monte_carlo_test.errors.txt")
os.remove("monte_carlo_test.inputs.txt")
os.remove("monte_carlo_test.outputs.txt")
for filepath in [
"monte_carlo_test.errors.txt",
"monte_carlo_test.inputs.txt",
"monte_carlo_test.outputs.txt",
]:
if os.path.exists(filepath):
os.remove(filepath)

Copilot uses AI. Check for mistakes.