Skip to content

Commit 9f428b3

Browse files
authored
Merge pull request #50 from alliander-opensource/feature/converters-doc
markdown for converters
2 parents 8d19666 + 50809d0 commit 9f428b3

17 files changed

+391
-73
lines changed

docs/conf.py

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -24,14 +24,9 @@
2424
"sphinx.ext.intersphinx",
2525
"numpydoc",
2626
"hoverxref.extension",
27-
"myst_parser",
27+
"myst_nb",
2828
]
2929

30-
source_suffix = {
31-
".rst": "restructuredtext",
32-
".md": "markdown",
33-
}
34-
3530
templates_path = ["_templates"]
3631
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
3732

@@ -71,3 +66,8 @@
7166
# -- sphinx.ext.intersphinx config -------------------------------------------
7267
# For linking to power-grid-model's documentation.
7368
intersphinx_mapping = {"power-grid-model": ("https://power-grid-model.readthedocs.io/en/stable/", None)}
69+
70+
71+
# -- Temporary config -----------
72+
# Disable notebook executions in examples directory
73+
nb_execution_excludepatterns = ["examples/*.ipynb"]

docs/converters/converter.md

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
<!--
2+
SPDX-FileCopyrightText: 2022 Contributors to the Power Grid Model IO project <[email protected]>
3+
4+
SPDX-License-Identifier: MPL-2.0
5+
-->
6+
7+
# Converters
8+
9+
There are 4 types of converters present as of now.
10+
Here, we shall discuss their basic structure and guidelines for building a custom converter.
11+
12+
Use the examples notebooks to understand how to convert data from the respective formats.
13+
14+
- **PGM JSON Converter:** Refer to the [PGM JSON Example](../examples/pgm_json_example.ipynb)
15+
- **VisonExcelConverter** and **GaiaExcelConverter:** Refer to the [Vision and Gaia Example](../examples/vision_gaia_example.ipynb)
16+
- **Pandapower Converter:** Converts [pandapower network](https://pandapower.readthedocs.io/en/stable/elements.html), which is a dictionary of dataframes, to power-grid-model data.
17+
18+
Refer to [converters](../power_grid_model_io.md#converters) in API documentation for more details
19+
20+
## Structure
21+
22+
`VisonExcelConverter` and `GaiaExcelConverter` are inherited from [tabular converters](tabular_converter.md) for excel exports of vision and gaia respectively.
23+
All 4 converters are derived from the base {py:class}`power_grid_model_io.converters.base_converter`.
24+
The usable functions for loading, saving and converting the data are located in the base class.
25+
The private functions (`_load_data`, `_parse_data` and `_serialize_data`) are overloaded based on the specific type of converter (ie. excel, json or pandapower).
26+
It is recommended to create any custom converter in a similar way.
27+
28+
## Instantiation
29+
30+
A converter object can be instantiated in the following way. For eg, for a `PgmJsonConverter`,
31+
32+
```python
33+
from power_grid_model_io.converters.pgm_json_converter import PgmJsonConverter
34+
35+
converter = PgmJsonConverter(source_file=source, destination_file=destination)
36+
```
37+
38+
The usable methods of converters for loading and saving the data are described below.
39+
40+
## Loading data
41+
42+
Use the methods load_input_data(), load_update_data(), load_sym_output_data() or load_asym_output_data() to load the relevant data to the converter.
43+
The Converter can be initialised with `source_file` containing path to source data. Or alternatively, the data can be provided as an argument to the load function.
44+
45+
In addition to the power-grid-model input data, other miscellaneous information in the source file not used in calculations by power-grid-model gets stored under `extra_info`
46+
47+
```python
48+
input_data, extra_info = converter.load_input_data(data=example_data)
49+
```
50+
51+
## Saving Data
52+
53+
It is possible to save the data in the format of the converter.
54+
The Converter can be instantiated with a path given to `destination_file`.
55+
Alternatively, the destination path can be provided in the save function.
56+
You can also add additional information about each component in the form of `extra_info` generated by [Load data](converter.md#load-data) to be saved along with it.
57+
58+
```python
59+
converter.save(example_data, extra_info=example_extra_info, destination=destination_path)
60+
```

docs/converters/tabular_converter.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ each with multiple `columns`, possibly with a specific `unit`.
1010
Others may have a categorical value which needs to be mapped (i.e. open: 0, closed: 1); in general we'll call these
1111
`substitutions`.
1212

13-
# Mapping file
13+
## Mapping file
1414
A mapping file is a yaml with three main sections `grid`, `units` and `substitutions`:
1515
```yaml
1616
grid:
@@ -146,7 +146,7 @@ The definitions above can be interpreted as:
146146
the word "none" should be replaced with the boolean `false`
147147
and the word "own" should be replaced with the value boolean `true`.
148148

149-
# AutoID
149+
## AutoID
150150
The `id` field is special in the sense that each object should have a unique numerical id in power grid model.
151151
Therefore, each id definition is mapped to a numerical ID.
152152
Also each field name that ends with `node` is converted into the matching numerical ID.
@@ -161,9 +161,9 @@ c = auto_id("Alpha") # c = 0 (because key "Alpha" already existed)
161161
item = auto_id[1] # item = "Bravo"
162162
```
163163

164-
See also [AutoID](/src/power_grid_model_io/utils/auto_id.py).
164+
See also {py:class}`power_grid_model_io.utils.AutoID`
165165

166-
## Vision and Gaia
166+
### Vision and Gaia
167167
For Vision and Gaia files, an extra trick is applied. Let's assume this mapping:
168168

169169
```yaml

docs/examples/pgm_json_example.ipynb

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# PGM JSON Converter\n",
8+
"\n",
9+
"Converts to and from the power-grid-model JSON data format. \n",
10+
"This json format makes it convenient to evaluate the grid data in perspective of input to power-grid-model. \n",
11+
"More details about the JSON format are mentioned in `power_grid_model.utils` and example of it is in [Make Test Dataset](power-grid-model:examples/Make%20Test%20Dataset.ipynb) in power-grid-model repository.\n",
12+
"\n",
13+
"Basic imports and configuration:"
14+
]
15+
},
16+
{
17+
"cell_type": "code",
18+
"execution_count": 1,
19+
"metadata": {},
20+
"outputs": [],
21+
"source": [
22+
"import logging\n",
23+
"from pathlib import Path\n",
24+
"\n",
25+
"import structlog\n",
26+
"from power_grid_model import PowerGridModel\n",
27+
"\n",
28+
"from power_grid_model_io.converters.pgm_json_converter import PgmJsonConverter\n",
29+
"\n",
30+
"DATA_DIR = Path(\".\") / \"data\""
31+
]
32+
},
33+
{
34+
"cell_type": "markdown",
35+
"metadata": {},
36+
"source": [
37+
"Define source and destination paths"
38+
]
39+
},
40+
{
41+
"cell_type": "code",
42+
"execution_count": 2,
43+
"metadata": {},
44+
"outputs": [],
45+
"source": [
46+
"input_file = DATA_DIR / \"tiny-net\" / \"input.json\"\n",
47+
"output_file = DATA_DIR / \"tiny-net\" / \"sym_output.json\""
48+
]
49+
},
50+
{
51+
"cell_type": "markdown",
52+
"metadata": {},
53+
"source": [
54+
"Instantiate the converter, optionally with source and destination file paths. Use `load_input_data()` to convert to power-grid-model data. Additional information is stored in `extra_info`. Run powerflow calculation with the `input_data`."
55+
]
56+
},
57+
{
58+
"cell_type": "code",
59+
"execution_count": 3,
60+
"metadata": {},
61+
"outputs": [
62+
{
63+
"name": "stdout",
64+
"output_type": "stream",
65+
"text": [
66+
"2022-11-01 16:08.58 [debug ] Loading PGM input data\n"
67+
]
68+
}
69+
],
70+
"source": [
71+
"converter = PgmJsonConverter(source_file=input_file, destination_file=output_file)\n",
72+
"input_data, extra_info = converter.load_input_data()\n",
73+
"pgm = PowerGridModel(input_data=input_data)\n",
74+
"output_data = pgm.calculate_state_estimation()"
75+
]
76+
},
77+
{
78+
"cell_type": "markdown",
79+
"metadata": {},
80+
"source": [
81+
"The converted data or loaded data can be saved. The file will be saved in `output_file` path or the destination path can be provided here optionally."
82+
]
83+
},
84+
{
85+
"cell_type": "code",
86+
"execution_count": 4,
87+
"metadata": {},
88+
"outputs": [],
89+
"source": [
90+
"converter.save(data=output_data, extra_info=extra_info)"
91+
]
92+
}
93+
],
94+
"metadata": {
95+
"kernelspec": {
96+
"display_name": "Python 3 (ipykernel)",
97+
"language": "python",
98+
"name": "python3"
99+
},
100+
"language_info": {
101+
"codemirror_mode": {
102+
"name": "ipython",
103+
"version": 3
104+
},
105+
"file_extension": ".py",
106+
"mimetype": "text/x-python",
107+
"name": "python",
108+
"nbconvert_exporter": "python",
109+
"pygments_lexer": "ipython3",
110+
"version": "3.9.13"
111+
}
112+
},
113+
"nbformat": 4,
114+
"nbformat_minor": 2
115+
}
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
SPDX-FileCopyrightText: 2022 Contributors to the Power Grid Model project <[email protected]>
2+
3+
SPDX-License-Identifier: MPL-2.0

0 commit comments

Comments
 (0)