Skip to content

Commit

Permalink
Merge pull request #239 from PowerGridModel/fix/General-retouching-on…
Browse files Browse the repository at this point in the history
…-the-documentation

Retouching on the converter documentations.
  • Loading branch information
Jerry-Jinfeng-Guo authored Mar 21, 2024
2 parents dee892a + fec8629 commit 9f20d1c
Show file tree
Hide file tree
Showing 5 changed files with 74 additions and 74 deletions.
37 changes: 21 additions & 16 deletions docs/converters/converter.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,49 +6,52 @@ SPDX-License-Identifier: MPL-2.0

# Converters

There are 4 types of converters present as of now.
Here, we shall discuss their basic structure and guidelines for building a custom converter.
We currently provide 4 types of converters.
Here, we discuss their basic structure and guidelines for building a custom converter.

Use the examples notebooks to understand how to convert data from the respective formats.

- **PGM JSON Converter:** Refer to the [PGM JSON Example](../examples/pgm_json_example.ipynb)
- **VisonExcelConverter** Refer to the [Vision Example](../examples/vision_example.ipynb)
- **Pandapower Converter:** Converts [pandapower network](https://pandapower.readthedocs.io/en/stable/elements.html), which is a dictionary of dataframes, to power-grid-model data.
- **Pandapower Converter:** Converts [pandapower network](https://pandapower.readthedocs.io/en/stable/elements.html) (a dictionary of dataframes) to power-grid-model data.

Refer to [converters](../power_grid_model_io.md#converters) in API documentation for more details

## Structure

The `VisonExcelConverter` extends the [tabular converters](tabular_converter.md) for Excel exports of Vision.
All converters are derived from the base {py:class}`power_grid_model_io.converters.base_converter`.
The usable functions for loading, saving and converting the data are located in the base class.
The private functions (`_load_data`, `_parse_data` and `_serialize_data`) are overloaded based on the specific type of converter (ie. excel, json or pandapower).
It is recommended to create any custom converter in a similar way.
The private functions (`_load_data`, `_parse_data` and `_serialize_data`) are overloaded based on the specific type of converter (i.e., excel, json or pandapower).
The `VisonExcelConverter` extends the [tabular converters](tabular_converter.md) for Excel exports of Vision.
It is recommended that custom converters be created in a similar way.

## Instantiation

A converter object can be instantiated in the following way. For eg, for a `PgmJsonConverter`,
Converter objects can be instantiated in the following way. For instance, for a `PgmJsonConverter`:

```python
from power_grid_model_io.converters.pgm_json_converter import PgmJsonConverter

converter = PgmJsonConverter(source_file=source, destination_file=destination)
```

The usable methods of converters for loading and saving the data are described below.
## Common methods for converters

The common methods across different converters, including data IO and log level configurations, are described below.
Converter specific methods are presented in corresponding sections.

## Loading data
### Loading data

Use the methods load_input_data(), load_update_data(), load_sym_output_data() or load_asym_output_data() to load the relevant data to the converter.
The Converter can be initialised with `source_file` containing path to source data. Or alternatively, the data can be provided as an argument to the load function.
Methods load_input_data(), load_update_data(), load_sym_output_data() and load_asym_output_data() are for loading the relevant data to the converter.
Input data can be configured at converter initialisation by passing the path to source data to parameter `source_file`. Alternatively, the data can also be provided as an argument to the load function after initialisation.

In addition to the power-grid-model input data, other miscellaneous information in the source file not used in calculations by power-grid-model gets stored under `extra_info`
In addition to the input data that will be used in calculations by power-grid-model, other miscellaneous information in the source file is stored in `extra_info`:

```python
input_data, extra_info = converter.load_input_data(data=example_data)
```

## Saving Data
### Saving data

It is possible to save the data in the format of the converter.
The Converter can be instantiated with a path given to `destination_file`.
Expand All @@ -59,9 +62,10 @@ You can also add additional information about each component in the form of `ext
converter.save(example_data, extra_info=example_extra_info, destination=destination_path)
```

## Configuring The Log Output
### Configuring the log output

We have provided an interface to configure the log level of the converter.
An interface is provided to configure the log level of converters.
This configuration can be done at converter initialisation or after.
Notice that this log level only belongs to the logger within the converter.
Users need to set their basic configuration of the `logging` module to a level that is below what is configuired for the converters.

Expand All @@ -78,5 +82,6 @@ converter_info = VisionExcelConverter(input_file) # Uses default INFO level
logging.basicConfig(level=logging.WARNING) # Only levels WARNING and above will be logged
assert converter_info.get_log_level() == logging.INFO # Previously created converters will retain their original log level, regardless of system wide configuration

converter_debug = VisionExcelConverter(input_file, log_level=logging.DEBUG) # Any logs on DEBUG and INFO level will not be logged
converter_ = VisionExcelConverter(input_file, log_level=logging.DEBUG) # Any logs on DEBUG and INFO level will not be logged
converter_.set_log_level(logging.WARNING) # Now the converter's log level is set to WARNING
```
10 changes: 5 additions & 5 deletions docs/converters/pandapower_converter.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@ If a `vector_group` is not available for transformer then a default is set in co
Similarly for three winding transformer, `YNynyn` is set for even clocks of `shift_mv_degree` and `shift_lv_degree`.
If the clocks are odd, then the vector group is converted as `YNynd`, `YNdyn` or `YNdd`.

## Modelling differences
## Modeling differences

The user must be aware of following unsupported features or differences in conversion.

* Currently, the conversions only support powerflow calculations and their relevant attributes.
* Any feature involving a PV bus, ie. generator, DC line are unsupported as of now.
* Any feature involving a PV bus, i.e., generator, DC line are unsupported as of now.

In addition, the following component-specific modelling differences exist.
In addition, the following component-specific modeling differences exist.

### Load

Expand All @@ -41,7 +41,7 @@ The features regarding `in_ka` and `z_ohm` attributes are currently unsupported.

### External grid

The external grid is modelled with a source impedance all sequence components in power-grid-model.
The external grid is modeled with a source impedance all sequence components in power-grid-model.
Whereas it is only present in negative and zero sequence networks in pandapower.
Hence, this impedance value has to be mentioned in the power-flow calculation when using the power-grid-model power flow calculation.

Expand All @@ -57,4 +57,4 @@ The default for transformer model to be used in pandapower is `t` model but powe
### Three winding transformer

The differences defined in [Transformer](#transformer) are applicable here as well.
Additionally, tap connection at star point, ie. `tap_at_star_point` is not supported.
Additionally, tap connection at star point, i.e., `tap_at_star_point` is not supported.
59 changes: 27 additions & 32 deletions docs/converters/tabular_converter.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,13 @@ SPDX-FileCopyrightText: Contributors to the Power Grid Model project <powergridm
SPDX-License-Identifier: MPL-2.0
-->
# Tabular converter
Tabular data can come from Excel files, a set of CSV files, GNF files, databases, pandas DataFrames, etc etc.
The similarity between all tabular data is that it contains multiple `tables`,
each with multiple `columns`, possibly with a specific `unit`.
Others may have a categorical value which needs to be mapped (i.e. open: 0, closed: 1); in general we'll call these
`substitutions`.
Tabular data is commonly stored in spreadsheet files: Excel files, CSV files, GNF files, databases, pandas DataFrames, etc.
The similarity between all tabular data is that it contains multiple `tables`, each with multiple `columns`, possibly with a specific `unit` row.
Others may have categorical values that need to be further mapped (i.e., open: 0, closed: 1).
These attributes are referred to as `substitutions`.

## Mapping file
A mapping file is a yaml with three main sections `grid`, `units` and `substitutions`:
A mapping file is a yaml file with three main sections `grid`, `units` and `substitutions`:
```yaml
grid:

Expand Down Expand Up @@ -53,13 +52,12 @@ substitutions:
```
## Grid
For each `table`, the target PGM `component` is listed (e.g. Nodes: node, Cables: line).
The for each PGM `column` the source column is supplied (e.g. u_rated: Unom, from_status: From.SwitchStatus).
For each `table`, the target PGM `component` is listed (e.g., Nodes: node, Cables: line).
The for each PGM `column` the source column is supplied (e.g., u_rated: Unom, from_status: From.SwitchStatus).

## Field Definitions
If the `column` definition is a one on one mapping,
the value is simply the name of the source column (e.g.u_rated: Unom),
but in many cases the mapping is a bit more complex.
If the `column` definition is a one-on-one mapping, the value is simply the name of the source column (e.g., u_rated: Unom).
In many other cases, however, mappings can be a bit more complex.
You can use the following `column` definitions:

* Column name `str`
Expand Down Expand Up @@ -93,7 +91,7 @@ You can use the following `column` definitions:
tan1: 0.0
```
* Pandas DataFrame functions `Dict[str, List[Any]]`
(`prod`, `sum`, `min`, `max`, etc and the alias `multiply` which translates to `prod`)
(`prod`, `sum`, `min`, `max`, etc. and the alias `multiply` which translates to `prod`)
```yaml
p_specified:
min:
Expand Down Expand Up @@ -130,15 +128,12 @@ You can use the following `column` definitions:
)
```
## Units
Power Grid Model uses SI units (e.g. "W" for Watts),
but source data may be supplied in different units (e.g. "MW" for Mega Watts).
If units are supplied in the tabular data,
the pandas DataFrame storing the data is expected to have `MultiIndexes` for the columns.
For our application, a `MultiIndex` can be interpreted as a tuple; the first element is the column name, the second
element is the column unit. For example: `("C0", "µF")`.
Power Grid Model uses SI units (e.g., "W" for Watts), but source data may be supplied in different units (e.g., "MW" for Mega Watts).
If units are supplied in the tabular data, the data stored using pandas DataFrame is expected to have `MultiIndexes` for columns.
For our application, a `MultiIndex` can be interpreted as a tuple; the first element is the column name, the second element is the column unit. For example: `("C0", "µF")`.

If a unit is supplied, it should be defined in the units section of the mapping.
Undefined units are not allowed to prevent errors.
If an optional unit is supplied, it should be defined in the unit section of the mapping.
Undefined units that do not exist in mapping are not allowed.

```yaml
units:
Expand All @@ -156,7 +151,7 @@ The definitions above can be interpreted as:
* 1 **ohm/km** = 0.001 **ohm/m**

## Substitutions
Some columns may contain categorical values (enums) which should be replaced. The column names can be defined as
Some columns may contain categorical values (enums) that should be mapped. The column names can be defined as
regular expressions.
```yaml
substitutions:
Expand All @@ -169,16 +164,16 @@ substitutions:
own: true
```
The definitions above can be interpreted as:
* In all columns that end with `SwitchState` (e.g. `From.Switch State`, `To.Switch State` or just `Switch State`),
the word "off" should be replaced with the integer 0 and the word "in" should be replaced with the value 1.
* In all columns that end with `SwitchState` (e.g., `From.Switch State`, `To.Switch State` or just `Switch State`),
the word "off" should be replaced with integer value 0, and the word "in" integer value 1.
* In all columns called "N1",
the word "none" should be replaced with the boolean `false`
and the word "own" should be replaced with the value boolean `true`.
the word "none" should be replaced with the boolean value `false`,
and the word "own" boolean value `true`.

## AutoID
The `id` field is special in the sense that each object should have a unique numerical id in power grid model.
Therefore, each id definition is mapped to a numerical ID.
Also each field name that ends with `node` is converted into the matching numerical ID.
Therefore, each id definition is mapped to a numerical (integer) ID.
Field names that end with `node` are also mapped to corresponding numerical IDs.

```python
from power_grid_model_io.utils.auto_id import AutoID
Expand All @@ -193,8 +188,8 @@ item = auto_id[1] # item = "Bravo"
See also {py:class}`power_grid_model_io.utils.AutoID`

## AutoID Mapping
Let's consider a very common example of the usage of `auto_id` in a mapping file.
(Note that we're focussing on the ids and references, so the other attributes have been disregarded.)
Let's consider a very common example of the usage of `auto_id` in a mapping file
(note that we're focussing on the ids and references, other attributes are therefore omitted).
```yaml
Nodes:
node:
Expand All @@ -217,7 +212,7 @@ Let's consider a very common example of the usage of `auto_id` in a mapping file
key:
Number: To_Number
```
This basically reads as:
This basically reads:
* For each row in the Nodes table, a PGM node instance is created.
* For each node instance, a numerical id is generated, which is unique for each value in the Number column. This
assumes that the Number column is unique in the source table. Let's say tha values of the Number column in that
Expand Down Expand Up @@ -247,7 +242,7 @@ This basically reads as:

## Advanced AutoID Mapping
In some cases, multiple components have to be created for each row in a source table.
In such cases, the `name` attribute may be necessary to create multiple PGM IDs for a single row. Let's consider
In such cases, the `name` attribute may be necessary to create multiple PGM IDs for a single row. Consider
this example:
```yaml
Transformer loads:
Expand Down Expand Up @@ -291,7 +286,7 @@ this example:
- Subnumber
```

Let's say we have one Transformer Load connected to the Node Number 103 and it's Subnumber is 1.
Suppose we have one Transformer Load connected to the Node Number 103 and its Subnumber is 1.
Then the following IDs will be generated / retrieved:
* `transformer.id`:
`{"table": "Transformer loads", "name": "transformer", "key" {"Node_Number": 103, "Subnumber": 1} -> 5`
Expand Down
Loading

0 comments on commit 9f20d1c

Please sign in to comment.