Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: improve downstream node performance #21

Merged
merged 40 commits into from
Feb 5, 2025
Merged
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
cbcad42
feat: improve downstream nodes performance with local search
jaapschoutenalliander Jan 23, 2025
3d72bab
test: add testing on sorting
jaapschoutenalliander Jan 24, 2025
da0c33d
chore: update performance test
jaapschoutenalliander Jan 28, 2025
6d791f5
Add .find_connected method and alter .get_downstream_nodes
Thijss Jan 29, 2025
fc99bc0
Update test_get_downstream_nodes
Thijss Jan 29, 2025
7cb6fa2
Update performance tests
Thijss Jan 29, 2025
098e3ac
Merge branch 'feat/update-performance-tests' into feature/downstream_…
Thijss Jan 29, 2025
5300faf
Merge remote-tracking branch 'origin/main' into feature/downstream_no…
Thijss Feb 3, 2025
3ccf1e9
rename to find_first_connected and update docstring
Thijss Feb 4, 2025
09595b9
Feature: add Grid.from_txt method
Thijss Feb 5, 2025
8441e89
support unordered branches
Thijss Feb 5, 2025
a1c17b5
update documentation
Thijss Feb 5, 2025
0dddcf2
update gitignore
Thijss Feb 5, 2025
960a09d
update documentation
Thijss Feb 5, 2025
ce2e0dd
fix test
Thijss Feb 5, 2025
6c39904
Merge branch 'feat/add-from-txt' into feat/improve-downstream-perform…
Thijss Feb 5, 2025
fa34b94
update downstream tests
Thijss Feb 5, 2025
5d541fb
ruff
Thijss Feb 5, 2025
a7bdd9d
switch to regex for better text support
Thijss Feb 5, 2025
4866e6b
add support for both list[str] and str
Thijss Feb 5, 2025
2860f04
add test for docstring
Thijss Feb 5, 2025
8124ef2
switch to args input
Thijss Feb 5, 2025
9d406ba
fix constants for graph performance tests (#28)
Thijss Feb 4, 2025
a6f0681
fix: delete_branch3 used wrong argument name (#29)
vincentkoppen Feb 4, 2025
6ead9c5
chore: remove unused/nonfunctional cache on graphcontainer (#30)
vincentkoppen Feb 4, 2025
5bcb179
Feature: add Grid.from_txt method
Thijss Feb 5, 2025
3f1df6a
support unordered branches
Thijss Feb 5, 2025
7693c0a
update documentation
Thijss Feb 5, 2025
a8a5c2c
update gitignore
Thijss Feb 5, 2025
3fe3a6a
update documentation
Thijss Feb 5, 2025
92eba6c
fix test
Thijss Feb 5, 2025
9dabf71
update downstream tests
Thijss Feb 5, 2025
afc8a17
merge
Thijss Feb 5, 2025
2b99ea3
add downstream test
Thijss Feb 5, 2025
b5713b0
add TestFindFirstConnected
Thijss Feb 5, 2025
84863dc
remove re module implementation
Thijss Feb 5, 2025
0018e2f
bump minor
Thijss Feb 5, 2025
82ec35e
Merge branch 'feat/add-from-txt' into feature/downstream_node_perform…
Thijss Feb 5, 2025
e7b9ab3
re-add grid logic from downstream nodes
Thijss Feb 5, 2025
e352085
merge main
Thijss Feb 5, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,6 @@ PYPI_VERSION
build/
wheelhouse/
_build/

# Jupyter Notebook
.ipynb_checkpoints
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.0
1.1
53 changes: 43 additions & 10 deletions docs/examples/utils/grid_from_txt_examples.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,22 @@
"- A _transformer_ is defined as `<from_node> <to_node> transformer`\n",
" - e.g.: `8 9 transformer`\n",
"- A _grid opening_ is defined by adding `open`\n",
" - e.g.: `4 5 open` for _lines_ or `6 7 transformer,open` for _transformers_\n",
"\n",
" - e.g.: `4 5 open` for _lines_ or `6 7 transformer,open` for _transformers_\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Loading a drawn grid into pgm-ds\n",
"\n",
"Once you've created a grid, copy the _Graph Data_ of your grid to a text file (e.g. `my_grid.txt`).\n",
"There are two ways of loading a text grid into a pgm-ds:\n",
"- load grid from a .txt file\n",
"- load grid from a list of strings\n",
"\n",
"For example, your file could contain the following data:\n",
"#### Load a grid from a .txt file\n",
"Copy the _Graph Data_ of your grid to a text file (e.g. `my_grid.txt`).\n",
"For the example above, the file should contain the following data:\n",
"\n",
"```text\n",
"S1 2\n",
Expand All @@ -52,7 +61,7 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -68,12 +77,36 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"**You should now have a grid loaded from your drawn graph data!**\n"
"\n",
"#### Load grid from a list of strings\n",
"You can also load a grid directly from a list of strings"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"from power_grid_model_ds import Grid\n",
"\n",
"grid = Grid.from_txt(\n",
" [\n",
" \"S1 2\",\n",
" \"S1 3 open\",\n",
" \"2 7\",\n",
" \"3 5\",\n",
" \"3 6 transformer\",\n",
" \"5 7\",\n",
" \"7 8\",\n",
" \"8 9\",\n",
" ]\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 10,
"metadata": {},
"outputs": [
{
Expand All @@ -99,7 +132,7 @@
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
Expand All @@ -113,9 +146,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
"version": "3.13.1"
}
},
"nbformat": 4,
"nbformat_minor": 2
"nbformat_minor": 4
}
44 changes: 44 additions & 0 deletions src/power_grid_model_ds/_core/model/graphs/models/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -235,8 +235,49 @@ def get_connected(
nodes_to_ignore=self._externals_to_internals(nodes_to_ignore),
inclusive=inclusive,
)

return self._internals_to_externals(nodes)

def find_first_connected(self, node_id: int, candidate_node_ids: list[int]) -> int:
"""Find the first connected node to the node_id from the candidate_node_ids

Note:
If multiple candidate nodes are connected to the node, the first one found is returned.
There is no guarantee that the same candidate node will be returned each time.

Raises:
MissingNodeError: if no connected node is found
ValueError: if the node_id is in candidate_node_ids
"""
internal_node_id = self.external_to_internal(node_id)
internal_candidates = self._externals_to_internals(candidate_node_ids)
if internal_node_id in internal_candidates:
raise ValueError("node_id cannot be in candidate_node_ids")
return self.internal_to_external(self._find_first_connected(internal_node_id, internal_candidates))

def get_downstream_nodes(self, node_id: int, start_node_ids: list[int], inclusive: bool = False) -> list[int]:
"""Find all nodes downstream of the node_id with respect to the start_node_ids

Example:
given this graph: [1] - [2] - [3] - [4]
>>> graph.get_downstream_nodes(2, [1]) == [3, 4]
>>> graph.get_downstream_nodes(2, [1], inclusive=True) == [2, 3, 4]

args:
node_id: node id to start the search from
start_node_ids: list of node ids considered 'above' the node_id
inclusive: whether to include the given node id in the result
returns:
list of node ids sorted by distance, downstream of to the node id
"""
connected_node = self.find_first_connected(node_id, start_node_ids)
path, _ = self.get_shortest_path(node_id, connected_node)
_, upstream_node, *_ = (
path # path is at least 2 elements long or find_first_connected would have raised an error
)

return self.get_connected(node_id, [upstream_node], inclusive)

def find_fundamental_cycles(self) -> list[list[int]]:
"""Find all fundamental cycles in the graph.
Returns:
Expand Down Expand Up @@ -273,6 +314,9 @@ def _branch_is_relevant(self, branch: BranchArray) -> bool:
@abstractmethod
def _get_connected(self, node_id: int, nodes_to_ignore: list[int], inclusive: bool = False) -> list[int]: ...

@abstractmethod
def _find_first_connected(self, node_id: int, candidate_node_ids: list[int]) -> int: ...

@abstractmethod
def _has_branch(self, from_node_id, to_node_id) -> bool: ...

Expand Down
22 changes: 21 additions & 1 deletion src/power_grid_model_ds/_core/model/graphs/models/rustworkx.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

import rustworkx as rx
from rustworkx import NoEdgeBetweenNodes
from rustworkx.visit import BFSVisitor, PruneSearch
from rustworkx.visit import BFSVisitor, PruneSearch, StopSearch

from power_grid_model_ds._core.model.graphs.errors import MissingBranchError, MissingNodeError, NoPathBetweenNodes
from power_grid_model_ds._core.model.graphs.models._rustworkx_search import find_fundamental_cycles_rustworkx
Expand Down Expand Up @@ -99,6 +99,13 @@ def _get_connected(self, node_id: int, nodes_to_ignore: list[int], inclusive: bo

return connected_nodes

def _find_first_connected(self, node_id: int, candidate_node_ids: list[int]) -> int:
visitor = _NodeFinder(candidate_nodes=candidate_node_ids)
rx.bfs_search(self._graph, [node_id], visitor)
if visitor.found_node is None:
raise MissingNodeError(f"node {node_id} is not connected to any of the candidate nodes")
return visitor.found_node

def _find_fundamental_cycles(self) -> list[list[int]]:
"""Find all fundamental cycles in the graph using Rustworkx.

Expand All @@ -117,3 +124,16 @@ def discover_vertex(self, v):
if v in self.nodes_to_ignore:
raise PruneSearch
self.nodes.append(v)


class _NodeFinder(BFSVisitor):
"""Visitor that stops the search when a candidate node is found"""

def __init__(self, candidate_nodes: list[int]):
self.candidate_nodes = candidate_nodes
self.found_node: int | None = None

def discover_vertex(self, v):
if v in self.candidate_nodes:
self.found_node = v
raise StopSearch
20 changes: 9 additions & 11 deletions src/power_grid_model_ds/_core/model/grids/_text_sources.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
"""Create a grid from text a text file"""

import logging
from pathlib import Path
from typing import TYPE_CHECKING

from power_grid_model_ds._core.model.enums.nodes import NodeType
Expand Down Expand Up @@ -35,29 +34,28 @@ class TextSource:
def __init__(self, grid_class: type["Grid"]):
self.grid = grid_class.empty()

def load_grid_from_path(self, path: Path):
"""Load assets from text file & sort them by id so that
they are ready to be appended to the grid"""
def load_from_txt(self, *args: str) -> "Grid":
"""Load a grid from text"""

txt_nodes, txt_branches = self.read_txt(path)
text_lines = [line for arg in args for line in arg.strip().split("\n")]

txt_nodes, txt_branches = self.read_txt(text_lines)
self.add_nodes(txt_nodes)
self.add_branches(txt_branches)
self.grid.set_feeder_ids()
return self.grid

@staticmethod
def read_txt(path: Path) -> tuple[set, dict]:
def read_txt(txt_lines: list[str]) -> tuple[set, dict]:
"""Extract assets from text"""
with open(path, "r", encoding="utf-8") as f:
txt_rows = f.readlines()

txt_nodes = set()
txt_branches = {}
for text_line in txt_rows:
for text_line in txt_lines:
if not text_line.strip() or text_line.startswith("#"):
continue # skip empty lines and comments
try:
from_node_str, to_node_str, *comments = text_line.strip().split(" ")
from_node_str, to_node_str, *comments = text_line.strip().split()
except ValueError as err:
raise ValueError(f"Text line '{text_line}' is invalid. Skipping...") from err
comments = comments[0].split(",") if comments else []
Expand Down Expand Up @@ -116,4 +114,4 @@ def add_branch(self, branch: tuple[str, str], comments: list[str]):
new_branch.to_status = 0
else:
new_branch.to_status = 1
self.grid.append(new_branch)
self.grid.append(new_branch, check_max_id=False)
20 changes: 18 additions & 2 deletions src/power_grid_model_ds/_core/model/grids/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -408,6 +408,21 @@ def _from_pickle(cls, pickle_path: Path):
raise TypeError(f"{pickle_path.name} is not a valid {cls.__name__} cache.")
return grid

@classmethod
def from_txt(cls, *args: str):
"""Build a grid from a list of strings

See the documentation for the expected format of the txt_lines

Args:
*args (str): The lines of the grid

Examples:
>>> Grid.from_txt("1 2", "2 3", "3 4 transformer", "4 5", "S1 6")
alternative: Grid.from_txt("1 2\n2 3\n3 4 transformer\n4 5\nS1 6")
"""
return TextSource(grid_class=cls).load_from_txt(*args)

@classmethod
# pylint: disable=arguments-differ
def from_txt_file(cls, txt_file_path: Path):
Expand All @@ -416,8 +431,9 @@ def from_txt_file(cls, txt_file_path: Path):
Args:
txt_file_path (Path): The path to the txt file
"""
text_source = TextSource(grid_class=cls)
return text_source.load_grid_from_path(txt_file_path)
with open(txt_file_path, "r", encoding="utf-8") as f:
txt_lines = f.readlines()
return TextSource(grid_class=cls).load_from_txt(*txt_lines)

def set_feeder_ids(self):
"""Sets feeder and substation id properties in the grids arrays"""
Expand Down
17 changes: 17 additions & 0 deletions tests/unit/model/graphs/test_graph_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -320,3 +320,20 @@ def test_get_connected_ignore_multiple_nodes(self, graph_with_2_routes):
connected_nodes = graph.get_connected(node_id=1, nodes_to_ignore=[2, 4])

assert {5} == set(connected_nodes)


class TestFindFirstConnected:
def test_find_first_connected(self, graph_with_2_routes):
graph = graph_with_2_routes
assert 2 == graph.find_first_connected(1, candidate_node_ids=[2, 3, 4])

def test_find_first_connected_same_node(self, graph_with_2_routes):
graph = graph_with_2_routes
with pytest.raises(ValueError):
graph.find_first_connected(1, candidate_node_ids=[1, 3, 5])

def test_find_first_connected_no_match(self, graph_with_2_routes):
graph = graph_with_2_routes
graph.add_node(99)
with pytest.raises(MissingNodeError):
graph.find_first_connected(1, candidate_node_ids=[99])
Loading