Merge pull request #236 from neo4j/rework-integrations-api

Slim down integrations api for pandas and gds

Author: FlorentinD

changelog.md

@@ -3,20 +3,29 @@
 ## Breaking changes
 
 - Do not automatically derive size and caption for `from_neo4j` and `from_gql_create`. Use the `size_property` and `node_caption` parameters to explicitly configure them.
+- Change API of integrations to only provide basic parameters. Any further configuration should happen ons the Visualization Graph object:
+  - `from_gds`
+    - Drop parameters size_property, node_radius_min_max. `Use VG.resize_nodes(property=...)` instead
+    - rename additional_node_properties to node_properties
+    - Don't derive fields from properties. Use `VG.map_properties_to_fields` instead
+  - `from_pandas`
+    - Drop `node_radius_min_max` parameter. `VG.resize_nodes(...)` instead
 
 ## New features
 
-- Allow to include db node properties in addition to the properties in the GDS Graph. Specify `additional_db_node_properties` in `from_gds`.
-
+- Allow to include db node properties in addition to the properties in the GDS Graph. Specify `db_node_properties` in `from_gds`.
 
 ## Bug fixes
 
 - fixed a bug in `from_neo4j`, where the node size would always be set to the `size` property.
 - fixed a bug in `from_neo4j`, where the node caption would always be set to the `caption` property.
+- Color nodes in `from_snowflake` only if there are less than 13 node tables used. This avoids reuse of colors for different tables.
 
 ## Improvements
 
 - Validate fields of a node and relationship not only at construction but also on assignment.
 - Allow resizing per node property such as `VG.resize_nodes(property="score")`.
+- Color nodes by label in `from_gds`.
+- Add `table` property to nodes and relationships created by `from_snowflake`. This is used as a default caption.
 
 ## Other changes

docs/source/conf.py

@@ -33,7 +33,10 @@
 
 # -- Options for autodoc extension -------------------------------------------
 autodoc_typehints = "description"
+autoclass_content = "both"
 
+# -- Options for napoleon extension -------------------------------------------
+napoleon_use_admonition_for_examples = True
 
 # -- Options for HTML output -------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output

docs/source/integration.rst

@@ -35,28 +35,17 @@ The ``from_dfs`` method takes two mandatory positional parameters:
 * A Pandas ``DataFrame``, or iterable (eg. list) of DataFrames representing the nodes of the graph.
   The rows of the DataFrame(s) should represent the individual nodes, and the columns should represent the node
   IDs and attributes.
-  If a column shares the name with a field of :doc:`Node <./api-reference/node>`, the values it contains will be set
-  on corresponding nodes under that field name.
-  Otherwise, the column name will be a key in each node's `properties` dictionary, that maps to the node's corresponding
+  The node ID will be set on the :doc:`Node <./api-reference/node>`,
+  Other columns will be a key in each node's `properties` dictionary, that maps to the node's corresponding
   value in the column.
   If the graph has no node properties, the nodes can be derived from the relationships DataFrame alone.
 * A Pandas ``DataFrame``, or iterable (eg. list) of DataFrames representing the relationships of the graph.
   The rows of the DataFrame(s) should represent the individual relationships, and the columns should represent the
   relationship IDs and attributes.
-  If a column shares the name with a field of :doc:`Relationship <./api-reference/relationship>`, the values it contains
-  will be set on corresponding relationships under that field name.
-  Otherwise, the column name will be a key in each node's `properties` dictionary, that maps to the node's corresponding
+  The relationship id, source and target node IDs will be set on the :doc:`Relationship <./api-reference/relationship>`.
+  Other columns will be a key in each relationship's `properties` dictionary, that maps to the relationship's corresponding
   value in the column.
 
-``from_dfs`` also takes an optional property, ``node_radius_min_max``, that can be used (and is used by default) to
-scale the node sizes for the visualization.
-It is a tuple of two numbers, representing the radii (sizes) in pixels of the smallest and largest nodes respectively in
-the visualization.
-The node sizes will be scaled such that the smallest node will have the size of the first value, and the largest node
-will have the size of the second value.
-The other nodes will be scaled linearly between these two values according to their relative size.
-This can be useful if node sizes vary a lot, or are all very small or very big.
-
 
 Example
 ~~~~~~~
@@ -111,33 +100,21 @@ If you want to have more control of the sampling, such as choosing a specific st
 a `sampling <https://neo4j.com/docs/graph-data-science/current/management-ops/graph-creation/sampling/>`_
 method yourself and passing the resulting projection to ``from_gds``.
 
-We can also provide an optional ``size_property`` parameter, which should refer to a node property of the projection,
-and will be used to determine the sizes of the nodes in the visualization.
-
-The ``additional_node_properties`` parameter is also optional, and should be a list of additional node properties of the
+The ``node_properties`` parameter is also optional, and should be a list of additional node properties of the
 projection that you want to include in the visualization.
 The default is ``None``, which means that all properties of the nodes in the projection will be included.
 Apart from being visible through on-hover tooltips, these properties could be used to color the nodes, or give captions
 to them in the visualization, or simply included in the nodes' ``Node.properties`` maps without directly impacting the
 visualization.
-If you want to include node properties stored at the Neo4j database, you can include them in the visualization by using the `additional_db_node_properties` parameter.
-
-The last optional property, ``node_radius_min_max``, can be used (and is used by default) to scale the node sizes for
-the visualization.
-It is a tuple of two numbers, representing the radii (sizes) in pixels of the smallest and largest nodes respectively in
-the visualization.
-The node sizes will be scaled such that the smallest node will have the size of the first value, and the largest node
-will have the size of the second value.
-The other nodes will be scaled linearly between these two values according to their relative size.
-This can be useful if node sizes vary a lot, or are all very small or very big.
+If you want to include node properties stored at the Neo4j database, you can include them in the visualization by using the `db_node_properties` parameter.
 
 
 Example
 ~~~~~~~
 
 In this small example, we import a graph projection from the GDS library, that has the node properties "pagerank" and
 "componentId".
-We use the "pagerank" property to determine the size of the nodes, and the "componentId" property to color the nodes.
+We use the "pagerank" property to compute the size of the nodes, and the "componentId" property to color the nodes.
 
 .. code-block:: python
 
@@ -156,9 +133,10 @@ We use the "pagerank" property to determine the size of the nodes, and the "comp
     VG = from_gds(
         gds,
         G,
-        size_property="pagerank",
-        additional_node_properties=["componentId"],
+        node_properties=["componentId"],
     )
+    # Size the nodes by the `pagerank` property
+    VG.resize_nodes(property="pagerank")
 
     # Color the nodes by the `componentId` property, so that the nodes are
     # colored by the connected component they belong to

python-wrapper/src/neo4j_viz/gds.py

@@ -8,6 +8,8 @@
 import pandas as pd
 from graphdatascience import Graph, GraphDataScience
 
+from neo4j_viz.colors import NEO4J_COLORS_DISCRETE, ColorSpace
+
 from .pandas import _from_dfs
 from .visualization_graph import VisualizationGraph
 
@@ -55,18 +57,20 @@ def _fetch_rel_dfs(gds: GraphDataScience, G: Graph) -> list[pd.DataFrame]:
 def from_gds(
     gds: GraphDataScience,
     G: Graph,
-    size_property: Optional[str] = None,
-    additional_node_properties: Optional[list[str]] = None,
-    additional_db_node_properties: Optional[list[str]] = None,
-    node_radius_min_max: Optional[tuple[float, float]] = (3, 60),
+    node_properties: Optional[list[str]] = None,
+    db_node_properties: Optional[list[str]] = None,
     max_node_count: int = 10_000,
 ) -> VisualizationGraph:
     """
     Create a VisualizationGraph from a GraphDataScience object and a Graph object.
 
-    All `additional_node_properties` will be included in the visualization graph.
-    If the properties are named as the fields of the `Node` class, they will be included as top level fields of the
-    created `Node` objects. Otherwise, they will be included in the `properties` dictionary.
+    By default:
+
+    * the caption of a node will be based on its `labels`.
+    * the caption of a relationship will be based on its `relationshipType`.
+    * the color of nodes will be set based on their label, unless there are more than 12 unique labels.
+
+    All `node_properties` and `db_node_properties` will be included in the visualization graph under the `properties` field.
     Additionally, a new "labels" node property will be added, containing the node labels of the node.
     Similarly for relationships, a new "relationshipType" property will be added.
 
@@ -76,49 +80,36 @@ def from_gds(
         GraphDataScience object.
     G : Graph
         Graph object.
-    size_property : str, optional
-        Property to use for node size, by default None.
-    additional_node_properties : list[str], optional
+    node_properties : list[str], optional
         Additional properties to include in the visualization node, by default None which means that all node
         properties from the Graph will be fetched.
-    additional_db_node_properties : list[str], optional
+    db_node_properties : list[str], optional
         Additional node properties to fetch from the database, by default None. Only works if the graph was projected from the database.
-    node_radius_min_max : tuple[float, float], optional
-        Minimum and maximum node radius, by default (3, 60).
-        To avoid tiny or huge nodes in the visualization, the node sizes are scaled to fit in the given range.
     max_node_count : int, optional
         The maximum number of nodes to fetch from the graph. The graph will be sampled using random walk with restarts
         if its node count exceeds this number.
     """
-    if additional_db_node_properties is None:
-        additional_db_node_properties = []
+    if db_node_properties is None:
+        db_node_properties = []
 
     node_properties_from_gds = G.node_properties()
     assert isinstance(node_properties_from_gds, pd.Series)
     actual_node_properties: dict[str, list[str]] = cast(dict[str, list[str]], node_properties_from_gds.to_dict())
     all_actual_node_properties = list(chain.from_iterable(actual_node_properties.values()))
 
-    if size_property is not None:
-        if size_property not in all_actual_node_properties:
-            raise ValueError(f"There is no node property '{size_property}' in graph '{G.name()}'")
-
     node_properties_by_label_sets: dict[str, set[str]] = dict()
-    if additional_node_properties is None:
+    if node_properties is None:
         node_properties_by_label_sets = {k: set(v) for k, v in actual_node_properties.items()}
     else:
-        for prop in additional_node_properties:
+        for prop in node_properties:
             if prop not in all_actual_node_properties:
                 raise ValueError(f"There is no node property '{prop}' in graph '{G.name()}'")
 
         for label, props in actual_node_properties.items():
             node_properties_by_label_sets[label] = {
-                prop for prop in actual_node_properties[label] if prop in additional_node_properties
+                prop for prop in actual_node_properties[label] if prop in node_properties
             }
 
-    if size_property is not None:
-        for label, label_props in node_properties_by_label_sets.items():
-            label_props.add(size_property)
-
     node_properties_by_label = {k: list(v) for k, v in node_properties_by_label_sets.items()}
 
     node_count = G.node_count()
@@ -143,7 +134,7 @@ def from_gds(
                 props.append(property_name)
 
         node_dfs = _fetch_node_dfs(
-            gds, G_fetched, node_properties_by_label, G_fetched.node_labels(), additional_db_node_properties
+            gds, G_fetched, node_properties_by_label, G_fetched.node_labels(), db_node_properties
         )
         if property_name is not None:
             for df in node_dfs.values():
@@ -161,13 +152,6 @@ def from_gds(
             df.drop(columns=[property_name], inplace=True)
 
     node_props_df = pd.concat(node_dfs.values(), ignore_index=True, axis=0).drop_duplicates()
-    if size_property is not None:
-        if "size" in all_actual_node_properties and size_property != "size":
-            node_props_df.rename(columns={"size": "__size"}, inplace=True)
-        if additional_node_properties is not None and size_property not in additional_node_properties:
-            node_props_df.rename(columns={size_property: "size"}, inplace=True)
-        else:
-            node_props_df["size"] = node_props_df[size_property]
 
     for lbl, df in node_dfs.items():
         if "labels" in all_actual_node_properties:
@@ -179,22 +163,22 @@ def from_gds(
 
     node_df = node_props_df.merge(node_labels_df, on="nodeId")
 
-    if "caption" not in all_actual_node_properties:
-        node_df["caption"] = node_df["labels"].astype(str)
+    try:
+        VG = _from_dfs(node_df, rel_dfs, dropna=True)
 
-    for rel_df in rel_dfs:
-        if "caption" not in rel_df.columns:
-            rel_df["caption"] = rel_df["relationshipType"]
+        for node in VG.nodes:
+            node.caption = str(node.properties.get("labels"))
+        for rel in VG.relationships:
+            rel.caption = rel.properties.get("relationshipType")
 
-    try:
-        return _from_dfs(
-            node_df, rel_dfs, node_radius_min_max=node_radius_min_max, rename_properties={"__size": "size"}, dropna=True
-        )
+        number_of_colors = node_df["labels"].drop_duplicates().count()
+        if number_of_colors <= len(NEO4J_COLORS_DISCRETE):
+            VG.color_nodes(property="labels", color_space=ColorSpace.DISCRETE)
+
+        return VG
     except ValueError as e:
         err_msg = str(e)
         if "column" in err_msg:
             err_msg = err_msg.replace("column", "property")
-            if ("'size'" in err_msg) and (size_property is not None):
-                err_msg = err_msg.replace("'size'", f"'{size_property}'")
             raise ValueError(err_msg)
         raise e

python-wrapper/src/neo4j_viz/node.py

@@ -98,3 +98,10 @@ def all_validation_aliases(exempted_fields: Optional[list[str]] = None) -> set[s
         by_field = [v.validation_alias.choices for k, v in Node.model_fields.items() if k not in exempted_fields]  # type: ignore
 
         return {str(alias) for aliases in by_field for alias in aliases}
+
+    @staticmethod
+    def basic_fields_validation_aliases() -> set[str]:
+        mandatory_fields = ["id"]
+        by_field = [v.validation_alias.choices for k, v in Node.model_fields.items() if k in mandatory_fields]  # type: ignore
+
+        return {str(alias) for aliases in by_field for alias in aliases}