Merge pull request #242 from neo4j/gql-and-neo4j-simplified-interface

gql and neo4j simplified interface

Author: FlorentinD

changelog.md

@@ -4,12 +4,12 @@
 
 - Do not automatically derive size and caption for `from_neo4j` and `from_gql_create`. Use the `size_property` and `node_caption` parameters to explicitly configure them.
 - Change API of integrations to only provide basic parameters. Any further configuration should happen ons the Visualization Graph object:
-  - `from_gds`
-    - Drop parameters size_property, node_radius_min_max. `Use VG.resize_nodes(property=...)` instead
-    - rename additional_node_properties to node_properties
-    - Don't derive fields from properties. Use `VG.map_properties_to_fields` instead
   - `from_pandas`
     - Drop `node_radius_min_max` parameter. `VG.resize_nodes(...)` instead
+  - `from_neo4j`, `from_gds`, `from_gql_create`
+    - Drop parameters `size_property`, `node_radius_min_max`. Use `VG.resize_nodes(property=...)` instead
+    - rename additional_node_properties to node_properties
+    - Don't derive fields from properties. Use `VG.map_properties_to_fields` instead
 
 ## New features
 
@@ -25,7 +25,7 @@
 
 - Validate fields of a node and relationship not only at construction but also on assignment.
 - Allow resizing per node property such as `VG.resize_nodes(property="score")`.
-- Color nodes by label in `from_gds`.
+- Color nodes by label in `from_gds` and `from_gql_create`.
 - Add `table` property to nodes and relationships created by `from_snowflake`. This is used as a default caption.
 
 ## Other changes

docs/source/integration.rst

@@ -164,22 +164,9 @@ The ``from_neo4j`` method takes one mandatory positional parameter:
 A ``data`` argument representing either a query result in the shape of a ``neo4j.graph.Graph`` or ``neo4j.Result``, or a
 ``neo4j.Driver`` in which case a simple default query will be executed internally to retrieve the graph data.
 
-We can also provide an optional ``size_property`` parameter, which should refer to a node property,
-and will be used to determine the sizes of the nodes in the visualization.
-
-The ``node_caption`` and ``relationship_caption`` parameters are also optional, and indicate the node and relationship
-properties to use for the captions of each element in the visualization.
-By default, the captions will be set to the node labels relationship types, but you can specify any property that
-exists on these entities.
-
-The last optional property, ``node_radius_min_max``, can be used (and is used by default) to scale the node sizes for
-the visualization.
-It is a tuple of two numbers, representing the radii (sizes) in pixels of the smallest and largest nodes respectively in
-the visualization.
-The node sizes will be scaled such that the smallest node will have the size of the first value, and the largest node
-will have the size of the second value.
-The other nodes will be scaled linearly between these two values according to their relative size.
-This can be useful if node sizes vary a lot, or are all very small or very big.
+The optional ``max_rows`` parameter can be used to limit the number of relationships shown in the visualization.
+By default, it is set to 10.000, meaning that if the database has more than 10.000 rows, a warning will be raised.
+Note, this only applies if the ``data`` parameter is a ``neo4j.Driver``.
 
 
 Example
@@ -222,20 +209,6 @@ The ``from_gql_create`` method takes one mandatory positional parameter:
 
 * A valid ``query`` representing a GQL ``CREATE`` query as a string.
 
-We can also provide an optional ``size_property`` parameter, which should refer to a node property,
-and will be used to determine the sizes of the nodes in the visualization.
-
-The ``node_caption`` and ``relationship_caption`` parameters are also optional, and indicate the node and relationship properties to use for the captions of each element in the visualization.
-
-The last optional property, ``node_radius_min_max``, can be used (and is used by default) to scale the node sizes for
-the visualization.
-It is a tuple of two numbers, representing the radii (sizes) in pixels of the smallest and largest nodes respectively in
-the visualization.
-The node sizes will be scaled such that the smallest node will have the size of the first value, and the largest node
-will have the size of the second value.
-The other nodes will be scaled linearly between these two values according to their relative size.
-This can be useful if node sizes vary a lot, or are all very small or very big.
-
 
 Example
 ~~~~~~~
@@ -283,39 +256,14 @@ The ``from_snowflake`` method takes two mandatory positional parameters:
 * A `project configuration <https://neo4j.com/docs/snowflake-graph-analytics/current/jobs/#jobs-project>`_ as a dictionary, that specifies how you want your tables to be projected as a graph.
   This configuration is the same as the project configuration of the `Neo4j Snowflake Graph Analytics application <https://neo4j.com/docs/snowflake-graph-analytics/current/>`_.
 
-``from_snowflake`` also takes an optional property, ``node_radius_min_max``, that can be used (and is used by default) to
-scale the node sizes for the visualization.
-It is a tuple of two numbers, representing the radii (sizes) in pixels of the smallest and largest nodes respectively in
-the visualization.
-The node sizes will be scaled such that the smallest node will have the size of the first value, and the largest node
-will have the size of the second value.
-The other nodes will be scaled linearly between these two values according to their relative size.
-This can be useful if node sizes vary a lot, or are all very small or very big.
-
-
-Special columns
-~~~~~~~~~~~~~~~
-
-It is possible to modify the visualization directly by including columns of certain specific names in the node and relationship tables.
-
-All such special columns can be found :doc:`here <./api-reference/node>` for nodes and :doc:`here <./api-reference/relationship>` for relationships.
-Though listed in ``snake_case`` here, ``SCREAMING_SNAKE_CASE`` and ``camelCase`` are also supported.
-Some of the most commonly used special columns are:
-
-* **Node sizes**: The sizes of nodes can be controlled by including a column named "SIZE" in node tables.
-  The values in these columns should be of a numeric type. This can be useful for visualizing the relative importance or size of nodes in the graph, for example using a computed centrality score.
-
-* **Captions**: The caption text of nodes and relationships can be controlled by including a column named "CAPTION" in the tables.
-  The values in these columns should be of a string type. This can be useful for displaying additional information about the nodes, such as their names or labels. If no "CAPTION" column is provided, the default captions in the visualization will be the names of the corresponding node and relationship tables.
-
-Please also note that you can further customize the visualization after the `VisualizationGraph` has been created, by using the methods described in the :doc:`Customizing the visualization <./customizing>` section.
+You can further customize the visualization after the `VisualizationGraph` has been created, by using the methods described in the :doc:`Customizing the visualization <./customizing>` section.
 
 
 Default behavior
 ~~~~~~~~~~~~~~~~
 
-Unless there are "CAPTION" columns in the tables, the node and relationship captions will be set to the names of the corresponding tables.
-Similarly, if there are are no "COLOR" node table columns, the nodes will be colored be colored so that nodes from the same table have the same color, and different tables have different colors.
+The node and relationship captions will be set to the names of the corresponding tables.
+The nodes will be colored so that nodes from the same table have the same color, and different tables have different colors.
 
 
 Example

python-wrapper/src/neo4j_viz/gds.py

@@ -167,7 +167,7 @@ def from_gds(
         VG = _from_dfs(node_df, rel_dfs, dropna=True)
 
         for node in VG.nodes:
-            node.caption = str(node.properties.get("labels"))
+            node.caption = ":".join([label for label in node.properties["labels"]])
         for rel in VG.relationships:
             rel.caption = rel.properties.get("relationshipType")
 

python-wrapper/src/neo4j_viz/gql_create.py

@@ -5,6 +5,7 @@
 from pydantic import BaseModel, ValidationError
 
 from neo4j_viz import Node, Relationship, VisualizationGraph
+from neo4j_viz.colors import NEO4J_COLORS_DISCRETE, ColorSpace
 
 
 def _parse_value(value_str: str) -> Any:
@@ -91,10 +92,7 @@ def _parse_value(value_str: str) -> Any:
     return value_str.strip("'\"")
 
 
-def _parse_prop_str(
-    query: str, prop_str: str, prop_start: int, top_level_keys: set[str]
-) -> tuple[dict[str, Any], dict[str, Any]]:
-    top_level: dict[str, Any] = {}
+def _parse_prop_str(query: str, prop_str: str, prop_start: int) -> dict[str, Any]:
     props: dict[str, Any] = {}
     depth = 0
     in_string = None
@@ -115,10 +113,7 @@ def _parse_prop_str(
                 k, v = pair.split(":", 1)
                 k = k.strip().strip("'\"")
 
-                if k in top_level_keys:
-                    top_level[k] = _parse_value(v)
-                else:
-                    props[k] = _parse_value(v)
+                props[k] = _parse_value(v)
 
                 start_idx = i + 1
         else:
@@ -133,17 +128,12 @@ def _parse_prop_str(
         k, v = pair.split(":", 1)
         k = k.strip().strip("'\"")
 
-        if k in top_level_keys:
-            top_level[k] = _parse_value(v)
-        else:
-            props[k] = _parse_value(v)
+        props[k] = _parse_value(v)
 
-    return top_level, props
+    return props
 
 
-def _parse_labels_and_props(
-    query: str, s: str, top_level_keys: set[str]
-) -> tuple[Optional[str], dict[str, Any], dict[str, Any]]:
+def _parse_labels_and_props(query: str, s: str) -> tuple[Optional[str], dict[str, Any]]:
     prop_match = re.search(r"\{(.*)\}", s)
     prop_str = ""
     if prop_match:
@@ -155,17 +145,16 @@ def _parse_labels_and_props(
     final_alias = raw_alias if raw_alias else None
 
     if prop_str:
-        top_level, props = _parse_prop_str(query, prop_str, prop_start, top_level_keys)
+        props = _parse_prop_str(query, prop_str, prop_start)
     else:
-        top_level = {}
         props = {}
 
     label_list = [lbl.strip() for lbl in alias_labels[1:]]
     if "labels" in props:
         props["__labels"] = props["labels"]
     props["labels"] = sorted(label_list)
 
-    return final_alias, top_level, props
+    return final_alias, props
 
 
 def _get_snippet(q: str, idx: int, context: int = 15) -> str:
@@ -175,21 +164,20 @@ def _get_snippet(q: str, idx: int, context: int = 15) -> str:
     return q[start:end].replace("\n", " ")
 
 
-def from_gql_create(
-    query: str,
-    size_property: Optional[str] = None,
-    node_caption: Optional[str] = "labels",
-    relationship_caption: Optional[str] = "type",
-    node_radius_min_max: Optional[tuple[float, float]] = (3, 60),
-) -> VisualizationGraph:
+def from_gql_create(query: str) -> VisualizationGraph:
     """
     Parse a GQL CREATE query and return a VisualizationGraph object representing the graph it creates.
 
     All node and relationship properties will be included in the visualization graph.
-    If the properties are named as the fields of the `Node` or `Relationship` classes, they will be included as
-    top level fields of the respective objects. Otherwise, they will be included in the `properties` dictionary.
+    All properties of nodes and relationships will be included in the `properties` dictionary of the respective objects.
     Additionally, a "labels" property will be added for nodes and a "type" property for relationships.
 
+    By default:
+
+    * the caption of a node will be based on its `labels`.
+    * the caption of a relationship will be based on its `type`.
+    * the color of nodes will be set based on their label, unless there are more than 12 unique labels.
+
     Please note that this function is not a full GQL parser, it only handles CREATE queries that do not contain
     other clauses like MATCH, WHERE, RETURN, etc, or any Cypher function calls.
     It also does not handle all possible GQL syntax, but it should work for most common cases.
@@ -199,15 +187,6 @@ def from_gql_create(
     ----------
     query : str
         The GQL CREATE query to parse
-    size_property : str, optional
-        Property to use for node size, by default None.
-    node_caption : str, optional
-        Property to use as the node caption, by default the node labels will be used.
-    relationship_caption : str, optional
-        Property to use as the relationship caption, by default the relationship type will be used.
-    node_radius_min_max : tuple[float, float], optional
-        Minimum and maximum node radius, by default (3, 60).
-        To avoid tiny or huge nodes in the visualization, the node sizes are scaled to fit in the given range.
     """
 
     query = query.strip()
@@ -251,19 +230,9 @@ def from_gql_create(
     node_pattern = re.compile(r"^\(([^)]*)\)$")
     rel_pattern = re.compile(r"^\(([^)]*)\)-\s*\[\s*:(\w+)\s*(\{[^}]*\})?\s*\]->\(([^)]*)\)$")
 
-    node_top_level_keys = Node.all_validation_aliases(exempted_fields=["id", "size", "caption"])
-    rel_top_level_keys = Relationship.all_validation_aliases(exempted_fields=["id", "source", "target", "caption"])
-
     def _parse_validation_error(e: ValidationError, entity_type: type[BaseModel]) -> None:
         for err in e.errors():
             loc = err["loc"][0]
-            if (loc == "size") and size_property is not None:
-                loc = size_property
-            if loc == "caption":
-                if (entity_type == Node) and (node_caption is not None):
-                    loc = node_caption
-                elif (entity_type == Relationship) and (relationship_caption is not None):
-                    loc = relationship_caption
             raise ValueError(
                 f"Error for {entity_type.__name__.lower()} property '{loc}' with provided input '{err['input']}'. Reason: {err['msg']}"
             )
@@ -277,14 +246,14 @@ def _parse_validation_error(e: ValidationError, entity_type: type[BaseModel]) ->
         node_m = node_pattern.match(part)
         if node_m:
             alias_labels_props = node_m.group(1).strip()
-            alias, top_level, props = _parse_labels_and_props(query, alias_labels_props, node_top_level_keys)
+            alias, props = _parse_labels_and_props(query, alias_labels_props)
             if not alias:
                 alias = f"_anon_{anonymous_count}"
                 anonymous_count += 1
             if alias not in alias_to_id:
                 alias_to_id[alias] = str(uuid.uuid4())
             try:
-                nodes.append(Node(id=alias_to_id[alias], **top_level, properties=props))
+                nodes.append(Node(id=alias_to_id[alias], properties=props))
             except ValidationError as e:
                 _parse_validation_error(e, Node)
 
@@ -296,29 +265,29 @@ def _parse_validation_error(e: ValidationError, entity_type: type[BaseModel]) ->
             right_node = rel_m.group(4).strip()
 
             # Parse left node pattern
-            left_alias, left_top_level, left_props = _parse_labels_and_props(query, left_node, node_top_level_keys)
+            left_alias, left_props = _parse_labels_and_props(query, left_node)
             if not left_alias:
                 left_alias = f"_anon_{anonymous_count}"
                 anonymous_count += 1
                 if left_alias not in alias_to_id:
                     alias_to_id[left_alias] = str(uuid.uuid4())
                 try:
-                    nodes.append(Node(id=alias_to_id[left_alias], **left_top_level, properties=left_props))
+                    nodes.append(Node(id=alias_to_id[left_alias], properties=left_props))
                 except ValidationError as e:
                     _parse_validation_error(e, Node)
             elif left_alias not in alias_to_id:
                 snippet = _get_snippet(query, query.index(left_node))
                 raise ValueError(f"Relationship references unknown node alias: '{left_alias}' near: `{snippet}`.")
 
             # Parse right node pattern
-            right_alias, right_top_level, right_props = _parse_labels_and_props(query, right_node, node_top_level_keys)
+            right_alias, right_props = _parse_labels_and_props(query, right_node)
             if not right_alias:
                 right_alias = f"_anon_{anonymous_count}"
                 anonymous_count += 1
                 if right_alias not in alias_to_id:
                     alias_to_id[right_alias] = str(uuid.uuid4())
                 try:
-                    nodes.append(Node(id=alias_to_id[right_alias], **right_top_level, properties=right_props))
+                    nodes.append(Node(id=alias_to_id[right_alias], properties=right_props))
                 except ValidationError as e:
                     _parse_validation_error(e, Node)
             elif right_alias not in alias_to_id:
@@ -331,9 +300,8 @@ def _parse_validation_error(e: ValidationError, entity_type: type[BaseModel]) ->
             if rel_props_str:
                 inner_str = rel_props_str.strip("{}").strip()
                 prop_start = query.index(inner_str, query.index(inner_str))
-                top_level, props = _parse_prop_str(query, inner_str, prop_start, rel_top_level_keys)
+                props = _parse_prop_str(query, inner_str, prop_start)
             else:
-                top_level = {}
                 props = {}
             if "type" in props:
                 props["__type"] = props["type"]
@@ -345,7 +313,6 @@ def _parse_validation_error(e: ValidationError, entity_type: type[BaseModel]) ->
                         id=rel_id,
                         source=alias_to_id[left_alias],
                         target=alias_to_id[right_alias],
-                        **top_level,
                         properties=props,
                     )
                 )
@@ -357,28 +324,15 @@ def _parse_validation_error(e: ValidationError, entity_type: type[BaseModel]) ->
         snippet = part[:30]
         raise ValueError(f"Invalid element in CREATE near: `{snippet}`.")
 
-    if size_property is not None:
-        try:
-            for node in nodes:
-                node.size = node.properties.get(size_property)
-        except ValidationError as e:
-            _parse_validation_error(e, Node)
-    if node_caption is not None:
-        for node in nodes:
-            if node_caption == "labels":
-                if len(node.properties["labels"]) > 0:
-                    node.caption = ":".join([label for label in node.properties["labels"]])
-            else:
-                node.caption = str(node.properties.get(node_caption))
-    if relationship_caption is not None:
-        for rel in relationships:
-            if relationship_caption == "type":
-                rel.caption = rel.properties["type"]
-            else:
-                rel.caption = str(rel.properties.get(relationship_caption))
-
     VG = VisualizationGraph(nodes=nodes, relationships=relationships)
-    if (node_radius_min_max is not None) and (size_property is not None):
-        VG.resize_nodes(node_radius_min_max=node_radius_min_max)
+
+    for node in VG.nodes:
+        node.caption = ":".join([label for label in node.properties["labels"]])
+    for rel in VG.relationships:
+        rel.caption = rel.properties.get("type")
+
+    number_of_colors = len({str(n.properties.get("labels")) for n in VG.nodes})
+    if number_of_colors <= len(NEO4J_COLORS_DISCRETE):
+        VG.color_nodes(property="labels", color_space=ColorSpace.DISCRETE)
 
     return VG