Refactor, improve API, documentation and packaging

Author: aradi

CHANGELOG.rst

@@ -0,0 +1,14 @@
+==========
+Change Log
+==========
+
+
+Unreleased
+==========
+
+Added
+-----
+
+* Basic functionality to manipulate HSD-data in Python.
+
+* Pip installation

LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2011-2020  DFTB+ developers group
+Copyright (c) 2011-2021  DFTB+ developers group
 
 All rights reserved.
 

README.rst

@@ -1,33 +1,40 @@
-************************************
-HSD — Human-friendly Structured Data
-************************************
+**********************************************
+HSD — Make your structured data human friendly
+**********************************************
 
-This Python package contains utilities to read and write files in
-the Human-friendly Structured Data (HSD) format.
+This package contains utilities to read and write files in the Human-friendly
+Structured Data (HSD) format.
 
-It is licensed under the *BSD 2-clause license*.
+The HSD-format is very similar to both JSON and YAML, but tries to minimize the
+effort for **humans** to read and write it. It ommits special characters as much
+as possible (in contrast to JSON) and is not indentation dependent (in contrast
+to YAML). It was developed originally as the input format for the scientific
+simulation tool (`DFTB+ <https://github.com/dftbplus/dftbplus>`_), but is
+of general purpose. Data stored in HSD can be easily mapped to a subset of JSON
+or XML andvica versa.
 
 
 Installation
 ============
 
-To install the python package in development mode use
+The package can be installed via conda-forge::
 
-.. code::
+  conda install hsd-python
 
-   pip install -e src
+Alternatively, the package can be downloaded and installed via pip into the
+active Python interpreter (preferably using a virtual python environment) by ::
 
+  pip install hsd
 
-The HSD format
-==============
+or into the user space issueing ::
+
+  pip install --user hsd
 
-The HSD-format is very similar to both JSON and XML, but tries to minimize the
-effort for humans to read and write it. It ommits special characters as much as
-possible but (in contrast to YAML for example) is not indentation dependent.
 
-It was developed originally as the input format for a scientific simulation tool
-(`DFTB+ <https://github.com/dftbplus/dftbplus>`_), but is absolutely general. A
-typical input written in HSD looks like ::
+Quick tutorial
+==============
+
+A typical, self-explaining input written in HSD looks like ::
 
   driver {
     conjugate_gradients {
@@ -45,11 +52,13 @@ typical input written in HSD looks like ::
       }
       filling {
         fermi {
-          temperature [kelvin] = 1e-8
+          # This is comment which will be ignored
+          # Note the attribute (unit) of the field below
+          temperature [kelvin] = 100
         }
       }
       k_points_and_weights {
-        supercell_folding = {
+        supercell_folding {
           2   0   0
           0   2   0
           0   0   2
@@ -59,13 +68,56 @@ typical input written in HSD looks like ::
     }
   }
 
-Content in HSD format can be represented as JSON. Content in JSON format can
-similarly be represented as HSD, provided it satisfies one restriction for
-arrays: Either all elements of an array must be objects or none of them. (This
-allows for a clear separation of structure and data and allows for the very
-simple input format.)
+The above input can be parsed into a Python dictionary with::
+
+  import hsd
+  hsdinput = hsd.load_file("test.hsd")
+
+The dictionary ``hsdinput`` will then look as::
+
+  {
+      "driver": {
+          "conjugate_gradients" {
+              "moved_atoms": [1, 2, "7:19"],
+              "max_steps": 100
+          }
+      },
+      "hamiltonian": {
+          "dftb": {
+              "scc": True,
+              "scc_tolerance": 1e-10,
+              "mixer": {
+                  "broyden": {}
+              },
+              "filling": {
+                  "fermi": {
+                      "temperature": 100,
+                      "temperature.attrib": "kelvin"
+                  }
+              }
+              "k_points_and_weights": {
+                  "supercell_folding": [
+                      [2, 0, 0],
+                      [0, 2, 0],
+                      [0, 0, 2],
+                      [0.5, 0.5, 0.5]
+                  ]
+              }
+          }
+      }
+  }
+
+Being a simple Python dictionary, it can be easily queried and manipulated in
+Python ::
+
+  hsdinput["driver"]["conjugate_gradients"]["max_steps"] = 200
+
+and then stored again in HSD format ::
+
+    hsd.dump_file(hsdinput, "test2.hsd")
+
+
+License
+========
 
-Content in HSD format can be represented as XML (DOM-tree). Likewise content in
-XML can be converted to HSD, provided it satisfies the restriction that every
-child has either data (text) or further children, but never both of
-them. (Again, this ensures the simplicity of the input format.)
+The hsd-python package is licensed under the `BSD 2-clause license <LICENSE>`_.

pyproject.toml

@@ -0,0 +1,3 @@
+[build-system]
+requires = ["setuptools", "wheel"]
+build-backend = "setuptools.build_meta"

setup.cfg

@@ -0,0 +1,31 @@
+[metadata]
+name = hsd-python
+version = 0.1
+author = DFTB+ developers group
+author_email = [email protected]
+url = https://github.com/dftbplus/hsd-python
+description =
+    Tools for reading, writing and manipulating data stored in the human-friendly
+    structured data (HSD) format
+long_description = file: README.rst
+long_description_content_type = text/x-rst
+license = BSD
+license_file = LICENSE
+platform = any
+classifiers =
+    Intended Audience :: Developers
+    License :: OSI Approved :: BSD License
+    Programming Language :: Python :: 3 :: Only
+    Programming Language :: Python :: 3.7
+    Programming Language :: Python :: 3.8
+    Programming Language :: Python :: 3.9
+    Programming Language :: Python :: 3.7
+
+[options]
+include_package_data = True
+package_dir =
+    = src
+packages = hsd
+
+[options.packages.find]
+where = src

src/LICENSE

@@ -8,6 +8,7 @@
 """
 Central module for the hsd package
 """
-from .dump import dump, dumps
-from .parser import HsdParser
 from .dictbuilder import HsdDictBuilder
+from .eventhandler import HsdEventHandler
+from .io import load, load_string, load_file, dump, dump_string, dump_file
+from .parser import HsdParser

src/MANIFEST.in

@@ -12,40 +12,10 @@
 
 class HsdException(Exception):
     """Base class for exceptions in the HSD package."""
-    pass
-
-
-class HsdQueryError(HsdException):
-    """Base class for errors detected by the HsdQuery object.
-
-
-    Attributes:
-        filename: Name of the file where error occured (or empty string).
-        line: Line where the error occurred (or -1).
-        tag: Name of the tag with the error (or empty string).
-    """
-
-    def __init__(self, msg="", node=None):
-        """Initializes the exception.
-
-        Args:
-            msg: Error message
-            node: HSD element where error occured (optional).
-        """
-        super().__init__(msg)
-        if node is not None:
-            self.tag = node.gethsd(HSDATTR_TAG, node.tag)
-            self.file = node.gethsd(HSDATTR_FILE, -1)
-            self.line = node.gethsd(HSDATTR_LINE, None)
-        else:
-            self.tag = ""
-            self.file = -1
-            self.line = None
 
 
 class HsdParserError(HsdException):
     """Base class for parser related errors."""
-    pass
 
 
 def unquote(txt):
@@ -56,11 +26,23 @@ def unquote(txt):
 
 
 # Name for default attribute (when attribute name is not specified)
-DEFAULT_ATTRIBUTE = "attribute"
+DEFAULT_ATTRIBUTE = "unit"
+
+# Suffix to mark attribute
+ATTRIB_SUFFIX = ".attrib"
+
+# Length of the attribute suffix
+LEN_ATTRIB_SUFFIX = len(ATTRIB_SUFFIX)
+
+# Suffix to mark hsd processing attributes
+HSD_ATTRIB_SUFFIX = ".hsdattrib"
+
+# Lengths of hsd processing attribute suffix
+LEN_HSD_ATTRIB_SUFFIX = len(HSD_ATTRIB_SUFFIX)
+
+
+HSD_ATTRIB_LINE = "line"
 
+HSD_ATTRIB_EQUAL = "equal"
 
-HSDATTR_PROC = "processed"
-HSDATTR_EQUAL = "equal"
-HSDATTR_FILE = "file"
-HSDATTR_LINE = "line"
-HSDATTR_TAG = "tag"
+HSD_ATTRIB_TAG = "tag"

src/hsd/__init__.py

@@ -9,9 +9,8 @@
 Contains an event-driven builder for dictionary based (JSON-like) structure
 """
 import re
-from .parser import HsdEventHandler
-
-__all__ = ['HsdDictBuilder']
+from .common import ATTRIB_SUFFIX, HSD_ATTRIB_SUFFIX
+from .eventhandler import HsdEventHandler
 
 
 _TOKEN_PATTERN = re.compile(r"""
@@ -29,21 +28,24 @@
 class HsdDictBuilder(HsdEventHandler):
     """Deserializes HSD into nested dictionaries
 
-    Note: hsdoptions passed by the generating events are ignored.
+    Note: hsdattrib passed by the generating events are ignored.
     """
 
-    def __init__(self, flatten_data=False):
-        HsdEventHandler.__init__(self)
+    def __init__(self, flatten_data=False, include_hsd_attribs=False):
+        super().__init__()
         self._hsddict = {}
         self._curblock = self._hsddict
         self._parentblocks = []
         self._data = None
         self._flatten_data = flatten_data
+        self._include_hsd_attribs = include_hsd_attribs
 
 
-    def open_tag(self, tagname, attrib, hsdoptions):
+    def open_tag(self, tagname, attrib, hsdattrib):
         if attrib is not None:
-            self._curblock[tagname + '.attribute'] = attrib
+            self._curblock[tagname + ATTRIB_SUFFIX] = attrib
+        if self._include_hsd_attribs and hsdattrib is not None:
+            self._curblock[tagname + HSD_ATTRIB_SUFFIX] = hsdattrib
         self._parentblocks.append(self._curblock)
         self._curblock = {}
 

src/hsd/common.py

@@ -0,0 +1,56 @@
+"""Contains an event handler base class."""
+
+
+class HsdEventHandler:
+    """Base class for event handler implementing simple printing"""
+
+    def __init__(self):
+        """Initializes the default event handler"""
+        self._indentlevel = 0
+        self._indentstr = "  "
+
+
+    def open_tag(self, tagname, attrib, hsdattrib):
+        """Handler which is called when a tag is opened.
+
+        It should be overriden in the application to handle the event in a
+        customized way.
+
+        Args:
+            tagname: Name of the tag which had been opened.
+            attrib: String containing the attribute of the tag or None.
+            hsdattrib: Dictionary of the options created during the processing
+                in the hsd-parser.
+        """
+        indentstr = self._indentlevel * self._indentstr
+        print("{}OPENING TAG: {}".format(indentstr, tagname))
+        print("{}ATTRIBUTE: {}".format(indentstr, attrib))
+        print("{}HSD OPTIONS: {}".format(indentstr, str(hsdattrib)))
+        self._indentlevel += 1
+
+
+    def close_tag(self, tagname):
+        """Handler which is called when a tag is closed.
+
+        It should be overriden in the application to handle the event in a
+        customized way.
+
+        Args:
+            tagname: Name of the tag which had been closed.
+        """
+        indentstr = self._indentlevel * self._indentstr
+        print("{}CLOSING TAG: {}".format(indentstr, tagname))
+        self._indentlevel -= 1
+
+
+    def add_text(self, text):
+        """Handler which is called with the text found inside a tag.
+
+        It should be overriden in the application to handle the event in a
+        customized way.
+
+        Args:
+           text: Text in the current tag.
+        """
+        indentstr = self._indentlevel * self._indentstr
+        print("{}Received text: {}".format(indentstr, text))