config2py development how tos

How to make yaml package easy to use

When using the python builtin yaml package, doing a yaml.safe_load(stream), I ran into this error:

ScannerError: mapping values are not allowed here

How can I load my stream, skipping any items that might not be safe?

When you encounter the ScannerError: mapping values are not allowed here error when using the yaml.safe_load() function in Python, it typically indicates that the YAML data being loaded contains syntax errors. In particular, the error message suggests that the problem is with a mapping (i.e., a key-value pair) in the YAML data.

To load a YAML stream in Python and skip any items that might not be safe, you can use the yaml.load_all() function instead of yaml.safe_load(). The yaml.load_all() function is similar to yaml.safe_load(), but it returns a generator that can be used to iterate over all of the YAML documents in a stream.

Here's an example of how to use yaml.load_all() to load a YAML stream and skip any unsafe items:

import yaml

def load_yaml_stream(stream):
    """
    Loads a YAML stream, skipping any items that might not be safe.

    Args:
        stream (Union[str, IO]): A YAML stream, as a string or file-like object.

    Returns:
        List[Dict]: A list of dictionaries representing the YAML documents in the stream.
    """
    documents = []
    for document in yaml.load_all(stream, Loader=yaml.FullLoader):
        if document is not None:
            documents.append(document)
    return documents

This function takes a YAML stream as input and returns a list of dictionaries representing the YAML documents in the stream. It uses the yaml.load_all() function to iterate over the documents in the stream, and skips any items that might not be safe by checking if the document is not None. This approach allows you to load as much of the YAML stream as possible, while ignoring any items that might be unsafe.

I haven't tried this with docs that have errors (the errors I thought I had that sparked this writing were due to feeding an html instead of a yaml), but the following might be useful if the iteration of the output of load_all needs to ignore errors:

from typing import Generator, Callable, Optional

def iterate_skipping_errors(g: Generator, error_callback : Optional[Callable] = None):
    iterator = iter(g)
    while True:
        try:
            yield next(iterator)
        except StopIteration:
            break
        except Exception as e:
            if error_callback:
                error_callback(e)