Merge pull request #2575 from Kodiologist/docs-content-2

More content edits for the manual

Author: Kodiologist

CONTRIBUTING.rst

@@ -92,7 +92,7 @@ NEWS and AUTHORS
 ----------------
 
 If you're making user-visible changes to the code, add one or more
-items describing it to the NEWS file.
+items describing them to the NEWS file.
 
 Finally, add yourself to the AUTHORS file (as a separate commit): you
 deserve it. :)

docs/api.rst

@@ -1,3 +1,5 @@
+.. _macros:
+
 ======
 Macros
 ======
@@ -161,6 +163,21 @@ Reader macros
 
 Reader macros allow you to hook directly into Hy's parser to customize how text is parsed into models. They're defined with :hy:func:`defreader <hy.core.macros.defreader>`, or, like regular macros, brought in from other modules with :hy:func:`require`. Rather than receiving function arguments, a reader macro has access to a :py:class:`HyReader <hy.reader.hy_reader.HyReader>` object named ``&reader``, which provides all the text-parsing logic that Hy uses to parse itself (see :py:class:`HyReader <hy.reader.hy_reader.HyReader>` and its base class :py:class:`Reader <hy.reader.reader.Reader>` for the available methods). A reader macro is called with the hash sign ``#``, and like a regular macro, it should return a model or something convertible to a model.
 
+The simplest kind of reader macro doesn't read anything::
+
+    (defreader hi
+      `(print "Hello."))
+
+    #hi #hi #hi
+
+A less trivial, and more common, usage of reader macros is to call :func:`HyReader.parse-one-form <hy.reader.hy_reader.HyReader.parse_one_form>` to get a single form from the following source text. Such a reader macro is like a unary regular macro that's called with ``#`` instead of parentheses. ::
+
+    (defreader do-twice
+      (setv x (.parse-one-form &reader))
+      `(do ~x ~x))
+
+    #do-twice (print "This line prints twice.")
+
 Here's a moderately complex example of a reader macro that couldn't be implemented as a regular macro. It reads in a list of lists in which the inner lists are newline-separated, but newlines are allowed inside elements. ::
 
     (defreader matrix

docs/macros.rst

@@ -27,13 +27,14 @@ Suppose you want to validate and extract the components of a form like::
         (foo8))))
 
 You could do this with loops and indexing, but it would take a lot of code and
-be error-prone. Model patterns concisely express the general form of an
-expression to be matched, like what a regular expression does for text. Here's
-a pattern for a ``try`` form of the above kind::
+be error-prone. Model patterns concisely express the general form of a model
+tree to be matched, like what a regular expression does for text. Here's a
+pattern for a ``try`` form of the above kind::
 
     (import
       funcparserlib.parser [maybe many]
       hy.model-patterns *)
+
     (setv parser (whole [
       (sym "try")
       (many (notpexpr "except" "else" "finally"))
@@ -78,15 +79,15 @@ built-in parsers:
 - ``(some function)`` takes a predicate ``function`` and matches a form if it
   satisfies the predicate.
 
-The best reference for Hy's parsers is the docstrings (use ``(help
-hy.model-patterns)``), but again, here are some of the more important ones:
+Some of the more important of Hy's own parsers are:
 
-- ``FORM`` matches anything.
-- ``SYM`` matches any symbol.
-- ``(sym "foo")`` or ``(sym ":foo")`` matches and discards (per ``skip``) the
+- :data:`FORM <hy.model_patterns.FORM>` matches anything.
+- :data:`SYM <hy.model_patterns.SYM>` matches any symbol.
+- :func:`sym <hy.model_patterns.sym>` matches and discards (per ``skip``) the
   named symbol or keyword.
-- ``(brackets ...)`` matches the arguments in square brackets.
-- ``(pexpr ...)`` matches the arguments in parentheses.
+- :func:`brackets <hy.model_patterns.brackets>` matches the arguments in square
+  brackets.
+- :func:`pexpr <hy.model_patterns.pexpr>` matches the arguments in parentheses.
 
 Here's how you could write a simple macro using model patterns::
 
@@ -105,3 +106,9 @@ Here's how you could write a simple macro using model patterns::
 A failed parse will raise ``funcparserlib.parser.NoParseError``.
 
 .. _funcparserlib: https://github.com/vlasovskikh/funcparserlib
+
+Reference
+---------
+
+.. automodule:: hy.model_patterns
+   :members:

docs/model_patterns.rst

@@ -51,44 +51,12 @@
   `(if ~test (do ~@body) None))
 
 (defmacro defreader [_hy_compiler key #* body]
-  "Define a new reader macro.
-
-  Reader macros are expanded at read time and allow you to modify the behavior
-  of the Hy reader. Access to the currently instantiated `HyReader` is available
-  in the ``body`` as ``&reader``. See :py:class:`HyReader <hy.reader.hy_reader.HyReader>`
-  and its base class :py:class:`Reader <hy.reader.reader.Reader>` for details
-  regarding the available processing methods.
-
-  Reader macro names can be any valid identifier and are callable by prefixing
-  the name with a ``#``. i.e. ``(defreader upper ...)`` is called with ``#upper``.
-
-  Examples:
-
-     The following is a primitive example of a reader macro that adds Python's
-     colon ``:`` slice sugar into Hy::
-
-        => (defreader slice
-        ...   (defn parse-node []
-        ...     (let [node (when (!= \":\" (.peekc &reader))
-        ...                  (.parse-one-form &reader))]
-        ...       (if (= node '...) 'Ellipse node)))
-        ...
-        ...   (with [(&reader.end-identifier \":\")]
-        ...     (let [nodes []]
-        ...       (&reader.slurp-space)
-        ...       (nodes.append (parse-node))
-        ...       (while (&reader.peek-and-getc \":\")
-        ...         (nodes.append (parse-node)))
-        ...
-        ...       `(slice ~@nodes))))
-
-        => (setv an-index 42)
-        => #slice a:(+ 1 2):\"column\"
-        (slice 42 3 column)
-
-     See the :ref:`reader macros docs <reader-macros>` for more detailed
-     information on how reader macros work and are defined.
-  "
+  "Define a reader macro, at both compile-time and run-time. After the name,
+  all arguments are body forms: there is no parameter list as for ``defmacro``,
+  since it's up to the reader macro to decide how to parse the source text
+  following its call position. See :ref:`reader-macros` for details and
+  examples."
+
   (when (not (isinstance _hy_compiler.scope hy.scoping.ScopeGlobal))
     (raise (_hy_compiler._syntax-error
              _hy_compiler.this

hy/core/macros.hy

@@ -461,7 +461,7 @@ def compile_augassign_expression(compiler, expr, root, target, values):
 
 
 @pattern_macro("setv", [many(maybe_annotated(FORM) + FORM)])
-@pattern_macro(((3, 8), "setx"), [times(1, 1, SYM + FORM)])
+@pattern_macro("setx", [times(1, 1, SYM + FORM)])
 def compile_def_expression(compiler, expr, root, decls):
     if not decls:
         return asty.Constant(expr, value=None)

hy/core/result_macros.py

@@ -38,15 +38,14 @@
 (setv _gensym_lock (Lock))
 
 (defn gensym [[g ""]]
-  #[[Generate a symbol with a unique name. The argument will be included in the
-  generated symbol name, as an aid to debugging. Typically one calls ``hy.gensym``
-  without an argument.
+
+  #[[Generate a symbol with a unique name. The argument, if provided,
+  will be included in the generated symbol name, as an aid to
+  debugging.
 
   The below example uses the return value of ``f`` twice but calls it only
   once, and uses ``hy.gensym`` for the temporary variable to avoid collisions
-  with any other variable names.
-
-  ::
+  with any other variable names. ::
 
       (defmacro selfadd [x]
         (setv g (hy.gensym))
@@ -59,6 +58,7 @@
         4)
 
       (print (selfadd (f)))]]
+
   (.acquire _gensym_lock)
   (try
     (global _gensym_counter)

hy/core/util.hy

@@ -31,20 +31,28 @@
     Tuple,
 )
 
+
+#: Match any token.
 FORM = some(lambda _: True).named('form')
+#: Match a :class:`Symbol <hy.models.Symbol>`.
 SYM = some(lambda x: isinstance(x, Symbol)).named('Symbol')
+#: Match a :class:`Keyword <hy.models.Keyword>`.
 KEYWORD = some(lambda x: isinstance(x, Keyword)).named('Keyword')
-STR = some(lambda x: isinstance(x, String)).named('String')  # matches literal strings only!
+#: Match a :class:`String <hy.models.String>`.
+STR = some(lambda x: isinstance(x, String)).named('String')
+#: Match any model type denoting a literal.
 LITERAL = some(lambda x: isinstance(x, (String, Integer, Float, Complex, Bytes))).named('literal')
 
 
 def sym(wanted):
-    "Parse and skip the given symbol or keyword."
+    """Match and skip a symbol with a name equal to the string ``wanted``.
+    You can begin the string with ``":"`` to check for a keyword instead."""
     return _sym(wanted, skip)
 
 
 def keepsym(wanted):
-    "Parse the given symbol or keyword."
+    """As :func:`sym`, but the object is kept instead of
+    skipped."""
     return _sym(wanted)
 
 
@@ -56,7 +64,7 @@ def _sym(wanted, f=lambda x: x):
 
 
 def whole(parsers):
-    """Parse the parsers in the given list one after another, then
+    """Match the parsers in the given list one after another, then
     expect the end of the input."""
     if len(parsers) == 0:
         return finished >> (lambda x: [])
@@ -72,34 +80,43 @@ def _grouped(group_type, syntax_example, name, parsers):
         (lambda x: group_type(whole(parsers).parse(x)).replace(x, recursive=False))
     )
 def brackets(*parsers, name = None):
-    "Parse the given parsers inside square brackets."
+    """Match the given parsers inside square brackets (a :class:`List
+    <hy.models.List>`)."""
     return _grouped(List, '[ … ]', name, parsers)
 def in_tuple(*parsers, name = None):
+    "Match the given parsers inside a :class:`Tuple <hy.models.Tuple>`."
     return _grouped(Tuple, '#( … )', name, parsers)
 def braces(*parsers, name = None):
-    "Parse the given parsers inside curly braces"
+    """Match the given parsers inside curly braces (a :class:`Dict
+    <hy.models.Dict>`)."""
     return _grouped(Dict, '{ … }', name, parsers)
 def pexpr(*parsers, name = None):
-    "Parse the given parsers inside a parenthesized expression."
+    """Match the given parsers inside a parenthesized :class:`Expression
+    <hy.models.Expression>`."""
     return _grouped(Expression, '( … )', name, parsers)
 
 
 def dolike(head):
-    "Parse a `do`-like form."
+    """Parse a :hy:func:`do`-like expression. ``head`` is a string used to
+    construct a symbol for the head."""
     return pexpr(sym(head), many(FORM))
 
 
 def notpexpr(*disallowed_heads):
-    """Parse any object other than a hy.models.Expression beginning with a
-    hy.models.Symbol equal to one of the disallowed_heads."""
+    """Parse any object other than an expression headed by a symbol whose name
+    is equal to one of the given strings."""
     disallowed_heads = list(map(Symbol, disallowed_heads))
     return some(
         lambda x: not (isinstance(x, Expression) and x and x[0] in disallowed_heads)
     )
 
 
 def unpack(kind, content_type = None):
-    "Parse an unpacking form, returning it unchanged."
+    """Parse an unpacking form, returning it unchanged. ``kind`` should be
+    ``"iterable"``, ``"mapping"``, or ``"either"``. If ``content_type`` is
+    provided, the parser also checks that the unpacking form has exactly one
+    argument and that argument inherits from ``content_type``."""
+
     return some(lambda x:
         isinstance(x, Expression) and
         len(x) > 0 and
@@ -110,8 +127,9 @@ def unpack(kind, content_type = None):
 
 
 def times(lo, hi, parser):
-    """Parse `parser` several times (`lo` to `hi`) in a row. `hi` can be
-    float('inf'). The result is a list no matter the number of instances."""
+    """Parse ``parser`` several times (from ``lo`` to ``hi``, inclusive) in a
+    row. ``hi`` can be ``float('inf')``. The result is a list no matter the
+    number of instances."""
 
     @Parser
     def f(tokens, s):
@@ -132,24 +150,33 @@ def f(tokens, s):
 
 
 Tag = namedtuple("Tag", ["tag", "value"])
+Tag.__doc__ = 'A named tuple; see :func:`collections.namedtuple` and :func:`tag`.'
 
 
 def tag(tag_name, parser):
-    """Matches the given parser and produces a named tuple `(Tag tag value)`
-    with `tag` set to the given tag name and `value` set to the parser's
-    value."""
+    """Match on ``parser`` and produce an instance of :class:`Tag`
+    with ``tag`` set to ``tag_name`` and ``value`` set to result of matching
+    ``parser``."""
     return parser >> (lambda x: Tag(tag_name, x))
 
 
 def parse_if(pred, parser):
-    """
-    Return a parser that parses a token with `parser` if it satisfies the
-    predicate `pred`.
-    """
+    """Return a parser that parses a token with ``parser`` if it satisfies the
+    predicate ``pred``."""
 
     @Parser
     def _parse_if(tokens, s):
         some(pred).run(tokens, s)
         return parser.run(tokens, s)
 
     return _parse_if
+
+
+__all__ = [
+   'FORM', 'SYM', 'KEYWORD', 'STR', 'LITERAL',
+   'sym', 'keepsym',
+   'whole',
+   'brackets', 'in_tuple', 'braces', 'pexpr',
+   'dolike', 'notpexpr',
+   'unpack', 'times',
+   'Tag', 'tag', 'parse_if']

hy/model_patterns.py

@@ -104,36 +104,17 @@ def __hash__(self):
 
 
 def as_model(x):
-    """Recursively promote an object ``x`` into its canonical model form.
+    """Convert ``x`` and any elements thereof into :ref:`models <models>`
+    recursively. This function is called implicitly by Hy in many situations,
+    such as when inserting the expansion of a macro into the surrounding code,
+    so you don't often need to call it. One use is to ensure that models are
+    used on both sides of a comparison::
 
-    When creating macros its possible to return non-Hy model objects or
-    even create an expression with non-Hy model elements::
+      (= 7 '7)                ; => False
+      (= (hy.as-model 7) '7)  ; => True
 
-       => (defmacro hello []
-       ...  "world!")
-
-       => (defmacro print-inc [a]
-       ...  `(print ~(+ a 1)))
-       => (print-inc 1)
-       2  ; in this case the unquote form (+ 1 1) would splice the literal
-          ; integer ``2`` into the print statement, *not* the model representation
-          ; ``(hy.model.Integer 2)``
-
-    This is perfectly fine, because Hy autoboxes these literal values into their
-    respective model forms at compilation time.
-
-    The one case where this distinction between the spliced composit form and
-    the canonical model tree representation matters, is when comparing some
-    spliced model tree with another known tree::
-
-       => (= `(print ~(+ 1 1)) '(print 2))
-       False  ; False because the literal int ``2`` in the spliced form is not
-              ; equal to the ``(hy.model.Integer 2)`` value in the known form.
-
-       => (= (hy.as-model `(print ~(+ 1 1)) '(print 2)))
-       True  ; True because ``as-model`` has walked the expression and promoted
-             ; the literal int ``2`` to its model for ``(hy.model.Integer 2)``
-    """
+    It's an error to call ``hy.as-model`` on an object that contains itself, or
+    an object that isn't representable as a Hy literal, such as a function."""
 
     if id(x) in _seen:
         raise HyWrapperError("Self-referential structure detected in {!r}".format(x))