Fix per symbol first set computation (#6)

Fix per symbol first set computation (Spec._firstSets()) for a non-epsilon
production to omit epsilon unless all RHS symbols' first sets contain epsilon.
Prior to this fix, epsilon was merged into the first set under the weaker
condition that a prefix of RHS contained epsilon in its first set. Note that
computeFirstSet(), which computes the first set for symbol strings (e.g. of RHS
prefixes during item set closure computation), does not contain this flaw.

In order for epsilon to be in a symbol's first set, the entire syntax subtree
rooted at the symbol must be capable of being constructed without containing any
tokens. The simplest case is for the symbol to have an epsilon production; the
more involved case (that this change fixes) is for the symbol to reduce one or
more RHS symbols which transitively contain only epsilon-reduced symbols at the
leaves.

Impacts:

- Per symbol first sets could erroneously contain epsilon.
- Per symbol follow sets could erroneously contain additional tokens. Follow
  sets are not used for any purpose other than logging.
- Parser tables were affected only if a symbol had multiple productions which
  could reduce on epsilon. But such productions would induce a parsing
  ambiguity, so it was impossible to for LR(1)-compatible parsers to be
  affected.

Author: jasone

parsing/automaton.py

@@ -1396,23 +1396,24 @@ def _firstSets(self) -> None:
             for name in self._nonterms:
                 nonterm = self._nonterms[name]
                 for prod in nonterm.productions:
-                    # Merge epsilon if there is an empty production.
-                    if len(prod.rhs) == 0:
-                        if not nonterm.firstSetMerge(epsilon):
-                            done = False
-
+                    mergeEpsilon = True
                     # Iterate through the RHS and merge the first sets into
                     # this symbol's, until a preceding symbol's first set does
                     # not contain epsilon.
                     for elm in prod.rhs:
-                        containsEpsilon = False
+                        hasEpsilon = False
                         for elmSym in elm.firstSet:
-                            if not nonterm.firstSetMerge(elmSym):
-                                done = False
                             if elmSym == epsilon:
-                                containsEpsilon = True
-                        if not containsEpsilon:
+                                hasEpsilon = True
+                            elif not nonterm.firstSetMerge(elmSym):
+                                done = False
+                        if not hasEpsilon:
+                            mergeEpsilon = False
                             break
+                    # Merge epsilon if it was in the first set of every symbol.
+                    if mergeEpsilon:
+                        if not nonterm.firstSetMerge(epsilon):
+                            done = False
 
     # Compute the follow sets for all symbols.
     def _followSets(self) -> None: