Improve error messages

[gilch]

One of the primary goals of Hissp is to have good error messages.

There are multiple stages to compilation and errors could happen in any of them, which makes debugging more difficult unless you really know what you are doing.

1. read time
   a. lexing to tokens
   b. parsing to hissp
   c. reader macros
2. compile time
   a. compiling to python
   b. compiler macros
   c. incremental execution
3. runtime

Currently the compile time error messages are pretty long. I've found them to be usable, but as the author of the compiler, I'm also the world's foremost expert on Hissp. My experience may not be typical of users, who may find it difficult to find the relevant information in all that noise.

Hissp has no line numbers to speak of. There's nowhere we could put the metadata. Line numbers wouldn't even make sense in readerless mode but the error message still needs to be good enough to point to the problem.

So if there's an error compiling a form, the error message will point to the exact subexpression that failed to compile by listing all surrounding forms. Really, we only need to print off the top level form and highlight the subform somehow. The rest is pretty much useless noise.

Once we've got the Python output, normal Python debugging and stack traces work. This part isn't too bad because Python has very good error messages, but it will point to the very unpythonic compiled code. This part could perhaps be improved with source maps, but it may take some compiler enhancement to implement them, perhaps by outputting special comments. Then do we map to just the Hissp tuples, or to the skin on top of that, like Lissp? Maybe not a high priority.

Currently the most difficult part is if there is an error during incremental execution (2.c). The error happens in the compiled output, but it hasn't actually been saved yet. Perhaps the output file could be opened in append mode and incrementally updated with each form before its compile-time execution?

[gilch]

Hissp evaluates all top-level forms at compile time, which makes it behave more consistently with the REPL. Emacs Lisp requires an opt-in with an (eval-and-compile). In contrast, Hissp is opt-out using a check if the module __name__ is not "<compiler>" instead. Disabling a compile-time evaluation of a problematic form is certainly a workaround that can at least generate a file to debug.

[gilch]

#38 and #40 should pretty well cover phase 2.

The lexer and parser could maybe do a better job of pointing out the location of problems. A proper Jupyter kernel/REPL #30 would also need to know when the input string is not (yet) in valid Lissp format.

[gilch]

The new REPL and incremental lexer from #62 have much improved read-time error messages.

[gilch]

I feel like there's still room for improvement in the reader error messages. It might need a trace decorator like the compiler.

[gilch]

#221 improves 3.b. compiler macros. It also adds the compiled-Hissp-in-filename trick to the REPL to improve the traceback and debugger experience. It honestly does better than native Python in this regard. (If Python doesn't have a source file, it doesn't fall back to showing you the bytecodes. It's just invisible.)

[gilch]

I'm finding the verbosity of the compiled-Hissp-in-filename trick in the REPL a bit annoying, now that I have it. I think I'd still rather have it than not though. I don't think there's much I can do to make it better short of removing it completely. It's more annoying with macros, especially long macros (like synexpand), which is probably not the usual situation, so maybe it's not as bad as it seems. Reloaded modules have an actual Python file to point to, and so don't have this problem.