Derive macros

[osa1]

TODO: Parse macro output and add it to the module.
TODO: Add tests.

[osa1]

Good stuff, but if I add a #[derive(...)] in the standard libs (actually, in any dependency of the derive macro), it causes recursion.

If the compiler was ready we would compile the derive macro with the current standard libraries, then add derives to prelude and recompile.

For now maybe we could only use derives in the compiler code, and once it's running start adding them to the standard libs.

For now we could manually copy/paste generated code in the standard libraries.

What matters is that we shouldn't be hand-rolling Eq and ToStr in the compiler code. We have many types of AST nodes, tokens etc. it's a lot of tedious, repetitive, error-prone work.

[osa1]

I think this is likely fine for now.

We will eventually need some kind of "procedural macro" that can generate anything (types, functions, modules, maybe even packages) anyway, we can start with derive macros.

Some (maybe most) of the macros will take time to run, and because they generate types and functions that can be used in the rest of the program, naively, the type checker will need the output before fully type checking the program.

That's not ideal in language servers. We should have the features in place to avoid this.

Some ideas:

• Macros should declare their dependencies and the front-end shouldn't run a macro unless one of the dependencies change in some way.

  For derive macros: this would mean that the front-end will never run derive macros on a type unless the type changes. (textually for now, then we can improve in the future with smarter change detection based on the parse tree or similar)

• Even better: macros should quickly return the types they generate and the type signatures of the functions (if any) they generate.

  The idea is that, whenever possible, we want to be able to type check (and provide other language server functionality) without having to actually run the macros.

  We can have a simple (but somewhat hacky) way to do this by just passing a flag to the macro functions indicating whether we're type checking or compiling. It would then be the macro's responsibility to generate function bodies with e.g. panic("Not generating code") so that it won't slow down type checking.

  Ideally though, I think the macro API should have a way to declare generated things based on inputs.

The principle is: type checking and other LSP functionality should still be response with macros. Do whatever it takes to make sure this is the case (without getting rid of macros, they're useful).

An open question is what kind of introspection to allow in macros. In my use cases (mainly in Rust, but with TemplateHaskell in the past as well) I've never needed anything more than the AST passed to the macro. That works for: generating serializers (with annotations on fields), implementing Eq, Hash, and similar simple traits. We should look at what kind of introspection other languages provide, and why introspection was needed and how it's used.

[osa1]

Re: introspection, I think I found one of the reasons why some of the other languages need it.

Consider this Dart class:

class Foo extends Bar { ... }

This top-level definition is not self-contained: some of the definition is in Bar.

So if I add something like Rust's derive macros:

@derive('Blah')
class Foo extends Bar { ... }

and pass the AST of the class Foo extends Bar { ... } to the macro, it won't have the entire definition of the type.

We don't have this problem in Rust and Fir (at least yet). A derive macro gets the whole type definition.

In principle there could be use cases where you have to look at the types in the fields, but I'm not aware of any such use cases.

[osa1]

Another use case for introspection is when you have generate code not in a macro that wraps a definition that you use, but somewhere else.

For example, instead of

#[derive(ToJson)]
type Foo: ...

you do something like (not real Fir syntax)

fooToJson(foo: Foo) Json:
    @deriveToJson(Foo)(foo)

(where @deriveToJson(...) is a macro invocation, Foo passes the type definition to the macro)

(This could be in another library than Foo)

I don't know if we need this kind of thing. My current thinking is: if Rust doesn't need this then we also probably don't.

[osa1]

I've merged the Fir code to derive Eq impls to main, so this branch now only contains the #[derive(...)] syntax and calling the derive macros.

[osa1]

Another concern here is that this will make bootstrapping much more difficult as the compiler will have to support derive macros from day 0.

I think we should copy-paste generated code for now.