CoffeeScript 3.0: Modern JavaScript with blazing fast Solar Parser and AST data nodes

[shreeve]

🔧 Technical Details

Key Changes:

• Solar parser generator (solar.coffee) with revolutionary grammar transformation
• Data-oriented grammar in src/syntax.coffee using Solar directives
• ES6 backend (src/es6.coffee) with pre-pass variable analysis
• Renamed parser.js → parser-cs2.js for clarity
• CS3 uses unified cs3 flag internally (not separate es6 flag)

Solar Directive System:

• $ast - Create AST nodes
• $ary - Handle arrays/lists
• $ops - Apply operations
• $use - Reference other values
• $pos - Position tracking

🌟 Why This Matters

1. Future-Proof: The data-oriented AST can compile to WebAssembly, Go, Rust, or any future target
2. Modern by Default: ES6+ is now the baseline, not an option
3. Maintainable: Pure data transformations are easier to understand and modify
4. Fast: 100x faster parser generation makes development iteration rapid

🧪 Testing

All existing tests pass. The test suite confirms:

• Full backward compatibility
• Correct ES6 output generation
• Proper const/let usage
• Modern JavaScript features work correctly

cake test:cs2  # Traditional parser - 455 tests pass
cake test:cs3  # Solar parser - 455 tests pass

🎯 Migration Path

CS3 is designed for gradual adoption:

• Default coffee command still uses CS2
• Add --cs3 flag to opt into modern output
• Both parsers coexist in the same codebase
• No breaking changes to existing code

📝 Example

# Input
class User
  constructor: (@name) ->

greet = (user) -> 
  console.log "Hello, #{user.name}"

users = [new User('Alice'), new User('Bob')]
for user in users
  greet user

CS2 Output (with var and wrapper):

(function() {
  var User, greet, user, users, i, len;
  // ... traditional output
}).call(this);

CS3 Output (with const/let, no wrapper):

let user, users;
const User = class User {
  constructor(name) {
    this.name = name;
  }
};

const greet = function(user) {
  return console.log(`Hello, ${user.name}`);
};

users = [new User('Alice'), new User('Bob')];
for (user of users) {
  greet(user);
}

---

This is the most significant architectural change in CoffeeScript's history. The move to data-oriented AST with Solar directives represents a fundamental rethinking of how transpilers should work.

Breaking changes: None - full backward compatibility maintained.

[shreeve]

Sky's the limit on what output code can be generated.

I'm working on a totally modernized version of CoffeeScript with several new features and primary support for bun, but also running on deno, nodejs, and browsers.

This code shows what can be done and how a totally revamped parser (12.5 seconds down to under 100 ms) and AST data nodes can be used to make it possible to target not only ES6, but anything really from WASM to Rust, etc...

[cosmicexplorer]

I had to perform some incredibly tricky work to fix scoping to support block scopes for const/let correctly, as well as fix several miscompiles of top-level import scoping in #5475. I would be quite impressed if you had also solved the same thing (but please feel free to reuse my work).

I don't see how swapping out the parser is quite so revolutionary in itself. For example, see #5474, where I discuss quite thoroughly how doing more work in the parser actually makes our analysis less powerful.

I have been considering reimplementing bison's GLR parsing in rust-compiled wasm--this would not only produce the far more powerful and less frustrating GLR grammars, but also open up extension of parsing logic to any language that can compile to wasm. Making the parser faster is the easy part and not indicative of greater extensibility in the future.

Jison was just never a good dependency in the first place: #5479 (comment). Maintenance of a programming language requires really deep, long-term investment not captured by a single benchmark or any collection of benchmarks.

In particular, primary support for bun alone is terribly uninteresting to me. Decontextualized benchmarks (https://bun.com/docs/api/file-io) without any explanation of what bun does to make things faster are not very persuasive.

All possible permutations are handled using the fastest available system calls on the current platform.

This is word salad and very unserious language when talking about performance. How long has bun's support of the node fs API been "nearly complete"?

The rebar benchmarks from the rust regex crate (that's Andrew Gallant) are some of the only performance benchmarks I've ever seen that have any value whatsoever: https://github.com/BurntSushi/rebar/blob/master/README.md. And they're still hampered by failing to incorporate how regex engines are typically invoked within a larger logical scheme, i.e. multiple times in different ways over the same input.

Optimizing things is easy. I've made pip take <1 second to execute a full resolve thanks to metadata caching (pypa/pip#12921), and I'm now taking the time to standardize the cached output so that everyone else can rely on it too.

Performance is a highly contextual question. See my docs for my hyperscan wrapper: https://docs.rs/vectorscan-async/latest/vectorscan/state/index.html. Regular expression performance is highly dependent on usage patterns not codified in standard benchmarks. The fastest program is a NOP instruction, but that's not very useful. Performance while adhering to standards for interoperability--now that's interesting.

bun:ffi has known bugs and limitations, and should not be relied on in production

Disappointing. Performance is useless if it's not robust.

If bun wants to hire a runtime/compiler/toolchain engineer who can fix their performance and robustness issues, please tell them to contact me (dmc2 at hypnicjerk dot ai). But otherwise, parser performance is not really a sufficient excuse to call this "CS3".

I am hoping to create a shell-like language on top of CoffeeScript soon without any attempts at node js compatibility (process startup time is really easy to fix btw). I don't know that I would still call it CoffeeScript unless I was trying to preserve CoffeeScript pretty precisely, which it doesn't seem like you're doing. When I proposed an extension of the existing destructuring rules #5471, I was also urged to consider making a new language instead. That seems like a good idea to do here.

[GeoffreyBooth]

This PR seems like a product of a lot of AI coding without much review. Like I don't know why all of the GitHub files like CODE_OF_CONDUCT.md were deleted. The sloppiness doesn't inspire confidence, especially for a substantial refactor as proposed here.

If all the tests pass, and therefore there are no breaking changes introduced, then this would be a minor version bump on 2.x, not a new major version. Therefore this also wouldn't need a new flag to enable it, and there wouldn't be a reason to keep around the old parser and old flow.

What is the “Solar parser generator” and what benefits does it bring? I understand that the parser generation itself is much faster, but that only happens during development of CoffeeScript itself when people change the grammar. That's nice, but not a performance improvement that users would see.

[phil294]

I think you can even be more concrete than that: This PR is rude and disrespectful, and should be closed.

[shreeve]

@phil294 - Chill my friend. Cancel culture is on the way out...

There is a lot more coming in this patch, sorry for the AI comments and yes, I did use a ton for the initial work. Who wouldn't?

The next code to land here should be significantly cleaned up and on the path towards a much more polished version.

The use of solar as the parser generator not solely to save 12.4 seconds one time, but for a far more rich polyglot platform that supports CoffeeScript as one of many languages.

The AST data nodes piece is also significant, as it abstracts the front end parsing from back end code generation. This approach will support ES6 code generation as well as WASM, others...

Hold your horses on the hate, there hasn't been much progress since 2017 on coffeescript, it's not going to hurt your feelings if some new work is started.

Nobody has to inline this code ever or even tomorrow... consider it some work done by a community member to try to see if new life can be breathed into a great language.

I'll update the PR as time progresses, don't get your panties up in a bunch -- realize that your comment was as rude and disrespectful to me. So, calm down, get a gatorade and relax.