Update web formatter and playground to support formatting code blocks in markdown

[dsherret]

Title is specific, but generally update the web formatter and playground to support formatting with other plugins from within a plugin.

[tamird]

+1, it's not clear how (or if it's possible) to use this to format code blocks in markdown.

[rattrayalex]

@dsherret (or other maintainers), would you accept a PR for this and do you have a sense of how a contributor could get started on it / how much work it might be?

[dsherret]

For sure, that would be great! It wouldn't be a ton of work. Essentially, the js formatter would need a way to have multiple plugins registered and determine which plugin to format with based on the extension of the path (which is found in the PluginInfo:

js-formatter/mod.ts

Line 63 in 75182af

fileExtensions: string[];

) or the file names (

js-formatter/mod.ts

Line 64 in 75182af

fileNames: string[];

) of the path given. Then the wasm import would need to be implemented (

js-formatter/mod.ts

Line 72 in 75182af

export function createImportObject(): WebAssembly.Imports {

) which would call into another plugin in order to format with it. The low level documentation is here: https://github.com/dprint/dprint/blob/main/docs/wasm-plugin-development.md#wasm-imports (the shared buffer that communication happens on is what this returns, so the code needs to be restructured somewhat:

js-formatter/mod.ts

Line 171 in 75182af

get_wasm_memory_buffer,

)

Let me know if you have any questions on this one.

[rattrayalex]

Awesome, thanks @dsherret !

[tamird]

@dsherret can you clarify how the steps you've outlined would allow the markdown plugin to format code blocks?

Essentially, the js formatter would need a way to have multiple plugins registered and determine which plugin to format with based on the extension of the path

There wouldn't be a path for each code block in this case, right?

However as I write this I'm realizing we can do this dispatch ourselves by looking for code blocks and formatting their contents before or after formatting the markdown.

[dsherret]

dprint-plugin-markdown looks at the tag of the code block and converts it to a fake file path (ex. ts tag to file.ts). Then it calls those wasm imports to see if the host (ex. dprint's CLI or in this case the JS code) can format that file path and if so then it sends over the file text to format.

However as I write this I'm realizing we can do this dispatch ourselves by looking for code blocks and formatting their contents before or after formatting the markdown.

This solution of using the wasm imports is general and benefits more than just dprint-plugin-markdown. For example, the dprint-plugin-jupyter plugin formats jupyter notebook codeblocks the same way.

[ilanashapiro]

Hi @dsherret! I am working on this issue and I am a bit confused on what you mean by needing to implement the wasm import. Am I essentially needing to rewrite the import functions from https://github.com/dprint/dprint/blob/5813e59f3d4804f794aa3acbac1e9e41467d1ff7/crates/dprint/src/plugins/implementations/wasm/import_object.rs in TypeScript? I also looked at the format_with_host function the dprint-plugin-jupyter plugin calls (https://github.com/dprint/dprint/blob/main/crates/core/src/plugins/wasm/mod.rs#L35C10-L35C26) -- it appears that the jupyter plugin exclusively relies on format_with_host to do the formatting (https://github.com/dprint/dprint-plugin-jupyter/blob/main/src/format_text.rs#L103), rather than calling another plugin first. Would I similarly need to reproduce this function in TypeScript for js-formatter? Right now, I'm able to register two (nested) plugins, where the import object of the outer plugin contains exports of the inner plugin, but I'm not sure that's what you meant.

Finally, can we assume we'd be able to access the tags of the code blocks in the input text indicating which plugin to format the input text with? Currently, the examples in js-formatter are not marked up with tags (which is something I can certainly change). Thank you so much for your help!

[dsherret]

it appears that the jupyter plugin exclusively relies on format_with_host to do the formatting (...), rather than calling another plugin first.

It's important to note that plugins have no knowledge of other plugins. The host formatting is how it calls another plugin. It asks the host (CLI/JS formatter) to format and then the host decides what plugins to format with.

So in this case, the JS formatter should hold a list of plugins and when it formats the plugin may ask the JS formatter (host) to format with another plugin. The JS formatter must then decide how to format it and sends back the result.

Am I essentially needing to rewrite the import functions from ...

Yes, but it will be slightly different here. There is an overview of the low level interface here: https://github.com/dprint/dprint/blob/main/docs/wasm-plugin-development.md#wasm-imports

These are the low level functions that a plugin uses to call back to the host (JS formatter) in order for the host to decide what plugin to format the code with.

Finally, can we assume we'd be able to access the tags of the code blocks in the input text indicating which plugin to format the input text with?

dprint-plugin-markdown and dprint-plugin-jupyter provide a filename of the code being formatted (ex. the markdown plugin converts a ts tag to a file name with .ts). That filename can be used to determine what plugin to format with.

[dsherret]

The low level ability to use host formatting landed in #11

Now the public API of this library just needs to change to support loading multiple plugins at the same time using the same formatter instance.

[g-plane]

IMHO it's hard to support multiple plugins under current playground architecture, but I would like to create a new dprint playground (in a separated repository).

[dsherret]

I think before any playground changes are done, there should be a higher level api here that allows passing a bunch of wasm plugins and it will format accordingly. That will make integration into the playground easier.

[g-plane]

It's unrelated to API. I mean the current UI of playground only allows us to select single plugin and the config editor is only for that single plugin. What we need to do is to allow to add or remove multiple plugins, and the config editor should look like dprint.json, not for single plugin.

[dsherret]

Yeah, for sure. It would be much better. It's only like the way it is because it's only supported one plugin at a time. I don't think it would be much work to change the existing playground to do that once the API improvement is done here.

[g-plane]

Should the high-level API be added to this package?