Support qmd as the output format

[alyst]

Description

The idea is that in the Quarto projects the source .qmd files may depend a lot on project/folder settings, include the code snippets etc.
It would be nice to generate .qmd files that are as standalone as possible:

• incorporate project and folder metadata settings into the frontmatter
• resolve {{<include ...>}} (and maybe other shortcodes as well)
• apply the filters

Sorry if this is a duplicate of another feature request -- it looks like something many people may ask, but I was not able to identify a specific request since the relevant keywords are too generic.

[cderv]

it looks like something many people may ask, but I was not able to identify a specific request since the relevant keywords are too generic.

I don't think a lot of people ask that. Can you share more about your use case?

What you asked could maybe be possible in a very narrow set of contexts where the Quarto project structure is used to render a single-file document. However, in a Website or Book Project context, for example, the .qmd file can't be standalone as it is part of a whole project. Some part of the output depends on the rest of the website.

Regarding applying filters, this is similar: Some filters rely on the Project context. Rendering a single document as a standalone doesn't trigger the same processing. Also, I don't see what a standalone document means with "apply filters". Filters are processing that applies between the source and the output - they can't really be "transformed" to be part of the source. Some very simple filters could probably, but not all.

As you see it as a common request, I am really interested in understanding more about your use case—why the need for something more standalone than the reproducibility that Quarto already offers?

Thank you!

[alyst]

I am really interested in understanding more about your use case—why the need for something more standalone than the reproducibility that Quarto already offers?

I think of a website that provides a gallery of downloadable interactive notebook examples.

Naturally, the original .qmd files for these notebooks will share common settings and code.
For the website these notebooks will be rendered into .html.
Quarto already supports adding ipynb rendering target that generates standalone Jupyter notebooks with includes resolved, filters applied etc.
The users can download the Jupyter notebooks and start using them without any modifications.

So the feature request is about supporting .qmd in the same manner.

[cderv]

Quarto already supports adding ipynb rendering target that generates standalone Jupyter notebooks with includes resolved, filters applied etc.

Are you referring to the "Including Notebook" feature from Manuscript ?
https://quarto.org/docs/manuscripts/components.html#including-notebooks

These notebooks will be rendered to an HTML notebook view, and will be linked from your manuscript webpage under “Notebooks”.

What happens here is that Quarto will leverage Pandoc's ability to render --to ipynb. The rendered notebook is included.

Is the aim to have a .qmd version of this notebook and not just .ipynb ?

[alyst]

Are you referring to the "Including Notebook" feature from Manuscript ?

I'm referring to "Including Other Formats" for the project of "website" type, not "manuscript".
I don't know how much related are the two features.

What happens here is that Quarto will leverage Pandoc's ability to render --to ipynb. The rendered notebook is included.

Thanks for clarifying how this is handled internally.
I have also tried quarto convert: it generates a ipynb notebook too, but the shortcodes are unresolved.

Is the aim to have a .qmd version of this notebook and not just .ipynb ?

Yes, as another option under "Other Formats" in the sidebar menu.

[cderv]

I'm referring to "Including Other Formats" for the project of "website" type, not "manuscript".

Thanks for clarifying. They are a bit different but similar - they both leverage format: ipynb, based on Pandoc's support to write IPYNB file as a target format (i.e. --to ipynb` and Pandoc's notebook support: https://pandoc.org/MANUAL.html#jupyter-notebooks

To be clear, this means that the ipynb linked in other formats is a rendered version of the Quarto source file. It is not the same as a quarto input file.

I have also tried quarto convert: it generates a ipynb notebook too, but the shortcodes are unresolved.

quarto convert is a way to move from .qmd source file to .ipynb source file for quarto render. This is different than quarto render test.qmd --to ipynb or quarto render test.ipynb --to ipynb.

With Quarto, an ipynb can be used as a format for the source document, AND can also be a format for a target document.

.qmd is a source format only! It contains syntax that is only for Quarto rendering. However, this is using Markdown syntax. So maybe what you are looking for is just to produce a Markdown text file as output?
Or are you looking for the source document itself? Quarto has a feature to include source document in a modal, or even as a link to download: https://quarto.org/docs/output-formats/html-code.html#code-tools

Is that closer to what you are looking for?

I hope I have helped clarify the current situation. If this is still not covered, please keep explaining to me your expected usage of this file. What would a reader of your website do with this .qmd file it could download ?

[alyst]

.qmd is a source format only! It contains syntax that is only for Quarto rendering. However, this is using Markdown syntax. So maybe what you are looking for is just to produce a Markdown text file as output?
What would a reader of your website do with this .qmd file it could download ?

The goal is to provide .qmd file in the "Other formats" section that the user can download and run quarto render <qmd-file> locally on their computer without any preliminary tweaks.
And then the user can start modifying the code & text in the notebook to their needs.
So just the Markdown output will not work.

As we discussed, for the ipynb format the goal is already achieved -- the code in the notebook does not require any preliminary configuration of the execution environment.
The user can download the ipynb notebook, it will work out-of-the-box.

For .qmd the problem is that, taken out of the project, it lacks frontmatter config options that were specified in _quarto.yml or _metadata.yml and contains {<<include >>} shortcodes instead of the actual code.

Naively, I was thinking that Quarto pipeline loads .qmd, creates its AST representation and then applies a series of transformations -- merges metadata from the parent folders, resolves shortcodes etc.
So there should be a way to output this transformed AST back into .qmd file.

[cderv]

Naively, I was thinking that Quarto pipeline loads .qmd, creates its AST representation and then applies a series of transformations -- merges metadata from the parent folders, resolves shortcodes etc.
So there should be a way to output this transformed AST back into .qmd file.

Unfortunately, that is not that simple.

Such a feature would need to be something like --to ipynb, which would be a different IPYNB file from the one you would provide to quarto render.
It would require Pandoc's writers to produce a markdown output, but with the code cell unprocessed and untouched, I think.

So a .qmd file with no other Quarto feature that computation cell to run - if I do a parallel to what the --to ipynb is doing.

This is also highly hypothetical because .ipynb is a specific format that contains much more information than a .qmd simple text file. For example, a .ipynb can contain source cell information, and also its results, and the Jupyter kernel information + other metadata.

Possibly, we could try to improve the code-tools feature to maybe resolve the includes shortcode in the linked source 🤔

Thanks for the discussion !