Write unix man pages in markdown and convert?

[torognes]

I find the unix man format (roff) difficult to write, but I think it is very nice to have man pages handy. Perhaps we could write the documentation in markdown instead and convert it? There are tools for that, for instance ronn and pandoc.

[frederic-mahe]

I also think we should continue to maintain man pages, but I also think we should try to add documentation as close to the binary as possible. For vsearch, it would be interesting and clearer for users to have a --help output for each vsearch sub-commands (vsearch --uchime_denovo -h for example).

But I digress. The unix man format (roff) is difficult to write, but I grew found of it. Maybe it would help if I describe the most current tricks and commands?

• .SH SYNOPSIS declare a section header title (here "SYNOPSIS"),
• .SS SUBSECTION declare a subsection title (here "SUBSECTION"),
• .RS increase indentation level,
• .RE decrease indentation level,
• .TP 9 declare a block with a definition followed by an indented text,
• .B text declare a block starting with a bold (B) text,
• .BI text1 \0text2 declare a block starting with a bold (B) text, followed by an italic (I) text, \0 separates the two styles (bold and italic),
• \fBtext\fR set "text" to bold (B) and go back to Roman (R),
• \fItext\fR set "text" to italics (I) and go back to Roman (R),
• .PP declare a new paragraph (add a space),
• .nr step 1 1 in a numbered list, start numbering at 1, increment by 1,
• .IP \n[step]. 4 item in numbered list, use step's value, use a space of size 4 between the item number and the text in the item.

(try man 7 groff for more information)

I can also add comments to the vsearch.1 file to explain the different commands, and how to make a table or a numbered list.

[frederic-mahe]

Earlier this year, I tried to write a manpage in markdown format. It is quite promising, but also very time consuming.

If I find time to do it (summer 2024?), the goal is to break down the monolithic vsearch.1 manpage into smaller files, one markdown file per vsearch command (very similar to what git does, see for example man git-clone). I think it can be done in a progressive manner, and does not require to be entirely complete before being released.

The downside is that it creates a dependency to a third-party tool to convert from markdown to roff.

[frederic-mahe]

I've finally produced a small example of three manpages for vsearch commands (--fastq_chars, --fasta2fastq, --orient), written in (pandoc-)markdown (see commit b6fc6a3).

@torognes @colinbrislawn you both have expressed interest for an alternative to groff. Do you think the generated markdown files could be used to build an online linkable manual?

The goal is to address limitations of the current monolithic manpage:

• page is way too long,
• some parts are redundant,
• some parts are not developed enough,
• some commands don't have a list of available options,
• groff is hard (old-school format) (Write unix man pages in markdown and convert? #269),
• current format not compatible with the web (Feature request: online linkable manual #280)

The proposed solution is to write manpages in markdown and to convert
them with pandoc. A short manpage for vsearch (dispatch), and a
detailed manpage for each vsearch command (as described in #340).

Pros:

• easier format (pandoc-markdown),
• easy manpage generation,
• eliminates most text duplication (factorize redundant sections),
• usable online? (put generated markdown files on a website?)
• possibility to have man(7) documentation pages (e.g.; fastq format,
  SHA1/MD5 hashing, quality values and offset, etc.)

Cons:

• adds new dependencies (perl and pandoc),
• pandoc-markdown, not vanilla markdown,
• slightly less expressive than groff, but that should be ok

Preview:

git pull
git checkout markdown2manpage
cd ./man/commands/
bash visualize_manpage.sh vsearch-orient.1.md

[colinbrislawn]

That's fantastic!

Zooming out a bit, there are many ways to do this, which means there is no 'one right way' to do this.

usable online? (put generated markdown files on a website?)

Yes! This is called static site generation, and I've done it a few times.
We host a seperate, small codebase, that has all the static site generation code, say in a special branch. We set up GitHub actions to generate/regenerate the site whenever changes are made to the core vsearch codebase and docs.

I will leave the decision of how to get make the .md files up to you!
I was assuming we would go
whatever we have now -> md -> static site gen with GitHub actions -> website

It looks like md2man was last updated in 2018, which either means it dead or stable.

[torognes]

Sounds very nice! But I cannot find the visualize_manpage.sh script. Where can I find that? Is it part of pandoc?

[frederic-mahe]

Sounds very nice! But I cannot find the visualize_manpage.sh script. Where can I find that? Is it part of pandoc?

My mistake, I forgot to include the scripts.

I've explored that idea of building a static website from our markdown pages. It seems that GitHub Pages now does most of the job:

https://mryap.github.io/markdown-to-github-pages/

I'll try to come up with a working pipeline tomorrow.