Remove HTML from text with rehype-retext?

[kentcdodds]

Hi friends.

This is what I've tried:

async function stripHtml(htmlString: string) {
  const {unified} = await import('unified')
  const {default: parseHtml} = await import('rehype-parse')
  const {default: rehypeRetext} = await import('rehype-retext')
  const {default: retextStringify} = await import('retext-stringify')
  const result = await unified()
    .use(parseHtml)
    .use(rehypeRetext)
    .use(retextStringify)
    .process(htmlString)

  return result.value.toString()
}

async function go() {
  const result = await stripHtml('<p>Hello</p>')
  console.log(result)
}

void go()

But I'm getting this:

file:///Users/kentcdodds/code/remix-kentcdodds/node_modules/hast-util-to-nlcst/lib/index.js:90
    throw new Error('hast-util-to-nlcst expected parser')
          ^

Error: hast-util-to-nlcst expected parser
    at toNlcst (file:///Users/kentcdodds/code/remix-kentcdodds/node_modules/hast-util-to-nlcst/lib/index.js:90:11)
    at file:///Users/kentcdodds/code/remix-kentcdodds/node_modules/rehype-retext/index.js:49:26
    at wrapped (file:///Users/kentcdodds/code/remix-kentcdodds/node_modules/trough/index.js:111:16)
    at next (file:///Users/kentcdodds/code/remix-kentcdodds/node_modules/trough/index.js:62:23)
    at Object.run (file:///Users/kentcdodds/code/remix-kentcdodds/node_modules/trough/index.js:33:5)
    at executor (file:///Users/kentcdodds/code/remix-kentcdodds/node_modules/unified/lib/index.js:321:20)
    at Function.run (file:///Users/kentcdodds/code/remix-kentcdodds/node_modules/unified/lib/index.js:312:5)
    at executor (file:///Users/kentcdodds/code/remix-kentcdodds/node_modules/unified/lib/index.js:393:17)
    at new Promise (<anonymous>)
    at Function.process (file:///Users/kentcdodds/code/remix-kentcdodds/node_modules/unified/lib/index.js:380:14)

I'm not sure what I need to do to get this to work 🙃 Based on what I understand of unified you give it a parser (rehype-parse), some transforms (rehype-retext), and then a stringifier (retext-stringify) and you should get a string out. Do I maybe have a version mismatch somewhere?

[wooorm]

Yeah, I believe that should work fine, I think you found a bug! let me see

[wooorm]

Ah, no, you’re missing a natural language parser:

import {unified} from 'unified'
import rehypeParse from 'rehype-parse'
import rehypeRetext from 'rehype-retext'
import {Parser as English} from 'retext-english'
import retextStringify from 'retext-stringify'

unified()
  .use(rehypeParse)
  .use(rehypeRetext, English)
  .use(retextStringify)
  .process('<h1>hello')
  .then((file) => {
    console.log(String(file))
  })

Yields:

hello

Or:

import {ParseEnglish} from 'parse-english'
...
  .use(rehypeRetext, ParseEnglish)

(which is exactly the same)

The reason is that natural language (retext) could be English, Dutch, or something else (other ones aren‘t supported, except that retext-latin parses most Latin-script languages okay-ish).

This is documented somewhat in rehype-retexts readme: destination does not have a ? in the API signature, and the prose mentions that something must be given.

Would showing an example in the readme perhaps help?

---

Note: the approach you’re taking here is more useful for checking spelling and more complex natural language linting (some more inspiration: https://github.com/unifiedjs/unifiedjs.github.io/blob/25d55ecc50968f4c0d5dadb12a13978283ef321b/.remarkrc.js#L27-L37). but if you just want plain text, then the utilities hast-util-to-text and hast-util-to-string perform the DOMs innerText and textContent algorithms, which are simpler and much faster

[kentcdodds]

Ah, yes, those do look much more like what I want. Thanks!

[wooorm]

Good, and good luck!

I did add an improved error message for future users that would try the same.
And I’m thinking of adding sections to most readmes in the future on “When (not) to use this” with alternatives, something like that might’ve helped in this case!