Skip to content

Conversation

avivkeller
Copy link
Member

@avivkeller avivkeller commented Sep 1, 2025

Fixes #7932


Introduces @shikijs/twoslash support in @node-core/rehype-shiki. Since twoslash depends on TypeScript declaration files, it cannot be bundled and must instead be listed under serverExternalPackages.

Because TypeScript declarations aren’t imported like regular files (they’re loaded through a TSVFS), they need to be explicitly included in the output bundle.

Because Cloudflare doesn't support WASM or Top-Level awaits, rehype-shiki has been rewritten to not rely on them.

@avivkeller avivkeller self-assigned this Sep 1, 2025
Copy link

vercel bot commented Sep 1, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Updated (UTC)
nodejs-org Ready Ready Preview Sep 6, 2025 9:03pm

Copy link

codecov bot commented Sep 1, 2025

Codecov Report

❌ Patch coverage is 45.85987% with 85 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.41%. Comparing base (c10ebc1) to head (285ed69).

Files with missing lines Patch % Lines
packages/rehype-shiki/src/index.mjs 11.25% 71 Missing ⚠️
packages/rehype-shiki/src/plugin.mjs 76.74% 10 Missing ⚠️
packages/rehype-shiki/src/highlighter.mjs 85.71% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8132      +/-   ##
==========================================
- Coverage   76.59%   76.41%   -0.19%     
==========================================
  Files         115      115              
  Lines        9602     9621      +19     
  Branches      322      316       -6     
==========================================
- Hits         7355     7352       -3     
- Misses       2246     2268      +22     
  Partials        1        1              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@AugustinMauroy
Copy link
Member

Hype 😁 !

@avivkeller
Copy link
Member Author

Locally:
image

@avivkeller avivkeller marked this pull request as ready for review September 4, 2025 21:19
@Copilot Copilot AI review requested due to automatic review settings September 4, 2025 21:19
@avivkeller avivkeller requested review from a team as code owners September 4, 2025 21:19
@avivkeller avivkeller changed the title [WIP] feat(shiki): add twoslash support feat(shiki): add twoslash support Sep 4, 2025
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds TypeScript TwoSlash support to the syntax highlighting system, enabling interactive TypeScript error displays and type information tooltips in code blocks.

Key Changes:

  • Integrated TwoSlash transformer for TypeScript/JavaScript code highlighting
  • Refactored metadata parsing system from string-based to object-based approach
  • Added CSS styling for TwoSlash popup containers and error displays

Reviewed Changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
packages/rehype-shiki/src/index.mjs Added TwoSlash transformer import and configuration
packages/rehype-shiki/src/highlighter.mjs Updated functions to accept metadata parameter and pass transformers
packages/rehype-shiki/src/plugin.mjs Replaced string-based metadata parsing with comprehensive object-based system
packages/rehype-shiki/src/twoslash.css Added CSS styling for TwoSlash popup containers
packages/rehype-shiki/package.json Added TwoSlash dependency and exported CSS file
packages/ui-components/src/Common/BaseCodeBox/index.module.css Added CSS transforms for proper positioning
apps/site/next.config.mjs Configured external packages and file tracing for TwoSlash
apps/site/package.json Added twoslash dependency
apps/site/styles/index.css Imported TwoSlash CSS styles
apps/site/pages/en/learn/typescript/transpile.md Updated TypeScript example to use TwoSlash error annotations
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@jakebailey
Copy link
Member

Is this all server side?

I'm asking because given the TS port to Go, I'm not sure what the deal will be for shiki on the web; probably having to include a large Go binary in Wasm to work with all of the fancy stuff if not prerendered...

@avivkeller
Copy link
Member Author

avivkeller commented Sep 4, 2025

Is this all server side?

I'm asking because given the TS port to Go, I'm not sure what the deal will be for shiki on the web; probably having to include a large Go binary in Wasm to work with all of the fancy stuff if not prerendered...

All server-side, yes. We don't pass the complicated WASM binaries and Shiki configurations to the client. The client gets a basic syntax highlighter w/ Shiki's RegEx engine.

@avivkeller
Copy link
Member Author

avivkeller commented Sep 4, 2025

@dario-piotrowicz Does this setup not support OpenNext? If not, we can make it conditional, like we do for the WASM engine

@AugustinMauroy
Copy link
Member

image

The box have a light mode but it's shouldn't since the shiki theme is always dark so the contrast isn't good

@avivkeller
Copy link
Member Author

@AugustinMauroy fixed

Copy link
Member

@AugustinMauroy AugustinMauroy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow soo cool

@@ -88,31 +89,4 @@ const justine: User = {
const isJustineAnAdult: string = isAdult(justine, "I shouldn't be here!");
```

And this is what TypeScript has to say about this:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is kind of useful because it's the output that the reader/user will see in this terminal

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file is minimified during publish process ?

Also should add docs somewhere about why here we don't use tailwind.

@canerakdas
Copy link
Member

On the line where the error messages are displayed, the scrolled section looks cut off. Additionally, the text and background color used here don’t seem to have sufficient contrast ratio
image

The popover that’s supposed to display the types on horizontally scrollable sections doesn’t open in the correct position
image

For some colors, the contrast ratio doesn’t meet the requirement. The simplest solution might be to use the same background color as the codebox
image

Copy to clipboard also copies the types into the clipboard, for example;

// server.mjs
import { function createServer<Request extends typeof IncomingMessage = typeof IncomingMessage, Response extends typeof ServerResponse = typeof ServerResponse>(requestListener?: RequestListener<Request, Response>): Server<Request, Response> (+1 overload)createServer } from 'node:http';

const const server: Server<typeof IncomingMessage, typeof ServerResponse>server = createServer<typeof IncomingMessage, typeof ServerResponse>(requestListener?: RequestListener<typeof IncomingMessage, typeof ServerResponse> | undefined): Server<...> (+1 overload)createServer((req: IncomingMessagereq, res: ServerResponse<IncomingMessage> & {
    req: IncomingMessage;
}res) => {
  res: ServerResponse<IncomingMessage> & {
    req: IncomingMessage;
}res.ServerResponse<IncomingMessage>.writeHead(statusCode: number, headers?: OutgoingHttpHeaders | OutgoingHttpHeader[]): ServerResponse<IncomingMessage> & {
    ...;
} (+1 overload)writeHead(200, { 'Content-Type': 'text/plain' });
  res: ServerResponse<IncomingMessage> & {
    req: IncomingMessage;
}res.Stream.Writable.end(chunk: any, cb?: () => void): ServerResponse<IncomingMessage> & {
    req: IncomingMessage;
} (+2 overloads)end('Hello World!\n');
});

// starts a simple http server locally on port 3000
const server: Server<typeof IncomingMessage, typeof ServerResponse>server.Server.listen(port?: number, hostname?: string, listeningListener?: () => void): Server<typeof IncomingMessage, typeof ServerResponse> (+8 overloads)listen(3000, '127.0.0.1', () => {
  var console: Consoleconsole.Console.log(message?: any, ...optionalParams: any[]): void (+1 overload)log('Listening on 127.0.0.1:3000');
});

// run with \`node server.mjs\`

Lastly, we also need to look at how this impacts page performance.
When checking the output HTML file sizes for our home page, I see:

Nodejs.org: 175.73 KB
Preview: 228.79 KB

It might be good to run a comparison with @next/bundle-analyzer to see how much load this adds to the pages 🤔

@avivkeller
Copy link
Member Author

avivkeller commented Sep 6, 2025

Lastly, we also need to look at how this impacts page performance.

When checking the output HTML file sizes for our home page, I see:

Nodejs.org: 175.73 KB

Preview: 228.79 KB

It might be good to run a comparison with @next/bundle-analyzer to see how much load this adds to the pages 🤔

It's not supposed to be passed to the client at all, so we should definitely investigate that

@avivkeller avivkeller marked this pull request as draft September 6, 2025 21:01
@avivkeller
Copy link
Member Author

The popover that’s supposed to display the types on horizontally scrollable sections doesn’t open in the correct position

Copy to clipboard also copies the types into the clipboard

Fixed in the latest commit, which introduced some styling issues (regarding wrapping).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Type Info on the Docs & Guides
4 participants