Skip to content

Conversation

junhaoliao
Copy link
Member

@junhaoliao junhaoliao commented Aug 19, 2025

Description

Add Cache-Control headers for static files to reduce network traffic

  • Configure no-cache for specific files (e.g., index.html, settings.json)
  • Set long-term caching for all other static files
  • Improve caching strategy for both log viewer and client directories

Checklist

  • The PR satisfies the [contribution guidelines][yscope-contrib-guidelines].
  • This is a breaking change and that has been indicated in the PR title, OR this isn't a
    breaking change.
  • Necessary docs have been updated, OR no docs need to be updated.

Validation performed

Below test was done without the changes in #1234

Stress test

  1. Built and started the package
    task package
    cd build/clp-package/sbin
    ./start-clp.sh
    
  2. Visited http://localhost:4000 in Microsoft Edge Version 139.0.3405.102 (Official build) (64-bit)
  3. Clear browser cache for the site:
    1. F12 to open Developer Console in the browser.
    2. Right-clicked on the "Refresh" button on the left of the browser address bar
    3. Clicked on "Empty cache and hard refresh"
  4. Idled for 1 minute.
  5. Kept refreshing the http://localhost:4000/ingest page at a rate of roughly twice a second. After roughly 30 seconds, observed "Rate limit exceeded, retry in 30 seconds".

Inspect headers

Idled for 1 minute so that the rate limit recovered from the above stress test.

The Web UI SPA

  1. Opened Developer Console in Microsoft Edge Version 139.0.3405.102 (Official build) (64-bit). Switched to the "Network" tab in the Developer Console.
  2. Visited http://localhost:4000/ingest
  3. Observed most requests were loaded from the cache:
    image
  4. ... except requests for "ingest", "settings.json", "sql" (ignored "content-fontface-intercept.js" because that's caused by a Chrome extension i installed). Inspected the requests and observed "Cache-Control: public, max-age=0" was set for those.
  5. Visited http://localhost:4000/ (a different path from http://localhost:4000/ingest though index.html is also served) and repeated Step 4 to ensure the no cache header was set.

The log viewer SPA

  1. Compressed sample logs: clp-package/sbin/compress.sh ~/sample/hive-24hr
  2. Visited http://localhost:4000/search and performed a search with query string "1". Observed results were shown.
  3. Clicked on any "Open file" link in any search result. The log viewer showed up.
  4. Clicked the Refresh button on the left of the browser address bar to refresh the page.
  5. Opened Developer Console. Switched to the "Network" tab in the Developer Console.
  6. Observed most requests were loaded from the cache:
    image
  7. ... except requests for "index.html?filePath=/streams/...", "dc4314df-6284-4555-824a-548de0932ebe_0_641.clp.zst" (ignored "content-fontface-intercept.js" because that's caused by a Chrome extension i installed). Inspected the requests and observed "Cache-Control: public, max-age=0" was set for those.

Summary by CodeRabbit

  • New Features

    • Enhanced cache-control for static assets: long-term caching for files like JS/CSS/images, with index.html and settings.json always served fresh.
    • Faster repeat page loads while ensuring users receive up-to-date app shell and configuration.
  • Bug Fixes

    • Prevents stale content after deployments, reducing issues with outdated pages or settings.

@junhaoliao junhaoliao requested a review from a team as a code owner August 19, 2025 15:47
Copy link
Contributor

coderabbitai bot commented Aug 19, 2025

Walkthrough

Implements explicit Cache-Control handling for static assets in the web UI server. Switches to default import for @fastify/static. Adds a helper to set headers based on request paths, configuring no-cache for index.html (and settings.json in SPA) and long-term caching for other assets. Updates static route registrations to use setHeaders.

Changes

Cohort / File(s) Summary
Static serving & cache headers
components/webui/server/src/routes/static.ts
Switches to default import fastifyStatic. Adds constants for Cache-Control values and a setCacheHeaders(res, reqPath, extraNoCachePaths=[]) helper. Registers static handlers for /log-viewer and / with setHeaders, disabling built-in cacheControl. Ensures /index.html and /settings.json (SPA) are no-cache; other assets get 1-year immutable caching. Aligns types via fastifyStatic.FastifyStaticOptions.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Fastify
  participant StaticPlugin as @fastify/static
  participant CacheHelper as setCacheHeaders

  Client->>Fastify: GET /... (static asset)
  Fastify->>StaticPlugin: serve file with setHeaders
  StaticPlugin->>CacheHelper: setCacheHeaders(res, req.path, extraNoCachePaths)
  alt Path is /index.html or in extraNoCachePaths
    CacheHelper-->>StaticPlugin: Cache-Control: public, max-age=0
  else
    CacheHelper-->>StaticPlugin: Cache-Control: public, max-age=31536000, immutable
  end
  StaticPlugin-->>Client: 200 OK + headers + file
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Tip

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@junhaoliao junhaoliao requested a review from davemarco August 19, 2025 15:47
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

📜 Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 51eaeeb and fd851c9.

📒 Files selected for processing (1)
  • components/webui/server/src/routes/static.ts (2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.{cpp,hpp,java,js,jsx,tpp,ts,tsx}

⚙️ CodeRabbit Configuration File

  • Prefer false == <expression> rather than !<expression>.

Files:

  • components/webui/server/src/routes/static.ts
🧠 Learnings (2)
📓 Common learnings
Learnt from: davemarco
PR: y-scope/clp#1015
File: components/log-viewer-webui/server/src/routes/static.ts:65-70
Timestamp: 2025-06-16T13:05:27.349Z
Learning: In components/log-viewer-webui/server/src/routes/static.ts, when decorateReply is set to true in fastifyStatic configuration, the reply.sendFile() method automatically uses the root directory configured in the static plugin registration, eliminating the need to pass the root directory as a second parameter.
📚 Learning: 2025-06-16T13:05:27.349Z
Learnt from: davemarco
PR: y-scope/clp#1015
File: components/log-viewer-webui/server/src/routes/static.ts:65-70
Timestamp: 2025-06-16T13:05:27.349Z
Learning: In components/log-viewer-webui/server/src/routes/static.ts, when decorateReply is set to true in fastifyStatic configuration, the reply.sendFile() method automatically uses the root directory configured in the static plugin registration, eliminating the need to pass the root directory as a second parameter.

Applied to files:

  • components/webui/server/src/routes/static.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: lint-check (ubuntu-24.04)
🔇 Additional comments (1)
components/webui/server/src/routes/static.ts (1)

104-107: SPA fallback looks correct and aligns with prior learning
With decorateReply: true on the client static registration, reply.sendFile("index.html") uses the configured root automatically, matching our prior learning.

import {fileURLToPath} from "node:url";

import {fastifyStatic} from "@fastify/static";
import fastifyStatic from "@fastify/static";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix type reference for FastifyStaticOptions — default import doesn’t expose types

fastifyStatic.FastifyStaticOptions will not resolve when using a default import. Bring the type in via a separate type-only import to avoid a TS compile error.

Apply this diff to add the type import:

 import fastifyStatic from "@fastify/static";
+import type { FastifyStaticOptions } from "@fastify/static";
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
import fastifyStatic from "@fastify/static";
import fastifyStatic from "@fastify/static";
import type { FastifyStaticOptions } from "@fastify/static";
🤖 Prompt for AI Agents
In components/webui/server/src/routes/static.ts around line 4, the code uses a
default import (import fastifyStatic from "@fastify/static") and references
fastifyStatic.FastifyStaticOptions which doesn't resolve; add a separate
type-only import like `import type { FastifyStaticOptions } from
"@fastify/static"` and replace usages of fastifyStatic.FastifyStaticOptions with
the imported FastifyStaticOptions type while keeping the existing default import
for runtime use.

Comment on lines +10 to +15
const CACHE_CONTROL_HEADER_KEY = "Cache-Control";
const CACHE_CONTROL_HEADER_VALUE_NO_CACHE = "public, max-age=0";

// Cache all other static files for 1 year without revalidation.
const CACHE_CONTROL_HEADER_VALUE_LONG_TERM_CACHE = "public, max-age=31536000, immutable";

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick (assertive)

Optional: strengthen revalidation for no-cache responses
If you want stricter behaviour for intermediaries, consider appending must-revalidate to the no-cache value. Current value is broadly fine; this is a belt-and-suspenders tweak.

-const CACHE_CONTROL_HEADER_VALUE_NO_CACHE = "public, max-age=0";
+const CACHE_CONTROL_HEADER_VALUE_NO_CACHE = "public, max-age=0, must-revalidate";
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const CACHE_CONTROL_HEADER_KEY = "Cache-Control";
const CACHE_CONTROL_HEADER_VALUE_NO_CACHE = "public, max-age=0";
// Cache all other static files for 1 year without revalidation.
const CACHE_CONTROL_HEADER_VALUE_LONG_TERM_CACHE = "public, max-age=31536000, immutable";
const CACHE_CONTROL_HEADER_KEY = "Cache-Control";
const CACHE_CONTROL_HEADER_VALUE_NO_CACHE = "public, max-age=0, must-revalidate";
// Cache all other static files for 1 year without revalidation.
const CACHE_CONTROL_HEADER_VALUE_LONG_TERM_CACHE = "public, max-age=31536000, immutable";
🤖 Prompt for AI Agents
In components/webui/server/src/routes/static.ts around lines 10 to 15, the
no-cache header value should enforce stronger revalidation by intermediaries;
change the CACHE_CONTROL_HEADER_VALUE_NO_CACHE constant to include
must-revalidate (e.g. "public, max-age=0, must-revalidate") so that proxies and
caches are required to revalidate before serving cached responses.

Comment on lines +24 to +41
const setCacheHeaders = (
res: Parameters<NonNullable<fastifyStatic.FastifyStaticOptions["setHeaders"]>>[0],
reqPath: string,
extraNoCachePaths: string[] = []
): void => {
const noCachePaths = [
"/index.html",
...extraNoCachePaths,
];

if (noCachePaths.some((noCachePath) => reqPath.endsWith(noCachePath))) {
res.setHeader(CACHE_CONTROL_HEADER_KEY, CACHE_CONTROL_HEADER_VALUE_NO_CACHE);

return;
}

res.setHeader(CACHE_CONTROL_HEADER_KEY, CACHE_CONTROL_HEADER_VALUE_LONG_TERM_CACHE);
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Make no-cache matching OS-agnostic (Windows path separators)
reqPath passed to setHeaders is a filesystem path. On Windows it uses backslashes, so endsWith("/index.html") won’t match. Normalise path separators before checking suffixes.

 const setCacheHeaders = (
     res: Parameters<NonNullable<FastifyStaticOptions["setHeaders"]>>[0],
     reqPath: string,
     extraNoCachePaths: string[] = []
 ): void => {
+    // Normalise to POSIX-style separators so suffix checks work on all OSes
+    const normalizedReqPath = reqPath.split(path.win32.sep).join("/");
     const noCachePaths = [
         "/index.html",
         ...extraNoCachePaths,
     ];
 
-    if (noCachePaths.some((noCachePath) => reqPath.endsWith(noCachePath))) {
+    if (noCachePaths.some((noCachePath) => normalizedReqPath.endsWith(noCachePath))) {
         res.setHeader(CACHE_CONTROL_HEADER_KEY, CACHE_CONTROL_HEADER_VALUE_NO_CACHE);
 
         return;
     }
 
     res.setHeader(CACHE_CONTROL_HEADER_KEY, CACHE_CONTROL_HEADER_VALUE_LONG_TERM_CACHE);
 };
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const setCacheHeaders = (
res: Parameters<NonNullable<fastifyStatic.FastifyStaticOptions["setHeaders"]>>[0],
reqPath: string,
extraNoCachePaths: string[] = []
): void => {
const noCachePaths = [
"/index.html",
...extraNoCachePaths,
];
if (noCachePaths.some((noCachePath) => reqPath.endsWith(noCachePath))) {
res.setHeader(CACHE_CONTROL_HEADER_KEY, CACHE_CONTROL_HEADER_VALUE_NO_CACHE);
return;
}
res.setHeader(CACHE_CONTROL_HEADER_KEY, CACHE_CONTROL_HEADER_VALUE_LONG_TERM_CACHE);
};
const setCacheHeaders = (
res: Parameters<NonNullable<fastifyStatic.FastifyStaticOptions["setHeaders"]>>[0],
reqPath: string,
extraNoCachePaths: string[] = []
): void => {
// Normalise to POSIX-style separators so suffix checks work on all OSes
const normalizedReqPath = reqPath.split(path.win32.sep).join("/");
const noCachePaths = [
"/index.html",
...extraNoCachePaths,
];
if (noCachePaths.some((noCachePath) => normalizedReqPath.endsWith(noCachePath))) {
res.setHeader(CACHE_CONTROL_HEADER_KEY, CACHE_CONTROL_HEADER_VALUE_NO_CACHE);
return;
}
res.setHeader(CACHE_CONTROL_HEADER_KEY, CACHE_CONTROL_HEADER_VALUE_LONG_TERM_CACHE);
};
🤖 Prompt for AI Agents
In components/webui/server/src/routes/static.ts around lines 24 to 41, the
no-cache check uses reqPath.endsWith("/index.html") which fails on Windows
because reqPath can contain backslashes; normalize path separators (e.g. replace
backslashes with forward slashes or use a path-normalization utility) before
performing the endsWith checks so the noCachePaths comparisons are OS-agnostic.

* @param extraNoCachePaths
*/
const setCacheHeaders = (
res: Parameters<NonNullable<fastifyStatic.FastifyStaticOptions["setHeaders"]>>[0],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Use the proper type name instead of fastifyStatic.FastifyStaticOptions

This reference will fail under default import; switch to the named type.

-    res: Parameters<NonNullable<fastifyStatic.FastifyStaticOptions["setHeaders"]>>[0],
+    res: Parameters<NonNullable<FastifyStaticOptions["setHeaders"]>>[0],
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
res: Parameters<NonNullable<fastifyStatic.FastifyStaticOptions["setHeaders"]>>[0],
res: Parameters<NonNullable<FastifyStaticOptions["setHeaders"]>>[0],
🤖 Prompt for AI Agents
In components/webui/server/src/routes/static.ts around line 25, the parameter
type references fastifyStatic.FastifyStaticOptions which breaks under a default
import; change to the named type from the package (e.g. import type {
FastifyStaticOptions } from 'fastify-static') and use FastifyStaticOptions in
the signature, ensuring the import is a named type import rather than relying on
a default namespace import.

Comment on lines +80 to 85
// Prevent fastify-static from adding its own cache headers and provide our own.
cacheControl: false,
setHeaders: (res, reqPath) => {
setCacheHeaders(res, reqPath);
},
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick (assertive)

Minor TS ergonomics: include the third ‘stat’ parameter for signature parity
Not required at runtime, but including the third arg avoids potential typing friction if lib definitions tighten.

-        setHeaders: (res, reqPath) => {
-            setCacheHeaders(res, reqPath);
-        },
+        setHeaders: (res, reqPath, _stat) => {
+            setCacheHeaders(res, reqPath);
+        },
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// Prevent fastify-static from adding its own cache headers and provide our own.
cacheControl: false,
setHeaders: (res, reqPath) => {
setCacheHeaders(res, reqPath);
},
});
// Prevent fastify-static from adding its own cache headers and provide our own.
cacheControl: false,
setHeaders: (res, reqPath, _stat) => {
setCacheHeaders(res, reqPath);
},
});
🤖 Prompt for AI Agents
In components/webui/server/src/routes/static.ts around lines 80 to 85, the
setHeaders callback currently declares only (res, reqPath) which mismatches the
expected signature; add the third parameter (stat) to the function signature for
TypeScript parity (e.g. (res, reqPath, _stat) => { setCacheHeaders(res,
reqPath); }) and name it with a leading underscore to avoid unused-variable lint
errors; no runtime change is needed beyond the signature update.

Comment on lines +97 to 102
// Prevent fastify-static from adding its own cache headers and provide our own.
cacheControl: false,
setHeaders: (res, reqPath) => {
setCacheHeaders(res, reqPath, ["/settings.json"]);
},
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick (assertive)

Same as above: include third ‘stat’ param and simplify wrapper
Keeps callback shape consistent with @fastify/static definitions.

-        setHeaders: (res, reqPath) => {
-            setCacheHeaders(res, reqPath, ["/settings.json"]);
-        },
+        setHeaders: (res, reqPath, _stat) => {
+            setCacheHeaders(res, reqPath, ["/settings.json"]);
+        },
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// Prevent fastify-static from adding its own cache headers and provide our own.
cacheControl: false,
setHeaders: (res, reqPath) => {
setCacheHeaders(res, reqPath, ["/settings.json"]);
},
});
// Prevent fastify-static from adding its own cache headers and provide our own.
cacheControl: false,
setHeaders: (res, reqPath, _stat) => {
setCacheHeaders(res, reqPath, ["/settings.json"]);
},
});
🤖 Prompt for AI Agents
In components/webui/server/src/routes/static.ts around lines 97 to 102, the
setHeaders wrapper currently has only (res, reqPath) but must match
@fastify/static's signature; change it to accept the third stat parameter and
pass it through to setCacheHeaders (i.e. setHeaders: (res, reqPath, stat) =>
setCacheHeaders(res, reqPath, stat, ["/settings.json"])), simplifying the
wrapper by forwarding all three args instead of omitting stat.

Copy link
Contributor

@davemarco davemarco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the cache control be setup in Nginx when it is deployed in production, rather than fastify?

@junhaoliao
Copy link
Member Author

Should the cache control be setup in Nginx when it is deployed in production, rather than fastify?

Good point. do note adding cache control headers does help reduce load on the Fastify server in all cases. While it's technically possible, and probably most efficient, to handle this in any proxy server's configuration, doing so requires the maintainer to understand our webui's architecture, which is less convenient. our arch may also change over time

by setting the headers in the Fastify server itself, we make the caching hints more self-contained. If a maintainer wants to apply additional or alternative caching rules, they can still override these headers in their proxy configuration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants