Skip to content

feat(rules page): rebuild directory rules page with vendored lists#1147

Merged
tomcasaburi merged 4 commits into
masterfrom
codex/feature/directory-rules-page
May 31, 2026
Merged

feat(rules page): rebuild directory rules page with vendored lists#1147
tomcasaburi merged 4 commits into
masterfrom
codex/feature/directory-rules-page

Conversation

@tomcasaburi
Copy link
Copy Markdown
Member

@tomcasaburi tomcasaburi commented May 30, 2026

Summary

  • Replace the monolithic 5chan-directory-lists.json vendored snapshot with a mirrored src/data/5chan-directories/ folder synced from bitsocialnet/lists
  • Rebuild /rules to match 4chan’s layout: green sidebar directory nav, category rule boxes, compact Load/Go controls, and mobile-friendly list spacing
  • Add parseSpoilers={false} on rules markdown so instructional [spoiler]…[/spoiler] examples render literally

Test plan

  • Open /rules on desktop and confirm sidebar width, typography, and category navigation match 4chan
  • Open /rules/a and confirm Anime & Manga spoiler syntax shows as plain text, not hidden spoiler UI
  • Check /rules on a mobile viewport: bullets visible, adequate left padding, desktop font size
  • Load board rules via P2P input and confirm Clear/Load button sizing matches homepage search Go
  • yarn build, yarn lint, yarn type-check, rules + markdown tests pass

Note

Medium Risk
Large offline directory data path change affects board resolution when GitHub is down; Rules and defaults caching are user-facing but covered by new tests.

Overview
Replaces the single vendored 5chan-directory-lists.json snapshot with a src/data/5chan-directories/ mirror of upstream lists plus vendored-directory-lists.ts, which merges per-directory files with 5chan-directories-defaults.json at build time. sync-directories.js now copies JSON verbatim (validates before write, prunes removed files) instead of normalizing into one blob.

use-directories gains useDirectoryDefaults, a separate defaults localStorage cache, and keeps defaults in step with GitHub refresh. The Rules page is rebuilt in a 4chan-style layout: sidebar directory nav, Image vs Upload rule sections from defaults, deep links to /rules/:code, and optional P2P rules via address input only (dropdown removed). Markdown adds parseSpoilers (off on rules so [spoiler] examples stay visible).

Reviewed by Cursor Bugbot for commit 658d928. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • New Features

    • Optional spoiler parsing in Markdown.
    • Rules page: categorized directory rules, deep-link deep‑links, improved navigation, and a P2P "load rules from any board" flow.
  • Bug Fixes

    • Directory sync now mirrors upstream manifests, validates payloads, updates/removes local files, and logs sync status without failing builds.
  • Chores

    • Added many per-directory manifests and an offline vendored fallback for directory data.

Mirror per-directory JSON from lists into src/data/5chan-directories, rework /rules layout to match 4chan (sidebar nav, category boxes, P2P load), and keep spoiler markup visible in rule text via parseSpoilers.
@vercel
Copy link
Copy Markdown

vercel Bot commented May 30, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
5chan Ready Ready Preview, Comment May 31, 2026 4:28am

Request Review

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 30, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e0141547-23e9-473a-a905-b9fb4ab8dffb

📥 Commits

Reviewing files that changed from the base of the PR and between 1d1fbc9 and 658d928.

📒 Files selected for processing (5)
  • scripts/sync-directories.js
  • src/hooks/__tests__/use-directories.test.tsx
  • src/hooks/use-directories.ts
  • src/views/rules/__tests__/rules.test.tsx
  • src/views/rules/rules.tsx
🚧 Files skipped from review as they are similar to previous changes (5)
  • src/hooks/tests/use-directories.test.tsx
  • src/views/rules/rules.tsx
  • src/views/rules/tests/rules.test.tsx
  • scripts/sync-directories.js
  • src/hooks/use-directories.ts

📝 Walkthrough

Walkthrough

This PR mirrors per-directory JSON from GitHub (with validation and pruning), adds a vendored assembly module, persists and exposes directory defaults via hooks, refactors the Rules page to use directory defaults and a P2P loader with deep-link scrolling, updates styles and tests, and adds optional spoiler parsing to Markdown.

Changes

Directory Data Infrastructure Refactor and Rules UI Overhaul

Layer / File(s) Summary
Per-directory JSON data files and defaults
src/data/5chan-directories/*.json, src/data/5chan-directories/5chan-directories-defaults.json
Adds 60+ per-directory JSON files and a comprehensive defaults payload with per-directory features and rules.
Directory sync script rewrite
scripts/sync-directories.js
Replaces aggregation with byte-for-byte mirroring: fetches GitHub listing or reads local mirror, downloads .json files with timeout, validates JSON before writes, updates only changed files, prunes removed files, and warns on failure without failing the build.
Vendored data assembly module
src/data/vendored-directory-lists.ts
New TypeScript module loads vendored JSONs at build time, normalizes into DirectoryList entries, computes aggregate timestamps, and exports vendored fallback data plus raw defaults.
Hooks and utilities updated for new data format
src/hooks/use-directories.ts, src/lib/utils/directory-list-lookup-utils.ts
Imports vendored defaults, adds caching and localStorage persistence for normalized defaults, provides getFallbackDirectoryDefaults() and useDirectoryDefaults() hooks, and updates lookup utils to import vendored data from the new module.
Rules page component refactored
src/views/rules/rules.tsx
Renders directory defaults grouped into Image/Upload categories with a quick-jump sidebar, implements deep-link scrolling /rules/:code, replaces navigation selection with a stateful P2P LoadBoardRules loader, and renders rule Markdown with spoiler parsing disabled.
Rules page styling
src/views/rules/rules.module.css
Adds two-column layout, expands directory-nav styling, adjusts ordered-list spacing to line-height, removes old selector styles, and includes mobile responsive rules.
Rules and hooks tests
src/views/rules/__tests__/*, src/hooks/__tests__/use-directories.test.tsx
Wraps Rules in MemoryRouter, mocks scrolling, adds input helpers, extends test scenarios to cover defaults-only rendering, deep-link scrolling, P2P load states, deferred update ordering, and localStorage defaults hydration.
Smoke test update
scripts/smoke-web-app.js
Updates expected visible text for the Rules route to the new phrasing.

Markdown Spoiler Parsing Control

Layer / File(s) Summary
Markdown component with spoiler toggle
src/components/markdown/markdown.tsx
Adds optional parseSpoilers prop (default true) to conditionally exclude spoiler markup from tokenization, introduces a spoiler-free combined regex, updates tokenize to accept and branch on the flag, and wires the flag into RenderContext and useMemo dependencies.
Markdown test helper and spoiler parsing test
src/components/markdown/__tests__/markdown.test.tsx
Updates renderMarkdown test helper to accept optional parseSpoilers and adds a test ensuring raw spoiler markup remains when parseSpoilers is false.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Poem

🐇 I mirrored directories, one by one,
Defaults vendored, fetched, and spun.
Rules now jump, scroll, and load P2P,
Spoilers obey the flag — hush, shh — from me.
A tidy rabbit hop, small and fun.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main change: rebuilding the rules page with vendored directory lists. It accurately reflects the primary objective of the pull request.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/feature/directory-rules-page

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Comment thread src/hooks/use-directories.ts Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
src/views/rules/__tests__/rules.test.tsx (1)

34-42: ⚡ Quick win

Route tests still bypass React Router's param parsing.

Because useParams() is mocked here, the deep-link assertions don't prove that /rules/:boardIdentifier is actually wired correctly. For this feature, it's worth rendering through a real <Route> and driving the param via MemoryRouter.initialEntries.

Suggested test shape
-import { MemoryRouter } from 'react-router-dom';
+import { MemoryRouter, Route, Routes } from 'react-router-dom';
 vi.mock('react-router-dom', async () => {
-  const actual = await vi.importActual<typeof import('react-router-dom')>('react-router-dom');
-  return {
-    ...actual,
-    useParams: () => ({
-      boardIdentifier: testState.boardIdentifier,
-    }),
-  };
+  return vi.importActual<typeof import('react-router-dom')>('react-router-dom');
 });
-const renderRules = async () => {
+const renderRules = async (initialEntry = '/rules') => {
   await act(async () => {
-    root.render(createElement(MemoryRouter, null, createElement(Rules)));
+    root.render(
+      createElement(
+        MemoryRouter,
+        { initialEntries: [initialEntry] },
+        createElement(
+          Routes,
+          null,
+          createElement(Route, { path: '/rules/:boardIdentifier?', element: createElement(Rules) }),
+        ),
+      ),
+    );
   });
 };
-    testState.boardIdentifier = 'a';
-    await renderRules();
+    await renderRules('/rules/a');

As per coding guidelines, "Add or update tests for bug fixes and non-trivial logic changes when the code is reasonably testable."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/views/rules/__tests__/rules.test.tsx` around lines 34 - 42, The test
currently mocks useParams which bypasses React Router; remove or stop mocking
useParams in this test and instead render the component through a real router by
wrapping the tested component in a MemoryRouter with initialEntries set to
[`/rules/${testState.boardIdentifier}`] and a Route
path="/rules/:boardIdentifier" so params are parsed naturally; update the test
render call (the render import from `@testing-library/react`) to use <MemoryRouter
initialEntries={...}><Routes><Route path="/rules/:boardIdentifier"
element={<YourComponent />} /></Routes></MemoryRouter>, drive assertions against
the rendered output and keep references to testState.boardIdentifier and any
existing render/test utilities (e.g., render, Routes, Route, MemoryRouter) so
the deep-link behavior is validated without mocking useParams.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/sync-directories.js`:
- Around line 127-128: The warning hardcodes "from GitHub" which hides the real
source; change the catch block message to include the actual source by building
a source description (e.g., use DIRECTORIES_SOURCE_PATH when present, otherwise
compose the GitHub identifier like `${GITHUB_REPO}@${GITHUB_BRANCH}`) and
interpolate that into the process message instead of the fixed "from GitHub"
text so failures show the real command/URL/path being acted on.

In `@src/hooks/use-directories.ts`:
- Around line 477-497: Remove the local useState/useEffect and GitHub refresh
from useDirectoryDefaults; instead call the shared directories hook (e.g.,
invoke useDirectories() or useDirectoriesState() inside useDirectoryDefaults)
and simply return cacheDefaults ?? getFallbackDirectoryDefaults(); delete the
isMounted wrapper and the fetchDirectoriesFromGitHubDeduped() call so defaults
are derived from the shared cache (cacheDefaults) and no duplicate fetch path is
created.

In `@src/views/rules/rules.tsx`:
- Around line 255-270: The effect currently always calls window.scrollTo(0, 0)
whenever boardIdentifier is falsy (e.g., on /rules) even if directories change;
change the no-board branch to only clear and scroll when the previous state
indicated we had scrolled to a specific board. Concretely, inside the useEffect
that depends on [boardIdentifier, directories], update the if (!boardIdentifier)
branch to check scrolledForRef.current and only call window.scrollTo(0, 0) and
set scrolledForRef.current = null when scrolledForRef.current !== null; keep the
early return otherwise so directory refreshes no longer yank readers to the top.

---

Nitpick comments:
In `@src/views/rules/__tests__/rules.test.tsx`:
- Around line 34-42: The test currently mocks useParams which bypasses React
Router; remove or stop mocking useParams in this test and instead render the
component through a real router by wrapping the tested component in a
MemoryRouter with initialEntries set to [`/rules/${testState.boardIdentifier}`]
and a Route path="/rules/:boardIdentifier" so params are parsed naturally;
update the test render call (the render import from `@testing-library/react`) to
use <MemoryRouter initialEntries={...}><Routes><Route
path="/rules/:boardIdentifier" element={<YourComponent />}
/></Routes></MemoryRouter>, drive assertions against the rendered output and
keep references to testState.boardIdentifier and any existing render/test
utilities (e.g., render, Routes, Route, MemoryRouter) so the deep-link behavior
is validated without mocking useParams.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 50196014-b6b1-4873-933d-15bbc218d767

📥 Commits

Reviewing files that changed from the base of the PR and between e4638ac and 155341f.

📒 Files selected for processing (73)
  • scripts/sync-directories.js
  • src/components/markdown/__tests__/markdown.test.tsx
  • src/components/markdown/markdown.tsx
  • src/data/5chan-directories/5chan-3-directory.json
  • src/data/5chan-directories/5chan-a-directory.json
  • src/data/5chan-directories/5chan-adv-directory.json
  • src/data/5chan-directories/5chan-an-directory.json
  • src/data/5chan-directories/5chan-b-directory.json
  • src/data/5chan-directories/5chan-bant-directory.json
  • src/data/5chan-directories/5chan-biz-directory.json
  • src/data/5chan-directories/5chan-c-directory.json
  • src/data/5chan-directories/5chan-ck-directory.json
  • src/data/5chan-directories/5chan-co-directory.json
  • src/data/5chan-directories/5chan-directories-defaults.json
  • src/data/5chan-directories/5chan-diy-directory.json
  • src/data/5chan-directories/5chan-f-directory.json
  • src/data/5chan-directories/5chan-fa-directory.json
  • src/data/5chan-directories/5chan-fit-directory.json
  • src/data/5chan-directories/5chan-g-directory.json
  • src/data/5chan-directories/5chan-gd-directory.json
  • src/data/5chan-directories/5chan-gif-directory.json
  • src/data/5chan-directories/5chan-his-directory.json
  • src/data/5chan-directories/5chan-i-directory.json
  • src/data/5chan-directories/5chan-ic-directory.json
  • src/data/5chan-directories/5chan-int-directory.json
  • src/data/5chan-directories/5chan-jp-directory.json
  • src/data/5chan-directories/5chan-k-directory.json
  • src/data/5chan-directories/5chan-lit-directory.json
  • src/data/5chan-directories/5chan-m-directory.json
  • src/data/5chan-directories/5chan-mlp-directory.json
  • src/data/5chan-directories/5chan-mu-directory.json
  • src/data/5chan-directories/5chan-n-directory.json
  • src/data/5chan-directories/5chan-news-directory.json
  • src/data/5chan-directories/5chan-o-directory.json
  • src/data/5chan-directories/5chan-out-directory.json
  • src/data/5chan-directories/5chan-p-directory.json
  • src/data/5chan-directories/5chan-po-directory.json
  • src/data/5chan-directories/5chan-pol-directory.json
  • src/data/5chan-directories/5chan-pw-directory.json
  • src/data/5chan-directories/5chan-qst-directory.json
  • src/data/5chan-directories/5chan-r9k-directory.json
  • src/data/5chan-directories/5chan-s5s-directory.json
  • src/data/5chan-directories/5chan-sci-directory.json
  • src/data/5chan-directories/5chan-soc-directory.json
  • src/data/5chan-directories/5chan-sp-directory.json
  • src/data/5chan-directories/5chan-t-directory.json
  • src/data/5chan-directories/5chan-tg-directory.json
  • src/data/5chan-directories/5chan-toy-directory.json
  • src/data/5chan-directories/5chan-trv-directory.json
  • src/data/5chan-directories/5chan-tv-directory.json
  • src/data/5chan-directories/5chan-v-directory.json
  • src/data/5chan-directories/5chan-vg-directory.json
  • src/data/5chan-directories/5chan-vip-directory.json
  • src/data/5chan-directories/5chan-vm-directory.json
  • src/data/5chan-directories/5chan-vmg-directory.json
  • src/data/5chan-directories/5chan-vp-directory.json
  • src/data/5chan-directories/5chan-vr-directory.json
  • src/data/5chan-directories/5chan-vrpg-directory.json
  • src/data/5chan-directories/5chan-vst-directory.json
  • src/data/5chan-directories/5chan-vt-directory.json
  • src/data/5chan-directories/5chan-w-directory.json
  • src/data/5chan-directories/5chan-wg-directory.json
  • src/data/5chan-directories/5chan-wsg-directory.json
  • src/data/5chan-directories/5chan-wsr-directory.json
  • src/data/5chan-directories/5chan-x-directory.json
  • src/data/5chan-directories/5chan-xs-directory.json
  • src/data/5chan-directory-lists.json
  • src/data/vendored-directory-lists.ts
  • src/hooks/use-directories.ts
  • src/lib/utils/directory-list-lookup-utils.ts
  • src/views/rules/__tests__/rules.test.tsx
  • src/views/rules/rules.module.css
  • src/views/rules/rules.tsx
💤 Files with no reviewable changes (1)
  • src/data/5chan-directory-lists.json

Comment thread scripts/sync-directories.js Outdated
Comment thread src/hooks/use-directories.ts Outdated
Comment thread src/views/rules/rules.tsx
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
src/hooks/use-directories.ts (1)

477-500: 🛠️ Refactor suggestion | 🟠 Major | 🏗️ Heavy lift

useDirectoryDefaults() still fetches inside a useEffect.

This hook adds a data-fetching useEffect (the fetchDirectoriesFromGitHubDeduped() call plus the defaults/setDefaults state that only mirrors cacheDefaults). Defaults can be derived from the shared directories hook instead, e.g. depend on useDirectories()/useDirectoriesState() and return cacheDefaults ?? getFallbackDirectoryDefaults(), dropping the effect and local state.

As per coding guidelines: "Do not use useEffect for data fetching; use bitsocial-react-hooks" and "Do not sync derived state with effects; compute during render".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/hooks/use-directories.ts` around lines 477 - 500, The hook
useDirectoryDefaults currently creates local state and a useEffect that calls
fetchDirectoriesFromGitHubDeduped and mirrors cacheDefaults into defaults;
remove the local useState, the useEffect, and the
fetchDirectoriesFromGitHubDeduped call, and instead derive the value directly
from the shared directories state by importing and using useDirectories() or
useDirectoriesState() (so fetching is handled by the shared hook); simplify the
function to return cacheDefaults ?? getFallbackDirectoryDefaults() (or
cacheDefaults ?? getFallbackDirectoryDefaults() combined with any value read
from useDirectories/useDirectoriesState() as needed) and drop
isMounted/setDefaults logic.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@src/hooks/use-directories.ts`:
- Around line 477-500: The hook useDirectoryDefaults currently creates local
state and a useEffect that calls fetchDirectoriesFromGitHubDeduped and mirrors
cacheDefaults into defaults; remove the local useState, the useEffect, and the
fetchDirectoriesFromGitHubDeduped call, and instead derive the value directly
from the shared directories state by importing and using useDirectories() or
useDirectoriesState() (so fetching is handled by the shared hook); simplify the
function to return cacheDefaults ?? getFallbackDirectoryDefaults() (or
cacheDefaults ?? getFallbackDirectoryDefaults() combined with any value read
from useDirectories/useDirectoriesState() as needed) and drop
isMounted/setDefaults logic.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 14aec99e-b5e7-4a25-a849-08dddaac23c3

📥 Commits

Reviewing files that changed from the base of the PR and between 155341f and 1d1fbc9.

📒 Files selected for processing (3)
  • scripts/smoke-web-app.js
  • src/hooks/__tests__/use-directories.test.tsx
  • src/hooks/use-directories.ts

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 51bd0d2. Configure here.

Comment thread src/hooks/use-directories.ts
Comment thread src/views/rules/rules.tsx
@tomcasaburi
Copy link
Copy Markdown
Member Author

Addressed the valid review findings in follow-up commits:\n\n- kept directory defaults and directory payload refresh/cache updates atomic, including reload fallback from localStorage\n- derived directory defaults through the shared directory refresh path instead of a duplicate fetch effect\n- updated the rules smoke assertion for the new heading\n- cleared loaded P2P rules when navigating between rules routes\n- avoided re-scrolling bare /rules on directory refresh\n- made the directory sync warning name the actual source path or GitHub folder\n\nLocal verification after the last review-driven change: test, lint, type-check, build, doctor, smoke, and cross-engine playwright-cli checks for Chrome/Firefox/WebKit desktop/mobile plus a throttled Chromium pass. Hosted checks are green; review threads are resolved.

@tomcasaburi tomcasaburi merged commit b442e05 into master May 31, 2026
11 checks passed
@tomcasaburi tomcasaburi deleted the codex/feature/directory-rules-page branch May 31, 2026 04:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant