fix(frontend): stop AdSense loading outside eligible marketing pages

[ihabkhaled]

Summary

AdSense rejected ClawAI for "low value content." Root cause: the AdSense
loader script was mounted in the root Next.js layout with no pathname check,
so it executed on every route — login, dashboard, chat, billing, settings —
not just on reviewed editorial pages. A correctly pathname-aware hook already
existed in the codebase but was never wired into anything actually mounted.

This PR implements the P0 fixes from the AdSense low-value-content
remediation brief: stop the script from loading anywhere ineligible, and
temporarily pull public shared chats out of ads/indexing/sitemap/RSS while
the account is under review.

• Move AdSenseHead out of the root layout into (marketing)/layout.tsx
  only — structurally impossible to load on (auth), (portal), (payment).
• Split the loader script into AdSenseScriptLoader, which re-checks
  isAdUnitEligible(pathname) even in review mode (reviewMode no longer
  bypasses eligibility — that bypass was the actual bug).
• Add CHAT_SHARE_REVIEW_LOCKDOWN_ENABLED: one flag that forces every public
  chat share to no-ads / noindex / excluded-from-sitemap / excluded-from-RSS,
  regardless of its own safety-scan verdict, for the review window. Wired
  into both RSS implementations (there are two — rss.service.ts and a
  separate global-rss.service.ts for the all-languages feed, easy to miss
  and caught by the full test run), the sitemap's chat chunks, and shared-chat
  metadata.
• Added a structural regression test (adsense-route-boundary.test.ts) that
  reads each layout's source and fails if AdSense is referenced outside
  (marketing) — guards the exact class of bug that caused the rejection.
• Filled a small robots.txt gap: 4 auth sub-routes were missing from the
  hand-maintained private-route prefix list.

Release notes

For the AdSense review: the ad script no longer loads anywhere except
genuinely eligible public marketing pages, and public shared AI chats are
temporarily excluded from ads, search indexing, the sitemap, and RSS feeds
until the account is approved and shared chats get a dedicated content
review. No user-facing behavior changes outside of ad/indexing exposure —
sharing conversations still works exactly as before.

Remaining work (P1, not in this PR)

Per the remediation brief's own "do not fix low-value-content by generating
filler" guidance, this PR intentionally does not touch /learn content,
run the full multilingual content audit, or rewrite locale copy — that is
real editorial judgment per article/locale, not something to do in bulk in
one pass. Also not done here: the full AdSense-eligible-page audit table,
Auto Ads audit, and ad-density review (sections 4, 13, 14 of the brief).
Recommend scoping those as a separate, content-focused follow-up.

Test plan

• npx tsgo --noEmit (claw-frontend) — 0 errors
• npm run lint (claw-frontend) — 0 errors (4 pre-existing warnings, unrelated)
• npm test (claw-frontend) — full suite green; 3 pre-existing flaky
  timeouts under full-suite load (roles/smart-router/contact-form
  forms), confirmed unrelated by passing in isolation
• npm run build (claw-frontend) — succeeds
• npm run knowledge:verify / npm run audit:check — OK
• Manual: verify in a real browser that no AdSense script tag appears on
  /login, /chat, /dashboard, /settings, or a /share/chat/* page

🤖 Generated with Claude Code