Fix: Robust SEC Filing Retrieval via Local Caching & Deep Scanning

[LutherAbel]

Description:
This PR addresses the "402 Payment Required" error that occurs when users do not have a paid subscription to the Financial Datasets API.

The Goal:
To allow the agent to seamlessly retrieve SEC filings (8-K, 6-K) directly from EDGAR when the paid API is unavailable, instead of failing or falling back to a generic web search.

Key Changes:

1. Hard Fallback Mechanism:

• Updated financial-search.ts to intercept 402 errors.
• If the paid API fails, the system now automatically executes a direct scraper (get_direct_sec_filings) to fetch the data from SEC.gov.

2. Direct SEC Scraper (sec_scraper.ts):

• Implemented a robust scraper that parses SEC filing pages to extract Press Releases (Exhibit 99.1).
• Includes logic to handle various HTML structures (e.g., NBIS, MCHP).

3. Local Ticker Cache:

• Added company_tickers.json as a static asset.
• The scraper prioritizes reading CIKs from this local file to ensure reliable lookups without needing to query the SEC API on every run.

Verification:
Verified that queries like "search NBIS 6-k" and "search SOFI 8-k" now return correct data even without a paid API key.

[LutherAbel]

Note on CI Failure: I noticed the CI build failed due to a frozen-lockfile error. This is because I modified package.json to fix the execution script (switching to tsx), but I am working in a Node.js/NPM environment and cannot regenerate the bun.lockb binary lockfile.

Please run bun install to update the lockfile on your end. The logic changes themselves are verified and working.

[LutherAbel]

Hey @virattt, I saw your excellent deep-dive on X regarding Dexter's architecture. The Subagent design and the "raw data straight to LLM" philosophy are incredibly elegant.

I built this PR specifically to harden the financial-search subagent you mentioned. It adds an offline CIK cache and a "Hard Fallback" mechanism to ensure the agent doesn't crash on 402/403 errors, keeping the data retrieval resilient even when APIs fail. It fully aligns with your design of keeping the decision space small and letting the subagent handle the heavy lifting of parsing non-standard SEC exhibits (like NBIS or SOFI Form 4s).

I know you're busy building FD. Let me know if you'd like to integrate this resilience layer into the core, or if I should just close this PR to keep your roadmap clean. Cheers!

[LutherAbel]

@virattt could you take a look pls