|
| 1 | +--- |
| 2 | +trigger: always_on |
| 3 | +--- |
| 4 | + |
| 5 | +Start of /source-prompt-2 a contiouation of /source-prompt |
| 6 | + |
| 7 | +Stax Decomposition |
| 8 | +Step 1: Identify competitors [95% confidence] ✓ |
| 9 | +Step 2: Determine scraping approach [65% confidence] ⚠️ |
| 10 | +Step 3: Storage schema design [80% confidence] |
| 11 | +Step 4: Alert logic [90% confidence] |
| 12 | +Step 5: Reporting dashboard [85% confidence] |
| 13 | + |
| 14 | +Step 2 flagged for Self-Research |
| 15 | + |
| 16 | +``` |
| 17 | +
|
| 18 | +#### **Self-Research on Step 2** |
| 19 | +
|
| 20 | +**Analogies Generated:** |
| 21 | +- "Like monitoring stock prices (Yahoo Finance pattern)" |
| 22 | +- "Like tracking Amazon prices (CamelCamelCamel approach)" |
| 23 | +- "Like RSS feed aggregation for blogs" |
| 24 | +
|
| 25 | +**Core Search (GitHub):** |
| 26 | +``` |
| 27 | + |
| 28 | +Found repos: |
| 29 | + |
| 30 | +- price-monitoring-saas (1.2k ⭐) - uses Puppeteer + Cron |
| 31 | +- competitor-tracker (890 ⭐) - uses Playwright + Webhooks |
| 32 | +- pricing-intelligence (340 ⭐) - uses Apify actors |
| 33 | + |
| 34 | +Pattern: Playwright preferred for JS-heavy sites |
| 35 | + |
| 36 | +``` |
| 37 | +
|
| 38 | +**Core Search (NPM):** |
| 39 | +``` |
| 40 | + |
| 41 | +- puppeteer (4M/week) ✓ |
| 42 | +- playwright (2M/week) ✓ |
| 43 | +- cheerio (6M/week) - HTML-only, not suitable |
| 44 | + |
| 45 | +``` |
| 46 | +
|
| 47 | +**Core Search (StackOverflow):** |
| 48 | +``` |
| 49 | + |
| 50 | +Query: "playwright scraping blocked captcha" |
| 51 | +Top answers: |
| 52 | + |
| 53 | +- Use stealth plugin |
| 54 | +- Rotate proxies |
| 55 | +- Add random delays |
| 56 | +- Handle CloudFlare |
| 57 | + |
| 58 | +Gotcha: "Don't scrape too fast or IP banned" |
| 59 | + |
| 60 | +``` |
| 61 | +
|
| 62 | +**Confidence after Core Search: 78%** (still need production guidance) |
| 63 | +
|
| 64 | +**Extended Search (NotebookLM):** |
| 65 | +
|
| 66 | +**Sources Fed:** |
| 67 | +1. https://github.com/price-monitoring-saas |
| 68 | +2. https://playwright.dev/docs/best-practices |
| 69 | +3. https://stackoverflow.com/questions/playwright-stealth |
| 70 | +4. https://www.youtube.com/watch?v=scraping-at-scale |
| 71 | +5. https://blog.apify.com/web-scraping-best-practices/ |
| 72 | +
|
| 73 | +**Asked NotebookLM:** |
| 74 | +> "I'm building a competitor pricing tracker checking every 15 minutes. Based on these 5 sources, what's the production-grade approach and what are the gotchas?" |
| 75 | +
|
| 76 | +**NotebookLM Response:** |
| 77 | +> "Analysis reveals: |
| 78 | +> |
| 79 | +> **Recommended Stack:** |
| 80 | +> - Playwright with stealth plugin (3 sources recommend) |
| 81 | +> - Proxy rotation service (mentioned in all 5 sources) |
| 82 | +> - Store screenshots for verification (blog post best practice) |
| 83 | +> - Redis queue for job management |
| 84 | +> |
| 85 | +> **Production Gotchas:** |
| 86 | +> 1. 15-min frequency may trigger rate limiting (YouTube tutorial) |
| 87 | +> 2. Need CloudFlare bypass strategy (SO thread + blog) |
| 88 | +> 3. CAPTCHA solving service needed (~$2/1000 solves) |
| 89 | +> 4. Memory leaks with long-running Playwright (GitHub issue #234) |
| 90 | +> |
| 91 | +> **Recommendation:** |
| 92 | +> Playwright + stealth + rotating proxies + 30-min frequency |
| 93 | +> (15-min too aggressive per blog analysis)" |
| 94 | +
|
| 95 | +**Confidence after NotebookLM: 92%** → Proceed |
| 96 | +
|
| 97 | +#### **Tier 2: Simulation Mode** |
| 98 | +``` |
| 99 | + |
| 100 | +AI: "Here's my implementation plan: |
| 101 | + |
| 102 | +1. Setup: |
| 103 | + - Playwright with playwright-extra-plugin-stealth |
| 104 | + - ProxyMesh for rotation (cheapest at scale) |
| 105 | + - Redis Bull for job queue |
| 106 | + - PostgreSQL for price history |
| 107 | + |
| 108 | +2. Scraping Logic: |
| 109 | + - 30-min intervals (15-min too aggressive per research) |
| 110 | + - Random delays 2-5 seconds between requests |
| 111 | + - Store raw HTML + parsed data |
| 112 | + - Screenshot on each run for verification |
| 113 | + |
| 114 | +3. Alert System: |
| 115 | + - Trigger on >5% price change |
| 116 | + - Email via SendGrid |
| 117 | + - Slack webhook for real-time |
| 118 | + |
| 119 | +4. Estimated Costs: |
| 120 | + - ProxyMesh: $100/month (10k requests) |
| 121 | + - Redis hosting: $15/month |
| 122 | + - PostgreSQL: $25/month |
| 123 | + - Total: ~$140/month |
| 124 | + |
| 125 | +Risks: |
| 126 | + |
| 127 | +- May still get blocked despite stealth (10% chance per research) |
| 128 | +- CAPTCHA may appear (need budget for solving service) |
| 129 | +- 30-min frequency means 5-30 min delay in price change detection |
| 130 | + |
| 131 | +Should I proceed with this approach?" |
| 132 | + |
| 133 | +User: "Yes, proceed" |
| 134 | + |
| 135 | +``` |
| 136 | +
|
| 137 | +**AI executes autonomously, logging each step** |
| 138 | +
|
| 139 | +--- |
| 140 | +
|
| 141 | +### Example 2: "Research and draft competitive strategy doc" |
| 142 | +
|
| 143 | +#### **Tier 0: Pre-Flight** |
| 144 | +``` |
| 145 | + |
| 146 | +AI: "I understand you want competitive analysis with strategic recommendations. |
| 147 | + |
| 148 | +I'm assuming: |
| 149 | + |
| 150 | +- Focus on direct competitors (not adjacent markets) |
| 151 | +- Last 12 months of data |
| 152 | +- Emphasis on pricing/features over marketing |
| 153 | +- Output: 15-20 page doc with recommendations |
| 154 | + |
| 155 | +Clarification: Should I include emerging startups or only established players?" |
| 156 | + |
| 157 | +User: "Include top 3 emerging startups" |
| 158 | + |
| 159 | +AI: "Got it. Proceeding with established + top 3 emerging." |
| 160 | + |
| 161 | +Stax Breakdown: |
| 162 | +Step 1: Identify competitors [90% confidence] |
| 163 | +Step 2: Data gathering [85% confidence] |
| 164 | +Step 3: Feature comparison [80% confidence] |
| 165 | +Step 4: Pricing analysis [85% confidence] |
| 166 | +Step 5: Market positioning [75% confidence] ⚠️ |
| 167 | +Step 6: Strategic recommendations [65% confidence] ⚠️ |
| 168 | + |
| 169 | +Steps 5 & 6 need Self-Research |
| 170 | + |
| 171 | +``` |
| 172 | +
|
| 173 | +#### **Self-Research on "Strategic Recommendations"** |
| 174 | +
|
| 175 | +**Analogies:** |
| 176 | +- "Like SWOT analysis in business strategy" |
| 177 | +- "Like chess position evaluation (strengths/weaknesses)" |
| 178 | +- "Like medical diagnosis (symptoms → treatment)" |
| 179 | +
|
| 180 | +**Core Search:** |
| 181 | +``` |
| 182 | + |
| 183 | +GitHub: "competitive-strategy-framework" |
| 184 | + |
| 185 | +- business-analysis-tools (2.3k ⭐) |
| 186 | +- market-intel-framework (1.1k ⭐) |
| 187 | + |
| 188 | +NPM: N/A (business strategy, not code) |
| 189 | + |
| 190 | +StackOverflow: N/A |
| 191 | + |
| 192 | +``` |
| 193 | +
|
| 194 | +**Extended Search (NotebookLM):** |
| 195 | +
|
| 196 | +**Sources:** |
| 197 | +1. Harvard Business Review articles on competitive strategy |
| 198 | +2. Porter's Five Forces framework |
| 199 | +3. Blue Ocean Strategy excerpts |
| 200 | +4. Y Combinator startup competition analysis |
| 201 | +5. SaaS metrics benchmarking reports |
| 202 | +
|
| 203 | +**Asked:** |
| 204 | +> "What framework should I use for strategic recommendations in competitive analysis? What do experts recommend?" |
| 205 | +
|
| 206 | +**NotebookLM:** |
| 207 | +> "Consensus framework combines: |
| 208 | +> 1. Porter's Five Forces (industry dynamics) |
| 209 | +> 2. SWOT per competitor (relative positioning) |
| 210 | +> 3. Feature gap analysis (product opportunities) |
| 211 | +> 4. Pricing strategy recommendations (monetization) |
| 212 | +> 5. Go-to-market differentiation (positioning) |
| 213 | +> |
| 214 | +> Key insight from YC: 'Focus on what competitors do poorly, |
| 215 | +> not just what they do well' - look for underserved segments" |
| 216 | +
|
| 217 | +**Confidence: 88%** → Proceed |
| 218 | +
|
| 219 | +#### **Execution** |
| 220 | +AI researches, drafts document, includes citations from NotebookLM sources |
| 221 | +
|
| 222 | +#### **Tier 2: Simulation** |
| 223 | +``` |
| 224 | + |
| 225 | +AI: "Draft complete. Here's the structure: |
| 226 | + |
| 227 | +- Executive Summary (2 pages) |
| 228 | +- Competitor Profiles (8 pages) |
| 229 | +- Feature Gap Analysis (3 pages) |
| 230 | +- Pricing Strategy (2 pages) |
| 231 | +- Strategic Recommendations (4 pages) |
| 232 | + |
| 233 | +Key recommendations: |
| 234 | + |
| 235 | +1. Undercut on pricing for SMB segment |
| 236 | +2. Build missing feature X (3 competitors lack it) |
| 237 | +3. Focus marketing on use case Y (underserved) |
| 238 | + |
| 239 | +Based on 15 sources including HBR, YC, and market data. |
| 240 | + |
| 241 | +Should I share the full draft?" |
| 242 | + |
| 243 | +User: "Yes, show me" |
| 244 | + |
| 245 | +Technical Setup |
| 246 | +MCP Server Configuration |
| 247 | +Minimal setup for Core Search: |
| 248 | + |
| 249 | +{ |
| 250 | +"mcpServers": { |
| 251 | +"github": { |
| 252 | +"command": "npx", |
| 253 | +"args": ["-y", "@modelcontextprotocol/server-github"], |
| 254 | +"env": { |
| 255 | +"GITHUB_PERSONAL_ACCESS_TOKEN": "your-token-here" |
| 256 | +} |
| 257 | +}, |
| 258 | +"brave-search": { |
| 259 | +"command": "npx", |
| 260 | +"args": ["-y", "@modelcontextprotocol/server-brave-search"], |
| 261 | +"env": { |
| 262 | +"BRAVE_API_KEY": "your-api-key" |
| 263 | +} |
| 264 | +}, |
| 265 | +"mcpServers": { |
| 266 | +"Notebooklm": { |
| 267 | +"command": "npx", |
| 268 | +"args": ["-y", "@modelcontextprotocol/server-notebooklm"], |
| 269 | +"env": { |
| 270 | +"Gemini_api_key": "your-api-here" |
| 271 | +} |
| 272 | +} |
| 273 | +} |
| 274 | + |
| 275 | +open /system-prompt |
0 commit comments