Repository navigation
fix(filters): Stage B2 sweep round 5 — 20 recurring off-topic domains - #195
Merged
Merged
Conversation
…ic domains With the domain-frequency gate now killing the single-shot tail, the remaining batch noise is recurring (>=2 appearances) off-topic domains that the gate cannot catch: car-deal/real-estate listings (cartube, dh-nadlan), gadget/expo/ franchising/ESG SEO, traffic-ticket and legal-admin SEO (eranwaisman, what2do, dabullaw), foreign academic/Danish outlets (discovery.researcher.life, fyens.dk), Torah essays (etzion), reality-TV (boss-il), gossip (ednakarnaval), custody advocacy (father4justice), film/therapy/sex-myth SEO (cinema, dvorih, eytanim), municipal tenders (gadish). Existing rows suppressed operationally per the batch scraping protocol. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the discovery candidate filters to block additional recurring off-topic domains that are slipping past the domain-frequency gate, reducing wasted scrape/classifier budget during Stage B2 batch operations.
Changes:
- Added 20 newly identified recurring off-topic domains to
_IRRELEVANT_CONTENT_DOMAINSunder the “Stage B2 sweep 2026-06 (round 5)” section. - Expanded coverage for several SEO/commerce/foreign/religious/entertainment sources that repeatedly surface in discovery results but are not relevant to enforcement-event discovery.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #195 +/- ##
=======================================
Coverage 93.03% 93.03%
=======================================
Files 80 80
Lines 11947 11947
=======================================
Hits 11115 11115
Misses 832 832
🚀 New features to boost your workflow:
|
|
pr-agent-context report: No unresolved review comments, failing checks, or actionable patch coverage gaps were found on PR
#195. Treat this PR as all clear unless new signals appear.Run metadata: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With the domain-frequency gate killing the single-shot tail, the remaining batch noise is recurring off-topic domains the gate can't catch (car/real-estate/gadget/franchising SEO, traffic-ticket & legal-admin SEO, foreign academic/Danish outlets, Torah essays, reality-TV, gossip, custody advocacy, film/therapy/sex-myth SEO). Blocklisted per
docs/batch_scraping_protocol.md.🤖 Generated with Claude Code