Skip to content

Pull requests: yasserg/crawler4j

New pull request New
Clear current search query, filters, and sorts
42 Open 147 Closed
42 Open 147 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Began work on an asynchronous crawling
#157 opened Sep 8, 2016 by lostmsu
subclasses of WebCrawler should be able to change the parser
#150 opened Aug 9, 2016 by cmacdonald
1
maxPagesToFetch bug
#155 opened Aug 9, 2016 by cmacdonald
2
Better management of proxies
#80 opened Jul 9, 2015 by Bouki Loading…
4
refactored some fields from private to protected to make it possible …
#79 opened Jul 8, 2015 by PatrickHuetter Loading…
1
allow parsing script tag and other html tags
#114 opened Feb 10, 2016 by code-911 Loading…
Making Maximum Download URL Sequentially configurable
#128 opened Apr 29, 2016 by Vineeth-Mohan Loading…
2
Allow URLCanonicalizer to handle multiple values for a single query param key
#124 opened Apr 21, 2016 by Phil4 Loading…
3
Canonical URL meta tag handling and AJAX crawling
#82 opened Jul 15, 2015 by EgbertW Loading…
2
Also notify waiting lists when a single URL has been schedules, instead
#59 opened May 20, 2015 by EgbertW Loading…
Improved delay handling
#57 opened May 20, 2015 by EgbertW Loading…
Allow to differentiate between queue sizes in Frontier
#60 opened May 20, 2015 by EgbertW Loading…
Feature: seed tracking
#63 opened May 20, 2015 by EgbertW Loading…
2
Allow negative priorities
#61 opened May 20, 2015 by EgbertW Loading…
7
Add the ability to choose a URLTransformer and not forced to stick wi…
#90 opened Aug 20, 2015 by tiboun Loading…
3
Create CrawlerConfigurationException Specialization. Avoid throw Exception.
#149 opened Aug 8, 2016 by gustavofoa Loading…
3
added custom html content filter
#168 opened Nov 7, 2016 by pdesmet Loading…
1
Add tests for util
#411 opened Aug 5, 2019 by romainbrenguier Loading…
2
Base clases provide more protected methods for subclasses
#432 opened Jan 25, 2020 by dgoiko Loading…
Timeoutable regular expressions in RobotstxtServer
#429 opened Jan 24, 2020 by dgoiko Loading…
1
Generic crawl controller
#434 opened Jan 25, 2020 by dgoiko Loading…
2
Granularity in exception
#428 opened Jan 24, 2020 by dgoiko Loading…
No DocIDs will be created if maxPagesToFetch is reached (most times).
#430 opened Jan 24, 2020 by dgoiko Loading…
1
Cloneable CrawlConfig
#431 opened Jan 25, 2020 by dgoiko Loading…
Feature/spring boot example
#382 opened Dec 18, 2018 by s17t Loading…
ProTip! Filter pull requests by the default branch with base:master.