API Docker

[busaud]

Why don't we create a docker image to run Spider with its API?

[j-mendez]

Hi do you mean something like what spider.cloud is doing running on a http server? Not sure what would be set with docker since it is mainly a library.

[busaud]

Yeah something like that. I want to run it locally but my project is not in Rust .

[j-mendez]

Yeah something like that. I want to run it locally but my project is not in Rust .

Thank you for showing interest in our Spider open-source project. We truly appreciate your enthusiasm however, running Spider as an HTTP server locally particularly for projects not based in Rust can be quite challenging. Our cloud service Spider.cloud was created to handle the complexities of efficiency, concurrency, and streaming. This service relies on bespoke code, and replicating this on a local setup would be a significant undertaking.

Given the extensive effort we've put into developing Spider.cloud we recommend utilizing our cloud service for your crawling needs. This ensures optimal performance and seamless integration without having to manage the complexities locally. If there is anything specific we can assist with regarding Spider.cloud, please let us know.

Also the costs of crawling for the service is currently the most cost effective for computing power and time for development/maintenance.