|
1 | 1 | # ProxyPool |
2 | 2 |
|
3 | | -## 安装 |
| 3 | +## Requirements |
4 | 4 |
|
5 | | -### 安装Python |
| 5 | +* Docker |
6 | 6 |
|
7 | | -至少Python3.5以上 |
| 7 | + or |
8 | 8 |
|
9 | | -### 安装Redis |
| 9 | +* Python: >=3.6 |
| 10 | +* Redis |
| 11 | +* Environment: Virtual Env |
10 | 12 |
|
11 | | -安装好之后将Redis服务开启 |
| 13 | +## Run with Docker |
12 | 14 |
|
13 | | -### 配置代理池 |
14 | | - |
15 | | -``` |
16 | | -cd proxypool |
| 15 | +```shell script |
| 16 | +docker-compose up |
17 | 17 | ``` |
18 | 18 |
|
19 | | -进入proxypool目录,修改settings.py文件 |
| 19 | +## Run without Docker |
20 | 20 |
|
21 | | -PASSWORD为Redis密码,如果为空,则设置为None |
| 21 | +Here are steps to run ProxyPool. |
22 | 22 |
|
23 | | -#### 安装依赖 |
| 23 | +### Install Redis |
24 | 24 |
|
25 | | -``` |
26 | | -pip3 install -r requirements.txt |
| 25 | +You need to install Redis locally or get a Redis server firstly. |
| 26 | + |
| 27 | +Next set Redis environment: |
| 28 | + |
| 29 | +```shell script |
| 30 | +export REDIS_HOST='localhost' |
| 31 | +export REDIS_PORT=6379 |
| 32 | +export REDIS_PASSWORD='foobar' |
27 | 33 | ``` |
28 | 34 |
|
29 | | -#### 打开代理池和API |
| 35 | +Also you can just set the Redis Connection String: |
30 | 36 |
|
| 37 | +```shell script |
| 38 | +export REDIS_CONNECTION_STRING='redis://[password]@host:port' |
31 | 39 | ``` |
32 | | -python3 run.py |
| 40 | + |
| 41 | +You can choose one method of above to set Redis environment. |
| 42 | + |
| 43 | +### Clone ProxyPool |
| 44 | + |
| 45 | +```shell script |
| 46 | +https://github.com/Python3WebSpider/ProxyPool |
| 47 | +cd ProxyPool |
33 | 48 | ``` |
34 | 49 |
|
35 | | -## 获取代理 |
| 50 | +### Install Requirements |
| 51 | + |
| 52 | +```shell script |
| 53 | +pip3 install -r requirements.txt |
| 54 | +``` |
36 | 55 |
|
| 56 | +### Run ProxyPool |
37 | 57 |
|
38 | | -利用requests获取方法如下 |
| 58 | +You can run all of the processors including Getter、Tester、 |
| 59 | +Server: |
39 | 60 |
|
40 | | -```python |
41 | | -import requests |
| 61 | +```shell script |
| 62 | +python3 run.py |
| 63 | +``` |
42 | 64 |
|
43 | | -PROXY_POOL_URL = 'http://localhost:5555/random' |
| 65 | +or run with args to run specific processor: |
44 | 66 |
|
45 | | -def get_proxy(): |
46 | | - try: |
47 | | - response = requests.get(PROXY_POOL_URL) |
48 | | - if response.status_code == 200: |
49 | | - return response.text |
50 | | - except ConnectionError: |
51 | | - return None |
| 67 | +```shell script |
| 68 | +python3 run.py --processor getter |
| 69 | +python3 run.py --processor tester |
| 70 | +python3 run.py --processor server |
52 | 71 | ``` |
| 72 | + |
| 73 | +### Usage |
| 74 | + |
| 75 | +After running the ProxyPool, you can visit |
| 76 | +[http://localhost:5555/random](http://localhost:5555/) to access random proxy. |
0 commit comments