- Python.3.6
- Redis
- Selenium webdriver
- rq
- Create a virtulaenv
- Activate it
- Install requirements
pip install -r requirements.txt
- Download the chrome driver from this link
- Run your redis server in one terminal:
redis-server
- start the rq workers in a separate terminal
rq worker
- Run the downloader
downloader.py 'cute alpaca' 10
Todo
- Async worker
- Clean the code
- Dockerize