Job Automation – Python Job Scraper

This repository contains a Python-based automation script that periodically scrapes job listings from python.org, filters them based on keywords, stores them in a local SQLite database, and sends email notifications when new jobs are found.

This project is intended for learning and experimentation with:

• Python automation
• Web scraping
• Persistence
• Error handling
• Scheduling

It is not a production system and not intended as a résumé project.

---

What the Script Does

On each run, the script:

1. Fetches job listings from https://www.python.org/jobs/
2. Parses job data from HTML
3. Filters jobs using predefined keywords
4. Stores jobs in a SQLite database (deduplicated by link)
5. Sends an email only if new jobs are added
6. Logs all activity and errors
7. Retries failed network requests with exponential backoff

---

Project Structure

job_automation/
│
├── main.py
├── config.py
├── requirements.txt
├── .env # not committed
│
├── automation/
│ ├── fetch.py
│ ├── process.py
│ ├── database.py
│ ├── emailer.py
│ └── logger.py
│
├── data/
│ └── processed/
│ └── jobs.db
│
├── logs/
│ └── job_automation.log
│
└── run_job_automation.bat

---

Requirements

• Python 3.9+
• Windows (for Task Scheduler setup)

Python packages are listed in requirements.txt.

---

Setup

1. Create virtual environment

python -m venv venv
venv\Scripts\activate

2. Install dependencies

pip install -r requirements.txt

---

Email Configuration

Email notifications use Gmail with an App Password.

Create a .env file in the project root:

EMAIL_ADDRESS=your_email@gmail.com
EMAIL_PASSWORD=your_app_password
EMAIL_TO=your_email@gmail.com

The .env file is intentionally excluded from version control.

---

Configuration

Edit config.py to adjust:

• Job URL
• Keyword filters
• Retry behavior

Example:

KEYWORDS = ["python", "django", "backend", "remote"]
MAX_RETRIES = 3
INITIAL_DELAY = 2

---

Running the Script

Manual run:

python main.py

Expected behavior:

• First run inserts jobs and sends an email
• Subsequent runs insert nothing and send no email

---

Scheduling (Windows)

Use Task Scheduler with the provided batch file:

run_job_automation.bat

This allows the script to run unattended on a schedule.

---

Logging

Logs are written to:

logs/job_automation.log

Includes timestamps, job counts, and full error tracebacks.

---

Notes

• SQLite is used for simplicity and local persistence
• Deduplication is enforced at the database level
• Network failures are handled with retries and backoff
• This project prioritizes correctness and learning over features

---

License

MIT