The aim of this website is to provide user a single place to find all the jobs from there favourite Job websites in one place you can visit the app at https://oneshopjobs.herokuapp.com/
https://oneshopjobs.herokuapp.com/
Client:
- Django-Templates
- Vanilla-Js
- HTML-5
- TailwindCSS
Server:
- Django ( Allauth , Django-Rest-Framework , Django-Extensions,django-cors-headers , Django-Clerey-Beat , Celery , Redis , Postgresql )
- Scrapper ( Aiohttp, Asyncio, Icecream , BeautifullSoup4 )
- fastapi ( Tortoise-Orm , Cors, Postgresql )
In the backend iam using a microservice architecture in which Django is my main server which handles all the user reqeust and serving the api where the fastapi is to run my webscrappers . Iam using celery to schedule the task to ping my fastapi server every 5-10mins to run the scrapper. the django server is hosting the main api that our frontend is consuming you can see the more detail about the api at the Redocs
I am using fastapi to host my scrapper these are not the normal webscrappers that you might have seen in the past this scrapper can scrape the whole stackoverflow jobs section in just 36 seconds and store the results in the database to achieve thses kind of results we are using aiohttp instead of the normal reqeusts library for accessing the web page of the stackoverflow because aiohttp is an async library the scrapper are async concurent scrappers which means that they fire the reqeusts to all the 9 pages of the stackoverflow at once so all the data is bieng scrapped at the same time this is possible due to the awesome library asyncio function called asyncio.ensurefuture() you can read more about it at python docs
The frontend is built using the HTMl5-Tailwindcss and Vanilla js the frontend is consuming our backend django api api
- Sign-in
- Registeraion
- log-out
- Infinite Scroll Pagination
- searching / filtering
- responsive
GET /api/v1/jobs/| Parameter | Type | Description |
|---|---|---|
api_key |
string |
Required. Your API key |
GET /api/v1/jobnames/| Parameter | Type | Description |
|---|---|---|
id |
string |
Required. Id of item to fetch |