Skip to content

Commit e86f151

Browse files
committed
2 parents e5ca8d2 + b498ab2 commit e86f151

File tree

2 files changed

+118
-1
lines changed

2 files changed

+118
-1
lines changed
Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# Docs for the Azure Web Apps Deploy action: https://github.com/Azure/webapps-deploy
2+
# More GitHub Actions for Azure: https://github.com/Azure/actions
3+
4+
name: Build and deploy ASP.Net Core app to Azure Web App - IBEX-Scraper-API
5+
6+
on:
7+
push:
8+
branches:
9+
- main
10+
workflow_dispatch:
11+
12+
jobs:
13+
build:
14+
runs-on: windows-latest
15+
permissions:
16+
contents: read #This is required for actions/checkout
17+
18+
steps:
19+
- uses: actions/checkout@v4
20+
21+
- name: Set up .NET Core
22+
uses: actions/setup-dotnet@v4
23+
with:
24+
dotnet-version: '3.1'
25+
26+
- name: Build with dotnet
27+
run: dotnet build --configuration Release
28+
29+
- name: dotnet publish
30+
run: dotnet publish -c Release -o "${{env.DOTNET_ROOT}}/myapp"
31+
32+
- name: Upload artifact for deployment job
33+
uses: actions/upload-artifact@v4
34+
with:
35+
name: .net-app
36+
path: ${{env.DOTNET_ROOT}}/myapp
37+
38+
deploy:
39+
runs-on: windows-latest
40+
needs: build
41+
permissions:
42+
id-token: write #This is required for requesting the JWT
43+
contents: read #This is required for actions/checkout
44+
45+
steps:
46+
- name: Download artifact from build job
47+
uses: actions/download-artifact@v4
48+
with:
49+
name: .net-app
50+
51+
- name: Login to Azure
52+
uses: azure/login@v2
53+
with:
54+
client-id: ${{ secrets.AZUREAPPSERVICE_CLIENTID_6BB5D16DE3054BC99DE8454AD92E7AC7 }}
55+
tenant-id: ${{ secrets.AZUREAPPSERVICE_TENANTID_B7BB4721B2B5402598B038933A6CADD7 }}
56+
subscription-id: ${{ secrets.AZUREAPPSERVICE_SUBSCRIPTIONID_74D8C97B2410464B848B67495AE72C03 }}
57+
58+
- name: Deploy to Azure Web App
59+
id: deploy-to-webapp
60+
uses: azure/webapps-deploy@v3
61+
with:
62+
app-name: 'IBEX-Scraper-API'
63+
slot-name: 'Production'
64+
package: .
65+

README.md

Lines changed: 53 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,53 @@
1-
# IBEX-Scraper-API
1+
# IBEX-Scraper-API
2+
3+
A secure ASP.NET Core Web API that automatically scrapes **day-ahead prices from the Free Electricity Market (Day Ahead segment) on IBEX** and stores them in a structured Azure SQL database for use by external applications.
4+
5+
## ⚡ Purpose
6+
7+
This API was built to **decouple IBEX access from dependent systems** by providing a secure, reliable, and filterable data backend.
8+
9+
It powers apps like [SunNext](https://github.com/JacksonJS12/SunNext) by allowing them to:
10+
- Access daily price data without scraping IBEX directly
11+
- Query historical electricity prices by date/hour
12+
- Build custom dashboards and energy trading strategies
13+
14+
## 📊 What It Does
15+
16+
- Scrapes **IBEX Day Ahead segment** prices for the next day
17+
- Extracts hourly energy prices (EUR/MWh)
18+
- Saves results into an Azure SQL `MarketPrices` table with:
19+
- `Date` (e.g. `day/month/year`)
20+
- `Hour` (1–24 using Central European Time – CET Time Zone)
21+
- `PricePerMWh` (BGN/MWh)
22+
23+
## 🕒 Scheduling
24+
25+
- Powered by an **Azure Logic App** that triggers daily at **14:00 BG time (UTC+3)**
26+
- Calls the `POST /api/Scrape/scrape-and-save` endpoint
27+
- Only stores new data (prevents duplicate entries for the same day)
28+
29+
## 🧱 Tech Stack
30+
31+
- **Backend:** ASP.NET Core Web API (.NET 8)
32+
- **Scraper:** AngleSharp
33+
- **Database:** Azure SQL
34+
- **Scheduler:** Azure Logic Apps
35+
- **Deployment:** Azure App Service
36+
- **CI/CD:** GitHub Actions
37+
38+
📁 Project Structure
39+
IBEX-Scraper-API/ <br/>
40+
├── Controllers/ <br/>
41+
│     └── ScrapeController.cs <br/>
42+
├── Services/ <br/>
43+
│     └── IbexScraper/ <br/>
44+
│         └── IbexScraperService.cs <br/>
45+
│         └── IIbexScraperService.cs <br/>
46+
├── Data/ <br/>
47+
│     └── Migrations/ <br/>
48+
│     └── Models/ <br/>
49+
│         └── MarketPrice.cs <br/>
50+
│     └── AppDbContext.cs <br/>
51+
│     └── AppDbContextFactory.cs <br/>
52+
└── Program.cs <br/>
53+

0 commit comments

Comments
 (0)