EN version on GitHub: https://scv2pl.github.io/scv2pl-en
EN version on HEROKU: https://blox-land.herokuapp.com/scv2pl_en
PL version on GitHub: https://scv2pl.github.io
The daily update of data on the SARS-CoV-2 virus in Poland usually takes place at 10:30AM GMT +0200 CEST local time for Warsaw.
Data sources from the API Polish Government Open Data Project:
When a failure occurs - the data can be updated later in the day, or be available only in the following days for download in a zip package. The "SARS-CoV-2_PL_V_2.0" version of the "EPIDEMIA" application is prepared for this and to complete the missing time series - it currently requires manually entering the required configurations and launches until the continuity of time series is obtained. In the near future, this process will be fully automated during a data source failure.
2023-03-13 update - the process described above has been fully automated and is being tested in order to detect possible errors and introduce corrections.
The application is originally designed to run automatically in the HEROKU cloud according to the "Heroku Scheduler" to execute the file: csv_internal_sources_from_zip_checker.py & csv_internal_sources_checker at every hour from 11:00AM to 6:00PM GMT +0200 CEST - it is also possible to perform the same process automatically locally using the CRON scheduler.
Both processes require pre-setting appropriate paths in the configuration file: config_create_sheets.yaml
For HEROKU:
MAIN: /app/,CORE: /app/CORE/,
For Local Environment:
MAIN: /home/blox_land/SARS-CoV-2_PL_V_2.0/,CORE: /home/blox_land/SARS-CoV-2_PL_V_2.0/CORE/,
The LUB folder should be placed in the main directory /home/blox_land/.
With the crontab -e command in terminal, we run the CRON scheduler to execute the file: csv_internal_sources_from_zip_checker.py & csv_internal_sources_checker at every hour on the 45th minute from 10:45AM to 5:45PM GMT +0200 CEST. Please place the contents of the crontab file in it:
#! /bin/bash
30 11 * * 0 /home/blox_land/LUB/LUB.sh
45 10-17 * * * /home/blox_land/LUB/local_startupscript.sh
# $ crontab -e
Each *.sh file must be granted permissions: chmod +x *.sh
"EPIDEMIA" application version: "SARS-CoV-2_PL_V_2.0" - is a solid, designed foundation for version 3.0, which will be implemented in the near future. The "EPIDEMIA" application in version 3.0 - will be able to generate and visualize time series presented in the "SARS-CoV-2_PL_V_2.0" version for a given date entered by the user.
Python files responsible for generating Plotly - Dash charts are under construction and testing. They will be made available only after the application version 3.0 is completed - together with the availability of the entire application under the appropriate Open Source license. The 3.0 application will rely solely on visualizations generated with Python in Plotly-Dash. Only the Pandas library will be used to generate the datasets - Google Sheets will be the perfect schematic diagram for this task - which is a challenge for me as I am still learning all these new things. Google Sheets will remain in use all the time for applications in version 2.0 - because they are also a great tool and an integral part of the application for daily updating, learning and discovering new solutions.
The application in version 1.0 - SCV2PL was also adapted to work in Google Cloud Platform when I had free credits to use. The Virtual Machine Computing Engine for the daily update was started using the CRON "Cloud Scheduler" and the functions: GCP with autostart my startupcript.sh and next was stopped after completing all computational tasks.