Skip to content

Merge updates to Dorieh into NSAPH repository #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mmcentre wants to merge 87 commits into
base: main
Choose a base branch
from
Open

Merge updates to Dorieh into NSAPH repository #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
782b650
Merge pull request #1 from ForomePlatform/dev-0.1.x
mmcentre Jun 29, 2024
f233523
Merge pull request #2 from ForomePlatform/dev-0.1.x
mmcentre Jun 30, 2024
2a89361
MAB Building docker image for Amazon Linux to avoid CVEs. Removed Hi…
mmcentre Aug 2, 2024
42d5714
MAB Cleaning up unused dependencies
mmcentre Aug 2, 2024
c3beb54
MAB Minor fixes
mmcentre Aug 3, 2024
1210ac2
MAB Cleaning up unused dependencies
mmcentre Aug 7, 2024
ca717ed
MAB Data lineage - generating Markdown
mmcentre Aug 8, 2024
e31a457
MAB Data lineage - generating Markdown
mmcentre Aug 8, 2024
c109090
MAB Data lineage - generating Markdown
mmcentre Aug 9, 2024
a033836
MAB Data lineage - generating Markdown
mmcentre Aug 9, 2024
b630155
MAB Data lineage - generating Markdown for Sphinx
mmcentre Aug 9, 2024
3cb956a
MAB Data lineage - generating Markdown for Sphinx
mmcentre Aug 9, 2024
7d31082
MAB Data lineage - generating Markdown for Sphinx
mmcentre Aug 9, 2024
73f9e68
MAB Data lineage - generating Markdown for Sphinx
mmcentre Aug 9, 2024
bcdbe18
MAB Data lineage - generating Markdown for Sphinx
mmcentre Aug 9, 2024
f9dd550
MAB Data lineage - generating Markdown for Sphinx
mmcentre Aug 9, 2024
0a20037
MAB Data lineage - generating Markdown for Sphinx
mmcentre Aug 9, 2024
7403c92
MAB Data lineage - generating Markdown for Sphinx
mmcentre Aug 9, 2024
490480a
MAB Data lineage - generating Markdown for Sphinx
mmcentre Aug 9, 2024
f6f6ea7
MAB Data lineage - generating Markdown for Sphinx
mmcentre Aug 9, 2024
f8b1957
MAB Data lineage - generating Markdown for Sphinx
mmcentre Aug 9, 2024
e6ef201
MAB Data lineage - generating Markdown for Sphinx
mmcentre Aug 9, 2024
4844fe2
MAB Cosmetic changes to documentation
mmcentre Aug 10, 2024
70a9b1c
MAB Cosmetic changes to documentation
mmcentre Aug 12, 2024
cb1e0a1
MAB Documenting data dictionary tools
mmcentre Aug 12, 2024
299c2d6
MAB Documenting data dictionary tools
mmcentre Aug 12, 2024
7425b02
MAB Documenting data dictionary tools
mmcentre Aug 12, 2024
bf46444
MAB Documenting data dictionary tools
mmcentre Aug 12, 2024
b969085
MAB Documenting data dictionary tools
mmcentre Aug 12, 2024
1fae89c
Merge pull request #3 from ForomePlatform/dev-0.2.x
mmcentre Aug 12, 2024
0749011
MAB Add building docker container for clean environment to the project
mmcentre Aug 15, 2024
51410c5
MAB Add workflows: version, handshake, monitor
mmcentre Sep 16, 2024
882e9c6
MAB Only display secret value in debug mode
mmcentre Sep 19, 2024
4dd61a1
MAB Run configurations
mmcentre Sep 21, 2024
a000753
MAB Fix a bug in psql, when there is no results from a query
mmcentre Oct 17, 2024
05f6a22
MAB Fixing export to Parquet, adding JSONLines, HDF5
mmcentre Oct 18, 2024
b073799
MAB Support Spark, add Parquet export to gridmet [test]
mmcentre Oct 19, 2024
8dcbd52
MAB Support Spark, add Parquet export to gridmet [test]
mmcentre Oct 19, 2024
7be4982
MAB Updating Gridmet workflows
mmcentre Oct 20, 2024
db5890b
MAB Updating workflows to export parquet correctly
mmcentre Oct 20, 2024
a7d7d98
MAB Improving Parquet and Spark support
mmcentre Oct 21, 2024
a0de93a
MAB Updating Dockerfiles
mmcentre Oct 22, 2024
6e73b78
MAB Add option to set Xmx option for JVM when using Spark
mmcentre Nov 1, 2024
51b475d
MAB Updating Spark utilities
mmcentre Nov 2, 2024
07a564d
MAB Updating Export Parquet to be more compatible with Spark
mmcentre Nov 2, 2024
0aaeb64
MAB Revert: Export Parquet to be more compatible with Spark
mmcentre Nov 2, 2024
a44a809
MAB Adding Groovy DSL
mmcentre Nov 19, 2024
e5d9ee5
MAB Adding Groovy DSL
mmcentre Dec 15, 2024
d9526a4
MAB Adding Groovy DSL
mmcentre Dec 15, 2024
ac285da
MAB Adding Groovy DSL
mmcentre Dec 15, 2024
f7bf9a6
MAB Fix Parquet export of non-string lists
mmcentre Mar 10, 2025
b022ffc
MAB Add an option to add sql when exporting a table
mmcentre Mar 11, 2025
cac2347
MAB Fix JDK reflection issue for Spark
mmcentre Mar 11, 2025
bcd3c08
MAB Docker files for Dorieh withSpark
mmcentre Mar 11, 2025
c2ae35a
MAB Fix incompatibility of hard partitioning with extra sql
mmcentre Mar 13, 2025
1529c38
Merge pull request #4 from ForomePlatform/dev-0.3x
mmcentre Mar 20, 2025
55f961f
MAB Docker files for Dorieh withSpark
mmcentre Mar 25, 2025
b48fae6
MAB Aggregating 2024 FOSSA data
mmcentre Apr 29, 2025
89ac83e
MAB Aggregating 2024 FOSSA data
mmcentre Apr 29, 2025
9bad322
MAB Improving documentation for handling Medicare data for years pr…
mmcentre May 10, 2025
2f66980
MAB Improving documentation for handling Medicare data for years pr…
mmcentre May 10, 2025
881ec74
MAB Improving documentation for handling Medicare data for years pr…
mmcentre May 11, 2025
e91e864
MAB Improving documentation for handling Medicare data for years pr…
mmcentre May 11, 2025
77862f8
MAB Improving documentation for handling Medicare data for years pr…
mmcentre May 11, 2025
242d1a2
MAB Improving documentation for handling Medicare data for years pr…
mmcentre May 11, 2025
109a542
MAB Improving documentation for handling Medicare data for years pr…
mmcentre May 11, 2025
978732f
MAB Port to Python 3.13 (GIS packages, remove pygeos)
mmcentre Oct 3, 2025
f497d35
MAB Update dependencies and test on Intel platform
mmcentre Oct 3, 2025
52f8052
MAB Port to Python 3.13 (Update Dockerfiles)
mmcentre Oct 3, 2025
77c8720
MAB Port to Python 3.13 (Update Dockerfiles)
mmcentre Oct 3, 2025
5ecdaec
MAB Port to Python 3.13 (Update Dockerfiles)
mmcentre Oct 3, 2025
bd9232f
MAB Working with examples
mmcentre Oct 4, 2025
2a014ce
MAB Working with examples
mmcentre Oct 4, 2025
6f8916a
MAB Working with examples
mmcentre Oct 4, 2025
1cca6a4
MAB Working with examples
mmcentre Oct 4, 2025
730c5d7
MAB Medicare processing examples
mmcentre Oct 6, 2025
0205fbb
MAB Reformatting Workflows documentation
mmcentre Oct 15, 2025
d517e23
MAB Amending code to align with the tutorial
mmcentre Nov 7, 2025
366cfbe
MAB Adding runtime config
mmcentre Nov 7, 2025
a8385a1
MAB Pre-release Cleanup
mmcentre Nov 7, 2025
66391a5
Merge pull request #5 from ForomePlatform/dev-0.4.x
mmcentre Nov 7, 2025
4e61ef9
MAB Updating Tutorial example
mmcentre Nov 7, 2025
e7cb646
MAB Updating Tutorial example
mmcentre Nov 11, 2025
8485752
MAB Cleaning up Data Dictionary Tool
mmcentre Nov 18, 2025
b8e352a
MAB Adding a figure illustrating a data preparation pipeline for …
mmcentre Dec 17, 2025
206173b
MAB Writing the Pipeline Building Tutorial
mmcentre Dec 21, 2025
9e5331b
Merge pull request #6 from ForomePlatform/dev-0.4.x
mmcentre Dec 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .idea/dorieh.iml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion .idea/runConfigurations/Aggregate_original_file___downscale__county__.xml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 17 additions & 0 deletions .idea/runConfigurations/Build_docker_image_Amzn.xml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion .idea/runConfigurations/Build_documentation.xml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 12 additions & 2 deletions .idea/runConfigurations/Census_help.xml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion .idea/runConfigurations/Create_AQS_SQL.xml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

36 changes: 36 additions & 0 deletions .idea/runConfigurations/Create_Airnow_SQL_.xml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

37 changes: 37 additions & 0 deletions .idea/runConfigurations/Create_PM25_components_SQL_.xml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

37 changes: 37 additions & 0 deletions .idea/runConfigurations/Create_climate_SQL__rmax.xml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

37 changes: 37 additions & 0 deletions .idea/runConfigurations/Create_climate_SQL__rmin_.xml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

36 changes: 36 additions & 0 deletions .idea/runConfigurations/DB_Handshake.xml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

37 changes: 37 additions & 0 deletions .idea/runConfigurations/DB_Handshake__ref_runner__.xml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

35 changes: 35 additions & 0 deletions .idea/runConfigurations/DB_Montior__Ref_runner_.xml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading