Skip to content

Features/505 fix cross environment database backup - #360

Merged
naomitodd merged 50 commits into
mainfrom
features/505-fix-cross-env-backup
May 19, 2026
Merged

Features/505 fix cross environment database backup#360
naomitodd merged 50 commits into
mainfrom
features/505-fix-cross-env-backup

Conversation

@naomitodd

@naomitodd naomitodd commented May 18, 2026

Copy link
Copy Markdown
Collaborator

Description

Update to original backup & restore workflow - now uses github artifacts to share backup file between azure subscription environments.
Increase timeouts and chunk restore job with retries similar to data pipeline to combat lost connection errors.

https://trello.com/c/rX9Ldere/505-look-into-how-we-can-push-data-to-production-without-wiping-the-gateway-users

Will raise placeholder tickets to look into amendments to data pipeline to allow safe running into production environment as an ideal production-ready solution:

  • can't reproduce but saw some duplicate edubase tables when restoring backups due to naming conventions maybe? tables not being dropped - shouldn't affect views as they are rebuilt but possibly storing unneeded data by not fully refreshing the data
  • need to action ticket to reduce data ingestion / may be addressed slightly by EES changes
  • amend to only update changed data or files?
  • add db unit testing for confidence (see data engineer review)
  • add smoke tests to pipeline
  • add maintenance page triggers to data pipeline
  • consider how data is managed for publication releases?
  • full overhaul of data ingestion unlikely?

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Tested cross environment
Prod > Test- https://github.com/DFE-Digital/sap-public/actions/runs/26044867072
Test > Prod - https://github.com/DFE-Digital/sap-public/actions/runs/26080886079
confirmed manually gateway users unaffected

@naomitodd naomitodd changed the title Features/505 fix cross env backup Features/505 fix cross environment database backup May 18, 2026
@github-actions

Copy link
Copy Markdown

Code Coverage Report: Only Changed Files listed

Package Coverage
Overall Coverage 🟢 60.23%

Minimum allowed coverage is 0%, this run produced 60.23%

@naomitodd
naomitodd marked this pull request as ready for review May 19, 2026 06:59

@DMurf DMurf left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks okay to me, happy to merge and give it some time to bed in :)

@naomitodd
naomitodd merged commit 05be0b3 into main May 19, 2026
13 checks passed
@naomitodd
naomitodd deleted the features/505-fix-cross-env-backup branch June 22, 2026 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants