DUP - Archive old passes

[JLWade]

NOTE: This ticket description depends on what comes out of the spike ticket bcgov/parks-reso-admin#385
DUP is currently storing 100s of thousands of expired passes - this ticket is to "archive" these passes so that it frees up some space in the system as well as improving security and privacy requirements. This ticket relates to bcgov/parks-data-register#356 and may be part of that solution.

November 21, 2024 - DUP prioritization session. This ticket/issue was identified as medium-high priority.

Additional notes

For devs this is a higher priority > lots of passes in DynamoDB, which are not useful to current system and slowing things down. Moving historic passes into their own table will improve speed and reduce carrying costs.
With current structure, admin portal is parsing through a lot of passes/data & information that isn't needed.
linked to ticket 425

Acceptance Criteria:

Given
When
Then

Development Checklist:
[ ]
[ ]
[ ]

Workflows:

Dependencies

• Relates to 105: Invalidate CF during deploys parks-data-register#356
• Dependent on Spike ticket to answer questions: DUP - Data report exploration - check-in times parks-reso-admin#385

Definition of Ready (Note: If any of these points are not applicable, mark N/A)

• User story is included
• User role and type are identified
• Acceptance criteria are included
• Wireframes are included
• Design / Solution is accepted by Product Owner
• Dependencies are identified (technical, business, regulatory/policy)
• Story has been estimated (under 13 pts)

Definition of Done

• In progress:
  • Acceptance criteria are tested (Functionality meets the acceptance criteria defined in the ticket)
  • UI meets accessibility requirements
  • Unit tests are written
  • Work is traceable in GitHub
  • PR linked to ticket number
  • If needed/required - Dev adds flag/label to highlight any migration steps necessary prior to PROD deployment
• Code review:
  • Code is peer reviewed and has passed CI/CD tests
• QA:
  • Acceptance criteria are tested (Functionality meets the acceptance criteria defined in the ticket)
  • Code is potentially shippable to the production environment
  • Functional features have been tested and passed by QA
  • UI components tested by designer
  • Code is deployed to PROD when moved to 'done' column (unless requested otherwise by PO)
• PO Review:
  • Acceptance criteria are tested (Functionality meets the acceptance criteria defined in the ticket)
  • Reviewed and approved by Product Owner

Notes:

[Dianadec]

Cam: Need to anonymize them and archive them by year, adding to a new table.

[Dianadec]

Refinement notes:

Marshall's feedback:

• When archiving - needs to do this in a way that makes sense with the way we use the data. Eg: for Winter, we don't want to have to pull 2 years worth. So we would want to archive it by season rather than by calendar year. Or even fiscal year?
• need bigger discussions about presentation of data

Jess: could still easily pull the data, it's just stored somewhere else.
Marshall: Capacity reports tell us the number of passes at a park at AM/PM. Don't need names etc, but we would like to analyze the check-in times. (ACTION: need separate ticket for future implementation - separate time report for check-in time). How would we analyze this data?
Mark: there are ways to still keep the relevant data.

Also, personal data isn't needed as soon as the pass is expired. - reference PIA policies

[Dianadec]

Cam: Not actionable until we determine when we want this to run and how far back (eg: if it runs on April 2024, what's the latest year that gets archived and removed from the active database?)

[Dianadec]

Idea:
Ticket 1: add a spike to clarify some things related to this ticket? (Created: bcgov/parks-reso-admin#385)
Ticket 2: first cut of archiving everything from 2021-2022 (Created: bcgov/parks-reso-admin#384)
Ticket 3: Future: (THIS TICKET) Automating things would come out of the spike.

[LindsayMacfarlane]

November 21, 2024 - DUP prioritization session. This ticket/issue was identified as medium-high priority.

Additional notes

• For devs this is a higher priority > lots of passes in DynamoDB, which are not useful to current system and slowing things down. Moving historic passes into their own table will improve speed and reduce carrying costs.
• With current structure, admin portal is parsing through a lot of passes/data & information that isn't needed.
• linked to ticket 425

https://app.mural.co/t/bcparks2575/m/bcparks2575/1729124366479/8adea59d9e76fb3eb1e33395f1c1857a8c07fb7b

[Dianadec]

The work we discussed that is needed is captured in #384