Update to TSDB by default for Loki (and latest schema version)

[mjnagel]

Description

TSDB has been the recommended store for Loki for several releases now. UDS Core should update to use it by default in its schemaConfig. We should also upgrade to the latest version of the schema at this time this issue is picked up (v13 as of this writing).

In order to "cut-over" between schemas, Loki requires that dates be set to determine when to switch over (see here). To effectively set this "by default" in core, we will need a mechanism to capture the user's upgrade date and switch the schema in the future (2 days). A few options have been explored for this:

• Use helm functions + secret lookups to manage schema change dates
• Use Pepr mutations to modify the loki config and set schema change dates
• Use a combination of the two above approaches - have pepr manage a configmap with dates + use helm lookups to query those dates

Definition of Done

• TSDB is used as the default store for Loki in UDS Core
• Upgrading properly handles the switch from BoltDB to TSDB

Additional Context

https://grafana.com/docs/loki/latest/operations/storage/tsdb/

https://lokidex.com/posts/tsdb/