getting CrashLoopBackOff error with grafana pods #1818

rkrath123 · 2024-12-07T12:50:20Z

Bug Report

Which version of the demo you are using? (please provide either a specific
commit
hash
or a specific
release).

Symptom

my-otel-demo-grafana-86f5656bf6-xpnh6                 0/1     CrashLoopBackOff   3 (46s ago)   6m53s

Events:
  Type     Reason     Age                   From               Message
  ----     ------     ----                  ----               -------
  Normal   Scheduled  7m7s                  default-scheduler  Successfully assigned default/my-otel-demo-grafana-86f5656bf6-xpnh6 to ubuntu-master
  Normal   Pulling    7m3s                  kubelet            Pulling image "docker.io/grafana/grafana:11.3.1"
  Normal   Pulled     2m37s                 kubelet            Successfully pulled image "docker.io/grafana/grafana:11.3.1" in 22.438s (4m25.297s including waiting). Image size: 485002425 bytes.
  Warning  Unhealthy  106s (x8 over 2m35s)  kubelet            Readiness probe failed: Get "http://192.168.172.202:3000/api/health": dial tcp 192.168.172.202:3000: connect: connection refused
  Warning  BackOff    83s (x5 over 2m14s)   kubelet            Back-off restarting failed container grafana in pod my-otel-demo-grafana-86f5656bf6-xpnh6_default(4b5f78a0-b149-4724-8835-5b10c559c954)
  Normal   Created    70s (x4 over 2m37s)   kubelet            Created container grafana
  Normal   Started    70s (x4 over 2m37s)   kubelet            Started container grafana
  Normal   Pulled     70s (x3 over 2m26s)   kubelet            Container image "docker.io/grafana/grafana:11.3.1" already present on machine

The text was updated successfully, but these errors were encountered:

julianocosta89 · 2024-12-09T09:09:55Z

Hello @rkrath123, thanks for reporting this.
Could you provide more info about your deployment?

Which version of Helm are you deploying, where are you deploying it?
Are you configuring anything on the deployment?

Thank you 😊

rkrath123 · 2024-12-09T09:32:36Z

Hi @julianocosta89 ,

I am installing it on my vanila k8s cluster on created in openstack, i have used below helm to install nothing else

helm install my-otel-demo open-telemetry/opentelemetry-demo

you can see all other pods are running instead of grafana one

root@ubuntu-master:~# kubectl  get nodes
NAME            STATUS   ROLES           AGE   VERSION
ubuntu-master   Ready    control-plane   46h   v1.30.7

root@ubuntu-master:~# kubectl  get pods
NAME                                                  READY   STATUS             RESTARTS        AGE
my-otel-demo-accountingservice-7c449bf79f-kvc8l       1/1     Running            0               43h
my-otel-demo-adservice-7d6cf47d66-f5x2b               1/1     Running            0               43h
my-otel-demo-cartservice-54d849946f-hkp9d             1/1     Running            0               43h
my-otel-demo-checkoutservice-67844cd86-khpmr          1/1     Running            0               43h
my-otel-demo-currencyservice-7b7b799fbb-5z2w4         1/1     Running            1 (9h ago)      43h
my-otel-demo-emailservice-5b6f8fb5cf-ncj6r            1/1     Running            2 (63m ago)     43h
my-otel-demo-flagd-7f4dcc6457-bkc7k                   2/2     Running            0               43h
my-otel-demo-frauddetectionservice-9b88df59-btfgf     1/1     Running            0               43h
my-otel-demo-frontend-5c9599c8f8-6s4lz                1/1     Running            0               43h
my-otel-demo-frontendproxy-5cb56558c4-j9zt4           1/1     Running            0               43h
my-otel-demo-grafana-86f5656bf6-f8wh2                 0/1     CrashLoopBackOff   503 (91s ago)   43h
my-otel-demo-imageprovider-c8f796548-2c8fr            1/1     Running            0               43h
my-otel-demo-jaeger-65748b457d-98n2d                  1/1     Running            0               43h
my-otel-demo-kafka-7b6f95d97d-p6qkw                   1/1     Running            0               43h
my-otel-demo-loadgenerator-8488df68f6-27276           1/1     Running            0               43h
my-otel-demo-otelcol-58cdf476c7-dlm29                 1/1     Running            0               43h
my-otel-demo-paymentservice-76d994698b-hlljr          1/1     Running            0               43h
my-otel-demo-productcatalogservice-749b75b559-w2ggs   1/1     Running            0               43h
my-otel-demo-prometheus-server-56dc5ffb9c-8jt9c       1/1     Running            0               43h
my-otel-demo-quoteservice-669f9578dc-vwm9l            1/1     Running            0               43h
my-otel-demo-recommendationservice-777fbdd578-llzv6   1/1     Running            0               43h
my-otel-demo-shippingservice-676dc67df-bzmrv          1/1     Running            0               43h
my-otel-demo-valkey-5b9ffb9bd6-gwprr                  1/1     Running            0               43h
otel-demo-opensearch-0                                1/1     Running            0               43h
root@ubuntu-master:~#

root@ubuntu-master:~# helm version
version.BuildInfo{Version:"v3.16.3", GitCommit:"cfd07493f46efc9debd9cc1b02a0961186df7fdf", GitTreeState:"clean", GoVersion:"go1.22.7"}
root@ubuntu-master:~#

julianocosta89 · 2024-12-09T09:56:11Z

Thank you, but which chart version are you deploying?

rkrath123 · 2024-12-09T09:59:59Z

root@ubuntu-master:~# helm ls -A
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                           APP VERSION
my-otel-demo    default         1               2024-12-07 13:33:25.508754793 +0000 UTC deployed        opentelemetry-demo-0.33.7       1.12.0
root@ubuntu-master:~#

julianocosta89 · 2024-12-09T10:06:43Z

I've just deployed the demo in minikube to test and I got everything up and running.

Even the events I get in my grafana pod are exactly the same as yours:

Events:
  Type     Reason     Age                    From               Message
  ----     ------     ----                   ----               -------
  Normal   Scheduled  6m57s                  default-scheduler  Successfully assigned default/my-otel-demo-grafana-86f5656bf6-nxb6d to minikube
  Normal   Pulling    6m56s                  kubelet            Pulling image "docker.io/grafana/grafana:11.3.1"
  Normal   Pulled     5m15s                  kubelet            Successfully pulled image "docker.io/grafana/grafana:11.3.1" in 9.244s (1m41.575s including waiting). Image size: 477934246 bytes.
  Normal   Created    5m14s                  kubelet            Created container grafana
  Normal   Started    5m14s                  kubelet            Started container grafana
  Warning  Unhealthy  5m11s (x3 over 5m13s)  kubelet            Readiness probe failed: Get "http://10.244.0.15:3000/api/health": dial tcp 10.244.0.15:3000: connect: connection refused

Would you be able to share the events from the deployment?

rkrath123 · 2024-12-09T10:40:49Z

for me same issue with crashloopbackoff error in grafana pods

root@ubuntu-master:~# kubectl  get pods | grep grafana
my-otel-demo-grafana-86f5656bf6-f8wh2                 0/1     CrashLoopBackOff   515 (4m45s ago)   45h
root@ubuntu-master:~#

root@ubuntu-master:~#
root@ubuntu-master:~# kubectl describe pod my-otel-demo-grafana-86f5656bf6-f8wh2
Events:
  Type     Reason     Age                    From     Message
  ----     ------     ----                   ----     -------
  Warning  Unhealthy  55m (x1492 over 45h)   kubelet  Readiness probe failed: Get "http://192.168.172.221:3000/api/health": dial tcp 192.168.172.221:3000: connect: connection refused
  Warning  BackOff    32s (x12337 over 45h)  kubelet  Back-off restarting failed container grafana in pod my-otel-demo-grafana-86f5656bf6-f8wh2_default(76bb1cb9-401b-4595-b853-9dc69901c692)

julianocosta89 · 2024-12-09T11:09:47Z

Could you do a describe on the deployment?

kubectl describe deployments.apps my-otel-demo-grafana

rkrath123 · 2024-12-09T11:23:42Z

root@ubuntu-master:~# kubectl describe deployments.apps my-otel-demo-grafana
Name:                   my-otel-demo-grafana
Namespace:              default
CreationTimestamp:      Sat, 07 Dec 2024 13:33:38 +0000
Labels:                 app.kubernetes.io/instance=my-otel-demo
                        app.kubernetes.io/managed-by=Helm
                        app.kubernetes.io/name=grafana
                        app.kubernetes.io/version=11.3.1
                        helm.sh/chart=grafana-8.6.4
Annotations:            deployment.kubernetes.io/revision: 1
                        meta.helm.sh/release-name: my-otel-demo
                        meta.helm.sh/release-namespace: default
Selector:               app.kubernetes.io/instance=my-otel-demo,app.kubernetes.io/name=grafana
Replicas:               1 desired | 1 updated | 1 total | 0 available | 1 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:           app.kubernetes.io/instance=my-otel-demo
                    app.kubernetes.io/name=grafana
                    app.kubernetes.io/version=11.3.1
                    helm.sh/chart=grafana-8.6.4
  Annotations:      checksum/config: 6176cd78468fe72b17c097d3ebad970c000b24bec85a2c6d3976faadf8cb5a78
                    checksum/sc-dashboard-provider-config: e70bf6a851099d385178a76de9757bb0bef8299da6d8443602590e44f05fdf24
                    checksum/secret: bed677784356b2af7fb0d87455db21f077853059b594101a4f6532bfbd962a7f
                    kubectl.kubernetes.io/default-container: grafana
  Service Account:  my-otel-demo-grafana
  Containers:
   grafana:
    Image:           docker.io/grafana/grafana:11.3.1
    Ports:           3000/TCP, 9094/TCP, 9094/UDP
    Host Ports:      0/TCP, 0/TCP, 0/UDP
    SeccompProfile:  RuntimeDefault
    Limits:
      memory:   150Mi
    Liveness:   http-get http://:3000/api/health delay=60s timeout=30s period=10s #success=1 #failure=10
    Readiness:  http-get http://:3000/api/health delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      POD_IP:                       (v1:status.podIP)
      GF_SECURITY_ADMIN_USER:      <set to the key 'admin-user' in secret 'my-otel-demo-grafana'>      Optional: false
      GF_SECURITY_ADMIN_PASSWORD:  <set to the key 'admin-password' in secret 'my-otel-demo-grafana'>  Optional: false
      GF_INSTALL_PLUGINS:          <set to the key 'plugins' of config map 'my-otel-demo-grafana'>     Optional: false
      GF_PATHS_DATA:               /var/lib/grafana/
      GF_PATHS_LOGS:               /var/log/grafana
      GF_PATHS_PLUGINS:            /var/lib/grafana/plugins
      GF_PATHS_PROVISIONING:       /etc/grafana/provisioning
    Mounts:
      /etc/grafana/grafana.ini from config (rw,path="grafana.ini")
      /etc/grafana/provisioning/dashboards/dashboardproviders.yaml from config (rw,path="dashboardproviders.yaml")
      /etc/grafana/provisioning/datasources/datasources.yaml from config (rw,path="datasources.yaml")
      /var/lib/grafana from storage (rw)
      /var/lib/grafana/dashboards/default from dashboards-default (rw)
  Volumes:
   config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      my-otel-demo-grafana
    Optional:  false
   dashboards-default:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      my-otel-demo-grafana-dashboards
    Optional:  false
   storage:
    Type:          EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:     <unset>
  Node-Selectors:  <none>
  Tolerations:     <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      False   MinimumReplicasUnavailable
  Progressing    False   ProgressDeadlineExceeded
OldReplicaSets:  <none>
NewReplicaSet:   my-otel-demo-grafana-86f5656bf6 (1/1 replicas created)
Events:          <none>
root@ubuntu-master:~#
root@ubuntu-master:~#

julianocosta89 · 2024-12-09T11:53:14Z

Okay, nothing wrong there :/

Have you tried to kill the pod already?

rkrath123 · 2024-12-09T11:59:12Z

the same issue even after deleting the pod

root@ubuntu-master:~# kubectl  get pods | grep grafana
my-otel-demo-grafana-86f5656bf6-f8wh2                 0/1     CrashLoopBackOff   531 (51s ago)   46h
root@ubuntu-master:~#

root@ubuntu-master:~#
root@ubuntu-master:~# kubectl  delete pods  my-otel-demo-grafana-86f5656bf6-f8wh2
pod "my-otel-demo-grafana-86f5656bf6-f8wh2" deleted
root@ubuntu-master:~#
root@ubuntu-master:~#

root@ubuntu-master:~#
root@ubuntu-master:~# kubectl  get pods -w | grep grafana
my-otel-demo-grafana-86f5656bf6-krdzr                 0/1     Running   1 (7s ago)      19s
my-otel-demo-grafana-86f5656bf6-krdzr                 0/1     Error     1 (12s ago)     24s
my-otel-demo-grafana-86f5656bf6-krdzr                 0/1     CrashLoopBackOff   1 (7s ago)      30s

julianocosta89 · 2024-12-09T12:32:57Z

Ok, let's do some troubleshooting then.

Could you delete the grafana deployment and run this?

kubectl run --image docker.io/grafana/grafana:11.3.1 -it grafana

rkrath123 · 2024-12-09T12:43:56Z

I could see below error

root@ubuntu-master:~# kubectl run --image docker.io/grafana/grafana:11.3.1 -it grafana
If you don't see a command prompt, try pressing enter.
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="add index alert state"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="add index alert state" duration=4.94517ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="add index alert dashboard_id"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="add index alert dashboard_id" duration=1.375159ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Create alert_rule_tag table v1"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Create alert_rule_tag table v1" duration=1.147673ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add unique index alert_rule_tag.alert_id_tag_id"

...
ERROR[12-09|12:41:34] Could not register plugin                logger=plugins.registration pluginId=xychart error="plugin xychart is already registered"
ERROR[12-09|12:41:34] Could not initialize plugin              logger=plugins.initialization pluginId=xychart error="plugin xychart is already registered"
WARN [12-09|12:41:34] Skipping finding plugins as directory does not exist logger=local.finder path=/usr/share/grafana/plugins-bundled

INFO [12-09|12:41:36] Adding GroupVersion iam.grafana.app v0alpha1 to ResourceManager logger=grafana-apiserver
INFO [12-09|12:41:36] Adding GroupVersion playlist.grafana.app v0alpha1 to ResourceManager logger=grafana-apiserver
INFO [12-09|12:41:36] Adding GroupVersion featuretoggle.grafana.app v0alpha1 to ResourceManager logger=grafana-apiserver

ERROR[12-09|12:41:44] Failed to install plugin                 logger=plugin.backgroundinstaller pluginId=grafana-lokiexplore-app version= error="Get \"https://grafana.com/api/plugins/grafana-lokiexplore-app/versions\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
ERROR[12-09|12:41:44] Update check failed                      logger=grafana.update.checker error="failed to get stable version from grafana.com: Get \"https://grafana.com/api/grafana/versions/stable\": dial tcp 34.120.177.193:443: i/o timeout" duration=10.010819827s
ERROR[12-09|12:41:44] Error while updating detectors           logger=plugin.angulardetectorsprovider.dynamic error="fetch: http do: Get \"https://grafana.com/api/plugins/angular_patterns\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
INFO [12-09|12:41:44] Patterns update finished                 logger=plugin.angulardetectorsprovider.dynamic duration=10.000769162s
ERROR[12-09|12:41:45] Error downloading plugin manifest keys   logger=plugin.signature.key_retriever error="Get \"https://grafana.com/api/plugins/ci/keys\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
^[[6~ERROR[12-09|12:42:55] Error downloading plugin manifest keys   logger=plugin.signature.key_retriever error="Get \"https://grafana.com/api/plugins/ci/keys\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
```"

rkrath123 · 2024-12-09T12:50:38Z

complete log message



root@ubuntu-master:~# kubectl run --image docker.io/grafana/grafana:11.3.1 -it grafana
If you don't see a command prompt, try pressing enter.
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="add index alert state"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="add index alert state" duration=4.94517ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="add index alert dashboard_id"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="add index alert dashboard_id" duration=1.375159ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Create alert_rule_tag table v1"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Create alert_rule_tag table v1" duration=1.147673ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add unique index alert_rule_tag.alert_id_tag_id"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add unique index alert_rule_tag.alert_id_tag_id" duration=1.6016ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" duration=1.035939ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" duration=7.098258ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Create alert_rule_tag table v2"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Create alert_rule_tag table v2" duration=1.137286ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" duration=722.044µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="copy alert_rule_tag v1 to v2"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="copy alert_rule_tag v1 to v2" duration=299.631µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="drop table alert_rule_tag_v1"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="drop table alert_rule_tag_v1" duration=1.764782ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="create alert_notification table v1"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="create alert_notification table v1" duration=810.229µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add column is_default"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add column is_default" duration=4.521628ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add column frequency"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add column frequency" duration=2.567172ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add column send_reminder"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add column send_reminder" duration=3.474203ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add column disable_resolve_message"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add column disable_resolve_message" duration=4.269609ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="add index alert_notification org_id & name"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="add index alert_notification org_id & name" duration=2.427691ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Update alert table charset"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Update alert table charset" duration=100.013µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Update alert_notification table charset"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Update alert_notification table charset" duration=13.874349ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="create notification_journal table v1"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="create notification_journal table v1" duration=8.794523ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="add index notification_journal org_id & alert_id & notifier_id"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="add index notification_journal org_id & alert_id & notifier_id" duration=13.187797ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="drop alert_notification_journal"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="drop alert_notification_journal" duration=904.257µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="create alert_notification_state table v1"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="create alert_notification_state table v1" duration=1.520683ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="add index alert_notification_state org_id & alert_id & notifier_id"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="add index alert_notification_state org_id & alert_id & notifier_id" duration=1.10211ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add for to alert table"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add for to alert table" duration=4.032251ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add column uid in alert_notification"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add column uid in alert_notification" duration=3.861877ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Update uid column values in alert_notification"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Update uid column values in alert_notification" duration=371.216µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add unique index alert_notification_org_id_uid"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add unique index alert_notification_org_id_uid" duration=758.816µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Remove unique index org_id_name"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Remove unique index org_id_name" duration=756.453µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add column secure_settings in alert_notification"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add column secure_settings in alert_notification" duration=2.266154ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="alter alert.settings to mediumtext"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="alter alert.settings to mediumtext" duration=229.009µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add non-unique index alert_notification_state_alert_id"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add non-unique index alert_notification_state_alert_id" duration=890.774µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add non-unique index alert_rule_tag_alert_id"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add non-unique index alert_rule_tag_alert_id" duration=1.653356ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Drop old annotation table v4"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Drop old annotation table v4" duration=3.149128ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="create annotation table v5"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="create annotation table v5" duration=1.832642ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="add index annotation 0 v3"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="add index annotation 0 v3" duration=1.250115ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="add index annotation 1 v3"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="add index annotation 1 v3" duration=944.778µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="add index annotation 2 v3"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="add index annotation 2 v3" duration=2.200615ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="add index annotation 3 v3"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="add index annotation 3 v3" duration=1.654752ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="add index annotation 4 v3"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="add index annotation 4 v3" duration=1.333416ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Update annotation table charset"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Update annotation table charset" duration=91.55µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add column region_id to annotation table"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add column region_id to annotation table" duration=16.734705ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Drop category_id index"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Drop category_id index" duration=4.810498ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add column tags to annotation table"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add column tags to annotation table" duration=10.115381ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Create annotation_tag table v2"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Create annotation_tag table v2" duration=904.811µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add unique index annotation_tag.annotation_id_tag_id"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add unique index annotation_tag.annotation_id_tag_id" duration=2.445303ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="drop index UQE_annotation_tag_annotation_id_tag_id - v2"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" duration=1.473673ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Rename table annotation_tag to annotation_tag_v2 - v2"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Rename table annotation_tag to annotation_tag_v2 - v2" duration=10.851956ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Create annotation_tag table v3"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Create annotation_tag table v3" duration=3.107727ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" duration=1.296733ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="copy annotation_tag v2 to v3"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="copy annotation_tag v2 to v3" duration=648.693µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="drop table annotation_tag_v2"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="drop table annotation_tag_v2" duration=805.088µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Update alert annotations and set TEXT to empty"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Update alert annotations and set TEXT to empty" duration=393.906µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add created time to annotation table"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add created time to annotation table" duration=3.688532ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add updated time to annotation table"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add updated time to annotation table" duration=5.213955ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add index for created in annotation table"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add index for created in annotation table" duration=1.326685ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add index for updated in annotation table"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add index for updated in annotation table" duration=923.069µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Convert existing annotations from seconds to milliseconds"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Convert existing annotations from seconds to milliseconds" duration=501.336µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add epoch_end column"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add epoch_end column" duration=3.140645ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Add index for epoch_end"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Add index for epoch_end" duration=1.990337ms
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Make epoch_end the same as epoch"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Make epoch_end the same as epoch" duration=529.253µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Move region to single row"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Move region to single row" duration=644.047µs
INFO [12-09|12:41:28] Executing migration                      logger=migrator id="Remove index org_id_epoch from annotation table"
INFO [12-09|12:41:28] Migration successfully executed          logger=migrator id="Remove index org_id_epoch from annotation table" duration=1.286766ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" duration=899.542µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" duration=11.105505ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add index for org_id_epoch_end_epoch on annotation table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add index for org_id_epoch_end_epoch on annotation table" duration=4.130441ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Remove index org_id_epoch_epoch_end from annotation table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Remove index org_id_epoch_epoch_end from annotation table" duration=3.069019ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add index for alert_id on annotation table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add index for alert_id on annotation table" duration=2.926069ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Increase tags column to length 4096"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Increase tags column to length 4096" duration=930.946µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="create test_data table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="create test_data table" duration=12.485468ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="create dashboard_version table v1"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="create dashboard_version table v1" duration=1.181937ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add index dashboard_version.dashboard_id"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add index dashboard_version.dashboard_id" duration=974.17µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add unique index dashboard_version.dashboard_id and dashboard_version.version"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add unique index dashboard_version.dashboard_id and dashboard_version.version" duration=1.355604ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Set dashboard version to 1 where 0"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Set dashboard version to 1 where 0" duration=508.777µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="save existing dashboard data in dashboard_version table v1"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="save existing dashboard data in dashboard_version table v1" duration=825.634µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="alter dashboard_version.data to mediumtext v1"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="alter dashboard_version.data to mediumtext v1" duration=8.348301ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="create team table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="create team table" duration=5.871852ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add index team.org_id"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add index team.org_id" duration=1.423663ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add unique index team_org_id_name"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add unique index team_org_id_name" duration=948.643µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add column uid in team"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add column uid in team" duration=3.70448ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Update uid column values in team"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Update uid column values in team" duration=416.568µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add unique index team_org_id_uid"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add unique index team_org_id_uid" duration=1.020338ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="create team member table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="create team member table" duration=837.618µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add index team_member.org_id"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add index team_member.org_id" duration=3.006753ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add unique index team_member_org_id_team_id_user_id"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add unique index team_member_org_id_team_id_user_id" duration=4.27893ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add index team_member.team_id"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add index team_member.team_id" duration=983.477µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add column email to team table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add column email to team table" duration=3.709081ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add column external to team_member table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add column external to team_member table" duration=5.928646ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add column permission to team_member table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add column permission to team_member table" duration=7.333631ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add unique index team_member_user_id_org_id"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add unique index team_member_user_id_org_id" duration=781.873µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="create dashboard acl table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="create dashboard acl table" duration=2.867287ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add index dashboard_acl_dashboard_id"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add index dashboard_acl_dashboard_id" duration=32.848052ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add unique index dashboard_acl_dashboard_id_user_id"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add unique index dashboard_acl_dashboard_id_user_id" duration=29.75777ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add unique index dashboard_acl_dashboard_id_team_id"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add unique index dashboard_acl_dashboard_id_team_id" duration=1.226053ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add index dashboard_acl_user_id"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add index dashboard_acl_user_id" duration=3.978474ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add index dashboard_acl_team_id"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add index dashboard_acl_team_id" duration=1.200605ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add index dashboard_acl_org_id_role"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add index dashboard_acl_org_id_role" duration=1.037762ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add index dashboard_permission"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add index dashboard_permission" duration=978.457µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="save default acl rules in dashboard_acl table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="save default acl rules in dashboard_acl table" duration=704.506µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="delete acl rules for deleted dashboards and folders"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="delete acl rules for deleted dashboards and folders" duration=295.077µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="create tag table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="create tag table" duration=755.403µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add index tag.key_value"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add index tag.key_value" duration=938.334µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="create login attempt table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="create login attempt table" duration=909.168µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add index login_attempt.username"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add index login_attempt.username" duration=1.226115ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="drop index IDX_login_attempt_username - v1"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="drop index IDX_login_attempt_username - v1" duration=754.833µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Rename table login_attempt to login_attempt_tmp_qwerty - v1"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" duration=9.6059ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="create login_attempt v2"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="create login_attempt v2" duration=676.183µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="create index IDX_login_attempt_username - v2"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="create index IDX_login_attempt_username - v2" duration=716.748µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="copy login_attempt v1 to v2"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="copy login_attempt v1 to v2" duration=286.44µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="drop login_attempt_tmp_qwerty"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="drop login_attempt_tmp_qwerty" duration=579.427µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="create user auth table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="create user auth table" duration=876.404µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="create index IDX_user_auth_auth_module_auth_id - v1"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="create index IDX_user_auth_auth_module_auth_id - v1" duration=1.107594ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="alter user_auth.auth_id to length 190"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="alter user_auth.auth_id to length 190" duration=91.795µs
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add OAuth access token to user_auth"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add OAuth access token to user_auth" duration=3.354902ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add OAuth refresh token to user_auth"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add OAuth refresh token to user_auth" duration=17.526785ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add OAuth token type to user_auth"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add OAuth token type to user_auth" duration=11.079182ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add OAuth expiry to user_auth"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add OAuth expiry to user_auth" duration=3.77227ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add index to user_id column in user_auth"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add index to user_id column in user_auth" duration=2.340257ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add OAuth ID token to user_auth"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add OAuth ID token to user_auth" duration=27.763546ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="create server_lock table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="create server_lock table" duration=1.178911ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add index server_lock.operation_uid"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add index server_lock.operation_uid" duration=1.321336ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="create user auth token table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="create user auth token table" duration=1.434762ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add unique index user_auth_token.auth_token"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add unique index user_auth_token.auth_token" duration=4.532439ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add unique index user_auth_token.prev_auth_token"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add unique index user_auth_token.prev_auth_token" duration=7.616516ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add index user_auth_token.user_id"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add index user_auth_token.user_id" duration=55.116219ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="Add revoked_at to the user auth token"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="Add revoked_at to the user auth token" duration=4.86747ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add index user_auth_token.revoked_at"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add index user_auth_token.revoked_at" duration=1.598229ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="add external_session_id to user_auth_token"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="add external_session_id to user_auth_token" duration=14.41918ms
INFO [12-09|12:41:29] Executing migration                      logger=migrator id="create cache_data table"
INFO [12-09|12:41:29] Migration successfully executed          logger=migrator id="create cache_data table" duration=1.113214ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add unique index cache_data.cache_key"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add unique index cache_data.cache_key" duration=1.987634ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="create short_url table v1"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="create short_url table v1" duration=1.075252ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index short_url.org_id-uid"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index short_url.org_id-uid" duration=1.348969ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="alter table short_url alter column created_by type to bigint"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="alter table short_url alter column created_by type to bigint" duration=169.521µs
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="delete alert_definition table"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="delete alert_definition table" duration=363.072µs
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="recreate alert_definition table"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="recreate alert_definition table" duration=2.813105ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index in alert_definition on org_id and title columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index in alert_definition on org_id and title columns" duration=1.256195ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index in alert_definition on org_id and uid columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index in alert_definition on org_id and uid columns" duration=2.174463ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="alter alert_definition table data column to mediumtext in mysql"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="alter alert_definition table data column to mediumtext in mysql" duration=211.093µs
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="drop index in alert_definition on org_id and title columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="drop index in alert_definition on org_id and title columns" duration=1.880359ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="drop index in alert_definition on org_id and uid columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="drop index in alert_definition on org_id and uid columns" duration=2.70323ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add unique index in alert_definition on org_id and title columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add unique index in alert_definition on org_id and title columns" duration=2.087938ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add unique index in alert_definition on org_id and uid columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add unique index in alert_definition on org_id and uid columns" duration=4.632931ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="Add column paused in alert_definition"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="Add column paused in alert_definition" duration=4.404588ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="drop alert_definition table"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="drop alert_definition table" duration=1.141076ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="delete alert_definition_version table"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="delete alert_definition_version table" duration=100.573µs
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="recreate alert_definition_version table"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="recreate alert_definition_version table" duration=2.041781ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index in alert_definition_version table on alert_definition_id and version columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index in alert_definition_version table on alert_definition_id and version columns" duration=4.698508ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index in alert_definition_version table on alert_definition_uid and version columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index in alert_definition_version table on alert_definition_uid and version columns" duration=17.273255ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="alter alert_definition_version table data column to mediumtext in mysql"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="alter alert_definition_version table data column to mediumtext in mysql" duration=132.588µs
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="drop alert_definition_version table"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="drop alert_definition_version table" duration=1.175241ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="create alert_instance table"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="create alert_instance table" duration=1.021609ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index in alert_instance table on def_org_id, def_uid and current_state columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index in alert_instance table on def_org_id, def_uid and current_state columns" duration=24.256968ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index in alert_instance table on def_org_id, current_state columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index in alert_instance table on def_org_id, current_state columns" duration=1.080349ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add column current_state_end to alert_instance"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add column current_state_end to alert_instance" duration=5.205326ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="remove index def_org_id, def_uid, current_state on alert_instance"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="remove index def_org_id, def_uid, current_state on alert_instance" duration=3.613854ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="remove index def_org_id, current_state on alert_instance"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="remove index def_org_id, current_state on alert_instance" duration=1.085286ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="rename def_org_id to rule_org_id in alert_instance"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="rename def_org_id to rule_org_id in alert_instance" duration=52.981971ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="rename def_uid to rule_uid in alert_instance"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="rename def_uid to rule_uid in alert_instance" duration=23.9494ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index rule_org_id, rule_uid, current_state on alert_instance"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index rule_org_id, rule_uid, current_state on alert_instance" duration=2.524816ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index rule_org_id, current_state on alert_instance"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index rule_org_id, current_state on alert_instance" duration=1.516958ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add current_reason column related to current_state"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add current_reason column related to current_state" duration=5.236123ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add result_fingerprint column to alert_instance"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add result_fingerprint column to alert_instance" duration=4.025307ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="create alert_rule table"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="create alert_rule table" duration=2.112564ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index in alert_rule on org_id and title columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index in alert_rule on org_id and title columns" duration=2.059784ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index in alert_rule on org_id and uid columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index in alert_rule on org_id and uid columns" duration=1.922449ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index in alert_rule on org_id, namespace_uid, group_uid columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index in alert_rule on org_id, namespace_uid, group_uid columns" duration=1.016842ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="alter alert_rule table data column to mediumtext in mysql"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="alter alert_rule table data column to mediumtext in mysql" duration=79.577µs
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add column for to alert_rule"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add column for to alert_rule" duration=12.406209ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add column annotations to alert_rule"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add column annotations to alert_rule" duration=37.227043ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add column labels to alert_rule"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add column labels to alert_rule" duration=45.529859ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="remove unique index from alert_rule on org_id, title columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="remove unique index from alert_rule on org_id, title columns" duration=2.618738ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index in alert_rule on org_id, namespase_uid and title columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index in alert_rule on org_id, namespase_uid and title columns" duration=1.551609ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add dashboard_uid column to alert_rule"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add dashboard_uid column to alert_rule" duration=6.916491ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add panel_id column to alert_rule"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add panel_id column to alert_rule" duration=9.604092ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index in alert_rule on org_id, dashboard_uid and panel_id columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" duration=1.94993ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add rule_group_idx column to alert_rule"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add rule_group_idx column to alert_rule" duration=5.216344ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add is_paused column to alert_rule table"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add is_paused column to alert_rule table" duration=36.617302ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="fix is_paused column for alert_rule table"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="fix is_paused column for alert_rule table" duration=374.808µs
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="create alert_rule_version table"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="create alert_rule_version table" duration=1.524774ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" duration=2.862431ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" duration=1.818314ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="alter alert_rule_version table data column to mediumtext in mysql"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="alter alert_rule_version table data column to mediumtext in mysql" duration=239.604µs
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add column for to alert_rule_version"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add column for to alert_rule_version" duration=6.339286ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add column annotations to alert_rule_version"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add column annotations to alert_rule_version" duration=5.561102ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add column labels to alert_rule_version"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add column labels to alert_rule_version" duration=5.262468ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add rule_group_idx column to alert_rule_version"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add rule_group_idx column to alert_rule_version" duration=9.253545ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="add is_paused column to alert_rule_versions table"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="add is_paused column to alert_rule_versions table" duration=6.124909ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="fix is_paused column for alert_rule_version table"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="fix is_paused column for alert_rule_version table" duration=212.438µs
INFO [12-09|12:41:30] Executing migration                      logger=migrator id=create_alert_configuration_table
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id=create_alert_configuration_table duration=40.632949ms
INFO [12-09|12:41:30] Executing migration                      logger=migrator id="Add column default in alert_configuration"
INFO [12-09|12:41:30] Migration successfully executed          logger=migrator id="Add column default in alert_configuration" duration=7.396994ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" duration=528.755µs
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add column org_id in alert_configuration"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add column org_id in alert_configuration" duration=5.5711ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add index in alert_configuration table on org_id column"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add index in alert_configuration table on org_id column" duration=1.042483ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add configuration_hash column to alert_configuration"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add configuration_hash column to alert_configuration" duration=5.770515ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id=create_ngalert_configuration_table
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id=create_ngalert_configuration_table duration=765.043µs
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add index in ngalert_configuration on org_id column"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add index in ngalert_configuration on org_id column" duration=1.317412ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add column send_alerts_to in ngalert_configuration"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add column send_alerts_to in ngalert_configuration" duration=8.149261ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="create provenance_type table"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="create provenance_type table" duration=1.279868ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add index to uniquify (record_key, record_type, org_id) columns"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add index to uniquify (record_key, record_type, org_id) columns" duration=2.880879ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="create alert_image table"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="create alert_image table" duration=3.796684ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add unique index on token to alert_image table"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add unique index on token to alert_image table" duration=1.277298ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="support longer URLs in alert_image table"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="support longer URLs in alert_image table" duration=17.887734ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id=create_alert_configuration_history_table
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id=create_alert_configuration_history_table duration=2.951274ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="drop non-unique orgID index on alert_configuration"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="drop non-unique orgID index on alert_configuration" duration=2.820674ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="drop unique orgID index on alert_configuration if exists"
WARN [12-09|12:41:31] Skipping migration: Already executed, but not recorded in migration log logger=migrator id="drop unique orgID index on alert_configuration if exists"
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="extract alertmanager configuration history to separate table"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="extract alertmanager configuration history to separate table" duration=931.257µs
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add unique index on orgID to alert_configuration"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add unique index on orgID to alert_configuration" duration=52.151688ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add last_applied column to alert_configuration_history"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add last_applied column to alert_configuration_history" duration=11.658496ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="create library_element table v1"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="create library_element table v1" duration=1.629531ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add index library_element org_id-folder_id-name-kind"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add index library_element org_id-folder_id-name-kind" duration=1.431215ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="create library_element_connection table v1"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="create library_element_connection table v1" duration=856.773µs
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add index library_element_connection element_id-kind-connection_id"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add index library_element_connection element_id-kind-connection_id" duration=1.168624ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add unique index library_element org_id_uid"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add unique index library_element org_id_uid" duration=1.667288ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="increase max description length to 2048"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="increase max description length to 2048" duration=82.045µs
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="alter library_element model to mediumtext"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="alter library_element model to mediumtext" duration=142.517µs
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add library_element folder uid"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add library_element folder uid" duration=8.981833ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="populate library_element folder_uid"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="populate library_element folder_uid" duration=553.034µs
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add index library_element org_id-folder_uid-name-kind"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add index library_element org_id-folder_uid-name-kind" duration=1.290819ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="clone move dashboard alerts to unified alerting"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="clone move dashboard alerts to unified alerting" duration=371.649µs
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="create data_keys table"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="create data_keys table" duration=886.396µs
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="create secrets table"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="create secrets table" duration=838.441µs
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="rename data_keys name column to id"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="rename data_keys name column to id" duration=27.549025ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add name column into data_keys"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add name column into data_keys" duration=6.795635ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="copy data_keys id column values into name"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="copy data_keys id column values into name" duration=490.879µs
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="rename data_keys name column to label"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="rename data_keys name column to label" duration=104.829045ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="rename data_keys id column back to name"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="rename data_keys id column back to name" duration=70.776112ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="create kv_store table v1"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="create kv_store table v1" duration=3.670352ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add index kv_store.org_id-namespace-key"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add index kv_store.org_id-namespace-key" duration=1.537479ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="update dashboard_uid and panel_id from existing annotations"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="update dashboard_uid and panel_id from existing annotations" duration=1.582091ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="create permission table"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="create permission table" duration=1.329163ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add unique index permission.role_id"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add unique index permission.role_id" duration=1.06601ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add unique index role_id_action_scope"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add unique index role_id_action_scope" duration=26.117327ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="create role table"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="create role table" duration=1.271204ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add column display_name"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add column display_name" duration=5.395764ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add column group_name"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add column group_name" duration=5.976637ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add index role.org_id"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add index role.org_id" duration=1.501935ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add unique index role_org_id_name"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add unique index role_org_id_name" duration=1.350531ms
INFO [12-09|12:41:31] Executing migration                      logger=migrator id="add index role_org_id_uid"
INFO [12-09|12:41:31] Migration successfully executed          logger=migrator id="add index role_org_id_uid" duration=11.330242ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create team role table"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create team role table" duration=1.594241ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add index team_role.org_id"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add index team_role.org_id" duration=1.465082ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add unique index team_role_org_id_team_id_role_id"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add unique index team_role_org_id_team_id_role_id" duration=1.890745ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add index team_role.team_id"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add index team_role.team_id" duration=1.252133ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create user role table"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create user role table" duration=1.273286ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add index user_role.org_id"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add index user_role.org_id" duration=2.634639ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add unique index user_role_org_id_user_id_role_id"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add unique index user_role_org_id_user_id_role_id" duration=22.450231ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add index user_role.user_id"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add index user_role.user_id" duration=8.17224ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create builtin role table"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create builtin role table" duration=3.483088ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add index builtin_role.role_id"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add index builtin_role.role_id" duration=1.510267ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add index builtin_role.name"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add index builtin_role.name" duration=4.548773ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="Add column org_id to builtin_role table"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="Add column org_id to builtin_role table" duration=9.527367ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add index builtin_role.org_id"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add index builtin_role.org_id" duration=3.803989ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add unique index builtin_role_org_id_role_id_role"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add unique index builtin_role_org_id_role_id_role" duration=3.067928ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="Remove unique index role_org_id_uid"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="Remove unique index role_org_id_uid" duration=1.646624ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add unique index role.uid"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add unique index role.uid" duration=3.887077ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create seed assignment table"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create seed assignment table" duration=3.404623ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add unique index builtin_role_role_name"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add unique index builtin_role_role_name" duration=2.131173ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add column hidden to role table"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add column hidden to role table" duration=8.301383ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="permission kind migration"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="permission kind migration" duration=7.776665ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="permission attribute migration"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="permission attribute migration" duration=8.797226ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="permission identifier migration"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="permission identifier migration" duration=6.145655ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add permission identifier index"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add permission identifier index" duration=1.937776ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add permission action scope role_id index"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add permission action scope role_id index" duration=1.118258ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="remove permission role_id action scope index"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="remove permission role_id action scope index" duration=1.396093ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add group mapping UID column to user_role table"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add group mapping UID column to user_role table" duration=8.689629ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add user_role org ID, user ID, role ID, group mapping UID index"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add user_role org ID, user ID, role ID, group mapping UID index" duration=1.270401ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="remove user_role org ID, user ID, role ID index"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="remove user_role org ID, user ID, role ID index" duration=958.701µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create query_history table v1"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create query_history table v1" duration=979.491µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add index query_history.org_id-created_by-datasource_uid"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add index query_history.org_id-created_by-datasource_uid" duration=1.007415ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="alter table query_history alter column created_by type to bigint"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="alter table query_history alter column created_by type to bigint" duration=77.145µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create query_history_details table v1"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create query_history_details table v1" duration=1.078604ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="rbac disabled migrator"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="rbac disabled migrator" duration=38.523µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="teams permissions migration"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="teams permissions migration" duration=757.173µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="dashboard permissions"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="dashboard permissions" duration=820.878µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="dashboard permissions uid scopes"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="dashboard permissions uid scopes" duration=735.918µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="drop managed folder create actions"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="drop managed folder create actions" duration=490.474µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="alerting notification permissions"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="alerting notification permissions" duration=1.005862ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create query_history_star table v1"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create query_history_star table v1" duration=1.338524ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add index query_history.user_id-query_uid"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add index query_history.user_id-query_uid" duration=1.137618ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add column org_id in query_history_star"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add column org_id in query_history_star" duration=5.499368ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="alter table query_history_star_mig column user_id type to bigint"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="alter table query_history_star_mig column user_id type to bigint" duration=68.743µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create correlation table v1"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create correlation table v1" duration=1.378306ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add index correlations.uid"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add index correlations.uid" duration=1.056585ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add index correlations.source_uid"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add index correlations.source_uid" duration=1.141954ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add correlation config column"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add correlation config column" duration=8.780079ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="drop index IDX_correlation_uid - v1"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="drop index IDX_correlation_uid - v1" duration=1.051905ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="drop index IDX_correlation_source_uid - v1"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="drop index IDX_correlation_source_uid - v1" duration=1.148057ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="Rename table correlation to correlation_tmp_qwerty - v1"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="Rename table correlation to correlation_tmp_qwerty - v1" duration=32.718619ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create correlation v2"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create correlation v2" duration=4.277095ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create index IDX_correlation_uid - v2"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create index IDX_correlation_uid - v2" duration=1.21459ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create index IDX_correlation_source_uid - v2"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create index IDX_correlation_source_uid - v2" duration=2.090541ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create index IDX_correlation_org_id - v2"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create index IDX_correlation_org_id - v2" duration=1.012577ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="copy correlation v1 to v2"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="copy correlation v1 to v2" duration=464.582µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="drop correlation_tmp_qwerty"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="drop correlation_tmp_qwerty" duration=25.370796ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add provisioning column"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add provisioning column" duration=10.495052ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add type column"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add type column" duration=8.12223ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create entity_events table"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create entity_events table" duration=1.093548ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create dashboard public config v1"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create dashboard public config v1" duration=921.291µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="drop index UQE_dashboard_public_config_uid - v1"
WARN [12-09|12:41:32] Skipping migration: Already executed, but not recorded in migration log logger=migrator id="drop index UQE_dashboard_public_config_uid - v1"
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1"
WARN [12-09|12:41:32] Skipping migration: Already executed, but not recorded in migration log logger=migrator id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1"
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="Drop old dashboard public config table"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="Drop old dashboard public config table" duration=827.791µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="recreate dashboard public config v1"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="recreate dashboard public config v1" duration=1.168305ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create index UQE_dashboard_public_config_uid - v1"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create index UQE_dashboard_public_config_uid - v1" duration=892.852µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" duration=1.040042ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="drop index UQE_dashboard_public_config_uid - v2"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="drop index UQE_dashboard_public_config_uid - v2" duration=2.231705ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=1.657521ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="Drop public config table"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="Drop public config table" duration=742.281µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="Recreate dashboard public config v2"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="Recreate dashboard public config v2" duration=2.511835ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create index UQE_dashboard_public_config_uid - v2"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create index UQE_dashboard_public_config_uid - v2" duration=1.075838ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=838.897µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create index UQE_dashboard_public_config_access_token - v2"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create index UQE_dashboard_public_config_access_token - v2" duration=1.662886ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="Rename table dashboard_public_config to dashboard_public - v2"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="Rename table dashboard_public_config to dashboard_public - v2" duration=23.556948ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add annotations_enabled column"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add annotations_enabled column" duration=9.569952ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add time_selection_enabled column"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add time_selection_enabled column" duration=7.056302ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="delete orphaned public dashboards"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="delete orphaned public dashboards" duration=430.636µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="add share column"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="add share column" duration=5.425419ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="backfill empty share column fields with default of public"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="backfill empty share column fields with default of public" duration=305.283µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create file table"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create file table" duration=995.571µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="file table idx: path natural pk"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="file table idx: path natural pk" duration=1.047942ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="file table idx: parent_folder_path_hash fast folder retrieval"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="file table idx: parent_folder_path_hash fast folder retrieval" duration=1.034668ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="create file_meta table"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="create file_meta table" duration=719.052µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="file table idx: path key"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="file table idx: path key" duration=1.092972ms
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="set path collation in file table"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="set path collation in file table" duration=73.559µs
INFO [12-09|12:41:32] Executing migration                      logger=migrator id="migrate contents column to mediumblob for MySQL"
INFO [12-09|12:41:32] Migration successfully executed          logger=migrator id="migrate contents column to mediumblob for MySQL" duration=79.116µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="managed permissions migration"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="managed permissions migration" duration=647.265µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="managed folder permissions alert actions migration"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="managed folder permissions alert actions migration" duration=167.237µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="RBAC action name migrator"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="RBAC action name migrator" duration=5.665883ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Add UID column to playlist"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Add UID column to playlist" duration=7.79088ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Update uid column values in playlist"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Update uid column values in playlist" duration=177.575µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Add index for uid in playlist"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Add index for uid in playlist" duration=2.020144ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="update group index for alert rules"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="update group index for alert rules" duration=430.771µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="managed folder permissions alert actions repeated migration"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="managed folder permissions alert actions repeated migration" duration=168.624µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="admin only folder/dashboard permission"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="admin only folder/dashboard permission" duration=788.662µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add action column to seed_assignment"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add action column to seed_assignment" duration=6.838311ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add scope column to seed_assignment"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add scope column to seed_assignment" duration=5.86922ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="remove unique index builtin_role_role_name before nullable update"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="remove unique index builtin_role_role_name before nullable update" duration=1.482752ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="update seed_assignment role_name column to nullable"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="update seed_assignment role_name column to nullable" duration=101.013005ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add unique index builtin_role_name back"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add unique index builtin_role_name back" duration=4.753226ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add unique index builtin_role_action_scope"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add unique index builtin_role_action_scope" duration=9.954144ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add primary key to seed_assigment"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add primary key to seed_assigment" duration=23.915389ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add origin column to seed_assignment"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add origin column to seed_assignment" duration=9.623188ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add origin to plugin seed_assignment"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add origin to plugin seed_assignment" duration=302.357µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="prevent seeding OnCall access"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="prevent seeding OnCall access" duration=167.324µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="managed folder permissions alert actions repeated fixed migration"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="managed folder permissions alert actions repeated fixed migration" duration=198.15µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="managed folder permissions library panel actions migration"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="managed folder permissions library panel actions migration" duration=199.351µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="migrate external alertmanagers to datsourcse"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="migrate external alertmanagers to datsourcse" duration=3.056048ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="create folder table"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="create folder table" duration=1.340929ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Add index for parent_uid"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Add index for parent_uid" duration=2.472041ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Add unique index for folder.uid and folder.org_id"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Add unique index for folder.uid and folder.org_id" duration=3.408184ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Update folder title length"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Update folder title length" duration=564.603µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Add unique index for folder.title and folder.parent_uid"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Add unique index for folder.title and folder.parent_uid" duration=2.191921ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Remove unique index for folder.title and folder.parent_uid"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Remove unique index for folder.title and folder.parent_uid" duration=8.550205ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Add unique index for title, parent_uid, and org_id"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Add unique index for title, parent_uid, and org_id" duration=1.941857ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Sync dashboard and folder table"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Sync dashboard and folder table" duration=1.184864ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Remove ghost folders from the folder table"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Remove ghost folders from the folder table" duration=673.38µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Remove unique index UQE_folder_uid_org_id"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Remove unique index UQE_folder_uid_org_id" duration=1.136889ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Add unique index UQE_folder_org_id_uid"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Add unique index UQE_folder_org_id_uid" duration=1.457872ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Remove unique index UQE_folder_title_parent_uid_org_id"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Remove unique index UQE_folder_title_parent_uid_org_id" duration=1.271363ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Add unique index UQE_folder_org_id_parent_uid_title"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Add unique index UQE_folder_org_id_parent_uid_title" duration=1.259196ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Remove index IDX_folder_parent_uid_org_id"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Remove index IDX_folder_parent_uid_org_id" duration=1.277062ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="create anon_device table"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="create anon_device table" duration=1.105663ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add unique index anon_device.device_id"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add unique index anon_device.device_id" duration=1.210608ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add index anon_device.updated_at"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add index anon_device.updated_at" duration=1.549419ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="create signing_key table"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="create signing_key table" duration=894.106µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add unique index signing_key.key_id"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add unique index signing_key.key_id" duration=1.3771ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="set legacy alert migration status in kvstore"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="set legacy alert migration status in kvstore" duration=2.008231ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="migrate record of created folders during legacy migration to kvstore"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="migrate record of created folders during legacy migration to kvstore" duration=866.321µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Add folder_uid for dashboard"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Add folder_uid for dashboard" duration=8.811139ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Populate dashboard folder_uid column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Populate dashboard folder_uid column" duration=843.747µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Add unique index for dashboard_org_id_folder_uid_title"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Add unique index for dashboard_org_id_folder_uid_title" duration=28.83µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Delete unique index for dashboard_org_id_folder_id_title"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Delete unique index for dashboard_org_id_folder_id_title" duration=2.201433ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Delete unique index for dashboard_org_id_folder_uid_title"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Delete unique index for dashboard_org_id_folder_uid_title" duration=113.281µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Add unique index for dashboard_org_id_folder_uid_title_is_folder"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" duration=1.323148ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Restore index for dashboard_org_id_folder_id_title"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Restore index for dashboard_org_id_folder_id_title" duration=999.248µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="create sso_setting table"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="create sso_setting table" duration=2.07427ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="copy kvstore migration status to each org"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="copy kvstore migration status to each org" duration=899.391µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add back entry for orgid=0 migrated status"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add back entry for orgid=0 migrated status" duration=1.832677ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="managed dashboard permissions annotation actions migration"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="managed dashboard permissions annotation actions migration" duration=1.509964ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="create cloud_migration table v1"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="create cloud_migration table v1" duration=1.720616ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="create cloud_migration_run table v1"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="create cloud_migration_run table v1" duration=1.286614ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add stack_id column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add stack_id column" duration=20.324316ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add region_slug column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add region_slug column" duration=6.347791ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add cluster_slug column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add cluster_slug column" duration=7.409586ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add migration uid column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add migration uid column" duration=6.579855ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Update uid column values for migration"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Update uid column values for migration" duration=157.891µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Add unique index migration_uid"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Add unique index migration_uid" duration=1.60142ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add migration run uid column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add migration run uid column" duration=6.820699ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Update uid column values for migration run"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Update uid column values for migration run" duration=246.959µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Add unique index migration_run_uid"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Add unique index migration_run_uid" duration=3.651766ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Rename table cloud_migration to cloud_migration_session_tmp_qwerty - v1"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Rename table cloud_migration to cloud_migration_session_tmp_qwerty - v1" duration=34.134575ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="create cloud_migration_session v2"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="create cloud_migration_session v2" duration=1.098546ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="create index UQE_cloud_migration_session_uid - v2"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="create index UQE_cloud_migration_session_uid - v2" duration=1.306365ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="copy cloud_migration_session v1 to v2"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="copy cloud_migration_session v1 to v2" duration=968.301µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="drop cloud_migration_session_tmp_qwerty"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="drop cloud_migration_session_tmp_qwerty" duration=1.220157ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="Rename table cloud_migration_run to cloud_migration_snapshot_tmp_qwerty - v1"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="Rename table cloud_migration_run to cloud_migration_snapshot_tmp_qwerty - v1" duration=17.052836ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="create cloud_migration_snapshot v2"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="create cloud_migration_snapshot v2" duration=1.003035ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="create index UQE_cloud_migration_snapshot_uid - v2"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="create index UQE_cloud_migration_snapshot_uid - v2" duration=1.170823ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="copy cloud_migration_snapshot v1 to v2"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="copy cloud_migration_snapshot v1 to v2" duration=485.823µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="drop cloud_migration_snapshot_tmp_qwerty"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="drop cloud_migration_snapshot_tmp_qwerty" duration=1.607106ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add snapshot upload_url column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add snapshot upload_url column" duration=9.384335ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add snapshot status column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add snapshot status column" duration=8.975262ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add snapshot local_directory column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add snapshot local_directory column" duration=8.862777ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add snapshot gms_snapshot_uid column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add snapshot gms_snapshot_uid column" duration=10.915234ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add snapshot encryption_key column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add snapshot encryption_key column" duration=10.011232ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add snapshot error_string column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add snapshot error_string column" duration=8.199227ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="create cloud_migration_resource table v1"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="create cloud_migration_resource table v1" duration=1.800261ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="delete cloud_migration_snapshot.result column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="delete cloud_migration_snapshot.result column" duration=49.390095ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add cloud_migration_resource.name column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add cloud_migration_resource.name column" duration=10.997325ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add cloud_migration_resource.parent_name column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add cloud_migration_resource.parent_name column" duration=7.269088ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add cloud_migration_session.org_id column"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add cloud_migration_session.org_id column" duration=5.62367ms
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="alter kv_store.value to longtext"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="alter kv_store.value to longtext" duration=72.895µs
INFO [12-09|12:41:33] Executing migration                      logger=migrator id="add notification_settings column to alert_rule table"
INFO [12-09|12:41:33] Migration successfully executed          logger=migrator id="add notification_settings column to alert_rule table" duration=5.746219ms
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="add notification_settings column to alert_rule_version table"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="add notification_settings column to alert_rule_version table" duration=10.356835ms
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="removing scope from alert.instances:read action migration"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="removing scope from alert.instances:read action migration" duration=531.893µs
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="managed folder permissions alerting silences actions migration"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="managed folder permissions alerting silences actions migration" duration=412.929µs
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="add record column to alert_rule table"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="add record column to alert_rule table" duration=5.959654ms
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="add record column to alert_rule_version table"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="add record column to alert_rule_version table" duration=6.224061ms
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="add resolved_at column to alert_instance table"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="add resolved_at column to alert_instance table" duration=6.974869ms
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="add last_sent_at column to alert_instance table"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="add last_sent_at column to alert_instance table" duration=6.294413ms
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="Enable traceQL streaming for all Tempo datasources"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="Enable traceQL streaming for all Tempo datasources" duration=45.559µs
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="Add scope to alert.notifications.receivers:read and alert.notifications.receivers.secrets:read"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="Add scope to alert.notifications.receivers:read and alert.notifications.receivers.secrets:read" duration=630.654µs
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="add metadata column to alert_rule table"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="add metadata column to alert_rule table" duration=6.326216ms
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="add metadata column to alert_rule_version table"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="add metadata column to alert_rule_version table" duration=7.470095ms
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="delete orphaned service account permissions"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="delete orphaned service account permissions" duration=208.562µs
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="adding action set permissions"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="adding action set permissions" duration=682.457µs
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="create user_external_session table"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="create user_external_session table" duration=1.268139ms
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="increase name_id column length to 1024"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="increase name_id column length to 1024" duration=72.994µs
INFO [12-09|12:41:34] Executing migration                      logger=migrator id="increase session_id column length to 1024"
INFO [12-09|12:41:34] Migration successfully executed          logger=migrator id="increase session_id column length to 1024" duration=70.413µs
INFO [12-09|12:41:34] migrations completed                     logger=migrator performed=611 skipped=0 duration=7.436569136s
INFO [12-09|12:41:34] Unlocking database                       logger=migrator
INFO [12-09|12:41:34] Created default admin                    logger=sqlstore user=admin
INFO [12-09|12:41:34] Created default organization             logger=sqlstore
INFO [12-09|12:41:34] Envelope encryption state                logger=secrets enabled=true current provider=secretKey.v1
INFO [12-09|12:41:34] Restored cache from database             logger=plugin.angulardetectorsprovider.dynamic duration=455.927µs
INFO [12-09|12:41:34] Loading plugins...                       logger=plugin.store
ERROR[12-09|12:41:34] Could not register plugin                logger=plugins.registration pluginId=xychart error="plugin xychart is already registered"
ERROR[12-09|12:41:34] Could not initialize plugin              logger=plugins.initialization pluginId=xychart error="plugin xychart is already registered"
WARN [12-09|12:41:34] Skipping finding plugins as directory does not exist logger=local.finder path=/usr/share/grafana/plugins-bundled
INFO [12-09|12:41:34] Plugins loaded                           logger=plugin.store count=54 duration=187.322337ms
INFO [12-09|12:41:34] Query Service initialization             logger=query_data
INFO [12-09|12:41:34] Live Push Gateway initialization         logger=live.push_http
INFO [12-09|12:41:34] Applying new configuration to Alertmanager logger=ngalert.notifier.alertmanager org=1 configHash=d2c56faca6af2a5772ff4253222f7386
INFO [12-09|12:41:34] Running in alternative execution of Error/NoData mode logger=ngalert.state.manager
INFO [12-09|12:41:34] registering usage stat providers         logger=infra.usagestats.collector usageStatsProvidersLen=2
INFO [12-09|12:41:34] starting to provision alerting           logger=provisioning.alerting
INFO [12-09|12:41:34] finished to provision alerting           logger=provisioning.alerting
INFO [12-09|12:41:34] Installing plugin                        logger=plugin.backgroundinstaller pluginId=grafana-lokiexplore-app version=
INFO [12-09|12:41:34] Storage starting                         logger=grafanaStorageLogger
INFO [12-09|12:41:34] HTTP Server Listen                       logger=http.server address=[::]:3000 protocol=http subUrl= socket=
INFO [12-09|12:41:34] Warming state cache for startup          logger=ngalert.state.manager
INFO [12-09|12:41:34] Starting MultiOrg Alertmanager           logger=ngalert.multiorg.alertmanager
INFO [12-09|12:41:34] starting to provision dashboards         logger=provisioning.dashboard
INFO [12-09|12:41:34] finished to provision dashboards         logger=provisioning.dashboard
INFO [12-09|12:41:35] State cache has been initialized         logger=ngalert.state.manager states=0 duration=163.237616ms
INFO [12-09|12:41:35] Starting scheduler                       logger=ngalert.scheduler tickInterval=10s maxAttempts=1
INFO [12-09|12:41:35] starting                                 logger=ticker first_tick=2024-12-09T12:41:40Z
INFO [12-09|12:41:36] Adding GroupVersion iam.grafana.app v0alpha1 to ResourceManager logger=grafana-apiserver
INFO [12-09|12:41:36] Adding GroupVersion playlist.grafana.app v0alpha1 to ResourceManager logger=grafana-apiserver
INFO [12-09|12:41:36] Adding GroupVersion featuretoggle.grafana.app v0alpha1 to ResourceManager logger=grafana-apiserver

ERROR[12-09|12:41:44] Failed to install plugin                 logger=plugin.backgroundinstaller pluginId=grafana-lokiexplore-app version= error="Get \"https://grafana.com/api/plugins/grafana-lokiexplore-app/versions\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
ERROR[12-09|12:41:44] Update check failed                      logger=grafana.update.checker error="failed to get stable version from grafana.com: Get \"https://grafana.com/api/grafana/versions/stable\": dial tcp 34.120.177.193:443: i/o timeout" duration=10.010819827s
ERROR[12-09|12:41:44] Error while updating detectors           logger=plugin.angulardetectorsprovider.dynamic error="fetch: http do: Get \"https://grafana.com/api/plugins/angular_patterns\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
INFO [12-09|12:41:44] Patterns update finished                 logger=plugin.angulardetectorsprovider.dynamic duration=10.000769162s
ERROR[12-09|12:41:45] Error downloading plugin manifest keys   logger=plugin.signature.key_retriever error="Get \"https://grafana.com/api/plugins/ci/keys\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
^[[6~ERROR[12-09|12:42:55] Error downloading plugin manifest keys   logger=plugin.signature.key_retriever error="Get \"https://grafana.com/api/plugins/ci/keys\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
INFO [12-09|12:43:14] Usage stats are ready to report          logger=infra.usagestats

julianocosta89 · 2024-12-09T13:17:13Z

In my case I don't have this last error:

...
logger=grafana-apiserver t=2024-12-09T13:10:52.455889625Z level=info msg="Adding GroupVersion iam.grafana.app v0alpha1 to ResourceManager"
logger=plugin.installer t=2024-12-09T13:10:52.551897708Z level=info msg="Installing plugin" pluginId=grafana-lokiexplore-app version=
logger=installer.fs t=2024-12-09T13:10:52.649395125Z level=info msg="Downloaded and extracted grafana-lokiexplore-app v1.0.4 zip successfully to /var/lib/grafana/plugins/grafana-lokiexplore-app"
logger=plugins.registration t=2024-12-09T13:10:52.663713458Z level=info msg="Plugin registered" pluginId=grafana-lokiexplore-app
logger=plugin.backgroundinstaller t=2024-12-09T13:10:52.663736958Z level=info msg="Plugin successfully installed" pluginId=grafana-lokiexplore-app version= duration=701.7015ms
logger=infra.usagestats t=2024-12-09T13:12:29.043229003Z level=info msg="Usage stats are ready to report"

@cedricziel would you be able to help @rkrath123?

puckpuck · 2024-12-09T16:15:03Z

Is this cluster air-gapped from the rest of the internet? All these errors are issues with communicating out to Grafana over https.

mviitane · 2024-12-11T16:33:41Z

I have the same problem with the latest helm chart.
It's not an air-gapped setup. I can pull the images just fine.
I'm running k0s on Ubuntu.

$ helm ls -A
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                           APP VERSION
my-otel-demo    demo            1               2024-12-11 18:02:11.94344249 +0200 EET  deployed        opentelemetry-demo-0.33.7       1.12.0     

$ helm version
version.BuildInfo{Version:"v3.16.2", GitCommit:"13654a52f7c70a143b1dd51416d633e1071faffb", GitTreeState:"clean", GoVersion:"go1.22.7"}

rkrath123 added the bug Something isn't working label Dec 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

getting CrashLoopBackOff error with grafana pods #1818

getting CrashLoopBackOff error with grafana pods #1818

rkrath123 commented Dec 7, 2024 •

edited by julianocosta89

Loading

julianocosta89 commented Dec 9, 2024

rkrath123 commented Dec 9, 2024 •

edited

Loading

julianocosta89 commented Dec 9, 2024

rkrath123 commented Dec 9, 2024

julianocosta89 commented Dec 9, 2024

rkrath123 commented Dec 9, 2024 •

edited

Loading

julianocosta89 commented Dec 9, 2024

rkrath123 commented Dec 9, 2024

julianocosta89 commented Dec 9, 2024

rkrath123 commented Dec 9, 2024

julianocosta89 commented Dec 9, 2024

rkrath123 commented Dec 9, 2024

rkrath123 commented Dec 9, 2024

julianocosta89 commented Dec 9, 2024

puckpuck commented Dec 9, 2024

mviitane commented Dec 11, 2024 •

edited

Loading

getting CrashLoopBackOff error with grafana pods #1818

getting CrashLoopBackOff error with grafana pods #1818

Comments

rkrath123 commented Dec 7, 2024 • edited by julianocosta89 Loading

Bug Report

Symptom

julianocosta89 commented Dec 9, 2024

rkrath123 commented Dec 9, 2024 • edited Loading

julianocosta89 commented Dec 9, 2024

rkrath123 commented Dec 9, 2024

julianocosta89 commented Dec 9, 2024

rkrath123 commented Dec 9, 2024 • edited Loading

julianocosta89 commented Dec 9, 2024

rkrath123 commented Dec 9, 2024

julianocosta89 commented Dec 9, 2024

rkrath123 commented Dec 9, 2024

julianocosta89 commented Dec 9, 2024

rkrath123 commented Dec 9, 2024

rkrath123 commented Dec 9, 2024

julianocosta89 commented Dec 9, 2024

puckpuck commented Dec 9, 2024

mviitane commented Dec 11, 2024 • edited Loading

rkrath123 commented Dec 7, 2024 •

edited by julianocosta89

Loading

rkrath123 commented Dec 9, 2024 •

edited

Loading

rkrath123 commented Dec 9, 2024 •

edited

Loading

mviitane commented Dec 11, 2024 •

edited

Loading