Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG]kbcli vscale success but ops shows failed #4950

Closed
ahjing99 opened this issue Sep 1, 2023 · 0 comments · Fixed by #4947
Closed

[BUG]kbcli vscale success but ops shows failed #4950

ahjing99 opened this issue Sep 1, 2023 · 0 comments · Fixed by #4947
Assignees
@free6om
Labels
bug kind/bug Something isn't working
Milestone
Release 0.7.0

Comments

@ahjing99
Copy link
Collaborator

ahjing99 commented Sep 1, 2023

➜ ~ kbcli version
Kubernetes: v1.27.3-gke.100
KubeBlocks: 0.7.0-alpha.4
kbcli: 0.7.0-alpha.4

➜  ~ kbcli cluster create  mysqltest             --termination-policy=DoNotTerminate             --monitoring-interval=0 --enable-all-logs=false --cluster-definition=apecloud-mysql --set cpu=100m,memory=0.5Gi,replicas=1,storage=1Gi  --namespace default
Info: --cluster-version is not specified, ClusterVersion ac-mysql-8.0.30 is applied by default
Cluster mysqltest created

➜  ~    kbcli class create custom-0.2c0.6g --cluster-definition apecloud-mysql         --type mysql --cpu 0.2 --memory 0.6Gi  --namespace default `

Successfully create class [custom-0.2c0.6g].

➜  ~ kbcli cluster vscale mysqltest --auto-approve                 --components mysql                 --cpu 200m                 --memory 0.6Gi --namespace default
OpsRequest mysqltest-verticalscaling-qgpml created successfully, you can view the progress:
	kbcli cluster describe-ops mysqltest-verticalscaling-qgpml -n default

➜  ~ k describe ops mysqltest-verticalscaling-qgpml
Name:         mysqltest-verticalscaling-qgpml
Namespace:    default
Labels:       app.kubernetes.io/instance=mysqltest
              ops.kubeblocks.io/ops-type=VerticalScaling
Annotations:  <none>
API Version:  apps.kubeblocks.io/v1alpha1
Kind:         OpsRequest
Metadata:
  Creation Timestamp:  2023-09-01T03:07:02Z
  Finalizers:
    opsrequest.kubeblocks.io/finalizer
  Generate Name:  mysqltest-verticalscaling-
  Generation:     2
  Managed Fields:
    API Version:  apps.kubeblocks.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:generateName:
      f:spec:
        .:
        f:clusterRef:
        f:ttlSecondsBeforeAbort:
        f:type:
        f:verticalScaling:
          .:
          k:{"componentName":"mysql"}:
            .:
            f:componentName:
            f:limits:
              .:
              f:cpu:
            f:requests:
              .:
              f:cpu:
    Manager:      kbcli
    Operation:    Update
    Time:         2023-09-01T03:07:02Z
    API Version:  apps.kubeblocks.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:finalizers:
          .:
          v:"opsrequest.kubeblocks.io/finalizer":
        f:labels:
          .:
          f:app.kubernetes.io/instance:
          f:ops.kubeblocks.io/ops-type:
        f:ownerReferences:
          .:
          k:{"uid":"99e8336e-cd49-4e0b-9f3a-c554fd6166f5"}:
      f:spec:
        f:verticalScaling:
          k:{"componentName":"mysql"}:
            f:limits:
              f:memory:
            f:requests:
              f:memory:
    Manager:      manager
    Operation:    Update
    Time:         2023-09-01T03:07:02Z
    API Version:  apps.kubeblocks.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:clusterGeneration:
        f:completionTimestamp:
        f:components:
          .:
          f:mysql:
            .:
            f:lastFailedTime:
            f:phase:
            f:progressDetails:
        f:conditions:
          .:
          k:{"type":"Failed"}:
            .:
            f:lastTransitionTime:
            f:message:
            f:reason:
            f:status:
            f:type:
          k:{"type":"Progressing"}:
            .:
            f:lastTransitionTime:
            f:message:
            f:reason:
            f:status:
            f:type:
          k:{"type":"Validated"}:
            .:
            f:lastTransitionTime:
            f:message:
            f:reason:
            f:status:
            f:type:
          k:{"type":"VerticalScaling"}:
            .:
            f:lastTransitionTime:
            f:message:
            f:reason:
            f:status:
            f:type:
        f:lastConfiguration:
          .:
          f:components:
            .:
            f:mysql:
              .:
              f:limits:
                .:
                f:cpu:
                f:memory:
              f:requests:
                .:
                f:cpu:
                f:memory:
        f:phase:
        f:progress:
        f:startTimestamp:
    Manager:      manager
    Operation:    Update
    Subresource:  status
    Time:         2023-09-01T03:07:34Z
  Owner References:
    API Version:     apps.kubeblocks.io/v1alpha1
    Kind:            Cluster
    Name:            mysqltest
    UID:             99e8336e-cd49-4e0b-9f3a-c554fd6166f5
  Resource Version:  707392
  UID:               87bf0861-d739-41ca-9189-07c6b7d7f3d9
Spec:
  Cluster Ref:               mysqltest
  Ttl Seconds Before Abort:  0
  Type:                      VerticalScaling
  Vertical Scaling:
    Component Name:  mysql
    Limits:
      Cpu:     200m
      Memory:  644245094400m
    Requests:
      Cpu:     200m
      Memory:  644245094400m
Status:
  Cluster Generation:    3
  Completion Timestamp:  2023-09-01T03:07:34Z
  Components:
    Mysql:
      Last Failed Time:  2023-09-01T03:07:04Z
      Phase:             Failed
      Progress Details:
        Message:     Start to vertical scale: Pod/mysqltest-mysql-0 in Component: mysql
        Object Key:  Pod/mysqltest-mysql-0
        Start Time:  2023-09-01T03:07:03Z
        Status:      Processing
  Conditions:
    Last Transition Time:  2023-09-01T03:07:02Z
    Message:               Start to process the OpsRequest: mysqltest-verticalscaling-qgpml in Cluster: mysqltest
    Reason:                OpsRequestProgressingStarted
    Status:                True
    Type:                  Progressing
    Last Transition Time:  2023-09-01T03:07:02Z
    Message:               OpsRequest: mysqltest-verticalscaling-qgpml is validated
    Reason:                ValidateOpsRequestPassed
    Status:                True
    Type:                  Validated
    Last Transition Time:  2023-09-01T03:07:02Z
    Message:               Start to vertical scale resources in Cluster: mysqltest
    Reason:                VerticalScalingStarted
    Status:                True
    Type:                  VerticalScaling
    Last Transition Time:  2023-09-01T03:07:34Z
    Message:               Failed to process OpsRequest: mysqltest-verticalscaling-qgpml in cluster: mysqltest
    Reason:                OpsRequestFailed
    Status:                False
    Type:                  Failed
  Last Configuration:
    Components:
      Mysql:
        Limits:
          Cpu:     100m
          Memory:  512Mi
        Requests:
          Cpu:      100m
          Memory:   512Mi
  Phase:            Failed
  Progress:         0/1
  Start Timestamp:  2023-09-01T03:07:02Z
Events:
  Type     Reason                        Age                From                    Message
  ----     ------                        ----               ----                    -------
  Normal   OpsRequestProgressingStarted  84s                ops-request-controller  Start to process the OpsRequest: mysqltest-verticalscaling-qgpml in Cluster: mysqltest
  Normal   ValidateOpsRequestPassed      84s                ops-request-controller  OpsRequest: mysqltest-verticalscaling-qgpml is validated
  Normal   VerticalScalingStarted        84s                ops-request-controller  Start to vertical scale resources in Cluster: mysqltest
  Normal   Processing                    83s                ops-request-controller  Start to vertical scale: Pod/mysqltest-mysql-0 in Component: mysql
  Warning  OpsRequestFailed              52s (x2 over 52s)  ops-request-controller  Failed to process OpsRequest: mysqltest-verticalscaling-qgpml in cluster: mysqltest

➜  ~ kbcli cluster describe mysqltest
Name: mysqltest	 Created Time: Sep 01,2023 11:05 UTC+0800
NAMESPACE   CLUSTER-DEFINITION   VERSION           STATUS    TERMINATION-POLICY
default     apecloud-mysql       ac-mysql-8.0.30   Running   DoNotTerminate

Endpoints:
COMPONENT   MODE        INTERNAL                                         EXTERNAL
mysql       ReadWrite   mysqltest-mysql.default.svc.cluster.local:3306   <none>

Topology:
COMPONENT   INSTANCE            ROLE     STATUS    AZ              NODE                                                CREATED-TIME
mysql       mysqltest-mysql-0   leader   Running   us-central1-c   gke-yjtest-default-pool-f59be211-53bj/10.128.0.47   Sep 01,2023 11:07 UTC+0800

Resources Allocation:
COMPONENT   DEDICATED   CPU(REQUEST/LIMIT)   MEMORY(REQUEST/LIMIT)           STORAGE-SIZE   STORAGE-CLASS
mysql       false       200m / 200m          644245094400m / 644245094400m   data:1Gi       kb-default-sc

Images:
COMPONENT   TYPE    IMAGE
mysql       mysql   registry.cn-hangzhou.aliyuncs.com/apecloud/apecloud-mysql-server:8.0.30-5.beta2.20230830.g4ca1eb8.13

Data Protection:
AUTO-BACKUP   BACKUP-SCHEDULE   TYPE     BACKUP-TTL   LAST-SCHEDULE   RECOVERABLE-TIME
Disabled      <none>            <none>   7d           <none>          <none>

Show cluster events: kbcli cluster list-events -n default mysqltest
@ahjing99 ahjing99 added the kind/bug Something isn't working label Sep 1, 2023
@ahjing99 ahjing99 added this to the Release 0.7.0 milestone Sep 1, 2023
@ahjing99 ahjing99 assigned free6om and unassigned wangyelei Sep 1, 2023
@free6om free6om linked a pull request Sep 2, 2023 that will close this issue
fix: cluster phase changed to running when role probe not ready #4947
Merged
@JashBook JashBook mentioned this issue Sep 7, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@free6om free6om

Labels
bug kind/bug Something isn't working
Projects
None yet
Milestone
Release 0.7.0
Development

Successfully merging a pull request may close this issue.

fix: cluster phase changed to running when role probe not ready apecloud/kubeblocks
4 participants
@free6om @wangyelei @ahjing99 @apecloud-bot