Fail to resize Tikv PVCs

[hongshaoyang]

Bug Report

What version of Kubernetes are you using?

v1.26.15-aliyun.1 running on Alicloud ACK

What version of TiDB Operator are you using?

TiDB Operator Version: version.Info{GitVersion:"v1.4.5", GitCommit:"e98d820d5421b028015c341e3fa2db2e4016dbb0", GitTreeState:"clean", BuildDate:"2023-06-26T02:21:14Z", GoVersion:"go1.19.10", Compiler:"gc", Platform:"linux/arm64"}

What storage classes exist in the Kubernetes cluster and what are used for PD/TiKV pods?

StorageClasses:

NAME                                 PROVISIONER                       RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
alicloud-disk-available              diskplugin.csi.alibabacloud.com   Delete          Immediate              true                   557d
alicloud-disk-efficiency (default)   diskplugin.csi.alibabacloud.com   Delete          Immediate              true                   557d
alicloud-disk-essd                   diskplugin.csi.alibabacloud.com   Delete          Immediate              true                   557d
alicloud-disk-ssd                    diskplugin.csi.alibabacloud.com   Delete          Immediate              true                   557d
alicloud-disk-topology               diskplugin.csi.alibabacloud.com   Delete          WaitForFirstConsumer   true                   557d
alicloud-disk-topology-alltype       diskplugin.csi.alibabacloud.com   Delete          WaitForFirstConsumer   true                   557d

PVCs:

tikv-prod-redash-tikv-0                Bound     d-t4n4m5tgnop7v2qian23   1500Gi     RWO            alicloud-disk-topology-alltype   4d21h
tikv-prod-redash-tikv-1                Bound     d-t4nip5kxoy249mdxefow   1500Gi     RWO            alicloud-disk-topology-alltype   4d21h
tikv-prod-redash-tikv-10               Bound     d-t4n38r14xjl6kfribg3l   1800Gi     RWO            alicloud-disk-topology-alltype   521d
tikv-prod-redash-tikv-11               Bound     d-t4nh7zxh3g0tkdwomqm1   1800Gi     RWO            alicloud-disk-topology-alltype   518d
tikv-prod-redash-tikv-12               Bound     d-t4n1wa6dqfemdg3dmdme   1800Gi     RWO            alicloud-disk-topology-alltype   518d
tikv-prod-redash-tikv-13               Bound     d-t4n030w32k3vrtsstbla   1800Gi     RWO            alicloud-disk-topology-alltype   518d
tikv-prod-redash-tikv-14               Bound     d-t4ner7z7lw8mukulptxb   1800Gi     RWO            alicloud-disk-topology-alltype   340d
tikv-prod-redash-tikv-15               Bound     d-t4nf1osbhztaa8ct3id3   1800Gi     RWO            alicloud-disk-topology-alltype   104d
tikv-prod-redash-tikv-16               Pending                                                      alicloud-disk-topology-alltype   4d21h
tikv-prod-redash-tikv-17               Pending                                                      alicloud-disk-topology-alltype   4d21h
tikv-prod-redash-tikv-2                Bound     d-t4n4hmsh6g4cio235r83   1500Gi     RWO            alicloud-disk-topology-alltype   4d21h
tikv-prod-redash-tikv-3                Bound     d-t4n1e280p6kqmsvrhhyw   1800Gi     RWO            alicloud-disk-topology-alltype   529d
tikv-prod-redash-tikv-4                Bound     d-t4ni68o0s2kb9mykou93   1800Gi     RWO            alicloud-disk-topology-alltype   529d
tikv-prod-redash-tikv-5                Bound     d-t4n21ikxof96qumbgya1   1800Gi     RWO            alicloud-disk-topology-alltype   529d
tikv-prod-redash-tikv-6                Bound     d-t4n2vf6dxr6m8b4i4vbh   1800Gi     RWO            alicloud-disk-topology-alltype   529d
tikv-prod-redash-tikv-7                Bound     d-t4nbsfny2g2q4uhd3dhw   1800Gi     RWO            alicloud-disk-topology-alltype   522d
tikv-prod-redash-tikv-8                Bound     d-t4nfmfhdp5p4acq6mxp5   1800Gi     RWO            alicloud-disk-topology-alltype   521d
tikv-prod-redash-tikv-9                Bound     d-t4nil0s0f10guflgq6cw   1800Gi     RWO            alicloud-disk-topology-alltype   521d

What's the status of the TiDB cluster pods?

Status of kind TidbCluster, .status.tikv:

  tikv:
    bootStrapped: true
    image: pingcap/tikv:v7.5.1
    phase: Scale
    statefulSet:
      collisionCount: 0
      currentReplicas: 14
      currentRevision: prod-redash-tikv-58bf4d69df
      observedGeneration: 284
      readyReplicas: 14
      replicas: 14
      updateRevision: prod-redash-tikv-58bf4d69df
      updatedReplicas: 14
    stores:
      "1729778":
        id: "1729778"
        ip: prod-redash-tikv-3.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: "2025-01-10T15:37:47Z"
        leaderCount: 33239
        podName: prod-redash-tikv-3
        state: Up
      "2829612":
        id: "2829612"
        ip: prod-redash-tikv-4.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: "2025-01-07T19:33:09Z"
        leaderCount: 33239
        podName: prod-redash-tikv-4
        state: Up
      "4156292":
        id: "4156292"
        ip: prod-redash-tikv-5.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: "2025-01-10T15:39:23Z"
        leaderCount: 33233
        podName: prod-redash-tikv-5
        state: Up
      "4179344":
        id: "4179344"
        ip: prod-redash-tikv-6.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: "2025-01-10T15:37:17Z"
        leaderCount: 33236
        podName: prod-redash-tikv-6
        state: Up
      "1271929623":
        id: "1271929623"
        ip: prod-redash-tikv-7.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: "2025-01-07T19:45:07Z"
        leaderCount: 33237
        podName: prod-redash-tikv-7
        state: Up
      "1272506199":
        id: "1272506199"
        ip: prod-redash-tikv-8.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: "2025-01-07T19:45:07Z"
        leaderCount: 33238
        podName: prod-redash-tikv-8
        state: Up
      "1272575884":
        id: "1272575884"
        ip: prod-redash-tikv-9.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: "2025-01-10T15:41:26Z"
        leaderCount: 33237
        podName: prod-redash-tikv-9
        state: Up
      "1272575957":
        id: "1272575957"
        ip: prod-redash-tikv-10.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: "2025-01-10T15:41:21Z"
        leaderCount: 33240
        podName: prod-redash-tikv-10
        state: Up
      "1495134941":
        id: "1495134941"
        ip: prod-redash-tikv-11.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: "2025-01-07T19:43:50Z"
        leaderCount: 33238
        podName: prod-redash-tikv-11
        state: Up
      "1563719691":
        id: "1563719691"
        ip: prod-redash-tikv-12.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: "2025-01-10T15:39:20Z"
        leaderCount: 33236
        podName: prod-redash-tikv-12
        state: Up
      "1563727636":
        id: "1563727636"
        ip: prod-redash-tikv-13.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: "2025-01-12T14:40:17Z"
        leaderCount: 2491
        podName: prod-redash-tikv-13
        state: Offline
      "2986667096":
        id: "2986667096"
        ip: prod-redash-tikv-1.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: "2025-01-09T05:29:07Z"
        leaderCount: 33238
        podName: prod-redash-tikv-1
        state: Up
      "2986667099":
        id: "2986667099"
        ip: prod-redash-tikv-2.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: "2025-01-09T05:29:08Z"
        leaderCount: 33239
        podName: prod-redash-tikv-2
        state: Up
      "2986667162":
        id: "2986667162"
        ip: prod-redash-tikv-0.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: "2025-01-09T05:29:35Z"
        leaderCount: 33238
        podName: prod-redash-tikv-0
        state: Up
    synced: true
    tombstoneStores:
      "2279292722":
        id: "2279292722"
        ip: prod-redash-tikv-14.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: null
        leaderCount: 0
        podName: prod-redash-tikv-14
        state: Tombstone
      "2963433803":
        id: "2963433803"
        ip: prod-redash-tikv-15.prod-redash-tikv-peer.prod-tidb.svc
        lastTransitionTime: null
        leaderCount: 0
        podName: prod-redash-tikv-15
        state: Tombstone
    volumes:
      tikv:
        boundCount: 14
        currentCapacity: 1800Gi
        currentCount: 14
        currentStorageClass: alicloud-disk-topology-alltype
        modifiedCapacity: 1900Gi
        modifiedCount: 0
        modifiedStorageClass: alicloud-disk-topology-alltype
        name: tikv
        resizedCapacity: 1900Gi
        resizedCount: 0

What did you do?

• Tidb-operator installed via Helm
• Modify kind TidbCluster .spec.tikv.requests.storage: (1800Gi -> 1900Gi)

What did you expect to see?

• I expect all Tikv PVCs to be resized to 1900Gi via the CSI provisioner in Alicloud ACK.

What did you see instead?

• PVCs are not resized.
• Could it be an issue with integrating Alicloud ACK CSI provisioner?

[hongshaoyang]

Logs from tidb-controller-manager:

[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:03.191745       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:03.191786       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:03.191795       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:03.191805       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:03.191813       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:03.191823       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:03.191835       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:03.191845       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:03.191855       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:03.191866       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:03.191879       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:03.191892       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:03.191902       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:03.191913       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:04.188820       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:06:04Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:04.213616       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:04.213666       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:04.779895       1 request.go:645] Throttling request took 1.024673525s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:05.780224       1 request.go:645] Throttling request took 1.390286564s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/core-vn-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:06.780783       1 request.go:645] Throttling request took 1.58866619s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:07.980500       1 request.go:645] Throttling request took 1.594298314s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:09.180419       1 request.go:645] Throttling request took 1.594362493s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.380681       1 request.go:645] Throttling request took 1.485720157s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.989498       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.989529       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.989536       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.989544       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.989552       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.989560       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.989566       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.989574       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.989582       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.989588       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.989595       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.989600       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.989607       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:10.989615       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:11.580567       1 request.go:645] Throttling request took 1.191284733s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/cluster-5-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:12.580813       1 request.go:645] Throttling request took 1.191815539s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/cluster-2-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:13.388564       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:06:13Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:13.417517       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:13.417567       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:13.780158       1 request.go:645] Throttling request took 1.192080334s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:14.780335       1 request.go:645] Throttling request took 1.090783201s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:15.980421       1 request.go:645] Throttling request took 1.191789381s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/core-id-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:17.180244       1 request.go:645] Throttling request took 1.191469829s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.180416       1 request.go:645] Throttling request took 1.192398506s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/prod-redash-tiflash-3438393
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.197483       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.197522       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.197531       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.197541       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.197554       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.197564       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.197572       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.197586       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.197597       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.197607       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.197616       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.197626       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.197639       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:18.197648       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:19.180766       1 request.go:645] Throttling request took 1.192384246s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:19.993675       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:06:19Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:19.997346       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:33.183051       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:33.183082       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:33.183095       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:33.183103       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:33.183110       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:33.183144       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:33.183768       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:33.183789       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:33.183796       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:33.183803       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:33.183809       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:33.183815       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:33.183821       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:33.183826       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:34.173509       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:06:34Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:34.176489       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:34.766402       1 request.go:645] Throttling request took 1.008783006s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:35.966181       1 request.go:645] Throttling request took 1.529297713s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:37.166459       1 request.go:645] Throttling request took 1.394519962s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:38.366060       1 request.go:645] Throttling request took 1.391889885s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/core-ph-dm-worker-3838313
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:39.566337       1 request.go:645] Throttling request took 1.392461112s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/cx-dm-worker-3632393
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.544668       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.544714       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.544727       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.544739       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.544749       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.544759       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.544769       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.544780       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.544790       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.544804       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.544814       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.544823       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.544864       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.544874       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.567564       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:06:53Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.598656       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.598717       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.747066       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.747098       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.747107       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.747116       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.747127       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.747134       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.747146       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.747156       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.747172       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.747186       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.747193       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.747201       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.747208       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.747219       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.911383       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:06:53Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:06:53.914485       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:03.172097       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:03.172126       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:03.172134       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:03.172141       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:03.172147       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:03.172154       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:03.172160       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:03.172166       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:03.172172       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:03.172179       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:03.172188       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:03.172198       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:03.172204       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:03.172209       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:04.171159       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:07:04Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:04.201035       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:04.201082       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:04.763899       1 request.go:645] Throttling request took 1.008024784s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:05.963534       1 request.go:645] Throttling request took 1.525584642s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:06.963744       1 request.go:645] Throttling request took 1.592977227s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/cluster-4-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:08.163774       1 request.go:645] Throttling request took 1.592893529s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/prod-redash-pd-6365646
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:09.163834       1 request.go:645] Throttling request took 1.594005162s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.363936       1 request.go:645] Throttling request took 1.48850483s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.972353       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.972417       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.972426       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.972432       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.972438       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.972445       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.972450       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.972456       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.972464       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.972472       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.972479       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.972486       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.972495       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:10.972504       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:11.563386       1 request.go:645] Throttling request took 1.19137803s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/cluster-2-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:12.563516       1 request.go:645] Throttling request took 1.192471805s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/core-id-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:13.372156       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:07:13Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:13.403509       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:13.403574       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:13.563600       1 request.go:645] Throttling request took 1.192440265s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/misc-dm-worker-3036353
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:14.763401       1 request.go:645] Throttling request took 1.090214574s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:15.763426       1 request.go:645] Throttling request took 1.193260146s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/prod-redash-pd-6365646
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:16.963609       1 request.go:645] Throttling request took 1.189036127s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.163845       1 request.go:645] Throttling request took 1.193325883s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/prod-redash-tiflash-3438393
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.173424       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.173454       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.173464       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.173472       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.173478       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.173484       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.173490       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.173497       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.173504       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.173511       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.173518       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.173525       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.173532       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:18.173542       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:19.363158       1 request.go:645] Throttling request took 1.189361779s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:19.970724       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:07:19Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.016011       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.016085       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.773045       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.773073       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.773080       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.773086       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.773095       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.773101       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.773107       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.773113       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.773119       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.773132       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.773140       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.773148       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.773157       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:20.773163       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:21.170163       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:07:21Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:21.172962       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[advanced-statefulset-controller-d9d467fb5-wdb55] I0114 03:07:31.774146       1 reflector.go:432] k8s.io/client-go/informers/factory.go:135: Watch close - *v1.PersistentVolumeClaim total 10 items received
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:33.174527       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:33.174553       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:33.174561       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:33.174567       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:33.174573       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:33.174579       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:33.174587       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:33.174595       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:33.174602       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:33.174609       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:33.174615       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:33.174625       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:33.174631       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:33.174636       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:34.170405       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:07:34Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:34.173288       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:34.763241       1 request.go:645] Throttling request took 1.00698147s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:35.962749       1 request.go:645] Throttling request took 1.518061842s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:36.963243       1 request.go:645] Throttling request took 1.39464364s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:38.163591       1 request.go:645] Throttling request took 1.394676138s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:07:39.363570       1 request.go:645] Throttling request took 1.294849771s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:03.174483       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:03.174513       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:03.174521       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:03.174528       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:03.174535       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:03.174542       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:03.174550       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:03.174558       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:03.174566       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:03.174577       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:03.174584       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:03.174591       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:03.174597       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:03.174605       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:04.173499       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:08:04Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:04.176199       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:04.765486       1 request.go:645] Throttling request took 1.007578555s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:05.965813       1 request.go:645] Throttling request took 1.517997364s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:07.165860       1 request.go:645] Throttling request took 1.394316467s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:08.366266       1 request.go:645] Throttling request took 1.391623472s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/cluster-5-dm-worker-3762376
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:09.565246       1 request.go:645] Throttling request took 1.383204153s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/route-dm-worker-6165333
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:33.187460       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:33.187492       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:33.187501       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:33.187511       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:33.187520       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:33.187528       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:33.187541       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:33.187552       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:33.187558       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:33.187565       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:33.187573       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:33.187580       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:33.187590       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:33.187597       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:34.182117       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:08:34Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:34.211414       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:34.211470       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:34.774241       1 request.go:645] Throttling request took 1.00816722s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:35.774640       1 request.go:645] Throttling request took 1.392132779s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/core-sg-th-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:36.974784       1 request.go:645] Throttling request took 1.592669554s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/driver-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:38.174419       1 request.go:645] Throttling request took 1.591076572s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/prod-redash-pd-6365646
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:39.175166       1 request.go:645] Throttling request took 1.593611371s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.374756       1 request.go:645] Throttling request took 1.471664773s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.984599       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.984626       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.984633       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.984639       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.984646       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.984652       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.984658       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.984664       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.984670       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.984677       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.984688       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.984697       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.984704       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:40.984711       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:41.574804       1 request.go:645] Throttling request took 1.192150212s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/core-vn-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:42.774646       1 request.go:645] Throttling request took 1.194480856s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:43.381428       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:08:43Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:43.441438       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:43.441491       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:43.974470       1 request.go:645] Throttling request took 1.192128312s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/core-vn-dm-worker-3832353
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:44.974857       1 request.go:645] Throttling request took 1.192769417s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/hub-dm-worker-6632316
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:46.175058       1 request.go:645] Throttling request took 1.093289821s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:47.374762       1 request.go:645] Throttling request took 1.191499787s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/cluster-4-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.188475       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.188510       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.188522       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.188533       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.188546       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.188616       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.188630       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.188637       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.188645       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.188651       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.188658       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.188667       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.188674       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.188683       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:48.574792       1 request.go:645] Throttling request took 1.194359699s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:49.985063       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:08:49Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.045095       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.045140       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.787621       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.787649       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.787660       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.787668       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.787675       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.787681       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.787687       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.787694       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.787701       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.787708       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.787716       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.787724       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.787730       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:50.787737       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.183047       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:08:51Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.238021       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.238067       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.987021       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.987055       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.987064       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.987072       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.987080       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.987087       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.987094       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.987100       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.987106       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.987111       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.987119       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.987126       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.987132       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:51.987140       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:52.382056       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:08:52Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:52.440137       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:52.440190       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.183773       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.183804       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.183811       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.183819       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.183825       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.183831       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.183841       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.183852       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.183871       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.183883       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.183889       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.183895       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.183902       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.183909       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.582613       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:08:53Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:08:53.585779       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:03.172616       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:03.172642       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:03.172656       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:03.172663       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:03.172669       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:03.172676       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:03.172685       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:03.172694       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:03.172701       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:03.172708       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:03.172720       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:03.172726       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:03.172733       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:03.172740       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:04.171321       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:09:04Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:04.203621       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:04.203665       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:04.764461       1 request.go:645] Throttling request took 1.00735214s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:05.964449       1 request.go:645] Throttling request took 1.519313892s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:07.163973       1 request.go:645] Throttling request took 1.591755895s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:08.164521       1 request.go:645] Throttling request took 1.591501042s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/prod-redash-pd-6365646
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:09.363961       1 request.go:645] Throttling request took 1.592592256s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/prod-redash-monitor-grafana
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.563594       1 request.go:645] Throttling request took 1.48932897s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.977879       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.977931       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.977945       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.977958       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.977974       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.977987       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.978000       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.978013       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.978028       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.978041       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.978056       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.978068       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.978085       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:10.978098       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:11.564289       1 request.go:645] Throttling request took 1.192393246s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/cluster-2-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:12.564441       1 request.go:645] Throttling request took 1.191682603s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/cluster-5-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:13.371366       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:09:13Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:13.379577       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:13.764494       1 request.go:645] Throttling request took 1.193816552s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:33.174779       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:33.174811       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:33.174819       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:33.174826       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:33.174835       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:33.174841       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:33.174847       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:33.174853       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:33.174861       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:33.174867       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:33.174900       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:33.174908       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:33.174915       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:33.174923       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:34.171466       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:09:34Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:34.202393       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:34.202475       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:34.964157       1 request.go:645] Throttling request took 1.165417553s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:36.163756       1 request.go:645] Throttling request took 1.392362274s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:37.363530       1 request.go:645] Throttling request took 1.593397637s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:38.364326       1 request.go:645] Throttling request took 1.593704129s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:39.563765       1 request.go:645] Throttling request took 1.592616476s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/prod-redash-dm-monitor-grafana
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:40.563795       1 request.go:645] Throttling request took 1.491698514s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.173958       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.173985       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.173992       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.173998       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.174004       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.174011       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.174019       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.174027       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.174036       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.174043       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.174051       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.174058       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.174065       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.174071       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:41.563820       1 request.go:645] Throttling request took 1.191308533s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/cluster-5-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:42.564257       1 request.go:645] Throttling request took 1.192807376s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/cluster-2-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:43.570838       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:09:43Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:43.573671       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:09:43.763637       1 request.go:645] Throttling request took 1.19388049s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:03.184104       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:03.184144       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:03.184155       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:03.184165       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:03.184174       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:03.184182       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:03.184189       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:03.184195       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:03.184202       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:03.184210       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:03.184222       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:03.184234       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:03.184245       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:03.184255       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[advanced-statefulset-controller-d9d467fb5-wdb55] I0114 03:10:03.772504       1 reflector.go:432] k8s.io/client-go/informers/factory.go:135: Watch close - *v1.ControllerRevision total 8 items received
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:04.178536       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:10:04Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:04.206371       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:04.206438       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:04.771563       1 request.go:645] Throttling request took 1.013579351s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:05.971488       1 request.go:645] Throttling request took 1.526991987s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:06.971546       1 request.go:645] Throttling request took 1.591965111s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/core-ph-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:07.971731       1 request.go:645] Throttling request took 1.592672994s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:08.971753       1 request.go:645] Throttling request took 1.592540877s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/hub-dm-worker-6632316
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.171775       1 request.go:645] Throttling request took 1.592993046s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/core-ph-dm-worker-3838313
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.980288       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.980315       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.980324       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.980331       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.980337       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.980345       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.980351       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.980380       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.980388       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.980395       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.980403       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.980411       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.980425       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:10.980432       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:11.371950       1 request.go:645] Throttling request took 1.082569058s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:12.571810       1 request.go:645] Throttling request took 1.192568122s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/core-my-mm-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:13.378678       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:10:13Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:13.407275       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:13.407348       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:13.571875       1 request.go:645] Throttling request took 1.192423426s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/cluster-5-dm-worker-3762376
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:14.771598       1 request.go:645] Throttling request took 1.094885082s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:15.772015       1 request.go:645] Throttling request took 1.192325588s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/prod-redash-pd-6365646
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:16.971221       1 request.go:645] Throttling request took 1.19052199s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:17.971841       1 request.go:645] Throttling request took 1.191992276s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/dp-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:18.180709       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:18.180735       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:18.180743       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:18.180749       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:18.180755       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:18.180762       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:18.180769       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:18.180778       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:18.180787       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:18.180794       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:18.180804       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:18.180811       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:18.180822       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:18.180827       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:19.171929       1 request.go:645] Throttling request took 1.194611333s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:19.979247       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:10:19Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.008871       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.008918       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.786673       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.786702       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.786711       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.786719       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.786726       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.786733       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.786740       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.786748       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.786756       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.786765       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.786772       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.786781       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.786787       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:20.786794       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.178925       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:10:21Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.239770       1 tidbcluster_control.go:71] TidbCluster: [prod-tidb/prod-redash] updated successfully
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.239834       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.981348       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.981373       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.981381       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.981388       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.981394       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.981400       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.981406       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.981412       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.981421       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.981428       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.981438       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.981446       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.981451       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:21.981459       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:22.378695       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:10:22Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:22.381206       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:33.180347       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-2 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:33.180407       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-0 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:33.180421       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-4 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:33.180427       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-6 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:33.180434       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-9 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:33.180440       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-3 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:33.180448       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-11 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:33.180455       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-7 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:33.180461       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-8 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:33.180470       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-12 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:33.180479       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-10 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:33.180491       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-1 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1500Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:33.180501       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-13 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:33.180512       1 phase.go:188] volume prod-tidb/tikv-prod-redash-tikv-5 is changed, sc (alicloud-disk-topology-alltype => alicloud-disk-topology-alltype), size (1800Gi => 1900Gi)
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:34.178035       1 tikv_scaler.go:116] scaling in tikv statefulset prod-tidb/prod-redash-tikv, ordinals: [13] (replicas: 13, delete slots: []), scaleInParallelism: 1, scaleInTime: 2025-01-14T03:10:34Z
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:34.180859       1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:34.772011       1 request.go:645] Throttling request took 1.014316154s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:35.772126       1 request.go:645] Throttling request took 1.392658427s, request: GET:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps/core-ph-dm-master-3735326
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:36.972067       1 request.go:645] Throttling request took 1.394693518s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:38.171805       1 request.go:645] Throttling request took 1.394075993s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps
[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:39.371533       1 request.go:645] Throttling request took 1.292585365s, request: POST:https://10.33.0.1:443/api/v1/namespaces/prod-tidb/configmaps

[hongshaoyang]

Note that .status.tikv.volumes.tikv.currentCapacity of kind TidbCluster = 1800Gi instead of 1900Gi

[csuzhangxc]

Can you wait for the TiKV-13 to be scaled in first?

[tidb-controller-manager-5f475c5bc-6zrfg] I0114 03:10:34.180859 1 tidb_cluster_controller.go:140] TidbCluster: prod-tidb/prod-redash, still need sync: TiKV prod-tidb/prod-redash-tikv-13 store 1563727636 is still in cluster, state: Offline, requeuing

Then, maybe you can upgrade TiDB Operator to v1.4.6?

https://docs.pingcap.com/tidb-in-kubernetes/stable/release-1.4.6