Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creating a RedisCluster stuck in Bootstrap state #1214

Open
trynocoding opened this issue Jan 20, 2025 · 0 comments
Open

Creating a RedisCluster stuck in Bootstrap state #1214

trynocoding opened this issue Jan 20, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@trynocoding
Copy link

trynocoding commented Jan 20, 2025

What version of redis operator are you using?
redis-operator version: v0.19.0

redis-operator log

{"level":"info","ts":"2025-01-20T20:04:37+08:00","logger":"setup","msg":"starting manager"}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","logger":"controller-runtime.metrics","msg":"Starting metrics server"}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"starting server","kind":"health probe","addr":"[::]:9081"}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","logger":"controller-runtime.metrics","msg":"Serving metrics server","bindAddress":":9080","secure":false}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting EventSource","controller":"rediscluster","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisCluster","source":"kind source: *v1beta2.RedisCluster"}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting EventSource","controller":"rediscluster","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisCluster","source":"kind source: *v1.StatefulSet"}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting Controller","controller":"rediscluster","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisCluster"}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting EventSource","controller":"redisreplication","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisReplication","source":"kind source: *v1beta2.RedisReplication"}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting EventSource","controller":"redissentinel","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisSentinel","source":"kind source: *v1beta2.RedisSentinel"}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting EventSource","controller":"redissentinel","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisSentinel","source":"kind source: *v1beta2.RedisReplication"}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting EventSource","controller":"redisreplication","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisReplication","source":"kind source: *v1.StatefulSet"}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting EventSource","controller":"redis","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"Redis","source":"kind source: *v1beta2.Redis"}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting Controller","controller":"redis","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"Redis"}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting Controller","controller":"redisreplication","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisReplication"}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting Controller","controller":"redissentinel","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisSentinel"}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting workers","controller":"redis","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"Redis","worker count":1}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting workers","controller":"redisreplication","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisReplication","worker count":1}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting workers","controller":"redissentinel","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisSentinel","worker count":1}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","msg":"Starting workers","controller":"rediscluster","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisCluster","worker count":1}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","logger":"KubeAPIWarningLogger","msg":"unknown field \"spec.storage.nodeConfVolumeClaimTemplate.metadata.creationTimestamp\""}
{"level":"info","ts":"2025-01-20T20:04:37+08:00","logger":"KubeAPIWarningLogger","msg":"unknown field \"spec.storage.volumeClaimTemplate.metadata.creationTimestamp\""}
W0120 20:04:37.628037 1360105 warnings.go:70] unknown field "spec.storage.nodeConfVolumeClaimTemplate.metadata.creationTimestamp"
W0120 20:04:37.628076 1360105 warnings.go:70] unknown field "spec.storage.volumeClaimTemplate.metadata.creationTimestamp"
W0120 20:04:48.463129 1360105 warnings.go:70] unknown field "spec.storage.nodeConfVolumeClaimTemplate.metadata.creationTimestamp"
W0120 20:04:48.463167 1360105 warnings.go:70] unknown field "spec.storage.volumeClaimTemplate.metadata.creationTimestamp"
W0120 20:05:04.867659 1360105 warnings.go:70] unknown field "spec.storage.nodeConfVolumeClaimTemplate.metadata.creationTimestamp"
W0120 20:05:04.867697 1360105 warnings.go:70] unknown field "spec.storage.volumeClaimTemplate.metadata.creationTimestamp"
{"level":"info","ts":"2025-01-20T20:05:05+08:00","msg":"Creating redis cluster by executing cluster creation commands","controller":"rediscluster","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisCluster","RedisCluster":{"name":"redis-cluster","namespace":"default"},"namespace":"default","name":"redis-cluster","reconcileID":"3023ddbc-6430-479f-ad84-239ca7ba7e60"}
{"level":"info","ts":"2025-01-20T20:05:05+08:00","msg":"Not all leader are part of the cluster...","controller":"rediscluster","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisCluster","RedisCluster":{"name":"redis-cluster","namespace":"default"},"namespace":"default","name":"redis-cluster","reconcileID":"3023ddbc-6430-479f-ad84-239ca7ba7e60","Leaders.Count":1,"Instance.Size":3}

What operating system and processor architecture are you using (kubectl version)?

[root@master redis]# kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.7", GitCommit:"07a61d861519c45ef5c89bc22dda289328f29343", GitTreeState:"clean", BuildDate:"2023-10-18T11:42:32Z", GoVersion:"go1.20.10", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.7", GitCommit:"07a61d861519c45ef5c89bc22dda289328f29343", GitTreeState:"clean", BuildDate:"2023-10-18T11:33:23Z", GoVersion:"go1.20.10", Compiler:"gc", Platform:"linux/amd64"}
[root@master redis]# 

Configuration of the reproduction problem

[root@master redis]# cat cluster.yaml 
---
apiVersion: redis.redis.opstreelabs.in/v1beta2
kind: RedisCluster
metadata:
  name: redis-cluster
spec:
  clusterSize: 3
  clusterVersion: v7
  podSecurityContext:
    runAsUser: 1000
    fsGroup: 1000
  persistenceEnabled: true
  kubernetesConfig:
    service:
      serviceType: NodePort
    image: quay.io/opstree/redis:v7.0.15
    imagePullPolicy: IfNotPresent
  storage:
    volumeClaimTemplate:
      spec:
        storageClassName: local-path
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 1Gi

kubectl apply -f cluster.yaml

What did you expect to see?
The cluster can be created normally and can be used

What did you see instead?
Creating a RedisCluster stuck in Bootstrap state

Related to #745

@trynocoding trynocoding added the bug Something isn't working label Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant