Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redis sentinel pod gets SIGTERM / crashloopbackoff #1215

Open
QuingKhaos opened this issue Jan 21, 2025 · 0 comments
Open

Redis sentinel pod gets SIGTERM / crashloopbackoff #1215

QuingKhaos opened this issue Jan 21, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@QuingKhaos
Copy link

What version of redis operator are you using?

redis-operator version: v0.19.0 / helm chart v0.19.1

What operating system and processor architecture are you using (kubectl version)?

OpenShift v4.16.8, Kubernetes v1.29.7

What did you do?

apiVersion: redis.redis.opstreelabs.in/v1beta1
kind: RedisReplication
metadata:
  name: dw-redis-sentinel-replication
spec:
  clusterSize: 3
  kubernetesConfig:
    image: 'quay.io/opstree/redis:v6.2.17'
    imagePullPolicy: IfNotPresent
    redisSecret:
      key: password
      name: dw-redis-sentinel-secret
    resources:
      limits:
        memory: 512Mi
      requests:
        memory: 512Mi
  nodeSelector:
    node-role.kubernetes.io/app: ''
  storage:
    volumeClaimTemplate:
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
        storageClassName: ocs-storagecluster-ceph-rbd
apiVersion: redis.redis.opstreelabs.in/v1beta1
kind: RedisSentinel
metadata:
  name: dw-redis-sentinel
spec:
  clusterSize: 3
  kubernetesConfig:
    image: 'quay.io/opstree/redis:v6.2.17'
    imagePullPolicy: IfNotPresent
    redisSecret:
      key: password
      name: dw-redis-sentinel-secret
    resources:
      limits:
        memory: 128Mi
      requests:
        memory: 128Mi
  nodeSelector:
    node-role.kubernetes.io/app: ''
  pdb:
    enabled: true
    maxUnavailable: 1
  priorityClassName: app-preprod
  redisSentinelConfig:
    masterGroupName: dw-redis-sentinel
    redisReplicationName: dw-redis-sentinel-replication
    redisReplicationPassword:
      secretKeyRef:
        key: password
        name: dw-redis-sentinel-secret
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: allow-from-redis-operator
spec:
  podSelector: {}
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: redis-operator
  policyTypes:
    - Ingress

there is also a allow-from-same-namespace network policy, that allows all communication within namespace on any port.

What did you expect to see?

Redis Sentinel should be up.

What did you see instead?

dw-redis-sentinel-sentinel-0 keeps restarting with the following logs:

Setting up redis in standalone mode
Running without persistence mode
Running without TLS mode
ACL_MODE is not true, skipping ACL file modification
Starting redis service in standalone mode.....
1:C 21 Jan 2025 13:16:20.124 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 21 Jan 2025 13:16:20.124 # Redis version=6.2.17, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 21 Jan 2025 13:16:20.124 # Configuration loaded
1:M 21 Jan 2025 13:16:20.125 * monotonic clock: POSIX clock_gettime
1:M 21 Jan 2025 13:16:20.125 * Running mode=standalone, port=6379.
1:M 21 Jan 2025 13:16:20.125 # Server initialized
1:M 21 Jan 2025 13:16:20.125 * Ready to accept connections
1:signal-handler (1737465405) Received SIGTERM scheduling shutdown...
1:M 21 Jan 2025 13:16:45.582 # User requested shutdown...
1:M 21 Jan 2025 13:16:45.582 * Saving the final RDB snapshot before exiting.
1:M 21 Jan 2025 13:16:45.582 * DB saved on disk
1:M 21 Jan 2025 13:16:45.582 * Removing the pid file.
1:M 21 Jan 2025 13:16:45.582 # Redis is now ready to exit, bye bye...

Operator log is just spammed with

{"level":"info","ts":"2025-01-21T13:20:54Z","msg":"Secret key found in the secret","controller":"redisreplication","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisReplication","RedisReplication":{"name":"dw-redis-sentinel-replication","namespace":"xxx-preprod"},"namespace":"xxx-preprod","name":"dw-redis-sentinel-replication","reconcileID":"f1e678cd-bb0b-4927-b6eb-856c2e0f407d","secretKey":"password"}
@QuingKhaos QuingKhaos added the bug Something isn't working label Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant