Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(RELEASE-1452): onboard sealights to release service #683

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jinqi7
Copy link
Contributor

@jinqi7 jinqi7 commented Feb 26, 2025

No description provided.

Copy link

openshift-ci bot commented Feb 26, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Copy link

codecov bot commented Feb 26, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.70%. Comparing base (f1cc9bd) to head (09c1ce1).
Report is 43 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #683      +/-   ##
==========================================
- Coverage   79.23%   77.70%   -1.54%     
==========================================
  Files          28       28              
  Lines        2283     2848     +565     
==========================================
+ Hits         1809     2213     +404     
- Misses        389      532     +143     
- Partials       85      103      +18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jinqi7 jinqi7 marked this pull request as ready for review February 26, 2025 09:50
@jinqi7 jinqi7 requested a review from a team as a code owner February 26, 2025 09:50
@jinqi7 jinqi7 removed the request for review from a team February 26, 2025 09:50
@jinqi7
Copy link
Contributor Author

jinqi7 commented Feb 26, 2025

/test

Copy link

openshift-ci bot commented Feb 26, 2025

@jinqi7: The /test command needs one or more targets.
The following commands are available to trigger required jobs:

/test images

The following commands are available to trigger optional jobs:

/test release-service-e2e

Use /test all to run the following jobs that were automatically triggered:

pull-ci-konflux-ci-release-service-main-images

In response to this:

/test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@jinqi7 jinqi7 requested review from a team and flacatus February 26, 2025 10:53
@jinqi7 jinqi7 marked this pull request as draft February 26, 2025 12:26
@jinqi7 jinqi7 marked this pull request as ready for review February 26, 2025 12:55
@jinqi7
Copy link
Contributor Author

jinqi7 commented Feb 26, 2025

/retest

@mmalina
Copy link
Collaborator

mmalina commented Feb 26, 2025

@jinqi7 could you add some more description? What is sealights? Would be nice to have more details without having to go to Jira.

@flacatus
Copy link

/retest

@jinqi7 jinqi7 force-pushed the RELEASE-1452 branch 2 times, most recently from 251bee2 to d7db378 Compare February 27, 2025 03:24
@jinqi7
Copy link
Contributor Author

jinqi7 commented Feb 27, 2025

/retest

tasks:
- name: sealights-refs

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for some reason I dont see this task creating in ur pipelines

Copy link
Contributor Author

@jinqi7 jinqi7 Feb 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we haven't merge this PR yet and "main" branch is being used for the pipeline in the IntegrationTestScenario CR.

@jinqi7
Copy link
Contributor Author

jinqi7 commented Feb 27, 2025

/retest

@flacatus
Copy link

/retest

@flacatus
Copy link

@johnbieren @jinqi7 Can this network policy https://github.com/konflux-ci/release-service/blob/main/config/manager/network_policy.yaml to block traffic to redhat.sealights.co?

@johnbieren
Copy link
Collaborator

@johnbieren @jinqi7 Can this network policy https://github.com/konflux-ci/release-service/blob/main/config/manager/network_policy.yaml to block traffic to redhat.sealights.co?

I have no idea, I didn't work on that. @happybhati ?

@happybhati
Copy link
Collaborator

@johnbieren @jinqi7 Can this network policy https://github.com/konflux-ci/release-service/blob/main/config/manager/network_policy.yaml to block traffic to redhat.sealights.co?

@flacatus We have Egress Network Policy to block all outgoing(Egress) traffic
from release service pod(controller-manager)

  • The "spec.podSelector" contains the name of a pod on which we want to apply network policy

  • "spec.egress" have Whitelist rules to allow the necessary traffic to certain pods and namespaces.

@happybhati
Copy link
Collaborator

If you need the agents to reach an external SeaLights server, you’d have to update the egress section to include an ipBlock or namespaceSelector that covers the external endpoint’s IP range and port. For example:

- to:
  - ipBlock:
      cidr: <SeaLights-server-IP-range>  # e.g., 203.0.113.0/24
  ports:
    - port: 443  # Or whatever port SeaLights uses
      protocol: TCP

Without such an addition, the agents are stuck inside the cluster’s restricted network boundaries.

@flacatus
Copy link

flacatus commented Feb 27, 2025

Can we disable this network policy for development? @johnbieren @happybhati

@happybhati
Copy link
Collaborator

I am not sure about disabling,
it was a prod sec requirement for the threat model of release service to have an egress network policy in place.

Screenshot 2025-02-27 at 1 48 29 PM

@jinqi7 jinqi7 force-pushed the RELEASE-1452 branch 4 times, most recently from c75b1eb to 47928d7 Compare February 28, 2025 07:20
@jinqi7
Copy link
Contributor Author

jinqi7 commented Feb 28, 2025

I did several tests with adding below IP range and port ( 6443/443 and 443/80). It didn't work. But it really works with disabling net work policy.

- to:
     - ipBlock:
         cidr: 159.60.128.0/24  # CIDR of Sealight server Endpoint.
     ports:
       - port: 6443
         protocol: TCP
       - port: 443
         protocol: TCP

@jinqi7
Copy link
Contributor Author

jinqi7 commented Feb 28, 2025

/retest

@jinqi7 jinqi7 force-pushed the RELEASE-1452 branch 7 times, most recently from f26e57d to ba04a25 Compare March 3, 2025 09:40
Signed-off-by: Jing Qi <[email protected]>

Added Sealights instrumentation task to the build process to enable tracking
 and monitoring of test coverage during builds. Adjusted build pipelines to
 create a new container image that includes the Sealights agent, making it
ready for instrumentation. Enabled e2e tests to sync with Sealights by
sending test reports for coverage tracking.Future improvements will include
test selection, optimizing test run times, and reducing cloud costs
@konflux-ci-qe-bot
Copy link

@jinqi7: The following test has Failed, say /retest to rerun failed tests.

PipelineRun Name Status Rerun command Build Log Test Log
konflux-e2e-tests-4mq8v Failed /retest View Pipeline Log View Test Logs

Inspecting Test Artifacts

To inspect your test artifacts, follow these steps:

  1. Install ORAS (see the ORAS installation guide).
  2. Download artifacts with the following commands:
mkdir -p oras-artifacts
cd oras-artifacts
oras pull quay.io/konflux-test-storage/konflux-team/release-service:konflux-e2e-tests-4mq8v

Test results analysis

🚨 Failed to provision a cluster, see the log for more details:

Click to view logs
INFO: Log in to your Red Hat account...
INFO: Configure AWS Credentials...
INFO: Logged in as 'konflux-ci-418295695583' on 'https://api.openshift.com'
INFO: Create ROSA with HCP cluster...
time=2025-03-03T09:48:41Z level=info msg=Ignored check for policy key 'sts_hcp_ec2_registry_permission_policy' (zero egress feature toggle is not enabled)
INFO: Creating cluster 'kx-a67114116a'
INFO: To view a list of clusters and their status, run 'rosa list clusters'
INFO: Cluster 'kx-a67114116a' has been created.
INFO: Once the cluster is installed you will need to add an Identity Provider before you can login into the cluster. See 'rosa create idp --help' for more information.

Name: kx-a67114116a
Domain Prefix: kx-a67114116a
Display Name: kx-a67114116a
ID: 2h9ipvrujv4sb7vm9qv1t4qtsk9hchmf
External ID: e2c01c01-d8ea-4c65-8a6d-b9950e046690
Control Plane: ROSA Service Hosted
OpenShift Version: 4.15.45
Channel Group: stable
DNS: Not ready
AWS Account: 418295695583
AWS Billing Account: 418295695583
API URL:
Console URL:
Region: us-east-1
Availability:

  • Control Plane: MultiAZ
  • Data Plane: SingleAZ

Nodes:

  • Compute (desired): 3
  • Compute (current): 0
    Network:
  • Type: OVNKubernetes
  • Service CIDR: 172.30.0.0/16
  • Machine CIDR: 10.0.0.0/16
  • Pod CIDR: 10.128.0.0/14
  • Host Prefix: /23
  • Subnets: subnet-05b9daa0609597f68, subnet-04cf6376374bf9e09
    EC2 Metadata Http Tokens: optional
    Role (STS) ARN: arn:aws:iam::418295695583:role/ManagedOpenShift-HCP-ROSA-Installer-Role
    Support Role ARN: arn:aws:iam::418295695583:role/ManagedOpenShift-HCP-ROSA-Support-Role
    Instance IAM Roles:
  • Worker: arn:aws:iam::418295695583:role/ManagedOpenShift-HCP-ROSA-Worker-Role
    Operator IAM Roles:
  • arn:aws:iam::418295695583:role/rosa-hcp-openshift-image-registry-installer-cloud-credentials
  • arn:aws:iam::418295695583:role/rosa-hcp-openshift-ingress-operator-cloud-credentials
  • arn:aws:iam::418295695583:role/rosa-hcp-openshift-cluster-csi-drivers-ebs-cloud-credentials
  • arn:aws:iam::418295695583:role/rosa-hcp-openshift-cloud-network-config-controller-cloud-credent
  • arn:aws:iam::418295695583:role/rosa-hcp-kube-system-capa-controller-manager
  • arn:aws:iam::418295695583:role/rosa-hcp-kube-system-control-plane-operator
  • arn:aws:iam::418295695583:role/rosa-hcp-kube-system-kms-provider
  • arn:aws:iam::418295695583:role/rosa-hcp-kube-system-kube-controller-manager
    Managed Policies: Yes
    State: waiting (Waiting for user action)
    Private: No
    Delete Protection: Disabled
    Created: Mar 3 2025 09:48:52 UTC
    User Workload Monitoring: Enabled
    Details Page: https://console.redhat.com/openshift/details/s/2tnkfuOmCqmfV5lVGb1dLWceDPq
    OIDC Endpoint URL: https://oidc.op1.openshiftapps.com/2du11g36ejmoo4624pofphlrgf4r9tf3 (Managed)
    Etcd Encryption: Disabled
    Audit Log Forwarding: Disabled
    External Authentication: Disabled
    Zero Egress: Disabled

INFO: Preparing to create operator roles.
INFO: Operator Roles already exists
INFO: Preparing to create OIDC Provider.
INFO: OIDC provider already exists
INFO: To determine when your cluster is Ready, run 'rosa describe cluster -c kx-a67114116a'.
INFO: To watch your cluster installation logs, run 'rosa logs install -c kx-a67114116a --watch'.
INFO: Track the progress of the cluster creation...
�[0;33mW:�[m Region flag will be removed from this command in future versions
INFO: Cluster 'kx-a67114116a' is in waiting state waiting for installation to begin. Logs will show up within 5 minutes
0001-01-01 00:00:00 +0000 UTC hostedclusters kx-a67114116a Version
2025-03-03 09:53:31 +0000 UTC hostedclusters kx-a67114116a ValidAWSIdentityProvider StatusUnknown
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a The hosted control plane is not found
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a The hosted control plane is not found
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a The hosted control plane is not found
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a The hosted control plane is not found
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a The hosted control plane is not found
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Waiting for hosted control plane to be healthy
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a The hosted control plane is not found
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Ignition server deployment not found
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Configuration passes validation
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a HostedCluster is supported by operator configuration
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Release image is valid
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a The hosted control plane is not found
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Reconciliation active on resource
2025-03-03 09:53:33 +0000 UTC hostedclusters kx-a67114116a HostedCluster is at expected version
2025-03-03 09:53:43 +0000 UTC hostedclusters kx-a67114116a Required platform credentials are found
2025-03-03 09:53:43 +0000 UTC hostedclusters kx-a67114116a failed to get referenced secret ocm-production-2h9ipvrujv4sb7vm9qv1t4qtsk9hchmf/cluster-api-cert: Secret "cluster-api-cert" not found
0001-01-01 00:00:00 +0000 UTC hostedclusters kx-a67114116a Version
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Release image is valid
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Ignition server deployment not found
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Waiting for hosted control plane kubeconfig to be created
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a HostedCluster is supported by operator configuration
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Reconciliation active on resource
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Configuration passes validation
2025-03-03 09:53:33 +0000 UTC hostedclusters kx-a67114116a HostedCluster is at expected version
2025-03-03 09:53:43 +0000 UTC hostedclusters kx-a67114116a Required platform credentials are found
2025-03-03 09:55:14 +0000 UTC hostedclusters kx-a67114116a OIDC configuration is valid
2025-03-03 09:55:14 +0000 UTC hostedclusters kx-a67114116a Reconciliation completed successfully
2025-03-03 09:55:15 +0000 UTC hostedclusters kx-a67114116a WebIdentityErr
2025-03-03 09:55:16 +0000 UTC hostedclusters kx-a67114116a All is well
2025-03-03 09:55:16 +0000 UTC hostedclusters kx-a67114116a lookup api.kx-a67114116a.2ctv.p3.openshiftapps.com on 172.30.0.10:53: no such host
2025-03-03 09:55:16 +0000 UTC hostedclusters kx-a67114116a capi-provider deployment has 1 unavailable replicas
2025-03-03 09:55:16 +0000 UTC hostedclusters kx-a67114116a Configuration passes validation
2025-03-03 09:55:16 +0000 UTC hostedclusters kx-a67114116a AWS KMS is not configured
2025-03-03 09:55:16 +0000 UTC hostedclusters kx-a67114116a EtcdAvailable StatefulSetNotFound
2025-03-03 09:55:16 +0000 UTC hostedclusters kx-a67114116a Kube APIServer deployment not found
2025-03-03 09:55:25 +0000 UTC hostedclusters kx-a67114116a All is well
2025-03-03 09:56:04 +0000 UTC hostedclusters kx-a67114116a EtcdAvailable QuorumAvailable
2025-03-03 09:57:08 +0000 UTC hostedclusters kx-a67114116a Kube APIServer deployment is available
2025-03-03 09:57:15 +0000 UTC hostedclusters kx-a67114116a All is well
2025-03-03 09:57:29 +0000 UTC hostedclusters kx-a67114116a The hosted cluster is not degraded
0001-01-01 00:00:00 +0000 UTC hostedclusters kx-a67114116a Version
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Release image is valid
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Configuration passes validation
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Get "https://a9aede7805f10446a82d5de8375e0c61-2bb8f6f51101692b.elb.us-east-1.amazonaws.com:443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Ignition server deployment is not yet available
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Reconciliation active on resource
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a HostedCluster is supported by operator configuration
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:32 +0000 UTC hostedclusters kx-a67114116a Condition not found in the CVO.
2025-03-03 09:53:33 +0000 UTC hostedclusters kx-a67114116a HostedCluster is at expected version
2025-03-03 09:53:43 +0000 UTC hostedclusters kx-a67114116a Required platform credentials are found
2025-03-03 09:55:14 +0000 UTC hostedclusters kx-a67114116a Reconciliation completed successfully
2025-03-03 09:55:14 +0000 UTC hostedclusters kx-a67114116a OIDC configuration is valid
2025-03-03 09:55:16 +0000 UTC hostedclusters kx-a67114116a AWS KMS is not configured
2025-03-03 09:55:16 +0000 UTC hostedclusters kx-a67114116a All is well
2025-03-03 09:55:16 +0000 UTC hostedclusters kx-a67114116a lookup api.kx-a67114116a.2ctv.p3.openshiftapps.com on 172.30.0.10:53: no such host
2025-03-03 09:55:16 +0000 UTC hostedclusters kx-a67114116a Configuration passes validation
2025-03-03 09:55:25 +0000 UTC hostedclusters kx-a67114116a All is well
2025-03-03 09:56:04 +0000 UTC hostedclusters kx-a67114116a EtcdAvailable QuorumAvailable
2025-03-03 09:57:08 +0000 UTC hostedclusters kx-a67114116a Kube APIServer deployment is available
2025-03-03 09:57:15 +0000 UTC hostedclusters kx-a67114116a All is well
2025-03-03 09:57:30 +0000 UTC hostedclusters kx-a67114116a [catalog-operator deployment has 1 unavailable replicas, certified-operators-catalog deployment has 2 unavailable replicas, cloud-credential-operator deployment has 1 unavailable replicas, cluster-network-operator deployment has 1 unavailable replicas, cluster-storage-operator deployment has 1 unavailable replicas, community-operators-catalog deployment has 2 unavailable replicas, csi-snapshot-controller-operator deployment has 1 unavailable replicas, dns-operator deployment has 1 unavailable replicas, hosted-cluster-config-operator deployment has 1 unavailable replicas, ignition-server deployment has 3 unavailable replicas, ingress-operator deployment has 1 unavailable replicas, oauth-openshift deployment has 2 unavailable replicas, olm-operator deployment has 1 unavailable replicas, packageserver deployment has 3 unavailable replicas, redhat-marketplace-catalog deployment has 2 unavailable replicas, redhat-operators-catalog deployment has 2 unavailable replicas, router deployment has 1 unavailable replicas]
2025-03-03 09:57:46 +0000 UTC hostedclusters kx-a67114116a All is well
2025-03-03 09:58:22 +0000 UTC hostedclusters kx-a67114116a Unable to apply 4.15.45: an unknown error has occurred: MultipleErrors
2025-03-03 09:58:22 +0000 UTC hostedclusters kx-a67114116a ClusterVersionAvailable FromClusterVersion
2025-03-03 09:58:22 +0000 UTC hostedclusters kx-a67114116a Payload loaded version="4.15.45" image="quay.io/openshift-release-dev/ocp-release@sha256:e645f0b6c90d201d79aa4b768f3a36dc38457329b858ca7054de721cb357ea0b" architecture="Multi"
2025-03-03 09:58:22 +0000 UTC hostedclusters kx-a67114116a Kubernetes 1.29 and therefore OpenShift 4.16 remove several APIs which require admin consideration. Please see the knowledge article https://access.redhat.com/articles/7031404 for details and instructions.
2025-03-03 09:58:25 +0000 UTC hostedclusters kx-a67114116a The hosted control plane is available
INFO: Cluster 'kx-a67114116a' is now ready
INFO: ROSA with HCP cluster is ready, create a cluster admin account for accessing the cluster
INFO: Storing login command...
INFO: Check if it's able to login to OCP cluster...
Retried 1 times...
INFO: Check if apiserver is ready...
Waiting for cluster operators to be accessible for 2m...
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console
csi-snapshot-controller 4.15.45 True False False 2m47s
dns 4.15.45 False False True 2m54s DNS "default" is unavailable.
image-registry False True True 2m28s Available: The deployment does not have available replicas...
ingress False True True 2m27s The "default" ingress controller reports Available=False: IngressControllerUnavailable: One or more status conditions indicate unavailable: DeploymentAvailable=False (DeploymentUnavailable: The deployment has Available status condition set to False (reason: MinimumReplicasUnavailable) with message: Deployment does not have minimum availability.)
insights
kube-apiserver 4.15.45 True False False 2m50s
kube-controller-manager 4.15.45 True False False 2m50s
kube-scheduler 4.15.45 True False False 2m50s
kube-storage-version-migrator
monitoring
network 4.15.45 True True False 2m22s Deployment "/openshift-network-diagnostics/network-check-source" is not available (awaiting 1 nodes)
node-tuning False True False 2m58s DaemonSet "tuned" has no available Pod(s)
openshift-apiserver 4.15.45 True False False 2m50s
openshift-controller-manager 4.15.45 True False False 2m50s
openshift-samples
operator-lifecycle-manager 4.15.45 True False False 2m55s
operator-lifecycle-manager-catalog 4.15.45 True False False 2m56s
operator-lifecycle-manager-packageserver 4.15.45 True False False 2m50s
service-ca
storage 4.15.45 False False False 2m47s AWSEBSCSIDriverOperatorCRAvailable: AWSEBSDriverNodeServiceControllerAvailable: Waiting for the DaemonSet to deploy the CSI Node Service
cluster operators to be accessible finished!
[INFO] Cluster operators are accessible.
Waiting for cluster to be reported as healthy for 60m...
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console
csi-snapshot-controller 4.15.45 True False False 2m48s
dns 4.15.45 False False True 2m55s DNS "default" is unavailable.
image-registry False True True 2m29s Available: The deployment does not have available replicas...
ingress False True True 2m28s The "default" ingress controller reports Available=False: IngressControllerUnavailable: One or more status conditions indicate unavailable: DeploymentAvailable=False (DeploymentUnavailable: The deployment has Available status condition set to False (reason: MinimumReplicasUnavailable) with message: Deployment does not have minimum availability.)
insights
kube-apiserver 4.15.45 True False False 2m51s
kube-controller-manager 4.15.45 True False False 2m51s
kube-scheduler 4.15.45 True False False 2m51s
kube-storage-version-migrator
monitoring
network 4.15.45 True True False 2m23s Deployment "/openshift-network-diagnostics/network-check-source" is not available (awaiting 1 nodes)
node-tuning False True False 2m59s DaemonSet "tuned" has no available Pod(s)
openshift-apiserver 4.15.45 True False False 2m51s
openshift-controller-manager 4.15.45 True False False 2m51s
openshift-samples
operator-lifecycle-manager 4.15.45 True False False 2m56s
operator-lifecycle-manager-catalog 4.15.45 True False False 2m57s
operator-lifecycle-manager-packageserver 4.15.45 True False False 2m51s
service-ca
storage 4.15.45 False False False 2m48s AWSEBSCSIDriverOperatorCRAvailable: AWSEBSDriverNodeServiceControllerAvailable: Waiting for the DaemonSet to deploy the CSI Node Service
Waiting for cluster to be reported as healthy... Trying again in 60s
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console
csi-snapshot-controller 4.15.45 True False False 3m48s
dns 4.15.45 False False True 3m55s DNS "default" is unavailable.
image-registry False True True 3m29s Available: The deployment does not have available replicas...
ingress False True True 3m28s The "default" ingress controller reports Available=False: IngressControllerUnavailable: One or more status conditions indicate unavailable: DeploymentAvailable=False (DeploymentUnavailable: The deployment has Available status condition set to False (reason: MinimumReplicasUnavailable) with message: Deployment does not have minimum availability.)
insights
kube-apiserver 4.15.45 True False False 3m51s
kube-controller-manager 4.15.45 True False False 3m51s
kube-scheduler 4.15.45 True False False 3m51s
kube-storage-version-migrator
monitoring
network 4.15.45 True True False 3m23s Deployment "/openshift-network-diagnostics/network-check-source" is not available (awaiting 1 nodes)
node-tuning False True False 3m59s DaemonSet "tuned" has no available Pod(s)
openshift-apiserver 4.15.45 True False False 3m51s
openshift-controller-manager 4.15.45 True False False 3m51s
openshift-samples
operator-lifecycle-manager 4.15.45 True False False 3m56s
operator-lifecycle-manager-catalog 4.15.45 True False False 3m57s
operator-lifecycle-manager-packageserver 4.15.45 True False False 3m51s
service-ca
storage 4.15.45 False False False 3m48s AWSEBSCSIDriverOperatorCRAvailable: AWSEBSDriverNodeServiceControllerAvailable: Waiting for the DaemonSet to deploy the CSI Node Service
Waiting for cluster to be reported as healthy... Trying again in 60s
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console
csi-snapshot-controller 4.15.45 True False False 4m48s
dns 4.15.45 False False True 4m55s DNS "default" is unavailable.
image-registry False True True 4m29s Available: The deployment does not have available replicas...
ingress False True True 4m28s The "default" ingress controller reports Available=False: IngressControllerUnavailable: One or more status conditions indicate unavailable: DeploymentAvailable=False (DeploymentUnavailable: The deployment has Available status condition set to False (reason: MinimumReplicasUnavailable) with message: Deployment does not have minimum availability.)
insights
kube-apiserver 4.15.45 True False False 4m51s
kube-controller-manager 4.15.45 True False False 4m51s
kube-scheduler 4.15.45 True False False 4m51s
kube-storage-version-migrator
monitoring
network 4.15.45 True True False 4m23s Deployment "/openshift-network-diagnostics/network-check-source" is not available (awaiting 1 nodes)
node-tuning False True False 4m59s DaemonSet "tuned" has no available Pod(s)
openshift-apiserver 4.15.45 True False False 4m51s
openshift-controller-manager 4.15.45 True False False 4m51s
openshift-samples
operator-lifecycle-manager 4.15.45 True False False 4m56s
operator-lifecycle-manager-catalog 4.15.45 True False False 4m57s
operator-lifecycle-manager-packageserver 4.15.45 True False False 4m51s
service-ca
storage 4.15.45 False False False 4m48s AWSEBSCSIDriverOperatorCRAvailable: AWSEBSDriverNodeServiceControllerAvailable: Waiting for the DaemonSet to deploy the CSI Node Service
Waiting for cluster to be reported as healthy... Trying again in 60s
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console
csi-snapshot-controller 4.15.45 True False False 5m49s
dns 4.15.45 False False True 5m56s DNS "default" is unavailable.
image-registry False True True 5m30s Available: The deployment does not have available replicas...
ingress False True True 5m29s The "default" ingress controller reports Available=False: IngressControllerUnavailable: One or more status conditions indicate unavailable: DeploymentAvailable=False (DeploymentUnavailable: The deployment has Available status condition set to False (reason: MinimumReplicasUnavailable) with message: Deployment does not have minimum availability.)
insights
kube-apiserver 4.15.45 True False False 5m52s
kube-controller-manager 4.15.45 True False False 5m52s
kube-scheduler 4.15.45 True False False 5m52s
kube-storage-version-migrator
monitoring
network 4.15.45 True True False 5m24s Deployment "/openshift-network-diagnostics/network-check-source" is not available (awaiting 1 nodes)
node-tuning False True False 6m DaemonSet "tuned" has no available Pod(s)
openshift-apiserver 4.15.45 True False False 5m52s
openshift-controller-manager 4.15.45 True False False 5m52s
openshift-samples
operator-lifecycle-manager 4.15.45 True False False 5m57s
operator-lifecycle-manager-catalog 4.15.45 True False False 5m58s
operator-lifecycle-manager-packageserver 4.15.45 True False False 5m52s
service-ca
storage 4.15.45 False False False 5m49s AWSEBSCSIDriverOperatorCRAvailable: AWSEBSDriverNodeServiceControllerAvailable: Waiting for the DaemonSet to deploy the CSI Node Service
Waiting for cluster to be reported as healthy... Trying again in 60s
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console
csi-snapshot-controller 4.15.45 True False False 6m49s
dns 4.15.45 False False True 6m56s DNS "default" is unavailable.
image-registry False True True 6m30s Available: The deployment does not have available replicas...
ingress False True True 6m29s The "default" ingress controller reports Available=False: IngressControllerUnavailable: One or more status conditions indicate unavailable: DeploymentAvailable=False (DeploymentUnavailable: The deployment has Available status condition set to False (reason: MinimumReplicasUnavailable) with message: Deployment does not have minimum availability.)
insights
kube-apiserver 4.15.45 True False False 6m52s
kube-controller-manager 4.15.45 True False False 6m52s
kube-scheduler 4.15.45 True False False 6m52s
kube-storage-version-migrator
monitoring
network 4.15.45 True True False 6m24s Deployment "/openshift-network-diagnostics/network-check-source" is not available (awaiting 1 nodes)
node-tuning False True False 7m DaemonSet "tuned" has no available Pod(s)
openshift-apiserver 4.15.45 True False False 6m52s
openshift-controller-manager 4.15.45 True False False 6m52s
openshift-samples
operator-lifecycle-manager 4.15.45 True False False 6m57s
operator-lifecycle-manager-catalog 4.15.45 True False False 6m58s
operator-lifecycle-manager-packageserver 4.15.45 True False False 6m52s
service-ca
storage 4.15.45 False False False 6m49s AWSEBSCSIDriverOperatorCRAvailable: AWSEBSDriverNodeServiceControllerAvailable: Waiting for the DaemonSet to deploy the CSI Node Service
Waiting for cluster to be reported as healthy... Trying again in 60s
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console
csi-snapshot-controller 4.15.45 True False False 7m49s
dns 4.15.45 False True True 7m56s DNS "default" is unavailable.
image-registry False True True 7m30s Available: The deployment does not have available replicas...
ingress False True True 7m29s The "default" ingress controller reports Available=False: IngressControllerUnavailable: One or more status conditions indicate unavailable: DeploymentAvailable=False (DeploymentUnavailable: The deployment has Available status condition set to False (reason: MinimumReplicasUnavailable) with message: Deployment does not have minimum availability.)
insights
kube-apiserver 4.15.45 True False False 7m52s
kube-controller-manager 4.15.45 True False False 7m52s
kube-scheduler 4.15.45 True False False 7m52s
kube-storage-version-migrator
monitoring
network 4.15.45 True True False 7m24s DaemonSet "/openshift-multus/multus-additional-cni-plugins" is not available (awaiting 2 nodes)...
node-tuning 4.15.45 True True False 2s Waiting for 2/2 Profiles to be applied
openshift-apiserver 4.15.45 True False False 7m52s
openshift-controller-manager 4.15.45 True False False 7m52s
openshift-samples
operator-lifecycle-manager 4.15.45 True False False 7m57s
operator-lifecycle-manager-catalog 4.15.45 True False False 7m58s
operator-lifecycle-manager-packageserver 4.15.45 True False False 7m52s
service-ca
storage 4.15.45 False True False 7m49s AWSEBSCSIDriverOperatorCRAvailable: AWSEBSDriverNodeServiceControllerAvailable: Waiting for the DaemonSet to deploy the CSI Node Service
Waiting for cluster to be reported as healthy... Trying again in 60s
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console
csi-snapshot-controller 4.15.45 True False False 8m49s
dns 4.15.45 False False True 8m56s DNS "default" is unavailable.
image-registry False True True 8m30s Available: The deployment does not have available replicas...
ingress False True True 8m29s The "default" ingress controller reports Available=False: IngressControllerUnavailable: One or more status conditions indicate unavailable: DeploymentAvailable=False (DeploymentUnavailable: The deployment has Available status condition set to False (reason: MinimumReplicasUnavailable) with message: Deployment does not have minimum availability.)
insights 4.15.45 True False False 4s
kube-apiserver 4.15.45 True False False 8m52s
kube-controller-manager 4.15.45 True False False 8m52s
kube-scheduler 4.15.45 True False False 8m52s
kube-storage-version-migrator 4.15.45 False True False 2s KubeStorageVersionMigratorAvailable: Waiting for Deployment
monitoring
network 4.15.45 True True False 8m24s DaemonSet "/openshift-network-diagnostics/network-check-target" is not available (awaiting 2 nodes)...
node-tuning 4.15.45 True False False 62s
openshift-apiserver 4.15.45 True False False 8m52s
openshift-controller-manager 4.15.45 True False False 8m52s
openshift-samples
operator-lifecycle-manager 4.15.45 True False False 8m57s
operator-lifecycle-manager-catalog 4.15.45 True False False 8m58s
operator-lifecycle-manager-packageserver 4.15.45 True False False 8m52s
service-ca True True False 0s Progressing: ...
storage 4.15.45 True False False 49s
Waiting for cluster to be reported as healthy... Trying again in 60s
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console 4.15.45 False True False 22s DeploymentAvailable: 0 replicas available for console deployment
csi-snapshot-controller 4.15.45 True False False 9m50s
dns 4.15.45 False True True 9m57s DNS "default" is unavailable.
image-registry False True True 9m31s Available: The deployment does not have available replicas...
ingress False True True 9m30s The "default" ingress controller reports Available=False: IngressControllerUnavailable: One or more status conditions indicate unavailable: DeploymentAvailable=False (DeploymentUnavailable: The deployment has Available status condition set to False (reason: MinimumReplicasUnavailable) with message: Deployment does not have minimum availability.)
insights 4.15.45 True False False 65s
kube-apiserver 4.15.45 True False False 9m53s
kube-controller-manager 4.15.45 True False False 9m53s
kube-scheduler 4.15.45 True False False 9m53s
kube-storage-version-migrator 4.15.45 True False False 42s
monitoring Unknown True Unknown 33s Rolling out the stack.
network 4.15.45 True True False 9m25s DaemonSet "/openshift-multus/network-metrics-daemon" is not available (awaiting 2 nodes)
node-tuning 4.15.45 True False False 2m3s
openshift-apiserver 4.15.45 True False False 9m53s
openshift-controller-manager 4.15.45 True False False 9m53s
openshift-samples
operator-lifecycle-manager 4.15.45 True False False 9m58s
operator-lifecycle-manager-catalog 4.15.45 True False False 9m59s
operator-lifecycle-manager-packageserver 4.15.45 True False False 9m53s
service-ca 4.15.45 True False False 61s
storage 4.15.45 True False False 110s
Waiting for cluster to be reported as healthy... Trying again in 60s
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console 4.15.45 True False False 48s
csi-snapshot-controller 4.15.45 True False False 10m
dns 4.15.45 True False False 59s
image-registry 4.15.45 True True False 10s Progressing: The deployment has not completed...
ingress 4.15.45 True False False 58s
insights 4.15.45 True False False 2m5s
kube-apiserver 4.15.45 True False False 10m
kube-controller-manager 4.15.45 True False False 10m
kube-scheduler 4.15.45 True False False 10m
kube-storage-version-migrator 4.15.45 True False False 102s
monitoring Unknown True Unknown 93s Rolling out the stack.
network 4.15.45 True False False 10m
node-tuning 4.15.45 True False False 3m3s
openshift-apiserver 4.15.45 True False False 10m
openshift-controller-manager 4.15.45 True False False 10m
openshift-samples 4.15.45 True False False 37s
operator-lifecycle-manager 4.15.45 True False False 10m
operator-lifecycle-manager-catalog 4.15.45 True False False 10m
operator-lifecycle-manager-packageserver 4.15.45 True False False 10m
service-ca 4.15.45 True False False 2m1s
storage 4.15.45 True False False 2m50s
Waiting for cluster to be reported as healthy... Trying again in 60s
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console 4.15.45 True False False 108s
csi-snapshot-controller 4.15.45 True False False 11m
dns 4.15.45 True False False 119s
image-registry 4.15.45 True False False 70s
ingress 4.15.45 True False False 118s
insights 4.15.45 True False False 3m5s
kube-apiserver 4.15.45 True False False 11m
kube-controller-manager 4.15.45 True False False 11m
kube-scheduler 4.15.45 True False False 11m
kube-storage-version-migrator 4.15.45 True False False 2m42s
monitoring Unknown True Unknown 2m33s Rolling out the stack.
network 4.15.45 True False False 11m
node-tuning 4.15.45 True False False 4m3s
openshift-apiserver 4.15.45 True False False 11m
openshift-controller-manager 4.15.45 True False False 11m
openshift-samples 4.15.45 True False False 97s
operator-lifecycle-manager 4.15.45 True False False 11m
operator-lifecycle-manager-catalog 4.15.45 True False False 11m
operator-lifecycle-manager-packageserver 4.15.45 True False False 11m
service-ca 4.15.45 True False False 3m1s
storage 4.15.45 True False False 3m50s
Waiting for cluster to be reported as healthy... Trying again in 60s
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console 4.15.45 True False False 2m49s
csi-snapshot-controller 4.15.45 True False False 12m
dns 4.15.45 True True False 3m DNS "default" reports Progressing=True: "Have 2 available node-resolver pods, want 3."
image-registry 4.15.45 True True False 2m11s Progressing: The registry is ready...
ingress 4.15.45 True False False 2m59s
insights 4.15.45 True False False 4m6s
kube-apiserver 4.15.45 True False False 12m
kube-controller-manager 4.15.45 True False False 12m
kube-scheduler 4.15.45 True False False 12m
kube-storage-version-migrator 4.15.45 True False False 3m43s
monitoring Unknown True Unknown 3m34s Rolling out the stack.
network 4.15.45 True True False 12m DaemonSet "/openshift-multus/multus" is not available (awaiting 1 nodes)...
node-tuning 4.15.45 True True False 3s Waiting for 1/3 Profiles to be applied
openshift-apiserver 4.15.45 True False False 12m
openshift-controller-manager 4.15.45 True False False 12m
openshift-samples 4.15.45 True False False 2m38s
operator-lifecycle-manager 4.15.45 True False False 12m
operator-lifecycle-manager-catalog 4.15.45 True False False 13m
operator-lifecycle-manager-packageserver 4.15.45 True False False 12m
service-ca 4.15.45 True False False 4m2s
storage 4.15.45 True True False 4m51s AWSEBSCSIDriverOperatorCRProgressing: AWSEBSDriverNodeServiceControllerProgressing: Waiting for DaemonSet to deploy node pods
Waiting for cluster to be reported as healthy... Trying again in 60s
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console 4.15.45 True True False 3m49s SyncLoopRefreshProgressing: Changes made during sync updates, additional sync expected.
csi-snapshot-controller 4.15.45 True False False 13m
dns 4.15.45 True False False 4m
image-registry 4.15.45 True False False 3m11s
ingress 4.15.45 True False False 3m59s
insights 4.15.45 True False False 5m6s
kube-apiserver 4.15.45 True False False 13m
kube-controller-manager 4.15.45 True False False 13m
kube-scheduler 4.15.45 True False False 13m
kube-storage-version-migrator 4.15.45 True False False 4m43s
monitoring 4.15.45 True False False 9s
network 4.15.45 True True False 13m DaemonSet "/openshift-network-diagnostics/network-check-target" is not available (awaiting 1 nodes)...
node-tuning 4.15.45 True True False 63s Waiting for 1/3 Profiles to be applied
openshift-apiserver 4.15.45 True False False 13m
openshift-controller-manager 4.15.45 True False False 13m
openshift-samples 4.15.45 True False False 3m38s
operator-lifecycle-manager 4.15.45 True False False 13m
operator-lifecycle-manager-catalog 4.15.45 True False False 14m
operator-lifecycle-manager-packageserver 4.15.45 True False False 13m
service-ca 4.15.45 True False False 5m2s
storage 4.15.45 True False False 5m51s
Waiting for cluster to be reported as healthy... Trying again in 60s
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console 4.15.45 True False False 4m49s
csi-snapshot-controller 4.15.45 True False False 14m
dns 4.15.45 True False False 5m
image-registry 4.15.45 True False False 4m11s
ingress 4.15.45 True False False 4m59s
insights 4.15.45 True False False 6m6s
kube-apiserver 4.15.45 True False False 14m
kube-controller-manager 4.15.45 True False False 14m
kube-scheduler 4.15.45 True False False 14m
kube-storage-version-migrator 4.15.45 True False False 5m43s
monitoring 4.15.45 True False False 69s
network 4.15.45 True True False 14m DaemonSet "/openshift-network-diagnostics/network-check-target" is not available (awaiting 1 nodes)
node-tuning 4.15.45 True True False 2m3s Waiting for 1/3 Profiles to be applied
openshift-apiserver 4.15.45 True False False 14m
openshift-controller-manager 4.15.45 True False False 14m
openshift-samples 4.15.45 True False False 4m38s
operator-lifecycle-manager 4.15.45 True False False 14m
operator-lifecycle-manager-catalog 4.15.45 True False False 15m
operator-lifecycle-manager-packageserver 4.15.45 True False False 14m
service-ca 4.15.45 True False False 6m2s
storage 4.15.45 True False False 6m51s
Waiting for cluster to be reported as healthy... Trying again in 60s
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
console 4.15.45 True False False 5m49s
csi-snapshot-controller 4.15.45 True False False 15m
dns 4.15.45 True False False 6m
image-registry 4.15.45 True False False 5m11s
ingress 4.15.45 True False False 5m59s
insights 4.15.45 True False False 7m6s
kube-apiserver 4.15.45 True False False 15m
kube-controller-manager 4.15.45 True False False 15m
kube-scheduler 4.15.45 True False False 15m
kube-storage-version-migrator 4.15.45 True False False 6m43s
monitoring 4.15.45 True False False 2m9s
network 4.15.45 True False False 15m
node-tuning 4.15.45 True True False 3m3s Waiting for 1/3 Profiles to be applied
openshift-apiserver 4.15.45 True False False 15m
openshift-controller-manager 4.15.45 True False False 15m
openshift-samples 4.15.45 True False False 5m38s
operator-lifecycle-manager 4.15.45 True False False 15m
operator-lifecycle-manager-catalog 4.15.45 True False False 16m
operator-lifecycle-manager-packageserver 4.15.45 True False False 15m
service-ca 4.15.45 True False False 7m2s
storage 4.15.45 True False False 7m51s
Waiting for cluster to be reported as healthy... Trying again in 60s
healthy
cluster to be reported as healthy finished!


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants