Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Changing eks.Cluster.authenticationMode should trigger a re-create #1493

Open
jkodroff opened this issue Nov 17, 2024 · 1 comment
Open

Changing eks.Cluster.authenticationMode should trigger a re-create #1493

jkodroff opened this issue Nov 17, 2024 · 1 comment
Labels
blocked The issue cannot be resolved without 3rd party action. kind/bug Some behavior is incorrect or out of spec

Comments

@jkodroff
Copy link
Member

What happened?

Changing eks.Cluster.authenticationMode from the default value of CONFIG_MAP to API throws an error. (Also note that the default value is not noted in the docs: https://www.pulumi.com/registry/packages/eks/api-docs/cluster/#authenticationmode_nodejs):

Diagnostics:
  pulumi:pulumi:Stack (k8s-better-together-eks-cluster-dev):
    (node:5547) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
    (Use `node --trace-deprecation ...` to show where the warning was created)

    error: update failed
    error: eks:index:Cluster resource 'k8s-better-together' has a problem: grpc: the client connection is closing

  aws:eks:Cluster (k8s-better-together-eksCluster):
    error: 1 error occurred:
        * updating urn:pulumi:dev::k8s-better-together-eks-cluster::eks:index:Cluster$aws:eks/cluster:Cluster::k8s-better-together-eksCluster: 1 error occurred:
        * updating EKS Cluster (k8s-better-together-eksCluster-6be490b) access configuration: operation error EKS: UpdateClusterConfig, https response error StatusCode: 400, RequestID: a7ebfa93-b66e-4d41-9214-0d6cb9020ee3, InvalidParameterException: Unsupported authentication mode update from CONFIG_MAP to API

Example

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";
import * as pcloud from "@pulumi/pulumiservice";

const vpc = new awsx.ec2.Vpc("k8s-better-together", {
  // Using a single NAT Gateway (as opposed to the default of one per AZ) helps
  // reduce cost and may improve provisioning time. In production scenarios, you
  // should typically use one per AZ.
  natGateways: {
    strategy: "Single"
  }
});

const eksCluster = new eks.Cluster("k8s-better-together", {
  authenticationMode: "API", // <-- Omit this line on the first run of the problem.
  vpcId: vpc.vpcId,
  publicSubnetIds: vpc.publicSubnetIds,
  privateSubnetIds: vpc.privateSubnetIds,
  // The CoreDNS add-on takes a while to install (about 10 minutes). Installing
  // the add-on keeps it automatically up to date and is helpful for day 2
  // operations. Since this is just a temporary cluster, we'll leave it
  // disabled.
  corednsAddonOptions: {
    enabled: false,
  },
  createOidcProvider: true
});

Output of pulumi about

CLI          
Version      3.138.0
Go Version   go1.23.3
Go Compiler  gc

Plugins
KIND      NAME           VERSION
resource  aws            6.59.1
resource  awsx           2.18.1
resource  docker         4.5.7
resource  docker         3.6.1
resource  eks            3.0.2
resource  kubernetes     4.18.3
language  nodejs         3.138.0-dev.0
resource  pulumiservice  0.27.0

Host     
OS       darwin
Version  14.6.1
Arch     arm64

This project is written in nodejs: executable='/opt/homebrew/bin/node' version='v22.5.1'

Current Stack: jkodrofftest/k8s-better-together-eks-cluster/dev

TYPE                                                 URN
pulumi:pulumi:Stack                                  urn:pulumi:dev::k8s-better-together-eks-cluster::pulumi:pulumi:Stack::k8s-better-together-eks-cluster-dev
pulumi:providers:awsx                                urn:pulumi:dev::k8s-better-together-eks-cluster::pulumi:providers:awsx::default_2_18_1
awsx:ec2:Vpc                                         urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc::k8s-better-together
pulumi:providers:aws                                 urn:pulumi:dev::k8s-better-together-eks-cluster::pulumi:providers:aws::default_6_58_0
aws:ec2/vpc:Vpc                                      urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc::k8s-better-together
aws:ec2/subnet:Subnet                                urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet::k8s-better-together-public-2
aws:ec2/subnet:Subnet                                urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet::k8s-better-together-private-3
aws:ec2/subnet:Subnet                                urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet::k8s-better-together-private-2
aws:ec2/subnet:Subnet                                urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet::k8s-better-together-public-3
aws:ec2/subnet:Subnet                                urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet::k8s-better-together-public-1
aws:ec2/subnet:Subnet                                urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet::k8s-better-together-private-1
aws:ec2/internetGateway:InternetGateway              urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/internetGateway:InternetGateway::k8s-better-together
aws:ec2/routeTable:RouteTable                        urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable::k8s-better-together-public-2
aws:ec2/routeTable:RouteTable                        urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable::k8s-better-together-private-3
aws:ec2/routeTable:RouteTable                        urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable::k8s-better-together-private-2
aws:ec2/routeTable:RouteTable                        urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable::k8s-better-together-public-3
aws:ec2/routeTable:RouteTable                        urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable::k8s-better-together-public-1
aws:ec2/eip:Eip                                      urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/eip:Eip::k8s-better-together-1
aws:ec2/routeTable:RouteTable                        urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable::k8s-better-together-private-1
aws:ec2/routeTableAssociation:RouteTableAssociation  urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable$aws:ec2/routeTableAssociation:RouteTableAssociation::k8s-better-together-public-2
aws:ec2/routeTableAssociation:RouteTableAssociation  urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable$aws:ec2/routeTableAssociation:RouteTableAssociation::k8s-better-together-public-3
aws:ec2/route:Route                                  urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable$aws:ec2/route:Route::k8s-better-together-public-2
aws:ec2/route:Route                                  urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable$aws:ec2/route:Route::k8s-better-together-public-3
aws:ec2/routeTableAssociation:RouteTableAssociation  urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable$aws:ec2/routeTableAssociation:RouteTableAssociation::k8s-better-together-private-3
aws:ec2/routeTableAssociation:RouteTableAssociation  urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable$aws:ec2/routeTableAssociation:RouteTableAssociation::k8s-better-together-private-2
aws:ec2/routeTableAssociation:RouteTableAssociation  urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable$aws:ec2/routeTableAssociation:RouteTableAssociation::k8s-better-together-public-1
aws:ec2/routeTableAssociation:RouteTableAssociation  urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable$aws:ec2/routeTableAssociation:RouteTableAssociation::k8s-better-together-private-1
aws:ec2/natGateway:NatGateway                        urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/natGateway:NatGateway::k8s-better-together-1
aws:ec2/route:Route                                  urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable$aws:ec2/route:Route::k8s-better-together-public-1
aws:ec2/route:Route                                  urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable$aws:ec2/route:Route::k8s-better-together-private-3
aws:ec2/route:Route                                  urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable$aws:ec2/route:Route::k8s-better-together-private-2
aws:ec2/route:Route                                  urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ec2:Vpc$aws:ec2/vpc:Vpc$aws:ec2/subnet:Subnet$aws:ec2/routeTable:RouteTable$aws:ec2/route:Route::k8s-better-together-private-1
pulumi:providers:pulumi                              urn:pulumi:dev::k8s-better-together-eks-cluster::pulumi:providers:pulumi::default
pulumi:providers:pulumiservice                       urn:pulumi:dev::k8s-better-together-eks-cluster::pulumi:providers:pulumiservice::default_0_27_0
pulumiservice:index:Environment                      urn:pulumi:dev::k8s-better-together-eks-cluster::pulumiservice:index:Environment::esc-environment
awsx:ecr:Repository                                  urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ecr:Repository::k8s-better-together
aws:ecr/repository:Repository                        urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ecr:Repository$aws:ecr/repository:Repository::k8s-better-together
aws:ecr/lifecyclePolicy:LifecyclePolicy              urn:pulumi:dev::k8s-better-together-eks-cluster::awsx:ecr:Repository$aws:ecr/lifecyclePolicy:LifecyclePolicy::k8s-better-together
pulumi:providers:eks                                 urn:pulumi:dev::k8s-better-together-eks-cluster::pulumi:providers:eks::default_3_0_2
eks:index:Cluster                                    urn:pulumi:dev::k8s-better-together-eks-cluster::eks:index:Cluster::k8s-better-together
eks:index:ServiceRole                                urn:pulumi:dev::k8s-better-together-eks-cluster::eks:index:Cluster$eks:index:ServiceRole::k8s-better-together-eksRole
eks:index:ServiceRole                                urn:pulumi:dev::k8s-better-together-eks-cluster::eks:index:Cluster$eks:index:ServiceRole::k8s-better-together-instanceRole
pulumi:providers:aws                                 urn:pulumi:dev::k8s-better-together-eks-cluster::pulumi:providers:aws::default_6_45_0
aws:ec2/securityGroup:SecurityGroup                  urn:pulumi:dev::k8s-better-together-eks-cluster::eks:index:Cluster$aws:ec2/securityGroup:SecurityGroup::k8s-better-together-eksClusterSecurityGroup
aws:iam/role:Role                                    urn:pulumi:dev::k8s-better-together-eks-cluster::eks:index:Cluster$eks:index:ServiceRole$aws:iam/role:Role::k8s-better-together-eksRole-role
aws:iam/role:Role                                    urn:pulumi:dev::k8s-better-together-eks-cluster::eks:index:Cluster$eks:index:ServiceRole$aws:iam/role:Role::k8s-better-together-instanceRole-role
aws:iam/rolePolicyAttachment:RolePolicyAttachment    urn:pulumi:dev::k8s-better-together-eks-cluster::eks:index:Cluster$eks:index:ServiceRole$aws:iam/rolePolicyAttachment:RolePolicyAttachment::k8s-better-together-eksRole-4b490823
aws:ec2/securityGroupRule:SecurityGroupRule          urn:pulumi:dev::k8s-better-together-eks-cluster::eks:index:Cluster$aws:ec2/securityGroupRule:SecurityGroupRule::k8s-better-together-eksClusterInternetEgressRule
aws:iam/rolePolicyAttachment:RolePolicyAttachment    urn:pulumi:dev::k8s-better-together-eks-cluster::eks:index:Cluster$eks:index:ServiceRole$aws:iam/rolePolicyAttachment:RolePolicyAttachment::k8s-better-together-instanceRole-03516f97
aws:iam/rolePolicyAttachment:RolePolicyAttachment    urn:pulumi:dev::k8s-better-together-eks-cluster::eks:index:Cluster$eks:index:ServiceRole$aws:iam/rolePolicyAttachment:RolePolicyAttachment::k8s-better-together-instanceRole-3eb088f2
aws:iam/rolePolicyAttachment:RolePolicyAttachment    urn:pulumi:dev::k8s-better-together-eks-cluster::eks:index:Cluster$eks:index:ServiceRole$aws:iam/rolePolicyAttachment:RolePolicyAttachment::k8s-better-together-instanceRole-e1b295bd
aws:eks/cluster:Cluster                              urn:pulumi:dev::k8s-better-together-eks-cluster::eks:index:Cluster$aws:eks/cluster:Cluster::k8s-better-together-eksCluster
pulumi:providers:aws                                 urn:pulumi:dev::k8s-better-together-eks-cluster::pulumi:providers:aws::default_6_59_1


Found no pending operations associated with dev

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/josh-pulumi-corp
User           josh-pulumi-corp
Organizations  josh-pulumi-corp, jkodrofftest, aws-partnership, zephyr, jkodroff-team, pulumi
Token type     personal

Dependencies:
NAME                   VERSION
@pulumi/pulumiservice  0.27.0
@types/node            18.19.64
typescript             5.6.3
@pulumi/aws            6.59.1
@pulumi/awsx           2.18.1
@pulumi/eks            3.0.2
@pulumi/pulumi         3.138.0

Pulumi locates its logs in /var/folders/5m/4n1x3f8151s35wc80w06z5k80000gn/T/ by default

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).
@jkodroff jkodroff added kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels Nov 17, 2024
@flostadler
Copy link
Contributor

flostadler commented Nov 18, 2024 •
edited
Loading

Thanks for opening this issue @jkodroff!

AWS EKS does not support migrating from CONFIG_MAP directly to API. You'll need to take an intermediate step to go to API_AND_CONFIG_MAP. This will not require a re-create.
But I agree, it would be great if the provider could detect that certain authentication mode changes require replacements. I'll open a feature request in pulumi-aws for this and link it here (opened pulumi/pulumi-aws#4789).

The reason why no default is listed on the pulumi side is that this is not set by pulumi, but rather by the EKS service. This will most likely change with newer EKS cluster versions given that the config map is already deprecated.

@flostadler flostadler added blocked The issue cannot be resolved without 3rd party action. and removed needs-triage Needs attention from the triage team labels Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
blocked The issue cannot be resolved without 3rd party action. kind/bug Some behavior is incorrect or out of spec
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jkodroff @flostadler