Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue# 306: Added tagging controller #308

Merged
merged 42 commits into from
Mar 30, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
aeaaf45
Add tagging controller configuration
nguyenkndinh Feb 23, 2022
6c7fbb1
add log
nguyenkndinh Feb 23, 2022
4e40c99
rearrange the controllers
nguyenkndinh Feb 23, 2022
793403b
remove debugging log
nguyenkndinh Feb 24, 2022
92d5743
removed route controller
nguyenkndinh Feb 24, 2022
1e8be5a
added a blank test file for the tagging controller
nguyenkndinh Feb 24, 2022
014b607
remove predefined tag
nguyenkndinh Feb 24, 2022
dc0227c
Refactoring based on recommendations
nguyenkndinh Mar 1, 2022
5821607
Sticking to the naming convention
nguyenkndinh Mar 1, 2022
615f200
address more comments on naming
nguyenkndinh Mar 8, 2022
ac31741
Merge branch 'kubernetes:master' into master
nguyenkndinh Mar 9, 2022
f2d58ab
Using ListNode to get nodes entering and leaving the cluster.
nguyenkndinh Mar 15, 2022
cc464d6
refactor
nguyenkndinh Mar 15, 2022
a3f19e5
Add testing and controller config skeletons
nguyenkndinh Mar 19, 2022
1f12385
Added tagging and flags mechanisms
nguyenkndinh Mar 22, 2022
a580ac2
Disabled the tagging controller by default
nguyenkndinh Mar 22, 2022
966e97f
Updated test structure
nguyenkndinh Mar 23, 2022
e458d6f
Merge branch 'kubernetes:master' into master
nguyenkndinh Mar 23, 2022
5546277
Removed logging statements.
nguyenkndinh Mar 23, 2022
374a18e
Making the tests more robust
nguyenkndinh Mar 23, 2022
332ac7e
Renaming the maps in tagging controller
nguyenkndinh Mar 23, 2022
b5375b7
Refactoring names and remove debugging logs
nguyenkndinh Mar 23, 2022
dd2bd14
Add failure test cases for when EC2 return error
nguyenkndinh Mar 23, 2022
0330b61
adding details for --resources
nguyenkndinh Mar 24, 2022
0c9bd00
add in Copyright message
nguyenkndinh Mar 24, 2022
633768d
Using NodeInformer and Workqueue for tagging resources
nguyenkndinh Mar 25, 2022
2511565
Used workqueue for both tag and untag actions
nguyenkndinh Mar 25, 2022
dda006d
Update docs/tagging_controller.md
nguyenkndinh Mar 25, 2022
22067dc
Renamed fields in the tagging controller to be more user friendly
nguyenkndinh Mar 26, 2022
40f3796
Added in a loop to make sure all messages are processed before shutti…
nguyenkndinh Mar 26, 2022
6247587
Added more logging
nguyenkndinh Mar 26, 2022
4466163
Added more testing
nguyenkndinh Mar 26, 2022
fa80a6a
cosmetic change
nguyenkndinh Mar 28, 2022
2f2f639
use array instead of map for supported resources
nguyenkndinh Mar 28, 2022
df3d102
Reworked the workqueue with workitem
nguyenkndinh Mar 28, 2022
19a931d
Addressed comments
nguyenkndinh Mar 29, 2022
a27a550
addressed verify-lint errors
nguyenkndinh Mar 29, 2022
a17f858
addressed comments and verify-lint
nguyenkndinh Mar 29, 2022
4fdd368
address validate-lint error
nguyenkndinh Mar 29, 2022
2e4b7a8
missed a couple more lint errors
nguyenkndinh Mar 29, 2022
5fb201b
Updated doc to be clearer
nguyenkndinh Mar 29, 2022
781d0e2
Add TODOs for e2e testing and non-retryable workitem
nguyenkndinh Mar 30, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@ docs/book/_book/
site/
.vscode/
e2e.test
.idea/
29 changes: 20 additions & 9 deletions cmd/aws-cloud-controller-manager/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,22 +26,21 @@ limitations under the License.
package main

import (
"math/rand"
"os"
"time"

"k8s.io/apimachinery/pkg/util/wait"
cloudprovider "k8s.io/cloud-provider"
"k8s.io/cloud-provider-aws/pkg/controllers/tagging"
awsv1 "k8s.io/cloud-provider-aws/pkg/providers/v1"
awsv2 "k8s.io/cloud-provider-aws/pkg/providers/v2"
"k8s.io/cloud-provider/app"
"k8s.io/cloud-provider/options"
cliflag "k8s.io/component-base/cli/flag"
"k8s.io/component-base/logs"
_ "k8s.io/component-base/metrics/prometheus/clientgo" // for client metric registration
_ "k8s.io/component-base/metrics/prometheus/version" // for version metric registration
"k8s.io/klog/v2"

cloudprovider "k8s.io/cloud-provider"
awsv1 "k8s.io/cloud-provider-aws/pkg/providers/v1"
awsv2 "k8s.io/cloud-provider-aws/pkg/providers/v2"
"math/rand"
"os"
"time"

cloudcontrollerconfig "k8s.io/cloud-provider/app/config"
)
Expand All @@ -62,11 +61,23 @@ func main() {
}

controllerInitializers := app.DefaultInitFuncConstructors
taggingControllerWrapper := tagging.ControllerWrapper{}
fss := cliflag.NamedFlagSets{}
taggingControllerWrapper.Options.AddFlags(fss.FlagSet("tagging controller"))

taggingControllerConstructor := app.ControllerInitFuncConstructor{
InitContext: app.ControllerInitContext{
ClientName: tagging.TaggingControllerClientName,
},
Constructor: taggingControllerWrapper.StartTaggingControllerWrapper,
}

controllerInitializers[tagging.TaggingControllerKey] = taggingControllerConstructor
app.ControllersDisabledByDefault.Insert(tagging.TaggingControllerKey)
command := app.NewCloudControllerManagerCommand(opts, cloudInitializer, controllerInitializers, fss, wait.NeverStop)

if err := command.Execute(); err != nil {
os.Exit(1)
klog.Fatalf("unable to execute command: %v", err)
}
}

Expand Down
5 changes: 5 additions & 0 deletions docs/TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,8 @@
### Kops

* Add a full example (ideally with IAM roles)

### Tagging Controller

* Add e2e testing which enables the controller, and monitors if the resources are tagged properly
* Handle the case where potential non-retryable errors are enqeueued indefinitely.
7 changes: 7 additions & 0 deletions docs/tagging_controller.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# The Tagging Controller

The tagging controller is responsible for tagging and untagging node resources when they join and leave the cluster, respectively. It can add and remove tags based on user input. Additionally, if a tag is updated, it would leave the updated tag and reapply the user-provided tag. Unlike the existing controllers, the tagging controller works exclusively with AWS. The AWS APIs it uses are `ec2:CreateTags` and `ec2:DeleteTags`.

| Flag | Valid Values | Default | Description |
|------| --- | --- | --- |
| tags | Comma-separated list of key=value | - | A comma-separated list of key-value pairs which will be recorded as nodes' additional tags. For example: "Key1=Val1,Key2=Val2,KeyNoVal1=,KeyNoVal2" |
24 changes: 24 additions & 0 deletions pkg/controllers/options/resources.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
/*
Copyright 2016 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package options

const (
// Instance presenting the string literal "instance"
Instance string = "instance"
)

// SupportedResources contains the resources that can be tagged by the controller at the moment
var SupportedResources = []string{
Instance,
}
53 changes: 53 additions & 0 deletions pkg/controllers/options/tagging_controller.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
/*
Copyright 2016 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package options

import (
"fmt"
"github.com/spf13/pflag"
)

// TaggingControllerOptions contains the inputs that can
// be used in the tagging controller
type TaggingControllerOptions struct {
Tags map[string]string
Resources []string
}

// AddFlags add the additional flags for the controller
func (o *TaggingControllerOptions) AddFlags(fs *pflag.FlagSet) {
fs.StringToStringVar(&o.Tags, "tags", o.Tags, "Tags to apply to AWS resources in the tagging controller, in a form of key=value.")
fs.StringArrayVar(&o.Resources, "resources", o.Resources, "AWS resources name to add/remove tags in the tagging controller.")
}

// Validate checks for errors from user input
func (o *TaggingControllerOptions) Validate() error {
if len(o.Tags) == 0 {
return fmt.Errorf("--tags must not be empty and must be a form of key=value")
}

if len(o.Resources) == 0 {
return fmt.Errorf("--resources must not be empty")
}

for _, r := range o.Resources {
for _, resource := range SupportedResources {
if r != resource {
return fmt.Errorf("%s is not a supported resource. Current supported resources %v", r, SupportedResources)
}
}
}

return nil
}
Loading