feat: support cue http

Signed-off-by: kaku <[email protected]>

README-zh.md

@@ -0,0 +1,116 @@
+# kinitiras
+![kinitiras-logo](docs/images/kinitiras.png)
+
+[![Build Status](https://github.com/k-cloud-labs/kinitiras/actions/workflows/ci.yml/badge.svg)](https://github.com/k-cloud-labs/kinitiras/actions?query=workflow%3Abuild)
+[![codecov](https://codecov.io/gh/k-cloud-labs/kinitiras/branch/main/graph/badge.svg?token=74uYpOiawR)](https://codecov.io/gh/k-cloud-labs/kinitiras)
+[![Go Report Card](https://goreportcard.com/badge/github.com/k-cloud-labs/kinitiras)](https://goreportcard.com/report/github.com/k-cloud-labs/kinitiras)
+[![Go doc](https://img.shields.io/badge/go.dev-reference-brightgreen?logo=go&logoColor=white&style=flat)](https://pkg.go.dev/github.com/k-cloud-labs/kinitiras)
+
+[[English](README.md)]
+
+**轻量**、**功能强大**、**可编程的** k8s admission webhook 规则引擎。
+
+如果你想在客户端实现类似能力，请使用 [pidalio](https://github.com/k-cloud-labs/pidalio)。
+
+## 快速开始
+
+### 部署 CRD
+```shell
+kubectl apply -f https://raw.githubusercontent.com/k-cloud-labs/pkg/main/charts/_crds/bases/policy.kcloudlabs.io_overridepolicies.yaml
+kubectl apply -f https://raw.githubusercontent.com/k-cloud-labs/pkg/main/charts/_crds/bases/policy.kcloudlabs.io_clusteroverridepolicies.yaml
+kubectl apply -f https://raw.githubusercontent.com/k-cloud-labs/pkg/main/charts/_crds/bases/policy.kcloudlabs.io_clustervalidatepolicies.yaml
+```
+
+### 部署应用
+所有资源将会被默认部署在 `kinitiras-system` 命名空间下，你可以按需修改部署文件 `deploy/deploy.yaml`。
+
+默认 webhook 配置会对所有包含 `kinitiras.kcloudlabs.io/webhook: enabled` 标签的资源对象进行拦截，你可以按需修改对应文件 `deploy/webhook-configuration.yaml`。
+**_部署前请按需修改所有 `deploy` 下的部署文件._**
+
+修改完之后执行如下命令部署到集群即可。
+
+```shell
+kubectl apply -f deploy/
+```
+
+### 创建策略
+支持三种策略，作用和生效范围如下：
+
+`OverridePolicy` 可以修改同命名空间下的资源对象。
+`ClusterOverridePolicy` 可以修改任意命名空间下的资源对象。
+`CLusterValidatePolciy` 可以校验任意命名空间下的资源对象的操作。
+
+针对集群级别的资源:
+- 按照匹配的 `ClusterOverridePolicy` 策略名称的字母顺序进行应用；
+
+针对命名空间级别的资源对象:
+- 首先应用所有匹配的 `ClusterOverridePolicy`;
+- 其次应用虽有匹配的 `OverridePolicy`;
+
+策略的可编程能力依赖 [CUE](https://cuelang.org/).
+
+### 约束
+1. K8s 资源对象通过 `object` 参数传递，针对修改请求，老资源对象将通过 `oldObject` 参数传递，无需入参时可省略，但参数名不可修改；
+2. Mutating 结果将以 `patches` 参数返回；
+3. Validating 结果将以 `validate` 参数返回； 
+4. 数据传输在 `processing` 节点定义，包含 `http` 和 `output` 两个子节点
+    1. `http` 用来发送 http(s) 请求. 参考: [http](https://pkg.go.dev/cuelang.org/go/pkg/tool/http)；
+    2. `output` 用来接受返回结果，按需定义其结构即可；
+
+结构定义:
+
+```cue
+// oldObject 只针对 `clustervalidatepolicy` 策略中的 `UPDATE` 操作  
+object: _ @tag(object) 
+oldObject: _ @tag(oldObject)
+
+processing: {
+	output: {
+		// 按需自定义返回体结构	
+	}
+	http: {
+	    method: *"GET" | string
+	    url: parameter.serviceURL
+	    request: {
+	    	body ?: bytes
+	    	header: {}
+	    	trailer: {}
+	    }
+	}
+}
+
+patch: {
+	op: string
+	path: string
+	value: string
+}
+
+// mutating 返回结构
+patches: [...patch] 
+
+// validating 返回结构
+validate: { 
+	reason?: string
+	valid: bool
+}
+```
+
+
+## 例子
+example 文件夹下有如下实例供参考。
+
+`deletens-cvp.yaml` 保护带有 `kinitiras.kcloudlabs.io/webhook=enabled` 标签的命名空间被删除。
+
+`addanno-op.yaml` 将会给默认命名空间下带有 `kinitiras.kcloudlabs.io/webhook=enabled` 标签的 pod 添加 `added-by=op` annotation。
+
+`addanno-cop.yaml` 将会给默认命名空间下带有 `kinitiras.kcloudlabs.io/webhook=enabled` 标签的 pod 添加 `added-by=cue` annotation。
+
+## 特性
+- [x] 支持通过在 (Cluster)OverridePolicy 策略中以 plaintext 方式实现对 k8s 资源对象的修改。
+- [x] 支持通过在 (Cluster)OverridePolicy 策略中以 cue 可编程的方式实现对 k8s 资源对象的修改。
+- [x] 支持通过在 ClusterValidatePolicy 策略中以 cue 可编程的方式实现对 k8s 资源对象的校验。
+- [x] 支持在策略中使用 CUE 发送 http 请求。
+- [ ] 支持使用 kubectl plugin 进行 CUE 内容校验。
+- [ ] ...
+
+更多详细内容，请参考 [roadmap](./ROADMAP.md)。

README.md

@@ -6,9 +6,11 @@
 [![Go Report Card](https://goreportcard.com/badge/github.com/k-cloud-labs/kinitiras)](https://goreportcard.com/report/github.com/k-cloud-labs/kinitiras)
 [![Go doc](https://img.shields.io/badge/go.dev-reference-brightgreen?logo=go&logoColor=white&style=flat)](https://pkg.go.dev/github.com/k-cloud-labs/kinitiras)
 
+[[中文](README-zh.md)]
+
 A **lightweight** but **powerful** and **programmable** rule engine for kubernetes admission webhook.
 
-If you want to use it in clientside with client-go, please use https://github.com/k-cloud-labs/pidolio.
+If you want to use it in clientside with client-go, please use [pidalio](https://github.com/k-cloud-labs/pidalio).
 
 ## Quick Start
 
@@ -48,8 +50,13 @@ For namespaced scoped resource, apply order is:
 
 Both mutate and validate policy are programmable via [CUE](https://cuelang.org/).   
 
-There is a **constraint** that the kubernetes object will be passed to CUE by `object` parameter and the mutating result will be returned by `patches`
-parameter and validating result will be returned by `validate` parameter.
+### Constraint
+1. The kubernetes object will be passed to CUE by `object` parameter.
+2. The mutating result will be returned by `patches` parameter. 
+3. The Validating result will be returned by `validate` parameter.  
+4. Use `processing` to support data passing. It contains `http` and `output` schema.
+   1. `http` used to make a http(s) request. Refer to: [http](https://pkg.go.dev/cuelang.org/go/pkg/tool/http) 
+   2. `output` used to receive response. You should add some properties you need to it.
 
 Schema:  
 
@@ -58,11 +65,28 @@ Schema:
 object: _ @tag(object) 
 oldObject: _ @tag(oldObject)
 
+// use processing to pass data. A http reqeust will be make and output contains the response.
+processing: {
+	output: {
+		// add what you need	
+	}
+	http: {
+	    method: *"GET" | string
+	    url: parameter.serviceURL
+	    request: {
+	    	body ?: bytes
+	    	header: {}
+	    	trailer: {}
+	    }
+	}
+}
+
 patch: {
 	op: string
 	path: string
 	value: string
 }
+
 // for mutating result
 patches: [...patch] 
 
@@ -86,7 +110,8 @@ The `addanno-cop.yaml` will add annotation `added-by=cue` to pod labeled with `k
 ## Feature
 - [x] Support mutate k8s resource by (Cluster)OverridePolicy via plaintext jsonpatch.
 - [x] Support mutate k8s resource by (Cluster)OverridePolicy programmable via CUE.
-- [x] Support validate k8s resource by ClusterValdiatePolicy programmable via CUE.
+- [x] Support validate k8s resource by ClusterValidatePolicy programmable via CUE.
+- [x] Support Data passing by http request via CUE.
 - [ ] kubectl plugin to validate CUE.
 - [ ] ...
 

ROADMAP.md

@@ -5,6 +5,9 @@
 - ClusterValidatingPolicy for validate everything of `CREATE`、`UPDATE` and `DELETE`.
 - Policy is fully programmable via cue.
 
+## v0.1.1
+- Support data passing by http request via cue.
+
 ## v0.2.0
 - More admission request info passed to cue script. e.g. userinfo.
 - Kinitiras ctl to validate cue script, also kubectl plugin.  

cmd/app/webhook.go

@@ -9,9 +9,9 @@ import (
 	"os"
 
 	policyv1alpha1 "github.com/k-cloud-labs/pkg/apis/policy/v1alpha1"
-	"github.com/k-cloud-labs/pkg/util/informermanager"
-	"github.com/k-cloud-labs/pkg/util/overridemanager"
-	"github.com/k-cloud-labs/pkg/util/validatemanager"
+	"github.com/k-cloud-labs/pkg/utils/informermanager"
+	"github.com/k-cloud-labs/pkg/utils/overridemanager"
+	"github.com/k-cloud-labs/pkg/utils/validatemanager"
 	"github.com/spf13/cobra"
 	"k8s.io/apimachinery/pkg/runtime/schema"
 	"k8s.io/client-go/dynamic"

deploy/deploy.yaml

@@ -42,7 +42,7 @@ spec:
             - --v=2
             - --logtostderr=false
             - --log_dir=./log
-          image: registry.cn-hangzhou.aliyuncs.com/k-cloud-labs/kinitiras-webhook:0.1.0
+          image: registry.cn-hangzhou.aliyuncs.com/k-cloud-labs/kinitiras-webhook:0.2.0
           imagePullPolicy: IfNotPresent
           name: kinitiras-webhook
           env:

go.mod

@@ -4,7 +4,7 @@ go 1.17
 
 require (
 	github.com/go-logr/logr v1.2.3
-	github.com/k-cloud-labs/pkg v0.2.0
+	github.com/k-cloud-labs/pkg v0.3.0
 	github.com/open-policy-agent/cert-controller v0.3.0
 	github.com/spf13/cobra v1.4.0
 	github.com/spf13/pflag v1.0.5

go.sum

@@ -83,6 +83,8 @@ github.com/bketelsen/crypt v0.0.3-0.20200106085610-5cbc8cc4026c/go.mod h1:MKsuJm
 github.com/bketelsen/crypt v0.0.4/go.mod h1:aI6NrJ0pMGgvZKL1iVgXLnfIFJtfV+bKCoqOes/6LfM=
 github.com/blang/semver v3.5.1+incompatible h1:cQNTCjp13qL8KC3Nbxr/y2Bqb63oX6wdnnjpJbkM4JQ=
 github.com/blang/semver v3.5.1+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk=
+github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869 h1:DDGfHa7BWjL4YnC6+E63dPcxHo2sUxDIu8g3QgEJdRY=
+github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869/go.mod h1:Ekp36dRnpXw/yCqJaO+ZrUyxD+3VXMFFr56k5XYrpB4=
 github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
 github.com/certifi/gocertifi v0.0.0-20191021191039-0944d244cd40/go.mod h1:sGbDF6GwGcLpkNXPUTkMRoywsNa/ol15pxFe6ERfguA=
 github.com/certifi/gocertifi v0.0.0-20200922220541-2c3bb06c6054/go.mod h1:sGbDF6GwGcLpkNXPUTkMRoywsNa/ol15pxFe6ERfguA=
@@ -329,8 +331,8 @@ github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/X
 github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
 github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
 github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
-github.com/k-cloud-labs/pkg v0.2.0 h1:kO1O3h7mhHKXdXh8ERMein8um/wSVk46qzoqvqUZdtk=
-github.com/k-cloud-labs/pkg v0.2.0/go.mod h1:+5RsSMC7VOdDXdRLwTBZzcY8W931ETI4r4DBqUSBnCk=
+github.com/k-cloud-labs/pkg v0.3.0 h1:yM3SFCnM4tr+QJfNoUtbg1lKIQsA/GLppGWBD0OYSvY=
+github.com/k-cloud-labs/pkg v0.3.0/go.mod h1:EjvI8cW+qngQ4mT6QaMPbFE0BT1CmG8o27Foj4p22lg=
 github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
 github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
 github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=

pkg/lister/clusteroverridepolicylister.go

@@ -19,12 +19,11 @@ package lister
 import (
 	policyv1alpha1 "github.com/k-cloud-labs/pkg/apis/policy/v1alpha1"
 	"github.com/k-cloud-labs/pkg/client/listers/policy/v1alpha1"
+	"github.com/k-cloud-labs/pkg/utils/util"
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
 	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
 	"k8s.io/apimachinery/pkg/labels"
 	"k8s.io/client-go/tools/cache"
-
-	"github.com/k-cloud-labs/pkg/util/converter"
 )
 
 // clusterOverridePolicyLister implements the ClusterOverridePolicyLister interface.
@@ -40,7 +39,7 @@ func NewUnstructuredClusterOverridePolicyLister(indexer cache.Indexer) v1alpha1.
 // List lists all ClusterOverridePolicies in the indexer.
 func (s *unstructuredClusterOverridePolicyLister) List(selector labels.Selector) (ret []*policyv1alpha1.ClusterOverridePolicy, err error) {
 	err = cache.ListAll(s.indexer, selector, func(m interface{}) {
-		cop, _ := converter.ConvertToClusterOverridePolicy(m.(*unstructured.Unstructured))
+		cop, _ := util.ConvertToClusterOverridePolicy(m.(*unstructured.Unstructured))
 		ret = append(ret, cop)
 	})
 	return ret, err
@@ -55,6 +54,6 @@ func (s *unstructuredClusterOverridePolicyLister) Get(name string) (*policyv1alp
 	if !exists {
 		return nil, apierrors.NewNotFound(policyv1alpha1.Resource("clusteroverridepolicy"), name)
 	}
-	cop, _ := converter.ConvertToClusterOverridePolicy(obj.(*unstructured.Unstructured))
+	cop, _ := util.ConvertToClusterOverridePolicy(obj.(*unstructured.Unstructured))
 	return cop, nil
 }

pkg/lister/overridepolicylister.go

@@ -18,13 +18,12 @@ package lister
 
 import (
 	policyv1alpha1 "github.com/k-cloud-labs/pkg/apis/policy/v1alpha1"
-	"github.com/k-cloud-labs/pkg/util/converter"
+	"github.com/k-cloud-labs/pkg/client/listers/policy/v1alpha1"
+	"github.com/k-cloud-labs/pkg/utils/util"
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
 	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
 	"k8s.io/apimachinery/pkg/labels"
 	"k8s.io/client-go/tools/cache"
-
-	"github.com/k-cloud-labs/pkg/client/listers/policy/v1alpha1"
 )
 
 // unstructuredOverridePolicyLister implements the OverridePolicyLister interface.
@@ -40,7 +39,7 @@ func NewUnstructuredOverridePolicyLister(indexer cache.Indexer) v1alpha1.Overrid
 // List lists all OverridePolicies in the indexer.
 func (s *unstructuredOverridePolicyLister) List(selector labels.Selector) (ret []*policyv1alpha1.OverridePolicy, err error) {
 	err = cache.ListAll(s.indexer, selector, func(m interface{}) {
-		op, _ := converter.ConvertToOverridePolicy(m.(*unstructured.Unstructured))
+		op, _ := util.ConvertToOverridePolicy(m.(*unstructured.Unstructured))
 		ret = append(ret, op)
 	})
 	return ret, err
@@ -61,7 +60,7 @@ type unstructuredOverridePolicyNamespaceLister struct {
 // List lists all OverridePolicies in the indexer for a given namespace.
 func (s unstructuredOverridePolicyNamespaceLister) List(selector labels.Selector) (ret []*policyv1alpha1.OverridePolicy, err error) {
 	err = cache.ListAllByNamespace(s.indexer, s.namespace, selector, func(m interface{}) {
-		op, _ := converter.ConvertToOverridePolicy(m.(*unstructured.Unstructured))
+		op, _ := util.ConvertToOverridePolicy(m.(*unstructured.Unstructured))
 		ret = append(ret, op)
 	})
 	return ret, err
@@ -76,6 +75,6 @@ func (s unstructuredOverridePolicyNamespaceLister) Get(name string) (*policyv1al
 	if !exists {
 		return nil, apierrors.NewNotFound(policyv1alpha1.Resource("overridepolicy"), name)
 	}
-	op, _ := converter.ConvertToOverridePolicy(obj.(*unstructured.Unstructured))
+	op, _ := util.ConvertToOverridePolicy(obj.(*unstructured.Unstructured))
 	return op, nil
 }

pkg/lister/validatepolicylister.go

@@ -19,12 +19,11 @@ package lister
 import (
 	policyv1alpha1 "github.com/k-cloud-labs/pkg/apis/policy/v1alpha1"
 	"github.com/k-cloud-labs/pkg/client/listers/policy/v1alpha1"
+	"github.com/k-cloud-labs/pkg/utils/util"
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
 	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
 	"k8s.io/apimachinery/pkg/labels"
 	"k8s.io/client-go/tools/cache"
-
-	"github.com/k-cloud-labs/pkg/util/converter"
 )
 
 // clusterValidatePolicyLister implements the ClusterValidatePolicyLister interface.
@@ -40,7 +39,7 @@ func NewUnstructuredClusterValidatePolicyLister(indexer cache.Indexer) v1alpha1.
 // List lists all ClusterValidatePolicies in the indexer.
 func (s *unstructuredClusterValidatePolicyLister) List(selector labels.Selector) (ret []*policyv1alpha1.ClusterValidatePolicy, err error) {
 	err = cache.ListAll(s.indexer, selector, func(m interface{}) {
-		cvp, _ := converter.ConvertToClusterValidatePolicy(m.(*unstructured.Unstructured))
+		cvp, _ := util.ConvertToClusterValidatePolicy(m.(*unstructured.Unstructured))
 		ret = append(ret, cvp)
 	})
 	return ret, err
@@ -55,6 +54,6 @@ func (s *unstructuredClusterValidatePolicyLister) Get(name string) (*policyv1alp
 	if !exists {
 		return nil, apierrors.NewNotFound(policyv1alpha1.Resource("clustervalidatepolicy"), name)
 	}
-	cvp, _ := converter.ConvertToClusterValidatePolicy(obj.(*unstructured.Unstructured))
+	cvp, _ := util.ConvertToClusterValidatePolicy(obj.(*unstructured.Unstructured))
 	return cvp, nil
 }

pkg/webhook/mutating.go

@@ -6,8 +6,8 @@ import (
 	"errors"
 	"net/http"
 
-	"github.com/k-cloud-labs/pkg/util"
-	"github.com/k-cloud-labs/pkg/util/overridemanager"
+	"github.com/k-cloud-labs/pkg/utils"
+	"github.com/k-cloud-labs/pkg/utils/overridemanager"
 	admissionv1 "k8s.io/api/admission/v1"
 	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
 	"k8s.io/klog/v2"
@@ -46,7 +46,7 @@ func (a *MutatingAdmission) Handle(ctx context.Context, req admission.Request) a
 		if err != nil {
 			return admission.Errored(http.StatusInternalServerError, err)
 		}
-		klog.V(4).InfoS("override policy applied.", "resource", klog.KObj(obj), util.AppliedOverrides, string(opBytes), util.AppliedClusterOverrides, string(copBytes))
+		klog.V(4).InfoS("override policy applied.", "resource", klog.KObj(obj), utils.AppliedOverrides, string(opBytes), utils.AppliedClusterOverrides, string(copBytes))
 	} else {
 		klog.InfoS("override policy applied.", "resource", klog.KObj(obj))
 	}

pkg/webhook/validateing.go

@@ -4,10 +4,11 @@ import (
 	"context"
 	"net/http"
 
-	pkgadmission "github.com/k-cloud-labs/kinitiras/pkg/admission"
-	"github.com/k-cloud-labs/pkg/util/validatemanager"
+	"github.com/k-cloud-labs/pkg/utils/validatemanager"
 	"sigs.k8s.io/controller-runtime/pkg/webhook"
 	"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
+
+	pkgadmission "github.com/k-cloud-labs/kinitiras/pkg/admission"
 )
 
 type ValidatingAdmission struct {