Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: consolidate all the conditions that trigger a transformer in one place #882

Open
HarikrishnanBalagopal opened this issue Sep 29, 2022 · 3 comments
Labels
kind/feature Categorizes issue or PR as related to a new feature.

Comments

@HarikrishnanBalagopal
Copy link
Contributor

HarikrishnanBalagopal commented Sep 29, 2022

Is your feature request related to a problem? Please describe.

Currently we have a bunch of different fields to indicate when a transformer might run.

Examples:

https://github.com/konveyor/move2kube-transformers/blob/525d3854a18e9c85582178395099544e3e351363/add-custom-files-directories-in-custom-locations/transformer.yaml#L9-L16

  directoryDetect:
    levels: -1
  consumes:
    Service:
      merge: false
  override:
    matchLabels: 
      move2kube.konveyor.io/built-in: "true"

https://github.com/konveyor/move2kube-transformers/blob/525d3854a18e9c85582178395099544e3e351363/custom-default-transformer/transformer.yaml#L9-L14

  directoryDetect:
    levels: 0
  config:
    starFile: "defaulttransformer.star"
  invokedByDefault:
    enabled: true

Describe the solution you'd like

We can consolidate all the different conditions under a single field similar to Github workflows:

on:
issues:
types: [opened]

For default transformers:

on:
  transform:
    always: true

For normal transformers that consume artifacts of certain types:

on:
  transform:
    artifacts:
      - type: ArtifactType1
        merge: false
      - type: ArtifactType2

We could also trigger on path mappings.

on:
  transform:
    artifacts:
      - type: ArtifactType1
      - type: ArtifactType2
    pathMappings:
      - type: Source
      - type: Template

This could be useful if we want to add a common LICENSE on top of each file we generate.

Running after other transformers:

on:
  transform:
    transformers:
      - runAfter: true
        matchLabels:
          move2kube.konveyor.io/name: CloudFoundry

For transformers that run during both the plan and transform phase:

on:
  plan:
    directories: all
  transform:
    artifacts:
      - type: ArtifactType1
        merge: true
      - type: ArtifactType2
        merge: false

This is also a good chance to move away from 1, 0, -1 for the levels and use more meaningful names.

Additional context

Since we are moving to v0.4 this is a good time to add any breaking changes we would like.

@HarikrishnanBalagopal HarikrishnanBalagopal added the kind/feature Categorizes issue or PR as related to a new feature. label Sep 29, 2022
@HarikrishnanBalagopal
Copy link
Contributor Author

HarikrishnanBalagopal commented Sep 29, 2022

Example of how an existing Helm chart transformer would change:
https://github.com/konveyor/move2kube-transformers/blob/525d3854a18e9c85582178395099544e3e351363/add-custom-files-directories-in-custom-locations/transformer.yaml

Also note the change to apiVersion: move2kube.konveyor.io/v1beta1 so that we can be backwards-compatible and continue to support the older transformer yaml format in v0.4. Or at least detect when we are given the older format and throw a warning/error.

apiVersion: move2kube.konveyor.io/v1alpha1
kind: Transformer
metadata:
  name: CustomHelmChartGen
  labels: 
    move2kube.konveyor.io/built-in: false
spec:
  class: "Starlark"
  directoryDetect:
    levels: -1
  consumes:
    Service:
      merge: false
  override:
    matchLabels: 
      move2kube.konveyor.io/built-in: "true"
  config:
    starFile: "customhelmchartgen.star"

turns into

apiVersion: move2kube.konveyor.io/v1beta1
kind: Transformer
metadata:
  name: CustomHelmChartGen
  labels: 
    move2kube.konveyor.io/built-in: false
spec:
  class: "Starlark"
  on:
    plan:
      directories: all
    transform:
      artifacts:
        - type: Service
          merge: false
      transformers:
        - override: true
          matchLabels: 
            move2kube.konveyor.io/built-in: "true"
  config:
    starFile: "customhelmchartgen.star"

@HarikrishnanBalagopal
Copy link
Contributor Author

HarikrishnanBalagopal commented Sep 29, 2022

Example:
https://github.com/konveyor/move2kube-transformers/blob/525d3854a18e9c85582178395099544e3e351363/cloud-foundry-to-ce-iks-roks/k8s-yamls/iks/transformer.yaml

apiVersion: move2kube.konveyor.io/v1alpha1
kind: Transformer
metadata:
  name: IKSKubernetes
  labels:
    move2kube.konveyor.io/built-in: false
spec:
  class: "Kubernetes"
  directoryDetect:
    levels: 0
  consumes:
    IR:
      merge: true
  produces:
    KubernetesYamls:
      disabled: false
  override:
    matchLabels: 
      move2kube.konveyor.io/name: Kubernetes
  dependency:
    matchLabels:
      move2kube.konveyor.io/name: IKSClusterSelector
  config:
    outputPath: "deploy/yamls/iks"
    ingressName: "{{ .ProjectName }}"

turns into

apiVersion: move2kube.konveyor.io/v1beta1
kind: Transformer
metadata:
  name: IKSKubernetes
  labels:
    move2kube.konveyor.io/built-in: false
spec:
  class: "Kubernetes"
  on:
    transform:
      artifacts:
        - type: IR
      transformers:
        - override: true
          matchLabels: 
            move2kube.konveyor.io/name: Kubernetes
        - runAfter: true
          matchLabels:
            move2kube.konveyor.io/name: IKSClusterSelector
  produces:
    KubernetesYamls:
      disabled: false
  config:
    outputPath: "deploy/yamls/iks"
    ingressName: "{{ .ProjectName }}"

Since we don't want the transformer to run during the plan phase, we can omit that trigger.

@HarikrishnanBalagopal
Copy link
Contributor Author

Example:
https://github.com/konveyor/move2kube-transformers/blob/525d3854a18e9c85582178395099544e3e351363/custom-helm-kustomize-octemplates-parameterization/transformer.yaml

apiVersion: move2kube.konveyor.io/v1alpha1
kind: Transformer
metadata:
  name: AdvancedCustomParameterizer
  labels: 
    move2kube.konveyor.io/built-in: false
spec:
  class: "Parameterizer"
  directoryDetect:
    levels: -1
  consumes:
    Service:
      merge: false
    KubernetesYamlsInSource:
      merge: false
    KubernetesYamls:
      merge: true
  override:
    matchLabels: 
      move2kube.konveyor.io/name: Parameterizer
  config:
    helmPath: "{{ $pathType := EnvPathType .YamlsPath}}{{ $rel := Rel .YamlsPath }}{{ if eq $pathType \"Source\" }}source/{{end}}{{ $rel }}{{ if ne $rel \".\" }}/..{{end}}/{{ FilePathBase .YamlsPath }}-parameterized/helm-chart"
    ocTemplatePath: "{{ $pathType := EnvPathType .YamlsPath}}{{ $rel := Rel .YamlsPath }}{{ if eq $pathType \"Source\" }}source/{{end}}{{ $rel }}{{ if ne $rel \".\" }}/..{{end}}/{{ FilePathBase .YamlsPath }}-parameterized/openshift-template"
    kustomizePath: "{{ $pathType := EnvPathType .YamlsPath}}{{ $rel := Rel .YamlsPath }}{{ if eq $pathType \"Source\" }}source/{{end}}{{ $rel }}{{ if ne $rel \".\" }}/..{{end}}/{{ FilePathBase .YamlsPath }}-parameterized/kustomize"
    projectName: "{{ if eq .ArtifactType \"KubernetesYamls\" }}{{ .ProjectName }}{{ else }}{{ if eq .ArtifactType \"KubernetesYamlsInSource\" }}{{ .ArtifactName }}{{ else }}{{ .ServiceName }}{{end}}{{end}}"

turns into

apiVersion: move2kube.konveyor.io/v1beta1
kind: Transformer
metadata:
  name: AdvancedCustomParameterizer
  labels: 
    move2kube.konveyor.io/built-in: false
spec:
  class: "Parameterizer"
  on:
    plan:
      directories: all
    transform:
      artifacts:
        - type: Service
          merge: false
        - type: KubernetesYamlsInSource
          merge: false
        - type: KubernetesYamls
          merge: true
      transformers:
       - override: true
         matchLabels: 
           move2kube.konveyor.io/name: Parameterizer
  config:
    helmPath: "{{ $pathType := EnvPathType .YamlsPath}}{{ $rel := Rel .YamlsPath }}{{ if eq $pathType \"Source\" }}source/{{end}}{{ $rel }}{{ if ne $rel \".\" }}/..{{end}}/{{ FilePathBase .YamlsPath }}-parameterized/helm-chart"
    ocTemplatePath: "{{ $pathType := EnvPathType .YamlsPath}}{{ $rel := Rel .YamlsPath }}{{ if eq $pathType \"Source\" }}source/{{end}}{{ $rel }}{{ if ne $rel \".\" }}/..{{end}}/{{ FilePathBase .YamlsPath }}-parameterized/openshift-template"
    kustomizePath: "{{ $pathType := EnvPathType .YamlsPath}}{{ $rel := Rel .YamlsPath }}{{ if eq $pathType \"Source\" }}source/{{end}}{{ $rel }}{{ if ne $rel \".\" }}/..{{end}}/{{ FilePathBase .YamlsPath }}-parameterized/kustomize"
    projectName: "{{ if eq .ArtifactType \"KubernetesYamls\" }}{{ .ProjectName }}{{ else }}{{ if eq .ArtifactType \"KubernetesYamlsInSource\" }}{{ .ArtifactName }}{{ else }}{{ .ServiceName }}{{end}}{{end}}"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

No branches or pull requests

1 participant