providers: Identify Metadata configuration needs for resources marked with the manual-intervention annotation

[ulucinar]

What problem are you facing?

Provider Name: All

We are using the upjet.upbound.io/manual-intervention annotation to mark the resources across all providers that need manual operations (such as provisioning a password/cryptographic key secret, or patching the value of a field from a dependent resource, etc.) to be successfully tested. Uptest skips such resources and we need to manually test them.

How could Official Providers help solve your problem?

We may be able to address the need for manual intervention for certain cases. For example, certain resources need an input secret (such as a password secret) and the idea is we can extend the metadata configuration API so that we can include a secret (whose data is to be generated by Uptest) in the generated unified manifests (i.e., example manifests with their dependencies). What we would like to do is to identify candidates for such configuration options. Some are:

• Configuration API to Include (generate) a password secret -> Different resources may have different password policies. Do we need to be able to specify a password policy for the generated passphrase? (probably one with upper/lower/alphanumeric/symbol mix with a reasonable length should suffice in the first shot)
• Configuration API to Include (generate) a cryptographic key secret -> support for different key parameters? In the first shot support for asymmetric keys (probably RSA with a reasonable length (say 2048 bits) should suffice)
• References to file uploads to Cloud storage systems (S3 etc.) -> In some cases, content can also be inlined so that file uploads won't be necessary. However, probably it will not pay off to implement file uploads to cloud storage.
• Cross-resource manual patches between dependents -> was already implemented on top of the example manifests with dependencies feature (small delta) though no PR has been opened.
• Unique names for resources -> Has already been implemented and is currently available in upjet (the Metaresource.ExternalName API)
• Syntax corrections in HCL configurations scraped from the Terraform registry -> Already doable (available in upjet)
• Introducing new resource dependencies -> Sometimes the Terraform registry does not include the required resources on which a target (child) resource depends. Being able to include such parent dependencies would increase the level of automation. This is already doable however, a higher level API that would eliminate the need to inline HCL configuration would be appreciated.
• ...

We may try to classify the configuration needs of the resources with manual intervention needs by:

• Annotating them with configuration categories such as password, cryptokey-sym, cryptokey-asym, etc.
• Under this issue or separate issues, reference resources with special configuration needs
• Include an indicator (such as the category names suggested above) in the value of the upjet.upbound.io/manual-intervention annotation
• ...