Add CustomServiceConfig

[amoralej]

This patch adds support for services config customization:

• A top level customServiceConfig in Watcher CR is used for configuration snippets that are desired in all the Watcher services.
• A second level customServiceConfig in each one of the templates, as APIServiceTemplate, etc.. defines per-service specific config.

Depends-On: openstack-k8s-operators/ci-framework#2658

[softwarefactory-project-zuul]

This change depends on a change that failed to merge.

Change openstack-k8s-operators/ci-framework#2658 is needed.

[SeanMooney]

So other service do not have two levels fo customServiceConfig
the just have the per component customServiceConfig within a service

I'm not sure we need a usecase to have

"A top level customServiceConfig in Watcher CR is used for configuration snippets that are desired in all the Watcher services."

Im also not convinced we need customServiceConfigSecerts

its possible but we do not need that for Nova for example today so I would like to know which usecases you are envisioning that requires this.

[cescgina]

So other service do not have two levels fo customServiceConfig the just have the per component customServiceConfig within a service

I'm not sure we need a usecase to have

"A top level customServiceConfig in Watcher CR is used for configuration snippets that are desired in all the Watcher services."

Im also not convinced we need customServiceConfigSecerts

its possible but we do not need that for Nova for example today so I would like to know which usecases you are envisioning that requires this.

my 2 cents after going through the PR, I think it's ok to keep the global and service-level customConfigs. Maybe it's not very likely, but I could imagine someone wanting to configure a certain logging format for all watcher services or something like that. Also, I don't find that it adds too much complexity to the CRD or the controller code, so don't see a huge drawback. I'm less sure about the customServiceConfigSecrets, I can't imagine any conf field sensitive enough for watcher outside the connection details, which is handled differently. That said, I haven't used watcher too much, so I can't be sure there is a usecase for it.

[amoralej]

So other service do not have two levels fo customServiceConfig the just have the per component customServiceConfig within a service
I'm not sure we need a usecase to have
"A top level customServiceConfig in Watcher CR is used for configuration snippets that are desired in all the Watcher services."
Im also not convinced we need customServiceConfigSecerts
its possible but we do not need that for Nova for example today so I would like to know which usecases you are envisioning that requires this.

my 2 cents after going through the PR, I think it's ok to keep the global and service-level customConfigs. Maybe it's not very likely, but I could imagine someone wanting to configure a certain logging format for all watcher services or something like that. Also, I don't find that it adds too much complexity to the CRD or the controller code, so don't see a huge drawback. I'm less sure about the customServiceConfigSecrets, I can't imagine any conf field sensitive enough for watcher outside the connection details, which is handled differently. That said, I haven't used watcher too much, so I can't be sure there is a usecase for it.

I still think that global config can be a good thing in some cases. i.e. what if a customer needs to adjust timeouts or tune database or rabbitmq connections. Obviously, they can do it per-service, but having a global one seems to me a convenient way. Also the cost of implementing and test looks small to me. Said that, nothing that can be done globally could not be done by-service so it's not a blocker for me.

WRT the customServiceConfigSecrets the fact that nova never required it convinced me :) .

[SeanMooney]

So other service do not have two levels fo customServiceConfig the just have the per component customServiceConfig within a service
I'm not sure we need a usecase to have
"A top level customServiceConfig in Watcher CR is used for configuration snippets that are desired in all the Watcher services."
Im also not convinced we need customServiceConfigSecerts
its possible but we do not need that for Nova for example today so I would like to know which usecases you are envisioning that requires this.

my 2 cents after going through the PR, I think it's ok to keep the global and service-level customConfigs. Maybe it's not very likely, but I could imagine someone wanting to configure a certain logging format for all watcher services or something like that. Also, I don't find that it adds too much complexity to the CRD or the controller code, so don't see a huge drawback. I'm less sure about the customServiceConfigSecrets, I can't imagine any conf field sensitive enough for watcher outside the connection details, which is handled differently. That said, I haven't used watcher too much, so I can't be sure there is a usecase for it.

I still think that global config can be a good thing in some cases. i.e. what if a customer needs to adjust timeouts or tune database or rabbitmq connections. Obviously, they can do it per-service, but having a global one seems to me a convenient way. Also the cost of implementing and test looks small to me. Said that, nothing that can be done globally could not be done by-service so it's not a blocker for me.

WRT the customServiceConfigSecrets the fact that nova never required it convinced me :) .

so that is something we do not want to allow them to do
specifically we do not want to allow them to provide custom log configuration or tun the database/rabbitmq connection via customServiceConfigSecrets or customServiceConfig

they can tweak logging to some degree via the oslo.log config options but that is the extent of the log configuration we want to expsoe.

[amoralej]

So other service do not have two levels fo customServiceConfig the just have the per component customServiceConfig within a service
I'm not sure we need a usecase to have
"A top level customServiceConfig in Watcher CR is used for configuration snippets that are desired in all the Watcher services."
Im also not convinced we need customServiceConfigSecerts
its possible but we do not need that for Nova for example today so I would like to know which usecases you are envisioning that requires this.

my 2 cents after going through the PR, I think it's ok to keep the global and service-level customConfigs. Maybe it's not very likely, but I could imagine someone wanting to configure a certain logging format for all watcher services or something like that. Also, I don't find that it adds too much complexity to the CRD or the controller code, so don't see a huge drawback. I'm less sure about the customServiceConfigSecrets, I can't imagine any conf field sensitive enough for watcher outside the connection details, which is handled differently. That said, I haven't used watcher too much, so I can't be sure there is a usecase for it.

I still think that global config can be a good thing in some cases. i.e. what if a customer needs to adjust timeouts or tune database or rabbitmq connections. Obviously, they can do it per-service, but having a global one seems to me a convenient way. Also the cost of implementing and test looks small to me. Said that, nothing that can be done globally could not be done by-service so it's not a blocker for me.
WRT the customServiceConfigSecrets the fact that nova never required it convinced me :) .

so that is something we do not want to allow them to do specifically we do not want to allow them to provide custom log configuration or tun the database/rabbitmq connection via customServiceConfigSecrets or customServiceConfig

they can tweak logging to some degree via the oslo.log config options but that is the extent of the log configuration we want to expsoe.

What'd be the recommended way to tune parameters for customers then?

There are a bunch of config parameters in watcher.conf that may be used in different sections, timeouts, pool sizes, etc... I understand in certain cases, maybe following recomendations from support engineers, customers may need to tweak them.

[SeanMooney]

ok, if we really want the two levels of defaulting then lets proceed with this as is.

[SeanMooney]

So other service do not have two levels fo customServiceConfig the just have the per component customServiceConfig within a service
I'm not sure we need a usecase to have
"A top level customServiceConfig in Watcher CR is used for configuration snippets that are desired in all the Watcher services."
Im also not convinced we need customServiceConfigSecerts
its possible but we do not need that for Nova for example today so I would like to know which usecases you are envisioning that requires this.

my 2 cents after going through the PR, I think it's ok to keep the global and service-level customConfigs. Maybe it's not very likely, but I could imagine someone wanting to configure a certain logging format for all watcher services or something like that. Also, I don't find that it adds too much complexity to the CRD or the controller code, so don't see a huge drawback. I'm less sure about the customServiceConfigSecrets, I can't imagine any conf field sensitive enough for watcher outside the connection details, which is handled differently. That said, I haven't used watcher too much, so I can't be sure there is a usecase for it.

I still think that global config can be a good thing in some cases. i.e. what if a customer needs to adjust timeouts or tune database or rabbitmq connections. Obviously, they can do it per-service, but having a global one seems to me a convenient way. Also the cost of implementing and test looks small to me. Said that, nothing that can be done globally could not be done by-service so it's not a blocker for me.
WRT the customServiceConfigSecrets the fact that nova never required it convinced me :) .

so that is something we do not want to allow them to do specifically we do not want to allow them to provide custom log configuration or tun the database/rabbitmq connection via customServiceConfigSecrets or customServiceConfig
they can tweak logging to some degree via the oslo.log config options but that is the extent of the log configuration we want to expsoe.

What'd be the recommended way to tune parameters for customers then?

There are a bunch of config parameters in watcher.conf that may be used in different sections, timeouts, pool sizes, etc... I understand in certain cases, maybe following recomendations from support engineers, customers may need to tweak them.

parameter in the watcher.conf can be tuned via the customServiceConfig

however, our goal is for our default to to work for most out of the box and to minimise the amount of tuning they require.

the majority of filed in https://docs.openstack.org/watcher/latest/configuration/watcher.html should not need to be adjusted.

we may override some of them in https://github.com/openstack-k8s-operators/watcher-operator/blob/main/templates/watcher/config/00-default.conf to better align with our deployment tool as the goal of the operator is to encode the operational knowledge of how to run the service so that our customer don't have to discover that by them selevs.

there will be some things that need to be changed on a per-deployment basis but i expect that to be rare. Unlike Nova or some other project that need customisation for specific hardware, watcher is much less coupled to the specific of the cloud. so i expect less customistaion to be needed.

[amoralej]

So other service do not have two levels fo customServiceConfig the just have the per component customServiceConfig within a service
I'm not sure we need a usecase to have
"A top level customServiceConfig in Watcher CR is used for configuration snippets that are desired in all the Watcher services."
Im also not convinced we need customServiceConfigSecerts
its possible but we do not need that for Nova for example today so I would like to know which usecases you are envisioning that requires this.

my 2 cents after going through the PR, I think it's ok to keep the global and service-level customConfigs. Maybe it's not very likely, but I could imagine someone wanting to configure a certain logging format for all watcher services or something like that. Also, I don't find that it adds too much complexity to the CRD or the controller code, so don't see a huge drawback. I'm less sure about the customServiceConfigSecrets, I can't imagine any conf field sensitive enough for watcher outside the connection details, which is handled differently. That said, I haven't used watcher too much, so I can't be sure there is a usecase for it.

I still think that global config can be a good thing in some cases. i.e. what if a customer needs to adjust timeouts or tune database or rabbitmq connections. Obviously, they can do it per-service, but having a global one seems to me a convenient way. Also the cost of implementing and test looks small to me. Said that, nothing that can be done globally could not be done by-service so it's not a blocker for me.
WRT the customServiceConfigSecrets the fact that nova never required it convinced me :) .

so that is something we do not want to allow them to do specifically we do not want to allow them to provide custom log configuration or tun the database/rabbitmq connection via customServiceConfigSecrets or customServiceConfig
they can tweak logging to some degree via the oslo.log config options but that is the extent of the log configuration we want to expsoe.

What'd be the recommended way to tune parameters for customers then?
There are a bunch of config parameters in watcher.conf that may be used in different sections, timeouts, pool sizes, etc... I understand in certain cases, maybe following recomendations from support engineers, customers may need to tweak them.

parameter in the watcher.conf can be tuned via the customServiceConfig

however, our goal is for our default to to work for most out of the box and to minimise the amount of tuning they require.

the majority of filed in https://docs.openstack.org/watcher/latest/configuration/watcher.html should not need to be adjusted.

we may override some of them in https://github.com/openstack-k8s-operators/watcher-operator/blob/main/templates/watcher/config/00-default.conf to better align with our deployment tool as the goal of the operator is to encode the operational knowledge of how to run the service so that our customer don't have to discover that by them selevs.

there will be some things that need to be changed on a per-deployment basis but i expect that to be rare. Unlike Nova or some other project that need customisation for specific hardware, watcher is much less coupled to the specific of the cloud. so i expect less customistaion to be needed.

I fully agreee with all you said here. We should be able to provide good defaults and I hope per-deployments adjustments to be rare.

I just thought on the best way for those (hopefully) rare cases.

[marios]

this was a bit easier to follow after the rebase (less going on) and the overview you gave me yesterday o/

lgtm but not confident enough for approve yet ;)

[cescgina]

/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cescgina

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [cescgina]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[cescgina]

let's keep both the global and per service custom config for now, and we can re-evaluate in the future