[Chore] Improve logging

[danail-branekov]

Background

Korifi is logging a lot. The overwhelming amount of logs we produce makes troubleshooting hard. Furthermore, we often get log severity wrong. For example, if a controller sees that the object it reconcile is missing a dependency, it logs an error. That dependency quite probably would appear eventually, so those errors simply introduce noise.

Action to take

• Come up with a guideline how to log temporary errors (e.g. agree on a log severity)
• Come up with a guideline what is considered an error that operators should be bothered with
• Come up with a guideline whether to log, or put a "not-ready" reason in the object status. Or maybe fire a k8s event?
• Check all the places (both api and controllers land) we log stuff and apply those guidelines