How to add functions that need charm libraries to chisme?

[ca-scribner]

Context

We've had times where we'd like to move a function that depends on a charm library to chisme (for example, in canonical/envoy-operator#87 and canonical/kfp-operators#436, which both have the same K8sServiceInfoComponent).
Because these depend on a charm library, chisme would then need that charm library.

An option could do a simple vendoring of the charm libraries in chisme, but then chisme would:

• always include all the libraries (say prometheus, grafana, etc, even if the charm using chisme doesn't need these libraries)
• the version of these libraries would be set by chisme directly (can't charmcraft fetch-lib within the charm to update libraries)

This likely is not manageable, at least not without some advanced stuff.

Some possible solutions:

1. Can we refactor our reusable tools to use dependency injection, where they accept as args the things they need from the libs rather than hard-code their dependency on the libs?
2. Can we use optional dependencies and somehow support multiple library versions? Chisme would then only ship with library-specific code when someone asks for it (pip install chisme[k8s-service-lib]). I think then in the chisme repo we'd still install the charm lib (somehow manually?) and use it for testing (maybe installing several versions in a matrix of tests) but the charm lib wouldn't ship those
3. Or maybe Chisme can ship without the libs, but still include all the lib-specific code? Which would only work if the charm also installed the necessary lib?

To me, (1) is probably the easiest

What needs to get done

1. prototype how this could be done and design a solution
2. implement at least one working example

Definition of Done

1. an approach is in place, with at least one example, of how to put code that uses charm libs in chisme

[syncronize-issues-to-jira]

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-5580.

This message was autogenerated

[orfeas-k]

My quick experience on this, I had played around with possible solution 3 (point 1 described in that spec section). AFAIR, this was working properly (example function) and the function was acting like it lived inside the charm's code.

Also, I had tried adding the libs to chisme and having them delivered with the code (by default) but I couldn't get the charm to locate those (not saying it's impossible). Example implementation here and here.

[rgildein]

What I tried to do is fetch-lib in root directory, like this:

$ charmcraft fetch-lib charms.prometheus_k8s.v0.prometheus_scrape       
Library charms.prometheus_k8s.v0.prometheus_scrape version 0.47 downloaded.
$ ll lib/charms 
total 4.0K
drwxrwxr-x 3 rgildein rgildein 4.0K May 22 16:10 prometheus_k8s           

then I changed the setup.cfg to include package_data, like this:

[options]
...
include_package_data = True

[options.package_data]
* = charms/*.py

With these changes, we can use libs directly in chisme, like this:

from lib.charms.prometheus_k8s.v0.prometheus_scrape import MetricsEndpointProvider

With this approach it should be possible to have different version of lib in charm and different version in chisme, so if chisme using lib smartly and charm will never import from lib.charms, it will be possible to do easy bump version of any lib in chisme. e.g. charm can keep using same function from chisme (foo(self, port)) and all logic to use new version of lib will be done on chisme side (foo change inside).