question about configuration/runtime keys and "staging" thereof

[hannesm]

So, I understand there's this approach that you have to define boot parameters in config.ml, and also configuration-time parameters.

I also understand that both kinds of keys are useful.

What I do not understand is: what is the usage of keys that are both available (can be filled) at configuration and runtime? E.g. the IP address of a network interface, there we have now 3 times when to define them:

• at development time in the key,
• at mirage configure --ipv4= time,
• at the startup of the unikernel.

I also understand from an academic point of view this may be an interesting property.

But from a practical point of view, I suggest to disentangle the keys that are used at configuration time (i.e. the target, and selection of features that require other packages), and those at runtime (e.g. the IP address).

Why do I propose this? Well, first of all, I went through all the unikernels I could find to figure out where exactly this "put another value at configuration time" is used and useful -- apart from the hello world example I couldn't find any. Another reason is that there is heavy machinery in functoria/functoria-runtime/mirage/mirage-runtime to deal with serialising and deserialising keys, which includes depending on ipaddr in mirage-runtime; but also all these poor unikernels that have the keys only available as base types (string / int -- but somehow Ipaddr is special enough that it got promoted as a hard dependency). Now, stuff like a Private_key.t is not there in mirage-runtime (and should not be).

So, codewise I don't quite understand functoria enough to deal with this, but my first question is: are there any real use-cases for having staged keys? The second question is: can we disentangle these keys (and (a) remove key_gen.ml, (b) the whole "defined stuff in config.ml which belongs to unikernel.ml", and (c) even the fork & modification of the cmdliner API, (d) unclutter the manpage)? What do you think?

My suspicion is that we do not even need to break the boot parameters (i.e. any existing unikernel), but we'd be able to remove a bunch of code (and the ipaddr dependency from mirage-runtime).

//cc @mirage/core @Drup

[hannesm]

I'd really like to get a discussion rolling on this one, maybe @TheLortex @samoht @dinosaure @avsm have an opinion on it? It would solve #1074 -- and may address #1178, and in a world where we don't have key_gen.ml, there's no need to generate an interface file for it (#1244).

So, what do you think? And how intertwined is functoria and the key processing/registration?

[samoht]

I think it is generally a good idea: I'm in favour of simplifying functoria to keep only what is actually needed.

One potential use-case for keys that I'd like to keep somehow is the following -- imagine that in a near future, you could configure your unikernel with a dune stanza:

(unikernel
  (name kv)
  (target hvt)
  (params (ipv4 10.0.0.4)))

(unikernel
  (name kv_ro)
  (target xen)
  (params (ipv4 10.0.0.5)))

and build both of these unikernels (with different parameters) with a single dune build -- note: we are not very far away from this already, as it's almost just a matter of calling mirage configure multiple times in the same repo.

Having said all of this, I also agree that what we need is a better way to handle runtime configurations. Maybe we should have the equivalent of a unikernel.conf file with some default runtime keys generated in there and the ability to change them easily, instead of having configure-time options.

[hannesm]

Thanks for your comment.

So, what would be the benefit to customize your IP address based on your target? I don't quite understand this use case. I think it is fine to "configure" with a "dune stanza", as described above, but I also think that "what can be configured" is the target, and setups for adding further packages as dependencies (i.e. enable-monitoring / tls, as in mirage-www).

What is "the equivalent of unikernel.conf"? Who should read and apply such a file? For e.g. solo5-hvt/spt I can see how such a thing could be done, for Xen and virtio and muen I don't see a path.

Back to my original post, what is the value of configure an IP address at configuration time? (Note that I'm in the camp that we should distribute general unikernels to everyone, and do configuration only at boot time (to support reproducible builds, and store no secrets in unikernels).)

[samoht]

We should distribute general unikernels to everyone and do configuration only at boot time (to support reproducible builds and store no secrets in unikernels).

That's a good argument. I'm very much in favour of supporting reproducible builds by default, so if removing configuration keys helps with this, let's try to do this.

But how far do you want to go? Right now, functoria's keys also give a "schema", i.e. you can check which parameter to use easily at each stage of your build/deploy pipeline by looking at the man page. If functoria would focus only on configuration keys, how would you signal to our users which runtime keys to use? Using "normal" cmdliner arguments? If yes, how do you do something like "query the help page" for runtime arguments?

Regarding "unikernel.conf" -- what I meant is that when you start having many keys to configure, knowing which ones you need and passing all of them as CLI arguments can be a bit cumbersome, and it would be great to ease this. We could have a way to serialise a set of runtime keys into a schema template (using JSON or other) and let it fill it by the user instead of guessing and passing each individual key.

[hannesm]

If yes, how do you do something like "query the help page" for runtime arguments?

Well, solo5-hvt -- dist/hello.hvt --help? which btw is displayed if the unikernel is missing arguments or gets unknown ones.

unikernel.conf .. serialise a set of runtime keys into a schema template

That's something you can do with the Cmdliner API (I suspect dbuenzli/cmdliner#1 is related). In any case, it is a slight diversion -- could you please open a different issue for this feature request.

It's likely a simple matter of programming, taking a unikernel image, to figure out how to render a manpage with the supported arguments -- even for an arbitrary image and not executing it under a tender / virtualization.

If it turns out to be hard, we can play a similar thing as solo5 does with the manifest -- put stuff into a custom ELF section, and provide tooling to extract that information (no, I'm not suggesting to use the very same section).

One of the goals of this issue is "how to remove the Ipaddr dependency from hello-world".

[samoht]

solo5-hvt -- dist/hello.hvt --help

My question was more: are you ok with losing the ability to query what are the runtime keys at configure time? ie. currently, we have a way (not super intuitive, but still a way) to view all the available keys in one place. If runtime keys just become cmdliner terms, then we can't easily query them at compile time, and so it might be harder to make sure we provide the suitable set of keys at runtime (which is maybe OK, but I want to check this is what you had in mind). For instance, mirage describe gives you what components are used but also what key they would need at runtime.

I'm happy to open another issue once I understand what you want to remove and keep. I'm also ok to start experimenting with removing code and see exactly what we lose (and open issue to keep track of introducing back what we want to keep).

[hannesm]

are you ok with losing the ability to query what are the runtime keys at configure time?

Thanks for asking, yes indeed I'm fine with that. If we want to get this back, as mentioned above, I'm sure there are ways. I never used mirage describe.

[hannesm]

So, motivation includes:

• remove dependencies & code from mirage/mirage-runtime (e.g. ipaddr)
• minimize "generation of code" -- i.e. no key_gen.ml anymore!
• be able to specify actual types for arguments (i.e. X509.Key_type.t instead of string and manual processing)
  --> specify keys in unikernel.ml rather than config.ml
• avoid the need for Key.v / Key.abstract calls and passing stuff to foreign

The question remains: how are arguments evaluated, and how does every piece of code gets their values for the keys? I tried to figure out how this is done today (both at runtime, but also what is needed / collected at configure time to run this). My current assumption is that in mirage-runtime (or functoria-runtime) we can have a "register_key : 'a key -> unit" and a "with_argv" (as we have now) -- where the former appends to a mutable list (expected to be called from the top-level only), and the latter to apply the argument vector to all keys in that list. likely some keys (such as of the network stack, e.g. parametrized by the stack name) will end up in main () -- but the "with_argv" will only be evaluated when "start" / "run" is invoked (i.e. after top-level initialization).

Key confusion / missing arguments / too many arguments will be discovered at that time only, but that's fine.

[samoht]

Another thing we will loose is a central place to define runtime arguments - we will need some standard CLI argument packages. For instance, to define an IP address - should this live in mirage-tcpip, mirage-tcpip.cli, or somewhere else? (it's a good thing if those are moved closer to where they are defined, so -- as you said -- we could then use richer data types to define those)

[hannesm]

What about in ipaddr.cli? Then they'd be usable outside of a mirage context as well (without depending on tcpip).

[samoht]

Also: do you want the values of the configure keys to be available at runtime? Or are you ok to loose this as well? For instance, are you ok with Key_gen.target disappearing?

[hannesm]

Yes, I'm fine with no target information inside of the binary itself.

[samoht]

Ok looking at the code again, there are various things which are more or less easy to do.

As a first step, I've extended the type of keys to expose if they are used at runtime or configure time; that'll make sure we are not using "both" kinds of keys in various places without breaking anything; I have a small patch doing this -- it's not complete, but it seems ok/doable. Then, need to map runtime keys to cmdliner terms directly. Then remove all the dead-code. All of the machinery to register runtime keys is already there, so it's not too difficult to do (I'll have a shot at completing my first patch later today).

I'm not fully convinced that we are actually loosing complexity yet, but let's try it out and see how it goes :-)

[samoht]

To come back to the initial question:

So, codewise I don't quite understand functoria enough to deal with this, but my first question is: are there any real use-cases for having staged keys?

The main use-case for staged keys is to define parameters for devices. I agree that we somehow have 2 layers (configure and runtime keys) but we need to be able to talk about runtime keys when we define the parameters of the devices. For instance, we need to say how the tcpip stack will get its runtime parameter (ie. what runtime keys will be needed, how are they called and what type do they have).

The second question is: can we disentangle these keys (and (a) remove key_gen.ml,

We need a place to generate the code for the runtime keys we talked about just above. Ie. we need to generate the runtime Cmdliner term and register it so the device could read the right parameter from the command line.

(b) the whole "defined stuff in config.ml which belongs to unikernel.ml",

In #1437 we can now define runtime key in unikernel.ml -- but we still need the ability to talk about runtime key to define runtime parameters of devices (as explained above). As you can create new devices in config.ml, I think you still need to define new runtime keys there.

But maybe what you are saying is that we should disallow adding keys to foreign -- as they should be added to their device directly or define in unikernel.ml ?

and (c) even the fork & modification of the cmdliner API

This is now done in #1437

(d) unclutter the manpage)? What do you think?

Do you mean not displaying configuration keys in the manpage? I think they are not (only Both and Runtime are if I remember properly).

[samoht]

Also: #1442 is removing the ability to have keys that are both runtime and configure time, and also force unikernel writer to define their global runtime keys in unikernel.ml. Let me know if that's what you had in mind.

The only thing which is remaining is removing key_gen.ml -- if that really matters to you we could always generate this glue code in main.ml :p

[hannesm]

What I just remembered is that muen doesn't do boot arguments, so there's only "configuration-time". It is still unclear to me whether this is worth the effort/complexity, or whether we want to go a different path for muen...

[samoht]

For muen, could we pass these arguments on a (default) block device somehow?

[hannesm]

since there's no solo5-block-muen, the answer is no. but maybe we should discuss this with adrian/reto in a separate issue?

[hannesm]

see #1459, it is actually possible to define boot parameters in the policy for muen.