[Proposal] Support a dependency graph in the packager

[henneberger]

This proposal is to increase the support of shared UDTs and shared libraries.

Dependencies resolved from the repository currently has the limitation that they must be fully self contained: shared libraries must be shaded or they will cause class conflicts at runtime. Therefore, they cannot be shared between packages. We handle jars manually and do not have access to a build system to refactor out common or shared dependencies.

We have a few options:

Maven repository dependency

Rather than including the jar in the package that is sent to the repository, we include a maven reference. We can then invoke gradle at compile-time to download the jars to do the necessary dependency deduplication. Local maven dependencies cannot be supported since we would have issues mapping the local repository into docker/k8s, so only remote dependencies would be supported. If the user uses an institutional maven repository, we would also need to include the ability to configure it in the package.json.

Packages can also have dependency

We manually resolve a dependency graph and assure we only download one copy of all dependencies. We can use semver to determine if dependencies are compatible and issue warnings if they are not.

This is open for comments.