Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to read artifacts from file #1331

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

ob
Copy link

@ob ob commented Feb 24, 2025

Add a way to read the artifacts from a file outside MODULE.bazel. Use it like this:

MODULE.bazel

maven = use_extension("@rules_jvm_external//:extensions.bzl", "maven")
maven.install(
    name = "external",
    artifacts_file = "//:external-dependencies.txt",
)

external-dependencies.txt

com.auth0:java-jwt:3.19.2
com.fasterxml.jackson.core:jackson-databind:2.17.2
com.github.ben-manes.caffeine:caffeine:3.1.8
com.google.api.grpc:proto-google-common-protos:2.22.0

@ob ob requested review from jin, shs96c and cheister as code owners February 24, 2025 06:41
Copy link
Collaborator

@shs96c shs96c left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'd originally planned for this to mirror the Gradle format, but a simple file with one entry per line is easy to understand and work with, and amenable to automated tooling, so I like this a lot.

I think that there are two things that would be useful which I know users will be clamouring for:

  1. Support for comments (ideally prefixed with a #)
  2. The ability to put version numbers into a variable and refer to that

Both are additive behaviour, so if we land this "as is", valid config files will continue to be valid, and 2 is a lot of work, but if you can, it'd be lovely to get 1 into this PR. If you'd rather land this as it is, and we can iterate from there, that's fine too :)

The only things I'd really like to see before this lands are:

  1. Workspace support
  2. Support for adding boms too

@@ -42,6 +42,7 @@ install = tag_class(

# Actual artifacts and overrides
"artifacts": attr.string_list(doc = "Maven artifact tuples, in `artifactId:groupId:version` format", allow_empty = True),
"artifacts_file": attr.label(doc = "A file containing a list of artifacts to install, one per line in `artifactId:groupId:version` format", allow_single_file = True),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also have a boms_file too?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


artifacts_file_path = mctx.path(artifacts_file_label)
artifacts_file_content = mctx.read(artifacts_file_path)
artifacts = artifacts_file_content.splitlines()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: do we want people to be able to do things like force the version or mark a dep as testonly from the file? Or do we want to make those dependencies be marked up in the normal way in the module file?

@@ -42,6 +42,7 @@ install = tag_class(

# Actual artifacts and overrides
"artifacts": attr.string_list(doc = "Maven artifact tuples, in `artifactId:groupId:version` format", allow_empty = True),
"artifacts_file": attr.label(doc = "A file containing a list of artifacts to install, one per line in `artifactId:groupId:version` format", allow_single_file = True),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also please add this to the maven_install repo rule too? People are still using that as well

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I'm trying to figure out how to add tests in-repo... if you have pointers I'd appreciate them.

@ob
Copy link
Author

ob commented Feb 24, 2025

We'd originally planned for this to mirror the Gradle format, but a simple file with one entry per line is easy to understand and work with, and amenable to automated tooling, so I like this a lot.

Thanks... does supporting the Gradle format means that we'd be getting most of the info from the file rather than the rule? So it would be either a file XOR fields in the rule? Full disclosure, I'm not super familiar with Java :-)

I think that there are two things that would be useful which I know users will be clamouring for:

  1. Support for comments (ideally prefixed with a #)
  2. The ability to put version numbers into a variable and refer to that

Both are additive behaviour, so if we land this "as is", valid config files will continue to be valid, and 2 is a lot of work, but if you can, it'd be lovely to get 1 into this PR. If you'd rather land this as it is, and we can iterate from there, that's fine too :)

Added these two features.

The only things I'd really like to see before this lands are:

  1. Workspace support
    Done.
  2. Support for adding boms too

Done.

@shs96c
Copy link
Collaborator

shs96c commented Feb 24, 2025

We'd originally planned for this to mirror the Gradle format, but a simple file with one entry per line is easy to understand and work with, and amenable to automated tooling, so I like this a lot.

Thanks... does supporting the Gradle format means that we'd be getting most of the info from the file rather than the rule? So it would be either a file XOR fields in the rule? Full disclosure, I'm not super familiar with Java :-)

The idea would be to make it easier for teams to migrate from Gradle to Bazel, particularly in that awkward phase where there may be both build systems running in parallel. It's not strictly a necessity, and being able to have the list of artifacts outside of the module file is a huge win.

In addition, there's tooling already out there that understands the gradle formatted file, so we'd get things like renovate support more swiftly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants