Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use jitpack jars and publish shadow jar #736

Merged
merged 2 commits into from
Apr 30, 2024
+205 −725
Merged

Use jitpack jars and publish shadow jar #736

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4c5301f
Use jitpack jars and publish shadow jar
MrCreosote Apr 8, 2024
00969f8
Improve release notes re shadow jar
MrCreosote Apr 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 66 additions & 68 deletions build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,19 @@ plugins {
id 'java'
id 'war'
id 'jacoco'
id 'maven-publish'
id 'org.ajoberstar.grgit' version '4.1.1'
id 'com.github.johnrengelman.shadow' version '8.1.1'
}

group = 'com.github.kbase'

var VER_JAVA_COMMON = "0.3.0"
var VER_AUTH2_CLIENT = "0.5.0"
var VER_AUTH2_SHADOW = "0.7.1"
var VER_JACKSON = "2.9.9"


var DEFAULT_URL = "https://ci.kbase.us/services/ws"

var BUILD_DOC_ROOT = "$buildDir/docs/"
Expand All @@ -32,6 +41,14 @@ var LOC_DOC_HTML = "$rootDir/docshtml"

repositories {
mavenCentral()
maven {
name = "Jitpack"
url = 'https://jitpack.io'
}
maven {
name = "Clojars"
url = "https://repo.clojars.org/"
}
}

task buildGitCommitFile {
Expand All @@ -55,19 +72,15 @@ javadoc {
* building the client javadocs. If we ever need the full javadocs make a new task for the
* client java docs
*/
/* TODO DOCS currently java-common and auth links don't work. ROI for fixing them is low.
* Ideally, in the future make github actions publish docs on release and ref them
* here.
* https://github.com/marketplace/actions/deploy-publish-javadoc
* https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-gradle-registry
*/
/* TODO DOCS the current sdk documenation looks like invalid html to javadoc
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fingers crossed this will be removed soon anyway, but there's a T missing in documentation

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

* need to go through and remove
* also some spots with < / > that need to be wrapped with {@code } in internal code
*/
failOnError = false
options {
links "https://docs.oracle.com/en/java/javase/11/docs/api/"
links "https://javadoc.jitpack.io/com/github/kbase/auth2_client_java/$VER_AUTH2_CLIENT/javadoc/"
links "https://javadoc.jitpack.io/com/github/kbase/java_common/$VER_JAVA_COMMON/javadoc/"
}
include "**/workspace/*.java"
exclude "**/workspace/WorkspaceServer.java"
Expand Down Expand Up @@ -163,12 +176,16 @@ shadowJar {
// Be careful when updating jars - you may want to set the duplicates strategy to WARN
// to see if any of the jars are shadowing the others when building the fat jar, which
// has been the case in the past
dependsOn buildDocs
// Can't include the full documentation as jitpack doesn't have sphinx installed
dependsOn javadoc
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
archiveAppendix = 'test-shadow'
from sourceSets.test.output
// we don't want to package the auth shadow jar, so no test runtime classpath
configurations = [project.configurations.runtimeClasspath]

// don't shadow the shadow jar, don't need it and it's big
dependencies {
exclude(dependency("com.github.kbase:auth2:$VER_AUTH2_SHADOW"))
}

enableRelocation true
relocationPrefix 'us.kbase.workspace.shadow'
Expand All @@ -186,6 +203,19 @@ war {
from(BUILD_OTHER_DOC_DIR) { into "/WEB-INF/classes/$IN_JAR_DOC_DIR" }
}

// This is a filthy hack to get jitpack.io to not try to build the war file.
// Since the war depends on buildDocs which depends on sphinx, the build will fail
// It'd be better if it didn't call assemble and just publishToMavenLocal,
// but no such luck
gradle.taskGraph.whenReady { taskGraph ->
def tasks = taskGraph.getAllTasks()
if (tasks.find {it.name == 'assemble'}) {
tasks.findAll {it.name == 'war' || it.name == "buildDocs"}.each { task ->
task.enabled = false
}
}
}

/* SDK compile notes:
* kb-sdk starts a docker container in interactive mode. Gradle's commandLine doesn't allocate
* a tty so the command fails.
Expand Down Expand Up @@ -291,49 +321,38 @@ configurations {
testimpl.extendsFrom testImplementation
}

def fromURL = { url, name ->
File file = new File("$buildDir/download/${name}.jar")
file.parentFile.mkdirs()
if (!file.exists()) {
new URL(url).withInputStream { downloadStream ->
file.withOutputStream { fileOut ->
fileOut << downloadStream
}
publishing {
publications {
shadow(MavenPublication) { publication ->
project.shadow.component(publication)
artifactId = "workspace_deluxe-test-shadow-all"
}
}
files(file.absolutePath)
}

dependencies {

// ### General application dependencies ###

implementation fromURL(
'https://github.com/kbase/jars/raw/master/lib/jars/kbase/handle/AbstractHandleClient-1.0.0.jar',
'AbstractHandleClient-1.0.0'
)
implementation fromURL(
'https://github.com/kbase/jars/raw/master/lib/jars/kbase/auth/kbase-auth-0.4.4.jar',
'kbase-auth-0.4.4'
)
implementation fromURL(
'https://github.com/kbase/jars/raw/master/lib/jars/kbase/kidl/kbase-kidl-parser-1409261812-7863aef.jar',
'kbase-kidl-parser-1409261812-7863aef'
)
implementation fromURL(
'https://github.com/kbase/jars/raw/master/lib/jars/kbase/sample/SampleServiceClient-0.1.1.jar',
'SampleServiceClient-0.1.1'
)
implementation 'ch.qos.logback:logback-classic:1.1.2'
implementation 'com.google.guava:guava:14.0.1'
implementation 'com.github.ben-manes.caffeine:caffeine:2.9.3'
implementation "com.github.kbase:auth2_client_java:$VER_AUTH2_CLIENT"
implementation("com.github.kbase:handle_service2:1.0.6") {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exciting!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IKR?

exclude group: 'net.java.dev.jna' // breaks shadow jar
}
implementation "com.github.kbase:java_kidl:0.1.0"
implementation("com.github.kbase:sample_service:0.2.6") {
exclude group: 'net.java.dev.jna' // breaks shadow jar
}
implementation "com.github.kbase:shock_java_client:0.2.0"
implementation 'com.google.guava:guava:14.0.1'
implementation 'commons-codec:commons-codec:1.8'
implementation 'commons-io:commons-io:2.4'
implementation 'com.fasterxml.jackson.core:jackson-annotations:2.9.9'
implementation 'com.fasterxml.jackson.core:jackson-core:2.9.9'
implementation 'com.fasterxml.jackson.core:jackson-databind:2.9.9'
implementation "com.fasterxml.jackson.core:jackson-annotations:$VER_JACKSON"
implementation "com.fasterxml.jackson.core:jackson-databind:$VER_JACKSON"
implementation 'info.picocli:picocli:4.6.1'
implementation 'org.apache.commons:commons-lang3:3.1'
implementation 'org.apache.httpcomponents:httpclient:4.5.9'
implementation 'org.apache.kafka:kafka-clients:2.1.0'
implementation 'org.ini4j:ini4j:0.5.2'
implementation 'org.mongodb:bson:4.11.1'
Expand All @@ -342,34 +361,18 @@ dependencies {
implementation 'org.mongodb:mongodb-driver-sync:4.11.1'
implementation 'org.slf4j:slf4j-api:1.7.30'

// ### Server dependencies, specifically for java_common JsonServerServlet ###
// ### Server dependencies for java_common JsonServerServlet & WorkspaceServer ###

implementation fromURL(
'https://github.com/kbase/jars/raw/master/lib/jars/kbase/common/kbase-common-0.2.0.jar',
'kbase-common-0.2.0'
)
// Pull from jars repo vs a maven repository to avoid the JNA dependency
// TODO DEPS Need to rework the java common logger to not use syslog4j at all since it's abandonware
// and has a ton of CVEs, even in the newer versions.
implementation fromURL(
'https://github.com/kbase/jars/raw/master/lib/jars/syslog4j/syslog4j-0.9.46.jar',
'syslog4j-0.9.46'
)
implementation("com.github.kbase:java_common:$VER_JAVA_COMMON") {
exclude group: 'net.java.dev.jna' // breaks shadow jar
}
// TODO DEPS Need to rework the java common logger to not use syslog4j at all since it's
// abandonware and has a ton of CVEs, even in the newer versions.
implementation "org.syslog4j:syslog4j:0.9.46"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AJ wins

implementation 'javax.annotation:javax.annotation-api:1.3.2'
implementation 'javax.servlet:servlet-api:2.5'
// joda-time is required for kbase-common and syslog4j
implementation 'joda-time:joda-time:2.2'
// this is OOOOOOLD. But that probably means updating java_common
implementation 'org.eclipse.jetty.aggregate:jetty-all:7.0.0.v20091005'

// ### Blobstore / Shock client and dependencies ###

implementation fromURL(
'https://github.com/kbase/jars/raw/master/lib/jars/kbase/shock/shock-client-0.1.0.jar',
'shock-client-0.1.0'
)
implementation 'org.apache.httpcomponents:httpclient:4.5.9'
implementation 'org.apache.httpcomponents:httpmime:4.5.8'

// ### Amazon S3 ###

Expand All @@ -381,22 +384,17 @@ dependencies {

// ### Test ###

testImplementation fromURL(
'https://github.com/kbase/jars/raw/master/lib/jars/kbase/auth2/kbase-auth2-test-shadow-all-0.7.0.jar',
'kbase-auth2-test-shadow-all-0.7.0'
)
testImplementation 'com.arangodb:arangodb-java-driver:6.7.2'
testImplementation 'com.github.kbase:java_test_utilities:0.1.0'
testImplementation "com.github.kbase:auth2:$VER_AUTH2_SHADOW"
testImplementation 'com.github.zafarkhaja:java-semver:0.9.0'
testImplementation 'junit:junit:4.12'
testImplementation 'nl.jqno.equalsverifier:equalsverifier:3.1.10'
testImplementation 'org.hamcrest:hamcrest-core:1.3'
testImplementation 'org.mockito:mockito-core:3.0.0'

}

task showTestClassPath {
doLast {
configurations.testimpl.each { println it }
}
}

48 changes: 24 additions & 24 deletions docsource/buildinitclient.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,22 +48,25 @@ Build the client::

The client jar is created in ``build/libs/workspace_deluxe-client.jar``.

For simplicity, copy the required jars into a single directory. You will also need the
`Jackson <https://github.com/FasterXML/jackson/>`_
annotations, core, and databind jars, which can be downloaded from maven or added to your build
tool::
For simplicity, copy the required jars into a single directory. You will also need the following
jars, which can be downloaded from a maven repository or https://jitpack.io:

* The `Jackson <https://github.com/FasterXML/jackson/>`_ annotations, core, and databind jars
(maven)
* The javax annotation api jar (maven)
* The `KBase auth client jar <https://github.com/kbase/auth2_client_java/>`_
* The `KBase java_common jar <https://github.com/kbase/java_common/>`_

::

bareubuntu@bu:~/ws$ mkdir tryjavaclient
bareubuntu@bu:~/ws$ cd tryjavaclient/
bareubuntu@bu:~/ws/tryjavaclient$ cp ../workspace_deluxe/build/libs/workspace_deluxe-client.jar .
bareubuntu@bu:~/ws/tryjavaclient$ cp ../workspace_deluxe/build/download/kbase-auth-0.4.4.jar .
bareubuntu@bu:~/ws/tryjavaclient$ cp ../workspace_deluxe/build/download/kbase-common-0.2.0.jar .
bareubuntu@bu:~/ws/tryjavaclient$ ls

jackson-annotations-2.9.9.jar kbase-auth-0.4.4.jar
jackson-core-2.9.9.jar kbase-common-0.2.0.jar
jackson-databind-2.9.9.jar workspace_deluxe-client.jar

auth2_client_java-0.5.0.jar java_common-0.3.0.jar
jackson-annotations-2.9.9.jar javax.annotation-api-1.3.2.jar
jackson-core-2.9.9.jar workspace_deluxe-client.jar
jackson-databind-2.9.9.jar

When creating an application using the WSS it's advisable to use a build tool
like ``ant``, ``maven``, or ``gradle`` to organize the required jars.
Comment on lines 71 to 72
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be updated to just recommend gradle?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, maven should work with jitpack, and you can download the jars for ant or use ivy

Expand All @@ -74,26 +77,23 @@ This simple program initializes and calls a method on the WSS client::

.. code-block:: java

import java.net.URI;
import java.net.URL;
import us.kbase.auth.AuthConfig;
import us.kbase.workspace.WorkspaceClient;
import us.kbase.auth.ConfigurableAuthService;
import us.kbase.auth.AuthToken;
import us.kbase.auth.client.AuthClient;
import us.kbase.workspace.WorkspaceClient;

public class TryWorkspaceClient {

public static void main(String[] args) throws Exception {
String authUrl =
"https://ci.kbase.us/services/auth/api/legacy/KBase/Sessions/Login/";

ConfigurableAuthService authService = new ConfigurableAuthService(
new AuthConfig().withKBaseAuthServerURL(new URL(authUrl));
final String authUrl = "https://appdev.kbase.us/services/auth/";
final AuthClient authcli = AuthClient.from(new URI(authUrl));
Comment on lines +89 to +90
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why switch from CI to appdev?

Copy link
Member Author

@MrCreosote MrCreosote Apr 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this documentation is targeted at 3rd app developers as well as kbase developers and they're much more likely to have an appdev/prod account than a ci account


String tokenString = YOUR_AUTH_TOKEN_HERE;
AuthToken token = authService.validateToken(tokenString);
final String tokenString = args[0];
final AuthToken token = authcli.validateToken(tokenString);

WorkspaceClient client = new WorkspaceClient(
new URL("https://ci.kbase.us/services/ws/"),
final WorkspaceClient client = new WorkspaceClient(
new URL("https://appdev.kbase.us/services/ws/"),
token);
System.out.println(client.ver());
}
Expand All @@ -102,7 +102,7 @@ This simple program initializes and calls a method on the WSS client::
Compile and run::

bareubuntu@bu:~/ws/tryjavaclient$ javac -cp "./*" TryWorkspaceClient.java
bareubuntu@bu:~/ws/tryjavaclient$ java -cp "./:./*" TryWorkspaceClient
bareubuntu@bu:~/ws/tryjavaclient$ java -cp "./:./*" TryWorkspaceClient $KBASE_TOKEN
0.14.2

For more client initialization and configuration options, see :ref:`apidocs`.
Expand Down
2 changes: 1 addition & 1 deletion docsource/developers.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Branches:

Recompiling the generated code
------------------------------
To compile, run ``./gradlew compile``.
To compile the Workspace generated code from ``workspace.spec``, run ``./gradlew compile``.

Note that:

Expand Down
1 change: 1 addition & 0 deletions docsource/releasenotes.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ UPDATES:
against Mongo 7.
* Gradle has replaced Ant as the build tool. As a consequence, all the built artifacts are now
located in the build directory, including the ``update_workspace_database_schema`` script.
* A shadow jar is published on jitpack.io for supporting tests in other repos.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a short explanation of what a shadow jar is so people can get the gist without having to google it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added



VERSION: 0.14.2 (Released 11/9/2023)
Expand Down
Loading
Loading