Document Spring recommendations/restrictions for Java 9 module setups [SPR-14579]

[spring-projects-issues]

Paul Bakker opened SPR-14579 and commented

As part of the work I'm doing for the upcoming Java 9 book for O'Reilly I'm experimenting with migration scenarios towards Java 9 modules.

The scenario is the following. The user migrated her code to a module (meaning that a module-info.java was added). The code relies on a Spring version which is not yet Java 9 compatible. This should't be a problem, because existing libraries should be able to work as "automatic modules". This works well with Spring, up until the point that namespaces in the xml configuration are used.

<context:component-scan base-package="org.example"/>

This results in the following error:

Exception in thread "main" org.springframework.beans.factory.parsing.BeanDefinitionParsingException: Configuration problem: Unable to locate Spring NamespaceHandler for XML schema namespace [http://www.springframework.org/schema/context]
Offending resource: class path resource [services.xml]

	at org.springframework.beans.factory.parsing.FailFastProblemReporter.error([email protected]/FailFastProblemReporter.java:70)
	at org.springframework.beans.factory.parsing.ReaderContext.error([email protected]/ReaderContext.java:85)
	at org.springframework.beans.factory.parsing.ReaderContext.error([email protected]/ReaderContext.java:80)
	at org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.error([email protected]/BeanDefinitionParserDelegate.java:301)
	at org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.parseCustomElement([email protected]/BeanDefinitionParserDelegate.java:1408)
	at org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.parseCustomElement([email protected]/BeanDefinitionParserDelegate.java:1401)
	at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.parseBeanDefinitions([email protected]/DefaultBeanDefinitionDocumentReader.java:172)
	at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.doRegisterBeanDefinitions([email protected]/DefaultBeanDefinitionDocumentReader.java:142)
	at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.registerBeanDefinitions([email protected]/DefaultBeanDefinitionDocumentReader.java:94)
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.registerBeanDefinitions([email protected]/XmlBeanDefinitionReader.java:508)
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions([email protected]/XmlBeanDefinitionReader.java:392)
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions([email protected]/XmlBeanDefinitionReader.java:336)
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions([email protected]/XmlBeanDefinitionReader.java:304)
	at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions([email protected]/AbstractBeanDefinitionReader.java:181)
	at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions([email protected]/AbstractBeanDefinitionReader.java:217)
	at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions([email protected]/AbstractBeanDefinitionReader.java:188)
	at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions([email protected]/AbstractBeanDefinitionReader.java:252)
	at org.springframework.context.support.AbstractXmlApplicationContext.loadBeanDefinitions([email protected]/AbstractXmlApplicationContext.java:127)
	at org.springframework.context.support.AbstractXmlApplicationContext.loadBeanDefinitions([email protected]/AbstractXmlApplicationContext.java:93)
	at org.springframework.context.support.AbstractRefreshableApplicationContext.refreshBeanFactory([email protected]/AbstractRefreshableApplicationContext.java:129)
	at org.springframework.context.support.AbstractApplicationContext.obtainFreshBeanFactory([email protected]/AbstractApplicationContext.java:612)
	at org.springframework.context.support.AbstractApplicationContext.refresh([email protected]/AbstractApplicationContext.java:513)
	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>([email protected]/ClassPathXmlApplicationContext.java:139)
	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>([email protected]/ClassPathXmlApplicationContext.java:93)
	at demo.bootstrapper.Main.setup(bootstrap/Main.java:11)
	at demo.bootstrapper.Main.main(bootstrap/Main.java:16)

I'm not suggesting this is an error in Spring, or that this must be fixed, but hopefully it's useful information for compatibility issues in the future.

A (pretty bad) workaround is to extract the META-INF folder from the JAR file and put that on the classpath.

---

Affects: 5.0 M1

Issue Links:

• Declare Spring modules with JDK 9 module metadata [SPR-13501] #18079 Declare Spring modules with JDK 9 module metadata
• Stable module names for Spring Framework jars on JDK 9 module path [SPR-13716] #18289 Stable module names for Spring Framework jars on JDK 9 module path
• DATACMNS-1033 Assert Java 9 compatibility
• Compatibility with merged JDK 9 mainline [SPR-13344] #17928 Compatibility with merged JDK 9 mainline
• Compatibility with JDK 9 GA [SPR-15992] #18941 Compatibility with JDK 9 GA

Referenced from: commits 6ef7dd4

0 votes, 9 watchers

[spring-projects-issues]

Juergen Hoeller commented

Hey Paul,

We are tracking JDK 9 compatibility for Spring 5: As of 5.0 M1, the test suite passes on JDK 9 in classpath mode (#17928). For use on the module path, we attempt to sort out the automatic modules option for 5.0 M3 (#18289).

As for finding the schemas, I suppose this only applies when actually using XML bean definitions? That is, not when using configuration classes or plain programmatic registration?

Also, I recall some recent discussions on the Jigsaw mailing list about the exposure of resources. This might be something worth addressing there, asking for default exposure of classpath resources from automatic modules.

[spring-projects-issues]

Paul Bakker commented

That's correct. The issue is related to the changes to resource loading (http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2015-October/000163.html). The "getResource*" methods don't find resources in modules. A new reflection API is available to iterate over modules and a module's resources.
I will add other issues I run into as comments to #18289 to keep track of them.

[spring-projects-issues]

Juergen Hoeller commented

Mark Reinhold's latest proposal suggests getting rid of resource encapsulation altogether: http://openjdk.java.net/projects/jigsaw/spec/issues/#ResourceEncapsulation. And as far as I remember, the idea was that automatic modules would see all resources in any case. This doesn't seem to be available in the mainline yet but is apparently changed in the Jigsaw branch: https://jdk9.java.net/jigsaw/.

[spring-projects-issues]

Paul Bakker commented

If I'm not mistaken (I'm just trying to get through the Spring codebase) the same problem exists for component scanning. E.g. a component scan like the one below ultimately uses ClassLoader.getResources, which doesn't currently return resources in named modules (in this case class files).

<context:component-scan base-package="org.example"/>

If this requirement is dropped as being discussed on the mailing list that should work again as well I assume.

[spring-projects-issues]

Paul Bakker commented

Hi Jurgen,

I retested with a Java 9 build that includes the dropped resource constraints things look a lot better!
The reported issue with spring.schemas is fixed, and component scanning works, but with a side note.

If I create a "modular JAR" containing the service that needs to be picked up by the component scan, it works. If I run in "exploded" mode however, it doesn't.
Looking at the code I believe this is because in exploded mode there's an extra directory (e.g. modulename/a/b, instead of a/b) that the code currently doesn't understand.

In case you're interested in further debugging, I pushed the setup to a git repo: https://github.com/paulbakker/spring-java9.

[spring-projects-issues]

Juergen Hoeller commented

Thanks for your efforts, Paul! That's great to hear, and I hope it'll get merged into the JDK 9 mainline soon.

I wasn't aware of the "exploded" mode at all. So this means that ClassLoader.getResources("mybasepackage") fails against such a mymodule/mybasepackage directory layout? I would actually expect the ClassLoader implementation to transparently resolve that... Anyway, if this won't get addressed in the JDK itself, we'll do something about it in Spring's classpath scanning algorithm.

[spring-projects-issues]

Juergen Hoeller commented

Paul Bakker, since we're about to revisit this from our side: Have you had any success with post-August JDK 9 builds? We're still waiting for CGLIB 3.2.5 for a JDK 9 compatible defineClass call, so any CGLIB-related features are expected to break against JDK 9 build 148+ at the moment... I'm rather wondering whether you attempted to run in modular JAR mode and/or "exploded" mode after the resource revision got merged into the JDK 9 mainline?

[spring-projects-issues]

Paul Bakker commented

Newer builds are a lot better when it comes to framework compatibility. See this Devoxx presentation (second part) about migrating a (old fashioned) Spring/Hibernate application. You can find the example source code here: https://github.com/sandermak/java9-migration-demos/tree/master/4_ModularSpring

[spring-projects-issues]

Juergen Hoeller commented

As far as my research goes, the problem is that ClassLoader.getResource("mypackageroot") simply returns null against an exploded module layout, whereas it properly returns a URL for the base package in all other scenarios... including module jars on the module path.

I've reported this to the jigsaw-dev list since there is nothing we can do about it from our end.

[spring-projects-issues]

Juergen Hoeller commented

Paul Bakker, it looks your sample application from back in November got away with just declaring requires spring.context in its main module. Against current JDK 9 builds, this doesn't seem to be sufficient anymore: main needs to explicitly declare requires spring.beans, requires spring.core etc as well, since they don't come transitively with spring-context (when deployed as an automatic module). This kind of makes sense semantically, even if it is quite a nuisance in practice.

[spring-projects-issues]

Paul Bakker commented

I've recently given an updated version of this talk again at DevoxxUS: https://www.youtube.com/watch?v=TEoexFsDP6A.
The example project repo is also updated: https://github.com/java9-modularity/java9-migration-demos

tl;dr: Spring is doing fine as automatic modules. No changes to how transitive dependencies work either.

Any progress with moving to named modules?

[spring-projects-issues]

Juergen Hoeller commented

Thanks for the update, that's good to hear!

As for explicit module declarations, we're tracking this in #18079. Unfortunately, according to the latest discussions around Jigsaw and Mark Reinhold's recent statements, it is absolutely not recommended to bake automatic module names for third-party dependencies into higher-level module descriptors. This means that we have to wait for all of our (optional) dependencies to ship module descriptors first, which single-handedly moves our target date to 2019+, possibly into Spring 6 :-(

For the time being, we'll have to keep focusing on the use of our framework jars as automatic modules. We might ship some aggregator modules with Jigsaw metadata (i.e. empty jars with just a module-info descriptor which aggregate a selection of automatic framework modules, including transitive dependencies, and third-party modules)... anything that helps early adopters who try to use Jigsaw idiomatically.

[spring-projects-issues]

Paul Bakker commented

Might be worth sharing that feedback on the mailing list. I understand the fundamental point they are making, but making it almost impossible to migrate frameworks to modules is a serious problem.

[spring-projects-issues]

Juergen Hoeller commented

Good point - done :-)

We are nevertheless supportive of Jigsaw even in Spring 5.x, just with a focus on automatic modules for the time being. I guess Spring 6 will raise its system requirements to JDK 9+ in any case, so a focus on proper module descriptors would go along nicely.

[spring-projects-issues]

Paul Bakker commented

Concrats on the first Spring 5 release. Recently the sentiment about automatic modules and releasing modules that use automatic modules seems to have changed a little. Would it be worth looking into writing module-infos for Spring 5 as well?

[spring-projects-issues]

Juergen Hoeller commented

The sentiment around automatic modules has certainly hardened. At least we have a mechanism for defining automatic module names without full descriptors now. However, we are still stuck with respect to explicit module-info descriptors: We cannot ship dependency-declaring descriptors of our own before the maintainers of all of our third-party dependencies agree on module names. We are at the end of the open source consumption chain, so we cannot be first movers in that respect.

The "Automatic-Module-Name" manifest entry might be an easy enough technical way out for the naming problem... but even that has to be adopted broadly first. Frankly, I don't see a well-established module namespace in the open source ecosystem so quickly. I still don't expect all of our dependencies to arrive there before 2019. At the Java EE level, the umbrella expert group has already clarified that they are not going to support official module name declarations for their APIs before EE 9, silencing attempts to do this e.g. for JPA. And with yesterday's rejection of the current Jigsaw proposal in the EC and the propaganda around it, it's even less likely that there will be broad early adoption. Even revisiting this with Spring 6, we might still find a rather incomplete ecosystem then.

So for the time being, all we can sensibly do is to focus on usage as automatic modules. I recently re-tested against JDK 9 build 167 and was quite pleased with the outcome, since even transitive resolution of automatic module dependencies works seamlessly now. For Spring applications, the experience is quite alright that way: e.g. requires spring.context is all it takes, with spring.core coming in transparently then, and our framework modules automatically seeing their optional dependencies when present. I even still like the filename-derived module names for our jars and consider defining those as our official Jigsaw module namespace. Ironically, the application development experience with Spring on Jigsaw won't get much better even once we ship module-info descriptors...

[spring-projects-issues]

Paul Bakker commented

I just downloaded Spring Core RC2 and noticed that there is no Automatic-Module-Name set in the manifest. It's probably a good idea to add that for this release (for all modules), until a future release adds a module-info.java.
The recommendation is now to use a fully qualified name, so probably something like: org.springframework.core, org.springframework.orm etc.

[spring-projects-issues]

Juergen Hoeller commented

Indeed, and we intend to do so as of 5.0 RC3, tracked through (the slightly repurposed) ticket #18289.

However, I'm still in favor of spring.core style module names, aligning with the original Jigsaw naming style in the JDK more than with the Stephen Colebourne driven reverse domain style. From my perspective, just because there are two Java frameworks named Spark out there doesn't meant that everybody has to use fully qualified domain names. Quite a few common libraries do not even use reverse domain names in their packages these days (e.g. akka., okhttp3.), so reverse domain names won't be the only policy out there in any case.

For us, quite a few of our jars do not have a single root package but rather aggregate several packages, so a reverse domain name would arguably suggest the false promise of a single contained package of the same name. A distinct module name space just like in the JDK arguably represents our arrangement much better, using the power of modules as fine-tuned aggregates of related functionality and not just as rather dumb units of deployment per root package. FWIW, several Java EE API jars intend to adopt the JDK naming style as well.

[spring-projects-issues]

Philippe Marschall commented

FWIW, several Java EE API jars intend to adopt the JDK naming style as well.

AFAIK everything under the JCP should do so, I would expect every Java EE API to follow this style.

[spring-projects-issues]

Juergen Hoeller commented

I've added a few further notes to applicable places in the reference documentation, with respect to component scanning as well as resource pattern matching. Since such arrangements effectively work fine on Jigsaw as well, with largely the same limitations as in portable classpath setups (in particular avoiding searches in jar file roots, putting resources into a directory instead), those notes are not too extensive and nicely fold into the general storyline of those chapters.

I've also added a note on exports vs opens for component classes that Spring is supposed to manage, depending on whether non-public members need to be invoked.