-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Infinispan support for distributed in-memory HttpSession caching #2843
Comments
Draft from @nmittles: Title: Infinispan Support for Distributed In-memory HttpSession CachingSummary: A key component of production-grade web applications is failover of non-persistent user session data (e.g. a shopping cart). In Open Liberty, the Prereq: Obtain the Infinispan 11 library Jars for running in client/server mode with MavenTo download the Infinispan jars and corresponding prerequisite jars for client/server mode use the following <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>io.openliberty</groupId>
<artifactId>openliberty-infinispan-client</artifactId>
<version>1.0</version>
<!-- https://mvnrepository.com/artifact/org.infinispan/infinispan-jcache-remote -->
<dependencies>
<dependency>
<groupId>org.infinispan</groupId>
<artifactId>infinispan-jcache-remote</artifactId>
<version>11.0.3.Final</version>
</dependency>
</dependencies>
</project> Commands to download and cleanup the jars:
Prereq: Obtain the Infinispan 11 library Jars for running in embedded mode with MavenTo download the Infinispan jars and corresponding prerequisite jars for embedded mode use the following <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>io.openliberty</groupId>
<artifactId>openliberty-infinispan</artifactId>
<version>1.0</version>
<!-- https://mvnrepository.com/artifact/org.infinispan/infinispan-jcache -->
<dependencies>
<dependency>
<groupId>org.infinispan</groupId>
<artifactId>infinispan-jcache</artifactId>
<version>11.0.3.Final</version>
</dependency>
</dependencies>
</project> Commands to download and cleanup the jars:
Configuring Infinispan in client/server mode with OpenShiftInfinispan can be configured in client/server mode. In this configuration, there is a standalone Infinispan server that houses all of the cache data. Each Open Liberty server then remotely connects to this Infinispan server to access an application's cached session data. Keep in mind that a cluster of Infinispan servers can be configured to share the workload and for failover purposes. This client/server mode can be beneficial in environments in which the Open Liberty servers are frequently starting and stopping because there is a cost associated with transferring the cache state to each Liberty server when running in embedded mode. <server>
<featureManager>
<feature>servlet-4.0</feature>
<feature>sessionCache-1.0</feature>
</featureManager>
<httpEndpoint host="*"
id="defaultHttpEndpoint"
httpPort="9080"
httpsPort="9443" />
<httpSessionCache libraryRef="InfinispanLib">
<properties infinispan.client.hotrod.server_list="infinispan-server:11222"/>
<properties infinispan.client.hotrod.auth_username="sampleUser"/>
<properties infinispan.client.hotrod.auth_password="samplePassword"/>
<properties infinispan.client.hotrod.auth_realm="default"/>
<properties infinispan.client.hotrod.sasl_mechanism="PLAIN"/>
<properties infinispan.client.hotrod.java_serial_whitelist=".*"/>
<properties infinispan.client.hotrod.marshaller="org.infinispan.commons.marshall.JavaSerializationMarshaller"/>
</httpSessionCache>
<library id="InfinispanLib">
<fileset dir="${shared.resource.dir}/infinispan" includes="*.jar"/>
</library>
</server> The key piece is the
In the example above Configuring Infinispan in embedded mode with OpenShiftInfinispan can be configured in embedded mode. In this configuration, the Infinispan server resides in the same JVM as the Open Liberty server. Infinispan utilizes JGroups to allow multiple embedded servers to form a cluster. In our context of the <server>
<featureManager>
<feature>servlet-4.0</feature>
<feature>mpMetrics-2.0</feature> <!-- one of the Infinispan JARs has a hard dependency on MicroProfile Metrics API -->
<feature>mpReactiveStreams-1.0</feature> <!-- one of the Infinispan JARs has a hard dependency on Reactive Streams API -->
<feature>sessionCache-1.0</feature>
</featureManager>
<httpEndpoint host="*"
id="defaultHttpEndpoint"
httpPort="9080"
httpsPort="9443" />
<httpSessionCache libraryRef="InfinispanLib" uri="file:${shared.resource.dir}/infinispan/infinispan.xml"/>
<library id="InfinispanLib">
<fileset dir="${shared.resource.dir}/infinispan" includes="*.jar"/>
</library>
</server> The key piece is the <infinispan>
<jgroups>
<stack-file name="jgroups-kubernetes" path="/default-configs/default-jgroups-kubernetes.xml"/>
</jgroups>
<cache-container>
<transport stack="jgroups-kubernetes" />
</cache-container>
</infinispan> The JGroups stack is what determines how the Infinispan servers form a cluster. The sample
apiVersion: v1
kind: Service
metadata:
name: infinispan-embedded
spec:
clusterIP: None
ports:
- name: discovery
port: 7800
protocol: TCP
targetPort: 7800
selector:
name: ol-runtime-infinispan-embedded
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {} Finally, create a # Set if IPv6 is not supported.
-Djava.net.preferIPv4Stack=true
# This value matches the DNS lookup of the headless service defined in the previous step. The domain might vary based on the OpenShift environment.
-Djgroups.dns.query=infinispan-embedded.myproject.svc.cluster.local For an OpenShift environment, <httpSessionCache libraryRef="InfinispanLib"/> |
@nmittles Please review: https://draft-openlibertyio.mybluemix.net/docs/latest/configuring-infinispan-support.html The draft mentions references to the JCache session persistence feature that will contain the proposed updates related to this topic that I mentioned in issue #1290 |
Feedback from @nmittles in slack: Also for me the sentence is hard to read, as it contains "in...", "that..", "that...", "to...". I'm not sure what the "to maintain high availability of data" adds to the sentence. Is it saying that it provides more flexibility to maintain high availability of data? What is the flexibility? Improved startup time (which is performance and not flexibility)? Or does the "to maintain high availability of data" mean this applies to servers that start and stop regularly for HA purposes, which doesn't make sense but is how it reads. Also the improved startup time is missing context, so it sounds like setting up session cache in this way improved startup performance in general, when I think you meant for it to be in comparison to embedded mode. The whole point of mentioning that the caches need to be synced when servers stop/start was to explain why you would use client/server vs embedded. I'm getting the impression that we're trying to really limit our word count for these docs? As even the sentence before this one is also a mouthful. I find myself rereading the sentence a couple times to unpack it. I know its long, but this is a really key piece to have the label and the selector match up. If this isn't setup right, it's really hard/frustrating to debug. Maybe we can find a way to shorten the label example. Also, throughout the doc I'm thinking mode with OpenShift might sound better as mode on OpenShift. I'll let you decide which you prefer. |
Infinispan doc review II: CURRENT: When you configure Infinispan in client/server mode on OpenShift, you can connect Open Liberty to a remote Infinispan server that accesses cached HTTP session data from applications. PROPOSED: When you configure Infinispan in client/server mode on OpenShift, you can connect Open Liberty to a remote Infinispan server that accesses cached HTTP session application data. (or remove 'application' completely?) CURRENT: This method improves performance by making data synchronization easier in application environments that must facilitate communication among multiple servers. PROPOSED: new section on deciding between client/server and embedded modes. CURRENT: When you configure Infinispan in embedded mode on OpenShift, you can connect to Infinispan servers with Open Liberty that are created in the same Java Virtual Machine (JVM). Infinispan provides access to JGroups, which enables servers that are in embedded in the JVM to form a cluster. This cluster of embedded servers in embedded mode provides high availability of user HTTP session data by performing user requests faster. To configure Infinispan in embedded mode, configure the server.xml file, as shown in the following example: PROPOSED: When you configure Infinispan in embedded mode on OpenShift, an Infinispan server runs in the same Java Virtual Machine (JVM) as Open Liberty. Multiple embedded Infinispan servers can utilize JGroups(http://www.jgroups.org) to form a cluster providing high availability of user HTTP session data across all members. To configure Infinispan in embedded mode, configure the server.xml file, as shown in the following example: CURRENT: After configuring embedded mode, configure the infinispan element in the server.xml file to create an infinispan.xml file: PROPOSED: After configuring embedded mode in the server.xml file, create the referenced infinispan.xml file to enable the JGroups Kubernetes transport stack: CURRENT: The JGroups stack determines how the Infinispan servers form a cluster. The infinispan.xml file uses the default Kubernetes template to enable clustering in OpenShift. For more information, see the JCache Session Persistence feature. PROPOSED: <The wrong link is referenced. Instead it should be "Setting Up Infinispan Clusters" https://infinispan.org/docs/11.0.x/titles/embedding/embedding.html#setting_up_clusters > CURRENT: This label matches the name value of the selector key that is defined in the server.xml file to integrate the application into the service. PROPOSED: This label matches the name value of the selector key that is defined in the service.yaml to integrate the application into the service. |
@nmittles Please review the latest update: https://docs-draft-openlibertyio.mybluemix.net/docs/latest/configuring-infinispan-support.html |
Worked with @Rwalls1 on a few minor tweaks. At this point, the Infinispan configuration doc looks good to me from a technical perspective. |
Content review:This topic looks really good- very clear and easy to follow. I just have a few comments and minor edit suggestions Intro
I think you should make it clear that infinispan is just one option- as stated the user might think you have to use it vs Hazelcast or another.
I'm not sure what the last part means- i think its based on this sentence from the blog post: This client/server mode can be beneficial in environments in which your Open Liberty servers are frequently starting and stopping because there is a cost associated with transferring the cache state to each Liberty server when running in embedded mode. but they don't seem to say the same thing- can you clarify the benefit of client/server? Before you begin
Configuring Infinispan in client/server mode on OpenShift
I think the remote server stores the data, rather than accesses it? The OP says:
Configuring Infinispan in embedded mode on OpenShift
|
@dmuelle Thanks, I've addressed your feedback: https://docs-draft-openlibertyio.mybluemix.net/docs/21.0.0.6/configuring-infinispan-support.html |
Looks good- You might add a heading for |
@dmuelle Thanks, I've made that update. |
@Rwalls1 The topic looks good! A few edit suggestions Peer reviewConfiguring Infinispan as a JCache provider
Before you begin
Configuring Infinispan in client/server mode on OpenShift
Configuring Infinispan in embedded mode on OpenShift
Configuring Infinispan in embedded mode without OpenShift
|
@ManasiGandhi Thanks, I've addressed your feedback: |
Doc has been merged to Vnext and will be published for 21.0.0.6 |
In OpenLiberty we now officially support using Infinispan (version 11) for the JCache provider used by the sessionCache-1.0 feature. We need to document how to configure this for server/client mode, embedded mode, and reference Liberty docker container documentation as well.
Infinispan beta support was already documented in a blog post. We will be reusing and updating this content to build the Liberty doc.
https://openliberty.io/blog/2020/01/08/infinispan-open-liberty-openshift.html
OpenLiberty epic:
OpenLiberty/open-liberty#8835
The text was updated successfully, but these errors were encountered: