Add Docker image to build natives with older dependencies

[jonahgraham]

To verify locally that this built a suitable eclipse/eclise*.so:

1. Add USER 1000 to end of Dockerfile
2. in dir of Dockerfile docker build --pull -t swtgtk3nativebuild .
3. EQUINOX=/scratch/eclipse/oomph/swt-master/git/equinox
4. LIBRARY_GTK=$EQUINOX/features/org.eclipse.equinox.executable.feature/library/gtk
5. docker run --rm -it -v$EQUINOX:$EQUINOX -w$LIBRARY_GTK swtgtk3nativebuild make -f make_linux.mak
6. make -f make_linux.mak install EXE_OUTPUT_DIR=#### LIB_OUTPUT_DIR=####
7. Test eclipse works
8. run objdump -R on the exe and so to ensure no new glibc symbols

I don't know how to verify this on the build machine. Do we just merge it so that new images are available

Prerequisite of eclipse-equinox/equinox#830

[laeubi]

I don't know how to verify this on the build machine. Do we just merge it so that new images are available

Yes that's basically the way not very comfortable but as we not very often create completely new images it seems acceptable.

[laeubi]

@HannesWell do we need to do anything after this merge to refresh the Jenkins jobs or will it happen automatically?

[HannesWell]

@HannesWell do we need to do anything after this merge to refresh the Jenkins jobs or will it happen automatically?

After merging nothing more has to be done (that's usually only necessary if groovy files are changed that are used by the job-DSL).

I had only a quick look at the phone but most looks sane to me. I assume the new docker image is similar to the existing one with only a few numbers adjusted?
If you want, I can perform a more detailed review this evening.

[jonahgraham]

I assume the new docker image is similar to the existing one with only a few numbers adjusted?

Yes, just changes to debian version, remove gtk4 and java. Here is the diff between the two of the Dockerfiles

$ diff -u swtnativebuild/Dockerfile swtgtk3nativebuild/Dockerfile 
--- swtnativebuild/Dockerfile   2025-02-02 16:10:42.942815876 -0500
+++ swtgtk3nativebuild/Dockerfile       2025-02-03 08:34:27.883882536 -0500
@@ -1,4 +1,4 @@
-FROM debian:12
+FROM debian:10
 
 ### user name recognition at runtime w/ an arbitrary uid - for OpenShift deployments
 COPY scripts/uid_entrypoint /usr/local/bin/uid_entrypoint
@@ -14,10 +14,15 @@
       procps \
       git \
       libgtk-3-dev \
-      libgtk-4-dev \
       freeglut3-dev \
       webkit2gtk-driver \
-      default-jdk
+      curl
+
+
+# Use Java17 from JustJ because Debian 10 comes with JDK11 only
+RUN mkdir -p /usr/lib/jvm/justj-17 && \
+      cd /usr/lib/jvm/justj-17 && \
+      curl "https://download.eclipse.org/justj/jres/17/downloads/20230428_1804/org.eclipse.justj.openjdk.hotspot.jre.minimal.stripped-17.0.7-linux-x86_64.tar.gz" | tar -xzf - include/ lib/
 
 ENV HOME=/home/swtbuild
 ENV DISPLAY :0
@@ -32,6 +37,8 @@
 
 RUN localedef -i en_US -f UTF-8 en_US.UTF-8
 ENV LANG=en_US.UTF-8
-ENV SWT_JAVA_HOME=/usr/lib/jvm/default-java/
+# This is only a partial Java, the headers needed to compile natives
+ENV SWT_JAVA_HOME=/usr/lib/jvm/justj-17
+ENV JAVA_HOME=/usr/lib/jvm/justj-17
 
-USER 10001
\ No newline at end of file
+USER 10001

(This is against latest commit, the USER line was missing from earlier commit)

[HannesWell]

Besides the installation of a JDK which seems unnecessary as explained below I think this is fine.

[jonahgraham]

Besides the installation of a JDK which seems unnecessary as explained below I think this is fine.

I'll remove the JDK - that seems cleanest option.

[HannesWell]

Besides the installation of a JDK which seems unnecessary as explained below I think this is fine.

I'll remove the JDK - that seems cleanest option.

Thanks! I think you can remove curl then as well (not too much of a concern though).

[jonahgraham]

With commit a311eae the diff to gtk4 dockerfile is:

$ diff -u swtnativebuild/Dockerfile swtgtk3nativebuild/Dockerfile 
--- swtnativebuild/Dockerfile   2025-02-03 15:59:40.171522419 -0500
+++ swtgtk3nativebuild/Dockerfile       2025-02-03 16:03:05.376049335 -0500
@@ -1,4 +1,4 @@
-FROM debian:12
+FROM debian:10
 
 ### user name recognition at runtime w/ an arbitrary uid - for OpenShift deployments
 COPY scripts/uid_entrypoint /usr/local/bin/uid_entrypoint
@@ -14,10 +14,11 @@
       procps \
       git \
       libgtk-3-dev \
-      libgtk-4-dev \
       freeglut3-dev \
-      webkit2gtk-driver \
-      default-jdk
+      webkit2gtk-driver
+
+# There is purposefully no JDK in this image. If a particular build needs JDK it installs
+# it in the Jenkinsfile.
 
 ENV HOME=/home/swtbuild
 ENV DISPLAY :0
@@ -32,6 +33,5 @@
 
 RUN localedef -i en_US -f UTF-8 en_US.UTF-8
 ENV LANG=en_US.UTF-8
-ENV SWT_JAVA_HOME=/usr/lib/jvm/default-java/
 
 USER 10001
\ No newline at end of file

[HannesWell]

Thanks for the update. Looks perfectly fine now!
I just rebased on master and added line-endings to the new files to get rid of the git indication that there is now new line.

[HannesWell]

I just submitted this immediately as the verification-build doesn't test it and started the build and deployment of the docker-images at:
https://ci.eclipse.org/releng/job/Builds/job/Build-Docker-images/32/

[HannesWell]

I just submitted this immediately as the verification-build doesn't test it and started the build and deployment of the docker-images at: https://ci.eclipse.org/releng/job/Builds/job/Build-Docker-images/32/

Unfortunately this build failed because the new docker-hub repository could not be created. I asked the infra-team for help at:
https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/5614

[jonahgraham]

Unfortunately this build failed because the new docker-hub repository could not be created. I asked the infra-team for help at:
https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/5614

Thanks @HannesWell for pushing this along.

added line-endings to the new files to get rid of the git indication that there is now new line.

I submitted #2803 so that the the two similar docker files have the fewest changes between each other as possible possible.

[laeubi]

As far as I could have tested it actually the image doesn't need to contain a JDK at all (as said the pipeline copies the necessary resources anyways).
Therefore I think it's best to remove all about downloading any JDK and setting environment variables for it (the other image could be cleaned-up too IMO).

Just FYI the jdk was there because the build requires the JNI, but it does not matter much what exact version of java so one can use the image as a standalone to compile the native code. Now one needs to supply the JDK externally what a bit breaks the idea of an isolated container and might be a bit challenging if we want to support cross-compiling.

The same is true for the ENV variables, of course everything can be somehow provided externally but this really limits the purpose of a docker container.

[jonahgraham]

@laeubi that is why I also initially included JDK include/lib. However I also needed to change USER to make it usable for me. Maybe we should provide some Dockerfiles that simply do a FROM our published images and add JDK + other needed tools.

For CDT I have also had the tradeoff problem of what goes in docker image vs what is added with Jenkins/pod. For CDT I have erred on the side of including everything, which is why CDT's releng docker image is huge.

[laeubi]

I just take a look and I used (replace 1000 with the output of id -u)

docker container run -u 1000 --rm -it --name swttest -v /<path to>/platform-sdk/git/eclipse.platform.swt/:/tmp/swt eclipse/swtbuild /bin/bash

and you can then run the build scripts inside the container or examine the results on your host system.

sure the JDK makes it bigger, but install it afterwards requires a similar amount of downloadspace + requires time to install and setup the JDK and having at least a basic java (as with default-jdk) is not that bad.

[jonahgraham]

I just take a look and I used (replace 1000 with the output of id -u)

Cool - thanks for sharing this way of doing it.

Can you make the comment on #2803 and if no objection from @HannesWell I'll add the JDK to the gtk3 image rather than remove it from the gtk4 one.

[HannesWell]

As far as I could have tested it actually the image doesn't need to contain a JDK at all (as said the pipeline copies the necessary resources anyways).
Therefore I think it's best to remove all about downloading any JDK and setting environment variables for it (the other image could be cleaned-up too IMO).

Just FYI the jdk was there because the build requires the JNI, but it does not matter much what exact version of java so one can use the image as a standalone to compile the native code. Now one needs to supply the JDK externally what a bit breaks the idea of an isolated container and might be a bit challenging if we want to support cross-compiling.

In fact when designing the the pipelines to build the native binaries for SWT and Equinox I had the opposite goal: Define and supply as much as possible from the Jenkins-pipeline and keep the requirements on the build-machine as low as possible. The reason is, if something has to be changed, one has to apply that change only to the affected Jenkinsfile but not to the executing build-machines.

The same is true for the ENV variables, of course everything can be somehow provided externally but this really limits the purpose of a docker container.

As an example, if we would rely on having the SWT_JAVA_HOME variable defined at the build-machine we would have to make sure it's properly defined on all six/seven different machines. If it changes for what every reason we would have to update all these six/seven build-machines and if a new one is added for a new configuration, it's an additional item on the list of setup tasks.
Now it's only one line in the Jenkinsfile for SWT respectively a different line in the one for Equinox:
https://github.com/eclipse-platform/eclipse.platform.swt/blob/9d3673c858777004c479d05cdd8277d80ccfe95a/Jenkinsfile#L227

And only the required JDK headers are already supplied to the build-machines by the Jenkins-pipeline during the build. So that's nothing one has to worry about. And I don't think that the docker-image is actively used by any person to compile the natives.

So in this case the docker-image is just a utility to get the necessary build-tools and headers onto the executing machine. If it would be possible to download/install it into an existing docker-image I would move all that into the Jenkins pipeline to have it defined there. I already tried that in the past, but failed:

• [Build] Define k8-agent for linux.x86_64 natives completely in pipeline eclipse-equinox/equinox#656

Of course it doesn't harm to add more to the docker-images, but it's not necessary. And if it's not necessary I prefer to remove it so that nobody has to wonder why it's there and if it has to be updated or synchronized if something else changes in the future.

[HannesWell]

The reason is, if something has to be changed, one has to apply that change only to the affected Jenkinsfile but not to the executing build-machines.

Another point I want to add to this is that we can do the change at the Jenkinsfile by our-self, while for changes at the build machines we have to ask the EF-infra team or the admins of that machines (since a few are external ones). And of course this takes time and generates double work.
So having more defined in the Jenkins file gives us more autarchy.