Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Failed to start polaris on UnionTech OS Desktop 20 Pro. #971

Closed
wants to merge 2 commits into from

Conversation

mlsstart
Copy link
Contributor

@mlsstart mlsstart commented Feb 8, 2025

Starting the Java application using /opt/jboss/container/java/run/run-java.sh ...
INFO exec -a "java" java -XX:MaxRAMPercentage=80.0 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -cp "." -jar /deployments/quarkus-run.jar
INFO running in /deployments
[0.005s][warning][os,thread] Failed to start thread "GC Thread#0" - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[0.005s][error ][gc,task ] Failed to create worker thread

@MonkeyCanCode
Copy link
Contributor

@mlsstart please sync your branch with latest main before raise for a code review.

@MonkeyCanCode
Copy link
Contributor

MonkeyCanCode commented Feb 8, 2025

So quick run on my local (Mac) shows it is working without --privileged flag:

➜  polaris git:(main) ✗ cd ~/polaris
➜  polaris git:(main) ✗ ./gradlew clean :polaris-quarkus-server:assemble -Dquarkus.container-image.build=true
Configuration on demand is an incubating feature.

> Task :polaris-api-management-service:openApiGenerate
user-defined server variable support is experimental.
################################################################################
# Thanks for using OpenAPI Generator.                                          #
# Please consider donation to help us maintain this project 🙏                 #
# https://opencollective.com/openapi_generator/donate                          #
################################################################################
Successfully generated code to /Users/yong/Desktop/GitHome/polaris/api/management-service/build/generated

> Task :polaris-api-management-model:openApiGenerate
user-defined server variable support is experimental.
Unexpected allVars for AwsStorageConfigInfo expecting:6 vars. actual:2 vars
Unexpected allVars for AzureStorageConfigInfo expecting:5 vars. actual:2 vars
Unexpected allVars for GcpStorageConfigInfo expecting:3 vars. actual:2 vars
################################################################################
# Thanks for using OpenAPI Generator.                                          #
# Please consider donation to help us maintain this project 🙏                 #
# https://opencollective.com/openapi_generator/donate                          #
################################################################################
Successfully generated code to /Users/yong/Desktop/GitHome/polaris/api/management-model/build/generated

> Task :polaris-api-iceberg-service:openApiGenerate
user-defined server variable support is experimental.
Failed to get the schema name: null
################################################################################
# Thanks for using OpenAPI Generator.                                          #
# Please consider donation to help us maintain this project 🙏                 #
# https://opencollective.com/openapi_generator/donate                          #
################################################################################
Successfully generated code to /Users/yong/Desktop/GitHome/polaris/api/iceberg-service/build/generated

[Incubating] Problems report is available at: file:///Users/yong/Desktop/GitHome/polaris/build/reports/problems/problems-report.html

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.12.1/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD SUCCESSFUL in 42s
80 actionable tasks: 66 executed, 4 from cache, 10 up-to-date
➜  polaris git:(main) ✗ docker run -p 8181:8181 -p 8182:8182 apache/polaris:latest
Starting the Java application using /opt/jboss/container/java/run/run-java.sh ...
INFO exec -a "java" java -XX:MaxRAMPercentage=80.0 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -cp "." -jar /deployments/quarkus-run.jar
INFO running in /deployments
realm: POLARIS root principal credentials: 566e29bd65a4bffe:1dbd7524aef7363fcdc47862dcf6227b

 @@@@   @@@  @       @    @@@@   @  @@@@    @@@@    @  @@@@@  @    @     @@@   @@@@
 @   @ @   @ @      @ @   @   @  @  @@     @       @ @   @   @ @   @    @   @ @
 @@@@  @   @ @     @@@@@  @@@@   @    @@   @      @@@@@  @  @@@@@  @    @   @ @  @@@
 @      @@@  @@@@ @     @ @  @@  @  @@@@    @@@@ @     @ @ @@   @@ @@@@  @@@   @@@@

                     *


                                      /////\
                                   //\\///T\\\
                                ///\\\////\\\\\\
                               //\\\\T////\\\\\\\\\
                          /T\ //\\\\\T///T\\//T\\\\\\
                        //\\\/////T\\////\\/////\\\\\\\  //\\
                     //\\\\\\T///////////////////T\\\\\\\T\\\\\
                  //\\\\/////T\//////////\///////T\\\\\T\\\\\\\\
                 //\\\\\/////\\\T////////////////\\\\\\/\\\\\\\\\
,,..,,,..,,,..,//\\\\////////\\\\\\\\\\/////////\\\\\///\\\\\\\\\\,,,..,,..,,,..,,,.
,,..,,,..,,,..,,,..,,,..,,,..,,,..,,,..,,,..,,,..,,,..,,,..,,,..,,,..,,,.,,,..,,,..,

                                                           Powered by Quarkus 3.18.2
2025-02-08 08:03:56,424 WARN  [io.mic.cor.ins.MeterRegistry] [,] [,,,] (vert.x-acceptor-thread-0) This Gauge has been already registered (MeterId{name='http.server.active.requests', tags=[tag(application=Polaris)]}), the Gauge registration will be ignored. Note that subsequent logs will be logged at debug level.
2025-02-08 08:03:56,427 INFO  [io.quarkus] [,] [,,,] (main) Apache Polaris Server (incubating) 1.0.0-incubating-SNAPSHOT on JVM (powered by Quarkus 3.18.2) started in 1.253s. Listening on: http://0.0.0.0:8181. Management interface listening on http://0.0.0.0:8182.
2025-02-08 08:03:56,428 INFO  [io.quarkus] [,] [,,,] (main) Profile prod activated.
2025-02-08 08:03:56,428 INFO  [io.quarkus] [,] [,,,] (main) Installed features: [cdi, hibernate-validator, micrometer, opentelemetry, reactive-routes, rest, rest-jackson, smallrye-context-propagation, smallrye-health, vertx]
^C2025-02-08 08:04:01,649 INFO  [io.quarkus] [,] [,,,] (main) Apache Polaris Server (incubating) stopped in 0.027s
➜  polaris git:(main) ✗

Do you mind share a bit more info about your setup (e.g. OS) and provide full trace if possible. There are couple reasons which can cause "pthread_create failed (EPERM) for attributes: stacksize" error including not having enough memory allocated to the docker (check if u have sufficient memory allocated to your docker process if using docker desktop. IMO, I don't think it is a good idea to run with privileges flag.

@mlsstart
Copy link
Contributor Author

mlsstart commented Feb 8, 2025

Do you mind share a bit more info about your setup (e.g. OS) and provide full trace if possible.

Hey, I'm with you on the --privileged flag thing. It's def not a good idea to use it for security reasons.
I'm running UnionTech OS Desktop 20 Pro. Here's the system deets:
From lsb_release -a:
No LSB modules available.
Distributor ID: Uos
Description: UnionTech OS Desktop 20 Pro
Release: 20
Codename: eagle
uname -r gives 4.19.0 - amd64 - desktop, so that's the kernel version and architecture.
I've also got the logs from the build and run. Here's the output when I ran the Docker container:
docker run -p 8181:8181 -p 8182:8182 apache/polaris:latest
Starting the Java application using /opt/jboss/container/java/run/run-java.sh ...
INFO exec -a "java" java -XX:MaxRAMPercentage=80.0 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -cp "." -jar /deployments/quarkus-run.jar
INFO running in /deployments
[0.006s][warning][os,thread] Failed to start thread "GC Thread#0" - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[0.007s][error ][gc,task ] Failed to create worker thread
Looks like there's an issue starting the garbage collection threads in the Java app. I'm looking into it. Let me know if you have any thoughts or need more info.

@mlsstart mlsstart changed the title Modify docker run command to include --privileged flag. This change is made because the previous command failed to start threads in the container, likely due to insufficient permissions. The --privileged flag gives the container almost the same privileges as the host, which may resolve the issue. Fix: Failed to start polaris Feb 8, 2025
@MonkeyCanCode
Copy link
Contributor

Do you mind share a bit more info about your setup (e.g. OS) and provide full trace if possible.

Hey, I'm with you on the --privileged flag thing. It's def not a good idea to use it for security reasons.
I'm running UnionTech OS Desktop 20 Pro. Here's the system deets:
From lsb_release -a:
No LSB modules available.
Distributor ID: Uos
Description: UnionTech OS Desktop 20 Pro
Release: 20
Codename: eagle
uname -r gives 4.19.0 - amd64 - desktop, so that's the kernel version and architecture.
I've also got the logs from the build and run. Here's the output when I ran the Docker container:
docker run -p 8181:8181 -p 8182:8182 apache/polaris:latest
Starting the Java application using /opt/jboss/container/java/run/run-java.sh ...
INFO exec -a "java" java -XX:MaxRAMPercentage=80.0 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -cp "." -jar /deployments/quarkus-run.jar
INFO running in /deployments
[0.006s][warning][os,thread] Failed to start thread "GC Thread#0" - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[0.007s][error ][gc,task ] Failed to create worker thread
Looks like there's an issue starting the garbage collection threads in the Java app. I'm looking into it. Let me know if you have any thoughts or need more info.

I don't have a local env that I can use to reproduce this, do you mind sharing the output of 'docker info' as well?

@mlsstart
Copy link
Contributor Author

mlsstart commented Feb 8, 2025

I don't have a local env that I can use to reproduce this

Here is the output of docker info:
Client:
Version: 27.2.0
Context: default
Debug Mode: false
Server:
Containers: 26
Running: 0
Paused: 0
Stopped: 26
Images: 455
Server Version: 27.2.0
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 2c90293f6964528176db4fc61d3defe7ee281779
runc version: 1.0.0~rc6+dfsg1-3
init version: N/A (expected: )
Security Options:
apparmor
seccomp
Profile: builtin
Kernel Version: 4.19.0-amd64-desktop
Operating System: UnionTech OS Desktop 20 Pro
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 15.64GiB
Name: fenhuCxXC738
ID: c61cf61b-4027-414f-84b8-bbe8797dfa67
Docker Root Dir: /data3/lib/docker
Debug Mode: false
Experimental: true
Insecure Registries:
registry.paas
registry.paas.cloudnative
100.76.6.186:7000
nexus.cmss.com:8086
nexus.cmss.com:8090
10.253.91.247:18080
100.71.8.213:5000
cicdcsy.harbor.cmss.com:18080
127.0.0.0/8
Registry Mirrors:
https://docker.nju.edu.cn/
Live Restore Enabled: false
Product License: Community Engine
Let me know if you need any more information.

@jbonofre
Copy link
Member

jbonofre commented Feb 8, 2025

Let me take a look. It worked for me earlier this week.

@flyrain flyrain changed the title Fix: Failed to start polaris Fix: Failed to start polaris on UnionTech OS Desktop 20 Pro. Feb 9, 2025
@flyrain
Copy link
Contributor

flyrain commented Feb 9, 2025

It seems like an issue of docker setting, or OS setting. I'm with @MonkeyCanCode that we should be careful of adding privileged to docker run, esp. for a general recommendation for all OSes.

@MonkeyCanCode
Copy link
Contributor

It seems like an issue of docker setting, or OS setting. I'm with @MonkeyCanCode that we should be careful of adding privileged to docker run, esp. for a general recommendation for all OSes.

Yes. This may be related to moby/moby#47297. I see you have seccomp profile set to builtin, maybe try to change it to unconfined instead? Here is what I have on my local:

 Security Options:
  seccomp
   Profile: unconfined

@mlsstart
Copy link
Contributor Author

Thank you for suggesting setting the seccomp profile to the "unconfined" mode. I followed your advice, and the problem has been resolved. I suggest adding information about Docker settings on UnionTech OS Desktop 20 Pro to the quickstart.md.

@mlsstart mlsstart closed this Feb 10, 2025
@flyrain
Copy link
Contributor

flyrain commented Feb 10, 2025

@mlsstart, Thanks for the confirmation! Feel free to file an issue/PR for the doc change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants