Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reoccurring null in LockableResourcesManager resources #677

Open
Naadwo opened this issue Jun 26, 2024 · 8 comments
Open

Reoccurring null in LockableResourcesManager resources #677

Naadwo opened this issue Jun 26, 2024 · 8 comments
Labels
bug Clarification needed Additional clarification needed Need test Automated tests shall be extended.

Comments

@Naadwo
Copy link

Naadwo commented Jun 26, 2024

Jenkins and plugins versions report

Environment
Jenkins: 2.426.2
OS: Linux - 5.15.0-1064-azure
Java: 17.0.9 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
lockable-resources:1218.va_3dd45e2b_fa_7

What Operating System are you using (both controller, and any agents involved in the problem)?

Jenkins is running in a container on Ubuntu 22

Reproduction steps

Unfortunately I do not have exact steps to reproduce the issue. I have a number of jobs that use lockable resources, so resources are locked/unlocked continuously minimum few times a minute. I've observed that every few days somehow a null is added to LockableResourcesManager resources list. This breaks not only the plugin (any call to step lock throws exception), but also global Jenkins configuration and the only option to fix it, is to directly replace resources list in LockableResourcesManager.
I'm not using any advanced features of this plugin, only simple step lock('lock-name') { }
I was tryin to find any logs from when this null was added, but I didn't find any.
I doubt it matters, but in my case job name is often used as resource name.

Expected Results

I don't know how is that even possible that a null resource is added to the LockableResourcesManager.resources, but I would expect that LockableResourcesManager should be able to remove it as clearly invalid entry.

Actual Results

I've observed two logs in case of failure (but I'm not sure why there are two variants) - both often appear mixed in log of the same job, that attempts to lock always the same resource.

Trying to acquire lock on [<lock name>]
[Pipeline] // lock
[Pipeline] End of Pipeline
java.lang.NullPointerException
Finished: FAILURE

and longer one:

[Pipeline] lock
Trying to acquire lock on [<lock name>]
[Pipeline] // lock
[Pipeline] End of Pipeline
Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: a326e53e-1b63-49e1-9293-60b260f97d14
java.lang.NullPointerException: Cannot invoke "org.jenkins.plugins.lockableresources.LockableResource.getName()" because "r" is null
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.fromName(LockableResourcesManager.java:272)
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.resourceExist(LockableResourcesManager.java:292)
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.addResource(LockableResourcesManager.java:803)
	at org.jenkins.plugins.lockableresources.LockableResourcesManager.createResource(LockableResourcesManager.java:753)
	at org.jenkins.plugins.lockableresources.LockStepExecution.start(LockStepExecution.java:55)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:323)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:196)
	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:124)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:47)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
	at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
	at org.jenkinsci.plugins.workflow.cps.LoggingInvoker.methodCall(LoggingInvoker.java:105)
	at discardOtherConcurrent.call(discardOtherConcurrent.groovy:6)
	at discardOtherConcurrent.call(discardOtherConcurrent.groovy)
	at WorkflowScript.run(WorkflowScript:32)
	at ___cps.transform___(Native Method)
	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:90)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:116)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:85)
	at jdk.internal.reflect.GeneratedMethodAccessor247.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46)
	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:152)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:146)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:146)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:187)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:423)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:331)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:295)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:97)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at jenkins.util.ErrorLoggingExecutorService.lambda$wrap$0(ErrorLoggingExecutorService.java:51)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)
Finished: FAILURE

Anything else?

No response

Are you interested in contributing a fix?

No response

@Naadwo Naadwo added the bug label Jun 26, 2024
@mPokornyETM
Copy link
Contributor

May you pls install last version. 1218 a little bit old.

@mPokornyETM mPokornyETM added the Clarification needed Additional clarification needed label Jul 19, 2024
@epliskin
Copy link

epliskin commented Dec 9, 2024

Same problem with Jenkins 2.426.2.
Message
( java.lang.NullPointerException Cannot invoke "org.jenkins.plugins.lockableresources.LockableResource.getName()" because "r" is null)
seemingly at the exit from steps.lock() pipeline block.
This occurred for the second time in a few days.
Before that the same Jenkins installation worked smoothly for several months.
Recently have added some agent but it seems to work well...

@mPokornyETM
Copy link
Contributor

Same problem with Jenkins 2.426.2. Message ( java.lang.NullPointerException Cannot invoke "org.jenkins.plugins.lockableresources.LockableResource.getName()" because "r" is null) seemingly at the exit from steps.lock() pipeline block. This occurred for the second time in a few days. Before that the same Jenkins installation worked smoothly for several months. Recently have added some agent but it seems to work well...

which plugin version do you used?

@epliskin
Copy link

epliskin commented Dec 9, 2024

Lockable Resources 1218.va_3dd45e2b_fa_7

@mPokornyETM
Copy link
Contributor

1218

may you pls update the plugin to the last one version. There was many fixes and this issue is porbably fixed.

thx

@epliskin
Copy link

epliskin commented Dec 9, 2024

This plugin update probably requires total Jenkins update. Not so easy in our working environment. Will try when I can...

@epliskin
Copy link

epliskin commented Jan 4, 2025

I believe the problem was this. There were three pipelines named say P1, P2, P3. The first and the third pipeline use resources (R1,...RN). The second pipeline uses resources (R2...RN) except for R1. Sequence of events could be like this:

  • P1 starts and locks (R1..RN) and does its work.
  • P2 starts and waits for (R2..RN).
  • P1 finishes, releasing (R1..RN).
  • P2 locks (R2..RN) and does its work. At this point R1 is presumably free or may even be destroyed.
  • P3 starts and waits for (R1..RN).
  • P2 completes and releases (R2..RN). At this moment an exception occures in P2 as described above in this issue.

The above model is my best guess. But the fact is that actually I was able to circumvent the problem by including R1 into the list of required resources for the P2 pipeline. So now when all the resources lock and release simultaneously it all works smoothly as before. That is my habit anyway. The problem initially arised some weeks ago when I forgot to include a new resource R1 into P2. Regards.

@mPokornyETM mPokornyETM added the Need test Automated tests shall be extended. label Jan 7, 2025
@mPokornyETM
Copy link
Contributor

Nice scenario. I will propose to create an automated test first. Then we can see if the issue is still rproducable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Clarification needed Additional clarification needed Need test Automated tests shall be extended.
Projects
None yet
Development

No branches or pull requests

3 participants