Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix exec race/hang on exit #1246

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

tsightler
Copy link
Contributor

@tsightler tsightler commented Jul 5, 2024

In go2rtc 1.9.4 it is easy to have hangs on exec processes, especially if they spawn child processes, due to a race condition caused by the fact that closer calls cmd.Wait() for RTSP sessions even though cmd.Wait() has already been called in handleRTSP. As far as I can tell, callling Wait() twice is bad practice and the behavior of doing so is undefined.

This small patch fixes the issue by passing in the existing channel from handleRTSP to the closer so that it can share the same wait() instead of attempting to call it a second time. For the handlePipe case, there is no initial call to cmd.Wait(), so the code behaves as previous.

I've tested various killsignals and killtimeout and this fixes all persistent hang conditions. There is still a scenario where, sigkill may kill the parent but the secondary process keeps the producer stream alive. I somewhat wonder if it should always return after sigkill, perhaps after a short timeout, since that should hard kill the parent and clearly the goal at that point is to stop the stream immediately, but I'm guessing this is a rare occurrence.

#1243

@dagleaves
Copy link

dagleaves commented Jul 13, 2024

Tested this as related to the similar hang with ffmpeg audio experienced in #1254. It did not resolve the issue in that case, it just now hangs on waiting for done. Likely due to your last statement because the secondary audio stream keeps it alive. Will likely need some additional way to kill all streams for that issue

@tsightler
Copy link
Contributor Author

tsightler commented Jul 13, 2024

Tested this as related to the similar hang with ffmpeg audio experienced in #1254. It did not resolve the issue in that case, it just now hangs on waiting for done. Likely due to your last statement because the secondary audio stream keeps it alive. Will likely need some additional way to kill all streams for that issue

I wouldn't expect the patch I submitted to fix this issue because, if I'm not mistaken, return audio is only supported via pipe, and the code addresses only the hang when using an RTSP producer via exec, but not for pipe exec. I suspect the actual cause is slightly different as well, as the pipe code needs to be modified to send kill to the secondary ffmpeg process since I believe go2rtc is responsible for starting it as well, but I'll admit I haven't tried to track that down because my project doesn't current support return audio, although I hope to add it eventually.

@AlexxIT AlexxIT self-assigned this Jul 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants