Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mac-capture: Set minimum frame interval in capture initialization #11896

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jcm93
Copy link
Contributor

@jcm93 jcm93 commented Feb 25, 2025

Description

When creating a macOS Screen Capture source, set the minimumFrameInterval property in the capture stream properties based on OBS's currently configured output FPS.

Motivation and Context

Because OBS does not set a minimumFrameInterval on the SCStreamConfiguration object, we are subject to whatever defaults ScreenCaptureKit imposes on capture streams. At some point, perhaps in macOS 15, a change in macOS seems to have made this default 1/60, even if the host display has a higher refresh rate or if the content is updating more frequently than 60 times per second.

OBS allows frame rates higher than 60, as does macOS, so we should set this property in accordance to OBS's configured FPS and ensure greater-than-60 frame rate captures are possible.

Importantly, setting this property to the exact inverse of our frame rate seems to provide a capture that drops many frames, so this PR targets a slightly shorter minimum interval. Presumably this allows the OBS and macOS render loops to better deal with timing variations between them.

The chosen buffer is 10% of our target interval. This value was determined through subjective testing. Oddly, setting a target interval significantly shorter than our target update rate (e.g. 1 / (target frame rate * 2)) also led to significant amounts of dropped frames, though less than with the exact interval.

Multiple rounds of testing were performed under different conditions (and between macOS 15.2 and 15.3) to verify that these results were as consistent as macOS would allow. As we have limited introspection ability for the internals of ScreenCaptureKit, we unfortunately do not seem to have better options here than significant amounts of trial-and-error testing.

Fixes #11778

How Has This Been Tested?

On an M1 Max MacBook Pro with "ProMotion" display on macOS 15.3.1, various trial recordings were made using https://www.testufo.com and the Display, Window and Application Capture modes of macOS Screen Capture and with OBS's target FPS set to 120. Verified that the chosen minimum frame interval value led to a relatively smooth capture at 120 fps.

Test recordings:

Existing behavior (default 1/60 minimum interval, OBS at 120 fps):

2025-02-25.11-20-01.mp4

PR behavior (1/132 minimum interval, OBS at 120 fps):

2025-02-25.11-21-07.mp4

Matching interval behavior (1/120 minimum interval, OBS at 120 fps):

1.120.mp4

Doubled interval behavior (1/240 minimum interval, OBS at 120 fps):

1.240.mp4

Since the 10% buffer seemed to have the subjectively best pacing across multiple rounds of testing, and presumably also asks less of the API than a doubled interval, it is what I went with for this PR.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • Tweak (non-breaking change to improve existing functionality)

Checklist:

  • My code has been run through clang-format.
  • I have read the contributing document.
  • My code is not on the master branch.
  • The code has been tested.
  • All commit messages are properly formatted and commits squashed where appropriate.
  • I have included updates to all appropriate documentation.

@jcm93 jcm93 force-pushed the fix/higher-sck-fps branch 2 times, most recently from 6827ded to f5d3013 Compare February 25, 2025 18:14
@WizardCM WizardCM added Bug Fix Non-breaking change which fixes an issue macOS Affects macOS labels Feb 26, 2025
@jcm93 jcm93 force-pushed the fix/higher-sck-fps branch from f5d3013 to f276025 Compare February 26, 2025 19:01
Copy link
Member

@PatTheMav PatTheMav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this effectively forces SCK to always capture slightly faster than OBS' current canvas frame rate, thus preventing macOS from throttling capture for energy efficiency or other reasons (because usually ProMotion wants to rapidly change the current device frame rate based on user interaction and thus be more reactive to what happens visually).

How does this cope when the canvas is set to a very high frame rate, say 360fps?

@jcm93
Copy link
Contributor Author

jcm93 commented Feb 26, 2025

So this effectively forces SCK to always capture slightly faster than OBS' current canvas frame rate,

I think we have to be wary of phrasing it this way; we can't ever force SCK to deliver a frame by any particular time or with any particular cadence. From the best of my understanding, SCK gets every frame that the internal macOS rendering system decides needs to be created, and if certain conditions are met, it passes that frame along to client applications. All minimumFrameInterval says is "don't pass this update to clients if we already had an update n or fewer seconds ago".

In my testing, ScreenCaptureKit will not ever deliver updates faster than there are 'genuine' updates to the captured frame content that are available. In other words, if we had content constantly updating at display refresh (120 Hz), and a minimum frame interval of 1/360, we would still receive new frames at (best case) 120 Hz. If we have static content not being updated whatsoever by macOS, we won't receive any new frames via SCK at any time even if the minimumFrameInterval is 1/9000 (any artificially high value).

@PatTheMav
Copy link
Member

So this effectively forces SCK to always capture slightly faster than OBS' current canvas frame rate,

I think we have to be wary of phrasing it this way; we can't ever force SCK to deliver a frame by any particular time or with any particular cadence. From the best of my understanding, SCK gets every frame that the internal macOS rendering system decides needs to be created, and if certain conditions are met, it passes that frame along to client applications. All minimumFrameInterval says is "don't pass this update to clients if we already had an update n or fewer seconds ago".

In my testing, ScreenCaptureKit will not ever deliver updates faster than there are 'genuine' updates to the captured frame content that are available. In other words, if we had content constantly updating at display refresh (120 Hz), and a minimum frame interval of 1/360, we would still receive new frames at (best case) 120 Hz. If we have static content not being updated whatsoever by macOS, we won't receive any new frames via SCK at any time even if the minimumFrameInterval is 1/9000 (any artificially high value).

Got it. Sounds uncontroversial to me then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Fix Non-breaking change which fixes an issue macOS Affects macOS
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"MacOS screen capture" only captures up to 60fps, despite using higher (120) fps configuration
3 participants