Hardware encoding doesn't work

[aWZHY0yQH81uOYvH]

When starting a stream with the Video Toolbox hardware encoder under macOS or using NVENC under Windows, the stream appears to start normally and the "Status -> Bitrate" displays approximately the selected video bitrate, but there is no video when the stream is viewed from VLC.

In the case of NVENC and Windows, the audio stream works with no video. With Video Toolbox under macOS, there is no audio and VLC eventually times out, saying "no data."

The plugin works as expected with x264 software encoding.

(Also clicking "stop" in the rtspserver dialog doesn't seem to stop OBS's encoding resource usage in some cases.)

Versions: obs-rtspserver 3.0.0, OBS 29.1.3, VLC 3.0.18, Windows 11, macOS 13.4.1

[aWZHY0yQH81uOYvH]

It seems like the "stop" button doesn't work when the advanced streaming options are selected in OBS instead of simple. Audio also doesn't seem to work in advanced mode (no audio stream is present in VLC).

[aWZHY0yQH81uOYvH]

After trying with v3.1.0-beta1, it seems like the Video Toolbox encoder works sometimes (maybe 10% of the time?). It can only be selected in the advanced streaming options mode, which still results in the audio being missing and the stop button not working.

[zgy-daodaodao]

me to

error :
20:11:37.465: NVENC supported
20:11:37.828: [NVENC] AV1 is not supported

detail log

20:11:35.313: CPU Name: AMD Ryzen 9 3900 12-Core Processor
20:11:35.313: CPU Speed: 3094MHz
20:11:35.313: Physical Cores: 12, Logical Cores: 24
20:11:35.313: Physical Memory: 32664MB Total, 26369MB Free
20:11:35.313: Windows Version: 10.0 Build 25915 (release: 23H2; revision: 1000; 64-bit)
20:11:35.313: Running as administrator: false
20:11:35.313: Windows 10/11 Gaming Features:
20:11:35.314: Game DVR: On
20:11:35.314: Game Mode: Probably On (no reg key set)
20:11:35.314: Sec. Software Status:
20:11:35.316: Microsoft Defender 防病毒: enabled (AV)
20:11:35.316: Windows 防火墙: enabled (FW)
20:11:35.316: Current Date/Time: 2023-08-02, 20:11:35
20:11:35.316: Browser Hardware Acceleration: true
20:11:35.316: Hide OBS windows from screen capture: true
20:11:35.316: Qt Version: 6.4.3 (runtime), 6.4.3 (compiled)
20:11:35.316: Portable mode: false
20:11:35.832: OBS 29.1.3 (64-bit, windows)
20:11:35.832: ---------------------------------
20:11:35.833: ---------------------------------
20:11:35.833: audio settings reset:
20:11:35.833: samples per sec: 48000
20:11:35.833: speakers: 2
20:11:35.833: max buffering: 960 milliseconds
20:11:35.833: buffering type: dynamically increasing
20:11:35.834: ---------------------------------
20:11:35.834: Initializing D3D11...
20:11:35.834: Available Video Adapters:
20:11:35.838: Adapter 0: NVIDIA GeForce RTX 2070
20:11:35.838: Dedicated VRAM: 8408530944
20:11:35.838: Shared VRAM: 17125439488
20:11:35.838: PCI ID: 10de:1f14
20:11:35.839: Driver Version: 31.0.15.3129
20:11:35.839: output 0:
20:11:35.839: name=
20:11:35.839: pos={0, 0}
20:11:35.839: size={1920, 1080}
20:11:35.839: attached=true
20:11:35.839: refresh=60
20:11:35.839: bits_per_color=8
20:11:35.839: space=RGB_FULL_G22_NONE_P709
20:11:35.839: sdr_white_nits=80
20:11:35.839: nit_range=[min=0.500000, max=270.000000, max_full_frame=270.000000]
20:11:35.839: dpi=120 (125%)
20:11:35.839: id=\?\DISPLAY#LGD0625#5&78754ca&0&UID28931#{e6f07b5f-ee97-4a90-b076-33f57bf4eaa7}
20:11:35.839: alt_id=\.\DISPLAY1
20:11:35.839: Adapter 1: NVIDIA GeForce RTX 2070
20:11:35.839: Dedicated VRAM: 8408530944
20:11:35.839: Shared VRAM: 17125439488
20:11:35.839: PCI ID: 10de:1f14
20:11:35.839: Driver Version: 31.0.15.3129
20:11:35.842: Loading up D3D11 on adapter NVIDIA GeForce RTX 2070 (0)
20:11:35.990: D3D11 loaded successfully, feature level used: b000
20:11:35.990: DXGI increase maximum frame latency success
20:11:35.990: Hardware-Accelerated GPU Scheduling enabled on adapter!
20:11:35.990: D3D11 GPU priority setup failed (not admin?)
20:11:37.387: ---------------------------------
20:11:37.387: video settings reset:
20:11:37.387: base resolution: 1920x1080
20:11:37.387: output resolution: 1920x1080
20:11:37.387: downscale filter: Bicubic
20:11:37.387: fps: 10/1
20:11:37.387: format: NV12
20:11:37.387: YUV mode: Rec. 709/Partial
20:11:37.387: NV12 texture support enabled
20:11:37.387: P010 texture support not available
20:11:37.390: Audio monitoring device:
20:11:37.390: name: 默认
20:11:37.390: id: default
20:11:37.390: ---------------------------------
20:11:37.393: No AJA devices found, skipping loading AJA UI plugin
20:11:37.393: Failed to initialize module 'aja-output-ui.dll'
20:11:37.394: No AJA devices found, skipping loading AJA plugin
20:11:37.394: Failed to initialize module 'aja.dll'
20:11:37.394: Skipping module '../../obs-plugins/64bit/chrome_elf.dll', not an OBS plugin
20:11:37.398: [CoreAudio encoder]: CoreAudio AAC encoder not installed on the system or couldn't be loaded
20:11:37.398: Failed to load 'en-US' text for module: 'decklink-captions.dll'
20:11:37.399: Failed to load 'en-US' text for module: 'decklink-output-ui.dll'
20:11:37.403: A DeckLink iterator could not be created. The DeckLink drivers may not be installed
20:11:37.403: Failed to initialize module 'decklink.dll'
20:11:37.433: [AMF] Unable to load 'amfrt64.dll', error code 126.
20:11:37.435: [AMF] AMF Test failed due to one or more errors.
20:11:37.435: Failed to initialize module 'enc-amf.dll'
20:11:37.444: Skipping module '../../obs-plugins/64bit/libcef.dll', not an OBS plugin
20:11:37.444: Skipping module '../../obs-plugins/64bit/libEGL.dll', not an OBS plugin
20:11:37.444: Skipping module '../../obs-plugins/64bit/libGLESv2.dll', not an OBS plugin
20:11:37.460: [obs-browser]: Version 2.21.1
20:11:37.460: [obs-browser]: CEF Version 103.0.5060.134 (runtime), 103.0.0-5060-shared-textures.2591+g4204d54+chromium-103.0.5060.134 (compiled)
20:11:37.465: NVENC supported
20:11:37.828: [NVENC] AV1 is not supported
20:11:37.830: [noise suppress]: NVIDIA denoiser disabled, redistributable not found or could not be loaded.
20:11:37.830: Failed to get NVVideoEffects.dll version info size
20:11:37.832: [NVIDIA VIDEO FX]: FX disabled, redistributable not found or could not be loaded.
20:11:37.881: [obs-websocket] [obs_module_load] you can haz websockets (Version: 5.2.3 | RPC Version: 1)
20:11:37.881: [obs-websocket] [obs_module_load] Qt version (compile-time): 6.4.3 | Qt version (run-time): 6.4.3
20:11:37.881: [obs-websocket] [obs_module_load] Linked ASIO Version: 101201
20:11:37.890: [obs-websocket] [obs_module_load] Module loaded.
20:11:37.897: [vlc-video]: VLC 3.0.18 Vetinari found, VLC video source enabled
20:11:37.905: ---------------------------------
20:11:37.905: Loaded Modules:
20:11:37.905: win-wasapi.dll
20:11:37.905: win-dshow.dll
20:11:37.905: win-capture.dll
20:11:37.905: vlc-video.dll
20:11:37.905: text-freetype2.dll
20:11:37.905: rtmp-services.dll
20:11:37.905: obs-x264.dll
20:11:37.905: obs-websocket.dll
20:11:37.905: obs-vst.dll
20:11:37.905: obs-transitions.dll
20:11:37.905: obs-text.dll
20:11:37.905: obs-rtspserver.dll
20:11:37.905: obs-qsv11.dll
20:11:37.905: obs-outputs.dll
20:11:37.905: obs-filters.dll
20:11:37.905: obs-ffmpeg.dll
20:11:37.905: obs-browser.dll
20:11:37.905: image-source.dll
20:11:37.905: frontend-tools.dll
20:11:37.905: decklink-output-ui.dll
20:11:37.905: decklink-captions.dll
20:11:37.905: coreaudio-encoder.dll
20:11:37.905: ---------------------------------
20:11:37.905: ==== Startup complete ===============================================
20:11:38.011: All scene data cleared
20:11:38.012: ------------------------------------------------
20:11:38.024: [win-wasapi: '桌面音频'] update settings:
20:11:38.024: device id: default
20:11:38.024: use device timing: 1
20:11:38.026: [Loaded global audio device]: '桌面音频'
20:11:38.026: [win-wasapi: '麦克风/Aux'] update settings:
20:11:38.026: device id: default
20:11:38.026: use device timing: 0
20:11:38.026: [noise suppress: '噪声抑制'] NVAFX redist is not installed.
20:11:38.026: [Loaded global audio device]: '麦克风/Aux'
20:11:38.026: - filter: '噪声抑制' (noise_suppress_filter_v2)
20:11:38.056: WASAPI: Device 'Microphone (High Definition Audio Device)' [44100 Hz] initialized
20:11:38.059: Switched to scene '场景'
20:11:38.059: ------------------------------------------------
20:11:38.059: Loaded scenes:
20:11:38.059: - scene '场景':
20:11:38.059: - source: 'VLC 视频源' (vlc_source)
20:11:38.059: ------------------------------------------------
20:11:38.064: WASAPI: Device '扬声器 (High Definition Audio Device)' [48000 Hz] initialized
20:11:38.117: adding 64 milliseconds of audio buffering, total audio buffering is now 64 milliseconds (source: 麦克风/Aux)
20:11:38.117:
20:11:38.283: [jim-nvenc: 'rtsp_output_video'] settings:
20:11:38.283: codec: H264
20:11:38.283: rate_control: CBR
20:11:38.283: bitrate: 2500
20:11:38.283: cqp: 20
20:11:38.283: keyint: 250
20:11:38.283: preset: p5
20:11:38.283: tuning: hq
20:11:38.283: multipass: qres
20:11:38.283: profile: high
20:11:38.283: width: 1920
20:11:38.283: height: 1080
20:11:38.283: b-frames: 2
20:11:38.283: lookahead: false
20:11:38.283: psycho_aq: true
20:11:38.283:
20:11:38.345: ---------------------------------
20:11:38.346: [FFmpeg aac encoder: 'rtsp_output_audio_track1'] bitrate: 128, channels: 2, channel_layout: stereo
20:11:38.346:
20:11:38.347: starting rtsp frame push thread
20:11:38.350: starting rstp server on port '554'

[iamscottxu]

This issue is a duplicate of #128.