Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【Server Core Audio Client On Win】【WIP】feat: add server core audio client for windows #134

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

Jamlee
Copy link

@Jamlee Jamlee commented Jan 27, 2025


English Version

This is a flawed PR. I feel that it's quite challenging for me to tackle the last problem on my own, so I've opened this PR to ask for help. The current code can be compiled successfully. The usage is as follows:

.\as-cmd.exe --connect=172.28.125.194:65530

I've spent approximately three weeks of my free time on this. During the implementation process, I've resolved the following issues:

  • Enabled the client (Windows) to play sounds. By utilizing the pAudioClient->SetEventHandle(hEvent) event, I addressed the problem where CPU usage during playback reached up to 12%.
  • Solved the problem of inconsistent audio format support between the server and the client. I used the pAudioClient->IsFormatSupported method to achieve comparable playback.
  • Resolved the issue where some hosts support audio playback objects in the WAVEFORMATEX format while others support the WAVEFORMATEXTENSIBLE format.
  • Added a ring buffer to deal with the issue that the UDP audio data received each time is too small (around a maximum of 1490 bytes).
  • Conducted basic studies on the asio await mode and MFC.

In the end, there's still one problem remaining:

  • The data received via UDP is actually faster than the audio playback speed, causing the played sound to lag behind the real - time sound. Gradually, there will be a delay of about 10 seconds or so.

The efforts I've made are as follows:

  1. A ring buffer was added to quickly discard the data when buffer overflow occurs. However, this has led to sound distortion.

中文版本(Chinese Version)

这是1个有缺陷的PR,最后的一个问题我感觉只凭我自己难以处理了,所以才开启这个PR,以寻求帮助。当前代码是完全可以编译的,使用方式如下:

.\as-cmd.exe --connect=172.28.125.194:65530

我大概花了3周的业余时间,在实现的过程中解决了如下问题:

  • 添加 client(windows) 播放声音能力,使用 pAudioClient->SetEventHandle(hEvent) 事件,解决播放过程占用 CPU 高达 12% 的问题。
  • 解决 server和client 支持音频格式不一致问题,使用接近的播放 pAudioClient->IsFormatSupported
  • 解决有的主机支持的音频播放对象是 WAVEFORMATEX 格式,而有的是 WAVEFORMATEXTENSIBLE 的问题。
  • 添加 ringbuffer 缓存 udp 音频数据每次 recv 数据太小的问题(约最大1490)。
  • asio await 模式入门学习、MFC 入门学习。

最终剩余下1个问题:

  • udp 接收的数据,实际上比播放音频的速度要快,导致播放的声音落后于实时的声音。慢慢会延时大约10几秒。

我做出的努力如下:

  1. 添加 ringbuffer 快速丢弃缓冲溢出的数据。这又导致声音变形。

Sorry, something went wrong.

@mkckr0
Copy link
Owner

mkckr0 commented Jan 28, 2025

You use WaitForSingleObject(hEvent, INFINITE); to wait for IAudioRenderClient available. That's to say you use the playback as the the timeline. It' will delay gradually by CPU scheduling.

Instead, you should use the receiving UDP packet as the timeline. They are directly controlled by IAudioCaptureClient. You can use condition variable to notify the playback thread when receive the UDP packet.

@Jamlee
Copy link
Author

Jamlee commented Jan 28, 2025

@mkckr0 At first, using UDP packets as the timeline causes high CPU usage. On an 8 - core machine, the CPU utilization can reach 12%. This is not good for a small tool. I'm very curious about why RDP has such low resource consumption and can play real - time audio without any problems.

@mkckr0
Copy link
Owner

mkckr0 commented Jan 28, 2025

The high CPU usage is always caused by busy waiting. Use condition variable to avoid it. The WaitForSingleObject has the same effect.

@Jamlee
Copy link
Author

Jamlee commented Jan 28, 2025

@mkckr0 It seems that the CPU usage has dropped to 8%. However, there is still a big gap from my target of around 0.1%. The main reason is that these two functions are called too many times. aee65cd
image

@mkckr0
Copy link
Owner

mkckr0 commented Jan 28, 2025

I test your branch. It seems like calculate_available_data() > 0 always be true even when there is no sound. You should make sure the ring buffer can work normally at first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants