Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Steam audio spatializer does not work correctly. FMOD+UNITY #362

Open
Denis-Semenovykh opened this issue Jul 11, 2024 · 12 comments
Open

Steam audio spatializer does not work correctly. FMOD+UNITY #362

Denis-Semenovykh opened this issue Jul 11, 2024 · 12 comments

Comments

@Denis-Semenovykh
Copy link

System Information
Please provide the following information about your system:

  • Steam Audio version: 4.5.3
  • (If applicable) Unity version: 2022.3.13
  • (If applicable) FMOD Studio version: 2.02.22
  • Operating System and version: win 11
  • (Optional) CPU architecture (e.g. x86-64, armv7): x64

Issue Description
As soon as in the event I add to the master Steam audio spatializer sound begins to play always on the left, as well as attenuation curves do not work, or rather with a linear curve sound only on the left, and when switching to any other curves, there is no sound. In fmod studio sound works and spatialize, but when you run in Unity, everything is on the left, or if you select any curve except linear, there is no sound. Also the distance set on the spatializer does not affect the sound, and when setting the distance in macros Fmod to a low value (for example, 10 meters) also does not perceive the sound, the sound appears if the distance in macros from 75 meters. If you install Fmod spatializer everything is fine....And another point, when using Steam audio spatializer output sound is always stereo, even if the platform fmod 5.1, and accordingly, when you play not in headphones sound is played only on the front speakers. So it turns out that Steam Audio is designed only for headphones? And if the game is planned several options of platforms (headphones, 5.1, 7.1) then the possibility of using Steam Audio is impossible, because you can not change the spatializer when changing the platform?

Снимок экрана 2024-07-11 153311
Снимок экрана 2024-07-11 153330

@lakulish
Copy link
Collaborator

@Denis-Semenovykh Unfortunately, we're unable to repro this issue on our end. Could you share a minimal FMOD and Unity project that reproduces this issue?

Also, you make a good point about headphones vs. 5.1, 7.1, etc. configurations. The following pull request: #329 updates the FMOD Studio and Unity plugins so that when the speaker configuration is something other than stereo, Steam Audio falls back to a panning algorithm. This pull request has been merged, and will be part of the next official release.

@Denis-Semenovykh
Copy link
Author

Denis-Semenovykh commented Jul 18, 2024

@lakulish I attach a demonstration of the screen, where you can see that with different curves on steam spatializer sound either works or not, and directly in the game it is always on the left. When changing to fmod spatializer everything is fine, everything works correctly. Reinstalling Steam in the fmod project and in Unity does not give any result...
And when is an update planned to work across platforms? And when changing to 5.1 for example, will Air absorbtion and Directivity still work?

https://drive.google.com/file/d/1GtQvBzoo7xW1idtwh1_k_3DkGQNUGYHu/view?usp=sharing

Update: Solved the problem with positioning, but we have another bugs, we try send some sounds to the channel with Steam Audio Reverb, but it has no effect, as if the reverb doesn't work. And when in event steam spatializer setting log curve, and the distance minimum 0, and maximum any value, the sound disappears and appears only if the listener and the source in 0 distance. When setting the minimum value to 1, and the maximum value to any value, then everything works correctly.

@lakulish
Copy link
Collaborator

@Denis-Semenovykh I tried matching my event settings to the ones visible in your video, but I'm still unable to repro the issue. I noticed that you have Live Update on in FMOD Studio, does the issue occur in the FMOD Studio editor even when Live Update is off?

@Denis-Semenovykh
Copy link
Author

Denis-Semenovykh commented Jul 19, 2024

@lakulish Update: Solved the problem with positioning, but we have another bugs, we try send some sounds to the channel with Steam Audio Reverb, but it has no effect, as if the reverb doesn't work (Or does it only work for real time reflections? Not for baked ones?). And when in event steam spatializer setting log curve, and the distance minimum 0, and maximum any value, the sound disappears and appears only if the listener and the source in 0 distance. When setting the minimum value to 1, and the maximum value to any value, then everything works correctly. And for me not clearly how working and what,s the main goal steam audio mixer return.

@lakulish
Copy link
Collaborator

lakulish commented Jul 19, 2024

@Denis-Semenovykh Steam Audio Reverb should work with either real-time or baked reflections/reverb. If it's not working, there might be an issue with how the Unity project is set up to send reverb information to the FMOD Studio plugin. Can you share more information (or a minimal project) about how the Steam Audio Sources, Steam Audio Listeners, etc. are configured in Unity?

As for the issue when setting the minimum distance to 0, what are the settings you're using for the distance curve? Can you share a screenshot of this?

The Steam Audio Mixer Return effect is way to optimize the rendering of multiple sources with reflections. It takes the reflected sound out of the event's signal processing chain, mixes all reflections together, and reinserts it into the bus containing the mixer return effect. Of course, this means that you can't add further DSP to the event after the spatializer, so the choice of whether or not to use this effect depends on your project's needs.

@Denis-Semenovykh
Copy link
Author

Denis-Semenovykh commented Jul 20, 2024

@lakulish I found out that reflections work only when in Steam Audio settings set Reflection effect settings to Parametric, in other variants there are no reflections, and TrueAudio Next crashes the editor. All done according to the instructions, materials on the level assigned, the scene exported, on the listener in addition to Fmod listener, added Steam Audio Listener with enabled reverb and in the position realtime. On the sound source also hangs Steam audio source with enabled reflections. Reinstalling Steam Audio in the project does not give any results, everything is the same. Also if occlusion on spatializer on sound disapear , even if it's not enclosed by any geometry, than if transmission set to on, sound appear but very quiet.

@Denis-Semenovykh
Copy link
Author

Denis-Semenovykh commented Jul 20, 2024

@lakulish Update! Found the cause of the problem, the player we have is not on the stage, and spawns at boot, apparently so not properly initialized Steam Audio listener. I think we will try to solve this situation. But now the next question arises - We have a level, it is not one scene, and it consists of 5 scenes. Will bake work for such a variant? Because it turns out we need to separately bake probes on each scene...but for example we need pathing for a moving object npc and there you can specify only one Probe Batch and it turns out Pathing will work only on one scene? What should we do in this situation, do you have any ideas or maybe we are missing something and there is a solution already?....And also, we have a lot of sound source objects that are not on stage, but appear during the game. And when they appear on the stage, even if they have enabled real-time reflections, they do not see the geometry, and accordingly do not play sound, if enabled occlusion for them, how can this be fixed? Also they are not initialized as dynamic, because export is possible beforehand, if the object is on the stage, and they are not there.... And is it possible to bake reflections for such objects? Thank you in advance!

@lakulish
Copy link
Collaborator

@Denis-Semenovykh At the moment we don't support calculating paths from one probe batch to another, so you will probably have to bake a single probe batch using all of the geometry, and save that probe batch in one of the scenes.

I'm not sure I fully understand the issue you're describing with sources being instantiated at run-time. Once the geometry has been exported, all sources should be able to make use of the geometry when simulating occlusion, reflections etc., even if they are instantiated at run-time. Note that depending on your settings, with real-time reflections there may be a small lag between the source instantiation and the simulated reflections becoming audible. If you want to use baked reflections, you will need to use a Steam Audio Baked Source on one or more GameObjects that are not dynamically instantiated, and then make sure that the Current Baked Source for the dynamically instantiated sources are set to point to the Steam Audio Baked Source. Hope this helps!

@Denis-Semenovykh
Copy link
Author

Denis-Semenovykh commented Jul 24, 2024

@lakulish Thanks for the reply! But I mean, when the object is not directly on the scene, for example Npc, which is loaded later, and it is just a prefab in the Assets folder, it is impossible to bake reflections to it and assign them in the Pathing and Reflections section. It doesn't see them in Assets or Scene. When you put it on the scene, it works, but when you remove it, the prefab shows "missing component" instead of data..

@lakulish
Copy link
Collaborator

@Denis-Semenovykh How about the following approach: on your prefab, make sure you have a Steam Audio Source, but not a Steam Audio Baked Source. Then, create some GameObject in the scene at the source's actual (static) position, and add a Steam Audio Baked Source to that GameObject, and bake reflections on it. At run-time, when you instantiate the prefab, set its Current Baked Source property to point to the GameObject on which you baked reflections. Does this approach work for you?

@Denis-Semenovykh
Copy link
Author

Denis-Semenovykh commented Aug 9, 2024

@lakulish Hello! I recorded a video for you. I managed to bake the reverb at the level. But the problem with occlusions is not solved. When you turn on physic-based sound disappears, when you turn it off it appears and reflections also do not work for these sounds. What can be the problem? And what about pathing, if the object is so initially not on the stage? I.e. you can't add an asset with probes in Steam Audio Source in prefab, it doesn't see it until it is on the stage.
https://drive.google.com/file/d/1Mpzg0lnT_-c-f28BZiSEvoxJQvFFxA5T/view?usp=sharing

@lakulish
Copy link
Collaborator

@Denis-Semenovykh Thanks, this video was very useful. It looks like the issue is that the source is inside the door object, so when occlusion is enabled, the door itself occludes the sound.

As a workaround, if only the player can open doors, you could just disable occlusion on the sound. Alternatively, just before playing the sound, you could move its position to be just outside the door, in the direction of the player.

In the meantime, we will work on a proper fix to this issue, which will be part of a future release of Steam Audio.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants