Binaural recording in Amnesia

[Luis]

Hi everyone! My sensors detected some inconsistencies posted in this thread and waked me up from my long sleep. Arrr.

@Ramsey: as the person in charge of being like all over sound technologies 3 years ago or so (that's been quite a while by the way), I feel the need to take you back on the right track

First of all, EAX has pretty much nothing to do with binaural sound recording or the like. It is actually a set of DSP (Digital Signal Processing) effects to be used in real time, which is also cool, but not even close.
Second, EAX 5.0 is by no means new, it has been around for a while actually (since around 2005). Also, it is not that nice, since there are very very few cards supporting it. Wikipedia even states it is being deprecated in terms of API, as it's functionality is being covered by the OpenAL's EFX extension.
Another nice thing you might not know, the HPL Engine supports EAX through EFX since Penumbra: Black Plague. You can actually find Black Plague listed in the old official EAX site, look in "EAX Games list", "N to S" (I just learned this ). All this means we can add it to Amnesia at any time, but still it's not binaural stuff. Anyway, Amnesia sounds awesome as is right now.

As Thomas said, we could look into getting this thingy or at least a similar one into the engine at some point, but this won't happen in a near future I'm afraid.

I hope I've been clear enough

[Tayrtahn]

It is definitely possible to implement this type of audio system in a game. You would have two "listener" objects in the world to represent the player's ears. When a sound is made in the game world, both listeners record their distances from the source of the sound - this information is used to slightly offset the playback of the sound in either the left or the right channel. Putting that together wouldn't be unreasonable, though it would definitely require more memory and CPU usage. Most developers opt for graphical improvements instead.

It's starts to get tricky when dealing with longer sounds though. Say you have a character speaking to the player in your game - the player is able to move and look freely while that character is talking. When the character starts speaking from a point to the right of where the player is looking, we measure the distances to the player's virtual ears and delay the playback of bob_speech01.ogg for the left channel by the appropriate amount. But now the player turns to look in the opposite direction. We have to slow the rate (but not the pitch) at which the right channel is playing while speeding up (but still maintaining the same pitch) the left channel. And in order to be convincing, that all has to be very smooth, which requires constant recalculation. Calculating all of that at least once per frame is bound to use up a lot of resources. It'd be cool if sound cards started implementing this themselves to take the load off the CPU.

For now, most games settle for panning the sound between the left and right channels to simulate direction. It's a reasonably good effect, and (at present) upgrading to true binaural sound is not enough of a benefit to justify the processing demand. Ah well.

[Adventurer4Life]

headphones are lame. Needs to work with surround sound speakers

[Talraega]

That was... Just freaky. I've never seen one of these recording videos before, and it just felt... So real. I actually reached up and felt if someone was actually cutting my hair. o_o

I understand the sound system most games employ work pretty well now, but... Really, that would be astounding to see this type of recording used in an FPS or something. That would be beautiful immersion...

Now all we need are those visors we see in movies about virtual reality games, and we could have some incredible stuff. GET ON IT, FUTURE. I'M COUNTING ON YOU.

[Tzajof]

(04-20-2010 09:38 PM)Tayrtahn Wrote:  It is definitely possible to implement this type of audio system in a game. You would have two "listener" objects in the world to represent the player's ears. When a sound is made in the game world, both listeners record their distances from the source of the sound - this information is used to slightly offset the playback of the sound in either the left or the right channel. Putting that together wouldn't be unreasonable, though it would definitely require more memory and CPU usage. Most developers opt for graphical improvements instead.

About the only problem I see there is that from my teensy experience with sound programming (OpenAL, just recently) it might not always be possible to have two listener objects within the same context. Or at least, in my OpenAL use so far I haven't seen anything regarding the creation of a second listener. That's not to say it's not possible, but it might not be a typically supported feature since there usually probably isn't much need for a second listener, not to mention of course the CPU problems you outlined.

Otherwise though, I agree with your post- it's more or less what I was thinking when I first read this thread.

[Adventurer4Life]

while your all wowing about that youtube demo... it is worth noting that youtube for streaming reasons collapses everything to mono....