Support NeoGAF

[Afro]

Binaural audio, in simple terms, is a method of recording sound with two microphones arranged in a way that results in an incredibly realistic, immersive 3D sensation, different from your typical Dolby Surround. All you need is a cheap pair of stereo headphones.

I'm baffled as to why sound designers have yet to incorporate this into their games seeing as how quickly and effortlessly the effect immerses the listener. It's almost cheating.

While Dolby Surround allows you to hear a bullet whiz passed your left ear, binaural audio improves upon this even further by allowing you to gauge the actual distance of that bullet pretty accurately, in inches almost.

Unfortunately this does not work with normal loudspeakers. I realize dedicated headphone users are in the minority, but I don't see why they can't add a simple "Binaural audio: On / Off" setting in the options menu, even if they just re-did some of the ambient background sounds such as wind, thunder, etc. I obviously don't expect every sound effect to be re-recorded using this method. The change in quality would immediately be apparent and appreciated even by non-audiophiles.

Not to mention combining this with Oculus Rift.

Youtube examples (use headphones):

Best example
Example #2
Classic example

 

[hbkajshdvfbkjhvgkahsgdfka]

I was wondering why it wasn't used in games after I spent 14 hours watching ASMR videos one day. Shit's cool.

 

[Persona7]

Silent Hill 3 has spooky binaural sound effects.

 

[Takao]

The Corpse Party games on PSP had their audio recorded like this. It enhances the creepyness factor.

 

[Robert at Zeboyd Games]

The Corpse Party games on PSP had their audio recorded like this. It enhances the creepyness factor.

This. Corpse Party was the first time I experienced binaural audio and man was it effective.

 

[GrizzNKev]

Welcome to a major focal point of my education.

Some of my classmates who are working on audio for Rift games have talked about how absolutely disorienting it is when this is missing. Seeing something happen and hearing it from the wrong place is crazy.

 

[orioto]

It's pretty awesome to record music to

My main problem with binaural audio is that i cannot hear anything in the front.

 

[Eric C]

I've often wondered why Binaural audio is not a standard option on portable/handheld games.

Handheld gamers are the ones most likely to be using headhpones, IMO.

 

[Ocaso]

Well, the obvious answer as to why it isn't used is that game audio isn't recorded, so such effects would have to be simulated. This is essentially what classic audio accelerators like Aureal 3D tried to do, albeit to lesser effect. I'm pretty sure the 360 has some implementation of this for their surround sound, but it's obviously not as good as the real deal.

There's also the fact most people don't game using headphones, so the impact would be lessened for the majority of gamers. I'd dig it nonetheless.

A neat example of A3D audio at work. Not bad, actually, though the fact that it's Thief helps.

 

[Spazznid]

I tried this thread a while ago... Everyone hated me....

I'd love a new sound revolution. There was this one dude who managed to get really good results in the source engine, but he never released his work.

there's a video somewhere.

 

[Mindlog]

I blame, Creative. After they killed Aureal EAX went nowhere. I was going to make a very similar thread because of the recent True Audio/Astound Sound whatever developments. Happy to see audio getting a little more attention.

 

[androvsky]

I blame, Creative. After they killed Aureal EAX went nowhere. I was going to make a very similar thread because of the recent True Audio/Astound Sound whatever developments. Happy to see audio getting a little more attention.

Yeah, Creative sued Aureal into bankruptcy (despite Creative ultimately losing), then Creative bought Aureal and promptly did absolutely nothing with the patents. We might have to wait for them to expire for all I know, which would be another 7 years? I mean if Aureal was getting pretty decent real time binaural audio from a consumer-level card back in the late 90's, I'm pretty sure someone could put all that into a USB dongle for a wireless headset for next to nothing now.

 

[commissar]

There was Papa Sangre on iPhone which used binaural audio. Wish I could play it :/

 

[CheesecakeRecipe]

Audio is a huge deal for me and the fact that more games haven't taken advantage of binaural audio pains me. A lot of games barely even have proper sound and have so much that's stationary that using headphones ends up pointing out flaws in the overall mix instead of providing better immersion. But it's not pretty graphics or home theater oriented, so it won't go anywhere.

 

[jediyoshi]

So what's the actual tech currently look like for recreating binaural sound recording setups virtually? I always just assumed it wasn't feasible without some bigger sacrifices because of the logic in how the sound is captured like that in the first place.

 

[TheListener]

Binaural audio (HRTF and all its derivatives) truly is the next generation for audio. The problem lies in the anatomy of everyone's ears. It radically affects the binaural sensation. Some people don't perceive any difference on the same setup where others can experience everything including height (the hardest to convey). There is a lot of research being done here by about every audio company on the planet so consumers should start to hear cool stuff over the next 5 years.

 

[androvsky]

So what's the actual tech currently look like for recreating binaural sound recording setups virtually? I always just assumed it wasn't feasible without some bigger sacrifices because of the logic in how the sound is captured like that in the first place.

I have sitting in my desk drawer an audio card that not only does audio raytracing (currently being promised as a potential PS4 feature with GPU compute), but it also does real time binaural calculations (also known as Head Related Transform Functions, or HRTF). I don't remember how much I paid for it since it's nearly 15 years old.

Granted, in the case of the Aureal 2.0 cards, the tradeoff with the wavetracing and echo calculations is you had to send scene geometry to the sound card, which was not only a performance problem back then, but a bit of a pain from an engine design standpoint. But the binaural part of the output only required 3D coordinates for the sound sources, which is standard in any modern sound engine.

 

[ramine]

It's something that personally I'd love to see developed more. There's a lot of benefits you can get from proper sound design, such as improved reflex times. The main problem is that very few players actually have the setups that could leverage it, especially on mobile.

 

[TheListener]

So what's the actual tech currently look like for recreating binaural sound recording setups virtually? I always just assumed it wasn't feasible without some bigger sacrifices because of the logic in how the sound is captured like that in the first place.

Capturing it is easy. Use a binaural microphone. But recreating it in a virtual environment is expensive to do right. You have to calculate and process each voice with true HRTF encoding. It's easier to do a virtual speaker array but it's way less effective.

If you want to do it virtually you'll need to cut your active voices down to a much smaller number than we usually have in games to make it feasible. Something like a survival horror game would be ideal where the scope is small and focused.

 

[Black_Stride]

Does it have to be headphones, forgive my ignorance.

The test videos sounded pretty damn good on my speakers, but ill try them out with headphones later.

Or maybe its how my placement is?

 

[Yasumi]

The Corpse Party games on PSP had their audio recorded like this. It enhances the creepyness factor.

This so much. That one scene in Book of Shadows in the radio booth legitimately scared the crap out of me.

Spoiler

I'M WATCHING YOU

 

[Arkanius]

I use CMSS-3D
Most OpenAL games have true Binaural Audio (256 sources)

 

[VanillaCakeIsBurning]

Reminds me of the final boss track for Final Fantasy 8. (The part in the beginning)

 

[Ding]

Does it have to be headphones, forgive my ignorance.

The test videos sounded pretty damn good on my speakers, but ill try them out with headphones later.

Headphones are preferred. If you listen via speakers, you will be getting some of the effects twice. Once as it was recorded, and then again as the various phase shifts and filtering happen as the sound hits your real head and ears. Also, the original effect will be badly mangled due to the fact that your speakers aren't perfectly and symmetrically positioned, and by the fact that both your ears are hearing both of your speakers at the same time.

Actually, headphones aren't actually ideal. An in-ear "canal phone" would be best. That way your outer ear doesn't affect the sound again. (But headphones still work well, since generally most of the sound is aiming right into your ear canal.)

I've used an Astro Mixamp with headphones for all my gaming for over a year now. This processes the 5-channel output from my 360 back into an "enhanced" stereo mix with the HRTF stuff applied to add front and back directional cues. It works okay, but not stunningly well. (Even real binaural recording don't work particularly well on me. I must have oddly shaped ears.)

Also, for this to really work, you need to incorporate head tracking. If you're are immersed in a stunningly realistic 360 degree soundscape, it's pretty disconcerting when you turn your head and the entire universe turns with you. This is particularly unfortunate since humans constantly turn or tilt their heads to "narrow down" where a sound is coming from. Not only does this not work with (non-tracked) headphones, the nonsensical cues you do receive act to confuse some of the "good cues" that were mentioned in earlier this thread.

So remember, kids. Hold your heads perfectly still if you want the illusion to seem its most convincing.