A nod to the late great Prof Hawking, you can print this out if you like and put it on your coffee table to pretend you've read it.

So before I launch into any GG specific sound stuff I am going to cover some of the basic stuff, it is difficult to know where to start so I cover all readers so please forgive me if this seems too much like teaching granny to suck eggs.

Firstly humans hear sound using two 'ears', I know this sounds (pun intended) obvious but we must keep in mind that video games are designed for human beings to play. Some creatures 'hear' sound through their feet (elephants and spiders for example), some through their jaw bones (mainly reptiles) and some lucky ones use their entire bodies (Shark anybody?) but I'll stick to the human method.

The interesting thing about the 'two ear' method of hearing is that although it's obvious how having two ears can help you to figure out whether a sound is coming from the 'left' or 'right' anybody reading this who has two functioning ears will attest to the fact that they can also tell (to some extent at least) whether a sound is 'in front' or 'behind' them!
How on earth does this work you may ask? Well it is very clever, the big flappy things on the side of your head are in fact very complex mechanisms that 'shape' sound in such a way that the other highly complex bit between them (your brain for the slower of you) can detect which direction a sound is coming from (a very important trick when half the creatures you share the planet with are trying to eat you!).
The flappy bit does several things, it changes the frequency of the sound, it alters the volume of the sound and it delays the sound. All of these (very subtle) changes are used by the brain to figure out the position that the sound is coming from.

Anyhow, what we are really interested in is how we can 'fake it' such that sounds we play in a video game are realistic enough to fool the brain into thinking that the sounds the player is hearing are actually coming from the visual entities within said game.
In addition, as we probably don't all possess a hugely expensive sound studio in which we can make these sounds, we need to be able to make the most out of sound samples that we can get from online libraries.

There are basically two kinds of sound sample we can use:
Stereo samples - These are produced by placing a couple of microphones in roughly the positions your ears would be and recording the sound from each into a 'left' channel and a 'right' channel. Again this sounds obvious but the point here is that the position of the sounds recorded wrt to the 'ears' is fixed at the time of recording, this means that these samples when played back will only be realistic if the position and orientation of the player *exactly* matches the position and orientation of the microphones at the time of recording wrt to *all* visible (or logically placeable) sound producers.
Mono samples - These are produced (usually) by placing a single microphone pointing straight at the sound producer and recording on a single channel, in cases where the sound producer is moving these samples can either be made by tracking the microphone with the sound producer or by a static microphone, in the latter case the amplitude and frequency of the sound will change in relation to the movement.

So stereo samples are not very useful except in the following cases:
1) Background atmospheric sounds, i.e. sounds so far away that the player cannot relate them to any visible or logically placeable entities in the level.
2) Mood music, except when that music is coming from (or supposed to be at least) a visible or logically placeable .. well you must be getting the idea by now.
3) Situations where the player position is fixed, for example when getting instructions from a quest giver.

What do I mean by 'logically placeable'? Well the best way of answering this is to give an example; let's imagine you have a factory building in your game and as you enter it there is a entity by the door which should be 'noisy', maybe a generator or air compressor, but when inside the building the player can't actually see this entity. The player will know where sound from the entity should be coming from even after they have entered the building and can no longer see it.

Mono samples are far more useful as they can be attached to an object and the 'stereo' effect will be produced by the engine (GG iow) dependent on player position relative to the entity (with a few restrictions that I will cover in the next post).

tbc

That’s a great explanation Dr Amen ,I think this will get very interesting.i will be following this one ,thanks

Dave

Yes I’m hooked by the slightly acerbic style and the content that is and will be very informative.

Looking forward to more Amen magic and maybe some funky math?

Cheers.

A word of caution; what you're referring to is binaural recording, which is when two mics are placed a head's width apart and usually have some sort of solid surface between them to simulate the blocking characteristics of a human head. There's even mics out there that recreate the complex shape of the ear to better simulate human hearing.



But this method of recording is quite uncommon, consequently binaural recordings are harder to find. A cheaper, smaller, and easier way of recording stereo sound is to use a pair of mics in an XY configuration. These are very popular for recording ambience because the offsetting of the mic directions help create a nice wide stereo image.



There's also plenty of products that package this into a single unit. My Zoom H1, for example, has been a great little entry-level recorder with XY mics.

When looking for stereo sound effects, be aware most are still either recorded with a mono mic and then treated in post to make them sound more stereo, or they are recorded by two or more mono mics if different locations that have been mixed together to create a "stereo" sound.

AE

@AE, As I said I was keeping it simple. What you say is correct, modern audio processing chips allow extremely small microphone arrays (in small Camcorders for example) to be made which give good a stereo effect.

Ok, so what facilities does GG have for adding sounds?

The most obvious is probably the 'Sound Zone' from the 'Markers' tab in 'Entities', and added recently there is now also an 'Ambience Zone'.
Both of these play a sound sample attached to the 'zone' marker, in fact sound samples can be attached to any entity.
The difference between these two methods is that the Sound Zone will play a sound (if it is a mono WAV file) as if it is coming from the centre of the zone, i.e. as a 3D sound, when the player enters the zone. The Ambience Zone will loop the attached sound as if it is following the player around, i.e. as a non-3D sound.
Under the covers all that is happening is that for sounds played with the Lua commands 'PlayNon3DSound' and 'LoopNon3DSound' the sound location is updated to the player position every frame!
You can examine the scripts that are attached to these zones to see the difference.

The positioning of the sound relative to the player is I believe handled by DX, which is probably why it only works for mono WAV files! (I seriously doubt that MS has incorporated ogg support in DX-11)

As explained in my first post, stereo samples already contain the audio positioning cues so DX will play those as is, i.e. they are by default non3D sounds!

The zones are limited to one sound sample but other entities can have multiple samples attached, up to 5 in fact in sound slots 0 to 4.

I will come back to entity attached sounds later but next post I'll look at another method, global sounds.

So I mentioned that a single entity can have 5 sound samples attached to it, but what happens if you want to have more than 5 sounds and rather than have them attached to an entity you want them to be able to play at any point in your game. For example maybe the player is in a space suit and you want a myriad of sounds to play during the game, maybe warning sounds to indicate low oxygen, or radio messages from your ship etc.
Well for that, and far better imo than the non-3D commands, GG provides global sounds.
From what I can see in the code you can have up to 9999 global sound samples loaded, given that GG is 32 bit you would probably run out of memory long before you filled up all the sound slots!
Global sound commands are: (taken from global.lua)
LoadGlobalSound ( filename, iID ) -- where iID is an index greater than zero and filename points to a file in your game installation
DeleteGlobalSound ( iID ) -- where iID is the index of the sound loaded in the load command
PlayGlobalSound ( iID ) -- where iID is the index of the sound to be played
LoopGlobalSound ( iID ) -- where iID is the index of the sound to be looped
StopGlobalSound ( iID ) -- where iID is the index of the sound to be stopped
SetGlobalSoundSpeed ( iID, speed ) -- where iID is the index of the sound to change the speed of
SetGlobalSoundVolume ( iID, volume ) -- where iID is the index of the sound to change the volume of

Also there are a few functions: (These probably all return 1 for 'true' and 0 for 'false)
GetGlobalSoundExist ( iID ) -- where iID is the index of the sound to check existence of
GetGlobalSoundPlaying ( iID ) -- where iID is the index of the sound to check if playing
GetGlobalSoundLooping ( iID ) -- where iID is the index of the sound to check if looping

The index referred to in these commands is simply an integer number, so the first sound sample you load should have an index of 1, the second 2 etc.
To find the first unused index you could use the following code:


Most of these commands are self explanatory but two need a bit of commenting on:
SetGlobalSoundSpeed ( iID, speed ) The speed value here is actually the bit-rate of the playback, for example 44100 for a 44khz sample.
SetGlobalSoundVolume ( iID, volume ) The volume value is 0 - 100, probably intended to be a percentage, but in reality values less than 50 are to low to be heard.

The global sounds are always non-3D so you can use either stereo or mono samples, it will really depend on what you are aiming to represent as to which you should use.

tbc

So I was going to describe yet another method that GG has, RawSounds, then realised I don't actually know myself what they are or how to use them. Even looked into the code to see if I could get some clues but just ended up very confused. (for example there are functions that refer to '2D' sounds as if to infer that there are 3D ones but I can't see anywhere that you can set the position of a 'RawSound'!)

So if anyone else has a clue what they are please feel free to chip in.

Ok, back to entity attached sounds, first thing to mention, and a personal bugbear of mine, is that even if you use a mono-WAV sound sample attached to an entity, once the sound sample starts to play its position is fixed, i.e. if the entity moves the sound does not move with it.
I originally considered this a bug myself and at one point added a PositionSound Lua command to get around it, the command works well but I haven't pushed it on GitHub yet because I would prefer to come up with a better solution.

Now you may be forgiven for thinking it is an easy fix, after all for non-3D sounds the engine already moves them to follow the player so why not do the same for all other entities? Well it's not that simple, think for a minute back to this little demo I did: , the 'boing' sound sample is attached to the balls and simply plays at the position the ball is in when the collision is detected, which if you think about it is correct, in this case we don't want the sound to follow the ball.
So this means we need some way to specify when playing a sound if the sound should be static or whether it should track with the entity, when I've worked out the best way of doing that I'll push a 'fix' (or enhancement?) for this.

In fact when you start to think out your level aurally you will discover that most sounds will have a specific fixed point of origin, it is only things like engine sounds which need to follow the entity making the sound and even then if the player is in the cab you may get a better effect using Global Sounds instead of entity attached ones.

Anyhow, I am going to find some mono sound samples and put together a little demo level to show off positional sounds in GG, then I'll post it here. I'll start off with sound zones to keep it nice and simple.

tbc

Thank you so very much for this explanation.
I have been searching for this info for a long time.
AmenMoses you ROCK!

The attached zip file has a really simple demonstration of positional sounds in GG. The crates are roughly in the centre of the zones to give you some idea where the sound positions are.
Probably best to try this out with headphones on!

One thing I noticed with this, and it is something I've brought up before but not really had an answer about, is that the sound fall off with distance seems to be wrong, not sure if that is down to DX but next post I'll show how a little bit of Lua magic (and some 'Funky' math) can improve matters.

So after you have the stuff from the previous post working and have heard how 'vanilla' GG behaves, try out the attached script.

(then look at the script to see if you understand what it is doing )

Edited to add: I've left in a PrompLocal call so you can a) See where the actual centre of the zone is, and b) see what the calculated volume value is, notice how the volume increase is not linear with the numerical value, more a sort of logarithmic relationship, so it is entirely possible that value being passed down to the audio system is in decibels!

S O U N D ! ! At last!!

HOLY MOSES !! Thank you, Mate! for starting this tutorial. I will definetly dive into my game again these days and redo all the sound-aspects. I really want to get this binaural-script to wotk in my scene properly... I ll probably ask dumb questions within the coming week.
To compensate in advance for those boring ones, I want to enlighten everyone here, that binaural recordings are very well doable with the fantastic and afordable OKM-set-of-microphones. Quite famous little trick to avoid larger and more expensive or bulky methods of immersive audio-capturing. The 2 x directional mics are positioned IN YOUR EARS (one foam is red one is blue, the colours point away from your ear-"hole") , literally giving you the possibility to record BINAURAL audio exactly how YOU perceive it. The quality is really good, crisp and all. The binaural effect just really depends from person to person. Check the audio-examples on their website first. Some people dont get the full surrounding immersion.

I also used them several times as a set of perfectly capturing directional "shotgun"-mics. Just stick them with their cables to something that can be used to point with the membrane at a sound-source (mouths from people talking dialogue usally) and bob's your uncle... The cheaper ones have a GOOD sound and the pricier ones have a GREAT sound, that you will get out of them by using anything proper as amps / converters. I have used them along with my Tascam DR-701D under a DSLR-camera and wearing the OKMs in my ears to get perfect 2 x 2 x channel ambience from nature / outsides sources and the build-in shotgun-mics of the Tascam to capture the perfect focus of the film and the OKMs to get a 3D-imagery of all sounds around me...

They sell it with a little amp(AC3) that you wear in your trouser's pocket ... or you can power them with any portable ZOOM, Tascam or whathaveyou... They can be found under 100,- $ at Ebay and their "Pro"-model costs some 240,- in new condition.

Thomann has them listed under "stereo microphones", but thei're really binaural. You can even buy a wooden artifical head for them (If you dont trust your own nutty shell): https://www.thomann.de/gb/soundman_okm_ii_studio_incl_adapter_a3.htm



FAQ
http://www.soundman.de/en/faq-2/

.. the white head above is NOT the OKM-dummhead. It is this guy over here:

Oh, and of course it makes MUCH sense to have the sounds that are placed somewhere in a scene to produce immersive / binaural sound in MONO, since the calculations needed to make it twirl / move around / behind you, must use DirectX-commands that involve "interaural-shifting" (basically 1. panning, 2. low-pass filtering-techniques and 3. delay-calculations resembling the distance of one ear to the other). In other words: If the sound-source for the binaural-script would be "stereo" the calculations might at least triple or worse in complexity, probably even more and all gets a tad too complicated - which on the other hand is not needed to get the result of convincing "synchresis" in a 3D or movie-scene (synchresis = http://filmsound.org/chion/sync.htm ).

I can only dream of artificial reverberation and delays-effects for large outdoor-mountain-scenes, right?

Why bother with artificial reverb when you can fake the real thing. (still on the basics but I'll get there!)

Anyhoo, those in-ear mics are great for recording sounds for you but other people don't have your ears! Every persons brain is wired to suit their own ear characteristics, basically as a baby develops the brain get's wired up as it learns where sounds come from wrt to the head position, that's why babies have trouble locating sounds at first and then get better at it over time.

It is also worth pointing out that a small percentage of our 'hearing' still comes the reptilian route through our skull and jaw bone, especially those frequencies that the eardrum is less sensitive to.

Thank you Amen Moses! You explain things so well!

Check out this video for Ambeo Orbit a binaural VST plugin for music producers. (the download link appears to not work....)

https://www.audiopluginsforfree.com/ambeo-orbit/

It will make your mixes sound much more vibrant.

scroll down and you will see a new link for it, not the large button but the smaller link below.

Just feed it with a stereo signal, then either widen to binaural or pan to left or up, up and down, for some amazing mixing
possibilities. I use this a lot not, only in a busy mix just to bring out different sounds. Really is a great plugin.

Also

https://store.steampowered.com/app/1071100/Boom_3D/

this is on steam and hooks into Windows really well and it works....... for music playback it is great.....

Right, so as there is a lack of feedback on the little demo I posted (did anyone try it? For those that did was the second script more 'realistic'?) I will just chat a bit about something sound related and how I would go about doing it with GG in Lua.

Let's say you have a cave, a nice big cave, in the cave there is a big stalactite above a pool of clear water, every 6 seconds or so a drip of water falls into the pool.
So how could we make a realistic audible experience to match?

Well one thing we need is the sound of the drip hitting the water, a nice gentle plopping sound, we also need this sound to come from the surface of the pond directly below the stalactite.

Given what I've already posted this should now be an easy thing to do (assuming you've read my particles tutorial you could even add a few particle effects, one for the drip itself, one for the splash and another for the ripple ) .

But, what about the fact that we are in a cave? Shouldn't there also be an echo?
Well yes and you could simply use a sound processing application to add an echo but would it be 'realistic'?
Think about what you would hear if you were really there in the cave, the echo would bounce off the walls nearest the pond and arrive at your ears at a time and volume dependent upon the combination of the distance of the pond from the wall and the distance from that wall to your 'ears'. But it would also bounce off the walls further away in a similar way!
So to make it sound 'realistic' you can't simply use a 'baked-in' echo, if you did the sound would be the same no matter where you were in the cave which the player would quickly pick up as 'false' and could destroy their immersion in the game.

So what are the alternatives?

Well let's go back to basics, what is an 'echo'? In simplistic terms it is a repeat of the original sound delayed in time from the original, coming from a different location (i.e. a wall of the cave) and reduced in volume by some amount which for simplicities sake we can assume to be related to the square of the distances involved.

Now the problem becomes; how do we make a sound come from the right position and the right volume to 'fake' a real echo?

Well how I would do it is to have some invisible entities with the same sound sample attached and position them in real time at the point at which a ray-cast from the sound origin intersects a wall, then at a time determined by the distance from the sound origin plus the players distance from the intersect point trigger the sound at a volume also determined by the distances.

Whew, does that all make sense?

One extra improvement we could make is based on the fact that an echo isn't an exact replication of the original sound, in particular the lower and higher frequencies tend to not reflect as well as the middle frequencies, so if you whistle in a cave the echo comes back sounding slightly lower in frequency and low frequency sounds tend not to echo much at all. We can fake this by simply having two sound samples, the first being the raw 'plop' sound with full frequency range, the second a special 'echo' version which we put through 'compression' processing with say Audacity (other apps are available).

Now all this is just me thinking out loud, whether the result is 'realistic' enough is unpredictable but I can't see any reason why it wouldn't be.

If anyone has a 'plop' sound and would be interested in making a cave level to try this out I will quite happily knock up some scripts to try it out.

To be fair you only posted them 25hrs ago, and it's taken me that long to read the thread

25hrs is a lifetime for us cyborgs!

Here is a sample

Just was able to download and test this for you.
The second script I think works better.
Obviously the helicopter sound would drown out the clock ticking no matter how close you where to the clock in real life. At the positions of the current objects. But editing the script for different objects should be simple enough.
But the fades seems to be working nicely.

You really do some outstanding work with lua. Thank you.

Cant wait to test the echo results. Very interesting concept for trying this out.

http://disq.us/url?url=http%3A%2F%2Fuploaded.net%2Ffile%2F8p9891pv%2FSennheiser_AMBEO_Orbit_1.0.1_WIN.zip%3AJS0QiVyJVraYqGPctfvq7zyHzqw&cuid=1954448

@ AM
This sound like funky math (pun intended).
Nice sound story.

@PCS, I'm glad you posted that as I was going to get around, eventually, to discussing sound samples fromf the interwebs so now is as good a time as any.

Firstly, the sound sample you posted is stereo but what we actually need (as discussed in the thread!) for positional sounds is a mono sample.
Secondly, the sample already has a fair amount of reverb, we haven't really covered this yet but basically the sample already has 'fake' echoes added to give the impression of a sound in an enclosed space. As my whole point was to show how to produce real 'fake' echoes () using simple samples and a bit of scripting this isn't actually that helpful!

Now for a bit of legal stuff, although there are many places on the interwebs advertising 'free' sound samples if you read the fine print on most of them you will find they are licenced for use and although the licences vary it is doubtful you would find one that says something along the lines of 'feel free to change this sound sample in any way you feel fit'.

So although I could put this through Audacity and re-reverb it and make it Mono (amongst other tweaks I would make) I would not feel happy doing so without knowing the source of the sample.

Now as 'reverb' has been mentioned a few times in the thread and some readers may not know what it is:
Any sound produced in an enclosed place will bounce around that space reflecting off of any object in that space, this creates hundreds of 'mini-echoes' that arrive at the ear of the listener at different times until the sound eventually fades away. If you record a sound in a room specially designed to produce no reflected sound at all and then play it back it will sound 'fake' or at the least 'not quite right' because your brain is expecting to hear a fuller 'reverb' sound most of the time.

the sound comes from this website and its really free.
https://freesound.org/people/dobroide/sounds/26266/

And you broke the licence agreement!

"Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use."



Anyhow, as I have now established that editing of the sample is allowed I'll do so for experimental purpose and see how it turns out. There are a few samples on that site with better licencing and which maybe more appropriate so I have a fallback.

Stay tuned!

So I promised some funky math and here it is, this basically attempts to produce 'real' echoes using the technique I outlined in an earlier post.

Have a play around with it and let me know what you think.

Slightly altered version so sound only plays when player is in the 'cave' and lowered the volume of the echoes a bit.

Further improvements that could be made:
1) Increase the number of 'echo producers', i.e. have more ray-casts in between current ones.
2) Calculate echo volume more accurately.
3) Have a second sound sample for the echo that is subtly different in tone (as an echo would be).

That works nice.

Just need someone to make a big nice cave with a little pool and stalactite.

I could even add an echo from the ceiling.

I'll change the script to do 12 echoes, see how that sounds.

(wear headphones!)

So in the video there are 8 side echoes and one from the roof directly above the drip position. I've added a ripple (really need to find a better one though) to show where the imaginary drip is falling.

Oh and as you can see/hear there are two drips.

Would be nice to have a proper cave for this and a stalactite.

amazing thread!

Very nice to read and you explain stuff very well!
This is a MUST READ for everyone... seriously

where the warter rippple script/system come from ?? ant file to download

effectbank\particles\water

The Lua command is TriggerWaterRipple(x,y,z) -- triggers a ripple decal animation at the xyz position

many thanks Amen... i'll give a look