Hey everyone,
I thought we could start a new topic for this. I've been looking up some stuff in my spare hours today regarding ways to input WAV/MP3 files, etc, and be able to analyse them in realtime, recognise beats, etc.
This is my simplistic (having not done anything like this before) view of how it would be done:
* Assuming the music tracks wouldn't be stop/started during play, the track can be parsed while each level loads, so not all the work need be done during play.
* A common method for making a simplified model of the audio waveform seems to be Fourier Transforms, with 'Fast Fourier Transforms' (FFT's) being a more simplified but... well, faster... method (ie realtime i believe). Fourier Transforms basically make an approximation of a complex waveform of the actual audio, by adding together a specific number of sine waves of differing wavelengths and magnitudes.
* Analysis of the FFT's allow recognition of spikes in audio in certain wavebands and at certain volumes.
I've added some links here for reading and discussion. I'm still looking into a good way for us to use it, but I've heard it mentioned as having been done using FMOD.
Some of the link contents are fairly simplistic, but will help you get the basic picture. There's some good stuff in the rest, and I'm still yet to finish reading read all of it.
Fourier Transforms:
* http://www.relisoft.com/Science/Physics/sound.html
* http://musikality.net/general/audio-waveforms-fouriertransform/
* http://resonanceswavesandfields.blogspot.com/2009/01/spectrum-of-wavefo…
Reading/Writing WAV files:
* http://www.codeguru.com/Cpp/G-M/multimedia/audio/article.php/c4739/
* http://people.msoe.edu/~taylor/examples/wav.htm
Beat Recognition:
* http://www.transcendentaxis.com/dthompson/blog/archives/14 (this one uses Actionscript, but still good to read).
* http://www.gamedev.net/reference/programming/features/beatdetection/
Anyways, have a look if you like, and let me know anything else you've found. This may not even be the best way to go, but in my searching today it seemed to be a commonly-used method. Hopefully I'll be able to finish reading them all tomorrow.
I'll see if i can dig up my uni notes on Fourier Series, but there may already be functions to generate some of this without us having to delve lots into the fourier side of the maths at least.
Cheers,
Craig.
another link
Just found another similar link on GameDev, where someone commented that creating a 'beat' file for each music track would be much better (though far less preferable for our purposes).
Let's look into it, but they seem to think that a suitably accurate algorithm is either not possible, or will be to CPU intensive anyway.
http://www.gamedev.net/community/forums/topic.asp?topic_id=473462
I'll see if i can get a look at FMOD tonight to see what it has to offer.
Just wanted to say
Good work guys. I expect me and Jarrod will use your research here for the programming considerations section of our musically inclined game proposals.
I've found out pretty much the same thing. I hadn't found any examples of actually implementing the FFT though. (I'm at work so haven't had time to go through your links).
Audiosurf parses the track and builds the level during level load too. Not sure if they use FFT's though, all I could really find mentioned was "Frequency Analysis".
If pausing the game and the sound occur instantaneously I can't see a problem with losing sync.