I’m an advanced programmer and game programmer, but I’m completly newbie with any kind of sound. So, I’m looking for a way to play a sound on BGE (using python) and get the spectrum + the current time of the sound… Or some idea to help me to recognize what is going on in the sound to program a lip sync method.
I’ve in mind how I’ll program this (the lip sync) this is not what I’m asking here, but I’ve to know a little bit more about the sound player on BGE…
Actually there is no built in FFT in Blender (except getSpectrum, but it needs fmod to be installed and used).
You can use numpy or other libraries to get it working.
I actually used python wave module to make something like this:
import wave
from array import array
from bge import logic
bands = 100 #number of lines
accuracy = 1
scene = logic.getCurrentScene()
def bounce(cont):
own = cont.owner
if "init" not in own:
own["init"] = True
own["pos"] = own.worldPosition.x.copy()
s = list(logic.amp)
dynamic = abs(sum(s)/(len(s)**2.5))*2
desired = abs(logic.amp[-int(own.worldPosition.x*1/accuracy)-1])/100/(own["pos"]/5)+dynamic
z = own.localScale.y
if z < desired:
own.localScale.y += (desired-z)/25
elif z > desired:
own.localScale.y -= (z-desired)/25
def start(cont):
own = cont.owner
for i in range(1,bands+1):
own.worldPosition.x = i*2
newObj = scene.addObject("Cube", own, 0) #The cube in a hidden layer will have an always sensors
#with true pulse mode eabled running the bounce function above^^^
def amplitude(own, wav, t):
frames = wav.getnframes()
rate = wav.getframerate()
head = int(t * rate)
delta = head - wav.tell()
if delta < 0:
delta = head
wav.rewind()
chunk = array('h', wav.readframes(delta))
return chunk if chunk else [1,1,1]
def analyse(cont): #Use a master object for this. Must have pulse mode running it.
own = cont.owner
s = cont.actuators["Sound"]
if not "init" in own:
music = own["music"] #put the file of the song in a string.
if not music.endswith(".wav"):
import subprocess
subprocess.call(logic.expandPath("//")+"ffmpeg.exe -y -i \""+music+"\" \""+music.split(".")[0]+".wav\"") #convert the non wav sound to wav, using ffmpeg.
own["music"] = music.split(".")[0]+".wav"
music = own["music"]
own["init"] = True
s.sound = aud.Factory.file(music)
s.mode = 4
cont.activate(s)
logic.wav = wave.open(music)
logic.power = 1
logic.soundTime = s.time
logic.amp = amplitude(own, logic.wav, logic.soundTime)
It should work probably with a bit of jiggling around perhaps because I modified it to be simpler than what I used.