Does anyone have a background in signal processing (DTFT, IIR filters) esp in relation to audio processing? I have an idea but need to bounce some ideas off someone so I don’t waste my time walking down a dead end.
What are your ideas?
Supposing I wanted to mimic someones voice – say Laurence Fishburne. I could take a voice sample of, say, 1 minute, and grab the frequency spectrum. Grab the largest 4 or 5 peaks. This is the basis for his voice. Do the same for my voice.
Now, I speak a specific sentence, paragraph, or speech. Every instant in time, my voice will deviate from the basis. If I superimpose that deviation (percentage-wise) on top of Laurence Fishburne’s basis – will I wind up with Laurence Fishburne’s voice, but the speech I made?
The problem isn’t just the frequency that Lawrence speaks at, it’s the way he speaks. Tempo, intonation, emphasis and pauses. It would be like wearing his clothes but not walking the same way.
I think you would need a larger frequency sample than one minute – I would say between 5 and 10 min of his speech would give you a good basis to start with. Then with some adjustment you should be able to closely match his voice.
That’s what my voice would be for – as my voice deviates from it’s basis – (fundamental frequency shifts in speed and magnitude, secondary does the same) that deviation would be done to Laurence Fishburne’s
Example…
LF has a fundamental freq. of 420 Hz @ 1 unit amplitude, secondary freq of 610 @0.3, so on so forth
I have fundamental of 515 Hz @ 1unit, secondary of 608 Hz @0.2 etc
As I say a specific phrase – “jump the moon!” at a certain instant my fundamental is 550 @ 1.7 units, secondary is 602 @ 0.4 units
Therefore, my fundamental shifted by approx a factor of 1.08 in freq. and 1.7 on magnitude, secondary shifted a factor of 0.98 in freq and 2 in magnitude–
So at that instant – I shift LF’s voice – his fundamental goes to 420*1.08 = 453 Hz @1.7 units amplitude, secondary goes to 604Hz @0.6 units –
Except do that for more of the peaks (maybe 5 or 6) – thereby mimicking (not perfectly, but something) his tone and tempo and such
It should work (in theory) but you might need to adjust it and clean the voice signal to accurately match his voice. I have a feeling it’s not going to be as straightforward as you’re thinking.
There’s a lot more going on than just stuff in the frequency domain. Theoretically you could do it with a convolution filter, but the problem is that there really isn’t a clean way to create an impulse sample of the human voice like you can with a physical space or DSP equipment.
I was kind of thinking like a moving modulation filter. It would be crazy CPU intensive, but for each sample, grab the FFT of the surrounding 50 ms, divide it by the basis spectrum, then multiply the target basis spectrum
my voice is c(t) -->> C(f) (the basis frequency spectrum, an FFT of perhaps 5 minutes of random talking)
Reference voice (Laurence Fishburne or whatever)is r(t) -->> R(f) (the refererence spectrum, 5 mins of him doing random talking)
Then I say my line “Jump the moon!”
this is x(t) (or x(k) since we are talking digital)
for each sample in my line (x(t))
FFT{ x(t-50ms) to x(t+50ms)} —>>> X(f)
modulation function M(f) == X(f)/C(f)
output Y(f) == M(f)*R(f)
output sample y(t) == IFFT{Y(f)}
end for