Feature request: Option to calculate delta and the delta of the delta

[liudger]

Now the mfcc uses 12 coefficients? Is it possible to calculate also the delta so the coefficients can get to double or triple the amount to have more accurate matching results?

We need to implement Rung Kutta method for this. I am not an expert in programming. Is this doable? Or is the performance hit too high?

https://joinerda.github.io/Solving-ODEs-in-Unity/

What are your thoughts?

[hecomi]

Sorry for the very late reply.

I would like to explore the option of calculating the DMFCC in multiple frames and adding it behind the current MFCC. In that case, it will have 2N+1 dimensional features, where MFCC is N dimensional and DMCC is N+1 dimensional. Currently, N=12 is fixed, so I will consider adjusting this as well.

[liudger]

I could try to clean up the code I now have and do a pull request. Not sure when I have time.

[liudger]

My previous code for calculating DMFCC was not working correctly. I wanted to start a new try. If you have any draft/ideas to share please let me know.

Meanwhile I found there are options for burst compile to improve speed in cost of precision. Did you test these?

[BurstCompile(FloatMode = FloatMode.Fast)]

and

[BurstCompile(FloatPrecision = FloatPrecision.High)]

[liudger]

Hi @hecomi ,

I think I am quite far along to deploy delta calculation. I will make a draft pull request so you could have a look. I hope to have Tuesday some time to make the pull request and add a demo video.
I don't have a solution for the changed profile that needs 24 array length. I had this in my previous code. But it did give some errors (memory leaks) with the different length of profiles.

[liudger]

Quick test. Still some memory leaks to tackle

new_buffer_delta_v2.mp4

[liudger]

every time I create in runtime a new phoneme I get this error:

A Native Collection has not been disposed, resulting in a memory leak. Enable Full StackTraces to get more details.

I shall push the pull request

[liudger]

Memory leaks were solved. Forgot to dispose an offset buffer for managing the memory in the buffer with melcepstrum data.

Pull request created #44