Displaying 1 result from an estimated 1 matches for "lpquantiz".
Did you mean:
llquantize
2007 Oct 14
0
format spec unclear on how to quantize the LPC prediction
Hello!
I just noticed how the LPC prediction is quantized by browsing the
source code of the java port jFlac:
:
int sum = 0;
for (int j = 0; j < order; j++)
sum += qlpCoeff[j] * data[startAt+i-j-1];
data[startAt + i] = residual[i] + (sum >> lpQuantization);
:
I checked the C version (libFLAC) and it matches. However, it was a
surprise because this way of quantizing the prediction introduces a bias
towards negative infinity which results in residual samples that have a
bias towards positive infinity and therefore a shifted probability
distribu...