flac dev - Jun 2014 - Residual bps and encoding speed

On Mon, Jun 30, 2014 at 03:27:18AM +0400, lvqcl wrote:
> lvqcl wrote:
> > FLAC 1.2.1 and 1.3.0 cannot encode snippet6.wav with -7 and -8
encoding modes.
> > But they are able to do this with --disable-fixed-subframes option.
This
> > implies that snippet6.wav triggers a problem somewthere inside
> > FLAC__fixed_compute_residual(data[], data_len, order, residual[])
function.
Interestingly, -8 -l 0 seems to avoid the encoding problem too.
> > And indeed, max. possible residual value is 16 times bigger than max.
value
> > in data[] array:
> >
> >      residual[i] = data[i] - 4*data[i-1] + 6*data[i-2] - 4*data[i-3] +
data[i-4]
> >
> > 16 == 2^4, so max. bps value for residual[] is equal to max. bps for
data[] + 4.
> > The value of FLAC__MAX_EXTRA_RESIDUAL_BPS == 4 is enough to fix this
problem
> > with FLAC__fixed_compute_residual().
> 
> 
> On the other hand, it is possible to set FLAC__MAX_EXTRA_RESIDUAL_BPS
> to 0, and change evaluate_fixed_subframe_() function instead: in the call
> of find_best_partition_order_() function, replace subframe_bps
> with subframe_bps + order.
Adding the extra bits to bps in evaluate_fixed_subframe_ instead of
precompute_partition_info_sums_ is ok with me, that's what I suggested
in the original thread discussing this problem, but I think it should
be done also in the lpc function. Adding order instead of 4 might work
for fixed frames, for LPC it looks too pessimistic.
> ...And maybe it's also better to use 'subframe_bps + order'
instead of 'order'
> in calculation of 'estimate' variable in
evaluate_fixed_subframe_()?
Why exactly?

To me it looks like (order * subframe_bps) in the calculation is the
number of bits for the warmup samples in the fixed subframe.

-- 
Miroslav Lichvar

Miroslav Lichvar wrote:
> Adding the extra bits to bps in evaluate_fixed_subframe_ instead of
> precompute_partition_info_sums_ is ok with me, that's what I suggested
> in the original thread discussing this problem,
found it: http://lists.xiph.org/pipermail/flac-dev/2013-July/004303.html
> but I think it should
> be done also in the lpc function. Adding order instead of 4 might work
> for fixed frames, for LPC it looks too pessimistic.
I'm OK with any version.
> To me it looks like (order * subframe_bps) in the calculation is the
> number of bits for the warmup samples in the fixed subframe.
Thanks!

I changed the condition in *_precompute_partition_info_sums_*()
functions from
     if(bps <= 16)
to
     if(FLAC__bitmath_ilog2(default_partition_samples) + bps < 32)

and then changed the 'subframe_bps' argument of
find_best_partition_order_()
in evaluate_fixed_subframe_() and evaluate_lpc_subframe_() as follows:

    evaluate_fixed_subframe_():    evaluate_lpc_subframe_():
1) subframe_bps + order           subframe_bps + 0
2) subframe_bps + order           subframe_bps + 2
3) subframe_bps + order           subframe_bps + 4
4) subframe_bps + 4               subframe_bps + 4

Version 0 hangs during encoding. So Miroslav was right, it is necessary
to increase bps for lpc subframes as well. Versions 1...3 have identical
encoding speed. So it's possible to play safe and just choose
"subframe_bps + 4" in both cases. This is equivalent to the patch in
http://lists.xiph.org/pipermail/flac-dev/2014-June/004771.html
(this patch also adds 4 to subframe_bps, but in different place).

So I think it's a matter of taste which patch to prefer: this --
http://lists.xiph.org/pipermail/flac-dev/2014-June/004771.html
or this --
http://lists.xiph.org/pipermail/flac-dev/2013-July/004303.html