flac dev - Oct 2015 - Supporting 32 bit data

Op 01-10-15 om 18:14 schreef lvqcl:
> Currently libFLAC stores residual signal as 32-bit signed int. And there
> are the following comments in stream_encoder.c:
The residual is stored as a Golomb/Rice code. As far as I know, 
that is not limited to 32-bit in the format itself, only in the 
implementation.

However, there are two residual coding methods now: rice and 
rice2. rice2 was added when it turned out rice (with a 4 bit 
rice parameter) wasn't able to cope with the larger residuals 
that 24-bit material. It looks like a 5-bit rice parameter is 
enough for 32-bit material as well (highest divisor is 2^30, 
whereas a 4-bit parameter is limited to 2^14), but maybe a rice 
partition with 6 bits has to be added, who knows?

Anyway, I don't think the format is in any way unsupportive of 
32-bit int material, it's just the current implementation of the 
codec.

On Oct 1, 2015, at 1:22 PM, Martijn van Beurden <mvanb1 at gmail.com>
wrote:
> Op 01-10-15 om 18:14 schreef lvqcl:
>> Currently libFLAC stores residual signal as 32-bit signed int. And
there
>> are the following comments in stream_encoder.c:
> 
> The residual is stored as a Golomb/Rice code. As far as I know, 
> that is not limited to 32-bit in the format itself, only in the 
> implementation.
> 
> However, there are two residual coding methods now: rice and 
> rice2. rice2 was added when it turned out rice (with a 4 bit 
> rice parameter) wasn't able to cope with the larger residuals 
> that 24-bit material. It looks like a 5-bit rice parameter is 
> enough for 32-bit material as well (highest divisor is 2^30, 
> whereas a 4-bit parameter is limited to 2^14), but maybe a rice 
> partition with 6 bits has to be added, who knows?
> 
> Anyway, I don't think the format is in any way unsupportive of 
> 32-bit int material, it's just the current implementation of the 
> codec.
I've been meaning to take a look at the header format to see whether 32-bit
would confuse old decoders, and today's messages reminded me to do that.

METADATA_BLOCK_STREAMINFO allows "bits per sample" to range from 4
bits to 32 bits. That means it should be much less complicated to support 32-bit
encoding without completely changing the format.

Decoders already have to check these headers to compare what they support
against the specifics of a given file. I don't think that error reporting is
very good on embedded FLAC implementations - they usually silently fail to play
if given an encoded file that they can't handle. Adding 32-bit support
should not make this any worse than it already is.

In contrast, attempting something crazy like support for 48-bit samples would
either completely break the format for backwards compatibility, or at the very
least would require using some of the "reserved" field values to
extend the headers in one way or another. I don't recommend going down this
route unless there is an extremely compelling reason.

A minor headache is that the FRAME_HEADER does not support 32-bit sample sizes
on its short list, but there is an option to defer to the STREAMINFO for sample
size. That might pose a challenge for 32-bit files. Not sure yet.

Brian Willoughby
Sound Consulting

Martijn van Beurden wrote:
> Anyway, I don't think the format is in any way unsupportive of
> 32-bit int material, it's just the current implementation of the
> codec.
Actually I'm more concerned about the current libFLAC implementation.
It looks like proper (fixed and lpc compression) support of 32-bit
input requires significant changes in libFLAC. The only easy way to
add 32-bit support is to allow only constant and verbatim subframes,
which means no compression.

Op 02-10-15 om 20:00 schreef lvqcl:
> It looks like proper (fixed and lpc compression) support of 32-bit
> input requires significant changes in libFLAC.
Sorry for my ignorance, but wouldn't it be as simple as 
duplicating every function working with the residual to work 
with a 64-bit int variant instead? I know it's pretty wasteful 
considering binary size, but it seems like the only solution 
that would in no way slow down encoding of 16/24-bit material 
but still being able to handle 32-bit material with minimal effort.

If you release a libFLAC version with only verbatim encoding but 
introduce lpc compression in a later stage, this might give 
crippled 3rd-party decoder implementations as soon as 
compression is introduced as bugs are revealed. On the other 
hand, 32-bit material will probably contain 18 bits of noise at 
any given time: there's not much to compress either way.