flac dev - Apr 2020 - identical audio but not identical unrecognized chunks

Dear all,

I've converted a wav file to flac but during the process three wrnings 
were casted. One of them I recall, it said that a BEXT chunk is 
unrecognized and thus ignored.

Wouldn't it be posible to keep the unrecognized information as is when 
decoding the file?

In general the extra chunks are located before or after the audio 
content, and represents a tiny part of the whole file, so it oould 
affect only very slightly the compression ratio attained.

Regards,

Federico Miyara
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.xiph.org/pipermail/flac-dev/attachments/20200429/b78908ca/attachment.html>

On Apr 29, 2020, at 4:46 PM, Federico Miyara <fmiyara at fceia.unr.edu.ar>
wrote:
> Dear all,
> 
> I've converted a wav file to flac but during the process three wrnings
were casted. One of them I recall, it said that a BEXT chunk is unrecognized and
thus ignored.
> 
> Wouldn't it be posible to keep the unrecognized information as is when
decoding the file?
> 
> In general the extra chunks are located before or after the audio content,
and represents a tiny part of the whole file, so it oould affect only very
slightly the compression ratio attained.
> 
> Regards,
> 
> Federico Miyara
Federico,

flac compresses an audio file by using characteristics of the audio waveform.
The algorithm cannot compress non-audio data. You probably already knew this. I
just wanted to point out that flac completely ignores the non-audio data, so it
does not even factor in to the operation, whether there are warnings, BEXT
chunks, or not.

However, I suggested to the team many years ago that FLAC APPLICATION metadata
blocks, which are intended for third-party applications, could be used for
specific audio file formats such as AIFF and WAVE in order to preserve the
non-audio chunks. As you mentioned, these extra chunks represent a small
fraction of the total size, so adding them does not have much affect on the
final file size.

I am happy that the suggestion was taken seriously and implemented. You will
find that the flac command line supports an option `--keep-foreign-metadata`
which does exactly what you seem to be suggesting, and it has been part of flac
for many years. One of the things we discussed was the fact that some chunks
occur before the audio chunk, and some occur after, so preserving the order
might have been difficult. I forget the details of the discussions, but I
believe that we eventually decided on a technique that also preserves the order
of non-audio chunks relative to the audio chunk. This feature treats AIFF
differently from WAVE, because it's not really possible to distinguish audio
from non-audio without knowing what format the file is in.

A minor caveat is that you'll see a warning when using
`--keep-foreign-metadata` - because this is a relatively new feature, the
original designer of flac decided to let users know that it might not work
perfectly. In my experience, it has always worked.

Another caveat is that you cannot combine `--delete-input-file` with
`--keep-foreign-metadata` - the reason being that it's not necessarily safe
to delete the original in the same command that you're compressing with a
new feature. In these cases, I just manually delete the original file after
compressing.

Yet another issue is that you cannot compress a WAVE file with
--keep-foreign-metadata and then uncompress it into AIFF format while expecting
the non-audio data to be preserved. flac cannot convert non-audio data from one
format to another. Similarly, you cannot compress an AIFF file with
--keep-foreign-metadata and then uncompress it into WAVE format while preserving
non-audio data. flac is perfectly capable of translating the audio between AIFF
and WAVE, but the metadata will get lost.

As a side-note, my mobile recording gear puts time stamps and other information
into the BEXT chunk, and I usually want to preserve that. flac
--keep-foreign-metadata does that perfectly.

On another note, there are applications like Logic Pro that add considerable
amounts of non-audio data to a chunked file - typically these are graphical
waveforms of the audio - and you probably do not want to waste the space needed
to preserve these large chunks. My habit is to back up the original audio
recordings with flac, using --keep-foreign-metadata, before loading them into
Logic where the extra chunks will be added. This gives me the archive that I
need without wasting extra space. Logic Pro will automatically add the waveform
image to a file that's been freshly restored from a flac backup.

Brian Willoughby