flac dev - Oct 2024 - C API: How to get a seektable for very long files?

Hi Martijn,

Am 13.10.24 um 21:00 schrieb Martijn van Beurden:
>
> There's actually quite a lot of documentation for this.
>
> Please review
https://xiph.org/flac/api/group__flac__stream__encoder.html#ga80d57f9069e354cbf1a15a3e3ad9ca78
>
> I quote:
>> SEEKTABLE blocks are handled specially. Since you will not know the
>> values for the seek point stream offsets, you should pass in a
>> SEEKTABLE 'template', that is, a SEEKTABLE object with the
required
>> sample numbers (or placeholder points), with 0 for the frame_samples
>> and stream_offset fields for each point. If the client has specified
>> that it supports seeking by providing a seek callback to
>> FLAC__stream_encoder_init_stream() or both seek AND read callback to
>> FLAC__stream_encoder_init_ogg_stream() (or by using
>> FLAC__stream_encoder_init*_file() or
>> FLAC__stream_encoder_init*_FILE()), then while it is encoding the
>> encoder will fill the stream offsets in for you and when encoding is
>> finished, it will seek back and write the real values into the
>> SEEKTABLE block in the stream. There are helper routines for
>> manipulating seektable template blocks; see metadata.h:
>> FLAC__metadata_object_seektable_template_*(). If the client does not
>> support seeking, the SEEKTABLE will have inaccurate offsets which will
>> slow down or remove the ability to seek in the FLAC stream.
>
>
> Also, take a look at this:
>
https://xiph.org/flac/api/group__flac__metadata__object.html#gab91c8b020a1da37d7524051ae82328cb
>
> Hope that helps.
Thanks, I have looked at the wrong place (at the encoder documentation,
not the metadata).

Is the seektable written at the beginning of the file in the metadata
block or can there also be a second metadata block at the end?

If it's written at the end could I just call
FLAC__metadata_object_seektable_template_append_point() in the encoding
loop? Should the sample already exist at that point or should the
seekpoint appended before that data is passed to the encoder?

If it's at the beginning, would it possible to reserve space for N seek
points and during encoding remember a seek point after X samples,
resulting in M seek points when encoding is finished. If M <= N all seek
points are written, otherwise only every 2nd, 3rd etc.
Is it possible to do that? The functions all expect a a total_samples
argument, which is not known at the beginning.
>
> Also, as I'm always extra curious when FLAC is used for non-audio
> purposes: could you perhaps say a little bit about what kind of
> signals you're compressing?
The signal is the FM-modulated video signal of video tapes (like VHS).
The idea is to capture the signal directly from the video head amplifier
in the VCR and later demodulate/decode it in software, providing higher
quality than traditional capture of analog video. See this project:
https://github.com/oyvindln/vhs-decode/
I started to design a capture device, as there is no 40 MHz continuous
sampling hardware available at consumer prices:
https://github.com/Stefan-Olt/MISRC

Best regards
Stefan Oltmanns

I think there is another major issue for me: In
METADATA_BLOCK_STREAMINFO the field for the length is only 36 bit,
that's not even half an hour at 40 MHz sample rate, resulting in that
the encoder sets it to 0 for longer captures. In the seekpoint the
sample number is 64 bit, which is more than enough.
But how does the decoder handle the seektable when the total number of
samples is unknown? Or does the seektable override the info from
METADATA_BLOCK_STREAMINFO?

I used this functions now to add seekpoints, but all remain placeholders
according to metaflac:

FLAC__metadata_object_new
FLAC__metadata_object_seektable_template_append_placeholders
FLAC__stream_encoder_set_metadata
(encoder init & loop)
FLAC__metadata_object_seektable_template_sort

Best regards
Stefan


Am 13.10.24 um 22:33 schrieb Stefan Oltmanns:
> Hi Martijn,
>
> Am 13.10.24 um 21:00 schrieb Martijn van Beurden:
>>
>> There's actually quite a lot of documentation for this.
>>
>> Please review https://xiph.org/flac/api/
>> group__flac__stream__encoder.html#ga80d57f9069e354cbf1a15a3e3ad9ca78
>>
>> I quote:
>>> SEEKTABLE blocks are handled specially. Since you will not know the
>>> values for the seek point stream offsets, you should pass in a
>>> SEEKTABLE 'template', that is, a SEEKTABLE object with the
required
>>> sample numbers (or placeholder points), with 0 for the
frame_samples
>>> and stream_offset fields for each point. If the client has
specified
>>> that it supports seeking by providing a seek callback to
>>> FLAC__stream_encoder_init_stream() or both seek AND read callback
to
>>> FLAC__stream_encoder_init_ogg_stream() (or by using
>>> FLAC__stream_encoder_init*_file() or
>>> FLAC__stream_encoder_init*_FILE()), then while it is encoding the
>>> encoder will fill the stream offsets in for you and when encoding
is
>>> finished, it will seek back and write the real values into the
>>> SEEKTABLE block in the stream. There are helper routines for
>>> manipulating seektable template blocks; see metadata.h:
>>> FLAC__metadata_object_seektable_template_*(). If the client does
not
>>> support seeking, the SEEKTABLE will have inaccurate offsets which
will
>>> slow down or remove the ability to seek in the FLAC stream.
>>
>>
>> Also, take a look at this:
>> https://xiph.org/flac/api/
>> group__flac__metadata__object.html#gab91c8b020a1da37d7524051ae82328cb
>>
>> Hope that helps.
>
> Thanks, I have looked at the wrong place (at the encoder documentation,
> not the metadata).
>
> Is the seektable written at the beginning of the file in the metadata
> block or can there also be a second metadata block at the end?
>
> If it's written at the end could I just call
> FLAC__metadata_object_seektable_template_append_point() in the encoding
> loop? Should the sample already exist at that point or should the
> seekpoint appended before that data is passed to the encoder?
>
> If it's at the beginning, would it possible to reserve space for N seek
> points and during encoding remember a seek point after X samples,
> resulting in M seek points when encoding is finished. If M <= N all seek
> points are written, otherwise only every 2nd, 3rd etc.
> Is it possible to do that? The functions all expect a a total_samples
> argument, which is not known at the beginning.
>
>>
>> Also, as I'm always extra curious when FLAC is used for non-audio
>> purposes: could you perhaps say a little bit about what kind of
>> signals you're compressing?
>
> The signal is the FM-modulated video signal of video tapes (like VHS).
> The idea is to capture the signal directly from the video head amplifier
> in the VCR and later demodulate/decode it in software, providing higher
> quality than traditional capture of analog video. See this project:
> https://github.com/oyvindln/vhs-decode/
> I started to design a capture device, as there is no 40 MHz continuous
> sampling hardware available at consumer prices: https://github.com/
> Stefan-Olt/MISRC
>
> Best regards
> Stefan Oltmanns
>