flac dev - Feb 2005 - Lossless audio codecs for video editing

Christian HJ Wiesner a ?crit :
> I was wondering how suitable both are with respect to video editing, 
> especially if it comes to sample precise cutting. What is the absolute 
> minimal number of samples per frame in a FLAC stream ? Can we change the 
> number of samples in the same track ?
> 
> Hopefully you understand where my question is coming from, we normally 
> cant cut audio streams in between frames during the video editing 
> process, but its not given in any case that frame boundaries of the DV 
> video stream will match those of the audio stream, and this could lead 
> to problems. For this reason we will concentrate on the one lossless 
> audio format for the DV editing that will allow us the finest sample 
> size, for precise editing.
Wavpack4 can do that fine.

Hi,

this email is sent to flac-dev AT xiph DOT org, copied to matroska-devel 
AT lists DOT matroska DOT org, and David Bryant, the author of Wavpack4.

We are about to implement the DV ( Digital Video ) video compression 
standard into matroska, to be able to offer an intelligent alternative 
to AVI for DV handling in editor tools. DV is using a high quality, high 
bitrate lossy video compression similar to MJPEG, so there are no I/P/B 
frames, you can cut the DV AVI wherever you like. The sound is typically 
PCM, and in a type 1 DV AVI the video and audio will form a single 
stream ( like in the DV format itself ). As most AVI editors cant handle 
a stream containing  audio and video, there is also type 2 DV AVI, where 
the PCM audio stream is repeated in a second track, to make it 
accessible. I heard Virtualdub has recently added support for type 1 DV 
AVI, but i couldnt verify this.

Our goal is to allow the use of lossless audio with a single DV video ( 
PCM audio stripped ), to save HDD space during the editing process. MKV 
has perfect support for FLAC since more than a year, and we are about to 
add Wavpack4 support right now.

I was wondering how suitable both are with respect to video editing, 
especially if it comes to sample precise cutting. What is the absolute 
minimal number of samples per frame in a FLAC stream ? Can we change the 
number of samples in the same track ?

Hopefully you understand where my question is coming from, we normally 
cant cut audio streams in between frames during the video editing 
process, but its not given in any case that frame boundaries of the DV 
video stream will match those of the audio stream, and this could lead 
to problems. For this reason we will concentrate on the one lossless 
audio format for the DV editing that will allow us the finest sample 
size, for precise editing.

I know we could try to dig this all up from your docs, but maybe you 
have an idea how we could best use your codecs for this purpose ?

Christian
matroska project admin
http://www.matroska.org

sorry about the delay... this it the first I'm seeing the message,
maybe it got stuck in moderation because of the cross posting.

--- Christian HJ Wiesner <chris@matroska.org>
wrote
> I was wondering how suitable both are with respect to video editing, 
> especially if it comes to sample precise cutting. What is the
> absolute  minimal number of samples per frame in a FLAC stream ?
minimum is 1 sample per frame.
> Can we change the number of samples in the same track ?
do you mean can the number of samples vary from frame to frame?

if you are putting it in matroska and the container is managing
the sample numbering and seeking, the answer is an easy "yes".

if you are relying on libFLAC to tell you the sample number for
a frame: due to a flaw in the native codec design, if you vary
the number of samples per frame it will not always be possible
for the decoder to give an accurate sample number.  then seeking
through libFLAC will also be a problem.

the flaw is that usually the frame number is encoded in the frame
header, not the sample number, so to get the sample number you
have to multiply the frame number by the block size.  it is
possible to encode the sample number in the frame header but
there is not a bit in the header that tells you which way it is
encoded, you have to infer it from other data that is complicated
and flawed.  and libFLAC only uses fixed blocksizes and only
writes frame numbers, so if you are editing such a thing in such
a way that you cut the first frame, the sample number calculation
will be off by the number of samples cut.

from this you can see though that if you are using your container
for the sample numbering and seeking, and ignoring libFLAC's
sample number there will be no problem.  however if the stream
is exported to a native FLAC stream it will have to be reframed
which unfortunately means reencoded (unless/until I get to fixing
the format).

I am contemplating fixing this problem in the format eventually
(along with others) but older decoders will not be able to read
such streams, so if I do it, it will have to be done carefully
to avoid disrupting users.  I would bundle it up with other
format changes and compression increases.

Josh



	
		
__________________________________ 
Do you Yahoo!? 
Yahoo! Mail - You care about security. So do we. 
http://promotions.yahoo.com/new_mail

On Fri, Feb 11, 2005 at 11:57:58AM -0800, Josh Coalson
wrote:
> I am contemplating fixing this problem in the format eventually
> (along with others) but older decoders will not be able to read
> such streams, so if I do it, it will have to be done carefully
> to avoid disrupting users.  I would bundle it up with other
> format changes and compression increases.
Would it be possible to add to your list a modification of partitioned
rice coding? Something like this:

 RESIDUAL_CODING_METHOD_PARTITIONED_RICE

   <4>              Partition order.
-  RICE_PARTITION+  There will be 2^order partitions.
+  RICE_PARTITION+  There will be
+                   (frame's blocksize - prediction order + s - 1) / s
+                   partitions, where s = frame's blocksize / 2^order
+                   is partition size.

 RICE_PARTITION

   <4(+5)> ...

   <?>  Encoded residual. The number of samples (n) in the partition is
        determined as follows:
 
        * if the partition order is zero, n = frame's blocksize -
          predictor order
        * else if this is not the first partition of the subframe,
          n = (frame's blocksize / (2^partition order))
-       * else n = (frame's blocksize / (2^partition order)) -
-         predictor order
+       * else n = (frame's blocksize - prediction order) % s


It would not break decoding of old files and it would allow:

* partition size <= prediction order, not really useful though
* reasonable partitioning for frames with blocksize != 2^something,
  would be good for variable blocksize coding

-- 
Miroslav Lichvar