Vorbis dev - Feb 2006 - padding in comment header

Having looked through the archives I've come to the conclusion that
to do padding for comments (in a file tagging context) the simplest
solution is to zero pad the comment header packet to fixed length[1]
and regenerate pages from #2 up to the first one containing the setup
header. Is this correct?

Not very closely related: should doing ogg_stream_pagein followed
by while(ogg_stream_pageout()>0) on a stream work (I could check
the code, but I'm sure someone here will just know), or must page
in/out always be followed by the complimentary packet in/out?

[1] i.e. some stepped length depending on the size of contents you'll
have; along the lines 325 bytes content => 512 bytes of comment header,
attempt to get 513 bytes content => 1024 bytes of comment header
(change => rewrite).  With possibly the default choice being a rewrite if
the header will be ridiculously sparse.

--
imalone

On Fri, Feb 03, 2006 at 03:36:17PM +0000, Ian Malone wrote:
> Having looked through the archives I've come to the conclusion that
> to do padding for comments (in a file tagging context) the simplest
> solution is to zero pad the comment header packet to fixed length[1]
> and regenerate pages from #2 up to the first one containing the setup
> header. Is this correct?
Well, I don't know about "simplest". If you're shortening an
existing
comment packet, it makes some sense to pad out to the original length 
because you can just update the ogg page crc values in place without
having to rewrite the rest of the file. If you're lengthening a comment 
packet or writing a newly encoded file, adding some padding after the 
end of the comment packet makes it easier for tagging applications to
make small changes without rewriting the entire file. And if you're 
going to do that the "round up to the nearest power of two" heuristic
is
a fine one. (I'd suggest "over 1024 bytes and more than half
padding" as
a definition of "ridiculously sparse" btw.)

Whether this is the simplest solution? I don't know. All our tools 
embed exact-size comment header packets, and any tagging software
has to handle the re-write case, so this is more of an optimization
than a simplification. Is making retagging faster worth wasting a
few hundred bytes per file? If I'm thinking of files on disk, I'd
say yes. If I'm thinking of streaming (as people usually have been
when developing vorbis) or files on disc, I'd say no.

The spec doesn't say anything about the padding either way, but it
should be legal and I'm not aware of a tool that doesn't handle it
properly. So if you want to do this, please go ahead.

As far as page regeneration, if you don't change the number of pages,
regenerating just the pages that contain some fraction of the comment
header works. If you have to change the number of pages, you have
to update the page number (and crc) of every subsequent page in the
stream.

Note that vorbiscomment repages everything. It started as laziness, but 
turns out to be a helpful normalization tool too.
> Not very closely related: should doing ogg_stream_pagein followed
> by while(ogg_stream_pageout()>0) on a stream work (I could check
> the code, but I'm sure someone here will just know), or must page
> in/out always be followed by the complimentary packet in/out?
It *should* work, but I don't know. I wouldn't be too surprised if
it buffered data forever, waiting for a corresponding packet_out, 
but that would be a bug. What are you trying to do here?

 -r