I haven't actually had many questions up to now. Despite the all-encompassing CP_INSTANCE monolith, things have been relatively easy going. Now we get into real grit: First off, CBitman looks like it has endianness issues; it's packing into host-order 32 bit arrays (the comments and symbol names seem to indicate that this was originally byte-based packing code that got upped to 32 bit. Either that or someone is a bit confused about a byte being 32 bits...) Whose responsibility is it to flip bytes before putting out a stream? Is native order big or little endian? I'm seriously considering tossing Ogg's bitpacker into the codec as it has defined endianness behavior and is already used throughout Ogg (the API is very similar). Also, this is probably my own confusion, but is it really the case that the encoder app has to guess about packet size and just hope the output buffer doesn't overflow? The Ogg bitpacker would eliminate this problem as well. Monty --- >8 ---- List archives: http://www.xiph.org/archives/ Ogg project homepage: http://www.xiph.org/ogg/ To unsubscribe from this list, send a message to 'theora-dev-request@xiph.org' containing only the word 'unsubscribe' in the body. No subject is needed. Unsubscribe messages sent to the list will be ignored/filtered.
> First off, CBitman looks like it has endianness issues; it's packing > into host-order 32 bit arrays (the comments and symbol names seem to > indicate that this was originally byte-based packing code that got > upped to 32 bit. Either that or someone is a bit confused about a > byte being 32 bits...)Ah, Write32ToBuffer is doign the flipping. What an odd little bitpacker. (At one point I wrote a 32 bit version of the Ogg bitpacker that did byte flipping when necessary, and it turned out to be slower than the 8 bit packer we use now. Also, you have the lowest level bithandling operations split between several C modules; this will kill any chance at compiler optimization). Anyway, it appears that the bitpacker used by VP3 is the exact opposite packing order used by Ogg... Ah well. Monty --- >8 ---- List archives: http://www.xiph.org/archives/ Ogg project homepage: http://www.xiph.org/ogg/ To unsubscribe from this list, send a message to 'theora-dev-request@xiph.org' containing only the word 'unsubscribe' in the body. No subject is needed. Unsubscribe messages sent to the list will be ignored/filtered.
I assume this is directed at me -- (apologies up front for the format of my post; I'm stuck with Outlook Web Access for the time being :( ) frankly I've never looked hard at this part of the code; I generally muck around with motion vectors, transforms, etc -- algorithm stuff. How the bitstream is created is fresh territory. I can of course ask some questions of the engineers responsible for this stuff, but I want to be sure I'm asking the right questions first.>Whose responsibility is it to flip bytes before putting out a stream?Good question. I suspect that our policy has been that memory is always considered a stream of unsigned chars. Therefore it would be the codec's decision how to order bytes. I can check on this but I'm 90% sure. The original Vp3 coders are pretty low-level folk; they tend to think of bits as bits. I've worked with programmers who violently disagree with this, and their interfaces always specify a size_T type parameter, and work with arrays of typed values, but I don't think that's the case with any of this code. I kind of agree with the byte-stream philosophy, if only because I want my disk-based data to look the same no matter where it is created. I think a file that has an 'endian flag' and a bunch of type info in the headers is perhaps over-complex. A file can always be considered a series of bytes without confusion, if we decide to think this way. However, perhaps OGG already has a different philosophy.>I'm seriously consideringtossing Ogg's bitpacker into the codec as it has defined endianness behavior and is already used throughout Ogg (the API is very similar). If we "toss Ogg's bitpacker" into the codec, that would imply a bitstream change, right?>Also, this is probably my own confusion, but is it really the casethat the encoder app has to guess about packet size and just hope the output buffer doesn't overflow? The Ogg bitpacker would eliminate this problem as well. have to check on this as well. I guess what you're thinking about is to put the OGG packer deep in the codec code; I assume it deals with buffer overflow gracefully, etc. -----Original Message----- From: Monty [mailto:xiphmont@xiph.org] Sent: Sat 9/14/2002 5:22 PM To: theora-dev@xiph.org Cc: Subject: [theora-dev] Specific code questions I haven't actually had many questions up to now. Despite the all-encompassing CP_INSTANCE monolith, things have been relatively easy going. Now we get into real grit: First off, CBitman looks like it has endianness issues; it's packing into host-order 32 bit arrays (the comments and symbol names seem to indicate that this was originally byte-based packing code that got upped to 32 bit. Either that or someone is a bit confused about a byte being 32 bits...) Whose responsibility is it to flip bytes before putting out a stream? Is native order big or little endian? I'm seriously considering tossing Ogg's bitpacker into the codec as it has defined endianness behavior and is already used throughout Ogg (the API is very similar). Also, this is probably my own confusion, but is it really the case that the encoder app has to guess about packet size and just hope the output buffer doesn't overflow? The Ogg bitpacker would eliminate this problem as well. Monty --- >8 ---- List archives: http://www.xiph.org/archives/ Ogg project homepage: http://www.xiph.org/ogg/ To unsubscribe from this list, send a message to 'theora-dev-request@xiph.org' containing only the word 'unsubscribe' in the body. No subject is needed. Unsubscribe messages sent to the list will be ignored/filtered. -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 6682 bytes Desc: winmail.dat Url : http://lists.xiph.org/pipermail/theora-dev/attachments/20020914/1e36088d/winmail-0001.bin
> Ah, Write32ToBuffer is doign the flipping. What an odd little bitpacker.hmm... so I guess my previous post was all wrong? Or perhaps I'm confused, and this routine is doing just what I said -- making sure the byte order is the same on all architectures? I'm going to check into this with those who know what is going on. I apologize for shooting off before I had my information straight. (At one point I wrote a 32 bit version of the Ogg bitpacker that did byte flipping when necessary, and it turned out to be slower than the 8 bit packer we use now. Also, you have the lowest level bithandling operations split between several C modules; this will kill any chance at compiler optimization). Anyway, it appears that the bitpacker used by VP3 is the exact opposite packing order used by Ogg... Ah well. Monty --- >8 ---- List archives: http://www.xiph.org/archives/ Ogg project homepage: http://www.xiph.org/ogg/ To unsubscribe from this list, send a message to 'theora-dev-request@xiph.org' containing only the word 'unsubscribe' in the body. No subject is needed. Unsubscribe messages sent to the list will be ignored/filtered. -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 4202 bytes Desc: winmail.dat Url : http://lists.xiph.org/pipermail/theora-dev/attachments/20020914/ab83b65e/winmail-0001.bin
[cursing Outlook... argh!]
From: Monty [mailto:xiphmont@xiph.org]
>I also apologize for doing some amount of thinking aloud as I put
nitty-gritty into place. VP3 is clearly defined as a big-endian codec
with MSb-first packing, and that is a sensible choice (makes fast
huffman decode easier).
well, on a big-endian machine I guess. Take a look at
"Read32FromBuffer" -- another curious specimen. Obvious room for
optimization -- like if we know we're little endian, just read a long!
In defense of the architects, Vp3 was originally designed as a
low-bitrate codec.
>Vorbis II and future Ogg still will be
MSb-first... so I think I'll just add that option to the Ogg
bitpacker
now and let Theora use it. It will simplify packet buffer management.
This sounds fine. Alternatively, we could change the VP3 bitstream to
little-endian, but if you don't mind fixing Ogg to support msb streams this
seems like the best choice.
My engineers confirmed that (paraphrasing Groucho Marx) yes, we have no
buffer overflow management.
Monty
--- >8 ----
List archives: http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'theora-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body. No subject
is needed.
Unsubscribe messages sent to the list will be ignored/filtered.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 4858 bytes
Desc: winmail.dat
Url :
http://lists.xiph.org/pipermail/theora-dev/attachments/20020916/36687719/winmail-0001.bin