Thank you very much! But how to deal with endianness in the case of bit stream? Some blocks (for example MinBlockSize) require 16bits (simply swap first and second), some block (e.g.MinFrameSize) require 3 byte-array to be reverted. Finally totalSamples is stored in 5 bytes ( only last 4 bits from first one byte are used). It was a real issue to make it little-endian (here is how I did it: http://code.google.com/p/sharpflac/source/browse/trunk/SharpFlac/Blocks/StreamInfo.cs). So to make stream little endian, I can't simple get each 4 bytes of the stream one by one and revert them... Anyways I've managed to decode StreamInfo, VorbisComments,SeekPoints and Application blocks without bitStream implementation. But I'd be really pleased if anyone pointed me out how to deal with endianness. Thank you! 2010/6/23 Brian Willoughby <brianw at sounds.wa.com>:> > On Jun 22, 2010, at 07:21, Ivailo Karamanolev wrote: >> Hello Ilia, >> >> The FLAC format by nature is not a byte stream, it's a bit stream. >> Therefore, in order to parse it you need too build a bit-reading >> infrastructure. Eg. a class that accepts a byte stream, implements >> buffering, etc, etc, and supports reading a specified number of >> bits, not bytes as you are used to. There is quite a lot of bit >> logic there, but nothing too scary. > > Ivailo is correct. ?You'll note, Ilia, that other compressed formats > such as MPEG and MP3 also have bit streams, so you might find other > open source examples of bit parsing. > > Brian Willoughby > Sound Consulting > > _______________________________________________ > Flac-dev mailing list > Flac-dev at xiph.org > http://lists.xiph.org/mailman/listinfo/flac-dev >
On Jun 22, 2010, at 21:27, Ilia Ternovich wrote:> Thank you very much! But how to deal with endianness in the case of > bit stream? Some blocks (for example MinBlockSize) require 16bits > (simply swap first and second), some block (e.g.MinFrameSize) require > 3 byte-array to be reverted. > > Finally totalSamples is stored in 5 bytes ( only last 4 bits from > first one byte are used). It was a real issue to make it little-endian > (here is how I did it: > http://code.google.com/p/sharpflac/source/browse/trunk/SharpFlac/ > Blocks/StreamInfo.cs). > > So to make stream little endian, I can't simple get each 4 bytes of > the stream one by one and revert them... > > Anyways I've managed to decode StreamInfo, VorbisComments,SeekPoints > and Application blocks without bitStream implementation. But I'd be > really pleased if anyone pointed me out how to deal with endianness.Endianness involves multiple bytes. Thus, if you never read more than one byte at a time, then you won't have endianness issues. You should convert from stream endianness to native endianness before combining bits into data of more than one byte. Your bitstream parser must read only one byte at a time and pull bits from it until they are completely exhausted before reading the next byte. The FLAC stream itself has an endianness within the byte that is predefined and will be the same across all platforms. When you need to read more than 8 bits for a field, you must only grab 8 bits at a time from the stream, then add them to your own accumulator that is large enough to hold the largest field (32 bits?). In other words, I think your only problem is that you're reading 4 bytes of the stream when you should only be reading 1 byte at a time. Another hint is that you should not try to maintain the same structure in your host-native data as in the FLAC stream. i.e. Do not try to convert the stream from big endian to little endian. Instead, convert each value to separate fields in a structure, or even to separate variables. Packing is only needed to save stream bandwidth - you should have plenty of memory in your program. Brian Willoughby Sound Consulting
On Jun 22, 2010, at 21:27, Ilia Ternovich wrote:> Thank you very much! But how to deal with endianness in the case of > bit stream? Some blocks (for example MinBlockSize) require 16bits > (simply swap first and second), some block (e.g.MinFrameSize) require > 3 byte-array to be reverted. > > Finally totalSamples is stored in 5 bytes ( only last 4 bits from > first one byte are used). It was a real issue to make it little-endian > (here is how I did it: > http://code.google.com/p/sharpflac/source/browse/trunk/SharpFlac/ > Blocks/StreamInfo.cs). > > So to make stream little endian, I can't simple get each 4 bytes of > the stream one by one and revert them... > > Anyways I've managed to decode StreamInfo, VorbisComments,SeekPoints > and Application blocks without bitStream implementation. But I'd be > really pleased if anyone pointed me out how to deal with endianness.Just because you've been able to hack around without a bitStream implementation doesn't mean you're doing it right! :-) I took a look at your code, and you're calling GetEndianBytes(adr, ofs, n); with n > 1, and that's wrong. You cannot look ahead in a stream, and so your code is already doing things in a way that is not to spec. You then use array indexes and bit shift operators to build multi-byte values. What's worse, I see that your code is looking at (contentOffset - 1), which is really bad. What you need to do is write a bitStream function. It should only read each byte from the stream once and completely deal with all 8 bits before dealing with the next byte. You should never look ahead beyond the current byte. Instead, you need a long long accumulator to hold the totalSamples, which is over 32 bits. Each time you read a byte, add 8 to your bit count, and shift any existing bits in your accumulator around to make room for the new bits. When pulling values out of the accumulator, you do that based upon the number of bits requested. In other words, instead of: GetEndianBytes(streamArray, streamOffset, numBytes); ... you should implement: (long long)GetBigEndianStreamBits(streamHandle, outputAddressPointer, (unsigned)numBits); The return value will have to be 64-bit to handle totalSamples. The nice thing is that you won't need any indexing or bit shifting, because GetBigEndianStreamBits() will directly return the value you ask for, based upon the number of bits requested. I mean, you can continue to try and hack things, or you can simply write your stream parser the way it's supposed to be implemented. The good news is that if you do it right, then your code is portable to situations where you have a real stream, such as over the internet. Your code hack so far will only work with files, not with real streams (and it's not really working completely, anyway). It's really not that hard. I've written MPEG metadata parsers in less than a day. Once you implement the bit function, the rest is super easy. Just don't try to map the exact stream structure into a literal C struct. Brian Willoughby Sound Consulting
Oops. I proofread my email a little too late. I corrected the example. Hopefully what I am suggesting is clear. Brian Willoughby Sound Consulting On Jun 22, 2010, at 22:15, Brian Willoughby wrote:> What you need to do is write a bitStream function. It should only > read each byte from the stream once and completely deal with all 8 > bits before reading the next byte. You should never look ahead > beyond the current byte. Instead, you need an accumulator to > collect the incoming bits. Each time you read a byte, add 8 to > your bit count, and shift any existing bits in your accumulator > around to make room for the new bits. When pulling values out of > the accumulator, you do that based upon the number of bits > requested. The accumulator will have to be (long long) because you > will sometimes request more than 32 bits. > > In other words, instead of: > > GetEndianBytes(streamArray, streamOffset, numBytes); > > ... you should implement: > > (long long)GetBigEndianStreamBits(streamHandle, (unsigned)numBits); > > The return value will have to be 64-bit to handle totalSamples. > The nice thing is that you won't need any indexing or bit shifting, > because GetBigEndianStreamBits() will directly return the value you > ask for, based upon the number of bits requested.