(I've already posted this message month ago, but nobody answered, may be it was not delivered to newsgroup?) Hello, All. I wrote an encoder-decoder based on example from OGG-Vorbis SDK. This encoder can encode a large amount of small WAV-files with equal parameters into one sound archive. Then I decode this sound archive back into large amount of small WAV-files. Theese files are different from source files, but each of them can be normally played by Windows. And now I can't correctly encode these decoded files into sound archive again. The behavior of encoder is the following: 1) it normally encodes first 10-20 files and after that the time of encoding for each file starts to grow The vorbis_analysis_blockout() function starts to return 0 (in normal mode it returns 1). The time of executing vorbis_analysis_buffer() function starts to grow (in normal mode this time is very small). It seems like the encoder processes vorbis_analysis_blockout() function only when the end-of-stream flag is set to 1 (at the end of the last file). 2) According to (1) the encoder hungs up when I'm trying to encode about 9000 files. 2) When I listen the resulting sound archive the first file is playing with artefacts. Does anybody know how can it be repaired? Thank you. Aleksey
On 11/09/06, Aleksey Surkov <Aleksey_S@abbyy.com> wrote:> (I've already posted this message month ago, but nobody answered, may be > it was not delivered to newsgroup?) >It arrived. I think there wasn't really enough detail to diagnose the problem.> Hello, All. > > I wrote an encoder-decoder based on example from OGG-Vorbis SDK. > This encoder can encode a large amount of small WAV-files with equal > parameters into one sound archive. > > Then I decode this sound archive back into large amount of small > WAV-files. > > Theese files are different from source files, but each of them can be > normally played by Windows. > > And now I can't correctly encode these decoded files into sound archive > again. > > The behavior of encoder is the following: > > 1) it normally encodes first 10-20 files and after that the time of > encoding for each file starts to grow The vorbis_analysis_blockout() > function starts to return 0 (in normal mode it returns 1). > The time of executing vorbis_analysis_buffer() function starts to grow > (in normal mode this time is very small). > > It seems like the encoder processes vorbis_analysis_blockout() function > only when the end-of-stream flag is set to 1 (at the end of the last > file). > > 2) According to (1) the encoder hungs up when I'm trying to encode about > 9000 files. > > 2) When I listen the resulting sound archive the first file is playing > with artefacts. > > Does anybody know how can it be repaired? >Well, without the code we can't say much. Since the problem occurs with large numbers of files I'd suggest there's some kind of memory leak somewhere, or some buffer is being filled up and never emptied. Your problem with round trippping the files may be related or entirely separate. Are you sure the .wav files being produced at the intermediate step are valid? Final aside: this is just a test isn't it? Round tripping audio through the encode/decode is probably a bad idea. -- imalone
Hi, Ian! Thank you for your response. Here is some details...> Well, without the code we can't say much. Since the problem occurswith> large numbers of files I'd suggest there's some kind of memory leak > somewhere, or some buffer is being filled up and never emptied.Not exactly so. It works correctly with other files (even with a big number of files) and fails just for several "bad" files.>Your problem with round trippping the files may be related or entirely > separate. Are you sure the .wav files being produced at theintermediate> step are valid?Yes, they are valid. I can play it. More, after converting (changing frequency) by CDex the files can be compressed again correctly. I just can't compress it immediately after decompressing, without any intermediate actions like frequency changing via third-party utility.> Final aside: this is just a test isn't it? Round tripping audiothrough> the encode/decode is probably a bad idea.No, unfortunately it`s not a test. It`s a real problem (we've lost the source files that are compressed into archive... And now the only way to make some changes in this archive is decompressing-compressing). And this problem occurs not only with recompressed files. For example, I have a test wav file (88 kb), the described bug occurs with it: http://rapidshare.de/files/32738670/0008.wav.html or I can send it by email, just write me to Aleksey_S(at)abbyy.com The Code I use is almost the same as in example below (it's a part that encodes files): void COggArchive::addFileToStream(CWAVFile & wavFile) { char readBuffer[ReadBufferSize]; int bytesRead; int channels = pcmDefaultSettings.channels; int bytesPerSample = wavFile.GetWaveFormat().nBitsPerSample / 8; int bytesPerChannel = bytesPerSample / channels; int totalSamples = wavFile.getDataChunkSize() / bytesPerSample; while( bytesRead = wavFile.ReadData( readBuffer, ReadBufferSize ) ) { int samplesRead = bytesRead / bytesPerSample; float **buffer = vorbis_analysis_buffer( &oggState.vd, samplesRead ); for (int ch = 0; ch < channels; ch++) { for (int i = 0; i < samplesRead; i++) { if ( bytesPerChannel == 2 ) { short ampl; int sampleChannelOffset = i * bytesPerSample + ch * 2/*bytesPerChannel*/; ampl = ( ( 0x00ff & (short)readBuffer[sampleChannelOffset] ) | ( readBuffer[sampleChannelOffset + 1] << 8 ) ); buffer[ch][i] = ampl / 32768.f; } else { buffer[ch][i] readBuffer[i*bytesPerSample + ch/* *bytesPerChannel=1*/] / 255.f; } } } vorbis_analysis_wrote( &oggState.vd, samplesRead ); EncodeReadSamples(); } } void COggArchive::EncodeReadSamples() { int eos = 0; while( vorbis_analysis_blockout( &oggState.vd, &oggState.vb ) =1 ) { /* analysis, assume we want to use bitrate management */ vorbis_analysis( &oggState.vb, 0 ); vorbis_bitrate_addblock( &oggState.vb ); while( vorbis_bitrate_flushpacket( &oggState.vd, &oggState.op ) ) { /* weld the packet into the bitstream */ ogg_stream_packetin( &oggState.os, &oggState.op ); /* write out pages (if any) */ // int eos = 0; while( !eos ) { if ( !ogg_stream_pageout( &oggState.os, &oggState.og ) ) break; streamBuffer->Write( oggState.og.header, oggState.og.header_len ); streamBuffer->Write( oggState.og.body, oggState.og.body_len ); if( ogg_page_eos( &oggState.og ) ) eos = 1; } } } }
Hi, R.L. Horn!>>> Are you sure the .wav files being produced at the intermediate step >>> are valid? >> Yes, they are valid. I can play it. >That's not a sufficient test of validity.How can I check them sufficiently?>> More, after converting (changing frequency) by CDex the files can be >> compressed again correctly. >That tends to support the "bad wav file" (or at least something yourcode isn't handling) theory. Could you tell me more about what is "bad wav file"? It can help me to understand troubles with recompressing.>Following the data chunk are 'bext', 'minf', 'elm1', 'regn', 'ovwf',and 'umid' chunks.>These will cause problems if your code isn't skipping unrecognizedchunks. Thanks a lot! The bug with this file was really here. Recompression I`ll check in odd-come-shortly, but I don`t sure, that decisions are the same.
Hi!> The simplest thing is just to do a hex dump and compare the file > contents against the standard.Thanks a lot! It seems I've found the bug. Decoded files was really corrupted. All of them contained some extra bytes (less then 4, so no artifacts appeared in 16bit mono). The cause of the problem was function vorbis_analysis_wrote( &vd, 0 ), that was called somewhere in midway of encoding (last portion of wav-stream in buffer was less then 4 bytes - less then one sample): file_n.wav: | readLength | readLength | 3bytes (0 samples in buffer) called: vorbis_analysis_wrote( &vd, 0 ); file_n+1.wav: | readLength | readLength | readLength | some samples + 3bytes After this moment (calling of vorbis_analysis_wrote( &vd, 0 )) encoder began to collect all passed to him stream-data. I don't know, is it a bug or no (such behaviour of encoder), but it was curious for me :)