flac dev - May 2011 - Can a libFLAC encoder be initialize and called from inside a libFLAC decoder callback?

I am getting large, corrupted native FLAC files when trying to call the
encoder from inside the decoder.  The metadata in the output file is OK.

 

Is initializing and calling an encoder from inside a libFLAC decoder write
callback supported?  The encoder is initialized on the first write callback.

 

Environment is Win 7 Ult, 64-bit, VS 2008, libFLAC 1.2.1.

 

Here is some context. The static callback calls the instance callback.
Encoder is initialized on line with "m_HaveEncoder =
InitializeEncoder();".
Samples are submitted to the encoder on line with "FLAC__bool OK
FLAC__stream_encoder_process(m_Encoder, Buffer, Blocksize);".

 

I have a version that decodes to a WAV file and then re-encodes the WAV
file, but would like to avoid all that I/O.

 

Regards.

 

David

 

 

FLAC__StreamDecoderWriteStatus FLACRecoder::StaticWriteCallback(const
FLAC__StreamDecoder *FLACStreamDecoder, const FLAC__Frame *Frame, const
FLAC__int32 *const Buffer[], void *ClientData)

{

  return ((FLACRecoder
*)(ClientData))->ClassWriteCallback(FLACStreamDecoder, Frame, Buffer);

}

 

FLAC__StreamDecoderWriteStatus  FLACRecoder::ClassWriteCallback(const
FLAC__StreamDecoder *WXUNUSED(FLACStreamDecoder), const FLAC__Frame *Frame,
const FLAC__int32 *const Buffer[])

{

  if (m_HaveError)

  {

    return FLAC__STREAM_DECODER_WRITE_STATUS_ABORT;

  }

 

  // Make sure we have an encoder.

  if (!m_TestOnly)

  {

    m_HaveEncoder = InitializeEncoder();

    

    if (!m_HaveEncoder)

    {

      return FLAC__STREAM_DECODER_WRITE_STATUS_ABORT;

    }

  }

 

  if (Frame)

  {

    if (m_TestOnly)

    {

      // Tell the decoder to continue without doing anything other than
count the samples.

      m_nInputSamplesProcessed += Frame->header.blocksize;

      return FLAC__STREAM_DECODER_WRITE_STATUS_CONTINUE;

    }

    else

    {

      // We have a frame and we are not in test mode. We need to send it
directly to the encoder

      unsigned int Blocksize = Frame->header.blocksize;

 

      // Is this line OK?

      FLAC__bool OK = FLAC__stream_encoder_process(m_Encoder, Buffer,
Blocksize);

 

      // Count the samples

      m_nInputSamplesProcessed += Blocksize;

 

      // Let decoder know if it can continue

      return OK ? FLAC__STREAM_DECODER_WRITE_STATUS_CONTINUE :
FLAC__STREAM_DECODER_WRITE_STATUS_ABORT;

    }

  }

  else

  {

    // We need a frame.  Let the decoder know we can't continue.

    return FLAC__STREAM_DECODER_WRITE_STATUS_ABORT;

  }

}

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/flac-dev/attachments/20110524/f9c81067/attachment-0001.htm

Have you tried decoding to memory, and encoding from that memory?   
This technique would avoid file I/O.

If you are reluctant to allocate enough memory for the full file (and  
why not, with a 64-bit system?), then you could implement a FIFO.   
You would need a semaphore to keep track of the decoder's progress,  
and multithreaded coding.  The decoder would pause, whenever the FIFO  
gets full, until the encoder empties out the samples so far.

It's also possible that you have made a mistake in InitializeEncoder 
() where the new file format does not match the incoming Buffer blocks.

Brian Willoughby
Sound Consulting


On May 24, 2011, at 10:16, David Troendle wrote:
> I am getting large, corrupted native FLAC files when trying to call  
> the encoder from inside the decoder.  The metadata in the output  
> file is OK.
>
> Is initializing and calling an encoder from inside a libFLAC  
> decoder write callback supported?  The encoder is initialized on  
> the first write callback.
>
> Here is some context. The static callback calls the instance  
> callback.  Encoder is initialized on line with ?m_HaveEncoder =  
> InitializeEncoder();?.  Samples are submitted to the encoder on  
> line with ?FLAC__bool OK = FLAC__stream_encoder_process(m_Encoder,  
> Buffer, Blocksize);?.
>
> I have a version that decodes to a WAV file and then re-encodes the  
> WAV file, but would like to avoid all that I/O.
>
>

Thanks for the tip, Brian.  I did have a version that does everything in
memory, but only had enough memory to get six threads going.  (Although my
system has 16GB, I have not taken the time to create 64-bit libraries for
wxWidgets, TagLib, libFLAC, etc.)

Based on the direction you are pointing me in, I assume that encoding from
within the decoder is not permitted.

I really like your idea of the FIFO, and will probably go in that direction.
What do you think of implementing the FIFO via a pipe?  That might simplify
the implementation.

Thanks for the advice.

Kindest regards to you and the flac-dev community.

David

-----Original Message-----
From: Brian Willoughby [mailto:brianw at sounds.wa.com] 
Sent: Tuesday, May 24, 2011 1:16 PM
To: David Troendle
Cc: flac-dev at xiph.org
Subject: Re: [Flac-dev] Can a libFLAC encoder be initialize and called from
inside a libFLAC decoder callback?

Have you tried decoding to memory, and encoding from that memory?   
This technique would avoid file I/O.

If you are reluctant to allocate enough memory for the full file (and  
why not, with a 64-bit system?), then you could implement a FIFO.   
You would need a semaphore to keep track of the decoder's progress, and
multithreaded coding.  The decoder would pause, whenever the FIFO gets full,
until the encoder empties out the samples so far.

It's also possible that you have made a mistake in InitializeEncoder
() where the new file format does not match the incoming Buffer blocks.

Brian Willoughby
Sound Consulting

As is evident from the code snippet below, I was initializing the encoder on
every write callback.  Once I fixed that it worked.

 

So, the encoder can be initialized and called from within a decoder
callback.

 

My apologies to the group for this post.    I should have caught this before
posting.

 

David

 

From: flac-dev-bounces at xiph.org [mailto:flac-dev-bounces at xiph.org] On
Behalf
Of David Troendle
Sent: Tuesday, May 24, 2011 12:16 PM
To: flac-dev at xiph.org
Subject: [Flac-dev] Can a libFLAC encoder be initialize and called from
inside a libFLAC decoder callback?

 

I am getting large, corrupted native FLAC files when trying to call the
encoder from inside the decoder.  The metadata in the output file is OK.

 

Is initializing and calling an encoder from inside a libFLAC decoder write
callback supported?  The encoder is initialized on the first write callback.

 

Environment is Win 7 Ult, 64-bit, VS 2008, libFLAC 1.2.1.

 

Here is some context. The static callback calls the instance callback.
Encoder is initialized on line with "m_HaveEncoder =
InitializeEncoder();".
Samples are submitted to the encoder on line with "FLAC__bool OK
FLAC__stream_encoder_process(m_Encoder, Buffer, Blocksize);".

 

I have a version that decodes to a WAV file and then re-encodes the WAV
file, but would like to avoid all that I/O.

 

Regards.

 

David

 

 

FLAC__StreamDecoderWriteStatus FLACRecoder::StaticWriteCallback(const
FLAC__StreamDecoder *FLACStreamDecoder, const FLAC__Frame *Frame, const
FLAC__int32 *const Buffer[], void *ClientData)

{

  return ((FLACRecoder
*)(ClientData))->ClassWriteCallback(FLACStreamDecoder, Frame, Buffer);

}

 

FLAC__StreamDecoderWriteStatus  FLACRecoder::ClassWriteCallback(const
FLAC__StreamDecoder *WXUNUSED(FLACStreamDecoder), const FLAC__Frame *Frame,
const FLAC__int32 *const Buffer[])

{

  if (m_HaveError)

  {

    return FLAC__STREAM_DECODER_WRITE_STATUS_ABORT;

  }

 

  // Make sure we have an encoder.

  if (!m_TestOnly)

  {

    m_HaveEncoder = InitializeEncoder();

    

    if (!m_HaveEncoder)

    {

      return FLAC__STREAM_DECODER_WRITE_STATUS_ABORT;

    }

  }

 

  if (Frame)

  {

    if (m_TestOnly)

    {

      // Tell the decoder to continue without doing anything other than
count the samples.

      m_nInputSamplesProcessed += Frame->header.blocksize;

      return FLAC__STREAM_DECODER_WRITE_STATUS_CONTINUE;

    }

    else

    {

      // We have a frame and we are not in test mode. We need to send it
directly to the encoder

      unsigned int Blocksize = Frame->header.blocksize;

 

      // Is this line OK?

      FLAC__bool OK = FLAC__stream_encoder_process(m_Encoder, Buffer,
Blocksize);

 

      // Count the samples

      m_nInputSamplesProcessed += Blocksize;

 

      // Let decoder know if it can continue

      return OK ? FLAC__STREAM_DECODER_WRITE_STATUS_CONTINUE :
FLAC__STREAM_DECODER_WRITE_STATUS_ABORT;

    }

  }

  else

  {

    // We need a frame.  Let the decoder know we can't continue.

    return FLAC__STREAM_DECODER_WRITE_STATUS_ABORT;

  }

}

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/flac-dev/attachments/20110524/69cb1332/attachment-0001.htm