Speex dev - Dec 2006 - Would be nice to conditionally compile out coding modes and code tables...

> -----Original Message-----
> From: Jean-Marc Valin [mailto:jean-marc.valin@usherbrooke.ca] 
> Sent: Thursday, December 14, 2006 2:51 AM
> To: Miles, Stewart
> Cc: speex-dev@xiph.org
> Subject: Re: [Speex-dev] Would be nice to conditionally 
> compile out coding modes and code tables...
> 
> Miles, Stewart a ?crit :
> > I'm only using the narrow-band encoder on my embedded 
> system and have 
> > removed all references to the wide-band and ultra-wide band encoder 
> > functions and tables. They will be pulled in by the linker 
> due to the 
> > references made by the structures in modes.c.  Attached is a 
> > replacement modes.c for compiling out modes that are not in use for 
> > anyone who's interested.
> 
> There's already a --disable-wideband (and corresponding
> DISABLE_WIDEBAND) option in the code, so I'm not sure what 
> your patch is supposed to add. 
Searching the code (speex 1.0.5) I don't find DISABLE_WIDEBAND.  This is new
to speex 1.2 and I wasn't aware of the update.
> Also, disabling narrowband 
> doesn't make sense because it's used by the wideband code.
After reading the code again, you're absolutely right (of course) each
progressively more complex codec references the previous one.  So disabling
narrow-band doesn't make sense.
> 
> > Also, I've removed the dynamic memory allocation of the encoder
and
> > decoder.  To do this I had to determine the required size of the 
> > encoder and decoder states since i'm working on a fixed 
> memory platform I simply
> > handle the worst case, see the attached statesize.h.   Now 
> I can do this
> > sort of thing...
> > 
> > MyDecoderClass decoders[5];
> > 
> > .. will means all of my decoder states are contiguous and 
> quick to DMA.
> > It also improves cache consistency when the input buffers 
> are placed 
> > in MyDecoderClass.
> 
> There's actually a much simpler way of making everything 
> contiguous. All you need to do is allocate (or reserve or 
> whatever) a big block yourself and override speex_alloc() and 
> speex_free() (and speex_alloc_scratch if you don't have a C99 
> compiler) to distribute that memory explicitly.
I completely understand that it's possible to override speex_alloc() and
speex_free(), in fact that's what i've done.  However, how much memory
do I reserve?  Without running the code I would like it to be possible to
reserve the appropriate memory for my required codecs.  Running the code to
determine the magic memory size to reserve is not very user-friendly and not
very good if future revisions of the library are released as the old code with
the magic allocation size will fail.  Determining whether an allocation is
possible at compile time is far more preferable than at run-time.

Which leads me to another point of irritation.  I noticed that you stated that
people shouldn't be using MAX_BYTES_PER_FRAME to reserve space for the
output of the encoder and that it's far better to just allow speex_bits to
realloc it's out of space
http://lists.xiph.org/pipermail/speex-dev/2003-August/000515.html .  For a lot
of embedded systems it's very important to know ahead of time how much
memory is required for an operation.  In fact the memory required should be
known at compile time so that the application will always function correctly. 
At the very least, the worst case scenario should be accounted for so the
application never fails due to being out of memory.  So something like
MAX_BYTES_PER_FRAME is a pretty good idea since it allows application developers
to prepare for the worst case.  What would be nice, would be a
MAX_BYTES_PER_FRAME type constant that defines the maximum packet size per
encoder mode.  How did MAX_BYTES_PER_FRAME end up at the magic number of 2000
bytes?  A frame of 2000 bytes is 1000 uncompressed mono 16-bit PCM samples.
> 
> > Would be nice for functionality like this to make it into the main 
> > line of code.
> 
> Anything that you need that isn't covered by the explanations above?
> 
> 	Jean-Marc
> 
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit 
> http://www.messagelabs.com/email 
> ______________________________________________________________________
> 
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
______________________________________________________________________

> Searching the code (speex 1.0.5) I don't find DISABLE_WIDEBAND.  This
> is new to speex 1.2 and I wasn't aware of the update.
Yes, Speex 1.0.x is old code and I'm not making changes to it (unless a
really bad bug is found).
> After reading the code again, you're absolutely right (of course)
> each progressively more complex codec references the previous one.
> So disabling narrow-band doesn't make sense.
Actually, I forgot to mention that disabling ultra-wideband is also
nearly useless since it uses the same functions as wideband.
> I completely understand that it's possible to override speex_alloc()
> and speex_free(), in fact that's what i've done.  However, how much
> memory do I reserve?  Without running the code I would like it to be
> possible to reserve the appropriate memory for my required codecs.
> Running the code to determine the magic memory size to reserve is not
> very user-friendly and not very good if future revisions of the
> library are released as the old code with the magic allocation size
> will fail.  Determining whether an allocation is possible at compile
> time is far more preferable than at run-time.
It's not easy to determine the size at compile time because it depends
on the compile options (fixed-point, float, SSE, ...) and the
architecture (the way structs are packed, the size of ints and pointers,
...). If you know of a generic solution that would work, I'm happy to
hear it.
> Which leads me to another point of irritation.  I noticed that you
> stated that people shouldn't be using MAX_BYTES_PER_FRAME to reserve
> space for the output of the encoder and that it's far better to just
> allow speex_bits to realloc it's out of space
> http://lists.xiph.org/pipermail/speex-dev/2003-August/000515.html .
> For a lot of embedded systems it's very important to know ahead of
> time how much memory is required for an operation.  In fact the
> memory required should be known at compile time so that the
> application will always function correctly.  At the very least, the
> worst case scenario should be accounted for so the application never
> fails due to being out of memory.  So something like
> MAX_BYTES_PER_FRAME is a pretty good idea since it allows application
> developers to prepare for the worst case.  What would be nice, would
> be a MAX_BYTES_PER_FRAME type constant that defines the maximum
> packet size per encoder mode.  How did MAX_BYTES_PER_FRAME end up at
> the magic number of 2000 bytes?  A frame of 2000 bytes is 1000
> uncompressed mono 16-bit PCM samples.
MAX_BYTES_PER_FRAME is totally unrelated to what you're asking. It's in
fact merely the initial size of the internal bit-packer to make sure
that realloc() is not needed for any sane use of libspeex. What you pass
to speex_bits_write() is the size of your own buffer. There's no maximum
value for that. It depends on what you're actually doing with Speex. If
you encode one frame at a time at 4 kbps, then you only need 10 bytes (4
kbps / 50 frames/sec / 8 bits/byte). If you decide you want to pack a
million frames in the same packet at 44 kbps, then the needed space will
be huge and there's nothing I can do for you. Now, if you're making an
embedded system and you're tight for memory, if could also be worth
using speex_bits_init_buffer() and pass your own buffer.

	Jean-Marc