Young, Milan wrote:> * Adding FEC seems to change the encoded audio bit-stream itself, i.e., it
doesn't just add additional protection bits, but also changes the encoded
bits. This is easy to show by comparing two decoded versions of the same audio
file: one encoded with no FEC, and another with FEC. Is this expected or is it a
bug?
If you encode with the same target rate with FEC on and off, then while
FEC is on, the encoder will reduce the rate of the non-FEC portion in
order to achieve the same total rate. So yes, this changes the resulting
bitstream.
> * Related to the above, what does the FEC information consist of? How does
the decoder use it? Does it simply repeat preceding packets, or interpolate
preceding and subsequent packets, or does it do something smarter?
The FEC information consists of "LBRR frames" in the SILK layer. The
bitstream syntax for these is largely the same as a regular SILK frame.
See <http://tools.ietf.org/html/rfc6716#section-4.2.5> for some details.
The decoder uses it to replace the last Opus frame of a dropped packet.
This is done by calling opus_decode() with a pointer to the next packet
and the decode_fec flag set to 1 (see
<https://mf4.xiph.org/jenkins/view/opus/job/opus/ws/doc/html/group__opus__decoder.html#ga7d1111f64c36027ddcb81799df9b3fc9>).
If no FEC is available in the packet, normal PLC is used instead.
Although it would be legal to have it simply be a repeat of the
preceding packet (as long as the preceding packet was an active frame),
the encoder actually requantizes it with a reduced bitrate. This lets it
do smarter rate allocation than a simple approach like RFC 2198 or RFC
6354 redundancy, which always repeats whole packets. Nothing prevents
you from using one of those codec-agnostic methods with Opus, however.
> * Based on my tests, the FEC in opus seems to have a limit, i.e., it adds
correction but only up to approx 4---10% expected packet loss, depending on the
file. What are the parameters/limits of the FEC?
You can find the details in silk_setup_LBRR() in silk/control_codec.c.
Basically, there is a base bitrate at which LBRR gets activated for each
audio bandwidth (NB, MB, and WB). For small amounts of packet loss (less
than 25%), this threshold gets boosted (by up to 25% when there is no
reported packet loss). As long as the target rate is above this
threshold, LBRR gets turned on.
When it's enabled, it boosts the quantization gain on LBRR frames
(w.r.t. normal frames) by a factor that decreases as the packet loss
percentage goes up. At a packet loss percentage of 0, this gain gets
increased by a factor of approximately 3. As the packet loss percentage
goes up, the increase in quantization goes down until it hits a factor
of approximately 1.37 at 12.5% packet loss.
> * Does FEC add any algorithmic delay? i.e., does it require the decoder
to wait until post-loss packets are received in order to generate a replacement
for the lost packets? If so, what is the maximum delay and can it be variably
set?
A decoder must wait for one future packet to fill in a missing hole
using FEC. This cannot be variably controlled (i.e., the maximum
additional delay is 1 packet). Of course, a decoder is always free to
ignore FEC if it does not want this additional delay, or
opportunistically use whatever happens to be in its jitter buffer, etc.