Hello everyone, I'm currently developing an adaptive videoconferencing application based on Ekiga which uses TFRC as a congestion control mechanism to adapt the video encoding rate according to the quality of the network experienced. My goal was to use the open-source Theora codec for video transmission. Unfortunately, it seemed that Theora 1.0 did not properly implement any correct CBR mode. The encoding rates can be adjusted "on-the-fly" during encoding by adjusting the "target_bitrate" and the "keyframe_target_bitrate" variables of the "theora_info" structure, but the resulting average bitrate is largely below the target bitrate desired. Furthermore, the bitrates vary depending on the amount of motion captured, which is a typical VBR feature. Here are the bitrates I measured (VGA resolution): tx = target_bitrate (Mbps); ox = max rate observed at output of encoder (Mbps; approximation) tx: 0.7 --> ox: 0.7 tx: 1.0 --> ox: 0.9 tx: 1.6 --> ox: 1.1 tx: 2.0 --> ox: 1.3 tx: 4.0 --> ox: 2.4 tx: 6.0 --> ox: 3.4 tx: 8.0 --> ox: 5.6 I also tested Theora 1.1 Thusnelda but no considerable improvement coud be noticed, though I found contradictory information: "According to Xiph.org's Ralph Giles, the most noticeable improvement in 1.1 is proper rate control, particularly for fixed bit rate encoding, where the user specifies either the number of bits per second desired in the output (a common use case for streaming applications), or the desired file size. "The 1.0 encoder relies a lot on heuristics, instead of trying to optimize directly the trade-off between quality of the coded images and the number of bits used to represent them," he said, "More significantly, the fixed bitrate mode in the 1.0 reference encoder didn't really work; it just guessed how to meet its target and often missed the requested bitrate, sometimes by quite a bit, which was a problem for streaming and fixed-size encodes." (http://lwn.net/Articles/326697) What is the exact behavior of rate controller in Theora and is or will be CBR possible to use? At least, adaptive quantization is implemented ( http://wiki.xiph.org/Theora) but apparently not correctly to support CBR, or am I missing something? I found that the quantizer values are adapted in oc_enc_select_qi() and oc_enc_calc_lamba() without being able to define the actual behavior of these functions... Having a constant bitrate that does not adapt to variations in the image content is really a crucial feature for my application. Thank you for your inputs and clarifications! :) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.xiph.org/pipermail/theora/attachments/20090708/62341d01/attachment-0001.htm
On Wed, Jul 8, 2009 at 12:40 PM, Thomas de Buman<tdbuman at gmail.com> wrote:> Hello everyone, > > I'm currently developing an adaptive videoconferencing application based on > Ekiga which uses TFRC as a congestion control mechanism to adapt the video > encoding rate according to the quality of the network experienced. My goal > was to use the open-source Theora codec for video transmission. > Unfortunately, it seemed that Theora 1.0 did not properly implement any > correct CBR mode.Mostly correct. The bitrate management in 1.0 was very fragile. There were a narrow set of constraints in which it worked, however it would silently fail and head off into the weeds if it got outside its working zone.> I also tested Theora 1.1? Thusnelda but no considerable improvement coud be > noticed, though I found contradictory information:That's quite surprising. I use the bitrate management in Thusnelda (FTR, 1.1 hasn't been released yet, these are alphas) and have found it to be robust and correct. That's not to say bugs don't remain. Perhaps what you're really running into is a long target rate window? The Thusnelda alphas, as shipped, constrain bitrate over the period of a buffering window that is equal to the max keyframe spacing. This window will be independently configurable in the final 1.1.> What is the exact behavior of rate controller in Theora and is or will be > CBR possible to use? At least, adaptive quantization is implemented > (http://wiki.xiph.org/Theora)I think adaptive quant is something different than what you expect (it's the use of multiple quantization matricies within a frame to properly handle lower-contrast details, not a change in quantization between frames to meet a rate target) and as yet Thusnelda does not have intra-frame AQ.> Having a constant bitrate that does not adapt to variations in the image > content is really a crucial feature for my application.I suspect the issue you're having is just a setup issue. If not, it's a bug and then I'm even more interested. Do you have code anywhere I could see it? Monty
The above-mentioned bitrate values were measured using Theora 1.0 at VGA
(640x480) resolution.
Concerning the implementation of the Theora encoder into Ekiga, it is
accessed via the underlying Opal library. This library is the basis of any
call management in Ekiga and supports different encoders via plugin
integration. Below, you will find the major steps involved in adjusting the
encoder options in the Theora plugin. The whole Opal package can is
downloadable from https://sourceforge.net/projects/opalvoip/files/.
1) The initialisation of the parameters:
theoraEncoderContext::theoraEncoderContext()
{
ogg_packet headerPacket, tablePacket;
_frameCounter=0;
_txTheoraFrame = new theoraFrame();
_txTheoraFrame->SetMaxPayloadSize(THEORA_PAYLOAD_SIZE);
theora_info_init( &_theoraInfo );
_theoraInfo.frame_width = 352; // Must be multiple of 16
_theoraInfo.frame_height = 288; // Must be multiple of 16
_theoraInfo.width = _theoraInfo.frame_width;
_theoraInfo.height = _theoraInfo.frame_height;
_theoraInfo.offset_x = 0;
_theoraInfo.offset_y = 0;
_theoraInfo.fps_numerator = THEORA_FRAME_RATE;
_theoraInfo.fps_denominator = 1;
_theoraInfo.aspect_numerator = _theoraInfo.width; // Aspect width/height
_theoraInfo.aspect_denominator = _theoraInfo.height; //
_theoraInfo.colorspace = OC_CS_UNSPECIFIED;
_theoraInfo.target_bitrate = THEORA_BITRATE * 2 / 3; // Anywhere
between 45kbps and 2000kbps
_theoraInfo.quality = 16;
_theoraInfo.dropframes_p = 0;
_theoraInfo.quick_p = 1;
_theoraInfo.keyframe_auto_p = 1;
_theoraInfo.keyframe_frequency = THEORA_KEY_FRAME_INTERVAL;
_theoraInfo.keyframe_frequency_force = _theoraInfo.keyframe_frequency;
_theoraInfo.keyframe_data_target_bitrate = THEORA_BITRATE;
_theoraInfo.keyframe_auto_threshold = 80;
_theoraInfo.keyframe_mindistance = 8;
_theoraInfo.noise_sensitivity = 1;
theora_encode_init( &_theoraState, &_theoraInfo );
theora_encode_header( &_theoraState, &headerPacket );
_txTheoraFrame->SetFromHeaderConfig(&headerPacket);
theora_encode_tables( &_theoraState, &tablePacket );
_txTheoraFrame->SetFromTableConfig(&tablePacket);
}
2) The only theoraInfo parameters that are updated "on-the-fly" during
a RTP
video session are the "target_bitrate" and
"keyframe_data_target_bitrate"
parameters:
void theoraEncoderContext::SetTargetBitrate(unsigned rate)
{
_theoraInfo.target_bitrate = rate * 2 / 3; // Anywhere between
45kbps and 2000kbps}
_theoraInfo.keyframe_data_target_bitrate = rate;
}
3) The changes are then applied by calling:
void theoraEncoderContext::ApplyOptions()
{
ogg_packet headerPacket, tablePacket;
theora_clear( &_theoraState );
theora_encode_init( &_theoraState, &_theoraInfo );
theora_encode_header( &_theoraState, &headerPacket );
_txTheoraFrame->SetFromHeaderConfig(&headerPacket);
theora_encode_tables( &_theoraState, &tablePacket );
_txTheoraFrame->SetFromTableConfig(&tablePacket);
}
4) The major function calls for encoding the frames are then:
theora_encode_YUVin( &_theoraState, &yuv );
followed by
theora_encode_packetout( &_theoraState, 0 /* not last Packet */,
&framePacket );
I hope this could help to find possible problems,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/theora/attachments/20090714/04e7870d/attachment-0001.htm