>> Consider a device that only has enough ROM to store one set of >> quantization tables (the limitation could also be about speed, network, >> ...). If you specify MUST be able to decode, then it means that this >> device simply *cannot* implement the spec *at all*. This is bad for >> interoperability. > > For me: device which don't have all mode implemented are already banned: > the SDP offer/answer model is based on codec and parameter *proposal*: > still the sender is *allowed* to not send the given mode. That said, UA > that don't support all modes are already not capable of > interoperability: they will receive sometimes modes that they do not > proposed and do not support.The same reasoning could apply to other codecs as well: if a client doesn't support G.729, you can't say it's broken if someone unexpectedly start sending that. Of course, in the case of Speex, we try and do our best...> One possible way would be to use "mode=any" to clearly indicate that the > UA support any modes. The prefered mode could be specified by order of > preference: "mode=5;mode=any". Other "limited device" would announce all > modes supported: "mode=5;mode=3;mode=1"Yes, I agree with that.> Samples for SDP negotiation must clearly indicates this behaviour: > > m=audio 8088 RTP/AVP 97 > a=rtpmap:97 speex/8000 > > -> equivalent to "mode=any" (for smooth evolution of current > implementations)Shouldn't clients have at least a preferred mode? Otherwise, if everyone says "any", which bit-rate do you choose? <ship>> The above would make sense and compliant application would > be interoperable? right?I think so.> That would be perfect for me, and I guess would be fine for you?Yes, I think it's the best compromise -- and actually not very far from what we have now. I would still keep the thing that says 8 kbps SHOULD be supported. Or maybe we can say "all modes SHOULD be supported and if it's not possible, then at least 8 kbps (mode 3) SHOULD be supported".>>> The other way would be to make it transparent like g279. >> >> Not sure what kind of transparence you mean? The Speex decoder (unless >> you remove some tables) is able to decode anything without even knowing >> how it was encoded. > > Except for limited device which are not capable of decoding the "mode" > that were removed from code ;) > > If I understood well the speex stuff: a decoder without wideband > implementation would be capable of locating the narrowband part > of the speex bitstream and return at least narrowband in audio? right?Actually, if you give wideband data to a narrowband decoder, it will decode it as narrowband without even *realising* it's wideband (the wideband bits are transparently ignored).> Is this possible? (I guess it's not that complex): > --->wideband data --> NARROW BAND DECODER --> narrow band decoded audio!yes, that's possible.> This does not seems possible: > --->mode=4 nband data --> LIMITED DEVICE ---> correct nband data > WITH MODE=3 ONLYThat's not possible.>> If all you mean is "do your best to decode anything you get no matter >> how different it is from what you asked for", then I agree. > > This has to be at least of recommendation so people are aware that speex > implementation allow this behavior: it was unclear to people that they > don't need to set option before starting decode process. Because they > get a lot of decode failure before, I think speex users are more prepared > now. Anyway, it's good to inform people.What do you mean by "they get a lot of decode failure"?> Sometimes, draft are providing some C code: I would love to see some kind > of annex with a sample routine to decode speex data. > > Something like: static void dec_process(MSFilter *f) > http://cvs.savannah.nongnu.org/viewvc/linphone/mediastreamer2/src/msspeex.c?root=linphone&view=markup > > I'm not sure of my implementation (mainly I have a strange behavior of > "speex_bits_remaining"...)I'm not really sure I understand your code, but it looks much more complicated than it should be. Normally, the decode code should look like: speex_bits_init(&bits); speex_bits_read_from(&bits, packet, length); while (1) { err = speex_decode_int(state, bits, out_buffer); if (err != 0) break; play_audio(out_buffer); } if (err == -2) ms_warning("there was an error decoding"); I see from your code that you use speex_bits_remaining(&bits) but I can't really understand why. Could you explain? Jean-Marc
On Thu, 17 May 2007, Jean-Marc Valin wrote:>>> Consider a device that only has enough ROM to store one set of >>> quantization tables (the limitation could also be about speed, network, >>> ...). If you specify MUST be able to decode, then it means that this >>> device simply *cannot* implement the spec *at all*. This is bad for >>> interoperability. >> >> For me: device which don't have all mode implemented are already banned: >> the SDP offer/answer model is based on codec and parameter *proposal*: >> still the sender is *allowed* to not send the given mode. That said, UA >> that don't support all modes are already not capable of >> interoperability: they will receive sometimes modes that they do not >> proposed and do not support. > > The same reasoning could apply to other codecs as well: if a client > doesn't support G.729, you can't say it's broken if someone unexpectedly > start sending that. Of course, in the case of Speex, we try and do our > best...I'm not talking about codec proposal: you cannot send G729 to a UA that didn't offered it. While you can send packetisation interval of 30ms even if the UA offered that he wish to receive 20ms. I think anyway, that adding "mode=any" is fixing the mode issue, so it's now possible to specify that you don't support some of the modes.>> One possible way would be to use "mode=any" to clearly indicate that the >> UA support any modes. The prefered mode could be specified by order of >> preference: "mode=5;mode=any". Other "limited device" would announce all >> modes supported: "mode=5;mode=3;mode=1" > > Yes, I agree with that. > >> Samples for SDP negotiation must clearly indicates this behaviour: >> >> m=audio 8088 RTP/AVP 97 >> a=rtpmap:97 speex/8000 >> >> -> equivalent to "mode=any" (for smooth evolution of current >> implementations) > > Shouldn't clients have at least a preferred mode? Otherwise, if everyone > says "any", which bit-rate do you choose?But I gave you example in previous mail! Clients that support any mode and prefer "mode=4" would say: mode=4:mode=any> <ship> > >> The above would make sense and compliant application would >> be interoperable? right? > > I think so. > >> That would be perfect for me, and I guess would be fine for you? > > Yes, I think it's the best compromise -- and actually not very far from > what we have now. I would still keep the thing that says 8 kbps SHOULD > be supported. Or maybe we can say "all modes SHOULD be supported and if > it's not possible, then at least 8 kbps (mode 3) SHOULD be supported".I think this is close to be equivalent anyway.>>>> The other way would be to make it transparent like g279. >>> >>> Not sure what kind of transparence you mean? The Speex decoder (unless >>> you remove some tables) is able to decode anything without even knowing >>> how it was encoded. >> >> Except for limited device which are not capable of decoding the "mode" >> that were removed from code ;) >> >> If I understood well the speex stuff: a decoder without wideband >> implementation would be capable of locating the narrowband part >> of the speex bitstream and return at least narrowband in audio? right? > > Actually, if you give wideband data to a narrowband decoder, it will > decode it as narrowband without even *realising* it's wideband (the > wideband bits are transparently ignored). > >> Is this possible? (I guess it's not that complex): >> --->wideband data --> NARROW BAND DECODER --> narrow band decoded audio! > > yes, that's possible. > >> This does not seems possible: >> --->mode=4 nband data --> LIMITED DEVICE ---> correct nband data >> WITH MODE=3 ONLY > > That's not possible. > >>> If all you mean is "do your best to decode anything you get no matter >>> how different it is from what you asked for", then I agree. >> >> This has to be at least of recommendation so people are aware that speex >> implementation allow this behavior: it was unclear to people that they >> don't need to set option before starting decode process. Because they >> get a lot of decode failure before, I think speex users are more prepared >> now. Anyway, it's good to inform people. > > What do you mean by "they get a lot of decode failure"?Just that implementation was full of bugs.>> Sometimes, draft are providing some C code: I would love to see some kind >> of annex with a sample routine to decode speex data. >> >> Something like: static void dec_process(MSFilter *f) >> http://cvs.savannah.nongnu.org/viewvc/linphone/mediastreamer2/src/msspeex.c?root=linphone&view=markup >> >> I'm not sure of my implementation (mainly I have a strange behavior of >> "speex_bits_remaining"...) > > I'm not really sure I understand your code, but it looks much more > complicated than it should be. Normally, the decode code should look like: > > speex_bits_init(&bits); > speex_bits_read_from(&bits, packet, length); > while (1) > { > err = speex_decode_int(state, bits, out_buffer); > if (err != 0) > break; > play_audio(out_buffer); > } > if (err == -2) > ms_warning("there was an error decoding"); > > > I see from your code that you use speex_bits_remaining(&bits) but I > can't really understand why. Could you explain?I don't know...... I think that when I implemented this, that helped me to get out the loop. It seems that in your code you call "speex_decode_int" one more time than me: I tried to call it only when "speex_decode_int" will succeed because I though it was bad to call it when there was no data any more. It seems I was wrong... Anyway: it's working fine... Would you mind if I rewrite the draft myself with the idea we discussed? Do you have the xml version? tks Aymeric> Jean-Marc > >
----draft text: " The encoding and decoding algorithm can change the bit rate at any 20 msec frame boundary, with the bit rate change notification provided in-band with the bit stream. Each frame contains both "mode" (narrowband, wideband or ultra-wideband) and "sub-mode" (bit-rate) information in the bit stream. No out-of-band notification is required for the decoder to process changes in the bit rate sent by the encoder." -----Comment: Why not set fields in the payload to indicating mode/sub-mode? In the current design, the decoder need to decode the frame to know its mode/sub-mode. That need time and not acceptable for real time system. Thus, it's not impossible for the decoder to know the mode/sub-mode which can change at any frame boundary. On 5/17/07, Aymeric Moizard <jack@atosc.org> wrote:> > > > On Thu, 17 May 2007, Jean-Marc Valin wrote: > > >>> Consider a device that only has enough ROM to store one set of > >>> quantization tables (the limitation could also be about speed, > network, > >>> ...). If you specify MUST be able to decode, then it means that this > >>> device simply *cannot* implement the spec *at all*. This is bad for > >>> interoperability. > >> > >> For me: device which don't have all mode implemented are already > banned: > >> the SDP offer/answer model is based on codec and parameter *proposal*: > >> still the sender is *allowed* to not send the given mode. That said, UA > >> that don't support all modes are already not capable of > >> interoperability: they will receive sometimes modes that they do not > >> proposed and do not support. > >> The same reasoning could apply to other codecs as well: if a client > > doesn't support G.729, you can't say it's broken if someone unexpectedly > > start sending that. Of course, in the case of Speex, we try and do our > > best... > > I'm not talking about codec proposal: you cannot send G729 to a UA that > didn't offered it. While you can send packetisation interval of 30ms even > if the UA offered that he wish to receive 20ms. > > I think anyway, that adding "mode=any" is fixing the mode issue, so it's > now possible to specify that you don't support some of the modes. > > >> One possible way would be to use "mode=any" to clearly indicate that > the > >> UA support any modes. The prefered mode could be specified by order of > >> preference: "mode=5;mode=any". Other "limited device" would announce > all > >> modes supported: "mode=5;mode=3;mode=1" > > > > Yes, I agree with that. > > > >> Samples for SDP negotiation must clearly indicates this behaviour: > >> > >> m=audio 8088 RTP/AVP 97 > >> a=rtpmap:97 speex/8000 > >> > >> -> equivalent to "mode=any" (for smooth evolution of current > >> implementations) > > > > Shouldn't clients have at least a preferred mode? Otherwise, if everyone > > says "any", which bit-rate do you choose? > > But I gave you example in previous mail! Clients that support any mode and > prefer "mode=4" would say: > > mode=4:mode=any > > > <ship> > > > >> The above would make sense and compliant application would > >> be interoperable? right? > > > > I think so. > > > >> That would be perfect for me, and I guess would be fine for you? > > > > Yes, I think it's the best compromise -- and actually not very far from > > what we have now. I would still keep the thing that says 8 kbps SHOULD > > be supported. Or maybe we can say "all modes SHOULD be supported and if > > it's not possible, then at least 8 kbps (mode 3) SHOULD be supported". > > I think this is close to be equivalent anyway. > > >>>> The other way would be to make it transparent like g279. > >>> > >>> Not sure what kind of transparence you mean? The Speex decoder (unless > >>> you remove some tables) is able to decode anything without even > knowing > >>> how it was encoded. > >> > >> Except for limited device which are not capable of decoding the "mode" > >> that were removed from code ;) > >> > >> If I understood well the speex stuff: a decoder without wideband > >> implementation would be capable of locating the narrowband part > >> of the speex bitstream and return at least narrowband in audio? right? > > > > Actually, if you give wideband data to a narrowband decoder, it will > > decode it as narrowband without even *realising* it's wideband (the > > wideband bits are transparently ignored). > > > >> Is this possible? (I guess it's not that complex): > >> --->wideband data --> NARROW BAND DECODER --> narrow band decoded > audio! > > > > yes, that's possible. > > > >> This does not seems possible: > >> --->mode=4 nband data --> LIMITED DEVICE ---> correct nband data > >> WITH MODE=3 ONLY > > > > That's not possible. > > > >>> If all you mean is "do your best to decode anything you get no matter > >>> how different it is from what you asked for", then I agree. > >> > >> This has to be at least of recommendation so people are aware that > speex > >> implementation allow this behavior: it was unclear to people that they > >> don't need to set option before starting decode process. Because they > >> get a lot of decode failure before, I think speex users are more > prepared > >> now. Anyway, it's good to inform people. > > > > What do you mean by "they get a lot of decode failure"? > > Just that implementation was full of bugs. > > >> Sometimes, draft are providing some C code: I would love to see some > kind > >> of annex with a sample routine to decode speex data. > >> > >> Something like: static void dec_process(MSFilter *f) > >> > http://cvs.savannah.nongnu.org/viewvc/linphone/mediastreamer2/src/msspeex.c?root=linphone&view=markup > >> > >> I'm not sure of my implementation (mainly I have a strange behavior of > >> "speex_bits_remaining"...) > > > > I'm not really sure I understand your code, but it looks much more > > complicated than it should be. Normally, the decode code should look > like: > > > > speex_bits_init(&bits); > > speex_bits_read_from(&bits, packet, length); > > while (1) > > { > > err = speex_decode_int(state, bits, out_buffer); > > if (err != 0) > > break; > > play_audio(out_buffer); > > } > > if (err == -2) > > ms_warning("there was an error decoding"); > > > > > > I see from your code that you use speex_bits_remaining(&bits) but I > > can't really understand why. Could you explain? > > I don't know...... > > I think that when I implemented this, that helped me to get out the loop. > It seems that in your code you call "speex_decode_int" one more time > than me: I tried to call it only when "speex_decode_int" will succeed > because I though it was bad to call it when there was no data any more. > It seems I was wrong... Anyway: it's working fine... > > Would you mind if I rewrite the draft myself with the idea we discussed? > Do you have the xml version? > > tks > Aymeric > > > Jean-Marc > > > > > > _______________________________________________ > Speex-dev mailing list > Speex-dev@xiph.org > http://lists.xiph.org/mailman/listinfo/speex-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.xiph.org/pipermail/speex-dev/attachments/20070518/32d3de01/attachment.htm
>> Yes, I think it's the best compromise -- and actually not very far from >> what we have now. I would still keep the thing that says 8 kbps SHOULD >> be supported. Or maybe we can say "all modes SHOULD be supported and if >> it's not possible, then at least 8 kbps (mode 3) SHOULD be supported". > > I think this is close to be equivalent anyway.Well, it's got one more thing: the recommendation that if you're only going to support one mode, it should be mode 3 (8 kbps).>> What do you mean by "they get a lot of decode failure"? > > Just that implementation was full of bugs.I'm just thinking one has to be really creative to screw up decoding of a Speex packet considering that any decoder can decode any packet.>> I see from your code that you use speex_bits_remaining(&bits) but I >> can't really understand why. Could you explain? > > I don't know...... > > I think that when I implemented this, that helped me to get out the loop. > It seems that in your code you call "speex_decode_int" one more time > than me: I tried to call it only when "speex_decode_int" will succeed > because I though it was bad to call it when there was no data any more. > It seems I was wrong... Anyway: it's working fine...It's still a bit weird and theoretically incorrect because you won't be able to handle null frames (indicating silence/CNG), that are only 5 bits. It probably worked because few (if any) people ever use them.> Would you mind if I rewrite the draft myself with the idea we discussed? > Do you have the xml version?Can you sort that out with Alfred? Jean-Marc