Hi, I implemented a small patch that allows the internal convergence state of the echo canceller to be saved in a file for later use, especially after a process restart or machine reboot. This enables immediate echo cancellation the second time the AEC is run. Of course this works only if the acoustic environment of the device doesn't change and if the soundcard latency is constant. To use this feature, proceeed as this: - use speex_echo_ctl() with SPEEX_ECHO_GET_BLOB to obtain a SpeexEchoStateBlob, at the end of an audio session, when the echo canceller is supposed to be converged. - save the blob to a disk file for example, using speex_echo_state_blob_get_data() and speex_echo_state_blob_get_size() to retrieve the actual data To restore the state later, after a reboot or a process restart, do the following: - read the data from the file where you previously save the blob's contents - instanciate a blob object from this data using speex_echo_state_blob_new_from_memory() - assign the blob to the echo canceller using speex_echo_ctl() with SPEEX_ECHO_SET_BLOB, before the echo canceller starts processing data. The way it works is that is saves the foreground weights of the echo canceller filter. When restored, it is also restored in the background filter. Some sanity checks on the configuration parameters ( frame size, filter length, channels) are performed to verify that the restoration is applicable to the current echo state object. I wonder if there are parameters other than weights that should be saved too. Feel free to comment, adapt, apply or reject. On a side comment, I would like to congratulate the author(s) of the Speex Echo Canceller, because it really works very well compared to competition and provide good cancellation even in difficult situations. The performance of the non-linear echo suppression with the speex_preprocess() api is really amazing. Best regards, Simon -------------- next part -------------- A non-text attachment was scrubbed... Name: echo_save_restore.patch Type: text/x-patch Size: 5906 bytes Desc: not available Url : http://lists.xiph.org/pipermail/speex-dev/attachments/20110404/18b1954d/attachment.bin
On 04/04/2011 06:58 PM, Simon Morlat wrote:> Hi, > > I implemented a small patch that allows the internal convergence state > of the echo canceller to be saved in a file for later use, especially > after a process restart or machine reboot. This enables immediate echo > cancellation the second time the AEC is run. > > Of course this works only if the acoustic environment of the device > doesn't change and if the soundcard latency is constant.That's probably a fair assumption for a desktop machine, but totally bogus for a notebook. Is there a clear way to tell the difference, and make a judgement as to whether this feature should be enabled.> To use this feature, proceeed as this: > - use speex_echo_ctl() with SPEEX_ECHO_GET_BLOB to obtain a > SpeexEchoStateBlob, at the end of an audio session, when the echo > canceller is supposed to be converged. > - save the blob to a disk file for example, using > speex_echo_state_blob_get_data() and speex_echo_state_blob_get_size() to > retrieve the actual data > > To restore the state later, after a reboot or a process restart, do the > following: > - read the data from the file where you previously save the blob's > contents > - instanciate a blob object from this data using > speex_echo_state_blob_new_from_memory() > - assign the blob to the echo canceller using speex_echo_ctl() with > SPEEX_ECHO_SET_BLOB, before the echo canceller starts processing data. > > The way it works is that is saves the foreground weights of the echo > canceller filter. When restored, it is also restored in the background > filter. Some sanity checks on the configuration parameters ( frame size, > filter length, channels) are performed to verify that the restoration is > applicable to the current echo state object. > I wonder if there are parameters other than weights that should be saved > too. Feel free to comment, adapt, apply or reject. > > On a side comment, I would like to congratulate the author(s) of the > Speex Echo Canceller, because it really works very well compared to > competition and provide good cancellation even in difficult situations. > The performance of the non-linear echo suppression with the > speex_preprocess() api is really amazing.It works well if the sound card has synchronised mic and speaker sampling, and is horrible otherwise. So, its great on Macs, and pot luck on PCs. Perhaps the canceller should detect one sampling rate sliding against the other, and talk alternate action? Steve
For what it's worth, I've tried various ways to fix the "slide" that happens when your microphone gives you (say) 20.5 ms or 19.5 ms worth of samples every 20 ms. Unfortunately, all of my attempted solutions have ended up making the echo cancellation problem worse. For instance, I would watch the microphone stream, and if, over time, I saw that it was sending (say) 640 bytes bytes every 19.5 ms instead of every 20 ms, I would buffer and downsample the stream so I could send 640 bytes to the echo canceller every 20 ms instead of 640 bytes every 19.5 ms. But it never seemed to work. Obviously the buffering screws with the AEC, but my assumption had been that so long as the buffering was consistent, the AEC would adjust. But perhaps I was just doing something wrong. I'd be interested in hearing other folks' attempts at a solution. For what it's worth, even when there's significant slide, if you combine the echo canceller with the echo suppressor, it works OK: I'm getting something like 90% of the echo removed in my testing. The problem really becomes noticeable when there are more than two or three people on the call: all the little bits of remaining echo combine to create a really nasty "subconversation" going on in the background that makes it sound like you're all in a very noisy coffee shop. Ken Smith Cell: 425-443-2359 Email: ken at alanta.com Blog: http://blog.wouldbetheologian.com/ On Mon, Apr 4, 2011 at 5:08 AM, Steve Underwood <steveu at coppice.org> wrote:> On 04/04/2011 06:58 PM, Simon Morlat wrote: > > Hi, > > > > I implemented a small patch that allows the internal convergence state > > of the echo canceller to be saved in a file for later use, especially > > after a process restart or machine reboot. This enables immediate echo > > cancellation the second time the AEC is run. > > > > Of course this works only if the acoustic environment of the device > > doesn't change and if the soundcard latency is constant. > That's probably a fair assumption for a desktop machine, but totally > bogus for a notebook. Is there a clear way to tell the difference, and > make a judgement as to whether this feature should be enabled. > > > To use this feature, proceeed as this: > > - use speex_echo_ctl() with SPEEX_ECHO_GET_BLOB to obtain a > > SpeexEchoStateBlob, at the end of an audio session, when the echo > > canceller is supposed to be converged. > > - save the blob to a disk file for example, using > > speex_echo_state_blob_get_data() and speex_echo_state_blob_get_size() to > > retrieve the actual data > > > > To restore the state later, after a reboot or a process restart, do the > > following: > > - read the data from the file where you previously save the blob's > > contents > > - instanciate a blob object from this data using > > speex_echo_state_blob_new_from_memory() > > - assign the blob to the echo canceller using speex_echo_ctl() with > > SPEEX_ECHO_SET_BLOB, before the echo canceller starts processing data. > > > > The way it works is that is saves the foreground weights of the echo > > canceller filter. When restored, it is also restored in the background > > filter. Some sanity checks on the configuration parameters ( frame size, > > filter length, channels) are performed to verify that the restoration is > > applicable to the current echo state object. > > I wonder if there are parameters other than weights that should be saved > > too. Feel free to comment, adapt, apply or reject. > > > > On a side comment, I would like to congratulate the author(s) of the > > Speex Echo Canceller, because it really works very well compared to > > competition and provide good cancellation even in difficult situations. > > The performance of the non-linear echo suppression with the > > speex_preprocess() api is really amazing. > It works well if the sound card has synchronised mic and speaker > sampling, and is horrible otherwise. So, its great on Macs, and pot luck > on PCs. Perhaps the canceller should detect one sampling rate sliding > against the other, and talk alternate action? > > Steve > _______________________________________________ > Speex-dev mailing list > Speex-dev at xiph.org > http://lists.xiph.org/mailman/listinfo/speex-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.xiph.org/pipermail/speex-dev/attachments/20110404/f623edc7/attachment.htm
Hi Steve,> That's probably a fair assumption for a desktop machine, but totally > bogus for a notebook. Is there a clear way to tell the difference, and > make a judgement as to whether this feature should be enabled.I would say that for a large ammount of embedded device that sit in a wall for their entire life, this assumption is true. Also, for smartphones where echo comes from mechanical coupling of mic and receiver, this is true also.> It works well if the sound card has synchronised mic and speaker > sampling, and is horrible otherwise. So, its great on Macs, and pot luck > on PCs. Perhaps the canceller should detect one sampling rate sliding > against the other, and talk alternate action?Yes this is true. The two streams synchronization is key to success. In mediastreamer2 (linphone's media engine) stream synchronisation is finally implemented (mediastreamer2/src/speexec.c). For each buffer read from a soundcard, the same data is expected on the rtp side. If not enough data, then silence is injected to the soundcard and used for speex_echo_cancel(). Some control is performed on the input that comes from rtp to monitor its size. If it grows too much (meaning that we receive more samples than the soundcard can consume), then I silently drop samples spread over a long time in a way it can't be heard. However, for cases where input is a webcam microphone and output a pci soundcard, this doesn't work well if the two clocks diverge rapidly. I have no solution yet for this case. Simon> > Steve > _______________________________________________ > Speex-dev mailing list > Speex-dev at xiph.org > http://lists.xiph.org/mailman/listinfo/speex-dev