Libguestfs - Oct 2021 - [PATCH nbdkit] New filter: request-request

On Thu, Oct 14, 2021 at 04:22:27PM -0500, Eric Blake
wrote:
> I know this is just an RFC, but the idea looks like it may have merit.
> Would we want to merge this functionality into the 'retry' filter
(a
> single filter, with a knob on whether to retry single requests
> vs. full connection), or would that be too heavyweight for one filter?
I did look at that, but the current retry filter is complicated enough
and this would make it quite a lot more complicated (unless you can
see some easy way to do it).

Also the two filters do have somewhat different behaviour and perhaps
use cases.

I really need a way to test this with the curl plugin to check that it
really does what I think it does.  In particular I'm not convinced
curl will actually go through the whole redirect phase in this case,
which would mean it doesn't actually solve Alexander's problem.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

On Thu, Oct 14, 2021 at 10:28:07PM +0100, Richard W.M. Jones
wrote:
> I really need a way to test this with the curl plugin to check that it
> really does what I think it does.  In particular I'm not convinced
> curl will actually go through the whole redirect phase in this case,
> which would mean it doesn't actually solve Alexander's problem.
Surprisingly to me this technique does actually work as expected.

I tested this now by setting up a mod_rewrite rule to do simple
round-robin with one broken mirror.  This was sending back external
redirects, similar to how mirrorserver works.

(similar to lb.pl example here:
https://publib.boulder.ibm.com/httpserv/manual60/misc/rewriteguide.html )

I ran nbdkit with the filter and curl pointing to the mirror, and used
a single nbdsh session to read the mirror.  I could tell by the data
being returned that it was redirected to a different mirror each
request, and it recovered transparently (after 2 seconds) when hitting
the broken mirror.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

On 10/14/21 23:28, Richard W.M. Jones wrote:
> On Thu, Oct 14, 2021 at 04:22:27PM -0500, Eric Blake wrote:
>> I know this is just an RFC, but the idea looks like it may have merit.
>> Would we want to merge this functionality into the 'retry'
filter (a
>> single filter, with a knob on whether to retry single requests
>> vs. full connection), or would that be too heavyweight for one filter?
> 
> I did look at that, but the current retry filter is complicated enough
> and this would make it quite a lot more complicated (unless you can
> see some easy way to do it).
> 
> Also the two filters do have somewhat different behaviour and perhaps
> use cases.
> 
> I really need a way to test this with the curl plugin to check that it
> really does what I think it does.  In particular I'm not convinced
> curl will actually go through the whole redirect phase in this case,
> which would mean it doesn't actually solve Alexander's problem.
I wonder if one of the commonly used proxies (squid perhaps?) can be
configured to inject errors / fail requests intentionally with some
user-specified frequency.

Laszlo