Libguestfs - Oct 2021 - [PATCH nbdkit] New filter: request-request

On 10/14/21 23:28, Richard W.M. Jones wrote:
> On Thu, Oct 14, 2021 at 04:22:27PM -0500, Eric Blake wrote:
>> I know this is just an RFC, but the idea looks like it may have merit.
>> Would we want to merge this functionality into the 'retry'
filter (a
>> single filter, with a knob on whether to retry single requests
>> vs. full connection), or would that be too heavyweight for one filter?
> 
> I did look at that, but the current retry filter is complicated enough
> and this would make it quite a lot more complicated (unless you can
> see some easy way to do it).
> 
> Also the two filters do have somewhat different behaviour and perhaps
> use cases.
> 
> I really need a way to test this with the curl plugin to check that it
> really does what I think it does.  In particular I'm not convinced
> curl will actually go through the whole redirect phase in this case,
> which would mean it doesn't actually solve Alexander's problem.
I wonder if one of the commonly used proxies (squid perhaps?) can be
configured to inject errors / fail requests intentionally with some
user-specified frequency.

Laszlo

On Fri, Oct 15, 2021 at 12:18:27PM +0200, Laszlo Ersek
wrote:
> On 10/14/21 23:28, Richard W.M. Jones wrote:
> > On Thu, Oct 14, 2021 at 04:22:27PM -0500, Eric Blake wrote:
> >> I know this is just an RFC, but the idea looks like it may have
merit.
> >> Would we want to merge this functionality into the 'retry'
filter (a
> >> single filter, with a knob on whether to retry single requests
> >> vs. full connection), or would that be too heavyweight for one
filter?
> > 
> > I did look at that, but the current retry filter is complicated enough
> > and this would make it quite a lot more complicated (unless you can
> > see some easy way to do it).
> > 
> > Also the two filters do have somewhat different behaviour and perhaps
> > use cases.
> > 
> > I really need a way to test this with the curl plugin to check that it
> > really does what I think it does.  In particular I'm not convinced
> > curl will actually go through the whole redirect phase in this case,
> > which would mean it doesn't actually solve Alexander's
problem.
> 
> I wonder if one of the commonly used proxies (squid perhaps?) can be
> configured to inject errors / fail requests intentionally with some
> user-specified frequency.
See other reply - I did check that curl does the right thing here.
However I do need to write a standalone test for this.  nbdkit carries
its own mini webserver for testing the curl plugin:

https://gitlab.com/nbdkit/nbdkit/-/blob/master/tests/web-server.c

It was only needed because the logical choice for testing -- Python's
built-in "python -m SimpleHTTPServer" -- doesn't support byte
ranges.

Ideally we'd not be extending this any further, but would use an
existing single file webserver which supports byte ranges and
redirects and has a BSD-like license.
https://www.gnu.org/software/libmicrohttpd/ has the wrong license but
links to several other interesting projects.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top