Xen devel - Aug 2006 - [PATCH 0 of 6] dm-userspace xen integration

In June, Dan Smith posted a patch to add dm-userspace to the -xen kernel*.
Since then, an io completion notification mechanism for metadata writing has
been added*.  The kernel module, along with the the tools, consistently pass
fsstress and dbench tests on both x86_64 and x86_32.

These patches integrate dm-userspace into the xen tree and allow a cow domain
to be created.  Hopefully some people can start experimenting with it and we
look forward to hearing some feedback.

Cowd and its associated plugins are a bit rough right now, but they work, and
we plan on smoothing things out in the near future.

This patch is broken up into 6 parts:

[1/6]: A patch to the xen kernel in the patches/ directory.  Also the default
config files will build dm_userspace as a module.

[2/6]: Adds the base for cowd, the userspace tool, to the tools directory.

[3/6]: Adds internal libdmu support to cowd

[4/6]: The dscow plugin for cowd

[5/6]: The qcow plugin for cowd

[6/6]: A script to start the dm-userspace backend as well as an xm example
configuration file.

*http://lists.xensource.com/archives/html/xen-devel/2006-06/msg00372.html
*http://lists.xensource.com/archives/html/xen-devel/2006-06/msg00773.html

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 8/21/06, Ryan Grimm <grimm@us.ibm.com> wrote:
> In June, Dan Smith posted a patch to add dm-userspace to the -xen kernel*.
> Since then, an io completion notification mechanism for metadata writing
has
> been added*.  The kernel module, along with the the tools, consistently
pass
> fsstress and dbench tests on both x86_64 and x86_32.
>
> These patches integrate dm-userspace into the xen tree and allow a cow
domain
> to be created.  Hopefully some people can start experimenting with it and
we
> look forward to hearing some feedback.
Has this been posted to the devicemapper list?  I think someone
mentioned it had been, could you post a reference to any follow-up
discussion?  Is there clear indication that this will be picked up by
devicemapper because I don''t think we''ll want to have to
maintain this
in-tree forever.
> Cowd and its associated plugins are a bit rough right now, but they work,
and
> we plan on smoothing things out in the near future.
>
> This patch is broken up into 6 parts:
>
> [1/6]: A patch to the xen kernel in the patches/ directory.  Also the
default
> config files will build dm_userspace as a module.
>
> [2/6]: Adds the base for cowd, the userspace tool, to the tools directory.
This needs to be changed to fail gracefully if the auto* tools are not
installed.
Is the use of auto* tools absolutely necessary?  Could we checkin the
generated files as well?

Also, I think the tools/Makefile could do with a bit of cleanup where
we define %-build, %-install and %-clean targets which do the default
action and then override them with specific actions for ioemu and I
guess cowd.
> [3/6]: Adds internal libdmu support to cowd
Is this there because libdevmapper doesn''t support this yet?  Is there
any version of libdevmapper which supports this yet?
> [4/6]: The dscow plugin for cowd
>
> [5/6]: The qcow plugin for cowd
>
> [6/6]: A script to start the dm-userspace backend as well as an xm example
> configuration file.
I don''t think an extra example config file is needed for a single line
change.  Just add the example to the existing config files.

Does this work for qemu domains?  If so how, if not, what are your
plans to make it work for qemu domains?

     christian

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

CL> Has this been posted to the devicemapper list?  I think someone
CL> mentioned it had been, could you post a reference to any follow-up
CL> discussion?  

It has been posted to dm-devel, and through a few on-list and (mostly)
off-list mails, the maintainer has expressed interest in getting this
pushed into device-mapper proper soon.

CL> Is there clear indication that this will be picked up by
CL> devicemapper because 

Yes.  An older version is already in his staging tree, and the latest
version has been sent to him.

CL> I don''t think we''ll want to have to maintain this
in-tree forever.

I strongly agree.

CL> This needs to be changed to fail gracefully if the auto* tools are
CL> not installed.  Is the use of auto* tools absolutely necessary?

Since xm-test was accepted with the use of autotools, we did not see
any reason why this wouldn''t.

CL> Could we checkin the generated files as well?

I suppose, if that''s what you want.  It will result in some relatively
large patches of apparent garbage every time we make a change to the
build system.

CL> Is this there because libdevmapper doesn''t support this yet?  Is
CL> there any version of libdevmapper which supports this yet?

The libdevmapper changes are also being pushed to the device-mapper
maintainer.  For now, we are just compiling those directly into cowd
to avoid forcing people to patch their libdevmapper.

CL> Does this work for qemu domains?  If so how, if not, what are your
CL> plans to make it work for qemu domains?

I don''t see why it would not work.  It just exports a block device, so
if a qemu domain can use an LVM or a partition, then this will work.
I can get access to an HVM machine and verify.

-- 
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: danms@us.ibm.com


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, Aug 22, 2006 at 10:03:40AM +0100, Christian Limpach
wrote:
> >Cowd and its associated plugins are a bit rough right now, but they
work,
> >and
> >we plan on smoothing things out in the near future.
> >
> >This patch is broken up into 6 parts:
> >
> >[1/6]: A patch to the xen kernel in the patches/ directory.  Also the 
> >default
> >config files will build dm_userspace as a module.
> >
> >[2/6]: Adds the base for cowd, the userspace tool, to the tools
directory.
> 
> This needs to be changed to fail gracefully if the auto* tools are not
> installed.
> Is the use of auto* tools absolutely necessary?  Could we checkin the
> generated files as well?
> 
> Also, I think the tools/Makefile could do with a bit of cleanup where
> we define %-build, %-install and %-clean targets which do the default
> action and then override them with specific actions for ioemu and I
> guess cowd.
How about cleaning up the Makefile like this?  Then, if we can get our
cowd tool in the tree, we can just add it to the SUBDIRS-y, define our
all, install, and clean, and ensure we exit gracefully.

Signed-off-by: Ryan Grimm <grimm@us.ibm.com>
 
diff -r 9ebba79efbe9 -r 4c4b0563656d tools/Makefile
--- a/tools/Makefile	Mon Aug 21 15:03:02 2006 -0500
+++ b/tools/Makefile	Tue Aug 22 11:37:05 2006 -0500
@@ -19,33 +19,35 @@ SUBDIRS-y += libaio
 SUBDIRS-y += libaio
 SUBDIRS-y += blktap
 
+ifdef CONFIG_IOEMU
+SUBDIRS-y += ioemu
+export IOEMU_DIR ?= ioemu
+endif
+
 # These don''t cross-compile
 ifeq ($(XEN_COMPILE_ARCH),$(XEN_TARGET_ARCH))
 SUBDIRS-y += python
 SUBDIRS-y += pygrub
 endif
 
-.PHONY: all
+.PHONY: all 
 all: check
 	@set -e; for subdir in $(SUBDIRS-y); do \
-		$(MAKE) -C $$subdir $@; \
+		$(MAKE) $$subdir-$@; \
 	done
-	$(MAKE) ioemu
 
-.PHONY: install
+.PHONY: install 
 install: check
 	@set -e; for subdir in $(SUBDIRS-y); do \
-		$(MAKE) -C $$subdir $@; \
+		$(MAKE) $$subdir-$@; \
 	done
-	$(MAKE) ioemuinstall
 	$(INSTALL_DIR) -p $(DESTDIR)/var/xen/dump
 
 .PHONY: clean
 clean: check_clean
 	@set -e; for subdir in $(SUBDIRS-y); do \
-		$(MAKE) -C $$subdir $@; \
+		$(MAKE) $$subdir-$@; \
 	done
-	$(MAKE) ioemuclean
 
 .PHONY: distclean
 distclean: clean
@@ -58,16 +60,22 @@ check_clean:
 check_clean:
 	$(MAKE) -C check clean
 
-.PHONY: ioemu ioemuinstall ioemuclean
-ifdef CONFIG_IOEMU
-export IOEMU_DIR ?= ioemu
-ioemu ioemuinstall:
+.PHONY: %-all
+%-all:
+	$(MAKE) -C $* all
+
+.PHONY: %-install
+%-install: 
+	$(MAKE) -C $* install
+
+.PHONY: %-install
+%-clean: 
+	$(MAKE) -C $* clean
+
+.PHONY: ioemu ioemu-all ioemu-install ioemu-clean
+ioemu ioemu-all ioemu-install:
 	[ -f $(IOEMU_DIR)/config-host.mak ] || \
 	  (cd $(IOEMU_DIR) && sh configure --prefix=/usr)
-	$(MAKE) -C $(IOEMU_DIR) $(patsubst ioemu%,%,$@)
-ioemuclean:
+	$(MAKE) -C $(IOEMU_DIR) $(patsubst ioemu-%,%,$@)
+ioemu-clean:
 	$(MAKE) -C $(IOEMU_DIR) distclean
-else
-ioemu ioemuinstall ioemuclean:
-endif
-

> 
> >[3/6]: Adds internal libdmu support to cowd
> 
> Is this there because libdevmapper doesn''t support this yet?  Is
there
> any version of libdevmapper which supports this yet?
> 
> >[4/6]: The dscow plugin for cowd
> >
> >[5/6]: The qcow plugin for cowd
> >
> >[6/6]: A script to start the dm-userspace backend as well as an xm
example
> >configuration file.
> 
> I don''t think an extra example config file is needed for a single
line
> change.  Just add the example to the existing config files.
> 
> Does this work for qemu domains?  If so how, if not, what are your
> plans to make it work for qemu domains?
> 
>     christian
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
-- 
Thanks,
Ryan Grimm
IBM Linux Technology Center

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RG> These patches integrate dm-userspace into the xen tree and allow a
RG> cow domain to be created.  Hopefully some people can start
RG> experimenting with it and we look forward to hearing some
RG> feedback.

I would really appreciate it if some people could test these patches
out and report their success/failure on-list.  It should be very easy
to test for anyone that builds xen-unstable frequently.

Thanks!

-- 
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: danms@us.ibm.com


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 8/22/06, Dan Smith <danms@us.ibm.com> wrote:
> CL> This needs to be changed to fail gracefully if the auto* tools are
> CL> not installed.  Is the use of auto* tools absolutely necessary?
>
> Since xm-test was accepted with the use of autotools, we did not see
> any reason why this wouldn''t.
Well, this is part of the tools build, while xm-test isn''t.
> CL> Could we checkin the generated files as well?
>
> I suppose, if that''s what you want.  It will result in some
relatively
> large patches of apparent garbage every time we make a change to the
> build system.
Yeah, I''m not sure about this.

We could do the same we do for the firmware build, i.e. test for the
existence of the required tools and if they are not present, then skip
the build.
> CL> Does this work for qemu domains?  If so how, if not, what are your
> CL> plans to make it work for qemu domains?
>
> I don''t see why it would not work.  It just exports a block
device, so
> if a qemu domain can use an LVM or a partition, then this will work.
> I can get access to an HVM machine and verify.
qemu expects to being able to open(2) the name of the file/partition.

I guess dm-userspace configures a devicemapper device from the backend
script and then has the backend driver use that.  This is probably too
late for qemu to use it since qemu starts immediately and will try to
open it right away.

    christian

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

CL> Well, this is part of the tools build, while xm-test isn''t.

Gotcha.

CL> We could do the same we do for the firmware build, i.e. test for
CL> the existence of the required tools and if they are not present,
CL> then skip the build.

I think that sounds good.

CL> I guess dm-userspace configures a devicemapper device from the
CL> backend script and then has the backend driver use that.  This is
CL> probably too late for qemu to use it since qemu starts immediately
CL> and will try to open it right away.

So, everything past the ioemu: is passed to qemu, which open()s it,
right?  I see there will need to be some changes made here to allow
qemu domains to work.  Wouldn''t it make more sense to have the normal
device scripts run for qemu domains as well? (With the exception of
file: devices).  That would make it easier to add advanced block
device support for both types of domains without a special case for
HVM domains.

-- 
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: danms@us.ibm.com


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel