Hi,
I''ve got a large ext3 file system, created on the Dom0 host, that
I''d
like to make available to a domU guest.
The filesystem is built like this:
- 8x500GB SATA drives combined as /dev/md2 (RAID6)
- /dev/md2 designated an LVM PV
- Volume Group vg_media created using /dev/md2
- Logical Volume lv_media created in vg_media
- ext3 filesystem created on lv_media
I''m using the following disk config in my xen config file:
disk = [ ''phy:vg_host/lv_slim,xvda,w'',
''phy:vg_media/lv_media,xvdb,w'', ]
However, /dev/xvdb is not appearing as the correct size in the DomU guest:
>From /proc/partitions:
major minor #blocks name
202 16 782819328 xvdb
When I look at the same partition in the host, I see this:
major minor #blocks name
9 2 2930303616 md2
There appears to be a problem in passing the size of the device to the
DomU guest.
Can anyone shed any light on this?
Thanks,
R.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
> - 8x500GB SATA drives combined as /dev/md2 (RAID6) > - /dev/md2 designated an LVM PV > - Volume Group vg_media created using /dev/md2 > - Logical Volume lv_media created in vg_media > - ext3 filesystem created on lv_media > > > I''m using the following disk config in my xen config file: > > disk = [ ''phy:vg_host/lv_slim,xvda,w'', > ''phy:vg_media/lv_media,xvdb,w'', ] > > > However, /dev/xvdb is not appearing as the correct size in the DomUguest:> > >From /proc/partitions: > > major minor #blocks name > 202 16 782819328 xvdb > > When I look at the same partition in the host, I see this: > major minor #blocks name > 9 2 2930303616 md2 > > There appears to be a problem in passing the size of the device to the > DomU guest.Hmm, 2930303616 - 2^31 = 782819968 Argh -- I can see the problem: see the connect function in blkfront.c. Fortunately, it can be fixed without an interface change. Just change ''sectors'' from an unsigned long to a blkif_sector_t and update the xenbus_gather to use: "sectors", "%llu", §ors You''ll also need to edit the xenbus_printf (to %llu) in the connect function in blkback.c too. Please post a patch! Thanks, Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt wrote:>> major minor #blocks name >> 202 16 782819328 xvdb >> >> When I look at the same partition in the host, I see this: >> major minor #blocks name >> 9 2 2930303616 md2 >> >> There appears to be a problem in passing the size of the device to the >> DomU guest. > > Hmm, 2930303616 - 2^31 = 782819968 > > Argh -- I can see the problem: see the connect function in blkfront.c. > > Fortunately, it can be fixed without an interface change. Just change > ''sectors'' from an unsigned long to a blkif_sector_t and update the > xenbus_gather to use: "sectors", "%llu", §ors > > You''ll also need to edit the xenbus_printf (to %llu) in the connect > function in blkback.c too. > > Please post a patch!Ian, I''d love to post a patch, but I''m afraid I''m not a coder. I''m downloading the SRPMS as I type and I''ll give it a go, but it might be an idea if someone with more coding skills fixes this. I''ll post an update when I''ve had a go. R. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Robin Bowes wrote:> I''d love to post a patch, but I''m afraid I''m not a coder. > > I''m downloading the SRPMS as I type and I''ll give it a go, but it might > be an idea if someone with more coding skills fixes this. > > I''ll post an update when I''ve had a go.OK, I''ve made the change to blkfront.c but there is no xenbus_printf in blkback.c so I didn''t make that change. (I''m using xen-3.0.3 from the FC6 SRPM) I''ve rebuilt the xen RPM with this patch: diff -ur xen-3.0.3-rc3/linux-2.6-xen-sparse/drivers/xen/blkfront/blkfront.c xen-3.0.3-rc3.patched/linux-2.6-xen-sp arse/drivers/xen/blkfront/blkfront.c --- xen-3.0.3-rc3/linux-2.6-xen-sparse/drivers/xen/blkfront/blkfront.c 2006-10-10 15:23:43.000000000 +0100 +++ xen-3.0.3-rc3.patched/linux-2.6-xen-sparse/drivers/xen/blkfront/blkfront.c 2006-11-21 01:10:54.000000000 +000 0 @@ -294,7 +294,8 @@ */ static void connect(struct blkfront_info *info) { - unsigned long sectors, sector_size; + blkif_sector_r sectors; + unsigned long sector_size; unsigned int binfo; int err; @@ -305,7 +306,7 @@ DPRINTK("blkfront.c:connect:%s.\n", info->xbdev->otherend); err = xenbus_gather(XBT_NIL, info->xbdev->otherend, - "sectors", "%lu", §ors, + "sectors", "%llu", §ors, "info", "%u", &binfo, "sector-size", "%lu", §or_size, NULL); I installed the resulting RPMs (xen and xen-libs) and rebooted the dom0 host. However, the xvdb device still only shows up like this: major minor #blocks name 202 16 782819328 xvdb Did I not do it right? R. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> Robin Bowes wrote: > > I''d love to post a patch, but I''m afraid I''m not a coder. > > > > I''m downloading the SRPMS as I type and I''ll give it a go, but itmight> > be an idea if someone with more coding skills fixes this. > > > > I''ll post an update when I''ve had a go. > > OK, I''ve made the change to blkfront.c but there is no xenbus_printfin> blkback.c so I didn''t make that change. (I''m using xen-3.0.3 from the > FC6 SRPM)I meant blkback/xenbus.c It would also be good to change tools/blktap/lib/xenbus.c for good measure. The rest of the patch looks OK, modulo use of tab vs spaces. Thanks, Ian> I''ve rebuilt the xen RPM with this patch: > > diff -ur > xen-3.0.3-rc3/linux-2.6-xen-sparse/drivers/xen/blkfront/blkfront.c > xen-3.0.3-rc3.patched/linux-2.6-xen-sp > arse/drivers/xen/blkfront/blkfront.c > --- xen-3.0.3-rc3/linux-2.6-xen-sparse/drivers/xen/blkfront/blkfront.c > 2006-10-10 15:23:43.000000000 +0100 > +++ >xen-3.0.3-rc3.patched/linux-2.6-xen-sparse/drivers/xen/blkfront/blkfront .c> 2006-11-21 01:10:54.000000000 +000 > 0 > @@ -294,7 +294,8 @@ > */ > static void connect(struct blkfront_info *info) > { > - unsigned long sectors, sector_size; > + blkif_sector_r sectors; > + unsigned long sector_size; > unsigned int binfo; > int err; > > @@ -305,7 +306,7 @@ > DPRINTK("blkfront.c:connect:%s.\n", info->xbdev->otherend); > > err = xenbus_gather(XBT_NIL, info->xbdev->otherend, > - "sectors", "%lu", §ors, > + "sectors", "%llu", §ors, > "info", "%u", &binfo, > "sector-size", "%lu", §or_size, > NULL); > > > I installed the resulting RPMs (xen and xen-libs) and rebooted thedom0> host. > > However, the xvdb device still only shows up like this: > > major minor #blocks name > 202 16 782819328 xvdb > > Did I not do it right? > > R. > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Daniel P. Berrange
2006-Nov-21 01:51 UTC
Re: [Xen-devel] Re: Getting xen to recognise large disks
On Tue, Nov 21, 2006 at 01:41:07AM +0000, Robin Bowes wrote:> Robin Bowes wrote: > > I''d love to post a patch, but I''m afraid I''m not a coder. > > > > I''m downloading the SRPMS as I type and I''ll give it a go, but it might > > be an idea if someone with more coding skills fixes this. > > > > I''ll post an update when I''ve had a go. > > OK, I''ve made the change to blkfront.c but there is no xenbus_printf in > blkback.c so I didn''t make that change. (I''m using xen-3.0.3 from the > FC6 SRPM) > > I''ve rebuilt the xen RPM with this patch: > > diff -ur > xen-3.0.3-rc3/linux-2.6-xen-sparse/drivers/xen/blkfront/blkfront.c > xen-3.0.3-rc3.patched/linux-2.6-xen-sp > arse/drivers/xen/blkfront/blkfront.c > --- xen-3.0.3-rc3/linux-2.6-xen-sparse/drivers/xen/blkfront/blkfront.c > 2006-10-10 15:23:43.000000000 +0100 > +++ > xen-3.0.3-rc3.patched/linux-2.6-xen-sparse/drivers/xen/blkfront/blkfront.c > 2006-11-21 01:10:54.000000000 +000 > 0 > @@ -294,7 +294,8 @@ > */ > static void connect(struct blkfront_info *info) > { > - unsigned long sectors, sector_size; > + blkif_sector_r sectors; > + unsigned long sector_size; > unsigned int binfo; > int err; > > @@ -305,7 +306,7 @@ > DPRINTK("blkfront.c:connect:%s.\n", info->xbdev->otherend); > > err = xenbus_gather(XBT_NIL, info->xbdev->otherend, > - "sectors", "%lu", §ors, > + "sectors", "%llu", §ors, > "info", "%u", &binfo, > "sector-size", "%lu", §or_size, > NULL); > > > I installed the resulting RPMs (xen and xen-libs) and rebooted the dom0 > host. > However, the xvdb device still only shows up like this: > > major minor #blocks name > 202 16 782819328 xvdb > > Did I not do it right?I''m afraid not. blkfront is the device driver for the DomU guest kernel, rather than Dom0. Also in Fedora, the ''xen'' RPM only contains the userspace parts of xen for Dom0. The hypervisor & kernel itself are in the kernel-xen RPM (which is one of many built from the kernel SRPM). FYI, I opened a bugzilla against Fedora to track this problem since I can also trivially reproduce it by creating a (sparse) 5 TB block device and exporting it to a guest. https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=216555 On x86_64 meanwhile everything is sizing up correctly - I succesfully exported a 15 PB (yes, PB) block device to a PV guest. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=| _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Daniel P. Berrange wrote:> blkfront is the device driver for the DomU guest kernel, rather than Dom0. > Also in Fedora, the ''xen'' RPM only contains the userspace parts of xen > for Dom0. The hypervisor & kernel itself are in the kernel-xen RPM (which > is one of many built from the kernel SRPM). > > FYI, I opened a bugzilla against Fedora to track this problem since I can > also trivially reproduce it by creating a (sparse) 5 TB block device and > exporting it to a guest. > > https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=216555 > > On x86_64 meanwhile everything is sizing up correctly - I succesfully > exported a 15 PB (yes, PB) block device to a PV guest.OK, I''ll track that bug. This is going to be one of the longest server builds on record! I''ve got 2x250GB drives mirrored as the "system" drive, plus 8x500GB drives as the "data" drive, configured in RAID6. After getting over all the hardware problems, I found that grub only supports up to 8 disks, so I had to patch it to support 16 disks. Now I''m finding that xen guests have problems with big disks. <sigh> I''ll get there eventually! R. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2006-Nov-21 07:46 UTC
Re: [Xen-devel] Re: Getting xen to recognise large disks
I''ll make a patch today. -- Keir On 21/11/06 2:13 am, "Robin Bowes" <robin-lists@robinbowes.com> wrote:>> FYI, I opened a bugzilla against Fedora to track this problem since I can >> also trivially reproduce it by creating a (sparse) 5 TB block device and >> exporting it to a guest. >> >> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=216555 >> >> On x86_64 meanwhile everything is sizing up correctly - I succesfully >> exported a 15 PB (yes, PB) block device to a PV guest. > > OK, I''ll track that bug._______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser wrote:> On 21/11/06 2:13 am, "Robin Bowes" <robin-lists@robinbowes.com> wrote: > >>> FYI, I opened a bugzilla against Fedora to track this problem since I can >>> also trivially reproduce it by creating a (sparse) 5 TB block device and >>> exporting it to a guest. >>> >>> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=216555 >>> >>> On x86_64 meanwhile everything is sizing up correctly - I succesfully >>> exported a 15 PB (yes, PB) block device to a PV guest. >> OK, I''ll track that bug. > > I''ll make a patch today. >Thanks Keir, looking forward to testing it. R. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2006-Nov-21 11:34 UTC
Re: [Xen-devel] Re: Getting xen to recognise large disks
On 21/11/06 11:21, "Robin Bowes" <robin-lists@robinbowes.com> wrote:> Keir Fraser wrote: >> On 21/11/06 2:13 am, "Robin Bowes" <robin-lists@robinbowes.com> wrote: >> >> I''ll make a patch today. >> > > Thanks Keir, looking forward to testing it.If you don''t mind using the xen-unstable source repository, it''s changeset 12496:0c0ef61de06b. It probably hasn''t reached the public repository just yet (should very shortly though). -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser wrote:> On 21/11/06 11:21, "Robin Bowes" <robin-lists@robinbowes.com> wrote: > >> Keir Fraser wrote: >>> On 21/11/06 2:13 am, "Robin Bowes" <robin-lists@robinbowes.com> wrote: >>> >>> I''ll make a patch today. >>> >> Thanks Keir, looking forward to testing it. > > If you don''t mind using the xen-unstable source repository, it''s changeset > 12496:0c0ef61de06b. It probably hasn''t reached the public repository just > yet (should very shortly though).Erm, I''m hoping for a modified RPM :) Is this a kernel-level change, i.e. will I need to rebuild and install a new kernel? Or is it just the userland tools that have changed? For example, on my Dom0 host I have the following RPMs installed with "xen" in their name: # rpm -qa | grep xen kernel-xen-2.6.18-1.2798.fc6 xen-3.0.3-0.1.rc3bigdisk kernel-xen-2.6.18-1.2849.fc6 xen-libs-3.0.3-0.1.rc3bigdisk (The packages suffixed "bigdisk" are the result of my own attempt to patch for this problem) On the guests, I just have the kernel. So, I''m rather suspecting that I''ll need to build a new kernel and update both the host and the guests? R. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2006-Nov-21 11:58 UTC
Re: [Xen-devel] Re: Getting xen to recognise large disks
On 21/11/06 11:53, "Robin Bowes" <robin-lists@robinbowes.com> wrote:> So, I''m rather suspecting that I''ll need to build a new kernel and > update both the host and the guests?I''m afraid so! -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser wrote:> > > On 21/11/06 11:53, "Robin Bowes" <robin-lists@robinbowes.com> wrote: > >> So, I''m rather suspecting that I''ll need to build a new kernel and >> update both the host and the guests? > > I''m afraid so!Bugger! I don''t suppose you happen to know of a guide to rebuilding the FC6 kernel from SRPMs do you? R. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, Where can I get working patches for the latest kernel 2.6.18.x? Regards Claes Lindblom _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, It is better to see FC6 SRPM. http://download.fedora.redhat.com/pub/fedora/linux/core/development/source/SRPMS/ Thanks Atushi Sakai>Hi, >Where can I get working patches for the latest kernel 2.6.18.x? > >Regards >Claes Lindblom > >_______________________________________________ >Xen-devel mailing list >Xen-devel@lists.xensource.com >http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Daniel P. Berrange
2006-Nov-21 12:36 UTC
Re: [Xen-devel] Re: Getting xen to recognise large disks
On Tue, Nov 21, 2006 at 12:04:32PM +0000, Robin Bowes wrote:> Keir Fraser wrote: > > > > > > On 21/11/06 11:53, "Robin Bowes" <robin-lists@robinbowes.com> wrote: > > > >> So, I''m rather suspecting that I''ll need to build a new kernel and > >> update both the host and the guests? > > > > I''m afraid so! > > Bugger! > > I don''t suppose you happen to know of a guide to rebuilding the FC6 > kernel from SRPMs do you?I''ll post some notes to the fedora-xen mailing list, so we don''t bother xen-devel with Fedora specific noise that most people don''t need to read. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=| _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Daniel P. Berrange
2006-Nov-21 21:11 UTC
Re: [Xen-devel] Re: Getting xen to recognise large disks
On Tue, Nov 21, 2006 at 11:34:45AM +0000, Keir Fraser wrote:> On 21/11/06 11:21, "Robin Bowes" <robin-lists@robinbowes.com> wrote: > > > Keir Fraser wrote: > >> On 21/11/06 2:13 am, "Robin Bowes" <robin-lists@robinbowes.com> wrote: > >> > >> I''ll make a patch today. > >> > > > > Thanks Keir, looking forward to testing it. > > If you don''t mind using the xen-unstable source repository, it''s changeset > 12496:0c0ef61de06b. It probably hasn''t reached the public repository just > yet (should very shortly though).I''ve tested that changeset with the following - phy: against a 5 TB partition - file: against a 7.3 TB file In both cases the # of sectors matches in Dom0 vs DomU. For good measure I also ran Stephen Tweedie''s verify-data tool in the DomU to verify no data I/O wraparound issues elsewhere in the code & it passed without trouble. Blktap, however, is a different story - it is showing wraparound for disk size at the 2 TB size mark stil. The userspace blktap tools have totally inconsistent data types. Sometimes using int, sometimes long, sometimes unsigned long & sometimes uint64. I''m working on a patch which makes it - ''unsigned long long'' for # sectors - ''unsigned long'' for sector size - ''unsigned int'' for info This makes it match the data types used in blkfront/blkback exactly. With this patch applied, the DomU sees correct disk size, however, the verify-data tool is showing nasty data consistency issues when writing/reading to such a disk. So I think there is 32-bit wrap around somewhere in the I/O codepath for blktap. I''ll get back when I''ve found out more info... Regards, Dan. [1] http://people.redhat.com/sct/src/verify-data/ -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=| _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Daniel P. Berrange
2006-Nov-21 22:41 UTC
Re: [Xen-devel] Re: Getting xen to recognise large disks
On Tue, Nov 21, 2006 at 09:11:18PM +0000, Daniel P. Berrange wrote:> On Tue, Nov 21, 2006 at 11:34:45AM +0000, Keir Fraser wrote: > > On 21/11/06 11:21, "Robin Bowes" <robin-lists@robinbowes.com> wrote: > > > > > Keir Fraser wrote: > > >> On 21/11/06 2:13 am, "Robin Bowes" <robin-lists@robinbowes.com> wrote: > > >> > > >> I''ll make a patch today. > > >> > > > > > > Thanks Keir, looking forward to testing it. > > > > If you don''t mind using the xen-unstable source repository, it''s changeset > > 12496:0c0ef61de06b. It probably hasn''t reached the public repository just > > yet (should very shortly though). > > I''ve tested that changeset with the following > > - phy: against a 5 TB partition > - file: against a 7.3 TB file > > In both cases the # of sectors matches in Dom0 vs DomU. For good measure > I also ran Stephen Tweedie''s verify-data tool in the DomU to verify no > data I/O wraparound issues elsewhere in the code & it passed without > trouble. > > Blktap, however, is a different story - it is showing wraparound for disk > size at the 2 TB size mark stil. The userspace blktap tools have totally > inconsistent data types. Sometimes using int, sometimes long, sometimes > unsigned long & sometimes uint64. I''m working on a patch which makes it > > - ''unsigned long long'' for # sectors > - ''unsigned long'' for sector size > - ''unsigned int'' for info > > This makes it match the data types used in blkfront/blkback exactly. > With this patch applied, the DomU sees correct disk size, however, > the verify-data tool is showing nasty data consistency issues when > writing/reading to such a disk. So I think there is 32-bit wrap > around somewhere in the I/O codepath for blktap. I''ll get back when > I''ve found out more info...It turns out that blktap wasn''t (directly) at fault here. I was storing my file based disk images on an XFS formatted partition in the host. Well it appears that XFS doesn''t play nice with the async I/O + O_DIRECT options that blktap likes so all your data goes to /dev/null :-) I re-tested blktap + large file backed disks on ext3 & GFS and everything is working as expected. So stay away from a XFS+blktap combo if you like your data :-) Attaching the patch to blktap to fix 32-bit wraparound of sector counts. Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=| _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I installed the patched kernel and xen RPMS this morning and can confirm that this fixed my problem. I can now see my 2.7TB partition correctly in the DomU guest. Thanks to all for the quick fix, and to Dan for help with building the modified kernel RPM. R. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Daniel P. Berrange
2006-Nov-28 21:52 UTC
Re: [Xen-devel] Re: Getting xen to recognise large disks
On Tue, Nov 21, 2006 at 10:41:41PM +0000, Daniel P. Berrange wrote:> On Tue, Nov 21, 2006 at 09:11:18PM +0000, Daniel P. Berrange wrote: > > On Tue, Nov 21, 2006 at 11:34:45AM +0000, Keir Fraser wrote: > > > On 21/11/06 11:21, "Robin Bowes" <robin-lists@robinbowes.com> wrote: > > > > > > > Keir Fraser wrote: > > > >> On 21/11/06 2:13 am, "Robin Bowes" <robin-lists@robinbowes.com> wrote: > > > >> > > > >> I''ll make a patch today. > > > >> > > > > > > > > Thanks Keir, looking forward to testing it. > > > > > > If you don''t mind using the xen-unstable source repository, it''s changeset > > > 12496:0c0ef61de06b. It probably hasn''t reached the public repository just > > > yet (should very shortly though). > > > > I''ve tested that changeset with the following > > > > - phy: against a 5 TB partition > > - file: against a 7.3 TB file > > > > In both cases the # of sectors matches in Dom0 vs DomU. For good measure > > I also ran Stephen Tweedie''s verify-data tool in the DomU to verify no > > data I/O wraparound issues elsewhere in the code & it passed without > > trouble. > > > > Blktap, however, is a different story - it is showing wraparound for disk > > size at the 2 TB size mark stil. The userspace blktap tools have totally > > inconsistent data types. Sometimes using int, sometimes long, sometimes > > unsigned long & sometimes uint64. I''m working on a patch which makes it > > > > - ''unsigned long long'' for # sectors > > - ''unsigned long'' for sector size > > - ''unsigned int'' for info > > > > This makes it match the data types used in blkfront/blkback exactly. > > With this patch applied, the DomU sees correct disk size, however, > > the verify-data tool is showing nasty data consistency issues when > > writing/reading to such a disk. So I think there is 32-bit wrap > > around somewhere in the I/O codepath for blktap. I''ll get back when > > I''ve found out more info... > > It turns out that blktap wasn''t (directly) at fault here. I was storing my > file based disk images on an XFS formatted partition in the host. Well it > appears that XFS doesn''t play nice with the async I/O + O_DIRECT options > that blktap likes so all your data goes to /dev/null :-) > > I re-tested blktap + large file backed disks on ext3 & GFS and everything > is working as expected. So stay away from a XFS+blktap combo if you like > your data :-)FYI, in case anyone else out there is reading the archives..it turns out there is a kernel bug which caused the data corruption problems with XFS in this case - it wasn''t a xen or blktap issue. The root cause was that if you used O_DIRECT + async-IO on a sparse file, XFS ended up writing data into the wrong region of the file! So if you''re using XFS for storing file backed images, make sure they''re not sparse images, or use the old loopback driver which avoids the O_DIRECT+AIO codepaths. Gory details in https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=217098 Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=| _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel