Hi! Say I export an LVM logical volume from dom0 as /dev/xvda to the domU: disk = [ ''phy:xenimages/stan,xvda,w'' ] No I can lvresize xenimages/stan in dom0, but the domU stays ignorant of this change. How could I propagate the resize to the domU without rebooting or temporarily breaking its connection to /dev/xvda? Sort of a SCSI rescan, perhaps? -- Thanks, Feri. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ferenc Wagner wrote:> Hi! > > Say I export an LVM logical volume from dom0 as /dev/xvda to the domU: > > disk = [ ''phy:xenimages/stan,xvda,w'' ] > > No I can lvresize xenimages/stan in dom0, but the domU stays ignorant > of this change. How could I propagate the resize to the domU without > rebooting or temporarily breaking its connection to /dev/xvda? Sort > of a SCSI rescan, perhaps? >You have to resize the *file system* on the partition. Whether you can use a ''rescue CD'' inside the virtualized environment, to resize your partitions, is an interesting question and may depend on your installed OS quite a bit. You can certainly resize non-''/'' partitions from the live OS, and use such an environment to re-run grub if your boot loader has moved or gotten confused. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Nico Kadel-Garcia <nkadel@gmail.com> writes:> Ferenc Wagner wrote: > >> Say I export an LVM logical volume from dom0 as /dev/xvda to the domU: >> >> disk = [ ''phy:xenimages/stan,xvda,w'' ] >> >> No I can lvresize xenimages/stan in dom0, but the domU stays ignorant >> of this change. How could I propagate the resize to the domU without >> rebooting or temporarily breaking its connection to /dev/xvda? Sort >> of a SCSI rescan, perhaps? > > You have to resize the *file system* on the partition.Sure, that would be the next step. But I can''t resize the file system until the underlying partition gets bigger also from the domU point of view.> Whether you can use a ''rescue CD'' inside the virtualized > environment, to resize your partitions, is an interesting question > and may depend on your installed OS quite a bit. You can certainly > resize non-''/'' partitions from the live OS, and use such an > environment to re-run grub if your boot loader has moved or gotten > confused.I''m using LVM, so resizing anything is very well possible from the running OS, as soon as I have the (virtual) raw disk space for it. A reboot would certainly provide that, but the question is exactly how to avoid a reboot. It''s a low level block device problem, below that of filesystems or even LVM. -- Thanks, Feri. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi, On Sun, Apr 06, 2008 at 12:09:37PM +0100, Nico Kadel-Garcia wrote:> Ferenc Wagner wrote: > >No I can lvresize xenimages/stan in dom0, but the domU stays ignorant > >of this change. How could I propagate the resize to the domU without > >rebooting or temporarily breaking its connection to /dev/xvda? Sort > >of a SCSI rescan, perhaps? > > > You have to resize the *file system* on the partition. Whether you can > use a ''rescue CD'' inside the virtualized environment, to resize your > partitions, is an interesting question and may depend on your installed > OS quite a bit. You can certainly resize non-''/'' partitions from the > live OS, and use such an environment to re-run grub if your boot loader > has moved or gotten confused.Have you ever actually tried this with a block device exported to a domU? Because as far as I know, you cannot make Xen see the change in size of the block device without detaching it and then attaching it again (from dom0). This question comes up repeatedly on this list and someone usually says "you need to resize the filesystem as well" without understanding that the domU is not seeing the change of size in the block device. Without which it is of course impossible to resize whatever filesystem is on it. Ferenc, A technique I have used for customers that required to add and remove disk space without any loss of service is to give them their / on one block device and then supply them with multiple other block devices which they use as LVM PVs. These can be added and removed with LVM inside the domU, with the data shifting off/on the different devices as needed. Obvious downsides to this approach are the added complexity, small performance loss from LVM-on-xvd*-on-LVM, and lack of easy access to the data from dom0. Or there is always NFS.. I don''t know how or even if iSCSI, AoE or NBD cope with changing sizes of block devices but that is perhaps something to explore. Cheers, Andy _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Andy Smith <andy@strugglers.net> writes:> Ferenc Wagner wrote: > >> Now I can lvresize xenimages/stan in dom0, but the domU stays ignorant >> of this change. How could I propagate the resize to the domU without >> rebooting or temporarily breaking its connection to /dev/xvda? Sort >> of a SCSI rescan, perhaps? > > A technique I have used for customers that required to add and > remove disk space without any loss of service is to give them their > / on one block device and then supply them with multiple other block > devices which they use as LVM PVs. These can be added and removed > with LVM inside the domU, with the data shifting off/on the > different devices as needed.Huh, good idea, if not particularly nice... :)> Obvious downsides to this approach are the added complexity, small > performance loss from LVM-on-xvd*-on-LVM, and lack of easy access to > the data from dom0.Well, if the domU has to do lots of pvmoves, that kills preformance much worse than the LVM layers... Actually, looks like one could do with two exported block devices: move data off one, detach/resize/ attach it again, move data onto the new device. Meanwhile / can reside on an LV without noticing anything at all... Idea: maybe multipathing could solve this issue without ever needing to physically move any data... If the multipath layer can copy with devices changing size underneath, which I don''t know.> Or there is always NFS.. I don''t know how or even if iSCSI, AoE or > NBD cope with changing sizes of block devices but that is perhaps > something to explore.Well, the kernel SCSI disk layer knows how to rescan a device, which may have changed its size (SAN devices routinely change exported LUN sizes, much like outsourced LVM): see /sys/block/sd*/device/rescan. My gut feeling is that the same mechanism wouldn''t be too hard to employ for xvd devices, too. Unfortunately, it''s not there yet. -- Regards, Feri. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I think the developers have kind of let this one go. Once upon a time, in addition to block-detach and block-attach there was also block-configure. Block-configure supposedly told the inner DomU to refresh its idea of the block device. It stopped working a while back, a few people complained, it was never fixed. I haven''t tested it, but this would be the "right way" to do it...if it worked. On Apr 6, 2008, at 6:23 AM, Andy Smith wrote:> Hi, > > On Sun, Apr 06, 2008 at 12:09:37PM +0100, Nico Kadel-Garcia wrote: >> Ferenc Wagner wrote: >>> No I can lvresize xenimages/stan in dom0, but the domU stays >>> ignorant >>> of this change. How could I propagate the resize to the domU >>> without >>> rebooting or temporarily breaking its connection to /dev/xvda? Sort >>> of a SCSI rescan, perhaps? >>> >> You have to resize the *file system* on the partition. Whether you >> can >> use a ''rescue CD'' inside the virtualized environment, to resize your >> partitions, is an interesting question and may depend on your >> installed >> OS quite a bit. You can certainly resize non-''/'' partitions from the >> live OS, and use such an environment to re-run grub if your boot >> loader >> has moved or gotten confused. > > Have you ever actually tried this with a block device exported to a > domU? Because as far as I know, you cannot make Xen see the change > in size of the block device without detaching it and then attaching it > again (from dom0). > > This question comes up repeatedly on this list and someone usually > says "you need to resize the filesystem as well" without > understanding that the domU is not seeing the change of size in the > block device. Without which it is of course impossible to resize > whatever filesystem is on it. > > Ferenc, > > A technique I have used for customers that required to add and > remove disk space without any loss of service is to give them their > / on one block device and then supply them with multiple other block > devices which they use as LVM PVs. These can be added and removed > with > LVM inside the domU, with the data shifting off/on the different > devices as needed. > > Obvious downsides to this approach are the added complexity, small > performance loss from LVM-on-xvd*-on-LVM, and lack of easy > access to the data from dom0. > > Or there is always NFS.. I don''t know how or even if iSCSI, AoE or > NBD cope with changing sizes of block devices but that is perhaps > something to explore. > > Cheers, > Andy > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users-- Jayson Vantuyl Systems Architect Engine Yard jvantuyl@engineyard.com 1 866 518 9275 ext 204 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
2008/4/7, Jayson Vantuyl <jvantuyl@engineyard.com>:> I think the developers have kind of let this one go. > > Once upon a time, in addition to block-detach and block-attach there was > also block-configure. Block-configure supposedly told the inner DomU to > refresh its idea of the block device. It stopped working a while back, a > few people complained, it was never fixed. > > I haven''t tested it, but this would be the "right way" to do it...if it > worked.that was highly enlightening... block-reconfigure would have allowed to reliably resize storage without shutting down domUs (yes, yes, with pvresize or other rescans and of course a capable filesystem, but lets stick to the root issue). I used to concatenate filesystem images using dd skip=xx but never found a way to notify the domU of a successful change. add in the fact, that large numbers of [ phy:... or, even worse file: ] disk definitions tend to be unreliable and it would be REALLY great to get the feature back. maybe we can start a little rally to make someone from -devel aware this is a NEEDED feature? rgds, florian -- ''Sie brauchen sich um Ihre Zukunft keine Gedanken zu machen'' _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Consider this to be my vote for block-reconfigure working. I suspect it wouldn''t take much work actually. In retrospect, it''s completely possible to concatenate devices at runtime using device mapper. It''s the same thing that LVM does under the sheets. If you somehow got your root on a DM device (initrd maybe?), then you could live-modify it to add in another device you pass through. It''s fraught with peril, but completely possible. On Apr 7, 2008, at 9:56 AM, Florian Heigl wrote:> 2008/4/7, Jayson Vantuyl <jvantuyl@engineyard.com>: >> I think the developers have kind of let this one go. >> >> Once upon a time, in addition to block-detach and block-attach >> there was >> also block-configure. Block-configure supposedly told the inner >> DomU to >> refresh its idea of the block device. It stopped working a while >> back, a >> few people complained, it was never fixed. >> >> I haven''t tested it, but this would be the "right way" to do >> it...if it >> worked. > > that was highly enlightening... > > block-reconfigure would have allowed to reliably resize storage > without shutting down domUs (yes, yes, with pvresize or other rescans > and of course a capable filesystem, but lets stick to the root issue). > I used to concatenate filesystem images using dd skip=xx but never > found a way to notify the domU of a successful change. add in the > fact, that large numbers of [ phy:... or, even worse file: ] disk > definitions tend to be unreliable and it would be REALLY great to get > the feature back. > > maybe we can start a little rally to make someone from -devel aware > this is a NEEDED feature? > > rgds, > florian > > -- > ''Sie brauchen sich um Ihre Zukunft keine Gedanken zu machen''-- Jayson Vantuyl Systems Architect Engine Yard jvantuyl@engineyard.com 1 866 518 9275 ext 204 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Apr 07, 2008 at 11:51:36AM -0700, Jayson Vantuyl wrote:> Consider this to be my vote for block-reconfigure working. I suspect > it wouldn''t take much work actually. > > In retrospect, it''s completely possible to concatenate devices at > runtime using device mapper. It''s the same thing that LVM does under > the sheets. If you somehow got your root on a DM device (initrd > maybe?), then you could live-modify it to add in another device you > pass through. It''s fraught with peril, but completely possible. >Please check this recent thread on xen-devel: http://lists.xensource.com/archives/html/xen-devel/2008-09/msg00158.html It seems block-reconfigure has never worked for online resizing xen domU xvd disks.. It is now being looked at and considered for Xen 3.4 as a new feature. -- Pasi> On Apr 7, 2008, at 9:56 AM, Florian Heigl wrote: > > >2008/4/7, Jayson Vantuyl <jvantuyl@engineyard.com>: > >>I think the developers have kind of let this one go. > >> > >>Once upon a time, in addition to block-detach and block-attach > >>there was > >>also block-configure. Block-configure supposedly told the inner > >>DomU to > >>refresh its idea of the block device. It stopped working a while > >>back, a > >>few people complained, it was never fixed. > >> > >>I haven''t tested it, but this would be the "right way" to do > >>it...if it > >>worked. > > > >that was highly enlightening... > > > >block-reconfigure would have allowed to reliably resize storage > >without shutting down domUs (yes, yes, with pvresize or other rescans > >and of course a capable filesystem, but lets stick to the root issue). > >I used to concatenate filesystem images using dd skip=xx but never > >found a way to notify the domU of a successful change. add in the > >fact, that large numbers of [ phy:... or, even worse file: ] disk > >definitions tend to be unreliable and it would be REALLY great to get > >the feature back. > > > >maybe we can start a little rally to make someone from -devel aware > >this is a NEEDED feature? > > > >rgds, > >florian > > > >-- > >''Sie brauchen sich um Ihre Zukunft keine Gedanken zu machen'' > > -- > Jayson Vantuyl > Systems Architect > Engine Yard > jvantuyl@engineyard.com > 1 866 518 9275 ext 204 >> _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users