ma qiang
2010-Oct-14 02:50 UTC
[Xen-users] copying or dding from a VM-template resulting in the whole system getting stuck
Hi all,
When I create a new VM by copying or dding from a VM-template, the
cp/dd operation takes almost all the memory and gets one cpu rate to almost
100%, resulting in the whole system getting stuck.
anyone knows how to perform this with less cost?
Thanks a lot.
Qiang
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Jeff Sturm
2010-Oct-14 13:17 UTC
RE: [Xen-users] copying or dding from a VM-template resulting in the whole system getting stuck
This on Linux? I find copying image files with O_DIRECT is helpful.
With the "dd" command, try "iflag=direct" and
"oflag=direct". Use a
reasonable buffer size, like bs=64k or so, to achieve a faster copy
operation.
-Jeff
From: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] On Behalf Of ma qiang
Sent: Wednesday, October 13, 2010 10:51 PM
To: xen-users@lists.xensource.com
Subject: [Xen-users] copying or dding from a VM-template resulting in
the whole system getting stuck
Hi all,
When I create a new VM by copying or dding from a VM-template, the
cp/dd operation takes almost all the memory and gets one cpu rate to
almost 100%, resulting in the whole system getting stuck.
anyone knows how to perform this with less cost?
Thanks a lot.
Qiang
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Fajar A. Nugraha
2010-Oct-15 01:48 UTC
Re: [Xen-users] copying or dding from a VM-template resulting in the whole system getting stuck
On Thu, Oct 14, 2010 at 9:50 AM, ma qiang <maqiang1984@gmail.com> wrote:> Hi all, > When I create a new VM by copying or dding from a VM-template, the > cp/dd operation takes almost all the memory and gets one cpu rate to almost > 100%, resulting in the whole system getting stuck. >Takes all the memory? How do you determine that?> anyone knows how to perform this with less cost? > >In common x86 systems today, disk (throughput and IOPS) is usually the bottleneck. Try ionice, something like ionice -n 7 -c 2 dd if= .... it should lower disk I/O priority for the dd process, giving room for other process. -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
ma qiang
2010-Oct-15 06:18 UTC
Re: [Xen-users] copying or dding from a VM-template resulting in the whole system getting stuck
Thanks a lot. It works much better. By the way, what is the reasonable buffer size. I have tested many based on different computers, and shows different results. On Thu, Oct 14, 2010 at 9:17 PM, Jeff Sturm <jeff.sturm@eprize.com> wrote:> This on Linux? I find copying image files with O_DIRECT is helpful. > > > > With the “dd” command, try “iflag=direct” and “oflag=direct”. Use a > reasonable buffer size, like bs=64k or so, to achieve a faster copy > operation. > > > > -Jeff > > > > *From:* xen-users-bounces@lists.xensource.com [mailto: > xen-users-bounces@lists.xensource.com] *On Behalf Of *ma qiang > *Sent:* Wednesday, October 13, 2010 10:51 PM > > *To:* xen-users@lists.xensource.com > *Subject:* [Xen-users] copying or dding from a VM-template resulting in > the whole system getting stuck > > > > Hi all, > > When I create a new VM by copying or dding from a VM-template, the > cp/dd operation takes almost all the memory and gets one cpu rate to almost > 100%, resulting in the whole system getting stuck. > > anyone knows how to perform this with less cost? > > > > Thanks a lot. > > Qiang > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Jeff Sturm
2010-Oct-15 14:06 UTC
RE: [Xen-users] copying or dding from a VM-template resulting in the whole system getting stuck
I''d consider 64KB reasonable. If you don''t give a block size
on the dd
command however, expect the copy to take a long time.
From: ma qiang [mailto:maqiang1984@gmail.com]
Sent: Friday, October 15, 2010 2:18 AM
To: Jeff Sturm
Cc: xen-users@lists.xensource.com
Subject: Re: [Xen-users] copying or dding from a VM-template resulting
in the whole system getting stuck
Thanks a lot. It works much better.
By the way, what is the reasonable buffer size. I have tested many based
on different computers, and shows different results.
On Thu, Oct 14, 2010 at 9:17 PM, Jeff Sturm <jeff.sturm@eprize.com>
wrote:
This on Linux? I find copying image files with O_DIRECT is helpful.
With the "dd" command, try "iflag=direct" and
"oflag=direct". Use a
reasonable buffer size, like bs=64k or so, to achieve a faster copy
operation.
-Jeff
From: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] On Behalf Of ma qiang
Sent: Wednesday, October 13, 2010 10:51 PM
To: xen-users@lists.xensource.com
Subject: [Xen-users] copying or dding from a VM-template resulting in
the whole system getting stuck
Hi all,
When I create a new VM by copying or dding from a VM-template, the
cp/dd operation takes almost all the memory and gets one cpu rate to
almost 100%, resulting in the whole system getting stuck.
anyone knows how to perform this with less cost?
Thanks a lot.
Qiang
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Greg Woods
2010-Oct-15 17:10 UTC
RE: [Xen-users] copying or dding from a VM-template resulting in the whole system getting stuck
On Fri, 2010-10-15 at 10:06 -0400, Jeff Sturm wrote:> > > With the “dd” command, try “iflag=direct” and “oflag=direct”. Use a > reasonable buffer size, like bs=64k or so, to achieve a faster copy > operation. >It looks like this may depend on the particulars of the storage devices you have. For me, direct I/O was much slower, even much slower than using "cp". But using dd with a 64k buffer size was a huge speedup over "cp", thanks for the tip on that. The difference is marked enough that it will probably be worth modifying my cloning script to copy this way rather than straight cp. This is using 5GB images, a hardware raid device running RAID-5 with SCSI disks, and a DRBD-replicated file system: root@vmi1 vmgroup1]# time cp paraivm.img test.img real 0m46.174s user 0m1.324s sys 0m21.913s [root@vmi1 vmgroup1]# time dd if=paraivm.img of=test.img bs=64k 80000+0 records in 80000+0 records out 5242880000 bytes (5.2 GB) copied, 8.6259 seconds, 608 MB/s real 0m8.646s user 0m0.076s sys 0m8.505s [root@vmi1 vmgroup1]# time dd if=paraivm.img of=test.img bs=64k iflag=direct oflag=direct 80000+0 records in 80000+0 records out 5242880000 bytes (5.2 GB) copied, 109.316 seconds, 48.0 MB/s real 1m49.319s user 0m0.012s sys 0m12.209s --Greg _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users