Hi, first i will explain my experiment : We have ESX server running virtual machines .We want to migrate to XEN. As i did not found any tool for transfering ESX to for example real server, or better to XEN --> please correct me if i am wrong , i found following approach : i will inside virtual machine on ESX boot from Live-CD of CentOS , using dd i will transfer images trought scp to CentOs machine with xen. I believe this one should work, machines on ESX are Windows2003 Servers. So what i need to find is how to write a command which will transfer image trought scp , but for faster transfer also compress it before using tar, or gzip. *so my input will be dd if=/dev/sda1 , my output will be gzipped or tarred image on second side. sda1.img.gz* ** Thanks in advance! D. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.centos.org/pipermail/centos-virt/attachments/20080425/f7a4f81d/attachment.htm
On Fri, Apr 25, 2008 at 10:48:17PM +0200, David Hl??ik wrote:> Hi, > > first i will explain my experiment : > > We have ESX server running virtual machines .We want to migrate to XEN. As i > did not found any tool for transfering ESX to for example real server, or > better to XEN --> please correct me if i am wrong , i found following approach > : > > i will inside virtual machine on ESX boot from Live-CD of CentOS , using dd i > will transfer images trought scp to CentOs machine with xen. I believe this one > should work, machines on ESX are Windows2003 Servers. > So what i need to find is how to write a command which will transfer image > trought scp , but for faster transfer also compress it before using tar, or > gzip. > > so my input will be dd if=/dev/sda1 , my output will be gzipped or tarred image > on second side. sda1.img.gz >dd if=/dev/sda1 | gzip -c | ssh host "cat > output.gz" Sanity check on that? If you're not worried about security of this transfer, might be faster to use NFS or something. Ray
David Hl?c(ik wrote:> *so my input will be dd if=/dev/sda1 , my output will be gzipped or > tarred image on second side. sda1.img.gz* > ** >I haven't had a chance to test this, but something like: dd if=/dev/sda1 | gzip | ssh user@host /tmp/somescript And /tmp/somescript on 'host' can be something like: gunzip | dd of=/dev/sdb1
David Hl?c(ik wrote:> Hi, > > first i will explain my experiment : > > We have ESX server running virtual machines .We want to migrate to XEN. > As i did not found any tool for transfering ESX to for example real > server, or better to XEN --> please correct me if i am wrong , i found > following approach : > > i will inside virtual machine on ESX boot from Live-CD of CentOS , using > dd i will transfer images trought scp to CentOs machine with xen. I > believe this one should work, machines on ESX are Windows2003 Servers. > So what i need to find is how to write a command which will transfer > image trought scp , but for faster transfer also compress it before > using tar, or gzip. > > *so my input will be dd if=/dev/sda1 , my output will be gzipped or > tarred image on second side. sda1.img.gz* > **First, I'd expect some problems with the device drivers when you try to restore a vmware image onto a xen virtual environment - unless they present similar virtual hardware. However, for the actual copy operation, I'd recommend clonezilla-live (http://www.clonezilla.org/clonezilla-live/). Boot its iso image into a menu system that will let you mount the image storage space via nfs, samba, or ssh-fs, then tell it to save the whole disk. The advantage over dd is that it knows enough about ntfs and most linux file systems to just save the used blocks and when you restore it will let you pick from the list of images you've saved. -- Les Mikesell lesmikesell at gmail.com
On Fri, Apr 25, 2008 at 10:48:17PM +0200, David Hl??ik wrote:> > i will inside virtual machine on ESX boot from Live-CD of CentOS , using dd > i will transfer images trought scp to CentOs machine with xen. I believe > this one should work, machines on ESX are Windows2003 Servers. > So what i need to find is how to write a command which will transfer image > trought scp , but for faster transfer also compress it before using tar, or > gzip. > > *so my input will be dd if=/dev/sda1 , my output will be gzipped or tarred > image on second side. sda1.img.gz* > **dd will produce a raw image of the disk; at that level, there aren''t any files for tar to work on, it''s just one long stream of bytes. scp also works from files, so unless you were to save your image locally first, e.g., # dd if=/dev/sda1 of=/tmp/sda1.img # scp /tmp/sda1.img me@whoever.wherever.com: ...scp isn''t the right tool to use either. You could do something like this. Where "foo.example.com" is the Xen machine which will receive the image, and we''re logged in to your system running the CentOS CD (note: this is one command line; the backslash is just there to make that explicit and isn''t necessary if you type this all in at the prompt): # dd if=/dev/sda1 | gzip | ssh root@foo.example.com \ "gzip -d > /path/to/vm/images/windows2003sda.img" HTH, --michael _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi, fastest way is netcat. No compression overhead or anything else. you can reach 12MByte/s on a 100MBit/s LAN. on the target machine run : nc -l -p 12345 | dd of=target.img on the source maschine run: dd if=/dev/device | nc target 12345 If you have a relly fast CPU you can also pipe it through gzip -1 on both sides. Martin _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
hi, 2008/4/25, David Hláčik <david@hlacik.eu>:> i will inside virtual machine on ESX boot from Live-CD of CentOS , using dd > i will transfer images trought scp to CentOs machine with xen. I believe > this one should work, machines on ESX are Windows2003 Servers. > So what i need to find is how to write a command which will transfer image > trought scp , but for faster transfer also compress it before using tar, or > gzip. > > so my input will be dd if=/dev/sda1 , my output will be gzipped or tarred > image on second side. sda1.img.gzThere's also some tools (ntfsclone for one ntfs-aware tool) or g4u for a more centralized whole-disk-image approach that might do the same. also you might be able to extract the Raw VMDK's data on your ESX host using VMWare cli utils and compress/scp that. I know this works with the normal VMWare server and would assume ESX also has some utility to ease that. good luck :) florian -- 'Sie brauchen sich um Ihre Zukunft keine Gedanken zu machen' _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
2008/4/26 Florian Heigl <florian.heigl@gmail.com>:> hi, > > 2008/4/25, David Hláčik <david@hlacik.eu>: > > > > i will inside virtual machine on ESX boot from Live-CD of CentOS , using dd > > i will transfer images trought scp to CentOs machine with xen. I believe > > this one should work, machines on ESX are Windows2003 Servers. > > So what i need to find is how to write a command which will transfer image > > trought scp , but for faster transfer also compress it before using tar, or > > gzip. > > > > so my input will be dd if=/dev/sda1 , my output will be gzipped or tarred > > image on second side. sda1.img.gz > > There's also some tools (ntfsclone for one ntfs-aware tool) or g4u for > a more centralized whole-disk-image approach that might do the same. >Clonezilla is also a good choice for this ghost/clone approach. It builds in the ntfsclone, partimage, and dd methods for capturing images. It is quite flexible, supporting many ways of storing the disk/partition image. See: http://www.clonezilla.org/> also you might be able to extract the Raw VMDK's data on your ESX host > using VMWare cli utils and compress/scp that. I know this works with > the normal VMWare server and would assume ESX also has some utility to > ease that. > > > > good luck :) > florian > > > -- > 'Sie brauchen sich um Ihre Zukunft keine Gedanken zu machen' > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Martin Hierling wrote:> Hi, > > fastest way is netcat. No compression overhead or anything else. you > can reach 12MByte/s on a 100MBit/s LAN. > on the target machine run : nc -l -p 12345 | dd of=target.img > on the source maschine run: dd if=/dev/device | nc target 12345 > > If you have a relly fast CPU you can also pipe it through gzip -1 on both sides. >It doesn''t need to be _that_ fast: you should be able to compress on-the-fly faster than that for any machine made in the last few years, you''d only need something really fast for a gigabit network and then you''re still more likely to be limited by disk bandwidth than cpu speed. Having said that, netcat does have a very low overhead and is a good choice for transferring large files over the net. If you have firewall problems or security is a concern though, then "scp -C" (to enable compression) is also a very good choice and "scp -C remotehost:/dev/sda1 local.img" is actually pretty simple and will work quite well. Whether the disk image compresses well depends on what is in corresponding file system: if it''s a fairly new and fairly large disk then chances are that most of the blocks are full of zeroes which compress very well :-) If it''s a disk practically full of, say, video files then it won''t noticeably compress. In other words, you''ll get somewhere between 0.1% and 99.9% compression depending what''s on the disk -- and it''s hard to predict what you''ll actually get. Finally, you may need to take into account the network reliability. The larger the disk image and the slower (or more distant) the link, the more likely something will drop the connection part way through. If this is a problem you may wish to look at the count, skip and seek dd options. jch _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi, Am Freitag, 25. April 2008 22:48 schrieb David Hláčik: [..]> *so my input will be dd if=/dev/sda1 , my output will be gzipped or tarred > image on second side. sda1.img.gz*i would boot both, the old ESX and the new Xen VM with a Live-CD and let both connect to the network. So the command from the old ESX VM would be as root: # dd if=/dev/sda | gzip -c | ssh $DOMUIP "gunzip -c | dd of=/dev/sda" Or just to make an image file: # dd if=/dev/sda | gzip -c > /tmp/sda.img.gz Instead of using the whole disk (sda) you can use a partition (e.g. sda1) but on the target you have to fdisk first. -- greetings eMHa _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users