Hi all, So I shutdown my domU (for backing it up) and do du -h /var/lib/xen/ images/host.img ... shows 18GB So I copy it to my NFS mounted backup machine and upon doing a du -h / nfs-share/backups/host.img ... show 7.6GB. I log on to that NFS server and its still shows 7.6GB So when I scp it to a remote filesystem the image shows as 300GB! This domU is actually set to have 300G but I didn''t allocate all of the disk space during domU creation so that it can grow when needed. Any ideas? I would prefer some predictability here. The domU (Centos 5.2) is using LVM. My dom0 is also Centos 5.2 and the version that ships with it is Xen 3.0.3 (according to rpm -qa | grep xen) - Brian _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Fri, Jan 16, 2009 at 12:54 PM, Brian Krusic <brian@krusic.com> wrote:> Hi all, > > So I shutdown my domU (for backing it up) and do du -h > /var/lib/xen/images/host.img > > ... shows 18GB > > So I copy it to my NFS mounted backup machine and upon doing a du -h > /nfs-share/backups/host.img > > ... show 7.6GB. > > I log on to that NFS server and its still shows 7.6GB > > So when I scp it to a remote filesystem the image shows as 300GB!just guessing.... is your image a raw sparse file?, and the Dom0 and NFS server have different blocksizes? -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Javier, My dom0 is 1TB and using LVM and so my domU files are on that 500GB LVM. My nfs server is around 10TB and is XFS. Would my image be a sparse file since I didn''t allocate full image size at creation? - Brian On Jan 16, 2009, at 10:02 AM, Javier Guerra wrote:> On Fri, Jan 16, 2009 at 12:54 PM, Brian Krusic <brian@krusic.com> > wrote: >> Hi all, >> >> So I shutdown my domU (for backing it up) and do du -h >> /var/lib/xen/images/host.img >> >> ... shows 18GB >> >> So I copy it to my NFS mounted backup machine and upon doing a du -h >> /nfs-share/backups/host.img >> >> ... show 7.6GB. >> >> I log on to that NFS server and its still shows 7.6GB >> >> So when I scp it to a remote filesystem the image shows as 300GB! > > just guessing.... is your image a raw sparse file?, and the Dom0 and > NFS server have different blocksizes? > > > > > -- > Javier_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I meant to say that my domUs are on that 1TB LVM and not 500GB. - Brian On Jan 16, 2009, at 10:19 AM, Brian Krusic wrote:> Hi Javier, > > My dom0 is 1TB and using LVM and so my domU files are on that 500GB > LVM. > > My nfs server is around 10TB and is XFS. > > Would my image be a sparse file since I didn''t allocate full image > size at creation? > > - Brian > > > > > On Jan 16, 2009, at 10:02 AM, Javier Guerra wrote: > >> On Fri, Jan 16, 2009 at 12:54 PM, Brian Krusic <brian@krusic.com> >> wrote: >>> Hi all, >>> >>> So I shutdown my domU (for backing it up) and do du -h >>> /var/lib/xen/images/host.img >>> >>> ... shows 18GB >>> >>> So I copy it to my NFS mounted backup machine and upon doing a du -h >>> /nfs-share/backups/host.img >>> >>> ... show 7.6GB. >>> >>> I log on to that NFS server and its still shows 7.6GB >>> >>> So when I scp it to a remote filesystem the image shows as 300GB! >> >> just guessing.... is your image a raw sparse file?, and the Dom0 and >> NFS server have different blocksizes? >> >> >> >> >> -- >> Javier > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Brian Krusic wrote on Fri, 16 Jan 2009 10:19:02 -0800:> Would my image be a sparse file since I didn''t allocate full image > size at creation?correct, that is what sparse files are for ;-) In my experience when you copy a sparse file the result is the "full" file. So, the result of the scp is expected. I''m surprised by the copy result to your nfs, maybe it''s a result of using xfs. Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Kai, Thanks for the reply. I''ve done more tests and have confirmed that wne I do a simple cp to an nfs or local storage wether its ext3 or xfs, the size of the file is roughly 1/3 of the size of that same file in its soruce dir of /var/ lib/xen/images. The backups are fully functional and seem to have no issues but its still puzzling to me. I don''t consider this a bonus that the copy ends up being a lot smaller because this could be a prelude to disaster. Any one care to comment and why I am seeing this? Has any one else done backups of the domU files to other locations and seen a diff in size? I''ve even had the backup machine be a smaller disk the the source with the same results. - Brian On Jan 16, 2009, at 3:31 PM, Kai Schaetzl wrote:> Brian Krusic wrote on Fri, 16 Jan 2009 10:19:02 -0800: > >> Would my image be a sparse file since I didn''t allocate full image >> size at creation? > > correct, that is what sparse files are for ;-) In my experience when > you > copy a sparse file the result is the "full" file. So, the result of > the > scp is expected. I''m surprised by the copy result to your nfs, maybe > it''s > a result of using xfs. > > Kai > > -- > Kai Schätzl, Berlin, Germany > Get your web at Conactive Internet Services: http://www.conactive.com > > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Brian Krusic wrote on Fri, 16 Jan 2009 17:50:59 -0800:> I''ve done more tests and have confirmed that wne I do a simple cp to > an nfs or local storage wether its ext3 or xfs, the size of the file > is roughly 1/3 of the size of that same file in its soruce dir of /var/ > lib/xen/images.I have verified that a local copy of a sparse file on ext3 actually produces another sparse file. I may have had this wrong in memory from the past. But I''m sure that under certain conditions I have seen that a copy of a sparse file created a non-sparse file with the maximum size. Here is what you can do to test with sparse files without playing with your VM all the time: create an empty sparse file (this one will have 2 MB): dd if=/dev/zero of=test.img bs=1k seek=2048 count=1 check "official" size: /home2/vm l test.img -rw-r--r-- 1 root root 2098176 Jan 17 16:34 test.img check real size on disk: /home2/vm du -s test.img 12 test.img Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Kai, I''ll try your suggestions. How can I find out if my domU file is really a sparse file? - Brian On Jan 17, 2009, at 8:31 AM, Kai Schaetzl wrote:> Brian Krusic wrote on Fri, 16 Jan 2009 17:50:59 -0800: > >> I''ve done more tests and have confirmed that wne I do a simple cp to >> an nfs or local storage wether its ext3 or xfs, the size of the file >> is roughly 1/3 of the size of that same file in its soruce dir of / >> var/ >> lib/xen/images. > > I have verified that a local copy of a sparse file on ext3 actually > produces another sparse file. I may have had this wrong in memory > from the > past. But I''m sure that under certain conditions I have seen that a > copy > of a sparse file created a non-sparse file with the maximum size. > > Here is what you can do to test with sparse files without playing with > your VM all the time: > > create an empty sparse file (this one will have 2 MB): > dd if=/dev/zero of=test.img bs=1k seek=2048 count=1 > > check "official" size: > /home2/vm l test.img > -rw-r--r-- 1 root root 2098176 Jan 17 16:34 test.img > > check real size on disk: > /home2/vm du -s test.img > 12 test.img > > > > > Kai > > -- > Kai Schätzl, Berlin, Germany > Get your web at Conactive Internet Services: http://www.conactive.com > > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Brian Krusic wrote on Sat, 17 Jan 2009 10:42:40 -0800:> How can I find out if my domU file is really a sparse file?In the way I showed you, e.g. use ls and du on it. If there is a considerable difference ... Maybe there are other methods that I don''t know of. Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sat, 17 Jan 2009, Brian Krusic wrote:> How can I find out if my domU file is really a sparse file?Do an "ls -sl" on it, and multiply the number of blocks (in the first column) by the file system block size (4096 for ext3). If this differs from the file size in bytes by 4096 or more, the file is definitely sparse. Steve _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sat, 17 Jan 2009, Steve Thompson wrote:> On Sat, 17 Jan 2009, Brian Krusic wrote: > >> How can I find out if my domU file is really a sparse file? > > Do an "ls -sl" on it, and multiply the number of blocks (in the first column) > by the file system block size (4096 for ext3). If this differs from the file > size in bytes by 4096 or more, the file is definitely sparse.Drat. I hate it when I do this; the first column is size in 1024-byte blocks. Steve _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users