Xen users - Jan 2009 - Why is xen allocating entire disk/file?

Hi,

I''m running xen on centos 5.2 x86_64 that is totally updated. When I  
build a new system I pick the store to file, and I chose not to  
allocate the entire disk yet, as I know I wont need all the room yet ,  
but will be adding more sooner than later.

Soon as I kick off the install, I look in /var/lib/xen/images and I  
see my image at 100GB, even though I specified it not to use the hole  
disk.

Any thoughts on how to correct this guy? Is this a bug of some sort  
that is doing this, or is this the expected behavior?

Many thanks,

Charlie

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Jan 29, 2009 at 12:09 PM, Charlie Reddington
<charlie.reddington@gmail.com> wrote:
> Soon as I kick off the install, I look in /var/lib/xen/images and I see my
> image at 100GB, even though I specified it not to use the hole disk.
How did you check? ls -la?
Try ls -lash. It''ll show how much it actually used.

Regards,

Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Jan 28, 2009, at 11:14 PM, Fajar A. Nugraha wrote:
> On Thu, Jan 29, 2009 at 12:09 PM, Charlie Reddington
> <charlie.reddington@gmail.com> wrote:
>> Soon as I kick off the install, I look in /var/lib/xen/images and I  
>> see my
>> image at 100GB, even though I specified it not to use the hole disk.
>
> How did you check? ls -la?
> Try ls -lash. It''ll show how much it actually used.
>
> Regards,
>
> Fajar
It''s kind of weird, it''s not exactly showing up correctly on
my system.

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             446G   43G  381G  10% /
tmpfs                 3.9G     0  3.9G   0% /dev/shm

Right here alone, has me kind of sketcy. Total - 446G - 43G should =  
403G

But in var/lib/xen/images.....

# du -sh *
9.8G	ns1.img
9.8G	ns2..img
119M	shared01.img
20G		www.img

Doing the command as you say.....

9.8G -rwxrwxr-x 1 root root 9.8G Jan 28 15:18 ns1.img
9.8G -rwxrwxr-x 1 root root 9.8G Jan 28 15:18 ns2.img
134M -rwxrwxr-x 1 root root  98G Jan 28 15:18 shared01.img
  20G -rwxrwxr-x 1 root root  20G Jan 28 15:18 www.img

I''m just overall kind of confused how this is working.  I should in  
theory be able to over allocate a number of vm''s, so that when I get  
close to running out, I can just add more disk space as cash allows.

Thanks,

Charlie



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Jan 29, 2009 at 12:20 PM, Charlie Reddington
<charlie.reddington@gmail.com> wrote:
> # df -h
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/sda1             446G   43G  381G  10% /
> tmpfs                 3.9G     0  3.9G   0% /dev/shm
>
> Right here alone, has me kind of sketcy. Total - 446G - 43G should = 403G
No comment on that one :)
>
> But in var/lib/xen/images.....
>
> # du -sh *
> 9.8G    ns1.img
> 9.8G    ns2..img
> 119M    shared01.img
> 20G             www.img
>
> Doing the command as you say.....
>
> 9.8G -rwxrwxr-x 1 root root 9.8G Jan 28 15:18 ns1.img
> 9.8G -rwxrwxr-x 1 root root 9.8G Jan 28 15:18 ns2.img
> 134M -rwxrwxr-x 1 root root  98G Jan 28 15:18 shared01.img
>  20G -rwxrwxr-x 1 root root  20G Jan 28 15:18 www.img
>
So it doesn''t use 100G right away, right?
Both "du -sh" and "ls -lash" (leftmost column) shows space
actually
used. It can be different from the right column if it''s created as
sparse file (in your case, "not to allocate the entire disk yet").
> I''m just overall kind of confused how this is working.  I should
in theory
> be able to over allocate a number of vm''s, so that when I get
close to
> running out, I can just add more disk space as cash allows.
Yes, it should work like that. Note however that using sparse file has
the downside of :
- lower I/O performance initially (i.e. only when it has to allocate
disk space previously used)
- there might be a condition when you run out of space (e.g. when all
domU unexpectedly needs more space) which can lead to unpredictable
result.

Regards,

Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Jan 28, 2009, at 11:30 PM, Fajar A. Nugraha wrote:
> On Thu, Jan 29, 2009 at 12:20 PM, Charlie Reddington
> <charlie.reddington@gmail.com> wrote:
>> # df -h
>> Filesystem            Size  Used Avail Use% Mounted on
>> /dev/sda1             446G   43G  381G  10% /
>> tmpfs                 3.9G     0  3.9G   0% /dev/shm
>>
>> Right here alone, has me kind of sketcy. Total - 446G - 43G should  
>> = 403G
>
> No comment on that one :)
>
>>
>> But in var/lib/xen/images.....
>>
>> # du -sh *
>> 9.8G    ns1.img
>> 9.8G    ns2..img
>> 119M    shared01.img
>> 20G             www.img
>>
>> Doing the command as you say.....
>>
>> 9.8G -rwxrwxr-x 1 root root 9.8G Jan 28 15:18 ns1.img
>> 9.8G -rwxrwxr-x 1 root root 9.8G Jan 28 15:18 ns2.img
>> 134M -rwxrwxr-x 1 root root  98G Jan 28 15:18 shared01.img
>> 20G -rwxrwxr-x 1 root root  20G Jan 28 15:18 www.img
>>
>
> So it doesn''t use 100G right away, right?
> Both "du -sh" and "ls -lash" (leftmost column) shows
space actually
> used. It can be different from the right column if it''s created as
> sparse file (in your case, "not to allocate the entire disk
yet").
>
>> I''m just overall kind of confused how this is working.  I
should in
>> theory
>> be able to over allocate a number of vm''s, so that when I get
close
>> to
>> running out, I can just add more disk space as cash allows.
>
> Yes, it should work like that. Note however that using sparse file has
> the downside of :
> - lower I/O performance initially (i.e. only when it has to allocate
> disk space previously used)
> - there might be a condition when you run out of space (e.g. when all
> domU unexpectedly needs more space) which can lead to unpredictable
> result.
>
> Regards,
>
> Fajar
Thanks for the reply. I''m just trying to get the most out of this  
stuff as we are not. We''ll be adding a lot more storage in the future,
but are working on getting things going. I just want everything to  
have the ability to expand as it needs, and we should have everything  
in place before running out of disk space is an issue.

So, since it''s calling the file 100gb, but only use 20, will I run  
into any problems on the host system thinking it''s out of memory?  
That''s really probably my only concern. If I setup 10vms and
I''m over
by a 100gb or so, will the host os complain?

Thanks,

Charlie

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Jan 29, 2009 at 12:41 PM, Charlie Reddington
<charlie.reddington@gmail.com> wrote:
> So, since it''s calling the file 100gb, but only use 20, will I run
into any
> problems on the host system thinking it''s out of memory?
That''s really
No. And anyway, it''s not memory. It''s disk.
> probably my only concern. If I setup 10vms and I''m over by a 100gb
or so,
> will the host os complain?
>
As long as you use sparse files, and your domUs doesn''t actually use
that much space, then it''s OK.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users