xen discuss - May 2008 - SNV82: Not enough memory is available, and dom0 cannot be shrunk any further

Hi All,

I am running nevada 79 BFU''ed to 82.  The machine is a Ultra 20 with
4GB
memory.  I have several Windows XP domU''s configured and registered.  
When ever I try to start the fourth domain I get an out of memory exception:

Not enough memory is available, and dom0 cannot be shrunk any further

Each of my domains only uses 256 so I thought there would be sufficient 
memory for 4 :-/

I am also using ZFS (maybe this is causing problems?) and have tried 
using the kernel flag:

dom0_mem=2048M

Any ideas?

Thanks,
Christian

Christian Smith wrote:
> Hi All,
> 
> I am running nevada 79 BFU''ed to 82.  The machine is a Ultra 20
with 4GB
> memory.  I have several Windows XP domU''s configured and
registered.
> When ever I try to start the fourth domain I get an out of memory
exception:
> 
> Not enough memory is available, and dom0 cannot be shrunk any further
> 
> Each of my domains only uses 256 so I thought there would be sufficient 
> memory for 4 :-/
> 
> I am also using ZFS (maybe this is causing problems?) and have tried 
> using the kernel flag:
> 
> dom0_mem=2048M
> 
> Any ideas?

When using zfs on dom0, you should do the following...



Normally, if you have zfs, you should set dom0_mem
to >= 2G.
   kernel$ /boot/$ISADIR/xen.gz com1=9600,8n1 console=com1 dom0_mem=2g


You should also limit the amount of memory can give away.
if you set dom0_mem to 2g. You should set the minimal
amount of memory for dom0 to 2000.

   svccfg -s xvm/xend setprop config/dom0-min-mem=2000
   svcadm refresh xvm/xend;svcadm restart xvm/xend


Limit the size of the arc.

   echo "set zfs:zfs_arc_max = 0x10000000" >> /etc/system


If your using a disk file (vs a zvol) on a zfs filesystem,
you should set the recordsize for that fs to 8k.  zvols
already default to 8k.

   zfs set recordsize=8k rpool/guests




MRJ

That did the trick, thanks Mark :-)

-Christian

Mark Johnson wrote:
> Christian Smith wrote:
>   
>> Hi All,
>>
>> I am running nevada 79 BFU''ed to 82.  The machine is a Ultra
20 with 4GB
>> memory.  I have several Windows XP domU''s configured and
registered.
>> When ever I try to start the fourth domain I get an out of memory
exception:
>>
>> Not enough memory is available, and dom0 cannot be shrunk any further
>>
>> Each of my domains only uses 256 so I thought there would be sufficient
>> memory for 4 :-/
>>
>> I am also using ZFS (maybe this is causing problems?) and have tried 
>> using the kernel flag:
>>
>> dom0_mem=2048M
>>
>> Any ideas?
>>     
>
>
> When using zfs on dom0, you should do the following...
>
>
>
> Normally, if you have zfs, you should set dom0_mem
> to >= 2G.
>    kernel$ /boot/$ISADIR/xen.gz com1=9600,8n1 console=com1 dom0_mem=2g
>
>
> You should also limit the amount of memory can give away.
> if you set dom0_mem to 2g. You should set the minimal
> amount of memory for dom0 to 2000.
>
>    svccfg -s xvm/xend setprop config/dom0-min-mem=2000
>    svcadm refresh xvm/xend;svcadm restart xvm/xend
>
>
> Limit the size of the arc.
>
>    echo "set zfs:zfs_arc_max = 0x10000000" >> /etc/system
>
>
> If your using a disk file (vs a zvol) on a zfs filesystem,
> you should set the recordsize for that fs to 8k.  zvols
> already default to 8k.
>
>    zfs set recordsize=8k rpool/guests
>
>
>
>
> MRJ
>
> _______________________________________________
> xen-discuss mailing list
> xen-discuss@opensolaris.org
>