Xen devel - Sep 2009 - Balloon down memory failed in x86 PAE mode when memory size>4G

Hi

I implemented balloon in windows pv driver, it works normally on x86 and 
x64 OS.
However, i hit problem to balloon down memory on x86 PAE mode VM when 
it''s memory
size is larger than 4G.

Furthermore, i tested linux PVHVM too. Balloon driver also failed to 
take effect
at the same situation.

So, is there anything missed in windows or linux PV driver when 
ballooning down
memory on x86 PAE VM(memory>4G)? or is there a limitation in Xen for PAE 
mode?

Thanks
Annie.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

We''d expect it to work. Please provide more details.

 -- Keir

On 21/09/2009 07:42, "ANNIE LI" <annie.li@oracle.com> wrote:
> Hi
> 
> I implemented balloon in windows pv driver, it works normally on x86 and
> x64 OS.
> However, i hit problem to balloon down memory on x86 PAE mode VM when
> it''s memory
> size is larger than 4G.
> 
> Furthermore, i tested linux PVHVM too. Balloon driver also failed to
> take effect
> at the same situation.
> 
> So, is there anything missed in windows or linux PV driver when
> ballooning down
> memory on x86 PAE VM(memory>4G)? or is there a limitation in Xen for PAE
> mode?
> 
> Thanks
> Annie.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

My xen version is Xen3.4.0, and i use el5u2 pvhvm to have test.
The memory size is 6114.  "xm info" dose not show any change on 
free_memory after command "xm mem-set domid 4096". It seems
free_memory
will change after i set memory size less than 3083M,  such as: free 
memory will increase 1 after command "xm mem-set domid 3082".

The vm configure file is following:

name = "OVM_EL5U2_X86_PVHVM_4GB"
builder = "hvm"
memory = "6114"
disk = [ 
''file:/mnt/running_pool/OVM_EL5U2_X86_PVHVM_4GB/System.img,hda,w'',
]
vif = [ ''type=netfront'', ]
device_model = "/usr/lib/xen/bin/qemu-dm"
kernel = "/usr/lib/xen/boot/hvmloader"
vnc=1
vncunused=1
vnclisten = ''0.0.0.0''
timer_mode=0
apic=1
acpi=1
pae=1
vcpus=1
serial = "pty" # enable serial console
on_reboot   = ''restart''
on_crash    = ''restart''

Thanks
Annie.

Keir Fraser wrote:
> We''d expect it to work. Please provide more details.
>
>  -- Keir
>
> On 21/09/2009 07:42, "ANNIE LI" <annie.li@oracle.com>
wrote:
>
>   
>> Hi
>>
>> I implemented balloon in windows pv driver, it works normally on x86
and
>> x64 OS.
>> However, i hit problem to balloon down memory on x86 PAE mode VM when
>> it''s memory
>> size is larger than 4G.
>>
>> Furthermore, i tested linux PVHVM too. Balloon driver also failed to
>> take effect
>> at the same situation.
>>
>> So, is there anything missed in windows or linux PV driver when
>> ballooning down
>> memory on x86 PAE VM(memory>4G)? or is there a limitation in Xen for
PAE
>> mode?
>>
>> Thanks
>> Annie.
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>     
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>   

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

ANNIE LI wrote:
>>> I implemented balloon in windows pv driver, it works normally on
x86 and
>>> x64 OS.
>>> However, i hit problem to balloon down memory on x86 PAE mode VM
when
>>> it''s memory
>>> size is larger than 4G.
>>>
Which version of windows are you booting? It may not be using PAE; check 
your boot flags.

   Paul

-- 
==============================Paul Durrant, Software Engineer

Citrix Systems (R&D) Ltd.
First Floor, Building 101
Cambridge Science Park
Milton Road
Cambridge CB4 0FY
United Kingdom

TEL: x35957 (+44 1223 225957)
==============================
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Paul Durrant wrote:
> ANNIE LI wrote:
>>>> I implemented balloon in windows pv driver, it works normally
on
>>>> x86 and
>>>> x64 OS.
>>>> However, i hit problem to balloon down memory on x86 PAE mode
VM when
>>>> it''s memory
>>>> size is larger than 4G.
>>>>
>
> Which version of windows are you booting? It may not be using PAE; 
> check your boot flags.
I am using Srv2008 X86. The memory size in this VM shows as 5.97GB. And 
I thought PAE mode was
enabled in this OS if the memory size is larger than 4G and pae was set 
1 in vm configure file. right?

Following is my Srv2008 X86 configure file:

acpi = 1
apic = 1
boot = ''c''
builder = ''hvm''
bootloader = ''/usr/bin/pygrub''
device_model = ''/usr/lib/xen/bin/qemu-dm''
disk = [
''file:/mnt/running_pool/win2k8/System.img,hda,w'',
]
kernel = ''/usr/lib/xen/boot/hvmloader''
memory = ''6114''
name = ''win2k8_32''
on_crash = ''''
on_reboot = ''''
pae = 1
#serial = ''pty''
timer_mode = 1
usbdevice = ''tablet''
vcpus = 1
vif = [
''mac=00:16:3E:5E:DF:29, bridge=xenbr0'',
]
vnc = 1
vnclisten = ''0.0.0.0''
vncunused = 1

Thanks
Annie.
>
>   Paul
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

ANNIE LI wrote:
> I am using Srv2008 X86. The memory size in this VM shows as 5.97GB. And 
> I thought PAE mode was
> enabled in this OS if the memory size is larger than 4G and pae was set 
> 1 in vm configure file. right?
> 
Yes, PAE should be on in 2k8 by default and if Windows tells you it''s 
got 5.97G of RAM then everything should be ok.

   Paul

-- 
==============================Paul Durrant, Software Engineer

Citrix Systems (R&D) Ltd.
First Floor, Building 101
Cambridge Science Park
Milton Road
Cambridge CB4 0FY
United Kingdom

TEL: x35957 (+44 1223 225957)
==============================
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 21/09/2009 10:15, "ANNIE LI" <annie.li@oracle.com> wrote:
> My xen version is Xen3.4.0, and i use el5u2 pvhvm to have test.
> The memory size is 6114.  "xm info" dose not show any change on
free_memory
> after command "xm mem-set domid 4096". It seems free_memory will
change after
> i set memory size less than 3083M,  such as: free memory will increase 1
after
> command "xm mem-set domid 3082".
And this is the exact same behaviour you see in a PAE Linux HVM guest with
PV drivers also? It''s weird, seems like maybe for some reason the
balloon
driver thinks the guest has 3083M and therefore doesn''t balloon down
until
it sees a target lower than that, and even then only by the difference
between 3083M and the target.

I suggest you add tracing to your PV drivers and see what the balloon driver
thinks is going on. This configuration really should work, so hopefully only
something simple and stupid is going wroing.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>> My xen version is Xen3.4.0, and i use el5u2 pvhvm to have test.
>> The memory size is 6114.  "xm info" dose not show any change
on free_memory
>> after command "xm mem-set domid 4096". It seems free_memory
will change after
>> i set memory size less than 3083M,  such as: free memory will increase
1 after
>> command "xm mem-set domid 3082".
>>     
>
> And this is the exact same behaviour you see in a PAE Linux HVM guest with
> PV drivers also? It''s weird, seems like maybe for some reason the
balloon
> driver thinks the guest has 3083M and therefore doesn''t balloon
down until
> it sees a target lower than that, and even then only by the difference
> between 3083M and the target.
>
> I suggest you add tracing to your PV drivers and see what the balloon
driver
> thinks is going on. This configuration really should work, so hopefully
only
> something simple and stupid is going wroing.
Sorry the false alarm.
The linux template el5u2 i use can not manage memory larger than 4G even 
if i set the
6G memory and PAE mode in vm configure file, because CONFIG_HIGHMEM4G is 
set
when compiling the OS kernel.

I did more test on el4u8 with CONFIG_HIGHMEM64G enabled, balloon works 
correctly.

I will check the windows pv driver again.

Thanks
Annie.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Yes. My winpv balloon driver works properly on x86 PAE mode when memory 
size > 4G now.

Thanks
Annie.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel