Fan, Huaxiang
2012-Aug-01 05:57 UTC
PCI passthrough for domU allocated with more than 4G memory
Hi, I have encountered some strange problems when I was trying to PCI passthrough Broadcom 5709/5716 NICs to domUs allocated with more than 4G memory. Please see below for details. My environment is: Hardware Platform: DELL R210 with 2 Broadcom 5709 NICs and 2 Broadcom 5716 NICs Xen: xen 4.2 unstable (64bits for hypervisor and 32bit for tools) Kernel for both dom0 and domUs: xenified kernel 2.6.32.57 (32bit) OS: CentOS 6.2 (32bit) The general info regarding to xen can be get via below command # xl info host : 7.8 release : 2.6.32.57 version : #1 SMP Fri Jul 6 18:44:16 CST 2012 machine : i686 nr_cpus : 8 max_cpu_id : 31 nr_nodes : 1 cores_per_socket : 4 threads_per_core : 2 cpu_mhz : 2660 hw_caps : bfebfbff:28100800:00000000:00003b40:0098e3fd:00000000:00000001:00000000 virt_caps : hvm hvm_directio total_memory : 8182 free_memory : 7046 sharing_freed_memory : 0 sharing_used_memory : 0 free_cpus : 0 xen_major : 4 xen_minor : 2 xen_extra : -unstable xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_scheduler : credit xen_pagesize : 4096 platform_params : virt_start=0xff400000 xen_changeset : unavailable xen_commandline : dom0_mem=1024M dom0_max_vcpus=2 dom0_vcpus_pin cc_compiler : gcc version 4.4.6 20110731 (Red Hat 4.4.6-3) (GCC) cc_compile_by : root cc_compile_domain : cc_compile_date : Thu Jul 12 11:20:56 CST 2012 xend_config_format : 4 case 1> I specified PV-domU config as below memory=3072 maxmem=6144 name="bs" vif=[''ip=169.254.254.1,script=vif-nat'',] disk=[''file:/root/bs.img,xvda1,w''] kernel=''/root/vmlinuz'' extra="iommu=soft console=hvc0" ramdisk=''/root/initrd.img'' root="/dev/xvda1 ro" pci=[''01:00.0'',''01:00.1''] and then start domU. After that I executed below command # xl list Name ID Mem VCPUs State Time(s) Domain-0 0 1024 2 r----- 88.7 bs 2 3072 1 -b---- 1.1 It seemed normal to me. But when I logon bs domU, and executed below command # cat /proc/meminfo | head MemTotal: 6158940 kB MemFree: 2944776 kB Buffers: 5108 kB Cached: 32292 kB SwapCached: 0 kB Active: 21456 kB Inactive: 22936 kB Active(anon): 7000 kB Inactive(anon): 108 kB Active(file): 14456 kB It indicated the total memory was 6G, why? When I back to dom0, I executed below command # xl mem-set bs 6144 # xl list Name ID Mem VCPUs State Time(s) Domain-0 0 1024 2 r----- 93.5 bs 2 6144 1 -b---- 10.5 It seemed normal to me. But when I logon bs domU again and executed below command # cat /proc/meminfo | head MemTotal: 9304668 kB MemFree: 6087540 kB Buffers: 5168 kB Cached: 32464 kB SwapCached: 0 kB Active: 22300 kB Inactive: 22408 kB Active(anon): 7080 kB Inactive(anon): 108 kB Active(file): 15220 kB It indicated total memory was 9G (6G + 3G). It was wired. Any idea about this? Case 2> I specified PV-domU config as below memory=6144 maxmem=6144 name="bs" vif=[''ip=169.254.254.1,script=vif-nat'',] disk=[''file:/root/bs.img,xvda1,w''] kernel=''/root/vmlinuz'' extra="iommu=soft console=hvc0" ramdisk=''/root/initrd.img'' root="/dev/xvda1 ro" pci=[''01:00.0'',''01:00.1''] and then start domU. After that I executed below command # xl list Name ID Mem VCPUs State Time(s) Domain-0 0 648 2 r----- 120.5 bs 3 3360 1 -b---- 7.0 the output was very confusing. Why dom0 memory had been shrank to 648M and only 3360M assigned to bs domU? My own analysis: I extracted the bios e820 memory map on bs domU as below [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable) [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved) [ 0.000000] Xen: 0000000000100000 - 00000000bf699000 (usable) [ 0.000000] Xen: 00000000bf699000 - 00000000bf6af000 (reserved) [ 0.000000] Xen: 00000000bf6af000 - 00000000bf6ce000 (ACPI data) [ 0.000000] Xen: 00000000bf6ce000 - 00000000c0000000 (reserved) [ 0.000000] Xen: 00000000e0000000 - 00000000f0000000 (reserved) [ 0.000000] Xen: 00000000fe000000 - 0000000100000000 (reserved) [ 0.000000] Xen: 0000000180000000 - 00000001c33ec000 (usable) I think the root cause might be related to the holes between c0000000 and e0000000 and between f0000000 and fe000000 and between 100000000 and 180000000. And I think the e820_host option set according to my tracking. Thanks in advance HUAXIANG FAN Software Engineer II WEBSENSE NETWORK SECURITY TECHNOLOGY R&D (BEIJING) CO. LTD. ph: +8610.5884.4327 fax: +8610.5884.4727 www.websense.cn<http://www.websense.cn> Websense TRITON(tm) For Essential Information Protection(tm) Web Security<http://www.websense.com/content/Regional/SCH/WebSecurityOverview.aspx> | Data Security<http://www.websense.com/content/Regional/SCH/DataSecurity.aspx> | Email Security<http://www.websense.com/content/Regional/SCH/MessagingSecurity.aspx> Protected by Websense Hosted Email Security -- www.websense.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Fan, Huaxiang
2012-Aug-01 05:57 UTC
PCI passthrough for domU allocated with more than 4G memory
Hi, I have encountered some strange problems when I was trying to PCI passthrough Broadcom 5709/5716 NICs to domUs allocated with more than 4G memory. Please see below for details. My environment is: Hardware Platform: DELL R210 with 2 Broadcom 5709 NICs and 2 Broadcom 5716 NICs Xen: xen 4.2 unstable (64bits for hypervisor and 32bit for tools) Kernel for both dom0 and domUs: xenified kernel 2.6.32.57 (32bit) OS: CentOS 6.2 (32bit) The general info regarding to xen can be get via below command # xl info host : 7.8 release : 2.6.32.57 version : #1 SMP Fri Jul 6 18:44:16 CST 2012 machine : i686 nr_cpus : 8 max_cpu_id : 31 nr_nodes : 1 cores_per_socket : 4 threads_per_core : 2 cpu_mhz : 2660 hw_caps : bfebfbff:28100800:00000000:00003b40:0098e3fd:00000000:00000001:00000000 virt_caps : hvm hvm_directio total_memory : 8182 free_memory : 7046 sharing_freed_memory : 0 sharing_used_memory : 0 free_cpus : 0 xen_major : 4 xen_minor : 2 xen_extra : -unstable xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_scheduler : credit xen_pagesize : 4096 platform_params : virt_start=0xff400000 xen_changeset : unavailable xen_commandline : dom0_mem=1024M dom0_max_vcpus=2 dom0_vcpus_pin cc_compiler : gcc version 4.4.6 20110731 (Red Hat 4.4.6-3) (GCC) cc_compile_by : root cc_compile_domain : cc_compile_date : Thu Jul 12 11:20:56 CST 2012 xend_config_format : 4 case 1> I specified PV-domU config as below memory=3072 maxmem=6144 name="bs" vif=[''ip=169.254.254.1,script=vif-nat'',] disk=[''file:/root/bs.img,xvda1,w''] kernel=''/root/vmlinuz'' extra="iommu=soft console=hvc0" ramdisk=''/root/initrd.img'' root="/dev/xvda1 ro" pci=[''01:00.0'',''01:00.1''] and then start domU. After that I executed below command # xl list Name ID Mem VCPUs State Time(s) Domain-0 0 1024 2 r----- 88.7 bs 2 3072 1 -b---- 1.1 It seemed normal to me. But when I logon bs domU, and executed below command # cat /proc/meminfo | head MemTotal: 6158940 kB MemFree: 2944776 kB Buffers: 5108 kB Cached: 32292 kB SwapCached: 0 kB Active: 21456 kB Inactive: 22936 kB Active(anon): 7000 kB Inactive(anon): 108 kB Active(file): 14456 kB It indicated the total memory was 6G, why? When I back to dom0, I executed below command # xl mem-set bs 6144 # xl list Name ID Mem VCPUs State Time(s) Domain-0 0 1024 2 r----- 93.5 bs 2 6144 1 -b---- 10.5 It seemed normal to me. But when I logon bs domU again and executed below command # cat /proc/meminfo | head MemTotal: 9304668 kB MemFree: 6087540 kB Buffers: 5168 kB Cached: 32464 kB SwapCached: 0 kB Active: 22300 kB Inactive: 22408 kB Active(anon): 7080 kB Inactive(anon): 108 kB Active(file): 15220 kB It indicated total memory was 9G (6G + 3G). It was wired. Any idea about this? Case 2> I specified PV-domU config as below memory=6144 maxmem=6144 name="bs" vif=[''ip=169.254.254.1,script=vif-nat'',] disk=[''file:/root/bs.img,xvda1,w''] kernel=''/root/vmlinuz'' extra="iommu=soft console=hvc0" ramdisk=''/root/initrd.img'' root="/dev/xvda1 ro" pci=[''01:00.0'',''01:00.1''] and then start domU. After that I executed below command # xl list Name ID Mem VCPUs State Time(s) Domain-0 0 648 2 r----- 120.5 bs 3 3360 1 -b---- 7.0 the output was very confusing. Why dom0 memory had been shrank to 648M and only 3360M assigned to bs domU? My own analysis: I extracted the bios e820 memory map on bs domU as below [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable) [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved) [ 0.000000] Xen: 0000000000100000 - 00000000bf699000 (usable) [ 0.000000] Xen: 00000000bf699000 - 00000000bf6af000 (reserved) [ 0.000000] Xen: 00000000bf6af000 - 00000000bf6ce000 (ACPI data) [ 0.000000] Xen: 00000000bf6ce000 - 00000000c0000000 (reserved) [ 0.000000] Xen: 00000000e0000000 - 00000000f0000000 (reserved) [ 0.000000] Xen: 00000000fe000000 - 0000000100000000 (reserved) [ 0.000000] Xen: 0000000180000000 - 00000001c33ec000 (usable) I think the root cause might be related to the holes between c0000000 and e0000000 and between f0000000 and fe000000 and between 100000000 and 180000000. And I think the e820_host option set according to my tracking. Thanks in advance HUAXIANG FAN Software Engineer II WEBSENSE NETWORK SECURITY TECHNOLOGY R&D (BEIJING) CO. LTD. ph: +8610.5884.4327 fax: +8610.5884.4727 www.websense.cn<http://www.websense.cn> Websense TRITON(tm) For Essential Information Protection(tm) Web Security<http://www.websense.com/content/Regional/SCH/WebSecurityOverview.aspx> | Data Security<http://www.websense.com/content/Regional/SCH/DataSecurity.aspx> | Email Security<http://www.websense.com/content/Regional/SCH/MessagingSecurity.aspx> Protected by Websense Hosted Email Security -- www.websense.com _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Konrad Rzeszutek Wilk
2012-Aug-04 11:12 UTC
Re: [Xen-devel] PCI passthrough for domU allocated with more than 4G memory
On Wed, Aug 01, 2012 at 05:57:13AM +0000, Fan, Huaxiang wrote:> Hi, > > I have encountered some strange problems when I was trying to PCI passthrough Broadcom 5709/5716 NICs to domUs allocated with more than 4G memory. Please see below for details. > > My environment is: > Hardware Platform: DELL R210 with 2 Broadcom 5709 NICs and 2 Broadcom 5716 NICs > Xen: xen 4.2 unstable (64bits for hypervisor and 32bit for tools) > Kernel for both dom0 and domUs: xenified kernel 2.6.32.57 (32bit) > OS: CentOS 6.2 (32bit) > > The general info regarding to xen can be get via below command > > # xl info > > host : 7.8 > > release : 2.6.32.57 > > version : #1 SMP Fri Jul 6 18:44:16 CST 2012 > > machine : i686 > > nr_cpus : 8 > > max_cpu_id : 31 > > nr_nodes : 1 > > cores_per_socket : 4 > > threads_per_core : 2 > > cpu_mhz : 2660 > > hw_caps : bfebfbff:28100800:00000000:00003b40:0098e3fd:00000000:00000001:00000000 > > virt_caps : hvm hvm_directio > > total_memory : 8182 > > free_memory : 7046 > > sharing_freed_memory : 0 > > sharing_used_memory : 0 > > free_cpus : 0 > > xen_major : 4 > > xen_minor : 2 > > xen_extra : -unstable > > xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 > > xen_scheduler : credit > > xen_pagesize : 4096 > > platform_params : virt_start=0xff400000 > > xen_changeset : unavailable > > xen_commandline : dom0_mem=1024M dom0_max_vcpus=2 dom0_vcpus_pin > > cc_compiler : gcc version 4.4.6 20110731 (Red Hat 4.4.6-3) (GCC) > > cc_compile_by : root > > cc_compile_domain : > > cc_compile_date : Thu Jul 12 11:20:56 CST 2012 > > xend_config_format : 4 > > case 1> I specified PV-domU config as below > > memory=3072 > > maxmem=6144 > > name="bs" > > vif=[''ip=169.254.254.1,script=vif-nat'',] > > disk=[''file:/root/bs.img,xvda1,w''] > > kernel=''/root/vmlinuz'' > > extra="iommu=soft console=hvc0" > > ramdisk=''/root/initrd.img'' > > root="/dev/xvda1 ro" > > pci=[''01:00.0'',''01:00.1''] > > and then start domU. After that I executed below command > > # xl list > > Name ID Mem VCPUs State Time(s) > > Domain-0 0 1024 2 r----- 88.7 > > bs 2 3072 1 -b---- 1.1 > > It seemed normal to me. But when I logon bs domU, and executed below command > > # cat /proc/meminfo | head > > MemTotal: 6158940 kB > > MemFree: 2944776 kBSo that is the right amount. It has around 3GB of free kernel space.> > Buffers: 5108 kB > > Cached: 32292 kB > > SwapCached: 0 kB > > Active: 21456 kB > > Inactive: 22936 kB > > Active(anon): 7000 kB > > Inactive(anon): 108 kB > > Active(file): 14456 kB > > It indicated the total memory was 6G, why?I wish you also included the DirectMap number. Irregardless of what /proc/meminfo says, what did your dmesg say in ''Memory'' section?> When I back to dom0, I executed below command > > # xl mem-set bs 6144 > > # xl list > > Name ID Mem VCPUs State Time(s) > > Domain-0 0 1024 2 r----- 93.5 > > bs 2 6144 1 -b---- 10.5 > It seemed normal to me. But when I logon bs domU again and executed below command > > # cat /proc/meminfo | head > > MemTotal: 9304668 kB > > MemFree: 6087540 kBSo that is right. 6GB of free space.> > Buffers: 5168 kB > > Cached: 32464 kB > > SwapCached: 0 kB > > Active: 22300 kB > > Inactive: 22408 kB > > Active(anon): 7080 kB > > Inactive(anon): 108 kB > > Active(file): 15220 kB > > > > It indicated total memory was 9G (6G + 3G). It was wired. Any idea about this?Presumarily b/c ''3G'' of it is the E820_UNUSUABLE or the big gap in the E820. But irregardless of that - do you have 6GB of ram in your guest?> Case 2> I specified PV-domU config as below > > memory=6144 > > maxmem=6144 > > name="bs" > > vif=[''ip=169.254.254.1,script=vif-nat'',] > > disk=[''file:/root/bs.img,xvda1,w''] > > kernel=''/root/vmlinuz'' > > extra="iommu=soft console=hvc0" > > ramdisk=''/root/initrd.img'' > > root="/dev/xvda1 ro" > > pci=[''01:00.0'',''01:00.1''] > > and then start domU. After that I executed below command > > # xl list > > Name ID Mem VCPUs State Time(s) > > Domain-0 0 648 2 r----- 120.5 > > bs 3 3360 1 -b---- 7.0 > > the output was very confusing. Why dom0 memory had been shrank to 648M and only 3360M assigned to bs domU?I think you are seeing a bug in the xl with the autoballoon. Smoebody mentioned this on the xen-devel.> > My own analysis: > I extracted the bios e820 memory map on bs domU as below > > [ 0.000000] BIOS-provided physical RAM map: > > [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable) > > [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved) > > [ 0.000000] Xen: 0000000000100000 - 00000000bf699000 (usable) > > [ 0.000000] Xen: 00000000bf699000 - 00000000bf6af000 (reserved) > > [ 0.000000] Xen: 00000000bf6af000 - 00000000bf6ce000 (ACPI data) > > [ 0.000000] Xen: 00000000bf6ce000 - 00000000c0000000 (reserved) > > [ 0.000000] Xen: 00000000e0000000 - 00000000f0000000 (reserved) > > [ 0.000000] Xen: 00000000fe000000 - 0000000100000000 (reserved) > > [ 0.000000] Xen: 0000000180000000 - 00000001c33ec000 (usable) > > I think the root cause might be related to the holes between c0000000 and e0000000 and between f0000000 and fe000000 and between 100000000 and 180000000. And I think the e820_host option set according to my tracking.Not sure what you mean by ''my tracking''. The holes are there to provide the PCI subsystem the space to stick the MMIO BARs of your PCI device. The memory that was "taken" out of those areas is then appended to the next E820_RAM region. It should have gone to 0x1000000, not to 0x18000000 though. Ohh, you are using the ancient 2.6.32 domU! Well, that is ancient - and it probably has bugs. Have you considered using something more recent?> > Thanks in advance > HUAXIANG FAN > Software Engineer II > > WEBSENSE NETWORK SECURITY TECHNOLOGY R&D (BEIJING) CO. LTD. > ph: +8610.5884.4327 > fax: +8610.5884.4727 > www.websense.cn<http://www.websense.cn> > > Websense TRITON(tm) > For Essential Information Protection(tm) > Web Security<http://www.websense.com/content/Regional/SCH/WebSecurityOverview.aspx> | Data Security<http://www.websense.com/content/Regional/SCH/DataSecurity.aspx> | Email Security<http://www.websense.com/content/Regional/SCH/MessagingSecurity.aspx> > > > > Protected by Websense Hosted Email Security -- www.websense.com> _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
Konrad Rzeszutek Wilk
2012-Aug-04 11:12 UTC
Re: PCI passthrough for domU allocated with more than 4G memory
On Wed, Aug 01, 2012 at 05:57:13AM +0000, Fan, Huaxiang wrote:> Hi, > > I have encountered some strange problems when I was trying to PCI passthrough Broadcom 5709/5716 NICs to domUs allocated with more than 4G memory. Please see below for details. > > My environment is: > Hardware Platform: DELL R210 with 2 Broadcom 5709 NICs and 2 Broadcom 5716 NICs > Xen: xen 4.2 unstable (64bits for hypervisor and 32bit for tools) > Kernel for both dom0 and domUs: xenified kernel 2.6.32.57 (32bit) > OS: CentOS 6.2 (32bit) > > The general info regarding to xen can be get via below command > > # xl info > > host : 7.8 > > release : 2.6.32.57 > > version : #1 SMP Fri Jul 6 18:44:16 CST 2012 > > machine : i686 > > nr_cpus : 8 > > max_cpu_id : 31 > > nr_nodes : 1 > > cores_per_socket : 4 > > threads_per_core : 2 > > cpu_mhz : 2660 > > hw_caps : bfebfbff:28100800:00000000:00003b40:0098e3fd:00000000:00000001:00000000 > > virt_caps : hvm hvm_directio > > total_memory : 8182 > > free_memory : 7046 > > sharing_freed_memory : 0 > > sharing_used_memory : 0 > > free_cpus : 0 > > xen_major : 4 > > xen_minor : 2 > > xen_extra : -unstable > > xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 > > xen_scheduler : credit > > xen_pagesize : 4096 > > platform_params : virt_start=0xff400000 > > xen_changeset : unavailable > > xen_commandline : dom0_mem=1024M dom0_max_vcpus=2 dom0_vcpus_pin > > cc_compiler : gcc version 4.4.6 20110731 (Red Hat 4.4.6-3) (GCC) > > cc_compile_by : root > > cc_compile_domain : > > cc_compile_date : Thu Jul 12 11:20:56 CST 2012 > > xend_config_format : 4 > > case 1> I specified PV-domU config as below > > memory=3072 > > maxmem=6144 > > name="bs" > > vif=[''ip=169.254.254.1,script=vif-nat'',] > > disk=[''file:/root/bs.img,xvda1,w''] > > kernel=''/root/vmlinuz'' > > extra="iommu=soft console=hvc0" > > ramdisk=''/root/initrd.img'' > > root="/dev/xvda1 ro" > > pci=[''01:00.0'',''01:00.1''] > > and then start domU. After that I executed below command > > # xl list > > Name ID Mem VCPUs State Time(s) > > Domain-0 0 1024 2 r----- 88.7 > > bs 2 3072 1 -b---- 1.1 > > It seemed normal to me. But when I logon bs domU, and executed below command > > # cat /proc/meminfo | head > > MemTotal: 6158940 kB > > MemFree: 2944776 kBSo that is the right amount. It has around 3GB of free kernel space.> > Buffers: 5108 kB > > Cached: 32292 kB > > SwapCached: 0 kB > > Active: 21456 kB > > Inactive: 22936 kB > > Active(anon): 7000 kB > > Inactive(anon): 108 kB > > Active(file): 14456 kB > > It indicated the total memory was 6G, why?I wish you also included the DirectMap number. Irregardless of what /proc/meminfo says, what did your dmesg say in ''Memory'' section?> When I back to dom0, I executed below command > > # xl mem-set bs 6144 > > # xl list > > Name ID Mem VCPUs State Time(s) > > Domain-0 0 1024 2 r----- 93.5 > > bs 2 6144 1 -b---- 10.5 > It seemed normal to me. But when I logon bs domU again and executed below command > > # cat /proc/meminfo | head > > MemTotal: 9304668 kB > > MemFree: 6087540 kBSo that is right. 6GB of free space.> > Buffers: 5168 kB > > Cached: 32464 kB > > SwapCached: 0 kB > > Active: 22300 kB > > Inactive: 22408 kB > > Active(anon): 7080 kB > > Inactive(anon): 108 kB > > Active(file): 15220 kB > > > > It indicated total memory was 9G (6G + 3G). It was wired. Any idea about this?Presumarily b/c ''3G'' of it is the E820_UNUSUABLE or the big gap in the E820. But irregardless of that - do you have 6GB of ram in your guest?> Case 2> I specified PV-domU config as below > > memory=6144 > > maxmem=6144 > > name="bs" > > vif=[''ip=169.254.254.1,script=vif-nat'',] > > disk=[''file:/root/bs.img,xvda1,w''] > > kernel=''/root/vmlinuz'' > > extra="iommu=soft console=hvc0" > > ramdisk=''/root/initrd.img'' > > root="/dev/xvda1 ro" > > pci=[''01:00.0'',''01:00.1''] > > and then start domU. After that I executed below command > > # xl list > > Name ID Mem VCPUs State Time(s) > > Domain-0 0 648 2 r----- 120.5 > > bs 3 3360 1 -b---- 7.0 > > the output was very confusing. Why dom0 memory had been shrank to 648M and only 3360M assigned to bs domU?I think you are seeing a bug in the xl with the autoballoon. Smoebody mentioned this on the xen-devel.> > My own analysis: > I extracted the bios e820 memory map on bs domU as below > > [ 0.000000] BIOS-provided physical RAM map: > > [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable) > > [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved) > > [ 0.000000] Xen: 0000000000100000 - 00000000bf699000 (usable) > > [ 0.000000] Xen: 00000000bf699000 - 00000000bf6af000 (reserved) > > [ 0.000000] Xen: 00000000bf6af000 - 00000000bf6ce000 (ACPI data) > > [ 0.000000] Xen: 00000000bf6ce000 - 00000000c0000000 (reserved) > > [ 0.000000] Xen: 00000000e0000000 - 00000000f0000000 (reserved) > > [ 0.000000] Xen: 00000000fe000000 - 0000000100000000 (reserved) > > [ 0.000000] Xen: 0000000180000000 - 00000001c33ec000 (usable) > > I think the root cause might be related to the holes between c0000000 and e0000000 and between f0000000 and fe000000 and between 100000000 and 180000000. And I think the e820_host option set according to my tracking.Not sure what you mean by ''my tracking''. The holes are there to provide the PCI subsystem the space to stick the MMIO BARs of your PCI device. The memory that was "taken" out of those areas is then appended to the next E820_RAM region. It should have gone to 0x1000000, not to 0x18000000 though. Ohh, you are using the ancient 2.6.32 domU! Well, that is ancient - and it probably has bugs. Have you considered using something more recent?> > Thanks in advance > HUAXIANG FAN > Software Engineer II > > WEBSENSE NETWORK SECURITY TECHNOLOGY R&D (BEIJING) CO. LTD. > ph: +8610.5884.4327 > fax: +8610.5884.4727 > www.websense.cn<http://www.websense.cn> > > Websense TRITON(tm) > For Essential Information Protection(tm) > Web Security<http://www.websense.com/content/Regional/SCH/WebSecurityOverview.aspx> | Data Security<http://www.websense.com/content/Regional/SCH/DataSecurity.aspx> | Email Security<http://www.websense.com/content/Regional/SCH/MessagingSecurity.aspx> > > > > Protected by Websense Hosted Email Security -- www.websense.com> _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel