Guanglin Xu
2013-Aug-09 22:18 UTC
Feature Request : dummy paddings for mapping memory holes in privcmd
Hi All, As a developer of high efficient VMI (Virtual Machine Introspection) applications, I appreciate hypervisor to map the whole guest memory for us. While Xen has been providing mmap_foreign_range() API by libxc and privcmd for a long time, it denies to map if the requested page range includes memory holes, such as page 160 - 191 legacy video buffer, VGA reserved memory, PCI hole, MMIO hole, etc. (see http://techfiles.de/dmelanchthon/files/memory_hole.pdf ). Maybe you would ask, "what the hell do you want dummy memory holes?!" It is because VMI applications are almost about memory searching where both the probing of derived separate memory regions and processing on them can slow it down a lot. In fact, we have recently added such feature for KVM hypervisor. However, we are not familiar with Xen. So we hope Xen community can help. Thanks, Guanglin
Guanglin Xu
2013-Aug-10 12:02 UTC
Feature Request : dummy paddings for mapping memory holes in privcmd
Hi Andres, David, Konrad and all, A couple weeks ago, I posted a thread named "Bug: privcmd failure in mmaping 160+ pages", but finally I found that it is not the fact. It''s just a feature of Xen - denying to map memory holes. However, as a developer of high efficient VMI (Virtual Machine Introspection) applications, I appreciate hypervisor to map the whole guest memory for us. While Xen has been providing mmap_foreign_range() API by libxc and privcmd for a long time, it denies to map if the requested page range includes memory holes, such as page 160 - 191 legacy video buffer, VGA reserved memory, PCI hole, MMIO hole, etc. (see http://techfiles.de/dmelanchthon/files/memory_hole.pdf ). Maybe you would ask, "what the hell do you want dummy memory holes?!" It is because VMI applications are almost about memory searching where both the probing of derived separate memory regions and processing on them can slow it down a lot. In fact, we have recently added such feature for KVM hypervisor. However, we are not familiar with Xen. So we hope Xen community can help. Thanks, Guanglin
Jan Beulich
2013-Aug-12 08:34 UTC
Re: Feature Request : dummy paddings for mapping memory holes in privcmd
>>> On 10.08.13 at 00:18, Guanglin Xu <mzguanglin@gmail.com> wrote: > Hi All, > > As a developer of high efficient VMI (Virtual Machine Introspection) > applications, I appreciate hypervisor to map the whole guest memory > for us. > > While Xen has been providing mmap_foreign_range() API by libxc and > privcmd for a long time, it denies to map if the requested page range > includes memory holes, such as page 160 - 191 legacy video buffer, VGA > reserved memory, PCI hole, MMIO hole, etc. (see > http://techfiles.de/dmelanchthon/files/memory_hole.pdf ). > > Maybe you would ask, "what the hell do you want dummy memory holes?!" > It is because VMI applications are almost about memory searching where > both the probing of derived separate memory regions and processing on > them can slow it down a lot. > > In fact, we have recently added such feature for KVM hypervisor. > However, we are not familiar with Xen. So we hope Xen community can > help.So what would you suggest to be mapped in place of those holes? Right now I think administrative tools are expected to learn about the guest''s physical memory first, and then only attempt to map the RAM portions of it. Jan
Andres Lagar-Cavilla
2013-Aug-12 14:55 UTC
Re: Feature Request : dummy paddings for mapping memory holes in privcmd
On Aug 9, 2013, at 6:18 PM, Guanglin Xu <mzguanglin@gmail.com> wrote:> Hi All, > > As a developer of high efficient VMI (Virtual Machine Introspection) > applications, I appreciate hypervisor to map the whole guest memory > for us. > > While Xen has been providing mmap_foreign_range() API by libxc and > privcmd for a long time, it denies to map if the requested page range > includes memory holes, such as page 160 - 191 legacy video buffer, VGA > reserved memory, PCI hole, MMIO hole, etc. (see > http://techfiles.de/dmelanchthon/files/memory_hole.pdf ). > > Maybe you would ask, "what the hell do you want dummy memory holes?!" > It is because VMI applications are almost about memory searching where > both the probing of derived separate memory regions and processing on > them can slow it down a lot.Sounds like a non-sequitur… "memory searching?". First of all, you application can easily deal with memory holes with some internal remapping data structure or bitmap. Second, if you don''t want to do that, in order to persistently map the whole memory in a contiguous range with non-holes, you should look at mremap. It *should* work with privcmd mappings, but your mileage may vary. Third, by keeping a persistent map to all pages, you effectively disable memory paging, sharing, and ballooning, so keep that in mind. Andres> > In fact, we have recently added such feature for KVM hypervisor.In KVM the "RAM" is a flat ram block upon which memory holes ROMs and BIOSes are overlaid, coming from separate ram blocks. So maybe you''re looking at zeros when you should be looking at something else. Hard to tell. Andres> However, we are not familiar with Xen. So we hope Xen community can > help. > > Thanks, > > Guanglin