Here are some changes for xenpaging in the 4.0 branch. Its just for review and comments. All changes are also needed for xen-unstable, I will prepare a patch queue also for this branch. There are still issues with xenpaging. My testsetup is a SLES11 SP1 guest with runlevel 6 as default runlevel to trigger automatic reboot. After a few iterations, there will be MMIO emulation errors. This happens with a startup delay of 15.0 seconds. Another issue: If the startup delay is 0.0, and over_allocated in hvmloader mem_alloc is forced to be active, the ASSERT in p2m_remove_page() triggers. This is because the passed mfn and the mfn returned by ->get_entry() do not match. So far I have no idea where the appearently missing set_gpfn_from_mfn() should go to. Normally over_allocated is not set because the XENMEM_populate_physmap call does not fail. Olaf _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 01/18] xenpaging: Fix-up xenpaging tool code.
This isn''t directly related to EPT checking, but does some general
fix-ups
to the xenpaging code (adds some extra frees, etc.)
Already-Signed-off-by: Patrick Colp <pjcolp@cs.ubc.ca>
Signed-off-by: Olaf Hering <olaf@aepfle.de>
---
tools/xenpaging/xenpaging.c | 22 ++++++++++++++++++----
1 file changed, 18 insertions(+), 4 deletions(-)
--- xen-4.0.1-testing.orig/tools/xenpaging/xenpaging.c
+++ xen-4.0.1-testing/tools/xenpaging/xenpaging.c
@@ -100,7 +100,7 @@ xenpaging_t *xenpaging_init(domid_t doma
paging->mem_event.ring_page = init_page();
if ( paging->mem_event.ring_page == NULL )
{
- ERROR("Error initialising shared page");
+ ERROR("Error initialising ring page");
goto err;
}
@@ -198,13 +198,27 @@ xenpaging_t *xenpaging_init(domid_t doma
return paging;
err:
- if ( paging->bitmap )
+ if ( paging )
+ {
+ if ( paging->mem_event.shared_page )
+ {
+ munlock(paging->mem_event.shared_page, PAGE_SIZE);
+ free(paging->mem_event.shared_page);
+ }
+
+ if ( paging->mem_event.ring_page )
+ {
+ munlock(paging->mem_event.ring_page, PAGE_SIZE);
+ free(paging->mem_event.ring_page);
+ }
+
free(paging->bitmap);
- if ( paging->platform_info )
free(paging->platform_info);
- if ( paging )
+ free(paging->domain_info);
free(paging);
+ }
+ err_iface:
return NULL;
}
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 02/18] xenpaging: call pageout policy function in xenpaging_evict_page
Notify policy about a page that was just paged out to disk.
Up to now the code called the opposite function, which clears the
(xenpaging internal) reference bit, instead of setting it and marking
the page as gone.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Already-Acked-by: Patrick Colp <pjcolp@cs.ubc.ca>
---
tools/xenpaging/xenpaging.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--- xen-4.0.1-testing.orig/tools/xenpaging/xenpaging.c
+++ xen-4.0.1-testing/tools/xenpaging/xenpaging.c
@@ -358,8 +358,8 @@ int xenpaging_evict_page(xenpaging_t *pa
goto out;
}
- /* Notify policy of page being paged in */
- policy_notify_paged_in(paging->mem_event.domain_id, victim->gfn);
+ /* Notify policy of page being paged out */
+ policy_notify_paged_out(paging->mem_event.domain_id, victim->gfn);
out:
return ret;
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 03/18] xenpaging: fix fd leak in xenstore
Missing from commit ''libxl: Backported stuff from unstable'' Without this change, xs_daemon_open/xs_daemon_close will leak filedescriptors. Signed-off-by: Olaf Hering <olaf@aepfle.de> --- tools/xenstore/xs.c | 2 ++ 1 file changed, 2 insertions(+) --- xen-4.0.1-testing.orig/tools/xenstore/xs.c +++ xen-4.0.1-testing/tools/xenstore/xs.c @@ -285,6 +285,8 @@ void xs_daemon_close(struct xs_handle *h mutex_unlock(&h->request_mutex); mutex_unlock(&h->reply_mutex); mutex_unlock(&h->watch_mutex); + + close_fds_free(h); } static bool read_all(int fd, void *data, unsigned int len) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 04/18] xenpaging: break endless loop during inital page-out with large pagefiles
To allow the starting for xenpaging right after ''xm start
XYZ'', I
specified a pagefile size equal to the guest memory size in the hope to
catch more errors where the paged-out state of a p2mt is not checked.
While doing that, xenpaging got into an endless loop because some pages
cant be paged out right away. Now the policy reports an error if the gfn
number wraps.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Already-Acked-by: Patrick Colp <pjcolp@cs.ubc.ca>
Already-Acked-by: Keir Fraser <keir.fraser@citrix.com>
---
tools/xenpaging/policy_default.c | 35 ++++++++++++++++++++++++++++-------
tools/xenpaging/xenpaging.c | 7 +++++--
2 files changed, 33 insertions(+), 9 deletions(-)
--- xen-4.0.1-testing.orig/tools/xenpaging/policy_default.c
+++ xen-4.0.1-testing/tools/xenpaging/policy_default.c
@@ -30,8 +30,12 @@
static unsigned long mru[MRU_SIZE];
-static unsigned int i_mru = 0;
+static unsigned int i_mru;
static unsigned long *bitmap;
+static unsigned long *unconsumed;
+static unsigned long current_gfn;
+static unsigned long bitmap_size;
+static unsigned long max_pages;
int policy_init(xenpaging_t *paging)
@@ -43,6 +47,14 @@ int policy_init(xenpaging_t *paging)
rc = alloc_bitmap(&bitmap, paging->bitmap_size);
if ( rc != 0 )
goto out;
+ /* Allocate bitmap to track unusable pages */
+ rc = alloc_bitmap(&unconsumed, paging->bitmap_size);
+ if ( rc != 0 )
+ goto out;
+
+ /* record bitmap_size */
+ bitmap_size = paging->bitmap_size;
+ max_pages = paging->domain_info->max_pages;
/* Initialise MRU list of paged in pages */
for ( i = 0; i < MRU_SIZE; i++ )
@@ -51,8 +63,6 @@ int policy_init(xenpaging_t *paging)
/* Don''t page out page 0 */
set_bit(0, bitmap);
- rc = 0;
-
out:
return rc;
}
@@ -60,17 +70,27 @@ int policy_init(xenpaging_t *paging)
int policy_choose_victim(xenpaging_t *paging, domid_t domain_id,
xenpaging_victim_t *victim)
{
+ unsigned long wrap = current_gfn;
ASSERT(victim != NULL);
/* Domain to pick on */
victim->domain_id = domain_id;
-
+
do
{
- /* Randomly choose a gfn to evict */
- victim->gfn = rand() % paging->domain_info->max_pages;
+ current_gfn++;
+ if ( current_gfn >= max_pages )
+ current_gfn = 0;
+ if ( wrap == current_gfn )
+ {
+ victim->gfn = INVALID_MFN;
+ return -ENOSPC;
+ }
}
- while ( test_bit(victim->gfn, bitmap) );
+ while ( test_bit(current_gfn, bitmap) || test_bit(current_gfn, unconsumed)
);
+
+ set_bit(current_gfn, unconsumed);
+ victim->gfn = current_gfn;
return 0;
}
@@ -78,6 +98,7 @@ int policy_choose_victim(xenpaging_t *pa
void policy_notify_paged_out(domid_t domain_id, unsigned long gfn)
{
set_bit(gfn, bitmap);
+ clear_bit(gfn, unconsumed);
}
void policy_notify_paged_in(domid_t domain_id, unsigned long gfn)
--- xen-4.0.1-testing.orig/tools/xenpaging/xenpaging.c
+++ xen-4.0.1-testing/tools/xenpaging/xenpaging.c
@@ -440,7 +440,8 @@ static int evict_victim(xenpaging_t *pag
ret = policy_choose_victim(paging, domain_id, victim);
if ( ret != 0 )
{
- ERROR("Error choosing victim");
+ if ( ret != -ENOSPC )
+ ERROR("Error choosing victim");
goto out;
}
@@ -518,7 +519,9 @@ int main(int argc, char *argv[])
memset(victims, 0, sizeof(xenpaging_victim_t) * num_pages);
for ( i = 0; i < num_pages; i++ )
{
- evict_victim(paging, domain_id, &victims[i], fd, i);
+ rc = evict_victim(paging, domain_id, &victims[i], fd, i);
+ if ( rc == -ENOSPC )
+ break;
if ( i % 100 == 0 )
DPRINTF("%d pages evicted\n", i);
}
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 05/18] xenpaging: Open paging file only if xenpaging_init() succeeds
Open paging file only if xenpaging_init() succeeds. It can fail if the host
does not support the required virtualization features such as EPT or if
xenpaging was already started for this domain_id.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Already-Acked-by: Patrick Colp <pjcolp@cs.ubc.ca>
Already-Acked-by: Keir Fraser <keir.fraser@citrix.com>
---
tools/xenpaging/xenpaging.c | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
--- xen-4.0.1-testing.orig/tools/xenpaging/xenpaging.c
+++ xen-4.0.1-testing/tools/xenpaging/xenpaging.c
@@ -495,15 +495,6 @@ int main(int argc, char *argv[])
victims = calloc(num_pages, sizeof(xenpaging_victim_t));
- /* Open file */
- sprintf(filename, "page_cache_%d", domain_id);
- fd = open(filename, open_flags, open_mode);
- if ( fd < 0 )
- {
- perror("failed to open file");
- return -1;
- }
-
/* Seed random-number generator */
srand(time(NULL));
@@ -515,6 +506,15 @@ int main(int argc, char *argv[])
goto out;
}
+ /* Open file */
+ sprintf(filename, "page_cache_%d", domain_id);
+ fd = open(filename, open_flags, open_mode);
+ if ( fd < 0 )
+ {
+ perror("failed to open file");
+ return -1;
+ }
+
/* Evict pages */
memset(victims, 0, sizeof(xenpaging_victim_t) * num_pages);
for ( i = 0; i < num_pages; i++ )
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 06/18] xenpaging: allow only one xenpaging binary per guest
Make sure only one xenpaging binary is active per domain.
Print info when the host lacks the required features for xenpaging.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Already-Acked-by: Patrick Colp <pjcolp@cs.ubc.ca>
Already-Acked-by: Keir Fraser <keir.fraser@citrix.com>
---
tools/xenpaging/xenpaging.c | 12 +++++++++++-
xen/arch/x86/mm/mem_event.c | 7 +++++++
2 files changed, 18 insertions(+), 1 deletion(-)
--- xen-4.0.1-testing.orig/tools/xenpaging/xenpaging.c
+++ xen-4.0.1-testing/tools/xenpaging/xenpaging.c
@@ -119,7 +119,17 @@ xenpaging_t *xenpaging_init(domid_t doma
paging->mem_event.ring_page);
if ( rc != 0 )
{
- ERROR("Error initialising shared page");
+ switch ( errno ) {
+ case EBUSY:
+ ERROR("xenpaging is (or was) active on this domain");
+ break;
+ case ENODEV:
+ ERROR("EPT not supported for this guest");
+ break;
+ default:
+ ERROR("Error initialising shared page");
+ break;
+ }
goto err;
}
--- xen-4.0.1-testing.orig/xen/arch/x86/mm/mem_event.c
+++ xen-4.0.1-testing/xen/arch/x86/mm/mem_event.c
@@ -226,6 +226,13 @@ int mem_event_domctl(struct domain *d, x
mfn_t ring_mfn;
mfn_t shared_mfn;
+ /* Only one xenpaging at a time. If xenpaging crashed,
+ * the cache is in an undefined state and so is the guest
+ */
+ rc = -EBUSY;
+ if ( d->mem_event.enabled )
+ break;
+
/* Currently only EPT is supported */
rc = -ENODEV;
if ( !(is_hvm_domain(d) &&
d->arch.hvm_domain.hap_enabled &&
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 07/18] xenpaging/qemu-dm: add command to flush buffer cache.
Add support for a xenstore dm command to flush qemu''s buffer cache.
qemu will just keep mapping pages and not release them, which causes problems
for the memory pager (since the page is mapped, it won''t get paged
out). When
the pager has trouble finding a page to page out, it asks qemu to flush its
buffer, which releases all the page mappings. This makes it possible to find
pages to swap out agian.
Already-Signed-off-by: Patrick Colp <Patrick.Colp@citrix.com>
Signed-off-by: Olaf Hering <olaf@aepfle.de>
---
tools/ioemu-remote/xenstore.c | 3 +++
1 file changed, 3 insertions(+)
--- xen-4.0.1-testing.orig/tools/ioemu-remote/xenstore.c
+++ xen-4.0.1-testing/tools/ioemu-remote/xenstore.c
@@ -1021,6 +1021,9 @@ static void xenstore_process_dm_command_
do_pci_add(par);
free(par);
#endif
+ } else if (!strncmp(command, "flush-cache", len)) {
+ fprintf(logfile, "dm-command: flush caches\n");
+ qemu_invalidate_map_cache();
} else {
fprintf(logfile, "dm-command: unknown
command\"%*s\"\n", len, command);
}
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 08/18] xenpaging: handle paged-out pages in XENMEM_* commands
Fix these two warings:
(XEN) Assertion ''__mfn_valid(mfn_x(omfn))'' failed at
p2m.c:2200
(XEN) memory.c:171:d1 Domain 1 page number 37ff0 invalid
Handle paged-out pages in xc_memory_op, guest_physmap_add_entry and
guest_remove_page. Use new do_xenmem_op_retry helper function.
In addition, export also xen/errno.h to hvmloader to get ENOENT define.
XENMEM_populate_physmap
populate_physmap
-> guest_physmap_add_entry
XENMEM_exchange
memory_exchange
-> guest_physmap_add_entry
XENMEM_add_to_physmap
guest_physmap_add_page
-> guest_physmap_add_entry
__gnttab_map_grant_ref
create_grant_host_mapping
create_grant_p2m_mapping
-> guest_physmap_add_entry
XENMEM_decrease_reservation
decrease_reservation
-> guest_remove_page
XENMEM_add_to_physmap
-> guest_remove_page
XENMEM_add_to_physmap
-> XENMAPSPACE_gmfn
Signed-off-by: Olaf Hering <olaf@aepfle.de>
---
tools/firmware/hvmloader/hvmloader.c | 9 +++-
tools/firmware/hvmloader/util.c | 26 +++++++++++-
tools/include/Makefile | 1
tools/ioemu-remote/hw/vga.c | 5 +-
tools/libxc/xc_domain.c | 71 +++++++++++++++++++++--------------
xen/arch/x86/mm.c | 26 ++++++++++--
xen/arch/x86/mm/p2m.c | 7 +++
xen/common/memory.c | 25 +++++++++++-
8 files changed, 131 insertions(+), 39 deletions(-)
--- xen-4.0.1-testing.orig/tools/firmware/hvmloader/hvmloader.c
+++ xen-4.0.1-testing/tools/firmware/hvmloader/hvmloader.c
@@ -29,6 +29,7 @@
#include "pci_regs.h"
#include "e820.h"
#include "option_rom.h"
+#include <xen/errno.h>
#include <xen/version.h>
#include <xen/hvm/params.h>
#include <xen/memory.h>
@@ -306,13 +307,19 @@ static void pci_setup(void)
while ( (pci_mem_start >> PAGE_SHIFT) < hvm_info->low_mem_pgend
)
{
struct xen_add_to_physmap xatp;
+ int rc;
if ( hvm_info->high_mem_pgend == 0 )
hvm_info->high_mem_pgend = 1ull << (32 - PAGE_SHIFT);
xatp.domid = DOMID_SELF;
xatp.space = XENMAPSPACE_gmfn;
xatp.idx = --hvm_info->low_mem_pgend;
xatp.gpfn = hvm_info->high_mem_pgend++;
- if ( hypercall_memory_op(XENMEM_add_to_physmap, &xatp) != 0 )
+ do {
+ rc = hypercall_memory_op(XENMEM_add_to_physmap, &xatp);
+ if ( rc == -ENOENT )
+ cpu_relax();
+ } while ( rc == -ENOENT );
+ if ( rc != 0 )
BUG();
}
--- xen-4.0.1-testing.orig/tools/firmware/hvmloader/util.c
+++ xen-4.0.1-testing/tools/firmware/hvmloader/util.c
@@ -23,6 +23,7 @@
#include "e820.h"
#include "hypercall.h"
#include <stdint.h>
+#include <xen/errno.h>
#include <xen/xen.h>
#include <xen/memory.h>
@@ -323,19 +324,27 @@ void *mem_alloc(uint32_t size, uint32_t
while ( (reserve >> PAGE_SHIFT) != (e >> PAGE_SHIFT) )
{
+ int rc;
reserve += PAGE_SIZE;
mfn = reserve >> PAGE_SHIFT;
/* Try to allocate a brand new page in the reserved area. */
if ( !over_allocated )
{
+ uint8_t delay = 0;
xmr.domid = DOMID_SELF;
xmr.mem_flags = 0;
xmr.extent_order = 0;
xmr.nr_extents = 1;
set_xen_guest_handle(xmr.extent_start, &mfn);
- if ( hypercall_memory_op(XENMEM_populate_physmap, &xmr) == 1 )
+ do {
+ rc = hypercall_memory_op(XENMEM_populate_physmap, &xmr);
+ if ( rc == 0 )
+ cpu_relax();
+ } while ( rc == 0 && ++delay );
+ if ( rc == 1 )
continue;
+ printf("%s: over_allocated\n", __func__);
over_allocated = 1;
}
@@ -353,7 +362,12 @@ void *mem_alloc(uint32_t size, uint32_t
xatp.domid = DOMID_SELF;
xatp.space = XENMAPSPACE_gmfn;
xatp.gpfn = mfn;
- if ( hypercall_memory_op(XENMEM_add_to_physmap, &xatp) != 0 )
+ do {
+ rc = hypercall_memory_op(XENMEM_add_to_physmap, &xatp);
+ if ( rc == -ENOENT )
+ cpu_relax();
+ } while ( rc == -ENOENT );
+ if ( rc != 0 )
BUG();
}
@@ -595,6 +609,7 @@ uint16_t get_cpu_mhz(void)
uint64_t cpu_khz;
uint32_t tsc_to_nsec_mul, version;
int8_t tsc_shift;
+ int rc;
static uint16_t cpu_mhz;
if ( cpu_mhz != 0 )
@@ -605,7 +620,12 @@ uint16_t get_cpu_mhz(void)
xatp.space = XENMAPSPACE_shared_info;
xatp.idx = 0;
xatp.gpfn = (unsigned long)shared_info >> 12;
- if ( hypercall_memory_op(XENMEM_add_to_physmap, &xatp) != 0 )
+ do {
+ rc = hypercall_memory_op(XENMEM_add_to_physmap, &xatp);
+ if ( rc == -ENOENT )
+ cpu_relax();
+ } while ( rc == -ENOENT );
+ if ( rc != 0 )
BUG();
/* Get a consistent snapshot of scale factor (multiplier and shift). */
--- xen-4.0.1-testing.orig/tools/include/Makefile
+++ xen-4.0.1-testing/tools/include/Makefile
@@ -12,6 +12,7 @@ xen/.dir:
@rm -rf xen
mkdir -p xen/libelf
ln -sf ../$(XEN_ROOT)/xen/include/public/COPYING xen
+ ln -sf ../$(XEN_ROOT)/xen/include/xen/errno.h xen
ln -sf $(addprefix ../,$(wildcard $(XEN_ROOT)/xen/include/public/*.h)) xen
ln -sf $(addprefix ../$(XEN_ROOT)/xen/include/public/,arch-ia64 arch-x86 hvm
io xsm) xen
ln -sf ../xen-sys/$(XEN_OS) xen/sys
--- xen-4.0.1-testing.orig/tools/ioemu-remote/hw/vga.c
+++ xen-4.0.1-testing/tools/ioemu-remote/hw/vga.c
@@ -2157,9 +2157,10 @@ void set_vram_mapping(void *opaque, unsi
for (i = 0; i < (end - begin) >> TARGET_PAGE_BITS; i++) {
xatp.idx = (s->vram_gmfn >> TARGET_PAGE_BITS) + i;
xatp.gpfn = (begin >> TARGET_PAGE_BITS) + i;
- rc = xc_memory_op(xc_handle, XENMEM_add_to_physmap, &xatp);
+ while ((rc = xc_memory_op(xc_handle, XENMEM_add_to_physmap, &xatp))
&& errno == ENOENT)
+ usleep(1000);
if (rc) {
- fprintf(stderr, "add_to_physmap MFN %"PRI_xen_pfn"
to PFN %"PRI_xen_pfn" failed: %d\n", xatp.idx, xatp.gpfn, rc);
+ fprintf(stderr, "add_to_physmap MFN %"PRI_xen_pfn"
to PFN %"PRI_xen_pfn" failed: %d\n", xatp.idx, xatp.gpfn, errno);
return;
}
}
--- xen-4.0.1-testing.orig/tools/libxc/xc_domain.c
+++ xen-4.0.1-testing/tools/libxc/xc_domain.c
@@ -536,6 +536,44 @@ int xc_domain_get_tsc_info(int xc_handle
return rc;
}
+static int do_xenmem_op_retry(int xc_handle, int cmd, struct
xen_memory_reservation *reservation, unsigned long nr_extents, xen_pfn_t
*extent_start)
+{
+ int err = 0;
+ unsigned long count = nr_extents;
+ unsigned long delay = 0;
+ unsigned long start = 0;
+
+ fprintf(stderr, "%s: cmd %d count %lx\n",__func__,cmd,count);
+ while ( count && start < nr_extents )
+ {
+ set_xen_guest_handle(reservation->extent_start, extent_start +
start);
+ reservation->nr_extents = count;
+
+ err = xc_memory_op(xc_handle, cmd, reservation);
+ if ( err == count )
+ {
+ err = 0;
+ break;
+ }
+
+ if ( err > count || err < 0 || delay > 1000 * 1000)
+ {
+ fprintf(stderr, "%s: %d err %x count %lx start %lx delay
%lu/%lu\n",__func__,cmd,err,count,start,delay,delay/666);
+ err = -1;
+ break;
+ }
+
+ if ( err )
+ delay = 0;
+
+ start += err;
+ count -= err;
+ usleep(delay);
+ delay += 666; /* 1500 iterations, 12 seconds */
+ }
+
+ return err;
+}
int xc_domain_memory_increase_reservation(int xc_handle,
uint32_t domid,
@@ -546,26 +584,18 @@ int xc_domain_memory_increase_reservatio
{
int err;
struct xen_memory_reservation reservation = {
- .nr_extents = nr_extents,
.extent_order = extent_order,
.mem_flags = mem_flags,
.domid = domid
};
- /* may be NULL */
- set_xen_guest_handle(reservation.extent_start, extent_start);
-
- err = xc_memory_op(xc_handle, XENMEM_increase_reservation,
&reservation);
- if ( err == nr_extents )
- return 0;
-
- if ( err >= 0 )
+ err = do_xenmem_op_retry(xc_handle, XENMEM_increase_reservation,
&reservation, nr_extents, extent_start);
+ if ( err < 0 )
{
DPRINTF("Failed allocation for dom %d: "
"%ld extents of order %d, mem_flags %x\n",
domid, nr_extents, extent_order, mem_flags);
errno = ENOMEM;
- err = -1;
}
return err;
@@ -579,14 +609,11 @@ int xc_domain_memory_decrease_reservatio
{
int err;
struct xen_memory_reservation reservation = {
- .nr_extents = nr_extents,
.extent_order = extent_order,
.mem_flags = 0,
.domid = domid
};
- set_xen_guest_handle(reservation.extent_start, extent_start);
-
if ( extent_start == NULL )
{
DPRINTF("decrease_reservation extent_start is NULL!\n");
@@ -594,16 +621,12 @@ int xc_domain_memory_decrease_reservatio
return -1;
}
- err = xc_memory_op(xc_handle, XENMEM_decrease_reservation,
&reservation);
- if ( err == nr_extents )
- return 0;
-
- if ( err >= 0 )
+ err = do_xenmem_op_retry(xc_handle, XENMEM_decrease_reservation,
&reservation, nr_extents, extent_start);
+ if ( err < 0 )
{
DPRINTF("Failed deallocation for dom %d: %ld extents of order
%d\n",
domid, nr_extents, extent_order);
errno = EINVAL;
- err = -1;
}
return err;
@@ -618,23 +641,17 @@ int xc_domain_memory_populate_physmap(in
{
int err;
struct xen_memory_reservation reservation = {
- .nr_extents = nr_extents,
.extent_order = extent_order,
.mem_flags = mem_flags,
.domid = domid
};
- set_xen_guest_handle(reservation.extent_start, extent_start);
-
- err = xc_memory_op(xc_handle, XENMEM_populate_physmap, &reservation);
- if ( err == nr_extents )
- return 0;
- if ( err >= 0 )
+ err = do_xenmem_op_retry(xc_handle, XENMEM_populate_physmap,
&reservation, nr_extents, extent_start);
+ if ( err < 0 )
{
DPRINTF("Failed allocation for dom %d: %ld extents of order
%d\n",
domid, nr_extents, extent_order);
errno = EBUSY;
- err = -1;
}
return err;
--- xen-4.0.1-testing.orig/xen/arch/x86/mm.c
+++ xen-4.0.1-testing/xen/arch/x86/mm.c
@@ -3660,6 +3660,8 @@ static int create_grant_p2m_mapping(uint
p2mt = p2m_grant_map_rw;
rc = guest_physmap_add_entry(current->domain, addr >> PAGE_SHIFT,
frame, 0, p2mt);
+ if ( rc == -ENOENT )
+ return GNTST_eagain;
if ( rc )
return GNTST_general_error;
else
@@ -4315,17 +4317,25 @@ long arch_memory_op(int op, XEN_GUEST_HA
case XENMAPSPACE_gmfn:
{
p2m_type_t p2mt;
+ unsigned long tmp_mfn;
- xatp.idx = mfn_x(gfn_to_mfn_unshare(d, xatp.idx, &p2mt, 0));
+ tmp_mfn = mfn_x(gfn_to_mfn_unshare(d, xatp.idx, &p2mt, 0));
+ if ( unlikely(p2m_is_paging(p2mt)) )
+ {
+ if ( p2m_is_paged(p2mt) )
+ p2m_mem_paging_populate(d, xatp.idx);
+ rcu_unlock_domain(d);
+ return -ENOENT;
+ }
/* If the page is still shared, exit early */
if ( p2m_is_shared(p2mt) )
{
rcu_unlock_domain(d);
return -ENOMEM;
}
- if ( !get_page_from_pagenr(xatp.idx, d) )
+ if ( !get_page_from_pagenr(tmp_mfn, d) )
break;
- mfn = xatp.idx;
+ mfn = tmp_mfn;
page = mfn_to_page(mfn);
break;
}
@@ -4354,8 +4364,16 @@ long arch_memory_op(int op, XEN_GUEST_HA
/* Xen heap frames are simply unhooked from this phys slot. */
guest_physmap_remove_page(d, xatp.gpfn, prev_mfn, 0);
else
+ {
/* Normal domain memory is freed, to avoid leaking memory. */
- guest_remove_page(d, xatp.gpfn);
+ rc = guest_remove_page(d, xatp.gpfn);
+ if ( rc == -ENOENT )
+ {
+ domain_unlock(d);
+ rcu_unlock_domain(d);
+ return rc;
+ }
+ }
}
/* Unmap from old location, if any. */
--- xen-4.0.1-testing.orig/xen/arch/x86/mm/p2m.c
+++ xen-4.0.1-testing/xen/arch/x86/mm/p2m.c
@@ -2186,6 +2186,13 @@ guest_physmap_add_entry(struct domain *d
P2M_DEBUG("aliased! mfn=%#lx, old gfn=%#lx, new
gfn=%#lx\n",
mfn + i, ogfn, gfn + i);
omfn = gfn_to_mfn_query(d, ogfn, &ot);
+ if ( unlikely(p2m_is_paging(ot)) )
+ {
+ p2m_unlock(d->arch.p2m);
+ if ( p2m_is_paged(ot) )
+ p2m_mem_paging_populate(d, ogfn);
+ return -ENOENT;
+ }
/* If we get here, we know the local domain owns the page,
so it can''t have been grant mapped in. */
BUG_ON( p2m_is_grant(ot) );
--- xen-4.0.1-testing.orig/xen/common/memory.c
+++ xen-4.0.1-testing/xen/common/memory.c
@@ -95,6 +95,7 @@ static void populate_physmap(struct memo
unsigned long i, j;
xen_pfn_t gpfn, mfn;
struct domain *d = a->domain;
+ int rc;
if ( !guest_handle_subrange_okay(a->extent_list, a->nr_done,
a->nr_extents-1) )
@@ -134,7 +135,12 @@ static void populate_physmap(struct memo
}
mfn = page_to_mfn(page);
- guest_physmap_add_page(d, gpfn, mfn, a->extent_order);
+ rc = guest_physmap_add_page(d, gpfn, mfn, a->extent_order);
+ if ( rc != 0 )
+ {
+ free_domheap_pages(page, a->extent_order);
+ goto out;
+ }
if ( !paging_mode_translate(d) )
{
@@ -162,6 +168,12 @@ int guest_remove_page(struct domain *d,
#ifdef CONFIG_X86
mfn = mfn_x(gfn_to_mfn(d, gmfn, &p2mt));
+ if ( unlikely(p2m_is_paging(p2mt)) )
+ {
+ if ( p2m_is_paged(p2mt) )
+ p2m_mem_paging_populate(d, gmfn);
+ return -ENOENT;
+ }
#else
mfn = gmfn_to_mfn(d, gmfn);
#endif
@@ -360,6 +372,13 @@ static long memory_exchange(XEN_GUEST_HA
/* Shared pages cannot be exchanged */
mfn = mfn_x(gfn_to_mfn_unshare(d, gmfn + k, &p2mt, 0));
+ if ( p2m_is_paging(p2mt) )
+ {
+ if ( p2m_is_paged(p2mt) )
+ p2m_mem_paging_populate(d, gmfn);
+ rc = -ENOENT;
+ goto fail;
+ }
if ( p2m_is_shared(p2mt) )
{
rc = -ENOMEM;
@@ -456,7 +475,9 @@ static long memory_exchange(XEN_GUEST_HA
&gpfn, exch.out.extent_start, (i<<out_chunk_order)+j,
1);
mfn = page_to_mfn(page);
- guest_physmap_add_page(d, gpfn, mfn, exch.out.extent_order);
+ rc = guest_physmap_add_page(d, gpfn, mfn, exch.out.extent_order);
+ if ( rc == -ENOENT )
+ goto fail;
if ( !paging_mode_translate(d) )
{
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 09/18] xenpaging: populate only paged-out pages
populdate a paged-out page only once to reduce pressure in the ringbuffer.
Several cpus may still request a page at once. xenpaging can handle this.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
---
xen/arch/x86/hvm/emulate.c | 3 ++-
xen/arch/x86/hvm/hvm.c | 17 ++++++++++-------
xen/arch/x86/mm/guest_walk.c | 3 ++-
xen/arch/x86/mm/hap/guest_walk.c | 6 ++++--
4 files changed, 18 insertions(+), 11 deletions(-)
--- xen-4.0.1-testing.orig/xen/arch/x86/hvm/emulate.c
+++ xen-4.0.1-testing/xen/arch/x86/hvm/emulate.c
@@ -65,7 +65,8 @@ static int hvmemul_do_io(
ram_mfn = gfn_to_mfn_unshare(current->domain, ram_gfn, &p2mt, 0);
if ( p2m_is_paging(p2mt) )
{
- p2m_mem_paging_populate(curr->domain, ram_gfn);
+ if ( p2m_is_paged(p2mt) )
+ p2m_mem_paging_populate(curr->domain, ram_gfn);
return X86EMUL_RETRY;
}
if ( p2m_is_shared(p2mt) )
--- xen-4.0.1-testing.orig/xen/arch/x86/hvm/hvm.c
+++ xen-4.0.1-testing/xen/arch/x86/hvm/hvm.c
@@ -291,7 +291,8 @@ static int hvm_set_ioreq_page(
return -EINVAL;
if ( p2m_is_paging(p2mt) )
{
- p2m_mem_paging_populate(d, gmfn);
+ if ( p2m_is_paged(p2mt) )
+ p2m_mem_paging_populate(d, gmfn);
return -ENOENT;
}
if ( p2m_is_shared(p2mt) )
@@ -1324,7 +1325,8 @@ static void *hvm_map_entry(unsigned long
mfn = mfn_x(gfn_to_mfn_unshare(current->domain, gfn, &p2mt, 0));
if ( p2m_is_paging(p2mt) )
{
- p2m_mem_paging_populate(current->domain, gfn);
+ if ( p2m_is_paged(p2mt) )
+ p2m_mem_paging_populate(current->domain, gfn);
return NULL;
}
if ( p2m_is_shared(p2mt) )
@@ -1723,7 +1725,8 @@ static enum hvm_copy_result __hvm_copy(
if ( p2m_is_paging(p2mt) )
{
- p2m_mem_paging_populate(curr->domain, gfn);
+ if ( p2m_is_paged(p2mt) )
+ p2m_mem_paging_populate(curr->domain, gfn);
return HVMCOPY_gfn_paged_out;
}
if ( p2m_is_shared(p2mt) )
@@ -3032,8 +3035,8 @@ long do_hvm_op(unsigned long op, XEN_GUE
mfn_t mfn = gfn_to_mfn(d, pfn, &t);
if ( p2m_is_paging(t) )
{
- p2m_mem_paging_populate(d, pfn);
-
+ if ( p2m_is_paged(t) )
+ p2m_mem_paging_populate(d, pfn);
rc = -EINVAL;
goto param_fail3;
}
@@ -3096,8 +3099,8 @@ long do_hvm_op(unsigned long op, XEN_GUE
mfn = gfn_to_mfn_unshare(d, pfn, &t, 0);
if ( p2m_is_paging(t) )
{
- p2m_mem_paging_populate(d, pfn);
-
+ if ( p2m_is_paged(t) )
+ p2m_mem_paging_populate(d, pfn);
rc = -EINVAL;
goto param_fail4;
}
--- xen-4.0.1-testing.orig/xen/arch/x86/mm/guest_walk.c
+++ xen-4.0.1-testing/xen/arch/x86/mm/guest_walk.c
@@ -96,7 +96,8 @@ static inline void *map_domain_gfn(struc
*mfn = gfn_to_mfn_unshare(d, gfn_x(gfn), p2mt, 0);
if ( p2m_is_paging(*p2mt) )
{
- p2m_mem_paging_populate(d, gfn_x(gfn));
+ if ( p2m_is_paged(*p2mt) )
+ p2m_mem_paging_populate(d, gfn_x(gfn));
*rc = _PAGE_PAGED;
return NULL;
--- xen-4.0.1-testing.orig/xen/arch/x86/mm/hap/guest_walk.c
+++ xen-4.0.1-testing/xen/arch/x86/mm/hap/guest_walk.c
@@ -49,7 +49,8 @@ unsigned long hap_gva_to_gfn(GUEST_PAGIN
top_mfn = gfn_to_mfn_unshare(v->domain, cr3 >> PAGE_SHIFT,
&p2mt, 0);
if ( p2m_is_paging(p2mt) )
{
- p2m_mem_paging_populate(v->domain, cr3 >> PAGE_SHIFT);
+ if ( p2m_is_paged(p2mt) )
+ p2m_mem_paging_populate(v->domain, cr3 >> PAGE_SHIFT);
pfec[0] = PFEC_page_paged;
return INVALID_GFN;
@@ -81,7 +82,8 @@ unsigned long hap_gva_to_gfn(GUEST_PAGIN
gfn_to_mfn_unshare(v->domain, gfn_x(gfn), &p2mt, 0);
if ( p2m_is_paging(p2mt) )
{
- p2m_mem_paging_populate(v->domain, gfn_x(gfn));
+ if ( p2m_is_paged(p2mt) )
+ p2m_mem_paging_populate(v->domain, gfn_x(gfn));
pfec[0] = PFEC_page_paged;
return INVALID_GFN;
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 10/18] xenpaging: reduce MINIMUM_RESTART_TIME
Allow runlevel 6 as default runlevel. Signed-off-by: Olaf Hering <olaf@aepfle.de> --- tools/python/xen/xend/XendConstants.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- xen-4.0.1-testing.orig/tools/python/xen/xend/XendConstants.py +++ xen-4.0.1-testing/tools/python/xen/xend/XendConstants.py @@ -92,7 +92,7 @@ DOM_STATES_OLD = [ SHUTDOWN_TIMEOUT = (60.0 * 5) """Minimum time between domain restarts in seconds.""" -MINIMUM_RESTART_TIME = 60 +MINIMUM_RESTART_TIME = 5 RESTART_IN_PROGRESS = ''xend/restart_in_progress'' DUMPCORE_IN_PROGRESS = ''xend/dumpcore_in_progress'' _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 11/18] xenpaging: start xenpaging via config option
Start xenpaging via config option.
TODO: add config option for pagefile directory
TODO: add libxl support
TODO: parse config values like 42K, 42M, 42G, 42%
Signed-off-by: Olaf Hering <olaf@aepfle.de>
---
tools/examples/xmexample.hvm | 3 +
tools/python/README.XendConfig | 1
tools/python/README.sxpcfg | 1
tools/python/xen/xend/XendConfig.py | 3 +
tools/python/xen/xend/XendDomainInfo.py | 6 ++
tools/python/xen/xend/image.py | 87 ++++++++++++++++++++++++++++++++
tools/python/xen/xm/create.py | 5 +
tools/python/xen/xm/xenapi_create.py | 1
8 files changed, 107 insertions(+)
--- xen-4.0.1-testing.orig/tools/examples/xmexample.hvm
+++ xen-4.0.1-testing/tools/examples/xmexample.hvm
@@ -127,6 +127,9 @@ disk = [ ''file:/var/lib/xen/images/disk.
# Device Model to be used
device_model = ''qemu-dm''
+# xenpaging, number of pages
+xenpaging = 42
+
#-----------------------------------------------------------------------------
# boot on floppy (a), hard disk (c), Network (n) or CD-ROM (d)
# default: hard disk, cd-rom, floppy
--- xen-4.0.1-testing.orig/tools/python/README.XendConfig
+++ xen-4.0.1-testing/tools/python/README.XendConfig
@@ -120,6 +120,7 @@ otherConfig
image.vncdisplay
image.vncunused
image.hvm.device_model
+ image.hvm.xenpaging
image.hvm.display
image.hvm.xauthority
image.hvm.vncconsole
--- xen-4.0.1-testing.orig/tools/python/README.sxpcfg
+++ xen-4.0.1-testing/tools/python/README.sxpcfg
@@ -51,6 +51,7 @@ image
- vncunused
(HVM)
- device_model
+ - xenpaging
- display
- xauthority
- vncconsole
--- xen-4.0.1-testing.orig/tools/python/xen/xend/XendConfig.py
+++ xen-4.0.1-testing/tools/python/xen/xend/XendConfig.py
@@ -145,6 +145,7 @@ XENAPI_PLATFORM_CFG_TYPES = {
''apic'': int,
''boot'': str,
''device_model'': str,
+ ''xenpaging'': int,
''loader'': str,
''display'' : str,
''fda'': str,
@@ -508,6 +509,8 @@ class XendConfig(dict):
self[''platform''][''nomigrate''] =
0
if self.is_hvm():
+ if ''xenpaging'' not in
self[''platform'']:
+
self[''platform''][''xenpaging''] = None
if ''timer_mode'' not in
self[''platform'']:
self[''platform''][''timer_mode''] = 1
if ''viridian'' not in
self[''platform'']:
--- xen-4.0.1-testing.orig/tools/python/xen/xend/XendDomainInfo.py
+++ xen-4.0.1-testing/tools/python/xen/xend/XendDomainInfo.py
@@ -2439,6 +2439,7 @@ class XendDomainInfo:
if self.image:
self.image.createDeviceModel()
+ self.image.createXenPaging()
#if have pass-through devs, need the virtual pci slots info from qemu
self.pci_device_configure_boot()
@@ -2451,6 +2452,11 @@ class XendDomainInfo:
self.image.destroyDeviceModel()
except Exception, e:
log.exception("Device model destroy failed %s" %
str(e))
+ try:
+ log.debug("stopping xenpaging")
+ self.image.destroyXenPaging()
+ except Exception, e:
+ log.exception("stopping xenpaging failed %s" %
str(e))
else:
log.debug("No device model")
--- xen-4.0.1-testing.orig/tools/python/xen/xend/image.py
+++ xen-4.0.1-testing/tools/python/xen/xend/image.py
@@ -122,12 +122,14 @@ class ImageHandler:
self.vm.permissionsVm("image/cmdline", {
''dom'': self.vm.getDomid(), ''read'': True } )
self.device_model =
vmConfig[''platform''].get(''device_model'')
+ self.xenpaging =
vmConfig[''platform''].get(''xenpaging'')
self.display =
vmConfig[''platform''].get(''display'')
self.xauthority =
vmConfig[''platform''].get(''xauthority'')
self.vncconsole =
int(vmConfig[''platform''].get(''vncconsole'',
0))
self.dmargs = self.parseDeviceModelArgs(vmConfig)
self.pid = None
+ self.xenpaging_pid = None
rtc_timeoffset =
int(vmConfig[''platform''].get(''rtc_timeoffset'',
0))
if
int(vmConfig[''platform''].get(''localtime'',
0)):
if time.localtime(time.time())[8]:
@@ -392,6 +394,91 @@ class ImageHandler:
sentinel_fifos_inuse[sentinel_path_fifo] = 1
self.sentinel_path_fifo = sentinel_path_fifo
+ def createXenPaging(self):
+ if self.xenpaging is None:
+ return
+ if self.xenpaging == 0:
+ return
+ if self.xenpaging_pid:
+ return
+ xenpaging_bin = auxbin.pathTo("xenpaging")
+ args = [xenpaging_bin]
+ args = args + ([ "%d" % self.vm.getDomid()])
+ args = args + ([ "%s" % self.xenpaging])
+ env = dict(os.environ)
+ self.xenpaging_logfile = "/var/log/xen/xenpaging-%s.log" %
str(self.vm.info[''name_label''])
+ logfile_mode = os.O_WRONLY|os.O_CREAT|os.O_APPEND|os.O_TRUNC
+ null = os.open("/dev/null", os.O_RDONLY)
+ logfd = os.open(self.xenpaging_logfile, logfile_mode, 0644)
+ sys.stderr.flush()
+ contract = osdep.prefork("%s:%d" % (self.vm.getName(),
self.vm.getDomid()))
+ xenpaging_pid = os.fork()
+ if xenpaging_pid == 0: #child
+ try:
+ xenpaging_dir = "/var/lib/xen/xenpaging"
+ osdep.postfork(contract)
+ os.dup2(null, 0)
+ os.dup2(logfd, 1)
+ os.dup2(logfd, 2)
+ try:
+ os.mkdir(xenpaging_dir)
+ except:
+ log.info("mkdir %s failed" % xenpaging_dir)
+ pass
+ try:
+ os.chdir(xenpaging_dir)
+ except:
+ log.warn("chdir %s failed" % xenpaging_dir)
+ try:
+ log.info("starting %s" % args)
+ os.execve(xenpaging_bin, args, env)
+ except Exception, e:
+ print >>sys.stderr, (
+ ''failed to execute xenpaging: %s: %s''
%
+ xenpaging_bin, utils.exception_string(e))
+ os._exit(126)
+ except Exception, e:
+ log.warn("staring xenpaging in %s failed" %
xenpaging_dir)
+ os._exit(127)
+ else:
+ osdep.postfork(contract, abandon=True)
+ self.xenpaging_pid = xenpaging_pid
+ os.close(null)
+ os.close(logfd)
+
+ def destroyXenPaging(self):
+ if self.xenpaging is None:
+ return
+ if self.xenpaging_pid:
+ try:
+ os.kill(self.xenpaging_pid, signal.SIGHUP)
+ except OSError, exn:
+ log.exception(exn)
+ for i in xrange(100):
+ try:
+ (p, rv) = os.waitpid(self.xenpaging_pid, os.WNOHANG)
+ if p == self.xenpaging_pid:
+ break
+ except OSError:
+ # This is expected if Xend has been restarted within
+ # the life of this domain. In this case, we can kill
+ # the process, but we can''t wait for it because
it''s
+ # not our child. We continue this loop, and after it is
+ # terminated make really sure the process is going away
+ # (SIGKILL).
+ pass
+ time.sleep(0.1)
+ else:
+ log.warning("xenpaging %d took more than 10s "
+ "to terminate: sending SIGKILL" %
self.xenpaging_pid)
+ try:
+ os.kill(self.xenpaging_pid, signal.SIGKILL)
+ os.waitpid(self.xenpaging_pid, 0)
+ except OSError:
+ # This happens if the process doesn''t exist.
+ pass
+ self.xenpaging_pid = None
+
def createDeviceModel(self, restore = False):
if self.device_model is None:
return
--- xen-4.0.1-testing.orig/tools/python/xen/xm/create.py
+++ xen-4.0.1-testing/tools/python/xen/xm/create.py
@@ -495,6 +495,10 @@ gopts.var(''nfs_root'',
val="PATH",
fn=set_value, default=None,
use="Set the path of the root NFS directory.")
+gopts.var(''xenpaging'', val=''NUM'',
+ fn=set_int, default=None,
+ use="Number of pages to swap.")
+
gopts.var(''device_model'', val=''FILE'',
fn=set_value, default=None,
use="Path to device model program.")
@@ -1080,6 +1084,7 @@ def configure_hvm(config_image, vals):
args = [ ''acpi'', ''apic'',
''boot'',
''cpuid'', ''cpuid_check'',
+ ''xenpaging'',
''device_model'', ''display'',
''fda'', ''fdb'',
''gfx_passthru'',
''guest_os_type'',
--- xen-4.0.1-testing.orig/tools/python/xen/xm/xenapi_create.py
+++ xen-4.0.1-testing/tools/python/xen/xm/xenapi_create.py
@@ -1086,6 +1086,7 @@ class sxp2xml:
''acpi'',
''apic'',
''boot'',
+ ''xenpaging'',
''device_model'',
''loader'',
''fda'',
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 12/18] xenpaging: add signal handling
Leave paging loop if xenpaging gets a signal.
Remove paging file on exit.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
---
tools/xenpaging/xenpaging.c | 39 +++++++++++++++++++++++++++++++--------
1 file changed, 31 insertions(+), 8 deletions(-)
--- xen-4.0.1-testing.orig/tools/xenpaging/xenpaging.c
+++ xen-4.0.1-testing/tools/xenpaging/xenpaging.c
@@ -22,6 +22,7 @@
#include <inttypes.h>
#include <stdlib.h>
+#include <signal.h>
#include <xc_private.h>
#include <xen/mem_event.h>
@@ -40,6 +41,11 @@
#define DPRINTF(...) ((void)0)
#endif
+static int interrupted;
+static void close_handler(int sig)
+{
+ interrupted = sig;
+}
static void *init_page(void)
{
@@ -244,7 +250,6 @@ int xenpaging_teardown(xenpaging_t *pagi
if ( rc != 0 )
{
ERROR("Error tearing down domain paging in xen");
- goto err;
}
/* Unbind VIRQ */
@@ -252,7 +257,6 @@ int xenpaging_teardown(xenpaging_t *pagi
if ( rc != 0 )
{
ERROR("Error unbinding event port");
- goto err;
}
paging->mem_event.port = -1;
@@ -261,7 +265,6 @@ int xenpaging_teardown(xenpaging_t *pagi
if ( rc != 0 )
{
ERROR("Error closing event channel");
- goto err;
}
paging->mem_event.xce_handle = -1;
@@ -270,7 +273,6 @@ int xenpaging_teardown(xenpaging_t *pagi
if ( rc != 0 )
{
ERROR("Error closing connection to xen");
- goto err;
}
paging->xc_handle = -1;
@@ -375,7 +377,7 @@ int xenpaging_evict_page(xenpaging_t *pa
return ret;
}
-int xenpaging_resume_page(xenpaging_t *paging, mem_event_response_t *rsp)
+static int xenpaging_resume_page(xenpaging_t *paging, mem_event_response_t
*rsp)
{
int ret;
@@ -455,6 +457,11 @@ static int evict_victim(xenpaging_t *pag
goto out;
}
+ if ( interrupted )
+ {
+ ret = -EINTR;
+ goto out;
+ }
ret = xc_mem_paging_nominate(paging->xc_handle,
paging->mem_event.domain_id,
victim->gfn);
if ( ret == 0 )
@@ -479,6 +486,7 @@ static int evict_victim(xenpaging_t *pag
int main(int argc, char *argv[])
{
+ struct sigaction act;
domid_t domain_id;
int num_pages;
xenpaging_t *paging;
@@ -513,7 +521,7 @@ int main(int argc, char *argv[])
if ( paging == NULL )
{
ERROR("Error initialising paging");
- goto out;
+ return 1;
}
/* Open file */
@@ -522,9 +530,18 @@ int main(int argc, char *argv[])
if ( fd < 0 )
{
perror("failed to open file");
- return -1;
+ return 2;
}
+ /* ensure that if we get a signal, we''ll do cleanup, then exit */
+ act.sa_handler = close_handler;
+ act.sa_flags = 0;
+ sigemptyset(&act.sa_mask);
+ sigaction(SIGHUP, &act, NULL);
+ sigaction(SIGTERM, &act, NULL);
+ sigaction(SIGINT, &act, NULL);
+ sigaction(SIGALRM, &act, NULL);
+
/* Evict pages */
memset(victims, 0, sizeof(xenpaging_victim_t) * num_pages);
for ( i = 0; i < num_pages; i++ )
@@ -532,6 +549,8 @@ int main(int argc, char *argv[])
rc = evict_victim(paging, domain_id, &victims[i], fd, i);
if ( rc == -ENOSPC )
break;
+ if ( rc == -EINTR )
+ break;
if ( i % 100 == 0 )
DPRINTF("%d pages evicted\n", i);
}
@@ -539,7 +558,7 @@ int main(int argc, char *argv[])
DPRINTF("pages evicted\n");
/* Swap pages in and out */
- while ( 1 )
+ while ( !interrupted )
{
/* Wait for Xen to signal that a page needs paged in */
rc = xc_wait_for_event_or_timeout(paging->mem_event.xce_handle,
100);
@@ -630,8 +649,11 @@ int main(int argc, char *argv[])
}
}
}
+ DPRINTF("xenpaging got signal %d\n", interrupted);
out:
+ unlink(filename);
+ close(fd);
free(victims);
/* Tear down domain paging */
@@ -642,6 +664,7 @@ int main(int argc, char *argv[])
if ( rc == 0 )
rc = rc1;
+ DPRINTF("xenpaging exit code %d\n", rc);
return rc;
}
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 13/18] xenpaging: increase recently used pages from 4MB to 64MB
Increase recently used pages from 4MB to 64MB. Keeping more pages in memory allows the guest to make more progress if the paging file spans the entire guest memory. Signed-off-by: Olaf Hering <olaf@aepfle.de> --- tools/xenpaging/policy_default.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- xen-4.0.1-testing.orig/tools/xenpaging/policy_default.c +++ xen-4.0.1-testing/tools/xenpaging/policy_default.c @@ -26,7 +26,7 @@ #include "policy.h" -#define MRU_SIZE 1024 +#define MRU_SIZE (1024 * 16) static unsigned long mru[MRU_SIZE]; _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 14/18] xenpaging: page-in granttable entries
When converting a gfn to mfn, check if the page is paged-out.
If it is, request a page-in and return GNTST_eagain to the caller
to indicate a retry of the hypercall is required.
This fixes granttable errors when xenpaging is enabled in the guest.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Already-Acked-by: Patrick Colp <pjcolp@cs.ubc.ca>
Already-Acked-by: Keir Fraser <keir.fraser@citrix.com>
---
xen/common/grant_table.c | 94 ++++++++++++++++++++++++++++++-----------------
1 file changed, 60 insertions(+), 34 deletions(-)
--- xen-4.0.1-testing.orig/xen/common/grant_table.c
+++ xen-4.0.1-testing/xen/common/grant_table.c
@@ -139,6 +139,37 @@ shared_entry_header(struct grant_table *
#define active_entry(t, e) \
((t)->active[(e)/ACGNT_PER_PAGE][(e)%ACGNT_PER_PAGE])
+/* Check if the page has been paged out */
+static int __get_paged_frame(unsigned long gfn, unsigned long *frame, int
readonly, struct domain *rd)
+{
+ p2m_type_t p2mt;
+ mfn_t mfn;
+ int rc = GNTST_okay;
+
+ if ( readonly )
+ mfn = gfn_to_mfn(rd, gfn, &p2mt);
+ else
+ mfn = gfn_to_mfn_unshare(rd, gfn, &p2mt, 1);
+
+ if ( p2m_is_valid(p2mt) )
+ {
+ *frame = mfn_x(mfn);
+ if ( p2m_is_paging(p2mt) )
+ {
+ if ( p2m_is_paged(p2mt) )
+ p2m_mem_paging_populate(rd, gfn);
+ rc = GNTST_eagain;
+ }
+ }
+ else
+ {
+ *frame = INVALID_MFN;
+ rc = GNTST_bad_page;
+ }
+
+ return rc;
+}
+
static inline int
__get_maptrack_handle(
struct grant_table *t)
@@ -527,14 +558,16 @@ __gnttab_map_grant_ref(
if ( !act->pin )
{
+ unsigned long gfn;
+ unsigned long frame;
+
+ gfn = sha1 ? sha1->frame : sha2->full_page.frame;
+ rc = __get_paged_frame(gfn, &frame, !!(op->flags &
GNTMAP_readonly), rd);
+ if ( rc != GNTST_okay )
+ goto unlock_out;
+ act->gfn = gfn;
act->domid = ld->domain_id;
- if ( sha1 )
- act->gfn = sha1->frame;
- else
- act->gfn = sha2->full_page.frame;
- act->frame = (op->flags & GNTMAP_readonly) ?
- gmfn_to_mfn(rd, act->gfn) :
- gfn_to_mfn_private(rd, act->gfn);
+ act->frame = frame;
act->start = 0;
act->length = PAGE_SIZE;
act->is_sub_page = 0;
@@ -1697,6 +1730,7 @@ __acquire_grant_for_copy(
domid_t trans_domid;
grant_ref_t trans_gref;
struct domain *rrd;
+ unsigned long gfn;
unsigned long grant_frame;
unsigned trans_page_off;
unsigned trans_length;
@@ -1814,9 +1848,11 @@ __acquire_grant_for_copy(
}
else if ( sha1 )
{
- act->gfn = sha1->frame;
- grant_frame = readonly ? gmfn_to_mfn(rd, act->gfn) :
- gfn_to_mfn_private(rd, act->gfn);
+ gfn = sha1->frame;
+ rc = __get_paged_frame(gfn, &grant_frame, readonly, rd);
+ if ( rc != GNTST_okay )
+ goto unlock_out;
+ act->gfn = gfn;
is_sub_page = 0;
trans_page_off = 0;
trans_length = PAGE_SIZE;
@@ -1824,9 +1860,11 @@ __acquire_grant_for_copy(
}
else if ( !(sha2->hdr.flags & GTF_sub_page) )
{
- act->gfn = sha2->full_page.frame;
- grant_frame = readonly ? gmfn_to_mfn(rd, act->gfn) :
- gfn_to_mfn_private(rd, act->gfn);
+ gfn = sha2->full_page.frame;
+ rc = __get_paged_frame(gfn, &grant_frame, readonly, rd);
+ if ( rc != GNTST_okay )
+ goto unlock_out;
+ act->gfn = gfn;
is_sub_page = 0;
trans_page_off = 0;
trans_length = PAGE_SIZE;
@@ -1834,9 +1872,11 @@ __acquire_grant_for_copy(
}
else
{
- act->gfn = sha2->sub_page.frame;
- grant_frame = readonly ? gmfn_to_mfn(rd, act->gfn) :
- gfn_to_mfn_private(rd, act->gfn);
+ gfn = sha2->sub_page.frame;
+ rc = __get_paged_frame(gfn, &grant_frame, readonly, rd);
+ if ( rc != GNTST_okay )
+ goto unlock_out;
+ act->gfn = gfn;
is_sub_page = 1;
trans_page_off = sha2->sub_page.page_off;
trans_length = sha2->sub_page.length;
@@ -1932,16 +1972,9 @@ __gnttab_copy(
else
{
#ifdef CONFIG_X86
- p2m_type_t p2mt;
- s_frame = mfn_x(gfn_to_mfn(sd, op->source.u.gmfn, &p2mt));
- if ( !p2m_is_valid(p2mt) )
- s_frame = INVALID_MFN;
- if ( p2m_is_paging(p2mt) )
- {
- p2m_mem_paging_populate(sd, op->source.u.gmfn);
- rc = -ENOENT;
+ rc = __get_paged_frame(op->source.u.gmfn, &s_frame, 1, sd);
+ if ( rc != GNTST_okay )
goto error_out;
- }
#else
s_frame = gmfn_to_mfn(sd, op->source.u.gmfn);
#endif
@@ -1978,16 +2011,9 @@ __gnttab_copy(
else
{
#ifdef CONFIG_X86
- p2m_type_t p2mt;
- d_frame = mfn_x(gfn_to_mfn_unshare(dd, op->dest.u.gmfn, &p2mt,
1));
- if ( !p2m_is_valid(p2mt) )
- d_frame = INVALID_MFN;
- if ( p2m_is_paging(p2mt) )
- {
- p2m_mem_paging_populate(dd, op->dest.u.gmfn);
- rc = -ENOENT;
+ rc = __get_paged_frame(op->dest.u.gmfn, &d_frame, 0, dd);
+ if ( rc != GNTST_okay )
goto error_out;
- }
#else
d_frame = gmfn_to_mfn(dd, op->dest.u.gmfn);
#endif
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 15/18] xenpaging: handle dying guest in notify_via_xen_event_channel
Avoid crash of Xen if a xenpaging enabled guest crashes and there are still
memevents in flight, as reported here:
http://lists.xensource.com/archives/html/xen-devel/2010-08/msg00516.html
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Already-Acked-by: Patrick Colp <pjcolp@cs.ubc.ca>
Already-Acked-by: Keir Fraser <keir.fraser@citrix.com>
---
xen/common/event_channel.c | 6 ++++++
1 file changed, 6 insertions(+)
--- xen-4.0.1-testing.orig/xen/common/event_channel.c
+++ xen-4.0.1-testing/xen/common/event_channel.c
@@ -994,6 +994,12 @@ void notify_via_xen_event_channel(struct
spin_lock(&ld->event_lock);
+ if ( unlikely(ld->is_dying) )
+ {
+ spin_unlock(&ld->event_lock);
+ return;
+ }
+
ASSERT(port_is_valid(ld, lport));
lchn = evtchn_from_port(ld, lport);
ASSERT(lchn->consumer_is_xen);
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 16/18] xenpaging: prevent page-out of first 16MB
This is more a workaround than a bugfix:
Don''t page out first 16MB of memory.
When the BIOS does its initialization process and xenpaging removes pages,
crashes will occour due to lack of support of xenpaging.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
---
tools/xenpaging/policy_default.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
--- xen-4.0.1-testing.orig/tools/xenpaging/policy_default.c
+++ xen-4.0.1-testing/tools/xenpaging/policy_default.c
@@ -60,8 +60,9 @@ int policy_init(xenpaging_t *paging)
for ( i = 0; i < MRU_SIZE; i++ )
mru[i] = INVALID_MFN;
- /* Don''t page out page 0 */
- set_bit(0, bitmap);
+ /* Don''t page out first 16MB */
+ for ( i = 0; i < ((16*1024*1024)/4096); i++ )
+ set_bit(i, bitmap);
out:
return rc;
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 17/18] xenpaging: add dynamic startup delay for xenpaging
This is a debug helper. Since the xenpaging support is still fragile, run
xenpaging at different stages in the bootprocess. Different delays will trigger
more bugs. This implementation starts without delay for 5 reboots, then
increments the delay by 0.1 seconds It uses xenstore for presistant storage of
delay values
TODO: find the correct place to remove the xenstore directory when the guest is
shutdown or crashed
Signed-off-by: Olaf Hering <olaf@aepfle.de>
---
tools/python/xen/xend/image.py | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
--- xen-4.0.1-testing.orig/tools/python/xen/xend/image.py
+++ xen-4.0.1-testing/tools/python/xen/xend/image.py
@@ -123,6 +123,18 @@ class ImageHandler:
self.device_model =
vmConfig[''platform''].get(''device_model'')
self.xenpaging =
vmConfig[''platform''].get(''xenpaging'')
+ self.xenpaging_delay =
xstransact.Read("/local/domain/0/xenpaging/%s/xenpaging_delay" %
self.vm.info[''name_label''])
+ if self.xenpaging_delay == None:
+ log.warn("XXX creating /local/domain/0/xenpaging/%s %
self.vm.info[''name_label''])
+ xstransact.Mkdir("/local/domain/0/xenpaging/%s" %
self.vm.info[''name_label''])
+ xstransact.Store("/local/domain/0/xenpaging/%s" %
self.vm.info[''name_label''],
(''xenpaging_delay'', ''0.0''))
+ xstransact.Store("/local/domain/0/xenpaging/%s" %
self.vm.info[''name_label''],
(''xenpaging_delay_inc'', ''0.1''))
+ xstransact.Store("/local/domain/0/xenpaging/%s" %
self.vm.info[''name_label''],
(''xenpaging_delay_use'', ''5''))
+ xstransact.Store("/local/domain/0/xenpaging/%s" %
self.vm.info[''name_label''],
(''xenpaging_delay_used'', ''0''))
+ self.xenpaging_delay =
float(xstransact.Read("/local/domain/0/xenpaging/%s/xenpaging_delay" %
self.vm.info[''name_label'']))
+ self.xenpaging_delay_inc =
float(xstransact.Read("/local/domain/0/xenpaging/%s/xenpaging_delay_inc"
% self.vm.info[''name_label'']))
+ self.xenpaging_delay_use =
int(xstransact.Read("/local/domain/0/xenpaging/%s/xenpaging_delay_use"
% self.vm.info[''name_label'']))
+ self.xenpaging_delay_used =
int(xstransact.Read("/local/domain/0/xenpaging/%s/xenpaging_delay_used"
% self.vm.info[''name_label'']))
self.display =
vmConfig[''platform''].get(''display'')
self.xauthority =
vmConfig[''platform''].get(''xauthority'')
@@ -401,6 +413,17 @@ class ImageHandler:
return
if self.xenpaging_pid:
return
+ if self.xenpaging_delay_used < self.xenpaging_delay_use:
+ self.xenpaging_delay_used += 1
+ else:
+ self.xenpaging_delay_used = 0
+ self.xenpaging_delay += self.xenpaging_delay_inc
+ log.info("delay_used %s" % self.xenpaging_delay_used)
+ log.info("delay_use %s" % self.xenpaging_delay_use)
+ log.info("delay %s" % self.xenpaging_delay)
+ log.info("delay_inc %s" % self.xenpaging_delay_inc)
+ xstransact.Store("/local/domain/0/xenpaging/%s" %
self.vm.info[''name_label''],
(''xenpaging_delay'', self.xenpaging_delay))
+ xstransact.Store("/local/domain/0/xenpaging/%s" %
self.vm.info[''name_label''],
(''xenpaging_delay_used'', self.xenpaging_delay_used))
xenpaging_bin = auxbin.pathTo("xenpaging")
args = [xenpaging_bin]
args = args + ([ "%d" % self.vm.getDomid()])
@@ -430,6 +453,9 @@ class ImageHandler:
except:
log.warn("chdir %s failed" % xenpaging_dir)
try:
+ if self.xenpaging_delay != 0.0:
+ log.info("delaying xenpaging startup %s seconds
..." % self.xenpaging_delay)
+ time.sleep(self.xenpaging_delay)
log.info("starting %s" % args)
os.execve(xenpaging_bin, args, env)
except Exception, e:
@@ -449,6 +475,8 @@ class ImageHandler:
def destroyXenPaging(self):
if self.xenpaging is None:
return
+ # FIXME find correct place for guest shutdown or crash
+ #xstransact.Remove("/local/domain/0/xenpaging/%s" %
self.vm.info[''name_label''])
if self.xenpaging_pid:
try:
os.kill(self.xenpaging_pid, signal.SIGHUP)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-15 14:12 UTC
[Xen-devel] [PATCH 18/18] xenpaging: random debug statements and partial fixes
Debug and hacks.
Presented-by: Olaf Hering <olaf@aepfle.de>
---
tools/firmware/hvmloader/util.c | 5 ++++-
tools/libxc/xc_dom_x86.c | 2 ++
tools/libxc/xc_domain.c | 3 +++
tools/libxc/xc_offline_page.c | 1 +
tools/xenpaging/Makefile | 2 +-
tools/xenpaging/xenpaging.c | 26 +++++++++++++++++++++++++-
tools/xentrace/formats | 5 +++++
xen/arch/x86/debug.c | 2 ++
xen/arch/x86/hvm/svm/svm.c | 2 ++
xen/arch/x86/hvm/vmx/vmx.c | 4 ++++
xen/arch/x86/mm.c | 26 +++++++++++++++++++++++++-
xen/arch/x86/mm/hap/guest_walk.c | 2 ++
xen/arch/x86/mm/mem_event.c | 9 +++++++++
xen/arch/x86/mm/mem_sharing.c | 4 ++++
xen/arch/x86/mm/p2m.c | 7 +++++++
xen/arch/x86/mm/shadow/common.c | 4 ++++
xen/arch/x86/mm/shadow/multi.c | 2 ++
xen/common/grant_table.c | 27 ++++++++++++++++++++++-----
xen/common/memory.c | 3 +++
xen/include/asm-x86/p2m.h | 4 ++++
20 files changed, 131 insertions(+), 9 deletions(-)
--- xen-4.0.1-testing.orig/tools/firmware/hvmloader/util.c
+++ xen-4.0.1-testing/tools/firmware/hvmloader/util.c
@@ -329,7 +329,7 @@ void *mem_alloc(uint32_t size, uint32_t
mfn = reserve >> PAGE_SHIFT;
/* Try to allocate a brand new page in the reserved area. */
- if ( !over_allocated )
+ if ( 0 && !over_allocated )
{
uint8_t delay = 0;
xmr.domid = DOMID_SELF;
@@ -339,6 +339,7 @@ void *mem_alloc(uint32_t size, uint32_t
set_xen_guest_handle(xmr.extent_start, &mfn);
do {
rc = hypercall_memory_op(XENMEM_populate_physmap, &xmr);
+ printf("%s(%u): %x %x\n", __func__,__LINE__,rc,
delay);
if ( rc == 0 )
cpu_relax();
} while ( rc == 0 && ++delay );
@@ -364,6 +365,7 @@ void *mem_alloc(uint32_t size, uint32_t
xatp.gpfn = mfn;
do {
rc = hypercall_memory_op(XENMEM_add_to_physmap, &xatp);
+ printf("%s(%u): %x\n", __func__,__LINE__,rc);
if ( rc == -ENOENT )
cpu_relax();
} while ( rc == -ENOENT );
@@ -622,6 +624,7 @@ uint16_t get_cpu_mhz(void)
xatp.gpfn = (unsigned long)shared_info >> 12;
do {
rc = hypercall_memory_op(XENMEM_add_to_physmap, &xatp);
+ printf("%s(%u): %x\n", __func__,__LINE__,rc);
if ( rc == -ENOENT )
cpu_relax();
} while ( rc == -ENOENT );
--- xen-4.0.1-testing.orig/tools/libxc/xc_dom_x86.c
+++ xen-4.0.1-testing/tools/libxc/xc_dom_x86.c
@@ -812,6 +812,7 @@ int arch_setup_bootlate(struct xc_dom_im
rc = xc_memory_op(dom->guest_xc, XENMEM_add_to_physmap, &xatp);
if ( rc != 0 )
{
+ fprintf(stderr, "%s(%u) rc %x errno %x\n",
__func__,__LINE__,rc,errno);
xc_dom_panic(XC_INTERNAL_ERROR, "%s: mapping shared_info
failed "
"(pfn=0x%" PRIpfn ", rc=%d)\n",
__FUNCTION__, xatp.gpfn, rc);
@@ -828,6 +829,7 @@ int arch_setup_bootlate(struct xc_dom_im
rc = xc_memory_op(dom->guest_xc, XENMEM_add_to_physmap,
&xatp);
if ( rc != 0 )
{
+ fprintf(stderr, "%s(%u) rc %x errno %x\n",
__func__,__LINE__,rc,errno);
if ( (i > 0) && (errno == EINVAL) )
{
xc_dom_printf("%s: %d grant tables mapped\n",
__FUNCTION__,
--- xen-4.0.1-testing.orig/tools/libxc/xc_domain.c
+++ xen-4.0.1-testing/tools/libxc/xc_domain.c
@@ -564,7 +564,10 @@ static int do_xenmem_op_retry(int xc_han
}
if ( err )
+ {
+ fprintf(stderr, "%s: delay reset: %d err %x count %lx start
%lx delay %lu/%lu\n",__func__,cmd,err,count,start,delay,delay/666);
delay = 0;
+ }
start += err;
count -= err;
--- xen-4.0.1-testing.orig/tools/libxc/xc_offline_page.c
+++ xen-4.0.1-testing/tools/libxc/xc_offline_page.c
@@ -517,6 +517,7 @@ static int exchange_page(int xc_handle,
set_xen_guest_handle(exchange.out.extent_start, &out_mfn);
rc = xc_memory_op(xc_handle, XENMEM_exchange, &exchange);
+ fprintf(stderr, "%s(%u) rc %x errno %x\n",
__func__,__LINE__,rc,errno);
if (!rc)
*new_mfn = out_mfn;
--- xen-4.0.1-testing.orig/tools/xenpaging/Makefile
+++ xen-4.0.1-testing/tools/xenpaging/Makefile
@@ -4,7 +4,7 @@ include $(XEN_ROOT)/tools/Rules.mk
CFLAGS += -I $(XEN_XC)
CFLAGS += -I ./
CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenstore)
-LDFLAGS += $(LDFLAGS_libxenctrl) $(LDFLAGS_libxenstore)
+LDFLAGS += $(LDFLAGS_libxenctrl) $(LDFLAGS_libxenstore) -lrt
POLICY = default
--- xen-4.0.1-testing.orig/tools/xenpaging/xenpaging.c
+++ xen-4.0.1-testing/tools/xenpaging/xenpaging.c
@@ -21,6 +21,7 @@
#include <inttypes.h>
+#include <time.h>
#include <stdlib.h>
#include <signal.h>
#include <xc_private.h>
@@ -249,6 +250,7 @@ int xenpaging_teardown(xenpaging_t *pagi
rc = xc_mem_event_disable(paging->xc_handle,
paging->mem_event.domain_id);
if ( rc != 0 )
{
+ fprintf(stderr, "%s: rc %x errno %x\n", __func__, rc, errno);
ERROR("Error tearing down domain paging in xen");
}
@@ -370,6 +372,7 @@ int xenpaging_evict_page(xenpaging_t *pa
goto out;
}
+ fprintf(stderr, "%s(%u) > gfn %lx pageslot %d\n", __func__,
__LINE__, victim->gfn, i);
/* Notify policy of page being paged out */
policy_notify_paged_out(paging->mem_event.domain_id, victim->gfn);
@@ -406,6 +409,8 @@ static int xenpaging_populate_page(
void *page;
int ret;
+ _gfn = *gfn;
+ fprintf(stderr, "%s(%u) < gfn %lx pageslot %d\n", __func__,
__LINE__, _gfn, i);
/* Tell Xen to allocate a page for the domain */
ret = xc_mem_paging_prep(paging->xc_handle,
paging->mem_event.domain_id,
*gfn);
@@ -441,6 +446,17 @@ static int xenpaging_populate_page(
return ret;
}
+static void time_diff(struct timespec *start, struct timespec *end, struct
timespec *diff)
+{
+ if ((end->tv_nsec - start->tv_nsec) < 0) {
+ diff->tv_sec = end->tv_sec - start->tv_sec - 1;
+ diff->tv_nsec = 1000000000 + end->tv_nsec - start->tv_nsec;
+ } else {
+ diff->tv_sec = end->tv_sec - start->tv_sec;
+ diff->tv_nsec = end->tv_nsec - start->tv_nsec;
+ }
+}
+
static int evict_victim(xenpaging_t *paging, domid_t domain_id,
xenpaging_victim_t *victim, int fd, int i)
{
@@ -501,6 +517,7 @@ int main(int argc, char *argv[])
mode_t open_mode = S_IRUSR | S_IRGRP | S_IROTH | S_IWUSR | S_IWGRP |
S_IWOTH;
char filename[80];
int fd;
+ struct timespec ce, cl, cd;
if ( argc != 3 )
{
@@ -544,6 +561,8 @@ int main(int argc, char *argv[])
/* Evict pages */
memset(victims, 0, sizeof(xenpaging_victim_t) * num_pages);
+ if (clock_gettime(CLOCK_MONOTONIC, &ce))
+ perror("clock_gettime");
for ( i = 0; i < num_pages; i++ )
{
rc = evict_victim(paging, domain_id, &victims[i], fd, i);
@@ -554,7 +573,10 @@ int main(int argc, char *argv[])
if ( i % 100 == 0 )
DPRINTF("%d pages evicted\n", i);
}
-
+ if (clock_gettime(CLOCK_MONOTONIC, &cl))
+ perror("clock_gettime");
+ time_diff(&ce, &cl, &cd);
+ DPRINTF("%s: c %d.%09d\n", __func__, (int)cd.tv_sec,
(int)cd.tv_nsec);
DPRINTF("pages evicted\n");
/* Swap pages in and out */
@@ -625,8 +647,10 @@ int main(int argc, char *argv[])
else
{
DPRINTF("page already populated (domain = %d; vcpu =
%d;"
+ " p2mt = %x;"
" gfn = %"PRIx64"; paused =
%"PRId64")\n",
paging->mem_event.domain_id, req.vcpu_id,
+ req.p2mt,
req.gfn, req.flags & MEM_EVENT_FLAG_VCPU_PAUSED);
/* Tell Xen to resume the vcpu */
--- xen-4.0.1-testing.orig/tools/xentrace/formats
+++ xen-4.0.1-testing/tools/xentrace/formats
@@ -5,6 +5,7 @@
0x0001f003 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) cpu_change 0x%(1)08x
0x0001f004 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) trace_irq [ vector = %(1)d,
count = %(2)d, tot_cycles = 0x%(3)08x, max_cycles = 0x%(4)08x ]
+0x00021002 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) continue_running [ dom:vcpu
= 0x%(1)08x ]
0x00021011 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) running_to_runnable [ dom:vcpu
= 0x%(1)08x ]
0x00021021 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) running_to_blocked [ dom:vcpu
= 0x%(1)08x ]
0x00021031 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) running_to_offline [ dom:vcpu
= 0x%(1)08x ]
@@ -70,10 +71,14 @@
0x00082018 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) CLTS
0x00082019 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) LMSW [ value = 0x%(1)08x
]
0x00082119 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) LMSW [ value =
0x%(2)08x%(1)08x ]
+0x00082020 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) INTR_WINDOW [ value = 0x%(1)08x
]
+0x00082021 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) NPF [ gpa =
0x%(2)08x%(1)08x mfn = 0x%(4)08x%(3)08x qual = 0x%(5)04x p2mt = 0x%(6)04x ]
0x0010f001 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) page_grant_map [ domid =
%(1)d ]
0x0010f002 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) page_grant_unmap [ domid =
%(1)d ]
0x0010f003 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) page_grant_transfer [ domid =
%(1)d ]
+0x0010f004 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) p2m_set_entry [ gfn
0x%(2)08x%(1)08x mfn 0x%(4)08x%(3)08x pmt 0x%(5)08x domain:order 0x%(6)08x ]
+0x0010f005 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) decrease_reservation [ gfn
0x%(2)08x%(1)08x domain:order 0x%(3)08x ]
0x0020f001 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) hypercall [ eip = 0x%(1)08x,
eax = 0x%(2)08x ]
0x0020f101 CPU%(cpu)d %(tsc)d (+%(reltsc)8d) hypercall [ rip =
0x%(2)08x%(1)08x, eax = 0x%(3)08x ]
--- xen-4.0.1-testing.orig/xen/arch/x86/debug.c
+++ xen-4.0.1-testing/xen/arch/x86/debug.c
@@ -62,6 +62,8 @@ dbg_hvm_va2mfn(dbgva_t vaddr, struct dom
}
mfn = mfn_x(gfn_to_mfn(dp, gfn, &gfntype));
+if ( p2m_is_paging(gfntype) )
+printk("%s: gfn %lx p2m %x\n",__func__,gfn,gfntype);
if ( p2m_is_readonly(gfntype) && toaddr )
{
DBGP2("kdb:p2m_is_readonly: gfntype:%x\n", gfntype);
--- xen-4.0.1-testing.orig/xen/arch/x86/hvm/svm/svm.c
+++ xen-4.0.1-testing/xen/arch/x86/hvm/svm/svm.c
@@ -232,6 +232,8 @@ static int svm_vmcb_restore(struct vcpu
if ( c->cr0 & X86_CR0_PG )
{
mfn = mfn_x(gfn_to_mfn(v->domain, c->cr3 >> PAGE_SHIFT,
&p2mt));
+if ( p2m_is_paging(p2mt) )
+printk("%s: gfn %"PRIx64" p2m
%x\n",__func__,c->cr3,p2mt);
if ( !p2m_is_ram(p2mt) || !get_page(mfn_to_page(mfn), v->domain)
)
{
gdprintk(XENLOG_ERR, "Invalid CR3
value=0x%"PRIx64"\n",
--- xen-4.0.1-testing.orig/xen/arch/x86/hvm/vmx/vmx.c
+++ xen-4.0.1-testing/xen/arch/x86/hvm/vmx/vmx.c
@@ -496,6 +496,8 @@ static int vmx_restore_cr0_cr3(
if ( cr0 & X86_CR0_PG )
{
mfn = mfn_x(gfn_to_mfn(v->domain, cr3 >> PAGE_SHIFT,
&p2mt));
+if ( p2m_is_paging(p2mt) )
+printk("%s: gfn %lx p2m %x\n",__func__,cr3,p2mt);
if ( !p2m_is_ram(p2mt) || !get_page(mfn_to_page(mfn), v->domain)
)
{
gdprintk(XENLOG_ERR, "Invalid CR3 value=0x%lx\n",
cr3);
@@ -1012,6 +1014,8 @@ static void vmx_load_pdptrs(struct vcpu
goto crash;
mfn = mfn_x(gfn_to_mfn(v->domain, cr3 >> PAGE_SHIFT, &p2mt));
+if ( p2m_is_paging(p2mt) )
+printk("%s: gfn %lx p2m %x\n",__func__,cr3,p2mt);
if ( !p2m_is_ram(p2mt) )
goto crash;
--- xen-4.0.1-testing.orig/xen/arch/x86/mm.c
+++ xen-4.0.1-testing/xen/arch/x86/mm.c
@@ -3148,7 +3148,8 @@ int do_mmu_update(
rc = -ENOENT;
break;
- }
+ } else if ( p2m_is_paging(p2mt) )
+ MEM_LOG("gfn %lx p2m %x", gmfn, p2mt);
if ( unlikely(!get_page_from_pagenr(mfn, pt_owner)) )
{
@@ -3224,6 +3225,8 @@ int do_mmu_update(
rc = -ENOENT;
break;
}
+ else if ( p2m_is_paging(l2e_p2mt) )
+ MEM_LOG("gfn %lx p2m %x", l2e_get_pfn(l2e),
l2e_p2mt);
else if ( p2m_ram_shared == l2e_p2mt )
{
MEM_LOG("Unexpected attempt to map shared
page.\n");
@@ -3254,6 +3257,8 @@ int do_mmu_update(
rc = -ENOENT;
break;
}
+ else if ( p2m_is_paging(l3e_p2mt) )
+ MEM_LOG("gfn %lx p2m %x", l3e_get_pfn(l3e),
l3e_p2mt);
else if ( p2m_ram_shared == l3e_p2mt )
{
MEM_LOG("Unexpected attempt to map shared
page.\n");
@@ -3285,6 +3290,8 @@ int do_mmu_update(
rc = -ENOENT;
break;
}
+ else if ( p2m_is_paging(l4e_p2mt) )
+ MEM_LOG("gfn %lx p2m %x", l4e_get_pfn(l4e),
l4e_p2mt);
else if ( p2m_ram_shared == l4e_p2mt )
{
MEM_LOG("Unexpected attempt to map shared
page.\n");
@@ -4322,6 +4329,7 @@ long arch_memory_op(int op, XEN_GUEST_HA
tmp_mfn = mfn_x(gfn_to_mfn_unshare(d, xatp.idx, &p2mt, 0));
if ( unlikely(p2m_is_paging(p2mt)) )
{
+printk("%s(%u) gfn %lx p2m %x\n",__func__,__LINE__,xatp.idx, p2mt);
if ( p2m_is_paged(p2mt) )
p2m_mem_paging_populate(d, xatp.idx);
rcu_unlock_domain(d);
@@ -4337,6 +4345,21 @@ long arch_memory_op(int op, XEN_GUEST_HA
break;
mfn = tmp_mfn;
page = mfn_to_page(mfn);
+
+ gpfn = get_gpfn_from_mfn(tmp_mfn);
+ ASSERT( gpfn != SHARED_M2P_ENTRY );
+ gfn_to_mfn(d, gpfn, &p2mt);
+ if ( unlikely(p2m_is_paging(p2mt)) )
+ {
+printk("%s(%u) gfn %lx p2m %x\n",__func__,__LINE__, gpfn, p2mt);
+ if ( page )
+ put_page(page);
+ if ( p2m_is_paged(p2mt) )
+ p2m_mem_paging_populate(d, gpfn);
+ rcu_unlock_domain(d);
+ return -ENOENT;
+ }
+
break;
}
default:
@@ -4369,6 +4392,7 @@ long arch_memory_op(int op, XEN_GUEST_HA
rc = guest_remove_page(d, xatp.gpfn);
if ( rc == -ENOENT )
{
+ MEM_LOG("gfn %lx", xatp.gpfn);
domain_unlock(d);
rcu_unlock_domain(d);
return rc;
--- xen-4.0.1-testing.orig/xen/arch/x86/mm/hap/guest_walk.c
+++ xen-4.0.1-testing/xen/arch/x86/mm/hap/guest_walk.c
@@ -49,6 +49,7 @@ unsigned long hap_gva_to_gfn(GUEST_PAGIN
top_mfn = gfn_to_mfn_unshare(v->domain, cr3 >> PAGE_SHIFT,
&p2mt, 0);
if ( p2m_is_paging(p2mt) )
{
+printk("%s: gfn %lx p2m %x\n",__func__,cr3 >> PAGE_SHIFT,p2mt);
if ( p2m_is_paged(p2mt) )
p2m_mem_paging_populate(v->domain, cr3 >> PAGE_SHIFT);
@@ -82,6 +83,7 @@ unsigned long hap_gva_to_gfn(GUEST_PAGIN
gfn_to_mfn_unshare(v->domain, gfn_x(gfn), &p2mt, 0);
if ( p2m_is_paging(p2mt) )
{
+printk("%s: gfn %lx p2m %x\n",__func__,gfn_x(gfn),p2mt);
if ( p2m_is_paged(p2mt) )
p2m_mem_paging_populate(v->domain, gfn_x(gfn));
--- xen-4.0.1-testing.orig/xen/arch/x86/mm/mem_event.c
+++ xen-4.0.1-testing/xen/arch/x86/mm/mem_event.c
@@ -68,6 +68,7 @@ int mem_event_enable(struct domain *d, m
d->mem_event.paused = 0;
d->mem_event.enabled = 1;
+printk("%s %u\n",__func__, d->domain_id);
return 0;
err_shared:
@@ -82,6 +83,7 @@ int mem_event_enable(struct domain *d, m
int mem_event_disable(struct domain *d)
{
+printk("%s %u\n",__func__, d->domain_id);
d->mem_event.enabled = 0;
d->mem_event.paused = 0;
@@ -168,6 +170,9 @@ int mem_event_check_ring(struct domain *
mem_event_ring_lock(d);
free_requests = RING_FREE_REQUESTS(&d->mem_event.front_ring);
+ if ( free_requests < 3 )
+ gdprintk(XENLOG_INFO, "frq %d\n", free_requests);
+ WARN_ON(free_requests == 0);
ring_full = free_requests < MEM_EVENT_RING_THRESHOLD;
if ( (current->domain->domain_id == d->domain_id) &&
ring_full )
@@ -243,6 +248,8 @@ int mem_event_domctl(struct domain *d, x
guest_get_eff_l1e(v, ring_addr, &l1e);
gfn = l1e_get_pfn(l1e);
ring_mfn = gfn_to_mfn(dom_mem_event, gfn, &p2mt);
+if ( p2m_is_paging(p2mt) )
+printk("%s: gfn %lx p2m %x\n",__func__,gfn,p2mt);
rc = -EINVAL;
if ( unlikely(!mfn_valid(mfn_x(ring_mfn))) )
@@ -252,6 +259,8 @@ int mem_event_domctl(struct domain *d, x
guest_get_eff_l1e(v, shared_addr, &l1e);
gfn = l1e_get_pfn(l1e);
shared_mfn = gfn_to_mfn(dom_mem_event, gfn, &p2mt);
+if ( p2m_is_paging(p2mt) )
+printk("%s: gfn %lx p2m %x\n",__func__,gfn,p2mt);
rc = -EINVAL;
if ( unlikely(!mfn_valid(mfn_x(shared_mfn))) )
--- xen-4.0.1-testing.orig/xen/arch/x86/mm/mem_sharing.c
+++ xen-4.0.1-testing/xen/arch/x86/mm/mem_sharing.c
@@ -381,6 +381,8 @@ int mem_sharing_debug_gfn(struct domain
struct page_info *page;
mfn = gfn_to_mfn(d, gfn, &p2mt);
+if ( p2m_is_paging(p2mt) )
+printk("%s: gfn %lx p2m %x\n",__func__,gfn,p2mt);
page = mfn_to_page(mfn);
printk("Debug for domain=%d, gfn=%lx, ",
@@ -636,6 +638,8 @@ int mem_sharing_unshare_page(struct doma
struct list_head *le;
mfn = gfn_to_mfn(d, gfn, &p2mt);
+if ( p2m_is_paging(p2mt) )
+printk("%s: gfn %lx p2m %x\n",__func__,gfn,p2mt);
page = mfn_to_page(mfn);
handle = page->shr_handle;
--- xen-4.0.1-testing.orig/xen/arch/x86/mm/p2m.c
+++ xen-4.0.1-testing/xen/arch/x86/mm/p2m.c
@@ -2005,6 +2005,8 @@ p2m_remove_page(struct domain *d, unsign
mfn_return = d->arch.p2m->get_entry(d, gfn + i, &t,
p2m_query);
if ( !p2m_is_grant(t) )
set_gpfn_from_mfn(mfn+i, INVALID_M2P_ENTRY);
+ if (!( !p2m_is_valid(t) || mfn + i == mfn_x(mfn_return) ))
+ printk("%s(%u) i %lu t %x gfn %lx mfn %lx ret %lx %lu
%d\n",__func__,__LINE__,i,t,gfn,mfn,mfn_x(mfn_return), p2m_is_valid(t), mfn
+ i == mfn_x(mfn_return));
ASSERT( !p2m_is_valid(t) || mfn + i == mfn_x(mfn_return) );
}
set_p2m_entry(d, gfn, _mfn(INVALID_MFN), page_order, p2m_invalid);
@@ -2115,6 +2117,8 @@ guest_physmap_add_entry(struct domain *d
int pod_count = 0;
int rc = 0;
+ if ( p2m_is_paging(t) )
+ gdprintk(XENLOG_ERR, "d %u gfn %lx mfn %lx o %x p2m %x\n",
d->domain_id, gfn, mfn, page_order, t);
if ( !paging_mode_translate(d) )
{
if ( need_iommu(d) && t == p2m_ram_rw )
@@ -2147,6 +2151,8 @@ guest_physmap_add_entry(struct domain *d
for ( i = 0; i < (1UL << page_order); i++ )
{
omfn = gfn_to_mfn_query(d, gfn + i, &ot);
+ if ( p2m_is_paging(ot) )
+ gdprintk(XENLOG_ERR, "d %u gfn %lx omfn %lx o %x i %lx p2m
%x\n", d->domain_id, gfn, mfn_x(omfn), page_order, i, ot);
if ( p2m_is_grant(ot) )
{
/* Really shouldn''t be unmapping grant maps this way */
@@ -2188,6 +2194,7 @@ guest_physmap_add_entry(struct domain *d
omfn = gfn_to_mfn_query(d, ogfn, &ot);
if ( unlikely(p2m_is_paging(ot)) )
{
+ gdprintk(XENLOG_ERR, "d %u gfn %lx omfn %lx o %x i %lx p2m
%x\n", d->domain_id, ogfn, mfn_x(omfn), page_order, i, ot);
p2m_unlock(d->arch.p2m);
if ( p2m_is_paged(ot) )
p2m_mem_paging_populate(d, ogfn);
--- xen-4.0.1-testing.orig/xen/arch/x86/mm/shadow/common.c
+++ xen-4.0.1-testing/xen/arch/x86/mm/shadow/common.c
@@ -3720,6 +3720,8 @@ int shadow_track_dirty_vram(struct domai
int dirty = 0;
paddr_t sl1ma = dirty_vram->sl1ma[i];
+if ( p2m_is_paging(t) )
+printk("%s: gfn %lx p2m %x\n",__func__,begin_pfn +i,t);
if (mfn_x(mfn) == INVALID_MFN)
{
dirty = 1;
@@ -3801,6 +3803,8 @@ int shadow_track_dirty_vram(struct domai
* write access */
for ( i = begin_pfn; i < end_pfn; i++ ) {
mfn_t mfn = gfn_to_mfn(d, i, &t);
+if ( p2m_is_paging(t) )
+printk("%s: gfn %lx p2m %x\n",__func__,i,t);
if (mfn_x(mfn) != INVALID_MFN)
flush_tlb |= sh_remove_write_access(d->vcpu[0], mfn,
1, 0);
}
--- xen-4.0.1-testing.orig/xen/arch/x86/mm/shadow/multi.c
+++ xen-4.0.1-testing/xen/arch/x86/mm/shadow/multi.c
@@ -4795,6 +4795,8 @@ static mfn_t emulate_gva_to_mfn(struct v
mfn = gfn_to_mfn_query(v->domain, _gfn(gfn), &p2mt);
else
mfn = gfn_to_mfn(v->domain, _gfn(gfn), &p2mt);
+if ( p2m_is_paging(p2mt) )
+printk("%s: gfn %lx p2m %x\n",__func__, gfn,p2mt);
if ( p2m_is_readonly(p2mt) )
return _mfn(READONLY_GFN);
--- xen-4.0.1-testing.orig/xen/common/grant_table.c
+++ xen-4.0.1-testing/xen/common/grant_table.c
@@ -1447,6 +1447,7 @@ gnttab_transfer(
struct domain *d = current->domain;
struct domain *e;
struct page_info *page;
+ int rc;
int i;
struct gnttab_transfer gop;
unsigned long mfn;
@@ -1465,7 +1466,12 @@ gnttab_transfer(
return -EFAULT;
}
- mfn = gfn_to_mfn_private(d, gop.mfn);
+ rc = __get_paged_frame(gop.mfn, &mfn, 0, d);
+ if ( rc == GNTST_eagain )
+ {
+ gop.status = GNTST_eagain;
+ goto copyback;
+ }
/* Check the passed page frame for basic validity. */
if ( unlikely(!mfn_valid(mfn)) )
@@ -1580,28 +1586,39 @@ gnttab_transfer(
/* Tell the guest about its new page frame. */
spin_lock(&e->grant_table->lock);
+ gop.status = GNTST_okay;
+
if ( e->grant_table->gt_version == 1 )
{
grant_entry_v1_t *sha = &shared_entry_v1(e->grant_table,
gop.ref);
- guest_physmap_add_page(e, sha->frame, mfn, 0);
+ rc = guest_physmap_add_page(e, sha->frame, mfn, 0);
+ if ( rc == -ENOENT )
+ {
+ gop.status = GNTST_eagain;
+ goto unlock_granttable_and_copyback;
+ }
sha->frame = mfn;
}
else
{
grant_entry_v2_t *sha = &shared_entry_v2(e->grant_table,
gop.ref);
- guest_physmap_add_page(e, sha->full_page.frame, mfn, 0);
+ rc = guest_physmap_add_page(e, sha->full_page.frame, mfn, 0);
+ if ( rc == -ENOENT )
+ {
+ gop.status = GNTST_eagain;
+ goto unlock_granttable_and_copyback;
+ }
sha->full_page.frame = mfn;
}
wmb();
shared_entry_header(e->grant_table, gop.ref)->flags |
GTF_transfer_completed;
+ unlock_granttable_and_copyback:
spin_unlock(&e->grant_table->lock);
rcu_unlock_domain(e);
- gop.status = GNTST_okay;
-
copyback:
if ( unlikely(__copy_to_guest_offset(uop, i, &gop, 1)) )
{
--- xen-4.0.1-testing.orig/xen/common/memory.c
+++ xen-4.0.1-testing/xen/common/memory.c
@@ -138,6 +138,7 @@ static void populate_physmap(struct memo
rc = guest_physmap_add_page(d, gpfn, mfn, a->extent_order);
if ( rc != 0 )
{
+ gdprintk(XENLOG_INFO, "%s: rc %x\n", __func__, rc);
free_domheap_pages(page, a->extent_order);
goto out;
}
@@ -170,6 +171,7 @@ int guest_remove_page(struct domain *d,
mfn = mfn_x(gfn_to_mfn(d, gmfn, &p2mt));
if ( unlikely(p2m_is_paging(p2mt)) )
{
+ gdprintk(XENLOG_INFO, "d %u gfn %lx mfn %lx p2m %x\n",
d->domain_id, gmfn, mfn, p2mt);
if ( p2m_is_paged(p2mt) )
p2m_mem_paging_populate(d, gmfn);
return -ENOENT;
@@ -374,6 +376,7 @@ static long memory_exchange(XEN_GUEST_HA
mfn = mfn_x(gfn_to_mfn_unshare(d, gmfn + k, &p2mt, 0));
if ( p2m_is_paging(p2mt) )
{
+ printk("%s: gfn %lx p2m
%x\n",__func__,gmfn+k,p2mt);
if ( p2m_is_paged(p2mt) )
p2m_mem_paging_populate(d, gmfn);
rc = -ENOENT;
--- xen-4.0.1-testing.orig/xen/include/asm-x86/p2m.h
+++ xen-4.0.1-testing/xen/include/asm-x86/p2m.h
@@ -329,6 +329,8 @@ static inline mfn_t gfn_to_mfn_unshare(s
return mfn;
}
mfn = gfn_to_mfn(d, gfn, p2mt);
+if ( p2m_is_paging(*p2mt) )
+printk("%s: gfn %lx p2m %x\n",__func__, gfn,*p2mt);
}
#endif
@@ -342,6 +344,8 @@ static inline unsigned long gmfn_to_mfn(
mfn_t mfn;
p2m_type_t t;
mfn = gfn_to_mfn(d, gpfn, &t);
+if ( p2m_is_paging(t) )
+printk("%s: gfn %lx p2m %x\n",__func__, gpfn,t);
if ( p2m_is_valid(t) )
return mfn_x(mfn);
return INVALID_MFN;
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Dan Magenheimer
2010-Oct-15 15:05 UTC
RE: [Xen-devel] [PATCH 00/18] xenpaging changes for 4.0
Hi Olaf -- Since you have obviously become an expert user of xenpaging, any chance you could write brief how-to documentation for it and maybe add a file to xen.hg/docs/misc? Thanks, Dan> -----Original Message----- > From: Olaf Hering [mailto:olaf@aepfle.de] > Sent: Friday, October 15, 2010 8:12 AM > To: xen-devel@lists.xensource.com > Subject: [Xen-devel] [PATCH 00/18] xenpaging changes for 4.0 > > > Here are some changes for xenpaging in the 4.0 branch. > > Its just for review and comments. All changes are also needed for > xen-unstable, I will prepare a patch queue also for this branch. > > There are still issues with xenpaging. > My testsetup is a SLES11 SP1 guest with runlevel 6 as default runlevel > to > trigger automatic reboot. After a few iterations, there will be MMIO > emulation > errors. This happens with a startup delay of 15.0 seconds. > > Another issue: > If the startup delay is 0.0, and over_allocated in hvmloader mem_alloc > is > forced to be active, the ASSERT in p2m_remove_page() triggers. This is > because > the passed mfn and the mfn returned by ->get_entry() do not match. So > far I > have no idea where the appearently missing set_gpfn_from_mfn() should > go to. > Normally over_allocated is not set because the XENMEM_populate_physmap > call > does not fail. > > Olaf > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Olaf, Thanks for all your work on this! These patches look pretty good to me, with a few caveats: - Is it possible to avoid having quite so many paths return ENOENT? In particular, there seemed to be paths where _removing_ a p2m entry that''s paged out caused an error. Would it be possible to have some of those cases fixed up from the paging code instead? - The usual way to get patches into the 4.0 tree is to submit them againts xen-unstable and then ask for specific patches to be backported; I see you''re already going that way with most of these. - I imagine the tools maintainers will encourage you to do your tools patches againsl xl/libxl as well. :) Cheers, Tim. At 15:12 +0100 on 15 Oct (1287155522), Olaf Hering wrote:> Here are some changes for xenpaging in the 4.0 branch. > > Its just for review and comments. All changes are also needed for > xen-unstable, I will prepare a patch queue also for this branch. > > There are still issues with xenpaging. > My testsetup is a SLES11 SP1 guest with runlevel 6 as default runlevel to > trigger automatic reboot. After a few iterations, there will be MMIO emulation > errors. This happens with a startup delay of 15.0 seconds. > > Another issue: > If the startup delay is 0.0, and over_allocated in hvmloader mem_alloc is > forced to be active, the ASSERT in p2m_remove_page() triggers. This is because > the passed mfn and the mfn returned by ->get_entry() do not match. So far I > have no idea where the appearently missing set_gpfn_from_mfn() should go to. > Normally over_allocated is not set because the XENMEM_populate_physmap call > does not fail. > > Olaf > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel-- Tim Deegan <Tim.Deegan@citrix.com> Principal Software Engineer, XenServer Engineering Citrix Systems UK Ltd. (Company #02937203, SL9 0BG) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, Oct 15, Dan Magenheimer wrote:> Hi Olaf -- > > Since you have obviously become an expert user of xenpaging, > any chance you could write brief how-to documentation for it > and maybe add a file to xen.hg/docs/misc?Dan, yes I will write something, which I will send with the next round of patches for xen-unstable. Olaf _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, Oct 18, Tim Deegan wrote:> These patches look pretty good to me, with a few caveats: > - Is it possible to avoid having quite so many paths return ENOENT? > In particular, there seemed to be paths where _removing_ a p2m entry > that''s paged out caused an error. Would it be possible to have some of > those cases fixed up from the paging code instead?I''m not sure if thats possible. Thats currently beyond my knowledge, sorry. Any pointer for how to do that are welcome.> - The usual way to get patches into the 4.0 tree is to submit them > againts xen-unstable and then ask for specific patches to be > backported; I see you''re already going that way with most of these.Yes, just wanted to send my current state and all of them are also needed for xen-unstable> - I imagine the tools maintainers will encourage you to do your tools > patches againsl xl/libxl as well. :)Sure, that will be done as well. Olaf _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2010-Oct-18 16:54 UTC
Re: [Xen-devel] [PATCH 06/18] xenpaging: allow only one xenpaging binary per guest
Olaf Hering writes ("[Xen-devel] [PATCH 06/18] xenpaging: allow only one
xenpaging binary per guest"):> - ERROR("Error initialising shared page");
> + switch ( errno ) {
> + case EBUSY:
> + ERROR("xenpaging is (or was) active on this
domain");
> + break;
> + case ENODEV:
> + ERROR("EPT not supported for this guest");
> + break;
> + default:
> + ERROR("Error initialising shared page");
> + break;
Surely this should print the actual errno value.
> + /* Only one xenpaging at a time. If xenpaging crashed,
> + * the cache is in an undefined state and so is the guest
> + */
> + rc = -EBUSY;
> + if ( d->mem_event.enabled )
> + break;
Is there any way to recover from this or is the guest just hosed ?
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Ian Jackson
2010-Oct-18 16:56 UTC
Re: [Xen-devel] [PATCH 18/18] xenpaging: random debug statements and partial fixes
Olaf Hering writes ("[Xen-devel] [PATCH 18/18] xenpaging: random debug
statements and partial fixes"):> Debug and hacks.
Are you sure this patch is intended to be in this series ?
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Tim Deegan writes ("Re: [Xen-devel] [PATCH 00/18] xenpaging changes for
4.0"):> - The usual way to get patches into the 4.0 tree is to submit them
> againts xen-unstable and then ask for specific patches to be
> backported; I see you''re already going that way with most of
these.
I would certainly like to try out these changes in xen-unstable
first.
> - I imagine the tools maintainers will encourage you to do your tools
> patches againsl xl/libxl as well. :)
Yes, in xen-unstable at least.
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-18 17:04 UTC
Re: [Xen-devel] [PATCH 06/18] xenpaging: allow only one xenpaging binary per guest
On Mon, Oct 18, Ian Jackson wrote:> Olaf Hering writes ("[Xen-devel] [PATCH 06/18] xenpaging: allow only one xenpaging binary per guest"): > > - ERROR("Error initialising shared page"); > > + switch ( errno ) { > > + case EBUSY: > > + ERROR("xenpaging is (or was) active on this domain"); > > + break; > > + case ENODEV: > > + ERROR("EPT not supported for this guest"); > > + break; > > + default: > > + ERROR("Error initialising shared page"); > > + break; > > Surely this should print the actual errno value.Ok, can be done.> > + /* Only one xenpaging at a time. If xenpaging crashed, > > + * the cache is in an undefined state and so is the guest > > + */ > > + rc = -EBUSY; > > + if ( d->mem_event.enabled ) > > + break; > > Is there any way to recover from this or is the guest just hosed ?If xenpaging dies for some reason, the bitmap which maps the "pageslot" in the pagefile to the gfn in the guest is gone. There is no way to recover from this situation. If the paging state should be persistant, then the maintained bitmaps have to be kept in memory and each page write has to be fsynced in some way. Some other way has to be found to prevent two xenpaging runs per domain_id. Olaf _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2010-Oct-18 17:05 UTC
Re: [Xen-devel] [PATCH 18/18] xenpaging: random debug statements and partial fixes
On Mon, Oct 18, Ian Jackson wrote:> Olaf Hering writes ("[Xen-devel] [PATCH 18/18] xenpaging: random debug statements and partial fixes"): > > Debug and hacks. > > Are you sure this patch is intended to be in this series ?Not really, its just my playground. Olaf _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2010-Oct-19 10:17 UTC
Re: [Xen-devel] [PATCH 06/18] xenpaging: allow only one xenpaging binary per guest
Olaf Hering writes ("Re: [Xen-devel] [PATCH 06/18] xenpaging: allow only
one xenpaging binary per guest"):> On Mon, Oct 18, Ian Jackson wrote:
> > Is there any way to recover from this or is the guest just hosed ?
>
> If xenpaging dies for some reason, the bitmap which maps the
"pageslot"
> in the pagefile to the gfn in the guest is gone. There is no way to
> recover from this situation. If the paging state should be persistant,
> then the maintained bitmaps have to be kept in memory and each page write
> has to be fsynced in some way.
> Some other way has to be found to prevent two xenpaging runs per
> domain_id.
OK, I just thought I''d ask.
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
On Fri, Oct 15, Dan Magenheimer wrote:> Hi Olaf -- > > Since you have obviously become an expert user of xenpaging, > any chance you could write brief how-to documentation for it > and maybe add a file to xen.hg/docs/misc?This is a first draft. Subject: xenpaging: (sparse) documenation Write up some sparse documentation about xenpaging usage. Signed-off-by: Olaf Hering <olaf@aepfle.de> --- docs/misc/xenpaging.txt | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) --- /dev/null +++ xen-4.0.1-testing/docs/misc/xenpaging.txt @@ -0,0 +1,40 @@ +Warning: + +The xenpaging code is new and not fully debugged. +Usage of xenpaging can crash Xen or cause severe data corruption in the +guest memory and its filesystems! + +Description: + +xenpaging writes memory pages of a given guest to a file and moves the +page back to the pool of available memory. Once the guests wants to +access the paged-out memory, the page is read from disk and placed into +memory. This allows the sum of all running guests to use more memory +than physically available on the host. + +Usage: + +Once the guest is running, run xenpaging with the guest_id and the +number of pages to page-out: + + mkdir /var/lib/xen/xenpaging + chdir /var/lib/xen/xenpaging + xenpaging <guest_id> <number_of_pages> + +To obtain the guest_id, run ''xm list''. +xenpaging will write the pagefile to the current directory. +Example with 128MB pagefile on guest 1: + + xenpaging 1 32768 + +Caution: stopping xenpaging manually will cause the guest to stall or +crash because the paged-out memory is not written back into the guest! + + + +Todo: +- implement stopping of xenpaging +- implement/test live migration + + +# vim: tw=72 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
At 14:06 +0100 on 18 Oct (1287410805), Olaf Hering wrote:> On Mon, Oct 18, Tim Deegan wrote: > > > These patches look pretty good to me, with a few caveats: > > - Is it possible to avoid having quite so many paths return ENOENT? > > In particular, there seemed to be paths where _removing_ a p2m entry > > that''s paged out caused an error. Would it be possible to have some of > > those cases fixed up from the paging code instead? > > I''m not sure if thats possible.OK, having had a look I''m not sure it''s possible either. :) Your current approach should be OK. Cheers, Tim. -- Tim Deegan <Tim.Deegan@citrix.com> Principal Software Engineer, XenServer Engineering Citrix Systems UK Ltd. (Company #02937203, SL9 0BG) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel