Olaf Hering
2011-Sep-15 06:16 UTC
[Xen-devel] [PATCH 0 of 9] xenpaging fixes for xen-unstable
The following series implements growing and shrinking of the paging file. A xenstore watch is used the retreive the new number of pages to page-out. This number is compared to the current amount of paged-out pages and, depending on the difference, new pages are written to disk or written back into the guest. Please review and apply. Olaf tools/xenpaging/Makefile | 4 tools/xenpaging/file_ops.c | 1 tools/xenpaging/policy_default.c | 11 - tools/xenpaging/xenpaging.c | 231 +++++++++++++++++++++++++++------------ tools/xenpaging/xenpaging.h | 8 - 5 files changed, 175 insertions(+), 80 deletions(-) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2011-Sep-15 06:16 UTC
[Xen-devel] [PATCH 1 of 9] xenpaging: install into LIBEXEC dir
# HG changeset patch # User Olaf Hering <olaf@aepfle.de> # Date 1316067221 -7200 # Node ID cd07fbf47c11055a30ba21e904bf5537c50b044b # Parent e90438f6e6d1585a71b18784a99c162b5d95f390 xenpaging: install into LIBEXEC dir In preparation of upcoming libxl integration, move xenpaging binary from /usr/sbin/ to /usr/lib/xen/bin/ Signed-off-by: Olaf Hering <olaf@aepfle.de> diff -r e90438f6e6d1 -r cd07fbf47c11 tools/xenpaging/Makefile --- a/tools/xenpaging/Makefile +++ b/tools/xenpaging/Makefile @@ -24,8 +24,8 @@ xenpaging: $(OBJS) install: all $(INSTALL_DIR) $(DESTDIR)/var/lib/xen/xenpaging - $(INSTALL_DIR) $(DESTDIR)$(SBINDIR) - $(INSTALL_PROG) $(IBINS) $(DESTDIR)$(SBINDIR) + $(INSTALL_DIR) $(DESTDIR)$(LIBEXEC) + $(INSTALL_PROG) $(IBINS) $(DESTDIR)$(LIBEXEC) clean: rm -f *.o *~ $(DEPS) xen TAGS $(IBINS) $(LIB) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2011-Sep-15 06:16 UTC
[Xen-devel] [PATCH 2 of 9] xenpaging: remove filename from comment
# HG changeset patch # User Olaf Hering <olaf@aepfle.de> # Date 1316067223 -7200 # Node ID eb968e957ee4789a7a06c5f5683eb4b0e49324cd # Parent cd07fbf47c11055a30ba21e904bf5537c50b044b xenpaging: remove filename from comment Signed-off-by: Olaf Hering <olaf@aepfle.de> diff -r cd07fbf47c11 -r eb968e957ee4 tools/xenpaging/file_ops.c --- a/tools/xenpaging/file_ops.c +++ b/tools/xenpaging/file_ops.c @@ -1,5 +1,4 @@ /****************************************************************************** - * tools/xenpaging/file_ops.c * * Common file operations. * diff -r cd07fbf47c11 -r eb968e957ee4 tools/xenpaging/policy_default.c --- a/tools/xenpaging/policy_default.c +++ b/tools/xenpaging/policy_default.c @@ -1,5 +1,4 @@ /****************************************************************************** - * tools/xenpaging/policy.c * * Xen domain paging default policy. * diff -r cd07fbf47c11 -r eb968e957ee4 tools/xenpaging/xenpaging.c --- a/tools/xenpaging/xenpaging.c +++ b/tools/xenpaging/xenpaging.c @@ -1,5 +1,4 @@ /****************************************************************************** - * tools/xenpaging/xenpaging.c * * Domain paging. * Copyright (c) 2009 by Citrix Systems, Inc. (Patrick Colp) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2011-Sep-15 06:16 UTC
[Xen-devel] [PATCH 3 of 9] xenpaging: update xenpaging_init
# HG changeset patch # User Olaf Hering <olaf@aepfle.de> # Date 1316067225 -7200 # Node ID 1e5697849aa62c08651a613901dcf054ee652ea4 # Parent eb968e957ee4789a7a06c5f5683eb4b0e49324cd xenpaging: update xenpaging_init Move comment about xc_handle to the right place. Allocate paging early and use calloc. Signed-off-by: Olaf Hering <olaf@aepfle.de> diff -r eb968e957ee4 -r 1e5697849aa6 tools/xenpaging/xenpaging.c --- a/tools/xenpaging/xenpaging.c +++ b/tools/xenpaging/xenpaging.c @@ -169,18 +169,21 @@ static xenpaging_t *xenpaging_init(domid char *p; int rc; + /* Allocate memory */ + paging = calloc(1, sizeof(xenpaging_t)); + if ( !paging ) + goto err; + if ( getenv("XENPAGING_DEBUG") ) dbg = (xentoollog_logger *)xtl_createlogger_stdiostream(stderr, XTL_DEBUG, 0); - xch = xc_interface_open(dbg, NULL, 0); + + /* Open connection to xen */ + paging->xc_handle = xch = xc_interface_open(dbg, NULL, 0); if ( !xch ) - goto err_iface; + goto err; DPRINTF("xenpaging init\n"); - /* Allocate memory */ - paging = malloc(sizeof(xenpaging_t)); - memset(paging, 0, sizeof(xenpaging_t)); - /* Open connection to xenstore */ paging->xs_handle = xs_open(0); if ( paging->xs_handle == NULL ) @@ -204,9 +207,6 @@ static xenpaging_t *xenpaging_init(domid DPRINTF("Setting policy mru_size to %d\n", paging->policy_mru_size); } - /* Open connection to xen */ - paging->xc_handle = xch; - /* Set domain id */ paging->mem_event.domain_id = domain_id; @@ -319,7 +319,8 @@ static xenpaging_t *xenpaging_init(domid { if ( paging->xs_handle ) xs_close(paging->xs_handle); - xc_interface_close(xch); + if ( xch ) + xc_interface_close(xch); if ( paging->mem_event.shared_page ) { munlock(paging->mem_event.shared_page, PAGE_SIZE); @@ -337,7 +338,6 @@ static xenpaging_t *xenpaging_init(domid free(paging); } - err_iface: return NULL; } _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2011-Sep-15 06:16 UTC
[Xen-devel] [PATCH 4 of 9] xenpaging: remove xc_dominfo_t from paging_t
# HG changeset patch # User Olaf Hering <olaf@aepfle.de> # Date 1316067227 -7200 # Node ID 2ff9871c02a1712d55addc2b30e4f7694e5b7ebe # Parent 1e5697849aa62c08651a613901dcf054ee652ea4 xenpaging: remove xc_dominfo_t from paging_t Remove xc_dominfo_t from paging_t, record only max_pages. This value is used to setup internal data structures. Signed-off-by: Olaf Hering <olaf@aepfle.de> diff -r 1e5697849aa6 -r 2ff9871c02a1 tools/xenpaging/policy_default.c --- a/tools/xenpaging/policy_default.c +++ b/tools/xenpaging/policy_default.c @@ -41,17 +41,17 @@ int policy_init(xenpaging_t *paging) int i; int rc = -ENOMEM; + max_pages = paging->max_pages; + /* Allocate bitmap for pages not to page out */ - bitmap = bitmap_alloc(paging->domain_info->max_pages); + bitmap = bitmap_alloc(max_pages); if ( !bitmap ) goto out; /* Allocate bitmap to track unusable pages */ - unconsumed = bitmap_alloc(paging->domain_info->max_pages); + unconsumed = bitmap_alloc(max_pages); if ( !unconsumed ) goto out; - max_pages = paging->domain_info->max_pages; - /* Initialise MRU list of paged in pages */ if ( paging->policy_mru_size > 0 ) mru_size = paging->policy_mru_size; diff -r 1e5697849aa6 -r 2ff9871c02a1 tools/xenpaging/xenpaging.c --- a/tools/xenpaging/xenpaging.c +++ b/tools/xenpaging/xenpaging.c @@ -164,6 +164,7 @@ static void *init_page(void) static xenpaging_t *xenpaging_init(domid_t domain_id, int num_pages) { xenpaging_t *paging; + xc_domaininfo_t domain_info; xc_interface *xch; xentoollog_logger *dbg = NULL; char *p; @@ -273,33 +274,29 @@ static xenpaging_t *xenpaging_init(domid paging->mem_event.port = rc; /* Get domaininfo */ - paging->domain_info = malloc(sizeof(xc_domaininfo_t)); - if ( paging->domain_info == NULL ) - { - ERROR("Error allocating memory for domain info"); - goto err; - } - rc = xc_domain_getinfolist(xch, paging->mem_event.domain_id, 1, - paging->domain_info); + &domain_info); if ( rc != 1 ) { ERROR("Error getting domain info"); goto err; } + /* Record number of max_pages */ + paging->max_pages = domain_info.max_pages; + /* Allocate bitmap for tracking pages that have been paged out */ - paging->bitmap = bitmap_alloc(paging->domain_info->max_pages); + paging->bitmap = bitmap_alloc(paging->max_pages); if ( !paging->bitmap ) { ERROR("Error allocating bitmap"); goto err; } - DPRINTF("max_pages = %"PRIx64"\n", paging->domain_info->max_pages); + DPRINTF("max_pages = %d\n", paging->max_pages); - if ( num_pages < 0 || num_pages > paging->domain_info->max_pages ) + if ( num_pages < 0 || num_pages > paging->max_pages ) { - num_pages = paging->domain_info->max_pages; + num_pages = paging->max_pages; DPRINTF("setting num_pages to %d\n", num_pages); } paging->num_pages = num_pages; @@ -334,7 +331,6 @@ static xenpaging_t *xenpaging_init(domid } free(paging->bitmap); - free(paging->domain_info); free(paging); } @@ -764,7 +760,7 @@ int main(int argc, char *argv[]) if ( interrupted == SIGTERM || interrupted == SIGINT ) { int num = 0; - for ( i = 0; i < paging->domain_info->max_pages; i++ ) + for ( i = 0; i < paging->max_pages; i++ ) { if ( test_bit(i, paging->bitmap) ) { @@ -780,7 +776,7 @@ int main(int argc, char *argv[]) */ if ( num ) page_in_trigger(); - else if ( i == paging->domain_info->max_pages ) + else if ( i == paging->max_pages ) break; } else diff -r 1e5697849aa6 -r 2ff9871c02a1 tools/xenpaging/xenpaging.h --- a/tools/xenpaging/xenpaging.h +++ b/tools/xenpaging/xenpaging.h @@ -44,11 +44,11 @@ typedef struct xenpaging { xc_interface *xc_handle; struct xs_handle *xs_handle; - xc_domaininfo_t *domain_info; - unsigned long *bitmap; mem_event_t mem_event; + /* number of pages for which data structures were allocated */ + int max_pages; int num_pages; int policy_mru_size; unsigned long pagein_queue[XENPAGING_PAGEIN_QUEUE_SIZE]; _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2011-Sep-15 06:16 UTC
[Xen-devel] [PATCH 5 of 9] xenpaging: track the number of paged-out pages
# HG changeset patch # User Olaf Hering <olaf@aepfle.de> # Date 1316067229 -7200 # Node ID 6beca8cbc2c92900859712f8738db17084bcebdb # Parent 2ff9871c02a1712d55addc2b30e4f7694e5b7ebe xenpaging: track the number of paged-out pages This change is required by subsequent changes. Signed-off-by: Olaf Hering <olaf@aepfle.de> diff -r 2ff9871c02a1 -r 6beca8cbc2c9 tools/xenpaging/xenpaging.c --- a/tools/xenpaging/xenpaging.c +++ b/tools/xenpaging/xenpaging.c @@ -468,6 +468,9 @@ static int xenpaging_evict_page(xenpagin /* Notify policy of page being paged out */ policy_notify_paged_out(victim->gfn); + /* Record number of evicted pages */ + paging->num_paged_out++; + out: return ret; } @@ -481,8 +484,13 @@ static int xenpaging_resume_page(xenpagi /* Notify policy of page being paged in */ if ( notify_policy ) + { policy_notify_paged_in(rsp->gfn); + /* Record number of resumed pages */ + paging->num_paged_out--; + } + /* Tell Xen page is ready */ ret = xc_mem_paging_resume(paging->xc_handle, paging->mem_event.domain_id, rsp->gfn); diff -r 2ff9871c02a1 -r 6beca8cbc2c9 tools/xenpaging/xenpaging.h --- a/tools/xenpaging/xenpaging.h +++ b/tools/xenpaging/xenpaging.h @@ -49,6 +49,7 @@ typedef struct xenpaging { mem_event_t mem_event; /* number of pages for which data structures were allocated */ int max_pages; + int num_paged_out; int num_pages; int policy_mru_size; unsigned long pagein_queue[XENPAGING_PAGEIN_QUEUE_SIZE]; _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2011-Sep-15 06:16 UTC
[Xen-devel] [PATCH 6 of 9] xenpaging: add evict_pages function
# HG changeset patch # User Olaf Hering <olaf@aepfle.de> # Date 1316067230 -7200 # Node ID 3a3a5979b799d948802183d10d65894ee84a872f # Parent 6beca8cbc2c92900859712f8738db17084bcebdb xenpaging: add evict_pages function Add new function to evict a couple of pages. Allocate all possible slots in a paging file to allow growing and shrinking of the number of paged-out pages. Adjust other places to iterate over all slots. This change is required by subsequent patches. Signed-off-by: Olaf Hering <olaf@aepfle.de> diff -r 6beca8cbc2c9 -r 3a3a5979b799 tools/xenpaging/xenpaging.c --- a/tools/xenpaging/xenpaging.c +++ b/tools/xenpaging/xenpaging.c @@ -597,6 +597,30 @@ static int evict_victim(xenpaging_t *pag return ret; } +/* Evict a couple of pages and write them to a free slot in the paging file */ +static int evict_pages(xenpaging_t *paging, int fd, xenpaging_victim_t *victims, int num_pages) +{ + xc_interface *xch = paging->xc_handle; + int rc, slot, num = 0; + + for ( slot = 0; slot < paging->max_pages && num < num_pages; slot++ ) + { + /* Slot is allocated */ + if ( victims[slot].gfn != INVALID_MFN ) + continue; + + rc = evict_victim(paging, &victims[slot], fd, slot); + if ( rc == -ENOSPC ) + break; + if ( rc == -EINTR ) + break; + if ( num && num % 100 == 0 ) + DPRINTF("%d pages evicted\n", num); + num++; + } + return num; +} + int main(int argc, char *argv[]) { struct sigaction act; @@ -639,7 +663,12 @@ int main(int argc, char *argv[]) return 2; } - victims = calloc(paging->num_pages, sizeof(xenpaging_victim_t)); + /* Allocate upper limit of pages to allow growing and shrinking */ + victims = calloc(paging->max_pages, sizeof(xenpaging_victim_t)); + + /* Mark all slots as unallocated */ + for ( i = 0; i < paging->max_pages; i++ ) + victims[i].gfn = INVALID_MFN; /* ensure that if we get a signal, we''ll do cleanup, then exit */ act.sa_handler = close_handler; @@ -653,18 +682,7 @@ int main(int argc, char *argv[]) /* listen for page-in events to stop pager */ create_page_in_thread(paging); - /* Evict pages */ - for ( i = 0; i < paging->num_pages; i++ ) - { - rc = evict_victim(paging, &victims[i], fd, i); - if ( rc == -ENOSPC ) - break; - if ( rc == -EINTR ) - break; - if ( i % 100 == 0 ) - DPRINTF("%d pages evicted\n", i); - } - + i = evict_pages(paging, fd, victims, paging->num_pages); DPRINTF("%d pages evicted. Done.\n", i); /* Swap pages in and out */ @@ -690,13 +708,13 @@ int main(int argc, char *argv[]) if ( test_and_clear_bit(req.gfn, paging->bitmap) ) { /* Find where in the paging file to read from */ - for ( i = 0; i < paging->num_pages; i++ ) + for ( i = 0; i < paging->max_pages; i++ ) { if ( victims[i].gfn == req.gfn ) break; } - if ( i >= paging->num_pages ) + if ( i >= paging->max_pages ) { DPRINTF("Couldn''t find page %"PRIx64"\n", req.gfn); goto out; _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2011-Sep-15 06:16 UTC
[Xen-devel] [PATCH 7 of 9] xenpaging: add resume_pages function
# HG changeset patch # User Olaf Hering <olaf@aepfle.de> # Date 1316067232 -7200 # Node ID 6987aa2dde4e93481f1599735ed1a26defb6d6b9 # Parent 3a3a5979b799d948802183d10d65894ee84a872f xenpaging: add resume_pages function Add new function to resume a couple of pages. This change is required by subsequent patches. Signed-off-by: Olaf Hering <olaf@aepfle.de> diff -r 3a3a5979b799 -r 6987aa2dde4e tools/xenpaging/xenpaging.c --- a/tools/xenpaging/xenpaging.c +++ b/tools/xenpaging/xenpaging.c @@ -554,6 +554,27 @@ static int xenpaging_populate_page(xenpa return ret; } +/* Trigger a page-in for a couple of pages */ +static void resume_pages(xenpaging_t *paging, int num_pages) +{ + xc_interface *xch = paging->xc_handle; + int i, num = 0; + + for ( i = 0; i < paging->max_pages && num < num_pages; i++ ) + { + if ( test_bit(i, paging->bitmap) ) + { + paging->pagein_queue[num] = i; + num++; + if ( num == XENPAGING_PAGEIN_QUEUE_SIZE ) + break; + } + } + /* num may be less than num_pages, caller has to try again */ + if ( num ) + page_in_trigger(); +} + static int evict_victim(xenpaging_t *paging, xenpaging_victim_t *victim, int fd, int i) { @@ -785,25 +806,12 @@ int main(int argc, char *argv[]) /* Write all pages back into the guest */ if ( interrupted == SIGTERM || interrupted == SIGINT ) { - int num = 0; - for ( i = 0; i < paging->max_pages; i++ ) - { - if ( test_bit(i, paging->bitmap) ) - { - paging->pagein_queue[num] = i; - num++; - if ( num == XENPAGING_PAGEIN_QUEUE_SIZE ) - break; - } - } - /* - * One more round if there are still pages to process. - * If no more pages to process, exit loop. - */ - if ( num ) - page_in_trigger(); - else if ( i == paging->max_pages ) + /* If no more pages to process, exit loop. */ + if ( !paging->num_paged_out ) break; + + /* One more round if there are still pages to process. */ + resume_pages(paging, paging->num_paged_out); } else { _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2011-Sep-15 06:16 UTC
[Xen-devel] [PATCH 8 of 9] xenpaging: compare both token and path when checking for @releaseDomain event
# HG changeset patch # User Olaf Hering <olaf@aepfle.de> # Date 1316067234 -7200 # Node ID 1addb9e81178ea164ef516f340df2607fddcb930 # Parent 6987aa2dde4e93481f1599735ed1a26defb6d6b9 xenpaging: compare both token and path when checking for @releaseDomain event Subsequent patches will use xenstored to store the numbers of pages xenpaging is suppose to page-out. A domain_id value could be misinterpreted as number of pages. Compare both path and token to recognize the @releaseDomain event. Signed-off-by: Olaf Hering <olaf@aepfle.de> diff -r 6987aa2dde4e -r 1addb9e81178 tools/xenpaging/xenpaging.c --- a/tools/xenpaging/xenpaging.c +++ b/tools/xenpaging/xenpaging.c @@ -101,7 +101,7 @@ static int xenpaging_wait_for_event_or_t vec = xs_read_watch(paging->xs_handle, &num); if ( vec ) { - if ( strcmp(vec[XS_WATCH_TOKEN], watch_token) == 0 ) + if ( strcmp(vec[XS_WATCH_PATH], "@releaseDomain") == 0 && strcmp(vec[XS_WATCH_TOKEN], watch_token) == 0 ) { /* If our guest disappeared, set interrupt flag and fall through */ if ( xs_is_domain_introduced(paging->xs_handle, paging->mem_event.domain_id) == false ) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2011-Sep-15 06:16 UTC
[Xen-devel] [PATCH 9 of 9] xenpaging: watch the domains /xenpaging/num_pages xenstore value
# HG changeset patch # User Olaf Hering <olaf@aepfle.de> # Date 1316067236 -7200 # Node ID ad903b34e0997e864539e36b92dde234f9ffbd55 # Parent 1addb9e81178ea164ef516f340df2607fddcb930 xenpaging: watch the domains /xenpaging/num_pages xenstore value Subsequent patches will use xenstored to store the numbers of pages xenpaging is suppose to page-out. Remove num_pages and use target_pages instead. Signed-off-by: Olaf Hering <olaf@aepfle.de> diff -r 1addb9e81178 -r ad903b34e099 tools/xenpaging/policy_default.c --- a/tools/xenpaging/policy_default.c +++ b/tools/xenpaging/policy_default.c @@ -70,7 +70,7 @@ int policy_init(xenpaging_t *paging) /* Start in the middle to avoid paging during BIOS startup */ current_gfn = max_pages / 2; - current_gfn -= paging->num_pages / 2; + current_gfn -= paging->target_pages / 2; rc = 0; out: diff -r 1addb9e81178 -r ad903b34e099 tools/xenpaging/xenpaging.c --- a/tools/xenpaging/xenpaging.c +++ b/tools/xenpaging/xenpaging.c @@ -35,6 +35,9 @@ #include "policy.h" #include "xenpaging.h" +/* Defines number of pages to page-out */ +#define WATCH_TARGETPAGES "/xenpaging/num_pages" +static char *watch_targetpages; static char watch_token[16]; static char filename[80]; static int interrupted; @@ -72,7 +75,7 @@ static int xenpaging_wait_for_event_or_t { xc_interface *xch = paging->xc_handle; xc_evtchn *xce = paging->mem_event.xce_handle; - char **vec; + char **vec, *val; unsigned int num; struct pollfd fd[2]; int port; @@ -101,6 +104,7 @@ static int xenpaging_wait_for_event_or_t vec = xs_read_watch(paging->xs_handle, &num); if ( vec ) { + DPRINTF("path ''%s'' token ''%s''\n", vec[XS_WATCH_PATH], vec[XS_WATCH_TOKEN]); if ( strcmp(vec[XS_WATCH_PATH], "@releaseDomain") == 0 && strcmp(vec[XS_WATCH_TOKEN], watch_token) == 0 ) { /* If our guest disappeared, set interrupt flag and fall through */ @@ -111,6 +115,22 @@ static int xenpaging_wait_for_event_or_t rc = 0; } } + else if ( strcmp(vec[XS_WATCH_PATH], watch_targetpages) == 0 ) + { + int ret, target_pages; + val = xs_read(paging->xs_handle, XBT_NULL, vec[XS_WATCH_PATH], NULL); + if (val) + { + ret = sscanf(val, "%d", &target_pages); + if ( ret > 0 ) + { + if ( target_pages < 0 || target_pages > paging->max_pages ) + target_pages = paging->max_pages; + paging->target_pages = target_pages; + DPRINTF("new target_pages %d\n", target_pages); + } + } + } free(vec); } } @@ -167,8 +187,8 @@ static xenpaging_t *xenpaging_init(domid xc_domaininfo_t domain_info; xc_interface *xch; xentoollog_logger *dbg = NULL; - char *p; - int rc; + char *p, *dom_path = NULL, *watchdir = WATCH_TARGETPAGES; + int rc, len; /* Allocate memory */ paging = calloc(1, sizeof(xenpaging_t)); @@ -201,6 +221,28 @@ static xenpaging_t *xenpaging_init(domid goto err; } + /* watch guests xenpaging directory */ + dom_path = xs_get_domain_path(paging->xs_handle, domain_id); + if ( !dom_path ) + { + ERROR("Could not find domain path\n"); + goto err; + } + len = strlen(dom_path) + strlen(watchdir) + 1; + watch_targetpages = malloc(len); + if ( !watch_targetpages ) + { + ERROR("Could not alloc domain path\n"); + goto err; + } + snprintf(watch_targetpages, len, "%s%s", dom_path, watchdir); + DPRINTF("watching ''%s''\n", watch_targetpages); + if ( xs_watch(paging->xs_handle, watch_targetpages, "") == false ) + { + ERROR("Could not bind to xenpaging watch\n"); + goto err; + } + p = getenv("XENPAGING_POLICY_MRU_SIZE"); if ( p && *p ) { @@ -299,7 +341,7 @@ static xenpaging_t *xenpaging_init(domid num_pages = paging->max_pages; DPRINTF("setting num_pages to %d\n", num_pages); } - paging->num_pages = num_pages; + paging->target_pages = num_pages; /* Initialise policy */ rc = policy_init(paging); @@ -330,6 +372,8 @@ static xenpaging_t *xenpaging_init(domid free(paging->mem_event.ring_page); } + free(dom_path); + free(watch_targetpages); free(paging->bitmap); free(paging); } @@ -345,6 +389,9 @@ static int xenpaging_teardown(xenpaging_ if ( paging == NULL ) return 0; + xs_unwatch(paging->xs_handle, watch_targetpages, ""); + xs_unwatch(paging->xs_handle, "@releaseDomain", watch_token); + xch = paging->xc_handle; paging->xc_handle = NULL; /* Tear down domain paging in Xen */ @@ -649,6 +696,7 @@ int main(int argc, char *argv[]) xenpaging_victim_t *victims; mem_event_request_t req; mem_event_response_t rsp; + int num, prev_num = 0; int i; int rc = -1; int rc1; @@ -673,7 +721,7 @@ int main(int argc, char *argv[]) } xch = paging->xc_handle; - DPRINTF("starting %s %u %d\n", argv[0], paging->mem_event.domain_id, paging->num_pages); + DPRINTF("starting %s %u %d\n", argv[0], paging->mem_event.domain_id, paging->target_pages); /* Open file */ sprintf(filename, "page_cache_%u", paging->mem_event.domain_id); @@ -703,9 +751,6 @@ int main(int argc, char *argv[]) /* listen for page-in events to stop pager */ create_page_in_thread(paging); - i = evict_pages(paging, fd, victims, paging->num_pages); - DPRINTF("%d pages evicted. Done.\n", i); - /* Swap pages in and out */ while ( 1 ) { @@ -774,8 +819,6 @@ int main(int argc, char *argv[]) * or clear this pagefile slot on exit */ if ( interrupted ) victims[i].gfn = INVALID_MFN; - else - evict_victim(paging, &victims[i], fd, i); } else { @@ -819,6 +862,29 @@ int main(int argc, char *argv[]) if ( interrupted ) break; } + if ( paging->num_paged_out < paging->target_pages ) + { + num = paging->target_pages - paging->num_paged_out; + if ( num != prev_num ) + { + DPRINTF("Need to evict %d pages to reach %d target_pages\n", num, paging->target_pages); + prev_num = num; + } + /* Limit the number of evicts to be able to process page-in requests */ + if ( num > 42 ) + num = 42; + evict_pages(paging, fd, victims, num); + } + else if ( paging->num_paged_out > paging->target_pages ) + { + num = paging->num_paged_out - paging->target_pages; + if ( num != prev_num ) + { + DPRINTF("Need to resume %d pages to reach %d target_pages\n", num, paging->target_pages); + prev_num = num; + } + resume_pages(paging, num); + } } DPRINTF("xenpaging got signal %d\n", interrupted); diff -r 1addb9e81178 -r ad903b34e099 tools/xenpaging/xenpaging.h --- a/tools/xenpaging/xenpaging.h +++ b/tools/xenpaging/xenpaging.h @@ -50,7 +50,8 @@ typedef struct xenpaging { /* number of pages for which data structures were allocated */ int max_pages; int num_paged_out; - int num_pages; + /* number of pages xenpaging is supposed to page-out */ + int target_pages; int policy_mru_size; unsigned long pagein_queue[XENPAGING_PAGEIN_QUEUE_SIZE]; } xenpaging_t; _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-Sep-15 08:16 UTC
Re: [Xen-devel] [PATCH 6 of 9] xenpaging: add evict_pages function
On Thu, 2011-09-15 at 02:16 -0400, Olaf Hering wrote:> # HG changeset patch > # User Olaf Hering <olaf@aepfle.de> > # Date 1316067230 -7200 > # Node ID 3a3a5979b799d948802183d10d65894ee84a872f > # Parent 6beca8cbc2c92900859712f8738db17084bcebdb > xenpaging: add evict_pages function > > Add new function to evict a couple of pages.Do you really mean "a couple" here? (that generally means exactly two).>From the implementation I think you mean evict a batch of pages?> Allocate all possible slots in a paging file to allow growing and > shrinking of the number of paged-out pages. Adjust other places to > iterate over all slots. > > This change is required by subsequent patches. > > Signed-off-by: Olaf Hering <olaf@aepfle.de> > > diff -r 6beca8cbc2c9 -r 3a3a5979b799 tools/xenpaging/xenpaging.c > --- a/tools/xenpaging/xenpaging.c > +++ b/tools/xenpaging/xenpaging.c > @@ -597,6 +597,30 @@ static int evict_victim(xenpaging_t *pag > return ret; > } > > +/* Evict a couple of pages and write them to a free slot in the paging file */ > +static int evict_pages(xenpaging_t *paging, int fd, xenpaging_victim_t *victims, int num_pages) > +{ > + xc_interface *xch = paging->xc_handle; > + int rc, slot, num = 0; > + > + for ( slot = 0; slot < paging->max_pages && num < num_pages; slot++ ) > + { > + /* Slot is allocated */ > + if ( victims[slot].gfn != INVALID_MFN ) > + continue; > + > + rc = evict_victim(paging, &victims[slot], fd, slot); > + if ( rc == -ENOSPC ) > + break; > + if ( rc == -EINTR ) > + break; > + if ( num && num % 100 == 0 ) > + DPRINTF("%d pages evicted\n", num); > + num++; > + } > + return num; > +} > + > int main(int argc, char *argv[]) > { > struct sigaction act; > @@ -639,7 +663,12 @@ int main(int argc, char *argv[]) > return 2; > } > > - victims = calloc(paging->num_pages, sizeof(xenpaging_victim_t)); > + /* Allocate upper limit of pages to allow growing and shrinking */ > + victims = calloc(paging->max_pages, sizeof(xenpaging_victim_t)); > + > + /* Mark all slots as unallocated */ > + for ( i = 0; i < paging->max_pages; i++ ) > + victims[i].gfn = INVALID_MFN; > > /* ensure that if we get a signal, we''ll do cleanup, then exit */ > act.sa_handler = close_handler; > @@ -653,18 +682,7 @@ int main(int argc, char *argv[]) > /* listen for page-in events to stop pager */ > create_page_in_thread(paging); > > - /* Evict pages */ > - for ( i = 0; i < paging->num_pages; i++ ) > - { > - rc = evict_victim(paging, &victims[i], fd, i); > - if ( rc == -ENOSPC ) > - break; > - if ( rc == -EINTR ) > - break; > - if ( i % 100 == 0 ) > - DPRINTF("%d pages evicted\n", i); > - } > - > + i = evict_pages(paging, fd, victims, paging->num_pages); > DPRINTF("%d pages evicted. Done.\n", i); > > /* Swap pages in and out */ > @@ -690,13 +708,13 @@ int main(int argc, char *argv[]) > if ( test_and_clear_bit(req.gfn, paging->bitmap) ) > { > /* Find where in the paging file to read from */ > - for ( i = 0; i < paging->num_pages; i++ ) > + for ( i = 0; i < paging->max_pages; i++ ) > { > if ( victims[i].gfn == req.gfn ) > break; > } > > - if ( i >= paging->num_pages ) > + if ( i >= paging->max_pages ) > { > DPRINTF("Couldn''t find page %"PRIx64"\n", req.gfn); > goto out; > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
George Dunlap
2011-Sep-15 09:05 UTC
Re: [Xen-devel] [PATCH 6 of 9] xenpaging: add evict_pages function
On Thu, Sep 15, 2011 at 9:16 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Thu, 2011-09-15 at 02:16 -0400, Olaf Hering wrote: >> # HG changeset patch >> # User Olaf Hering <olaf@aepfle.de> >> # Date 1316067230 -7200 >> # Node ID 3a3a5979b799d948802183d10d65894ee84a872f >> # Parent 6beca8cbc2c92900859712f8738db17084bcebdb >> xenpaging: add evict_pages function >> >> Add new function to evict a couple of pages. > > Do you really mean "a couple" here? (that generally means exactly two).LIterally "couple" means two, but at least in US idiom, "a couple of [foo]" means a small indeterminate number, usually 2-4. In any case, a more precise description seems like a better idea -- it looks like it takes an argument for the number of pages to evict; and it''s not adding a new function, it''s pulling existing code into a function. So, "Pull eviction loop into a function" would probably be a better description. -George> >From the implementation I think you mean evict a batch of pages? > >> Allocate all possible slots in a paging file to allow growing and >> shrinking of the number of paged-out pages. Adjust other places to >> iterate over all slots. >> >> This change is required by subsequent patches. >> >> Signed-off-by: Olaf Hering <olaf@aepfle.de> >> >> diff -r 6beca8cbc2c9 -r 3a3a5979b799 tools/xenpaging/xenpaging.c >> --- a/tools/xenpaging/xenpaging.c >> +++ b/tools/xenpaging/xenpaging.c >> @@ -597,6 +597,30 @@ static int evict_victim(xenpaging_t *pag >> return ret; >> } >> >> +/* Evict a couple of pages and write them to a free slot in the paging file */ >> +static int evict_pages(xenpaging_t *paging, int fd, xenpaging_victim_t *victims, int num_pages) >> +{ >> + xc_interface *xch = paging->xc_handle; >> + int rc, slot, num = 0; >> + >> + for ( slot = 0; slot < paging->max_pages && num < num_pages; slot++ ) >> + { >> + /* Slot is allocated */ >> + if ( victims[slot].gfn != INVALID_MFN ) >> + continue; >> + >> + rc = evict_victim(paging, &victims[slot], fd, slot); >> + if ( rc == -ENOSPC ) >> + break; >> + if ( rc == -EINTR ) >> + break; >> + if ( num && num % 100 == 0 ) >> + DPRINTF("%d pages evicted\n", num); >> + num++; >> + } >> + return num; >> +} >> + >> int main(int argc, char *argv[]) >> { >> struct sigaction act; >> @@ -639,7 +663,12 @@ int main(int argc, char *argv[]) >> return 2; >> } >> >> - victims = calloc(paging->num_pages, sizeof(xenpaging_victim_t)); >> + /* Allocate upper limit of pages to allow growing and shrinking */ >> + victims = calloc(paging->max_pages, sizeof(xenpaging_victim_t)); >> + >> + /* Mark all slots as unallocated */ >> + for ( i = 0; i < paging->max_pages; i++ ) >> + victims[i].gfn = INVALID_MFN; >> >> /* ensure that if we get a signal, we''ll do cleanup, then exit */ >> act.sa_handler = close_handler; >> @@ -653,18 +682,7 @@ int main(int argc, char *argv[]) >> /* listen for page-in events to stop pager */ >> create_page_in_thread(paging); >> >> - /* Evict pages */ >> - for ( i = 0; i < paging->num_pages; i++ ) >> - { >> - rc = evict_victim(paging, &victims[i], fd, i); >> - if ( rc == -ENOSPC ) >> - break; >> - if ( rc == -EINTR ) >> - break; >> - if ( i % 100 == 0 ) >> - DPRINTF("%d pages evicted\n", i); >> - } >> - >> + i = evict_pages(paging, fd, victims, paging->num_pages); >> DPRINTF("%d pages evicted. Done.\n", i); >> >> /* Swap pages in and out */ >> @@ -690,13 +708,13 @@ int main(int argc, char *argv[]) >> if ( test_and_clear_bit(req.gfn, paging->bitmap) ) >> { >> /* Find where in the paging file to read from */ >> - for ( i = 0; i < paging->num_pages; i++ ) >> + for ( i = 0; i < paging->max_pages; i++ ) >> { >> if ( victims[i].gfn == req.gfn ) >> break; >> } >> >> - if ( i >= paging->num_pages ) >> + if ( i >= paging->max_pages ) >> { >> DPRINTF("Couldn''t find page %"PRIx64"\n", req.gfn); >> goto out; >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
George Dunlap
2011-Sep-15 09:15 UTC
Re: [Xen-devel] [PATCH 7 of 9] xenpaging: add resume_pages function
On Thu, Sep 15, 2011 at 7:16 AM, Olaf Hering <olaf@aepfle.de> wrote:> # HG changeset patch > # User Olaf Hering <olaf@aepfle.de> > # Date 1316067232 -7200 > # Node ID 6987aa2dde4e93481f1599735ed1a26defb6d6b9 > # Parent 3a3a5979b799d948802183d10d65894ee84a872f > xenpaging: add resume_pages function > > Add new function to resume a couple of pages. > This change is required by subsequent patches.Olaf, if you end up re-submitting this patch series for any reason, I would combine this patch and the previous patch, and make the description something like this: "Pull page eviction and resume loops into their own functions." That gives a better idea what''s going on. Since neither really makes any functional changes, there''s no need to have separate patches, IMHO. (I don''t think that in itself is worth rejecting the series for.) -George> > Signed-off-by: Olaf Hering <olaf@aepfle.de> > > diff -r 3a3a5979b799 -r 6987aa2dde4e tools/xenpaging/xenpaging.c > --- a/tools/xenpaging/xenpaging.c > +++ b/tools/xenpaging/xenpaging.c > @@ -554,6 +554,27 @@ static int xenpaging_populate_page(xenpa > return ret; > } > > +/* Trigger a page-in for a couple of pages */ > +static void resume_pages(xenpaging_t *paging, int num_pages) > +{ > + xc_interface *xch = paging->xc_handle; > + int i, num = 0; > + > + for ( i = 0; i < paging->max_pages && num < num_pages; i++ ) > + { > + if ( test_bit(i, paging->bitmap) ) > + { > + paging->pagein_queue[num] = i; > + num++; > + if ( num == XENPAGING_PAGEIN_QUEUE_SIZE ) > + break; > + } > + } > + /* num may be less than num_pages, caller has to try again */ > + if ( num ) > + page_in_trigger(); > +} > + > static int evict_victim(xenpaging_t *paging, > xenpaging_victim_t *victim, int fd, int i) > { > @@ -785,25 +806,12 @@ int main(int argc, char *argv[]) > /* Write all pages back into the guest */ > if ( interrupted == SIGTERM || interrupted == SIGINT ) > { > - int num = 0; > - for ( i = 0; i < paging->max_pages; i++ ) > - { > - if ( test_bit(i, paging->bitmap) ) > - { > - paging->pagein_queue[num] = i; > - num++; > - if ( num == XENPAGING_PAGEIN_QUEUE_SIZE ) > - break; > - } > - } > - /* > - * One more round if there are still pages to process. > - * If no more pages to process, exit loop. > - */ > - if ( num ) > - page_in_trigger(); > - else if ( i == paging->max_pages ) > + /* If no more pages to process, exit loop. */ > + if ( !paging->num_paged_out ) > break; > + > + /* One more round if there are still pages to process. */ > + resume_pages(paging, paging->num_paged_out); > } > else > { > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2011-Sep-15 09:17 UTC
Re: [Xen-devel] [PATCH 6 of 9] xenpaging: add evict_pages function
On Thu, Sep 15, George Dunlap wrote:> On Thu, Sep 15, 2011 at 9:16 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > On Thu, 2011-09-15 at 02:16 -0400, Olaf Hering wrote: > >> # HG changeset patch > >> # User Olaf Hering <olaf@aepfle.de> > >> # Date 1316067230 -7200 > >> # Node ID 3a3a5979b799d948802183d10d65894ee84a872f > >> # Parent 6beca8cbc2c92900859712f8738db17084bcebdb > >> xenpaging: add evict_pages function > >> > >> Add new function to evict a couple of pages. > > > > Do you really mean "a couple" here? (that generally means exactly two). > > LIterally "couple" means two, but at least in US idiom, "a couple of > [foo]" means a small indeterminate number, usually 2-4. > > In any case, a more precise description seems like a better idea -- it > looks like it takes an argument for the number of pages to evict; and > it''s not adding a new function, it''s pulling existing code into a > function. So, "Pull eviction loop into a function" would probably be > a better description.Thanks to both of you, I will improve the description. Olaf _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2011-Sep-28 16:14 UTC
Re: [Xen-devel] [PATCH 6 of 9] xenpaging: add evict_pages function
Olaf Hering writes ("Re: [Xen-devel] [PATCH 6 of 9] xenpaging: add evict_pages function"):> On Thu, Sep 15, George Dunlap wrote: > > In any case, a more precise description seems like a better idea -- it > > looks like it takes an argument for the number of pages to evict; and > > it''s not adding a new function, it''s pulling existing code into a > > function. So, "Pull eviction loop into a function" would probably be > > a better description. > > Thanks to both of you, I will improve the description.Thanks. Just to be clear, I think I''m expecting a repost either of this patch, or perhaps of the whole series ? Ian C made a comment about combining two of your later patches, too. Thanks, Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Olaf Hering
2011-Sep-28 16:19 UTC
Re: [Xen-devel] [PATCH 6 of 9] xenpaging: add evict_pages function
On Wed, Sep 28, Ian Jackson wrote:> Olaf Hering writes ("Re: [Xen-devel] [PATCH 6 of 9] xenpaging: add evict_pages function"): > > On Thu, Sep 15, George Dunlap wrote: > > > In any case, a more precise description seems like a better idea -- it > > > looks like it takes an argument for the number of pages to evict; and > > > it''s not adding a new function, it''s pulling existing code into a > > > function. So, "Pull eviction loop into a function" would probably be > > > a better description. > > > > Thanks to both of you, I will improve the description. > > Thanks. Just to be clear, I think I''m expecting a repost either of > this patch, or perhaps of the whole series ? Ian C made a comment > about combining two of your later patches, too.Yes, I will post a new series, replacing this one. Olaf _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel