This patch series allows xenstored to run in a stub domian started by dom0. It is based on a patch series posted by Alex Zeffertt in 2009 - http://old-list-archives.xen.org/archives/html/xen-devel/2009-03/msg01488.html A domain configuration for starting xenstored looks like: kernel=''/home/daniel/xen/stubdom/mini-os-x86_64-xenstore/mini-os'' extra='''' memory=50 name=''xenstore'' Once xenstore is started, "xenstore_dom=1" needs to be added to other domain''s configurations in order to set up the xenstore connection to domain 1. The following program handles post-creation parts of xenstored. To use it, run "xl create -p xenstore" and then "init-xenstore $domid". The running xenstored must be stopped to prevent xl using the UNIX sockets, and xenconsoled needs to be restarted after switching xenstores. /* init-xenstore.c: link with -lxenctrl */ #include <fcntl.h> #include <stdio.h> #include <string.h> #include <stdint.h> #include <stdlib.h> #include <sys/ioctl.h> #include <sys/mman.h> #define __XEN_TOOLS__ #include <xen/domctl.h> #include "xenctrl.h" #define IOCTL_XENBUS_BACKEND_SETUP _IOC(_IOC_NONE, ''B'', 1, 0) #define IOCTL_XENBUS_BACKEND_COMMIT _IOC(_IOC_NONE, ''B'', 2, 0) static void set_virq(int domid, int virq) { struct xen_domctl command; xc_interface *xch; xch = xc_interface_open(NULL, NULL, 0); memset(&command, 0, sizeof(command)); command.cmd = XEN_DOMCTL_set_virq_handler; command.interface_version = XEN_DOMCTL_INTERFACE_VERSION; command.domain = domid; command.u.set_virq_handler.virq = virq; xc_domctl(xch, &command); xc_interface_close(xch); } int main(int argc, char** argv) { char buf[512]; int domid = atoi(argv[1]); set_virq(domid, VIRQ_DOM_EXC); int fd = open("/dev/xen/xenbus_backend", O_RDWR); void *map = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0); int rv = ioctl(fd, IOCTL_XENBUS_BACKEND_SETUP, domid); *(uint16_t*)(map + 0x810) = rv; snprintf(buf, 512, "xl unpause %d", domid); system(buf); ioctl(fd, IOCTL_XENBUS_BACKEND_COMMIT, 0); return 0; } ------------------------------------------------- Dom0 kernel changes: [PATCH] xenbus: Add support for xenbus backend in stub domain This is based on the new /dev/xen devices introduced in Linux 3.3. Hypervisor changes: [PATCH 01/18] xen: reinstate previously unused [PATCH 02/18] xen: allow global VIRQ handlers to be delegated to [PATCH 03/18] xsm: allow use of XEN_DOMCTL_getdomaininfo by [PATCH 04/18] xen: Preserve reserved grant entries when switching Patch 1 & 4 are required for setting up grant entries in new domains. Patch 2 & 3 allow xenstored to run in an unprivileged domain. This currently requires XSM to be enabled to avoid allowing all domUs access to XEN_DOMCTL_getdomaininfo, so the patch only allows this hypercall if XSM is being compiled in. Toolstack changes: [PATCH 05/18] tools/libxl: Add xenstore and console backend domain [PATCH 06/18] lib{xc,xl}: Seed grant tables with xenstore and These patches populate two of the eight reserved grant entries in new domains with the xenstore and console shared pages, which is required if xenstored is not run in a privileged domain. Minios and xenstored: [PATCH 07/18] mini-os: avoid crash if no console is provided [PATCH 08/18] mini-os: avoid crash if no xenstore is provided [PATCH 09/18] mini-os: remove per-fd evtchn limit [PATCH 10/18] xenstored: use grant references instead of [PATCH 11/18] xenstored: add NO_SOCKETS compilation option [PATCH 12/18] xenstored support for in-memory rather than FS based [PATCH 13/18] xenstored: support running in minios stubdom [PATCH 14/18] xenstored: always use xc_gnttab_munmap in stubdom [PATCH 15/18] xenstored: add --event parameter for bootstrapping [PATCH 16/18] xenstored: pull dom0 event port from shared page [PATCH 17/18] xenstored: use domain_is_unprivileged instead of [PATCH 18/18] xenstored: add --priv-domid parameter Support for running in a stub domain
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 01/18] xen: reinstate previously unused XENMEM_remove_from_physmap hypercall
From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> This patch reinstates the XENMEM_remove_from_physmap hypercall which was removed in 19041:ee62aaafff46 because it was not used. However, is now needed in order to support xenstored stub domains. The xenstored stub domain is not priviliged like dom0 and so cannot unilaterally map the xenbus page of other guests into it''s address space. Therefore, before creating a domU the domain builder needs to seed its grant table with a grant ref allowing the xenstored stub domain to access the new domU''s xenbus page. At present domU''s do not start with their grant table mapped. Instead it gets mapped when the guest requests a grant table from the hypervisor. In order to seed the grant table, the domain builder first needs to map it into dom0 address space. But the hypercall to do this requires a gpfn (guest pfn), which is an mfn for PV guest, but a pfn for HVM guests. Therfore, in order to seed the grant table of an HVM guest, dom0 needs to *temporarily* map it into the guest''s "physical" address space. Hence the need to reinstate the XENMEM_remove_from_physmap hypercall. Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- xen/arch/ia64/xen/mm.c | 34 ++++++++++++++++++++++++++++++++++ xen/arch/x86/mm.c | 35 +++++++++++++++++++++++++++++++++++ xen/arch/x86/x86_64/compat/mm.c | 14 ++++++++++++++ xen/include/public/memory.h | 16 ++++++++++++++++ xen/include/xlat.lst | 1 + xen/include/xsm/xsm.h | 6 ++++++ xen/xsm/dummy.c | 6 ++++++ xen/xsm/flask/hooks.c | 6 ++++++ 8 files changed, 118 insertions(+), 0 deletions(-) diff --git a/xen/arch/ia64/xen/mm.c b/xen/arch/ia64/xen/mm.c index 694bf95..1b42e25 100644 --- a/xen/arch/ia64/xen/mm.c +++ b/xen/arch/ia64/xen/mm.c @@ -3447,6 +3447,40 @@ arch_memory_op(int op, XEN_GUEST_HANDLE(void) arg) break; } + case XENMEM_remove_from_physmap: + { + struct xen_remove_from_physmap xrfp; + unsigned long mfn; + struct domain *d; + + if ( copy_from_guest(&xrfp, arg, 1) ) + return -EFAULT; + + rc = rcu_lock_target_domain_by_id(xrfp.domid, &d); + if ( rc != 0 ) + return rc; + + if ( xsm_remove_from_physmap(current->domain, d) ) + { + rcu_unlock_domain(d); + return -EPERM; + } + + domain_lock(d); + + mfn = gmfn_to_mfn(d, xrfp.gpfn); + + if ( mfn_valid(mfn) ) + guest_physmap_remove_page(d, xrfp.gpfn, mfn, 0); + + domain_unlock(d); + + rcu_unlock_domain(d); + + break; + } + + case XENMEM_machine_memory_map: { struct xen_memory_map memmap; diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 52222dd..0b5332b 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -4871,6 +4871,41 @@ long arch_memory_op(int op, XEN_GUEST_HANDLE(void) arg) return rc; } + case XENMEM_remove_from_physmap: + { + struct xen_remove_from_physmap xrfp; + unsigned long mfn; + struct domain *d; + + if ( copy_from_guest(&xrfp, arg, 1) ) + return -EFAULT; + + rc = rcu_lock_target_domain_by_id(xrfp.domid, &d); + if ( rc != 0 ) + return rc; + + if ( xsm_remove_from_physmap(current->domain, d) ) + { + rcu_unlock_domain(d); + return -EPERM; + } + + domain_lock(d); + + mfn = get_gfn_untyped(d, xrfp.gpfn); + + if ( mfn_valid(mfn) ) + guest_physmap_remove_page(d, xrfp.gpfn, mfn, PAGE_ORDER_4K); + + put_gfn(d, xrfp.gpfn); + + domain_unlock(d); + + rcu_unlock_domain(d); + + break; + } + case XENMEM_set_memory_map: { struct xen_foreign_memory_map fmap; diff --git a/xen/arch/x86/x86_64/compat/mm.c b/xen/arch/x86/x86_64/compat/mm.c index bea94fe..dde5430 100644 --- a/xen/arch/x86/x86_64/compat/mm.c +++ b/xen/arch/x86/x86_64/compat/mm.c @@ -82,6 +82,20 @@ int compat_arch_memory_op(int op, XEN_GUEST_HANDLE(void) arg) break; } + case XENMEM_remove_from_physmap: + { + struct compat_remove_from_physmap cmp; + struct xen_remove_from_physmap *nat = (void *)COMPAT_ARG_XLAT_VIRT_BASE; + + if ( copy_from_guest(&cmp, arg, 1) ) + return -EFAULT; + + XLAT_remove_from_physmap(nat, &cmp); + rc = arch_memory_op(op, guest_handle_from_ptr(nat, void)); + + break; + } + case XENMEM_set_memory_map: { struct compat_foreign_memory_map cmp; diff --git a/xen/include/public/memory.h b/xen/include/public/memory.h index c5b78a8..308deff 100644 --- a/xen/include/public/memory.h +++ b/xen/include/public/memory.h @@ -229,6 +229,22 @@ struct xen_add_to_physmap { typedef struct xen_add_to_physmap xen_add_to_physmap_t; DEFINE_XEN_GUEST_HANDLE(xen_add_to_physmap_t); +/* + * Unmaps the page appearing at a particular GPFN from the specified guest''s + * pseudophysical address space. + * arg == addr of xen_remove_from_physmap_t. + */ +#define XENMEM_remove_from_physmap 15 +struct xen_remove_from_physmap { + /* Which domain to change the mapping for. */ + domid_t domid; + + /* GPFN of the current mapping of the page. */ + xen_pfn_t gpfn; +}; +typedef struct xen_remove_from_physmap xen_remove_from_physmap_t; +DEFINE_XEN_GUEST_HANDLE(xen_remove_from_physmap_t); + /*** REMOVED ***/ /*#define XENMEM_translate_gpfn_list 8*/ diff --git a/xen/include/xlat.lst b/xen/include/xlat.lst index 3d92175..9e67259 100644 --- a/xen/include/xlat.lst +++ b/xen/include/xlat.lst @@ -54,6 +54,7 @@ ! kexec_image kexec.h ! kexec_range kexec.h ! add_to_physmap memory.h +! remove_from_physmap memory.h ! foreign_memory_map memory.h ! memory_exchange memory.h ! memory_map memory.h diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h index df6cec2..566c808 100644 --- a/xen/include/xsm/xsm.h +++ b/xen/include/xsm/xsm.h @@ -169,6 +169,7 @@ struct xsm_operations { int (*update_va_mapping) (struct domain *d, struct domain *f, l1_pgentry_t pte); int (*add_to_physmap) (struct domain *d1, struct domain *d2); + int (*remove_from_physmap) (struct domain *d1, struct domain *d2); int (*sendtrigger) (struct domain *d); int (*bind_pt_irq) (struct domain *d, struct xen_domctl_bind_pt_irq *bind); int (*unbind_pt_irq) (struct domain *d); @@ -738,6 +739,11 @@ static inline int xsm_add_to_physmap(struct domain *d1, struct domain *d2) return xsm_call(add_to_physmap(d1, d2)); } +static inline int xsm_remove_from_physmap(struct domain *d1, struct domain *d2) +{ + return xsm_call(remove_from_physmap(d1, d2)); +} + static inline int xsm_sendtrigger(struct domain *d) { return xsm_call(sendtrigger(d)); diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c index 4bbfbff..65daa4e 100644 --- a/xen/xsm/dummy.c +++ b/xen/xsm/dummy.c @@ -529,6 +529,11 @@ static int dummy_add_to_physmap (struct domain *d1, struct domain *d2) return 0; } +static int dummy_remove_from_physmap (struct domain *d1, struct domain *d2) +{ + return 0; +} + static int dummy_sendtrigger (struct domain *d) { return 0; @@ -690,6 +695,7 @@ void xsm_fixup_ops (struct xsm_operations *ops) set_to_dummy_if_null(ops, mmu_machphys_update); set_to_dummy_if_null(ops, update_va_mapping); set_to_dummy_if_null(ops, add_to_physmap); + set_to_dummy_if_null(ops, remove_from_physmap); set_to_dummy_if_null(ops, sendtrigger); set_to_dummy_if_null(ops, bind_pt_irq); set_to_dummy_if_null(ops, pin_mem_cacheattr); diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c index 0d35767..a2020a9 100644 --- a/xen/xsm/flask/hooks.c +++ b/xen/xsm/flask/hooks.c @@ -1283,6 +1283,11 @@ static int flask_add_to_physmap(struct domain *d1, struct domain *d2) return domain_has_perm(d1, d2, SECCLASS_MMU, MMU__PHYSMAP); } +static int flask_remove_from_physmap(struct domain *d1, struct domain *d2) +{ + return domain_has_perm(d1, d2, SECCLASS_MMU, MMU__PHYSMAP); +} + static int flask_sendtrigger(struct domain *d) { return domain_has_perm(current->domain, d, SECCLASS_DOMAIN, DOMAIN__TRIGGER); @@ -1550,6 +1555,7 @@ static struct xsm_operations flask_ops = { .mmu_machphys_update = flask_mmu_machphys_update, .update_va_mapping = flask_update_va_mapping, .add_to_physmap = flask_add_to_physmap, + .remove_from_physmap = flask_remove_from_physmap, .sendtrigger = flask_sendtrigger, .get_device_group = flask_get_device_group, .test_assign_device = flask_test_assign_device, -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 02/18] xen: allow global VIRQ handlers to be delegated to other domains
This patch sends global VIRQs to a domain designated as the VIRQ handler instead of sending all global VIRQ events to dom0. This is required in order to run xenstored in a stubdom, because VIRQ_DOM_EXC must be sent to xenstored for domain destruction to work properly. This patch was inspired by the xenstored stubdomain patch series sent to xen-devel by Alex Zeffertt in 2009. Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/flask/policy/policy/flask/access_vectors | 1 + xen/arch/x86/cpu/mcheck/amd_nonfatal.c | 2 +- xen/arch/x86/cpu/mcheck/mce.c | 2 +- xen/arch/x86/cpu/mcheck/mce_intel.c | 6 +- xen/arch/x86/cpu/mcheck/non-fatal.c | 2 +- xen/common/cpu.c | 4 +- xen/common/domain.c | 10 ++-- xen/common/domctl.c | 17 ++++++ xen/common/event_channel.c | 63 +++++++++++++++++++++++- xen/common/trace.c | 2 +- xen/drivers/char/console.c | 4 +- xen/include/public/domctl.h | 8 +++ xen/include/xen/event.h | 18 ++++++- xen/include/xsm/xsm.h | 6 ++ xen/xsm/dummy.c | 6 ++ xen/xsm/flask/hooks.c | 6 ++ xen/xsm/flask/include/av_perm_to_string.h | 1 + xen/xsm/flask/include/av_permissions.h | 1 + 18 files changed, 140 insertions(+), 19 deletions(-) diff --git a/tools/flask/policy/policy/flask/access_vectors b/tools/flask/policy/policy/flask/access_vectors index 644f2e1..5901911 100644 --- a/tools/flask/policy/policy/flask/access_vectors +++ b/tools/flask/policy/policy/flask/access_vectors @@ -85,6 +85,7 @@ class domain getpodtarget setpodtarget set_misc_info + set_virq_handler } class hvm diff --git a/xen/arch/x86/cpu/mcheck/amd_nonfatal.c b/xen/arch/x86/cpu/mcheck/amd_nonfatal.c index 50288bd..9222098 100644 --- a/xen/arch/x86/cpu/mcheck/amd_nonfatal.c +++ b/xen/arch/x86/cpu/mcheck/amd_nonfatal.c @@ -100,7 +100,7 @@ static void mce_amd_checkregs(void *info) if (dom0_vmce_enabled()) { mctelem_commit(mctc); - send_guest_global_virq(dom0, VIRQ_MCA); + send_global_virq(VIRQ_MCA); } else if (++dumpcount >= 10) { x86_mcinfo_dump((struct mc_info *)mctelem_dataptr(mctc)); mctelem_dismiss(mctc); diff --git a/xen/arch/x86/cpu/mcheck/mce.c b/xen/arch/x86/cpu/mcheck/mce.c index b592041..c4e4477 100644 --- a/xen/arch/x86/cpu/mcheck/mce.c +++ b/xen/arch/x86/cpu/mcheck/mce.c @@ -594,7 +594,7 @@ void mcheck_cmn_handler(struct cpu_user_regs *regs, long error_code, if (dom0_vmce_enabled()) { if (mctc != NULL) mctelem_commit(mctc); - send_guest_global_virq(dom0, VIRQ_MCA); + send_global_virq(VIRQ_MCA); } else { x86_mcinfo_dump(mci); if (mctc != NULL) diff --git a/xen/arch/x86/cpu/mcheck/mce_intel.c b/xen/arch/x86/cpu/mcheck/mce_intel.c index 0986025..0894080 100644 --- a/xen/arch/x86/cpu/mcheck/mce_intel.c +++ b/xen/arch/x86/cpu/mcheck/mce_intel.c @@ -354,7 +354,7 @@ static void mce_softirq(void) /* Step2: Send Log to DOM0 through vIRQ */ if (dom0_vmce_enabled()) { mce_printk(MCE_VERBOSE, "MCE: send MCE# to DOM0 through virq\n"); - send_guest_global_virq(dom0, VIRQ_MCA); + send_global_virq(VIRQ_MCA); } } @@ -1085,7 +1085,7 @@ static void cmci_discover(void) if (bs.errcnt && mctc != NULL) { if (dom0_vmce_enabled()) { mctelem_commit(mctc); - send_guest_global_virq(dom0, VIRQ_MCA); + send_global_virq(VIRQ_MCA); } else { x86_mcinfo_dump(mctelem_dataptr(mctc)); mctelem_dismiss(mctc); @@ -1205,7 +1205,7 @@ fastcall void smp_cmci_interrupt(struct cpu_user_regs *regs) if (dom0_vmce_enabled()) { mctelem_commit(mctc); mce_printk(MCE_VERBOSE, "CMCI: send CMCI to DOM0 through virq\n"); - send_guest_global_virq(dom0, VIRQ_MCA); + send_global_virq(VIRQ_MCA); } else { x86_mcinfo_dump(mctelem_dataptr(mctc)); mctelem_dismiss(mctc); diff --git a/xen/arch/x86/cpu/mcheck/non-fatal.c b/xen/arch/x86/cpu/mcheck/non-fatal.c index c57688f..1dded9b 100644 --- a/xen/arch/x86/cpu/mcheck/non-fatal.c +++ b/xen/arch/x86/cpu/mcheck/non-fatal.c @@ -55,7 +55,7 @@ static void mce_checkregs (void *info) if (dom0_vmce_enabled()) { mctelem_commit(mctc); - send_guest_global_virq(dom0, VIRQ_MCA); + send_global_virq(VIRQ_MCA); } else if (++dumpcount >= 10) { x86_mcinfo_dump((struct mc_info *)mctelem_dataptr(mctc)); mctelem_dismiss(mctc); diff --git a/xen/common/cpu.c b/xen/common/cpu.c index 79abdb7..630881e 100644 --- a/xen/common/cpu.c +++ b/xen/common/cpu.c @@ -108,7 +108,7 @@ int cpu_down(unsigned int cpu) notifier_rc = notifier_call_chain(&cpu_chain, CPU_DEAD, hcpu, NULL); BUG_ON(notifier_rc != NOTIFY_DONE); - send_guest_global_virq(dom0, VIRQ_PCPU_STATE); + send_global_virq(VIRQ_PCPU_STATE); cpu_hotplug_done(); return 0; @@ -148,7 +148,7 @@ int cpu_up(unsigned int cpu) notifier_rc = notifier_call_chain(&cpu_chain, CPU_ONLINE, hcpu, NULL); BUG_ON(notifier_rc != NOTIFY_DONE); - send_guest_global_virq(dom0, VIRQ_PCPU_STATE); + send_global_virq(VIRQ_PCPU_STATE); cpu_hotplug_done(); return 0; diff --git a/xen/common/domain.c b/xen/common/domain.c index 52a63ef..c4d98d9 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -116,7 +116,7 @@ static void __domain_finalise_shutdown(struct domain *d) if ( (d->shutdown_code == SHUTDOWN_suspend) && d->suspend_evtchn ) evtchn_send(d, d->suspend_evtchn); else - send_guest_global_virq(dom0, VIRQ_DOM_EXC); + send_global_virq(VIRQ_DOM_EXC); } static void vcpu_check_shutdown(struct vcpu *v) @@ -492,7 +492,7 @@ int domain_kill(struct domain *d) } d->is_dying = DOMDYING_dead; put_domain(d); - send_guest_global_virq(dom0, VIRQ_DOM_EXC); + send_global_virq(VIRQ_DOM_EXC); /* fallthrough */ case DOMDYING_dead: break; @@ -633,7 +633,7 @@ void domain_pause_for_debugger(void) for_each_vcpu ( d, v ) vcpu_sleep_nosync(v); - send_guest_global_virq(dom0, VIRQ_DEBUGGER); + send_global_virq(VIRQ_DEBUGGER); } /* Complete domain destroy after RCU readers are not holding old references. */ @@ -659,6 +659,8 @@ static void complete_domain_destroy(struct rcu_head *head) watchdog_domain_destroy(d); + clear_global_virq_handlers(d); + rangeset_domain_destroy(d); cpupool_rm_domain(d); @@ -690,7 +692,7 @@ static void complete_domain_destroy(struct rcu_head *head) free_cpumask_var(d->domain_dirty_cpumask); free_domain_struct(d); - send_guest_global_virq(dom0, VIRQ_DOM_EXC); + send_global_virq(VIRQ_DOM_EXC); } /* Release resources belonging to task @p. */ diff --git a/xen/common/domctl.c b/xen/common/domctl.c index d6ae09b..a775aa3 100644 --- a/xen/common/domctl.c +++ b/xen/common/domctl.c @@ -994,6 +994,23 @@ long do_domctl(XEN_GUEST_HANDLE(xen_domctl_t) u_domctl) } break; + case XEN_DOMCTL_set_virq_handler: + { + struct domain *d; + int virq = op->u.set_virq_handler.virq; + + ret = -ESRCH; + d = rcu_lock_domain_by_id(op->domain); + if ( d != NULL ) + { + ret = xsm_set_virq_handler(d, virq); + if ( !ret ) + ret = set_global_virq_handler(d, virq); + rcu_unlock_domain(d); + } + } + break; + default: ret = arch_do_domctl(op, u_domctl); break; diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c index 9212042..77c7a27 100644 --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -689,7 +689,7 @@ void send_guest_vcpu_virq(struct vcpu *v, int virq) spin_unlock_irqrestore(&v->virq_lock, flags); } -void send_guest_global_virq(struct domain *d, int virq) +static void send_guest_global_virq(struct domain *d, int virq) { unsigned long flags; int port; @@ -739,6 +739,67 @@ int send_guest_pirq(struct domain *d, const struct pirq *pirq) return evtchn_set_pending(d->vcpu[chn->notify_vcpu_id], port); } +static struct domain* global_virq_handlers[NR_VIRQS]; + +spinlock_t global_virq_handlers_lock = SPIN_LOCK_UNLOCKED; + +static struct domain* _get_global_virq_handler(int virq) +{ + struct domain *d; + + d = global_virq_handlers[virq]; + return d != NULL ? d : dom0; +} + +void send_global_virq(int virq) +{ + ASSERT(virq >= 0 && virq < NR_VIRQS); + ASSERT(virq_is_global(virq)); + + send_guest_global_virq(_get_global_virq_handler(virq), virq); +} + +int set_global_virq_handler(struct domain *d, int virq) +{ + struct domain *old; + + if (virq < 0 || virq >= NR_VIRQS) + return -EINVAL; + if (!virq_is_global(virq)) + return -EINVAL; + + if (global_virq_handlers[virq] == d) + return 0; + + if (unlikely(!get_domain(d))) + return -EINVAL; + + spin_lock(&global_virq_handlers_lock); + + old = global_virq_handlers[virq]; + global_virq_handlers[virq] = d; + if (old != NULL) + put_domain(old); + spin_unlock(&global_virq_handlers_lock); + + return 0; +} + +void clear_global_virq_handlers(struct domain *d) +{ + int virq; + + spin_lock(&global_virq_handlers_lock); + + for (virq = 0; virq < NR_VIRQS; virq++) { + if (global_virq_handlers[virq] == d) { + global_virq_handlers[virq] = NULL; + put_domain(d); + } + } + + spin_unlock(&global_virq_handlers_lock); +} static long evtchn_status(evtchn_status_t *status) { diff --git a/xen/common/trace.c b/xen/common/trace.c index 5772f24..58cbf39 100644 --- a/xen/common/trace.c +++ b/xen/common/trace.c @@ -661,7 +661,7 @@ static inline void insert_lost_records(struct t_buf *buf) */ static void trace_notify_dom0(unsigned long unused) { - send_guest_global_virq(dom0, VIRQ_TBUF); + send_global_virq(VIRQ_TBUF); } static DECLARE_SOFTIRQ_TASKLET(trace_notify_dom0_tasklet, trace_notify_dom0, 0); diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c index 8a4c684..79b266f 100644 --- a/xen/drivers/char/console.c +++ b/xen/drivers/char/console.c @@ -287,7 +287,7 @@ static void __serial_rx(char c, struct cpu_user_regs *regs) if ( (serial_rx_prod-serial_rx_cons) != SERIAL_RX_SIZE ) serial_rx_ring[SERIAL_RX_MASK(serial_rx_prod++)] = c; /* Always notify the guest: prevents receive path from getting stuck. */ - send_guest_global_virq(dom0, VIRQ_CONSOLE); + send_global_virq(VIRQ_CONSOLE); } static void serial_rx(char c, struct cpu_user_regs *regs) @@ -314,7 +314,7 @@ static void serial_rx(char c, struct cpu_user_regs *regs) static void notify_dom0_con_ring(unsigned long unused) { - send_guest_global_virq(dom0, VIRQ_CON_RING); + send_global_virq(VIRQ_CON_RING); } static DECLARE_SOFTIRQ_TASKLET(notify_dom0_con_ring_tasklet, notify_dom0_con_ring, 0); diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index c7640aa..b1eb425 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -813,6 +813,12 @@ struct xen_domctl_audit_p2m { typedef struct xen_domctl_audit_p2m xen_domctl_audit_p2m_t; DEFINE_XEN_GUEST_HANDLE(xen_domctl_audit_p2m_t); +struct xen_domctl_set_virq_handler { + uint32_t virq; /* IN */ +}; +typedef struct xen_domctl_set_virq_handler xen_domctl_set_virq_handler_t; +DEFINE_XEN_GUEST_HANDLE(xen_domctl_set_virq_handler_t); + #if defined(__i386__) || defined(__x86_64__) /* XEN_DOMCTL_setvcpuextstate */ /* XEN_DOMCTL_getvcpuextstate */ @@ -912,6 +918,7 @@ struct xen_domctl { #define XEN_DOMCTL_getvcpuextstate 63 #define XEN_DOMCTL_set_access_required 64 #define XEN_DOMCTL_audit_p2m 65 +#define XEN_DOMCTL_set_virq_handler 71 #define XEN_DOMCTL_gdbsx_guestmemio 1000 #define XEN_DOMCTL_gdbsx_pausevcpu 1001 #define XEN_DOMCTL_gdbsx_unpausevcpu 1002 @@ -966,6 +973,7 @@ struct xen_domctl { #endif struct xen_domctl_set_access_required access_required; struct xen_domctl_audit_p2m audit_p2m; + struct xen_domctl_set_virq_handler set_virq_handler; struct xen_domctl_gdbsx_memio gdbsx_guest_memio; struct xen_domctl_gdbsx_pauseunp_vcpu gdbsx_pauseunp_vcpu; struct xen_domctl_gdbsx_domstatus gdbsx_domstatus; diff --git a/xen/include/xen/event.h b/xen/include/xen/event.h index 7e5ad7b..2df65a0 100644 --- a/xen/include/xen/event.h +++ b/xen/include/xen/event.h @@ -24,11 +24,23 @@ void send_guest_vcpu_virq(struct vcpu *v, int virq); /* - * send_guest_global_virq: Notify guest via a global VIRQ. - * @d: Domain to which virtual IRQ should be sent + * send_global_virq: Notify the domain handling a global VIRQ. * @virq: Virtual IRQ number (VIRQ_*) */ -void send_guest_global_virq(struct domain *d, int virq); +void send_global_virq(int virq); + +/* + * sent_global_virq_handler: Set a global VIRQ handler. + * @d: New target domain for this VIRQ + * @virq: Virtual IRQ number (VIRQ_*), must be global + */ +int set_global_virq_handler(struct domain *d, int virq); + +/* + * clear_global_virq_handlers: Remove a domain as a handler for global VIRQs. + * @d: Domain to no longer handle global virtual IRQs + */ +void clear_global_virq_handlers(struct domain *d); /* * send_guest_pirq: diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h index 566c808..c89c6ed 100644 --- a/xen/include/xsm/xsm.h +++ b/xen/include/xsm/xsm.h @@ -64,6 +64,7 @@ struct xsm_operations { int (*domain_settime) (struct domain *d); int (*set_target) (struct domain *d, struct domain *e); int (*domctl) (struct domain *d, int cmd); + int (*set_virq_handler) (struct domain *d, int virq); int (*tbufcontrol) (void); int (*readconsole) (uint32_t clear); int (*sched_id) (void); @@ -265,6 +266,11 @@ static inline int xsm_domctl (struct domain *d, int cmd) return xsm_call(domctl(d, cmd)); } +static inline int xsm_set_virq_handler (struct domain *d, int virq) +{ + return xsm_call(set_virq_handler(d, virq)); +} + static inline int xsm_tbufcontrol (void) { return xsm_call(tbufcontrol()); diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c index 65daa4e..59db86d 100644 --- a/xen/xsm/dummy.c +++ b/xen/xsm/dummy.c @@ -94,6 +94,11 @@ static int dummy_domctl(struct domain *d, int cmd) return 0; } +static int dummy_set_virq_handler(struct domain *d, int virq) +{ + return 0; +} + static int dummy_tbufcontrol (void) { return 0; @@ -596,6 +601,7 @@ void xsm_fixup_ops (struct xsm_operations *ops) set_to_dummy_if_null(ops, domain_settime); set_to_dummy_if_null(ops, set_target); set_to_dummy_if_null(ops, domctl); + set_to_dummy_if_null(ops, set_virq_handler); set_to_dummy_if_null(ops, tbufcontrol); set_to_dummy_if_null(ops, readconsole); set_to_dummy_if_null(ops, sched_id); diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c index a2020a9..a1feb8d 100644 --- a/xen/xsm/flask/hooks.c +++ b/xen/xsm/flask/hooks.c @@ -597,6 +597,11 @@ static int flask_domctl(struct domain *d, int cmd) return domain_has_perm(current->domain, d, SECCLASS_DOMAIN, DOMAIN__SET_MISC_INFO); } +static int flask_set_virq_handler(struct domain *d, int virq) +{ + return domain_has_perm(current->domain, d, SECCLASS_DOMAIN, DOMAIN__SET_VIRQ_HANDLER); +} + static int flask_tbufcontrol(void) { return domain_has_xen(current->domain, XEN__TBUFCONTROL); @@ -1460,6 +1465,7 @@ static struct xsm_operations flask_ops = { .domain_settime = flask_domain_settime, .set_target = flask_set_target, .domctl = flask_domctl, + .set_virq_handler = flask_set_virq_handler, .tbufcontrol = flask_tbufcontrol, .readconsole = flask_readconsole, .sched_id = flask_sched_id, diff --git a/xen/xsm/flask/include/av_perm_to_string.h b/xen/xsm/flask/include/av_perm_to_string.h index 85cbffc..17a1c36 100644 --- a/xen/xsm/flask/include/av_perm_to_string.h +++ b/xen/xsm/flask/include/av_perm_to_string.h @@ -60,6 +60,7 @@ S_(SECCLASS_DOMAIN, DOMAIN__GETPODTARGET, "getpodtarget") S_(SECCLASS_DOMAIN, DOMAIN__SETPODTARGET, "setpodtarget") S_(SECCLASS_DOMAIN, DOMAIN__SET_MISC_INFO, "set_misc_info") + S_(SECCLASS_DOMAIN, DOMAIN__SET_VIRQ_HANDLER, "set_virq_handler") S_(SECCLASS_HVM, HVM__SETHVMC, "sethvmc") S_(SECCLASS_HVM, HVM__GETHVMC, "gethvmc") S_(SECCLASS_HVM, HVM__SETPARAM, "setparam") diff --git a/xen/xsm/flask/include/av_permissions.h b/xen/xsm/flask/include/av_permissions.h index 9e55a86..42eaf81 100644 --- a/xen/xsm/flask/include/av_permissions.h +++ b/xen/xsm/flask/include/av_permissions.h @@ -61,6 +61,7 @@ #define DOMAIN__GETPODTARGET 0x10000000UL #define DOMAIN__SETPODTARGET 0x20000000UL #define DOMAIN__SET_MISC_INFO 0x40000000UL +#define DOMAIN__SET_VIRQ_HANDLER 0x80000000UL #define HVM__SETHVMC 0x00000001UL #define HVM__GETHVMC 0x00000002UL -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 03/18] xsm: allow use of XEN_DOMCTL_getdomaininfo by non-IS_PRIV domains
This domctl does not allow manipulation of domains, only basic information such as size and state. XSM modules can also provide fine-grained control over what domains are visible to domains that call getdomaininfo. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- xen/common/domctl.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/xen/common/domctl.c b/xen/common/domctl.c index a775aa3..2c1ca85 100644 --- a/xen/common/domctl.c +++ b/xen/common/domctl.c @@ -263,6 +263,10 @@ long do_domctl(XEN_GUEST_HANDLE(xen_domctl_t) u_domctl) return -EPERM; break; } +#ifdef XSM_ENABLE + case XEN_DOMCTL_getdomaininfo: + break; +#endif default: if ( !IS_PRIV(current->domain) ) return -EPERM; -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 04/18] xen: Preserve reserved grant entries when switching versions
In order for the toolstack to use reserved grant table entries, the grant table for a guest must be initialized prior to the guest''s boot. When the guest switches grant table versions (necessary if the guest is using v2 grant tables, or on kexec if switching grant versions), these initial grants will be cleared. Instead of clearing them, preserve the grants across the type change. Attempting to use or preserve v2-only features such as sub-page grants will produce invalid v1 grant entries, which (while they will not work) is not a problem since a guest can always produce such invalid entries. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- xen/common/grant_table.c | 38 +++++++++++++++++++++++++++++++------- 1 files changed, 31 insertions(+), 7 deletions(-) diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c index 014734d..8d4a4cb 100644 --- a/xen/common/grant_table.c +++ b/xen/common/grant_table.c @@ -2106,6 +2106,7 @@ gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) struct domain *d = current->domain; struct grant_table *gt = d->grant_table; struct active_grant_entry *act; + grant_entry_v1_t reserved_entries[GNTTAB_NR_RESERVED_ENTRIES]; long res; int i; @@ -2126,7 +2127,7 @@ gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) /* (You need to change the version number for e.g. kexec.) */ if ( gt->gt_version != 0 ) { - for ( i = 0; i < nr_grant_entries(gt); i++ ) + for ( i = GNTTAB_NR_RESERVED_ENTRIES; i < nr_grant_entries(gt); i++ ) { act = &active_entry(gt, i); if ( act->pin != 0 ) @@ -2151,15 +2152,38 @@ gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) goto out_unlock; } + /* Preserve the first 8 entries (toolstack reserved grants) */ + if (gt->gt_version == 1) { + memcpy(reserved_entries, gt->shared_v1[0], sizeof(reserved_entries)); + } else if (gt->gt_version == 2) { + for ( i = 0; i < GNTTAB_NR_RESERVED_ENTRIES && i < nr_grant_entries(gt); i++ ) + { + reserved_entries[i].flags = shared_entry_v2(gt, i).hdr.flags; + reserved_entries[i].domid = shared_entry_v2(gt, i).hdr.domid; + reserved_entries[i].frame = shared_entry_v2(gt, i).full_page.frame; + reserved_entries[i].flags |= status_entry(gt, i); + } + } + if ( op.version < 2 && gt->gt_version == 2 ) gnttab_unpopulate_status_frames(d, gt); - if ( op.version != gt->gt_version ) - { - /* Make sure there''s no crud left over in the table from the - old version. */ - for ( i = 0; i < nr_grant_frames(gt); i++ ) - memset(gt->shared_raw[i], 0, PAGE_SIZE); + /* Make sure there''s no crud left over in the table from the + old version. */ + for ( i = 0; i < nr_grant_frames(gt); i++ ) + memset(gt->shared_raw[i], 0, PAGE_SIZE); + + /* Restore the first 8 entries (toolstack reserved grants) */ + if (gt->gt_version != 0 && op.version == 1) { + memcpy(gt->shared_v1[0], reserved_entries, sizeof(reserved_entries)); + } else if (gt->gt_version != 0 && op.version == 2) { + for ( i = 0; i < GNTTAB_NR_RESERVED_ENTRIES; i++ ) + { + status_entry(gt, i) = reserved_entries[i].flags & (GTF_reading|GTF_writing); + shared_entry_v2(gt, i).hdr.flags = reserved_entries[i].flags & ~(GTF_reading|GTF_writing); + shared_entry_v2(gt, i).hdr.domid = reserved_entries[i].domid; + shared_entry_v2(gt, i).full_page.frame = reserved_entries[i].frame; + } } gt->gt_version = op.version; -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 05/18] tools/libxl: Add xenstore and console backend domain IDs to config
This changes the event channels set up in the start_info structure of newly built domains to be associated with a domain other than dom0, to support xenstored or xenconsoled running in stub domains. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/libxl/libxl_dom.c | 4 ++-- tools/libxl/libxl_types.idl | 2 ++ tools/libxl/xl_cmdimpl.c | 6 ++++++ 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c index a4725fe..e12d687 100644 --- a/tools/libxl/libxl_dom.c +++ b/tools/libxl/libxl_dom.c @@ -104,8 +104,8 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid, xc_shadow_control(ctx->xch, domid, XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION, NULL, 0, &shadow, 0, NULL); } - state->store_port = xc_evtchn_alloc_unbound(ctx->xch, domid, 0); - state->console_port = xc_evtchn_alloc_unbound(ctx->xch, domid, 0); + state->store_port = xc_evtchn_alloc_unbound(ctx->xch, domid, info->xenstore_dom); + state->console_port = xc_evtchn_alloc_unbound(ctx->xch, domid, info->console_dom); state->vm_generationid_addr = 0; return 0; } diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index 574dec7..013f5c6 100644 --- a/tools/libxl/libxl_types.idl +++ b/tools/libxl/libxl_types.idl @@ -168,6 +168,8 @@ libxl_domain_build_info = Struct("domain_build_info",[ ("video_memkb", uint32), ("shadow_memkb", uint32), ("disable_migrate", bool), + ("xenstore_dom", uint32), + ("console_dom", uint32), ("cpuid", libxl_cpuid_policy_list), ("type", libxl_domain_type), ("u", KeyedUnion(None, libxl_domain_type, "type", diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c index 8c30de1..bb1bb5b 100644 --- a/tools/libxl/xl_cmdimpl.c +++ b/tools/libxl/xl_cmdimpl.c @@ -661,6 +661,12 @@ static void parse_config_data(const char *configfile_filename_report, if (!xlu_cfg_get_long (config, "maxvcpus", &l, 0)) b_info->max_vcpus = l; + if (!xlu_cfg_get_long (config, "xenstore_dom", &l, 0)) + b_info->xenstore_dom = l; + + if (!xlu_cfg_get_long (config, "console_dom", &l, 0)) + b_info->console_dom = l; + if (!xlu_cfg_get_long (config, "memory", &l, 0)) { b_info->max_memkb = l * 1024; b_info->target_memkb = b_info->max_memkb; -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants
From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> This patch claims one reserved grant entry for the console and another for the xenstore. It modifies the builder to fill in the grant table entries for the console and the xenstore. This has not been tested with any kind of migration. Previous versions of this patch have been sent to xen-devel. See http://lists.xensource.com/archives/html/xen-devel/2008-07/msg00610.html http://lists.xensource.com/archives/html/xen-devel/2009-03/msg01491.html Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/libxc/xc_dom.h | 13 +++ tools/libxc/xc_dom_boot.c | 164 +++++++++++++++++++++++++++++++++++++ tools/libxc/xc_dom_compat_linux.c | 2 + tools/libxc/xc_domain_restore.c | 17 ++++ tools/libxc/xc_hvm_build.c | 1 + tools/libxl/libxl_dom.c | 17 ++++- tools/libxl/libxl_internal.h | 2 + xen/include/public/grant_table.h | 6 ++ 8 files changed, 220 insertions(+), 2 deletions(-) diff --git a/tools/libxc/xc_dom.h b/tools/libxc/xc_dom.h index e72f066..6c36403 100644 --- a/tools/libxc/xc_dom.h +++ b/tools/libxc/xc_dom.h @@ -106,7 +106,9 @@ struct xc_dom_image { /* misc xen domain config stuff */ unsigned long flags; unsigned int console_evtchn; + unsigned int console_domid; unsigned int xenstore_evtchn; + unsigned int xenstore_domid; xen_pfn_t shared_info_mfn; xc_interface *xch; @@ -200,6 +202,17 @@ void *xc_dom_boot_domU_map(struct xc_dom_image *dom, xen_pfn_t pfn, xen_pfn_t count); int xc_dom_boot_image(struct xc_dom_image *dom); int xc_dom_compat_check(struct xc_dom_image *dom); +int xc_dom_gnttab_init(struct xc_dom_image *dom); +int xc_dom_gnttab_hvm_seed(xc_interface *xch, uint32_t domid, + unsigned long console_gmfn, + unsigned long xenstore_gmfn, + uint32_t console_domid, + uint32_t xenstore_domid); +int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid, + unsigned long console_gmfn, + unsigned long xenstore_gmfn, + uint32_t console_domid, + uint32_t xenstore_domid); /* --- debugging bits ---------------------------------------------- */ diff --git a/tools/libxc/xc_dom_boot.c b/tools/libxc/xc_dom_boot.c index 65f60df..1a55a6d 100644 --- a/tools/libxc/xc_dom_boot.c +++ b/tools/libxc/xc_dom_boot.c @@ -34,6 +34,7 @@ #include "xg_private.h" #include "xc_dom.h" #include <xen/hvm/params.h> +#include <xen/grant_table.h> /* ------------------------------------------------------------------------ */ @@ -275,6 +276,169 @@ int xc_dom_boot_image(struct xc_dom_image *dom) return rc; } +static unsigned long xc_dom_gnttab_setup(xc_interface *xch, uint32_t domid) +{ + DECLARE_HYPERCALL; + gnttab_setup_table_t setup_table; + DECLARE_HYPERCALL_BUFFER(unsigned long, gmfnp); + int rc; + unsigned long gmfn; + + gmfnp = xc_hypercall_buffer_alloc(xch, gmfnp, sizeof(*gmfnp)); + if (gmfnp == NULL) + return -1; + + setup_table.dom = domid; + setup_table.nr_frames = 1; + set_xen_guest_handle(setup_table.frame_list, gmfnp); + setup_table.status = 0; + + hypercall.op = __HYPERVISOR_grant_table_op; + hypercall.arg[0] = GNTTABOP_setup_table; + hypercall.arg[1] = (unsigned long) &setup_table; + hypercall.arg[2] = 1; + + rc = do_xen_hypercall(xch, &hypercall); + gmfn = *gmfnp; + xc_hypercall_buffer_free(xch, gmfnp); + + if ( rc != 0 || setup_table.status != GNTST_okay ) + { + xc_dom_panic(xch, XC_INTERNAL_ERROR, + "%s: failed to setup domU grant table " + "[errno=%d, status=%" PRId16 "]\n", + __FUNCTION__, rc != 0 ? errno : 0, setup_table.status); + return -1; + } + + return gmfn; +} + +int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid, + unsigned long console_gmfn, + unsigned long xenstore_gmfn, + uint32_t console_domid, + uint32_t xenstore_domid) +{ + + unsigned long gnttab_gmfn; + grant_entry_v1_t *gnttab; + + gnttab_gmfn = xc_dom_gnttab_setup(xch, domid); + if ( gnttab_gmfn == -1 ) + return -1; + + gnttab = xc_map_foreign_range(xch, + domid, + PAGE_SIZE, + PROT_READ|PROT_WRITE, + gnttab_gmfn); + if ( gnttab == NULL ) + { + xc_dom_panic(xch, XC_INTERNAL_ERROR, + "%s: failed to map domU grant table " + "[errno=%d]\n", + __FUNCTION__, errno); + return -1; + } + + if ( domid != console_domid && console_gmfn != -1) + { + gnttab[GNTTAB_RESERVED_CONSOLE].flags = GTF_permit_access; + gnttab[GNTTAB_RESERVED_CONSOLE].domid = console_domid; + gnttab[GNTTAB_RESERVED_CONSOLE].frame = console_gmfn; + } + if ( domid != xenstore_domid && xenstore_gmfn != -1) + { + gnttab[GNTTAB_RESERVED_XENSTORE].flags = GTF_permit_access; + gnttab[GNTTAB_RESERVED_XENSTORE].domid = xenstore_domid; + gnttab[GNTTAB_RESERVED_XENSTORE].frame = xenstore_gmfn; + } + + if ( munmap(gnttab, PAGE_SIZE) == -1 ) + { + xc_dom_panic(xch, XC_INTERNAL_ERROR, + "%s: failed to unmap domU grant table " + "[errno=%d]\n", + __FUNCTION__, errno); + return -1; + } + + return 0; +} + +int xc_dom_gnttab_hvm_seed(xc_interface *xch, uint32_t domid, + unsigned long console_gpfn, + unsigned long xenstore_gpfn, + uint32_t console_domid, + uint32_t xenstore_domid) +{ +#define SCRATCH_PFN_GNTTAB 0xFFFFE + + int rc; + struct xen_add_to_physmap xatp = { + .domid = domid, + .space = XENMAPSPACE_grant_table, + .idx = 0, /* TODO: what''s this? */ + .gpfn = SCRATCH_PFN_GNTTAB + }; + struct xen_remove_from_physmap xrfp = { + .domid = domid, + .gpfn = SCRATCH_PFN_GNTTAB + }; + + rc = do_memory_op(xch, XENMEM_add_to_physmap, &xatp, sizeof(xatp)); + if ( rc != 0 ) + { + xc_dom_panic(xch, XC_INTERNAL_ERROR, + "%s: failed to add gnttab to physmap " + "[errno=%d]\n", + __FUNCTION__, errno); + return -1; + } + + rc = xc_dom_gnttab_seed(xch, domid, + console_gpfn, xenstore_gpfn, + console_domid, xenstore_domid); + if (rc != 0) + { + xc_dom_panic(xch, XC_INTERNAL_ERROR, + "%s: failed to seed gnttab entries\n", + __FUNCTION__); + (void) do_memory_op(xch, XENMEM_remove_from_physmap, &xrfp, sizeof(xrfp)); + return -1; + } + + rc = do_memory_op(xch, XENMEM_remove_from_physmap, &xrfp, sizeof(xrfp)); + if (rc != 0) + { + xc_dom_panic(xch, XC_INTERNAL_ERROR, + "%s: failed to remove gnttab from physmap " + "[errno=%d]\n", + __FUNCTION__, errno); + return -1; + } + + return 0; +} + +int xc_dom_gnttab_init(struct xc_dom_image *dom) +{ + unsigned long console_gmfn; + unsigned long xenstore_gmfn; + int autotranslated; + + autotranslated = xc_dom_feature_translated(dom); + console_gmfn = autotranslated ? + dom->console_pfn : xc_dom_p2m_host(dom, dom->console_pfn); + xenstore_gmfn = autotranslated ? + dom->xenstore_pfn : xc_dom_p2m_host(dom, dom->xenstore_pfn); + + return xc_dom_gnttab_seed(dom->xch, dom->guest_domid, + console_gmfn, xenstore_gmfn, + dom->console_domid, dom->xenstore_domid); +} + /* * Local variables: * mode: C diff --git a/tools/libxc/xc_dom_compat_linux.c b/tools/libxc/xc_dom_compat_linux.c index 0e78842..2183a3b 100644 --- a/tools/libxc/xc_dom_compat_linux.c +++ b/tools/libxc/xc_dom_compat_linux.c @@ -62,6 +62,8 @@ static int xc_linux_build_internal(struct xc_dom_image *dom, goto out; if ( (rc = xc_dom_boot_image(dom)) != 0 ) goto out; + if ( (rc = xc_dom_gnttab_init(dom)) != 0) + goto out; *console_mfn = xc_dom_p2m_host(dom, dom->console_pfn); *store_mfn = xc_dom_p2m_host(dom, dom->xenstore_pfn); diff --git a/tools/libxc/xc_domain_restore.c b/tools/libxc/xc_domain_restore.c index 3fda6f8..23619da 100644 --- a/tools/libxc/xc_domain_restore.c +++ b/tools/libxc/xc_domain_restore.c @@ -2018,6 +2018,15 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, memcpy(ctx->live_p2m, ctx->p2m, dinfo->p2m_size * sizeof(xen_pfn_t)); munmap(ctx->live_p2m, P2M_FL_ENTRIES * PAGE_SIZE); +/* TODO don''t hardcode zero here */ + rc = xc_dom_gnttab_seed(xch, dom, + *console_mfn, *store_mfn, 0, 0); + if (rc != 0) + { + ERROR("error seeding grant table"); + goto out; + } + DPRINTF("Domain ready to be built.\n"); rc = 0; goto out; @@ -2076,6 +2085,14 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, goto out; } + /* TODO don''t hardcode zero here */ + rc = xc_dom_gnttab_hvm_seed(xch, dom, *console_mfn, *store_mfn, 0, 0); + if (rc != 0) + { + ERROR("error seeding grant table"); + goto out; + } + /* HVM success! */ rc = 0; diff --git a/tools/libxc/xc_hvm_build.c b/tools/libxc/xc_hvm_build.c index 9831bab..453f8a5 100644 --- a/tools/libxc/xc_hvm_build.c +++ b/tools/libxc/xc_hvm_build.c @@ -24,6 +24,7 @@ #include "xg_private.h" #include "xc_private.h" +#include "xc_dom.h" #include <xen/foreign/x86_32.h> #include <xen/foreign/x86_64.h> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c index e12d687..7f7bca6 100644 --- a/tools/libxl/libxl_dom.c +++ b/tools/libxl/libxl_dom.c @@ -104,7 +104,9 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid, xc_shadow_control(ctx->xch, domid, XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION, NULL, 0, &shadow, 0, NULL); } + state->store_domid = info->xenstore_dom; state->store_port = xc_evtchn_alloc_unbound(ctx->xch, domid, info->xenstore_dom); + state->console_domid = info->console_dom; state->console_port = xc_evtchn_alloc_unbound(ctx->xch, domid, info->console_dom); state->vm_generationid_addr = 0; return 0; @@ -223,7 +225,9 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid, dom->flags = flags; dom->console_evtchn = state->console_port; + dom->console_domid = state->console_domid; dom->xenstore_evtchn = state->store_port; + dom->xenstore_domid = state->store_domid; if ( (ret = xc_dom_boot_xen_init(dom, ctx->xch, domid)) != 0 ) { LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "xc_dom_boot_xen_init failed"); @@ -249,6 +253,10 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid, LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "xc_dom_boot_image failed"); goto out; } + if ( (ret = xc_dom_gnttab_init(dom)) != 0 ) { + LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "xc_dom_gnttab_init failed"); + goto out; + } state->console_mfn = xc_dom_p2m_host(dom, dom->console_pfn); state->store_mfn = xc_dom_p2m_host(dom, dom->xenstore_pfn); @@ -262,7 +270,8 @@ out: static int hvm_build_set_params(xc_interface *handle, uint32_t domid, libxl_domain_build_info *info, int store_evtchn, unsigned long *store_mfn, - int console_evtchn, unsigned long *console_mfn) + int console_evtchn, unsigned long *console_mfn, + uint32_t store_domid, uint32_t console_domid) { struct hvm_info_table *va_hvm; uint8_t *va_map, sum; @@ -295,6 +304,8 @@ static int hvm_build_set_params(xc_interface *handle, uint32_t domid, xc_set_hvm_param(handle, domid, HVM_PARAM_NESTEDHVM, info->u.hvm.nested_hvm); xc_set_hvm_param(handle, domid, HVM_PARAM_STORE_EVTCHN, store_evtchn); xc_set_hvm_param(handle, domid, HVM_PARAM_CONSOLE_EVTCHN, console_evtchn); + + xc_dom_gnttab_hvm_seed(handle, domid, *console_mfn, *store_mfn, console_domid, store_domid); return 0; } @@ -348,7 +359,9 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid, goto out; } ret = hvm_build_set_params(ctx->xch, domid, info, state->store_port, - &state->store_mfn, state->console_port, &state->console_mfn); + &state->store_mfn, state->console_port, + &state->console_mfn, state->store_domid, + state->console_domid); if (ret) { LIBXL__LOG_ERRNOVAL(ctx, LIBXL__LOG_ERROR, ret, "hvm build set params failed"); goto out; diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index 288c03c..97ead08 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -219,9 +219,11 @@ _hidden int libxl__domain_shutdown_reason(libxl__gc *gc, uint32_t domid); libxl__domain_type((gc), (domid)) == LIBXL_DOMAIN_TYPE_##type typedef struct { uint32_t store_port; + uint32_t store_domid; unsigned long store_mfn; uint32_t console_port; + uint32_t console_domid; unsigned long console_mfn; unsigned long vm_generationid_addr; } libxl__domain_build_state; diff --git a/xen/include/public/grant_table.h b/xen/include/public/grant_table.h index 0bf20bc..36d1ac7 100644 --- a/xen/include/public/grant_table.h +++ b/xen/include/public/grant_table.h @@ -117,6 +117,12 @@ struct grant_entry_v1 { }; typedef struct grant_entry_v1 grant_entry_v1_t; +/* External tools reserve first few grant table entries. */ +#define GNTTAB_NR_RESERVED_ENTRIES 8 +#define GNTTAB_RESERVED_CONSOLE 0 +#define GNTTAB_RESERVED_XENSTORE 1 +/* (the next 6 are reserved for future use) */ + /* * Type of grant entry. * GTF_invalid: This grant entry grants no privileges. -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 07/18] mini-os: avoid crash if no console is provided
Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- extras/mini-os/console/xencons_ring.c | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/extras/mini-os/console/xencons_ring.c b/extras/mini-os/console/xencons_ring.c index 22fd618..14a8bd1 100644 --- a/extras/mini-os/console/xencons_ring.c +++ b/extras/mini-os/console/xencons_ring.c @@ -25,7 +25,10 @@ static inline void notify_daemon(struct consfront_dev *dev) static inline struct xencons_interface *xencons_interface(void) { - return mfn_to_virt(start_info.console.domU.mfn); + if (start_info.console.domU.evtchn) + return mfn_to_virt(start_info.console.domU.mfn); + else + return NULL; } int xencons_ring_send_no_notify(struct consfront_dev *dev, const char *data, unsigned len) @@ -38,6 +41,8 @@ int xencons_ring_send_no_notify(struct consfront_dev *dev, const char *data, uns intf = xencons_interface(); else intf = dev->ring; + if (!intf) + return sent; cons = intf->out_cons; prod = intf->out_prod; -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 08/18] mini-os: avoid crash if no xenstore is provided
Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- extras/mini-os/xenbus/xenbus.c | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-) diff --git a/extras/mini-os/xenbus/xenbus.c b/extras/mini-os/xenbus/xenbus.c index a8081fd..e475e2c 100644 --- a/extras/mini-os/xenbus/xenbus.c +++ b/extras/mini-os/xenbus/xenbus.c @@ -328,6 +328,10 @@ static int allocate_xenbus_id(void) void init_xenbus(void) { int err; + if (!start_info.store_evtchn) { + printk("Skipping initialization of xenbus\n"); + return; + } printk("Initialising xenbus\n"); DEBUG("init_xenbus called.\n"); xenstore_buf = mfn_to_virt(start_info.store_mfn); @@ -435,6 +439,11 @@ xenbus_msg_reply(int type, DEFINE_WAIT(w); struct xsd_sockmsg *rep; + if (!xenstore_buf) { + printk("xenbus_msg_reply called but no xenstore!\n"); + return NULL; + } + id = allocate_xenbus_id(); add_waiter(w, req_info[id].waitq); -- 1.7.7.5
From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Changes the minios evtchn implementation to use a list instead of an array. This allows it to grow as necessary to support any number of ports. Unfortunately, it''s still limited by NR_EVS in events.c. Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- extras/mini-os/include/lib.h | 16 +++--- tools/libxc/xc_minios.c | 139 ++++++++++++++++++++++-------------------- 2 files changed, 81 insertions(+), 74 deletions(-) diff --git a/extras/mini-os/include/lib.h b/extras/mini-os/include/lib.h index bd3eeaf..12070c3 100644 --- a/extras/mini-os/include/lib.h +++ b/extras/mini-os/include/lib.h @@ -53,6 +53,7 @@ #include <xen/xen.h> #include <xen/event_channel.h> #include "gntmap.h" +#include "list.h" #ifdef HAVE_LIBC #include <stdio.h> @@ -143,7 +144,12 @@ enum fd_type { FTYPE_SAVEFILE, }; -#define MAX_EVTCHN_PORTS 16 +struct evtchn_port_info { + struct minios_list_head list; + evtchn_port_t port; + unsigned long pending; + int bound; +}; extern struct file { enum fd_type type; @@ -158,13 +164,7 @@ extern struct file { off_t offset; } file; struct { - /* To each event channel FD is associated a series of ports which - * wakes select for this FD. */ - struct { - evtchn_port_t port; - unsigned long pending; - int bound; - } ports[MAX_EVTCHN_PORTS]; + struct minios_list_head ports; } evtchn; struct gntmap gntmap; struct { diff --git a/tools/libxc/xc_minios.c b/tools/libxc/xc_minios.c index 8bbfd18..29cce63 100644 --- a/tools/libxc/xc_minios.c +++ b/tools/libxc/xc_minios.c @@ -210,15 +210,34 @@ static struct xc_osdep_ops minios_privcmd_ops = { }, }; + +/* XXX Note: This is not threadsafe */ +static struct evtchn_port_info* port_alloc(int fd) { + struct evtchn_port_info *port_info; + port_info = malloc(sizeof(struct evtchn_port_info)); + if (port_info == NULL) + return NULL; + port_info->pending = 0; + port_info->port = -1; + port_info->bound = 0; + + minios_list_add(&port_info->list, &files[fd].evtchn.ports); + return port_info; +} + +static void port_dealloc(struct evtchn_port_info *port_info) { + if (port_info->bound) + unbind_evtchn(port_info->port); + minios_list_del(&port_info->list); + free(port_info); +} + static xc_osdep_handle minios_evtchn_open(xc_evtchn *xce) { - int fd = alloc_fd(FTYPE_EVTCHN), i; + int fd = alloc_fd(FTYPE_EVTCHN); if ( fd == -1 ) return XC_OSDEP_OPEN_ERROR; - for (i = 0; i < MAX_EVTCHN_PORTS; i++) { - files[fd].evtchn.ports[i].port = -1; - files[fd].evtchn.ports[i].bound = 0; - } + MINIOS_INIT_LIST_HEAD(&files[fd].evtchn.ports); printf("evtchn_open() -> %d\n", fd); return (xc_osdep_handle)fd; } @@ -231,10 +250,10 @@ static int minios_evtchn_close(xc_evtchn *xce, xc_osdep_handle h) void minios_evtchn_close_fd(int fd) { - int i; - for (i = 0; i < MAX_EVTCHN_PORTS; i++) - if (files[fd].evtchn.ports[i].bound) - unbind_evtchn(files[fd].evtchn.ports[i].port); + struct evtchn_port_info *port_info, *tmp; + minios_list_for_each_entry_safe(port_info, tmp, &files[fd].evtchn.ports, list) + port_dealloc(port_info); + files[fd].type = FTYPE_NONE; } @@ -256,35 +275,21 @@ static int minios_evtchn_notify(xc_evtchn *xce, xc_osdep_handle h, evtchn_port_t return ret; } -/* XXX Note: This is not threadsafe */ -static int port_alloc(int fd) { - int i; - for (i= 0; i < MAX_EVTCHN_PORTS; i++) - if (files[fd].evtchn.ports[i].port == -1) - break; - if (i == MAX_EVTCHN_PORTS) { - printf("Too many ports in xc handle\n"); - errno = EMFILE; - return -1; - } - files[fd].evtchn.ports[i].pending = 0; - return i; -} - static void evtchn_handler(evtchn_port_t port, struct pt_regs *regs, void *data) { int fd = (int)(intptr_t)data; - int i; + struct evtchn_port_info *port_info; assert(files[fd].type == FTYPE_EVTCHN); mask_evtchn(port); - for (i= 0; i < MAX_EVTCHN_PORTS; i++) - if (files[fd].evtchn.ports[i].port == port) - break; - if (i == MAX_EVTCHN_PORTS) { - printk("Unknown port for handle %d\n", fd); - return; + minios_list_for_each_entry(port_info, &files[fd].evtchn.ports, list) { + if (port_info->port == port) + goto found; } - files[fd].evtchn.ports[i].pending = 1; + printk("Unknown port for handle %d\n", fd); + return; + + found: + port_info->pending = 1; files[fd].read = 1; wake_up(&event_queue); } @@ -292,12 +297,13 @@ static void evtchn_handler(evtchn_port_t port, struct pt_regs *regs, void *data) static evtchn_port_or_error_t minios_evtchn_bind_unbound_port(xc_evtchn *xce, xc_osdep_handle h, int domid) { int fd = (int)h; - int ret, i; + struct evtchn_port_info *port_info; + int ret; evtchn_port_t port; assert(get_current() == main_thread); - i = port_alloc(fd); - if (i == -1) + port_info = port_alloc(fd); + if (port_info == NULL) return -1; printf("xc_evtchn_bind_unbound_port(%d)", domid); @@ -305,11 +311,12 @@ static evtchn_port_or_error_t minios_evtchn_bind_unbound_port(xc_evtchn *xce, xc printf(" = %d\n", ret); if (ret < 0) { + port_dealloc(port_info); errno = -ret; return -1; } - files[fd].evtchn.ports[i].bound = 1; - files[fd].evtchn.ports[i].port = port; + port_info->bound = 1; + port_info->port = port; unmask_evtchn(port); return port; } @@ -318,12 +325,13 @@ static evtchn_port_or_error_t minios_evtchn_bind_interdomain(xc_evtchn *xce, xc_ evtchn_port_t remote_port) { int fd = (int)h; + struct evtchn_port_info *port_info; evtchn_port_t local_port; - int ret, i; + int ret; assert(get_current() == main_thread); - i = port_alloc(fd); - if (i == -1) + port_info = port_alloc(fd); + if (port_info == NULL) return -1; printf("xc_evtchn_bind_interdomain(%d, %"PRId32")", domid, remote_port); @@ -331,11 +339,12 @@ static evtchn_port_or_error_t minios_evtchn_bind_interdomain(xc_evtchn *xce, xc_ printf(" = %d\n", ret); if (ret < 0) { + port_dealloc(port_info); errno = -ret; return -1; } - files[fd].evtchn.ports[i].bound = 1; - files[fd].evtchn.ports[i].port = local_port; + port_info->bound = 1; + port_info->port = local_port; unmask_evtchn(local_port); return local_port; } @@ -343,42 +352,40 @@ static evtchn_port_or_error_t minios_evtchn_bind_interdomain(xc_evtchn *xce, xc_ static int minios_evtchn_unbind(xc_evtchn *xce, xc_osdep_handle h, evtchn_port_t port) { int fd = (int)h; - int i; - for (i = 0; i < MAX_EVTCHN_PORTS; i++) - if (files[fd].evtchn.ports[i].port == port) { - files[fd].evtchn.ports[i].port = -1; - break; - } - if (i == MAX_EVTCHN_PORTS) { - printf("Warning: couldn''t find port %"PRId32" for xc handle %x\n", port, fd); - errno = -EINVAL; - return -1; + struct evtchn_port_info *port_info; + + minios_list_for_each_entry(port_info, &files[fd].evtchn.ports, list) { + if (port_info->port == port) { + port_dealloc(port_info); + return 0; + } } - files[fd].evtchn.ports[i].bound = 0; - unbind_evtchn(port); - return 0; + printf("Warning: couldn''t find port %"PRId32" for xc handle %x\n", port, fd); + errno = -EINVAL; + return -1; } static evtchn_port_or_error_t minios_evtchn_bind_virq(xc_evtchn *xce, xc_osdep_handle h, unsigned int virq) { int fd = (int)h; + struct evtchn_port_info *port_info; evtchn_port_t port; - int i; assert(get_current() == main_thread); - i = port_alloc(fd); - if (i == -1) + port_info = port_alloc(fd); + if (port_info == NULL) return -1; printf("xc_evtchn_bind_virq(%d)", virq); port = bind_virq(virq, evtchn_handler, (void*)(intptr_t)fd); if (port < 0) { + port_dealloc(port_info); errno = -port; return -1; } - files[fd].evtchn.ports[i].bound = 1; - files[fd].evtchn.ports[i].port = port; + port_info->bound = 1; + port_info->port = port; unmask_evtchn(port); return port; } @@ -386,18 +393,18 @@ static evtchn_port_or_error_t minios_evtchn_bind_virq(xc_evtchn *xce, xc_osdep_h static evtchn_port_or_error_t minios_evtchn_pending(xc_evtchn *xce, xc_osdep_handle h) { int fd = (int)h; - int i; + struct evtchn_port_info *port_info; unsigned long flags; evtchn_port_t ret = -1; local_irq_save(flags); files[fd].read = 0; - for (i = 0; i < MAX_EVTCHN_PORTS; i++) { - evtchn_port_t port = files[fd].evtchn.ports[i].port; - if (port != -1 && files[fd].evtchn.ports[i].pending) { + + minios_list_for_each_entry(port_info, &files[fd].evtchn.ports, list) { + if (port_info->port != -1 && port_info->pending) { if (ret == -1) { - ret = port; - files[fd].evtchn.ports[i].pending = 0; + ret = port_info->port; + port_info->pending = 0; } else { files[fd].read = 1; break; -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 10/18] xenstored: use grant references instead of map_foreign_range
From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> make xenstored use grantref rather than map_foreign_range (which can only be used by privileged domains) This patch modifies the xenstore daemon to use xc_gnttab_map_grant_ref instead of xc_map_foreign_range where available. Previous versions of this patch have been sent to xen-devel. See http://lists.xensource.com/archives/html/xen-devel/2008-07/msg00610.html http://lists.xensource.com/archives/html/xen-devel/2009-03/msg01492.html Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/xenstored_domain.c | 45 ++++++++++++++++++++++++++++++++----- 1 files changed, 39 insertions(+), 6 deletions(-) diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c index 443af82..0b8353b 100644 --- a/tools/xenstore/xenstored_domain.c +++ b/tools/xenstore/xenstored_domain.c @@ -32,8 +32,10 @@ #include "xenstored_watch.h" #include <xenctrl.h> +#include <xen/grant_table.h> static xc_interface **xc_handle; +static xc_gnttab **xcg_handle; static evtchn_port_t virq_port; xc_evtchn *xce_handle = NULL; @@ -174,8 +176,12 @@ static int destroy_domain(void *_domain) eprintf("> Unbinding port %i failed!\n", domain->port); } - if (domain->interface) - munmap(domain->interface, getpagesize()); + if (domain->interface) { + if (*xcg_handle >= 0 && domain->domid != 0) + xc_gnttab_munmap(*xcg_handle, domain->interface, 1); + else + munmap(domain->interface, getpagesize()); + } fire_watches(NULL, "@releaseDomain", false); @@ -344,9 +350,17 @@ void do_introduce(struct connection *conn, struct buffered_data *in) domain = find_domain_by_domid(domid); if (domain == NULL) { - interface = xc_map_foreign_range( - *xc_handle, domid, - getpagesize(), PROT_READ|PROT_WRITE, mfn); + if (*xcg_handle >= 0) { + /* this is the preferred method */ + interface = xc_gnttab_map_grant_ref( + *xcg_handle, domid, + GNTTAB_RESERVED_XENSTORE, + PROT_READ|PROT_WRITE); + } else { + interface = xc_map_foreign_range( + *xc_handle, domid, + getpagesize(), PROT_READ|PROT_WRITE, mfn); + } if (!interface) { send_error(conn, errno); return; @@ -354,7 +368,10 @@ void do_introduce(struct connection *conn, struct buffered_data *in) /* Hang domain off "in" until we''re finished. */ domain = new_domain(in, domid, port); if (!domain) { - munmap(interface, getpagesize()); + if (*xcg_handle >= 0) + xc_gnttab_munmap(*xcg_handle, interface, 1); + else + munmap(interface, getpagesize()); send_error(conn, errno); return; } @@ -552,6 +569,12 @@ static int close_xc_handle(void *_handle) return 0; } +static int close_xcg_handle(void *_handle) +{ + xc_gnttab_close(*(xc_gnttab **)_handle); + return 0; +} + /* Returns the implicit path of a connection (only domains have this) */ const char *get_implicit_path(const struct connection *conn) { @@ -603,6 +626,16 @@ void domain_init(void) talloc_set_destructor(xc_handle, close_xc_handle); + xcg_handle = talloc(talloc_autofree_context(), xc_gnttab*); + if (!xcg_handle) + barf_perror("Failed to allocate domain gnttab handle"); + + *xcg_handle = xc_gnttab_open(NULL, 0); + if (*xcg_handle < 0) + xprintf("WARNING: Failed to open connection to gnttab\n"); + else + talloc_set_destructor(xcg_handle, close_xcg_handle); + xce_handle = xc_evtchn_open(NULL, 0); if (xce_handle == NULL) -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 11/18] xenstored: add NO_SOCKETS compilation option
From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> option for compiling xenstored without unix sockets to support running on mini-OS Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/xenstored_core.c | 35 +++++++++++++++++++++++++++++++---- tools/xenstore/xs.c | 2 ++ tools/xenstore/xs_lib.c | 4 ++++ 3 files changed, 37 insertions(+), 4 deletions(-) diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c index 9e6c2c7..0623aac 100644 --- a/tools/xenstore/xenstored_core.c +++ b/tools/xenstore/xenstored_core.c @@ -19,9 +19,11 @@ #include <sys/types.h> #include <sys/stat.h> -#include <sys/socket.h> #include <sys/select.h> +#ifndef NO_SOCKETS +#include <sys/socket.h> #include <sys/un.h> +#endif #include <sys/time.h> #include <time.h> #include <unistd.h> @@ -308,7 +310,10 @@ static void set_fd(int fd, fd_set *set, int *max) } -static int initialize_set(fd_set *inset, fd_set *outset, int sock, int ro_sock, +static int initialize_set(fd_set *inset, fd_set *outset, +#ifndef NO_SOCKETS + int sock, int ro_sock, +#endif struct timeval **ptimeout) { static struct timeval zero_timeout = { 0 }; @@ -320,8 +325,10 @@ static int initialize_set(fd_set *inset, fd_set *outset, int sock, int ro_sock, FD_ZERO(inset); FD_ZERO(outset); +#ifndef NO_SOCKETS set_fd(sock, inset, &max); set_fd(ro_sock, inset, &max); +#endif set_fd(reopen_log_pipe[0], inset, &max); if (xce_handle != NULL) @@ -343,12 +350,14 @@ static int initialize_set(fd_set *inset, fd_set *outset, int sock, int ro_sock, return max; } +#ifndef NO_SOCKETS static int destroy_fd(void *_fd) { int *fd = _fd; close(*fd); return 0; } +#endif /* Is child a subnode of parent, or equal? */ bool is_child(const char *child, const char *parent) @@ -1352,6 +1361,7 @@ struct connection *new_connection(connwritefn_t *write, connreadfn_t *read) return new; } +#ifndef NO_SOCKETS static int writefd(struct connection *conn, const void *data, unsigned int len) { int rc; @@ -1406,6 +1416,7 @@ static void accept_connection(int sock, bool canwrite) } else close(fd); } +#endif #define TDB_FLAGS 0 @@ -1752,8 +1763,11 @@ extern void dump_conn(struct connection *conn); int main(int argc, char *argv[]) { - int opt, *sock, *ro_sock, max; + int opt, max; +#ifndef NO_SOCKETS + int *sock, *ro_sock; struct sockaddr_un addr; +#endif fd_set inset, outset; bool dofork = true; bool outputpid = false; @@ -1837,6 +1851,7 @@ int main(int argc, char *argv[]) if (!dofork) talloc_enable_leak_report_full(); +#ifndef NO_SOCKETS /* Create sockets for them to listen to. */ sock = talloc(talloc_autofree_context(), int); *sock = socket(PF_UNIX, SOCK_STREAM, 0); @@ -1848,10 +1863,12 @@ int main(int argc, char *argv[]) barf_perror("Could not create socket"); talloc_set_destructor(sock, destroy_fd); talloc_set_destructor(ro_sock, destroy_fd); +#endif /* Don''t kill us with SIGPIPE. */ signal(SIGPIPE, SIG_IGN); +#ifndef NO_SOCKETS /* FIXME: Be more sophisticated, don''t mug running daemon. */ unlink(xs_daemon_socket()); unlink(xs_daemon_socket_ro()); @@ -1871,6 +1888,7 @@ int main(int argc, char *argv[]) if (listen(*sock, 1) != 0 || listen(*ro_sock, 1) != 0) barf_perror("Could not listen on sockets"); +#endif if (pipe(reopen_log_pipe)) { barf_perror("pipe"); @@ -1909,7 +1927,11 @@ int main(int argc, char *argv[]) evtchn_fd = xc_evtchn_fd(xce_handle); /* Get ready to listen to the tools. */ +#ifndef NO_SOCKETS max = initialize_set(&inset, &outset, *sock, *ro_sock, &timeout); +#else + max = initialize_set(&inset, &outset, &timeout); +#endif /* Tell the kernel we''re up and running. */ xenbus_notify_running(); @@ -1931,11 +1953,13 @@ int main(int argc, char *argv[]) reopen_log(); } +#ifndef NO_SOCKETS if (FD_ISSET(*sock, &inset)) accept_connection(*sock, true); if (FD_ISSET(*ro_sock, &inset)) accept_connection(*ro_sock, false); +#endif if (evtchn_fd != -1 && FD_ISSET(evtchn_fd, &inset)) handle_event(); @@ -1977,7 +2001,10 @@ int main(int argc, char *argv[]) } } - max = initialize_set(&inset, &outset, *sock, *ro_sock, + max = initialize_set(&inset, &outset, +#ifndef NO_SOCKETS + *sock, *ro_sock, +#endif &timeout); } } diff --git a/tools/xenstore/xs.c b/tools/xenstore/xs.c index 8e54fe0..119d945 100644 --- a/tools/xenstore/xs.c +++ b/tools/xenstore/xs.c @@ -271,10 +271,12 @@ struct xs_handle *xs_open(unsigned long flags) { struct xs_handle *xsh = NULL; +#ifndef NO_SOCKETS if (flags & XS_OPEN_READONLY) xsh = get_handle(xs_daemon_socket_ro()); else xsh = get_handle(xs_daemon_socket()); +#endif if (!xsh && !(flags & XS_OPEN_SOCKETONLY)) xsh = get_handle(xs_domain_dev()); diff --git a/tools/xenstore/xs_lib.c b/tools/xenstore/xs_lib.c index 03a9ee4..af3db6b 100644 --- a/tools/xenstore/xs_lib.c +++ b/tools/xenstore/xs_lib.c @@ -39,6 +39,7 @@ const char *xs_daemon_rundir(void) return (s ? s : "/var/run/xenstored"); } +#ifndef NO_SOCKETS static const char *xs_daemon_path(void) { static char buf[PATH_MAX]; @@ -50,6 +51,7 @@ static const char *xs_daemon_path(void) return NULL; return buf; } +#endif const char *xs_daemon_tdb(void) { @@ -58,6 +60,7 @@ const char *xs_daemon_tdb(void) return buf; } +#ifndef NO_SOCKETS const char *xs_daemon_socket(void) { return xs_daemon_path(); @@ -73,6 +76,7 @@ const char *xs_daemon_socket_ro(void) return NULL; return buf; } +#endif const char *xs_domain_dev(void) { -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 12/18] xenstored support for in-memory rather than FS based trivial DB (needed to run on mini-OS)
From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> tdb_copy (a xen modification to tdb?) should honor the TDB_INTERNAL flag for in-memory databases. TODO: leaks memory on error case Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/tdb.c | 36 ++++++++++++++++++++++++++++++++++++ 1 files changed, 36 insertions(+), 0 deletions(-) diff --git a/tools/xenstore/tdb.c b/tools/xenstore/tdb.c index 63205e1..639ce6e 100644 --- a/tools/xenstore/tdb.c +++ b/tools/xenstore/tdb.c @@ -2103,6 +2103,42 @@ TDB_CONTEXT *tdb_copy(TDB_CONTEXT *tdb, const char *outfile) int fd, saved_errno; TDB_CONTEXT *copy; + if (tdb->flags & TDB_INTERNAL) { + struct tdb_header *copydb; + + copy = talloc_zero(outfile, TDB_CONTEXT); + if (copy == NULL) { + errno = ENOMEM; + goto intfail; + } + memcpy(copy, tdb, sizeof(TDB_CONTEXT)); + + if (copy->name || copy->locked || copy->device || copy->inode) { + fprintf(stderr, "tdb_copy assumption(s) failed\n"); + goto intfail; + } + + copydb = talloc_zero_size(copy, copy->map_size); + if (copydb == NULL) { + errno = ENOMEM; + goto intfail; + } + memcpy(copydb, copy->map_ptr, copy->map_size); + copy->map_ptr = (char*) copydb; + + if (tdb_brlock(tdb, GLOBAL_LOCK, F_UNLCK, F_SETLKW, 0) == -1) + goto intfail; + + copy->next = tdbs; + tdbs = copy; + + + return copy; +intfail: + /* TODO (leaking memory is easier) */ + return NULL; + } + fd = open(outfile, O_TRUNC|O_CREAT|O_WRONLY, 0640); if (fd < 0) return NULL; -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 13/18] xenstored: support running in minios stubdom
A previous versions of this patch has been sent to xen-devel. See http://lists.xensource.com/archives/html/xen-devel/2009-03/msg01655.html Originally-by: Diego Ongaro <diego.ongaro@citrix.com> Originally-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- extras/mini-os/include/list.h | 6 ++-- extras/mini-os/main.c | 4 +++ stubdom/Makefile | 29 +++++++++++++++++-- tools/xenstore/Makefile | 9 +++++- tools/xenstore/tdb.c | 6 ++-- tools/xenstore/utils.h | 2 + tools/xenstore/xenstored_core.c | 47 ++++++++++++++++++++++++++++++- tools/xenstore/xenstored_domain.c | 7 +++++ tools/xenstore/xenstored_transaction.c | 2 + 9 files changed, 100 insertions(+), 12 deletions(-) diff --git a/extras/mini-os/include/list.h b/extras/mini-os/include/list.h index a60ae23..4e6a2ac 100644 --- a/extras/mini-os/include/list.h +++ b/extras/mini-os/include/list.h @@ -1,5 +1,5 @@ -#ifndef _LINUX_LIST_H -#define _LINUX_LIST_H +#ifndef _MINIOS_LIST_H +#define _MINIOS_LIST_H /* * Simple doubly linked list implementation. @@ -186,5 +186,5 @@ static __inline__ void minios_list_splice(struct minios_list_head *list, struct n = minios_list_entry(pos->member.next, typeof(*pos), member); \ &pos->member != (head); \ pos = n, n = minios_list_entry(n->member.next, typeof(*n), member)) -#endif /* _LINUX_LIST_H */ +#endif /* _MINIOS_LIST_H */ diff --git a/extras/mini-os/main.c b/extras/mini-os/main.c index b95b889..cd89849 100644 --- a/extras/mini-os/main.c +++ b/extras/mini-os/main.c @@ -60,6 +60,7 @@ static void call_main(void *p) * crashing. */ //sleep(1); +#ifndef CONFIG_XENSTORE #ifndef CONFIG_GRUB sparse((unsigned long) &__app_bss_start, &__app_bss_end - &__app_bss_start); #if defined(HAVE_LWIP) && !defined(CONFIG_QEMU) @@ -67,6 +68,7 @@ static void call_main(void *p) #endif #endif create_thread("pcifront", pcifront_watches, NULL); +#endif #ifdef CONFIG_QEMU /* Fetch argc, argv from XenStore */ @@ -169,9 +171,11 @@ void _exit(int ret) close_all_files(); __libc_fini_array(); printk("main returned %d\n", ret); +#ifndef CONFIG_XENSTORE #ifdef HAVE_LWIP stop_networking(); #endif +#endif stop_kernel(); if (!ret) { /* No problem, just shutdown. */ diff --git a/stubdom/Makefile b/stubdom/Makefile index 3705059..e0a90a9 100644 --- a/stubdom/Makefile +++ b/stubdom/Makefile @@ -74,14 +74,14 @@ TARGET_CPPFLAGS += -I$(XEN_ROOT)/xen/include TARGET_LDFLAGS += -nostdlib -L$(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf/lib -TARGETS=ioemu c caml grub +TARGETS=ioemu c caml grub xenstore CROSS_MAKE := $(MAKE) DESTDIR .PHONY: all all: build ifeq ($(STUBDOM_SUPPORTED),1) -build: genpath ioemu-stubdom c-stubdom pv-grub +build: genpath ioemu-stubdom c-stubdom pv-grub xenstore-stubdom else build: genpath endif @@ -262,6 +262,11 @@ mk-headers-$(XEN_TARGET_ARCH): ioemu/linkfarm.stamp ln -sf $(XEN_ROOT)/tools/libxc/$(XEN_TARGET_ARCH)/*.c . && \ ln -sf $(XEN_ROOT)/tools/libxc/$(XEN_TARGET_ARCH)/*.h . && \ ln -sf $(XEN_ROOT)/tools/libxc/$(XEN_TARGET_ARCH)/Makefile . ) + mkdir -p xenstore + [ -h xenstore/Makefile ] || ( cd xenstore && \ + ln -sf $(XEN_ROOT)/tools/xenstore/*.c . && \ + ln -sf $(XEN_ROOT)/tools/xenstore/*.h . && \ + ln -sf $(XEN_ROOT)/tools/xenstore/Makefile . ) $(CROSS_MAKE) -C $(MINI_OS) links touch mk-headers-$(XEN_TARGET_ARCH) @@ -334,6 +339,14 @@ grub: grub-upstream $(CROSS_ROOT) mkdir -p grub-$(XEN_TARGET_ARCH) CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(CROSS_MAKE) -C $@ OBJ_DIR=$(CURDIR)/grub-$(XEN_TARGET_ARCH) +########## +# xenstore +########## + +.PHONY: xenstore +xenstore: $(CROSS_ROOT) + CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(CROSS_MAKE) -C $@ LWIPDIR=$(CURDIR)/lwip xenstored.a CONFIG_STUBDOM=y + ######## # minios ######## @@ -355,12 +368,16 @@ c-stubdom: mini-os-$(XEN_TARGET_ARCH)-c lwip-$(XEN_TARGET_ARCH) libxc c pv-grub: mini-os-$(XEN_TARGET_ARCH)-grub libxc grub DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="-DCONFIG_GRUB $(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" $(CROSS_MAKE) -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/grub-$(XEN_TARGET_ARCH)/main.a +.PHONY: xenstore-stubdom +xenstore-stubdom: mini-os-$(XEN_TARGET_ARCH)-xenstore libxc xenstore + DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="-DCONFIG_XENSTORE $(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" $(MAKE) -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< LWIPDIR=$(CURDIR)/lwip-$(XEN_TARGET_ARCH) APP_OBJS=$(CURDIR)/xenstore/xenstored.a + ######### # install ######### ifeq ($(STUBDOM_SUPPORTED),1) -install: genpath install-readme install-ioemu install-grub +install: genpath install-readme install-ioemu install-grub install-xenstore else install: genpath endif @@ -379,6 +396,10 @@ install-grub: pv-grub $(INSTALL_DIR) "$(DESTDIR)$(XENFIRMWAREDIR)" $(INSTALL_DATA) mini-os-$(XEN_TARGET_ARCH)-grub/mini-os.gz "$(DESTDIR)$(XENFIRMWAREDIR)/pv-grub-$(XEN_TARGET_ARCH).gz" +install-xenstore: xenstore-stubdom + $(INSTALL_DIR) "$(DESTDIR)/usr/lib/xen/boot" + $(INSTALL_PROG) mini-os-$(XEN_TARGET_ARCH)-xenstore/mini-os.gz "$(DESTDIR)/usr/lib/xen/boot/xenstore-stubdom.gz" + ####### # clean ####### @@ -390,12 +411,14 @@ clean: rm -fr mini-os-$(XEN_TARGET_ARCH)-c rm -fr mini-os-$(XEN_TARGET_ARCH)-caml rm -fr mini-os-$(XEN_TARGET_ARCH)-grub + rm -fr mini-os-$(XEN_TARGET_ARCH)-xenstore $(CROSS_MAKE) -C caml clean $(CROSS_MAKE) -C c clean rm -fr grub-$(XEN_TARGET_ARCH) rm -f $(STUBDOMPATH) [ ! -d libxc-$(XEN_TARGET_ARCH) ] || $(CROSS_MAKE) -C libxc-$(XEN_TARGET_ARCH) clean -[ ! -d ioemu ] || $(CROSS_MAKE) -C ioemu clean + -[ ! -d xenstore ] || $(CROSS_MAKE) -C xenstore clean # clean the cross-compilation result .PHONY: crossclean diff --git a/tools/xenstore/Makefile b/tools/xenstore/Makefile index 4facb62..3a061d6 100644 --- a/tools/xenstore/Makefile +++ b/tools/xenstore/Makefile @@ -28,6 +28,10 @@ endif ALL_TARGETS = libxenstore.so libxenstore.a clients xs_tdb_dump xenstored +ifdef CONFIG_STUBDOM +CFLAGS += -DNO_SOCKETS=1 -DNO_LOCAL_XENBUS=1 -DNO_SYSLOG=1 -DNO_REOPEN_LOG=1 +endif + .PHONY: all all: $(ALL_TARGETS) @@ -45,10 +49,13 @@ xenstored_probes.o: xenstored_solaris.o CFLAGS += -DHAVE_DTRACE=1 endif - + xenstored: $(XENSTORED_OBJS) $(CC) $(LDFLAGS) $^ $(LDLIBS_libxenctrl) $(SOCKET_LIBS) -o $@ $(APPEND_LDFLAGS) +xenstored.a: $(XENSTORED_OBJS) + $(AR) cr $@ $^ + $(CLIENTS): xenstore ln -f xenstore $@ diff --git a/tools/xenstore/tdb.c b/tools/xenstore/tdb.c index 639ce6e..75ffd2a 100644 --- a/tools/xenstore/tdb.c +++ b/tools/xenstore/tdb.c @@ -1334,7 +1334,7 @@ static int tdb_next_lock(TDB_CONTEXT *tdb, struct tdb_traverse_lock *tlock, /* Iterate through chain */ while( tlock->off) { - tdb_off current; + tdb_off mycurrent; if (rec_read(tdb, tlock->off, rec) == -1) goto fail; @@ -1352,10 +1352,10 @@ static int tdb_next_lock(TDB_CONTEXT *tdb, struct tdb_traverse_lock *tlock, } /* Try to clean dead ones from old traverses */ - current = tlock->off; + mycurrent = tlock->off; tlock->off = rec->next; if (!tdb->read_only && - do_delete(tdb, current, rec) != 0) + do_delete(tdb, mycurrent, rec) != 0) goto fail; } tdb_unlock(tdb, tlock->hash, F_WRLCK); diff --git a/tools/xenstore/utils.h b/tools/xenstore/utils.h index f378343..2effd17 100644 --- a/tools/xenstore/utils.h +++ b/tools/xenstore/utils.h @@ -19,7 +19,9 @@ static inline bool strends(const char *a, const char *b) return streq(a + strlen(a) - strlen(b), b); } +#ifndef ARRAY_SIZE #define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0])) +#endif void barf(const char *fmt, ...) __attribute__((noreturn)); void barf_perror(const char *fmt, ...) __attribute__((noreturn)); diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c index 0623aac..2d4a7e1 100644 --- a/tools/xenstore/xenstored_core.c +++ b/tools/xenstore/xenstored_core.c @@ -32,7 +32,9 @@ #include <stdio.h> #include <stdarg.h> #include <stdlib.h> +#ifndef NO_SYSLOG #include <syslog.h> +#endif #include <string.h> #include <errno.h> #include <dirent.h> @@ -61,13 +63,24 @@ LIST_HEAD(connections); static int tracefd = -1; static bool recovery = true; static bool remove_local = true; +#ifndef NO_REOPEN_LOG static int reopen_log_pipe[2]; +#endif static char *tracefile = NULL; static TDB_CONTEXT *tdb_ctx; static void corrupt(struct connection *conn, const char *fmt, ...); static void check_store(void); +#ifdef __MINIOS__ +#define lockf(...) (-ENOSYS) +#endif + +#ifdef NO_SYSLOG +#define openlog(...) ((void) 0) +#define syslog(...) ((void) 0) +#endif + #define log(...) \ do { \ char *s = talloc_asprintf(NULL, __VA_ARGS__); \ @@ -92,8 +105,10 @@ TDB_CONTEXT *tdb_context(struct connection *conn) bool replace_tdb(const char *newname, TDB_CONTEXT *newtdb) { +#ifndef __MINIOS__ if (rename(newname, xs_daemon_tdb()) != 0) return false; +#endif tdb_close(tdb_ctx); tdb_ctx = talloc_steal(talloc_autofree_context(), newtdb); return true; @@ -195,6 +210,7 @@ void trace_destroy(const void *data, const char *type) trace("DESTROY %s %p\n", type, data); } +#ifndef NO_REOPEN_LOG /** * Signal handler for SIGHUP, which requests that the trace log is reopened * (in the main loop). A single byte is written to reopen_log_pipe, to awaken @@ -222,7 +238,7 @@ static void reopen_log(void) trace("\n***\n"); } } - +#endif static bool write_messages(struct connection *conn) { @@ -329,7 +345,9 @@ static int initialize_set(fd_set *inset, fd_set *outset, set_fd(sock, inset, &max); set_fd(ro_sock, inset, &max); #endif +#ifndef NO_REOPEN_LOG set_fd(reopen_log_pipe[0], inset, &max); +#endif if (xce_handle != NULL) set_fd(xc_evtchn_fd(xce_handle), inset, &max); @@ -1418,7 +1436,11 @@ static void accept_connection(int sock, bool canwrite) } #endif +#ifdef __MINIOS__ +#define TDB_FLAGS TDB_INTERNAL|TDB_NOLOCK +#else #define TDB_FLAGS 0 +#endif /* We create initial nodes manually. */ static void manual_node(const char *name, const char *child) @@ -1443,7 +1465,11 @@ static void setup_structure(void) { char *tdbname; tdbname = talloc_strdup(talloc_autofree_context(), xs_daemon_tdb()); +#ifdef __MINIOS__ + tdb_ctx = NULL; +#else tdb_ctx = tdb_open(tdbname, 0, TDB_FLAGS, O_RDWR, 0); +#endif if (tdb_ctx) { /* XXX When we make xenstored able to restart, this will have @@ -1669,6 +1695,7 @@ static void corrupt(struct connection *conn, const char *fmt, ...) } +#ifndef __MINIOS__ static void write_pidfile(const char *pidfile) { char buf[100]; @@ -1715,7 +1742,7 @@ static void daemonize(void) /* Discard our parent''s old-fashioned umask prejudices. */ umask(0); } - +#endif static void usage(void) { @@ -1769,7 +1796,11 @@ int main(int argc, char *argv[]) struct sockaddr_un addr; #endif fd_set inset, outset; +#ifdef __MINIOS__ + bool dofork = false; +#else bool dofork = true; +#endif bool outputpid = false; bool no_domain_init = false; const char *pidfile = NULL; @@ -1823,8 +1854,11 @@ int main(int argc, char *argv[]) if (optind != argc) barf("%s: No arguments desired", argv[0]); +#ifndef NO_REOPEN_LOG reopen_log(); +#endif +#ifndef __MINIOS__ /* make sure xenstored directory exists */ if (mkdir(xs_daemon_rundir(), 0755)) { if (errno != EEXIST) { @@ -1846,6 +1880,7 @@ int main(int argc, char *argv[]) } if (pidfile) write_pidfile(pidfile); +#endif /* Talloc leak reports go to stderr, which is closed if we fork. */ if (!dofork) @@ -1890,9 +1925,11 @@ int main(int argc, char *argv[]) barf_perror("Could not listen on sockets"); #endif +#ifndef NO_REOPEN_LOG if (pipe(reopen_log_pipe)) { barf_perror("pipe"); } +#endif /* Setup the database */ setup_structure(); @@ -1921,7 +1958,9 @@ int main(int argc, char *argv[]) xprintf = trace; } +#ifndef NO_REOPEN_LOG signal(SIGHUP, trigger_reopen_log); +#endif if (xce_handle != NULL) evtchn_fd = xc_evtchn_fd(xce_handle); @@ -1933,8 +1972,10 @@ int main(int argc, char *argv[]) max = initialize_set(&inset, &outset, &timeout); #endif +#ifndef __MINIOS__ /* Tell the kernel we''re up and running. */ xenbus_notify_running(); +#endif /* Main loop. */ for (;;) { @@ -1946,12 +1987,14 @@ int main(int argc, char *argv[]) barf_perror("Select failed"); } +#ifndef NO_REOPEN_LOG if (FD_ISSET(reopen_log_pipe[0], &inset)) { char c; if (read(reopen_log_pipe[0], &c, 1) != 1) barf_perror("read failed"); reopen_log(); } +#endif #ifndef NO_SOCKETS if (FD_ISSET(*sock, &inset)) diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c index 0b8353b..811ae3c 100644 --- a/tools/xenstore/xenstored_domain.c +++ b/tools/xenstore/xenstored_domain.c @@ -588,6 +588,12 @@ void restore_existing_connections(void) { } +#ifdef __MINIOS__ +static inline int dom0_init(void) +{ + return 0; +} +#else static int dom0_init(void) { evtchn_port_t port; @@ -611,6 +617,7 @@ static int dom0_init(void) return 0; } +#endif void domain_init(void) { diff --git a/tools/xenstore/xenstored_transaction.c b/tools/xenstore/xenstored_transaction.c index 380c691..c59acfb 100644 --- a/tools/xenstore/xenstored_transaction.c +++ b/tools/xenstore/xenstored_transaction.c @@ -120,7 +120,9 @@ static int destroy_transaction(void *_transaction) trace_destroy(trans, "transaction"); if (trans->tdb) tdb_close(trans->tdb); +#ifndef __MINIOS__ unlink(trans->tdb_name); +#endif return 0; } -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 14/18] xenstored: always use xc_gnttab_munmap in stubdom
Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/xenstored_domain.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c index 811ae3c..aca2149 100644 --- a/tools/xenstore/xenstored_domain.c +++ b/tools/xenstore/xenstored_domain.c @@ -177,10 +177,14 @@ static int destroy_domain(void *_domain) } if (domain->interface) { +#ifdef __MINIOS__ + xc_gnttab_munmap(*xcg_handle, domain->interface, 1); +#else if (*xcg_handle >= 0 && domain->domid != 0) xc_gnttab_munmap(*xcg_handle, domain->interface, 1); else munmap(domain->interface, getpagesize()); +#endif } fire_watches(NULL, "@releaseDomain", false); -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 15/18] xenstored: add --event parameter for bootstrapping
When xenstored is run in a minios domain, it needs a bootstrap connection to dom0 so that additional domain introduce messages can be sent to it. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/xenstored_core.c | 5 +++++ tools/xenstore/xenstored_core.h | 1 + tools/xenstore/xenstored_domain.c | 13 ++++++++++++- 3 files changed, 18 insertions(+), 1 deletions(-) diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c index 2d4a7e1..7b63a68 100644 --- a/tools/xenstore/xenstored_core.c +++ b/tools/xenstore/xenstored_core.c @@ -1774,6 +1774,7 @@ static struct option options[] = { { "no-domain-init", 0, NULL, ''D'' }, { "entry-nb", 1, NULL, ''E'' }, { "pid-file", 1, NULL, ''F'' }, + { "event", 1, NULL, ''e'' }, { "help", 0, NULL, ''H'' }, { "no-fork", 0, NULL, ''N'' }, { "output-pid", 0, NULL, ''P'' }, @@ -1787,6 +1788,7 @@ static struct option options[] = { { NULL, 0, NULL, 0 } }; extern void dump_conn(struct connection *conn); +int dom0_event = 0; int main(int argc, char *argv[]) { @@ -1849,6 +1851,9 @@ int main(int argc, char *argv[]) case ''W'': quota_nb_watch_per_domain = strtol(optarg, NULL, 10); break; + case ''e'': + dom0_event = strtol(optarg, NULL, 10); + break; } } if (optind != argc) diff --git a/tools/xenstore/xenstored_core.h b/tools/xenstore/xenstored_core.h index c487089..d3040ba 100644 --- a/tools/xenstore/xenstored_core.h +++ b/tools/xenstore/xenstored_core.h @@ -168,6 +168,7 @@ void trace(const char *fmt, ...); void dtrace_io(const struct connection *conn, const struct buffered_data *data, int out); extern int event_fd; +extern int dom0_event; /* Map the kernel''s xenstore page. */ void *xenbus_map(void); diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c index aca2149..648eb1d 100644 --- a/tools/xenstore/xenstored_domain.c +++ b/tools/xenstore/xenstored_domain.c @@ -593,8 +593,19 @@ void restore_existing_connections(void) } #ifdef __MINIOS__ -static inline int dom0_init(void) +static int dom0_init(void) { + struct domain *domain; + int domid = 0; + evtchn_port_t port = dom0_event; + + domain = new_domain(NULL, domid, port); + domain->interface = xc_gnttab_map_grant_ref(*xcg_handle, domid, + GNTTAB_RESERVED_XENSTORE, PROT_READ|PROT_WRITE); + talloc_steal(domain->conn, domain); + + xc_evtchn_notify(xce_handle, domain->port); + return 0; } #else -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 16/18] xenstored: pull dom0 event port from shared page
When building the xenstored domain using xl, it is difficult to specify the event channel number on the command line because the command line is parsed prior to domain creation, while the event channel cannot be created until after domain creation. To avoid needing a special domain builder for the xenstore domain, store the event channel port in an unused field of the shared page. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/xenstored_domain.c | 12 ++++++++++-- 1 files changed, 10 insertions(+), 2 deletions(-) diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c index 648eb1d..b100aa4 100644 --- a/tools/xenstore/xenstored_domain.c +++ b/tools/xenstore/xenstored_domain.c @@ -598,10 +598,18 @@ static int dom0_init(void) struct domain *domain; int domid = 0; evtchn_port_t port = dom0_event; + void *interface; - domain = new_domain(NULL, domid, port); - domain->interface = xc_gnttab_map_grant_ref(*xcg_handle, domid, + interface = xc_gnttab_map_grant_ref(*xcg_handle, domid, GNTTAB_RESERVED_XENSTORE, PROT_READ|PROT_WRITE); + + if (!port) { + port = *(uint16_t*)(interface + + sizeof(struct xenstore_domain_interface)); + } + + domain = new_domain(NULL, domid, port); + domain->interface = interface; talloc_steal(domain->conn, domain); xc_evtchn_notify(xce_handle, domain->port); -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:21 UTC
[PATCH 17/18] xenstored: use domain_is_unprivileged instead of checking conn->id
This centralizes all the permission checking for privileged domains in preparation for allowing domains other than dom0 to be privileged. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/xenstored_core.c | 6 +++--- tools/xenstore/xenstored_domain.c | 8 ++++---- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c index 7b63a68..65ceb2c 100644 --- a/tools/xenstore/xenstored_core.c +++ b/tools/xenstore/xenstored_core.c @@ -491,7 +491,7 @@ static enum xs_perm_type perm_for_conn(struct connection *conn, mask &= ~XS_PERM_WRITE; /* Owners and tools get it all... */ - if (!conn->id || perms[0].id == conn->id + if (!domain_is_unprivileged(conn) || perms[0].id == conn->id || (conn->target && perms[0].id == conn->target->id)) return (XS_PERM_READ|XS_PERM_WRITE|XS_PERM_OWNER) & mask; @@ -829,11 +829,11 @@ static struct node *construct_node(struct connection *conn, const char *name) node->tdb = tdb_context(conn); node->name = talloc_strdup(node, name); - /* Inherit permissions, except domains own what they create */ + /* Inherit permissions, except unprivileged domains own what they create */ node->num_perms = parent->num_perms; node->perms = talloc_memdup(node, parent->perms, node->num_perms * sizeof(node->perms[0])); - if (conn && conn->id) + if (domain_is_unprivileged(conn)) node->perms[0].id = conn->id; /* No children, no data */ diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c index b100aa4..5bf16e8 100644 --- a/tools/xenstore/xenstored_domain.c +++ b/tools/xenstore/xenstored_domain.c @@ -336,7 +336,7 @@ void do_introduce(struct connection *conn, struct buffered_data *in) return; } - if (conn->id != 0 || !conn->can_write) { + if (domain_is_unprivileged(conn) || !conn->can_write) { send_error(conn, EACCES); return; } @@ -413,7 +413,7 @@ void do_set_target(struct connection *conn, struct buffered_data *in) return; } - if (conn->id != 0 || !conn->can_write) { + if (domain_is_unprivileged(conn) || !conn->can_write) { send_error(conn, EACCES); return; } @@ -465,7 +465,7 @@ void do_release(struct connection *conn, const char *domid_str) return; } - if (conn->id != 0) { + if (domain_is_unprivileged(conn)) { send_error(conn, EACCES); return; } @@ -502,7 +502,7 @@ void do_resume(struct connection *conn, const char *domid_str) return; } - if (conn->id != 0) { + if (domain_is_unprivileged(conn)) { send_error(conn, EACCES); return; } -- 1.7.7.5
This parameter identifies an alternative service domain which has superuser access to the xenstore database, which is currently required to set up a new domain''s xenstore entries. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/xenstored_core.c | 5 +++++ tools/xenstore/xenstored_core.h | 1 + tools/xenstore/xenstored_domain.c | 2 +- 3 files changed, 7 insertions(+), 1 deletions(-) diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c index 65ceb2c..4437c8d 100644 --- a/tools/xenstore/xenstored_core.c +++ b/tools/xenstore/xenstored_core.c @@ -1777,6 +1777,7 @@ static struct option options[] = { { "event", 1, NULL, ''e'' }, { "help", 0, NULL, ''H'' }, { "no-fork", 0, NULL, ''N'' }, + { "priv-domid", 1, NULL, ''p'' }, { "output-pid", 0, NULL, ''P'' }, { "entry-size", 1, NULL, ''S'' }, { "trace-file", 1, NULL, ''T'' }, @@ -1789,6 +1790,7 @@ static struct option options[] = { extern void dump_conn(struct connection *conn); int dom0_event = 0; +int priv_domid = 0; int main(int argc, char *argv[]) { @@ -1854,6 +1856,9 @@ int main(int argc, char *argv[]) case ''e'': dom0_event = strtol(optarg, NULL, 10); break; + case ''p'': + priv_domid = strtol(optarg, NULL, 10); + break; } } if (optind != argc) diff --git a/tools/xenstore/xenstored_core.h b/tools/xenstore/xenstored_core.h index d3040ba..03e2e48 100644 --- a/tools/xenstore/xenstored_core.h +++ b/tools/xenstore/xenstored_core.h @@ -169,6 +169,7 @@ void dtrace_io(const struct connection *conn, const struct buffered_data *data, extern int event_fd; extern int dom0_event; +extern int priv_domid; /* Map the kernel''s xenstore page. */ void *xenbus_map(void); diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c index 5bf16e8..ba9a5ef 100644 --- a/tools/xenstore/xenstored_domain.c +++ b/tools/xenstore/xenstored_domain.c @@ -241,7 +241,7 @@ bool domain_can_read(struct connection *conn) bool domain_is_unprivileged(struct connection *conn) { - return (conn && conn->domain && conn->domain->domid != 0); + return (conn && conn->domain && conn->domain->domid != 0 && conn->domain->domid != priv_domid); } bool domain_can_write(struct connection *conn) -- 1.7.7.5
Daniel De Graaf
2012-Jan-11 17:22 UTC
[PATCH] xenbus: Add support for xenbus backend in stub domain
This adds two ioctls to the /dev/xen/xenbus_backend device allowing the xenbus backend to be started after the kernel has booted. This is intended to allow dom0 to start another domain to run xenstore. IOCTL_XENBUS_BACKEND_REMOTE_SETUP requires that the xenstore domain be started; the domain ID is passed as the ioctl argument. This sets up a listening event channel and grant reference to xenbus, but does not start using this interface. It returns the local event channel port. IOCTL_XENBUS_BACKEND_REMOTE_COMMIT causes the kernel to begin using the event channel set up in the previous ioctl, reregistering all watches and deallocating the previous xenstored event channel. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- drivers/xen/xenbus/xenbus_comms.c | 6 +++ drivers/xen/xenbus/xenbus_comms.h | 1 + drivers/xen/xenbus/xenbus_dev_backend.c | 57 +++++++++++++++++++++++++++++++ include/xen/grant_table.h | 2 + include/xen/xenbus_dev.h | 6 +++ 5 files changed, 72 insertions(+), 0 deletions(-) diff --git a/drivers/xen/xenbus/xenbus_comms.c b/drivers/xen/xenbus/xenbus_comms.c index 2eff7a6..52fe7ad 100644 --- a/drivers/xen/xenbus/xenbus_comms.c +++ b/drivers/xen/xenbus/xenbus_comms.c @@ -234,3 +234,9 @@ int xb_init_comms(void) return 0; } + +void xb_deinit_comms(void) +{ + unbind_from_irqhandler(xenbus_irq, &xb_waitq); + xenbus_irq = 0; +} diff --git a/drivers/xen/xenbus/xenbus_comms.h b/drivers/xen/xenbus/xenbus_comms.h index 6e42800..c8abd3b 100644 --- a/drivers/xen/xenbus/xenbus_comms.h +++ b/drivers/xen/xenbus/xenbus_comms.h @@ -35,6 +35,7 @@ int xs_init(void); int xb_init_comms(void); +void xb_deinit_comms(void); /* Low level routines. */ int xb_write(const void *data, unsigned len); diff --git a/drivers/xen/xenbus/xenbus_dev_backend.c b/drivers/xen/xenbus/xenbus_dev_backend.c index 3d3be78..4138ba2 100644 --- a/drivers/xen/xenbus/xenbus_dev_backend.c +++ b/drivers/xen/xenbus/xenbus_dev_backend.c @@ -8,7 +8,11 @@ #include <xen/xen.h> #include <xen/page.h> +#include <xen/xenbus.h> #include <xen/xenbus_dev.h> +#include <xen/grant_table.h> +#include <xen/events.h> +#include <asm/xen/hypervisor.h> #include "xenbus_comms.h" @@ -22,6 +26,53 @@ static int xenbus_backend_open(struct inode *inode, struct file *filp) return nonseekable_open(inode, filp); } +static int pending_xsd_port; + +static long xenbus_backend_remote_setup(domid_t domid) +{ + struct evtchn_alloc_unbound arg; + int err; + + gnttab_grant_foreign_access_ref(GNTTAB_RESERVED_XENSTORE, domid, + virt_to_mfn(xen_store_interface), 0 /* writable */); + + if (pending_xsd_port) { + struct evtchn_close close; + close.port = pending_xsd_port; + err = HYPERVISOR_event_channel_op(EVTCHNOP_close, &close); + WARN_ON(err); + } + + arg.dom = DOMID_SELF; + arg.remote_dom = domid; + + err = HYPERVISOR_event_channel_op(EVTCHNOP_alloc_unbound, &arg); + if (err) + return err; + + pending_xsd_port = arg.port; + + return arg.port; +} + +static long xenbus_backend_remote_commit(unsigned long data) +{ + if (!pending_xsd_port) + return -EINVAL; + + xs_suspend(); + + if (xen_store_evtchn > 0) + xb_deinit_comms(); + + xen_store_evtchn = pending_xsd_port; + pending_xsd_port = 0; + + xs_resume(); + + return 0; +} + static long xenbus_backend_ioctl(struct file *file, unsigned int cmd, unsigned long data) { if (!capable(CAP_SYS_ADMIN)) @@ -33,6 +84,12 @@ static long xenbus_backend_ioctl(struct file *file, unsigned int cmd, unsigned l return xen_store_evtchn; return -ENODEV; + case IOCTL_XENBUS_BACKEND_REMOTE_SETUP: + return xenbus_backend_remote_setup(data); + + case IOCTL_XENBUS_BACKEND_REMOTE_COMMIT: + return xenbus_backend_remote_commit(data); + default: return -ENOTTY; } diff --git a/include/xen/grant_table.h b/include/xen/grant_table.h index 15f8a00..11e27c3 100644 --- a/include/xen/grant_table.h +++ b/include/xen/grant_table.h @@ -46,6 +46,8 @@ #include <xen/features.h> +#define GNTTAB_RESERVED_XENSTORE 1 + /* NR_GRANT_FRAMES must be less than or equal to that configured in Xen */ #define NR_GRANT_FRAMES 4 diff --git a/include/xen/xenbus_dev.h b/include/xen/xenbus_dev.h index ac5f0fe..24d5028 100644 --- a/include/xen/xenbus_dev.h +++ b/include/xen/xenbus_dev.h @@ -38,4 +38,10 @@ #define IOCTL_XENBUS_BACKEND_EVTCHN \ _IOC(_IOC_NONE, ''B'', 0, 0) +#define IOCTL_XENBUS_BACKEND_REMOTE_SETUP \ + _IOC(_IOC_NONE, ''B'', 1, 0) + +#define IOCTL_XENBUS_BACKEND_REMOTE_COMMIT \ + _IOC(_IOC_NONE, ''B'', 2, 0) + #endif /* __LINUX_XEN_XENBUS_DEV_H__ */ -- 1.7.7.5
Keir Fraser
2012-Jan-11 17:27 UTC
Re: [PATCH 03/18] xsm: allow use of XEN_DOMCTL_getdomaininfo by non-IS_PRIV domains
On 11/01/2012 17:21, "Daniel De Graaf" <dgdegra@tycho.nsa.gov> wrote:> This domctl does not allow manipulation of domains, only basic > information such as size and state. XSM modules can also provide > fine-grained control over what domains are visible to domains that call > getdomaininfo.Well there''s a reason we might not disallow the hypercall. But why would we actually care to allow it? -- Keir> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > --- > xen/common/domctl.c | 4 ++++ > 1 files changed, 4 insertions(+), 0 deletions(-) > > diff --git a/xen/common/domctl.c b/xen/common/domctl.c > index a775aa3..2c1ca85 100644 > --- a/xen/common/domctl.c > +++ b/xen/common/domctl.c > @@ -263,6 +263,10 @@ long do_domctl(XEN_GUEST_HANDLE(xen_domctl_t) u_domctl) > return -EPERM; > break; > } > +#ifdef XSM_ENABLE > + case XEN_DOMCTL_getdomaininfo: > + break; > +#endif > default: > if ( !IS_PRIV(current->domain) ) > return -EPERM;
Daniel De Graaf
2012-Jan-11 17:36 UTC
Re: [PATCH 03/18] xsm: allow use of XEN_DOMCTL_getdomaininfo by non-IS_PRIV domains
On 01/11/2012 12:27 PM, Keir Fraser wrote:> On 11/01/2012 17:21, "Daniel De Graaf" <dgdegra@tycho.nsa.gov> wrote: > >> This domctl does not allow manipulation of domains, only basic >> information such as size and state. XSM modules can also provide >> fine-grained control over what domains are visible to domains that call >> getdomaininfo. > > Well there''s a reason we might not disallow the hypercall. But why would we > actually care to allow it? > > -- Keir >Xenstored needs to be able to call getdomaininfo to determine what domain(s) have been shut down when it receives the DOM_EXC VIRQ. This is used both to clean up references to the domain (event/grant) and to fire the @releaseDomain watch so that other domains also clean up after the domain. Other than this hypercall, there is no reason to make xenstored''s domain privileged (the VIRQ must be delegated by dom0).>> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> >> --- >> xen/common/domctl.c | 4 ++++ >> 1 files changed, 4 insertions(+), 0 deletions(-) >> >> diff --git a/xen/common/domctl.c b/xen/common/domctl.c >> index a775aa3..2c1ca85 100644 >> --- a/xen/common/domctl.c >> +++ b/xen/common/domctl.c >> @@ -263,6 +263,10 @@ long do_domctl(XEN_GUEST_HANDLE(xen_domctl_t) u_domctl) >> return -EPERM; >> break; >> } >> +#ifdef XSM_ENABLE >> + case XEN_DOMCTL_getdomaininfo: >> + break; >> +#endif >> default: >> if ( !IS_PRIV(current->domain) ) >> return -EPERM; >
Keir Fraser
2012-Jan-11 17:49 UTC
Re: [PATCH 03/18] xsm: allow use of XEN_DOMCTL_getdomaininfo by non-IS_PRIV domains
On 11/01/2012 17:27, "Keir Fraser" <keir@xen.org> wrote:> On 11/01/2012 17:21, "Daniel De Graaf" <dgdegra@tycho.nsa.gov> wrote: > >> This domctl does not allow manipulation of domains, only basic >> information such as size and state. XSM modules can also provide >> fine-grained control over what domains are visible to domains that call >> getdomaininfo. > > Well there''s a reason we might not disallow the hypercall. But why would we > actually care to allow it?Ah, I''ve now seen patch 00/18, so this is for xenstore stubdom. Also this applies only to the XSM-enabled case, and just allows you to get as far as the finer-grained xsm_getdomaininfo() check. Somehow I got the ifdef the wrong way round in my head! Okay, makes a lot of sense. However, if the dummy xsm module is supposed to behave very similarly to a !XSM_ENABLE build (which is what I personally would expect), then I think dummy_getdomaininfo() should be changed to return 0 only when IS_PRIV(current->domain). This of course will require a ''proper'' XSM setup to be able to use the xenstore stubdom, but probably setting eg XSM/Flask should be a core part of setting up such a hardened Xen host anyway. -- Keir> -- Keir > >> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> >> --- >> xen/common/domctl.c | 4 ++++ >> 1 files changed, 4 insertions(+), 0 deletions(-) >> >> diff --git a/xen/common/domctl.c b/xen/common/domctl.c >> index a775aa3..2c1ca85 100644 >> --- a/xen/common/domctl.c >> +++ b/xen/common/domctl.c >> @@ -263,6 +263,10 @@ long do_domctl(XEN_GUEST_HANDLE(xen_domctl_t) u_domctl) >> return -EPERM; >> break; >> } >> +#ifdef XSM_ENABLE >> + case XEN_DOMCTL_getdomaininfo: >> + break; >> +#endif >> default: >> if ( !IS_PRIV(current->domain) ) >> return -EPERM; > >
Jan Beulich
2012-Jan-12 08:22 UTC
Re: [PATCH 01/18] xen: reinstate previously unused XENMEM_remove_from_physmap hypercall
>>> On 11.01.12 at 18:21, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > --- a/xen/include/xlat.lst > +++ b/xen/include/xlat.lst > @@ -54,6 +54,7 @@ > ! kexec_image kexec.h > ! kexec_range kexec.h > ! add_to_physmap memory.h > +! remove_from_physmap memory.h > ! foreign_memory_map memory.h > ! memory_exchange memory.h > ! memory_map memory.hAlphabetically, please. Jan
Jan Beulich
2012-Jan-12 08:43 UTC
Re: [PATCH 02/18] xen: allow global VIRQ handlers to be delegated to other domains
>>> On 11.01.12 at 18:21, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > @@ -659,6 +659,8 @@ static void complete_domain_destroy(struct rcu_head *head) > > watchdog_domain_destroy(d); > > + clear_global_virq_handlers(d);This is too late, I''m afraid. The domain can''t possibly service the vIRQ(s) anymore as soon as its destruction begins.> + > rangeset_domain_destroy(d); > > cpupool_rm_domain(d); > @@ -739,6 +739,67 @@ int send_guest_pirq(struct domain *d, const struct pirq *pirq) > return evtchn_set_pending(d->vcpu[chn->notify_vcpu_id], port); > } > > +static struct domain* global_virq_handlers[NR_VIRQS];__read_mostly? Also, the formatting is wrong (also elsewhere in the patch) - the space should be before the star, not after.> + > +spinlock_t global_virq_handlers_lock = SPIN_LOCK_UNLOCKED;static? DEFINE_SPINLOCK()?> + > +static struct domain* _get_global_virq_handler(int virq) > +{ > + struct domain *d; > + > + d = global_virq_handlers[virq]; > + return d != NULL ? d : dom0;This can be done in a single line: return global_virq_handlers[virq] ?: dom0;> +} > + > +void send_global_virq(int virq) > +{ > + ASSERT(virq >= 0 && virq < NR_VIRQS); > + ASSERT(virq_is_global(virq)); > + > + send_guest_global_virq(_get_global_virq_handler(virq), virq); > +} > + > +int set_global_virq_handler(struct domain *d, int virq)In the domctl interface structure the virq (correctly) is uint32_t, so why is it (signed) int here (and elsewhere)?> +{ > + struct domain *old; > + > + if (virq < 0 || virq >= NR_VIRQS)The < 0 part would then become unnecessary.> + return -EINVAL; > + if (!virq_is_global(virq)) > + return -EINVAL; > + > + if (global_virq_handlers[virq] == d) > + return 0; > + > + if (unlikely(!get_domain(d))) > + return -EINVAL; > + > + spin_lock(&global_virq_handlers_lock); > + > + old = global_virq_handlers[virq]; > + global_virq_handlers[virq] = d; > + if (old != NULL) > + put_domain(old);This should happen outside the lock.> + spin_unlock(&global_virq_handlers_lock); > + > + return 0; > +} > + > +void clear_global_virq_handlers(struct domain *d) > +{ > + int virq; > + > + spin_lock(&global_virq_handlers_lock); > + > + for (virq = 0; virq < NR_VIRQS; virq++) { > + if (global_virq_handlers[virq] == d) { > + global_virq_handlers[virq] = NULL; > + put_domain(d);Same here (albeit resulting in some code growth).> + } > + } > + > + spin_unlock(&global_virq_handlers_lock); > +} > > static long evtchn_status(evtchn_status_t *status) > { > @@ -912,6 +918,7 @@ struct xen_domctl { > #define XEN_DOMCTL_getvcpuextstate 63 > #define XEN_DOMCTL_set_access_required 64 > #define XEN_DOMCTL_audit_p2m 65 > +#define XEN_DOMCTL_set_virq_handler 71Any reason for picking a non-contiguous value here?> #define XEN_DOMCTL_gdbsx_guestmemio 1000 > #define XEN_DOMCTL_gdbsx_pausevcpu 1001 > #define XEN_DOMCTL_gdbsx_unpausevcpu 1002Jan
Jan Beulich
2012-Jan-12 08:53 UTC
Re: [PATCH 04/18] xen: Preserve reserved grant entries when switching versions
>>> On 11.01.12 at 18:21, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > --- a/xen/common/grant_table.c > +++ b/xen/common/grant_table.c > @@ -2106,6 +2106,7 @@ gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) > struct domain *d = current->domain; > struct grant_table *gt = d->grant_table; > struct active_grant_entry *act; > + grant_entry_v1_t reserved_entries[GNTTAB_NR_RESERVED_ENTRIES];How much stack space does this consume? (Or in other words, where does GNTTAB_NR_RESERVED_ENTRIES get defined? It''s not in this patch, I can''t find it in earlier patches, and it also doesn''t exist in the current staging tree.) Jan> long res; > int i; > > @@ -2126,7 +2127,7 @@ > gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) > /* (You need to change the version number for e.g. kexec.) */ > if ( gt->gt_version != 0 ) > { > - for ( i = 0; i < nr_grant_entries(gt); i++ ) > + for ( i = GNTTAB_NR_RESERVED_ENTRIES; i < nr_grant_entries(gt); i++ > ) > { > act = &active_entry(gt, i); > if ( act->pin != 0 ) > @@ -2151,15 +2152,38 @@ > gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) > goto out_unlock; > } > > + /* Preserve the first 8 entries (toolstack reserved grants) */ > + if (gt->gt_version == 1) { > + memcpy(reserved_entries, gt->shared_v1[0], sizeof(reserved_entries)); > + } else if (gt->gt_version == 2) { > + for ( i = 0; i < GNTTAB_NR_RESERVED_ENTRIES && i < > nr_grant_entries(gt); i++ ) > + { > + reserved_entries[i].flags = shared_entry_v2(gt, i).hdr.flags; > + reserved_entries[i].domid = shared_entry_v2(gt, i).hdr.domid; > + reserved_entries[i].frame = shared_entry_v2(gt, > i).full_page.frame; > + reserved_entries[i].flags |= status_entry(gt, i); > + } > + } > + > if ( op.version < 2 && gt->gt_version == 2 ) > gnttab_unpopulate_status_frames(d, gt); > > - if ( op.version != gt->gt_version ) > - { > - /* Make sure there''s no crud left over in the table from the > - old version. */ > - for ( i = 0; i < nr_grant_frames(gt); i++ ) > - memset(gt->shared_raw[i], 0, PAGE_SIZE); > + /* Make sure there''s no crud left over in the table from the > + old version. */ > + for ( i = 0; i < nr_grant_frames(gt); i++ ) > + memset(gt->shared_raw[i], 0, PAGE_SIZE); > + > + /* Restore the first 8 entries (toolstack reserved grants) */ > + if (gt->gt_version != 0 && op.version == 1) { > + memcpy(gt->shared_v1[0], reserved_entries, sizeof(reserved_entries)); > + } else if (gt->gt_version != 0 && op.version == 2) { > + for ( i = 0; i < GNTTAB_NR_RESERVED_ENTRIES; i++ ) > + { > + status_entry(gt, i) = reserved_entries[i].flags & > (GTF_reading|GTF_writing); > + shared_entry_v2(gt, i).hdr.flags = reserved_entries[i].flags & > ~(GTF_reading|GTF_writing); > + shared_entry_v2(gt, i).hdr.domid = reserved_entries[i].domid; > + shared_entry_v2(gt, i).full_page.frame = > reserved_entries[i].frame; > + } > } > > gt->gt_version = op.version; > -- > 1.7.7.5 > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel
Jan Beulich
2012-Jan-12 08:59 UTC
Re: [PATCH] xenbus: Add support for xenbus backend in stub domain
>>> On 11.01.12 at 18:22, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > This adds two ioctls to the /dev/xen/xenbus_backend device allowing the > xenbus backend to be started after the kernel has booted. This is > intended to allow dom0 to start another domain to run xenstore. > > IOCTL_XENBUS_BACKEND_REMOTE_SETUP requires that the xenstore domain be > started; the domain ID is passed as the ioctl argument. This sets up a > listening event channel and grant reference to xenbus, but does not > start using this interface. It returns the local event channel port.Any chance to get at least the setup part matched with the legacy Linux implementation (defining IOCTL_XENBUS_ALLOC)? Jan> IOCTL_XENBUS_BACKEND_REMOTE_COMMIT causes the kernel to begin using the > event channel set up in the previous ioctl, reregistering all watches > and deallocating the previous xenstored event channel. > > Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > --- > drivers/xen/xenbus/xenbus_comms.c | 6 +++ > drivers/xen/xenbus/xenbus_comms.h | 1 + > drivers/xen/xenbus/xenbus_dev_backend.c | 57 > +++++++++++++++++++++++++++++++ > include/xen/grant_table.h | 2 + > include/xen/xenbus_dev.h | 6 +++ > 5 files changed, 72 insertions(+), 0 deletions(-) > > diff --git a/drivers/xen/xenbus/xenbus_comms.c > b/drivers/xen/xenbus/xenbus_comms.c > index 2eff7a6..52fe7ad 100644 > --- a/drivers/xen/xenbus/xenbus_comms.c > +++ b/drivers/xen/xenbus/xenbus_comms.c > @@ -234,3 +234,9 @@ int xb_init_comms(void) > > return 0; > } > + > +void xb_deinit_comms(void) > +{ > + unbind_from_irqhandler(xenbus_irq, &xb_waitq); > + xenbus_irq = 0; > +} > diff --git a/drivers/xen/xenbus/xenbus_comms.h > b/drivers/xen/xenbus/xenbus_comms.h > index 6e42800..c8abd3b 100644 > --- a/drivers/xen/xenbus/xenbus_comms.h > +++ b/drivers/xen/xenbus/xenbus_comms.h > @@ -35,6 +35,7 @@ > > int xs_init(void); > int xb_init_comms(void); > +void xb_deinit_comms(void); > > /* Low level routines. */ > int xb_write(const void *data, unsigned len); > diff --git a/drivers/xen/xenbus/xenbus_dev_backend.c > b/drivers/xen/xenbus/xenbus_dev_backend.c > index 3d3be78..4138ba2 100644 > --- a/drivers/xen/xenbus/xenbus_dev_backend.c > +++ b/drivers/xen/xenbus/xenbus_dev_backend.c > @@ -8,7 +8,11 @@ > > #include <xen/xen.h> > #include <xen/page.h> > +#include <xen/xenbus.h> > #include <xen/xenbus_dev.h> > +#include <xen/grant_table.h> > +#include <xen/events.h> > +#include <asm/xen/hypervisor.h> > > #include "xenbus_comms.h" > > @@ -22,6 +26,53 @@ static int xenbus_backend_open(struct inode *inode, struct > file *filp) > return nonseekable_open(inode, filp); > } > > +static int pending_xsd_port; > + > +static long xenbus_backend_remote_setup(domid_t domid) > +{ > + struct evtchn_alloc_unbound arg; > + int err; > + > + gnttab_grant_foreign_access_ref(GNTTAB_RESERVED_XENSTORE, domid, > + virt_to_mfn(xen_store_interface), 0 /* writable */); > + > + if (pending_xsd_port) { > + struct evtchn_close close; > + close.port = pending_xsd_port; > + err = HYPERVISOR_event_channel_op(EVTCHNOP_close, &close); > + WARN_ON(err); > + } > + > + arg.dom = DOMID_SELF; > + arg.remote_dom = domid; > + > + err = HYPERVISOR_event_channel_op(EVTCHNOP_alloc_unbound, &arg); > + if (err) > + return err; > + > + pending_xsd_port = arg.port; > + > + return arg.port; > +} > + > +static long xenbus_backend_remote_commit(unsigned long data) > +{ > + if (!pending_xsd_port) > + return -EINVAL; > + > + xs_suspend(); > + > + if (xen_store_evtchn > 0) > + xb_deinit_comms(); > + > + xen_store_evtchn = pending_xsd_port; > + pending_xsd_port = 0; > + > + xs_resume(); > + > + return 0; > +} > + > static long xenbus_backend_ioctl(struct file *file, unsigned int cmd, > unsigned long data) > { > if (!capable(CAP_SYS_ADMIN)) > @@ -33,6 +84,12 @@ static long xenbus_backend_ioctl(struct file *file, > unsigned int cmd, unsigned l > return xen_store_evtchn; > return -ENODEV; > > + case IOCTL_XENBUS_BACKEND_REMOTE_SETUP: > + return xenbus_backend_remote_setup(data); > + > + case IOCTL_XENBUS_BACKEND_REMOTE_COMMIT: > + return xenbus_backend_remote_commit(data); > + > default: > return -ENOTTY; > } > diff --git a/include/xen/grant_table.h b/include/xen/grant_table.h > index 15f8a00..11e27c3 100644 > --- a/include/xen/grant_table.h > +++ b/include/xen/grant_table.h > @@ -46,6 +46,8 @@ > > #include <xen/features.h> > > +#define GNTTAB_RESERVED_XENSTORE 1 > + > /* NR_GRANT_FRAMES must be less than or equal to that configured in Xen */ > #define NR_GRANT_FRAMES 4 > > diff --git a/include/xen/xenbus_dev.h b/include/xen/xenbus_dev.h > index ac5f0fe..24d5028 100644 > --- a/include/xen/xenbus_dev.h > +++ b/include/xen/xenbus_dev.h > @@ -38,4 +38,10 @@ > #define IOCTL_XENBUS_BACKEND_EVTCHN \ > _IOC(_IOC_NONE, ''B'', 0, 0) > > +#define IOCTL_XENBUS_BACKEND_REMOTE_SETUP \ > + _IOC(_IOC_NONE, ''B'', 1, 0) > + > +#define IOCTL_XENBUS_BACKEND_REMOTE_COMMIT \ > + _IOC(_IOC_NONE, ''B'', 2, 0) > + > #endif /* __LINUX_XEN_XENBUS_DEV_H__ */ > -- > 1.7.7.5 > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel
Ian Campbell
2012-Jan-12 09:49 UTC
Re: [PATCH 04/18] xen: Preserve reserved grant entries when switching versions
On Thu, 2012-01-12 at 08:53 +0000, Jan Beulich wrote:> >>> On 11.01.12 at 18:21, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > > --- a/xen/common/grant_table.c > > +++ b/xen/common/grant_table.c > > @@ -2106,6 +2106,7 @@ gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) > > struct domain *d = current->domain; > > struct grant_table *gt = d->grant_table; > > struct active_grant_entry *act; > > + grant_entry_v1_t reserved_entries[GNTTAB_NR_RESERVED_ENTRIES]; > > How much stack space does this consume? (Or in other words, where > does GNTTAB_NR_RESERVED_ENTRIES get defined? It''s not in this > patch, I can''t find it in earlier patches, and it also doesn''t exist in the > current staging tree.)I could have sworn I''d seen this somewhere but I can''t for the life of me find it now :-/ One place I did find it is in the Linux gnttab driver, but that''s not canonical. Ian.> > Jan > > > long res; > > int i; > > > > @@ -2126,7 +2127,7 @@ > > gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) > > /* (You need to change the version number for e.g. kexec.) */ > > if ( gt->gt_version != 0 ) > > { > > - for ( i = 0; i < nr_grant_entries(gt); i++ ) > > + for ( i = GNTTAB_NR_RESERVED_ENTRIES; i < nr_grant_entries(gt); i++ > > ) > > { > > act = &active_entry(gt, i); > > if ( act->pin != 0 ) > > @@ -2151,15 +2152,38 @@ > > gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) > > goto out_unlock; > > } > > > > + /* Preserve the first 8 entries (toolstack reserved grants) */ > > + if (gt->gt_version == 1) { > > + memcpy(reserved_entries, gt->shared_v1[0], sizeof(reserved_entries)); > > + } else if (gt->gt_version == 2) { > > + for ( i = 0; i < GNTTAB_NR_RESERVED_ENTRIES && i < > > nr_grant_entries(gt); i++ ) > > + { > > + reserved_entries[i].flags = shared_entry_v2(gt, i).hdr.flags; > > + reserved_entries[i].domid = shared_entry_v2(gt, i).hdr.domid; > > + reserved_entries[i].frame = shared_entry_v2(gt, > > i).full_page.frame; > > + reserved_entries[i].flags |= status_entry(gt, i); > > + } > > + } > > + > > if ( op.version < 2 && gt->gt_version == 2 ) > > gnttab_unpopulate_status_frames(d, gt); > > > > - if ( op.version != gt->gt_version ) > > - { > > - /* Make sure there''s no crud left over in the table from the > > - old version. */ > > - for ( i = 0; i < nr_grant_frames(gt); i++ ) > > - memset(gt->shared_raw[i], 0, PAGE_SIZE); > > + /* Make sure there''s no crud left over in the table from the > > + old version. */ > > + for ( i = 0; i < nr_grant_frames(gt); i++ ) > > + memset(gt->shared_raw[i], 0, PAGE_SIZE); > > + > > + /* Restore the first 8 entries (toolstack reserved grants) */ > > + if (gt->gt_version != 0 && op.version == 1) { > > + memcpy(gt->shared_v1[0], reserved_entries, sizeof(reserved_entries)); > > + } else if (gt->gt_version != 0 && op.version == 2) { > > + for ( i = 0; i < GNTTAB_NR_RESERVED_ENTRIES; i++ ) > > + { > > + status_entry(gt, i) = reserved_entries[i].flags & > > (GTF_reading|GTF_writing); > > + shared_entry_v2(gt, i).hdr.flags = reserved_entries[i].flags & > > ~(GTF_reading|GTF_writing); > > + shared_entry_v2(gt, i).hdr.domid = reserved_entries[i].domid; > > + shared_entry_v2(gt, i).full_page.frame = > > reserved_entries[i].frame; > > + } > > } > > > > gt->gt_version = op.version; > > -- > > 1.7.7.5 > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel
On Wed, 2012-01-11 at 17:21 +0000, Daniel De Graaf wrote:> This patch series allows xenstored to run in a stub domian started by > dom0. It is based on a patch series posted by Alex Zeffertt in 2009 - > http://old-list-archives.xen.org/archives/html/xen-devel/2009-03/msg01488.html > > > A domain configuration for starting xenstored looks like: > > kernel=''/home/daniel/xen/stubdom/mini-os-x86_64-xenstore/mini-os'' > extra='''' > memory=50 > name=''xenstore'' > > Once xenstore is started, "xenstore_dom=1" needs to be added to other > domain''s configurations in order to set up the xenstore connection to > domain 1.The domid should be written to xenstore so that the toolstack can pull it back out as necessary.> The following program handles post-creation parts of xenstored. To use > it, run "xl create -p xenstore" and then "init-xenstore $domid". The > running xenstored must be stopped to prevent xl using the UNIX sockets, > and xenconsoled needs to be restarted after switching xenstores.So the model is that you start a normal xenstored process in dom0 and then use it to start the stub-xenstore before switching over? How does that work wrt watches which are already registered, e.g. by backend drivers? I had imagined a init-xenstore.c which actually built the domain using libxc directly from a mostly hardcoded configuration as well as performing the necessary IOCTLs to get around the handover issue. Ian,> > /* init-xenstore.c: link with -lxenctrl */ > > #include <fcntl.h> > #include <stdio.h> > #include <string.h> > #include <stdint.h> > #include <stdlib.h> > #include <sys/ioctl.h> > #include <sys/mman.h> > > #define __XEN_TOOLS__ > #include <xen/domctl.h> > #include "xenctrl.h" > > #define IOCTL_XENBUS_BACKEND_SETUP _IOC(_IOC_NONE, ''B'', 1, 0) > #define IOCTL_XENBUS_BACKEND_COMMIT _IOC(_IOC_NONE, ''B'', 2, 0) > > static void set_virq(int domid, int virq) > { > struct xen_domctl command; > xc_interface *xch; > > xch = xc_interface_open(NULL, NULL, 0); > > memset(&command, 0, sizeof(command)); > command.cmd = XEN_DOMCTL_set_virq_handler; > command.interface_version = XEN_DOMCTL_INTERFACE_VERSION; > command.domain = domid; > command.u.set_virq_handler.virq = virq; > xc_domctl(xch, &command); > xc_interface_close(xch); > } > > int main(int argc, char** argv) > { > char buf[512]; > int domid = atoi(argv[1]); > > set_virq(domid, VIRQ_DOM_EXC); > > int fd = open("/dev/xen/xenbus_backend", O_RDWR); > void *map = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0); > int rv = ioctl(fd, IOCTL_XENBUS_BACKEND_SETUP, domid); > *(uint16_t*)(map + 0x810) = rv; > snprintf(buf, 512, "xl unpause %d", domid); > system(buf); > ioctl(fd, IOCTL_XENBUS_BACKEND_COMMIT, 0); > return 0; > } > > ------------------------------------------------- > > Dom0 kernel changes: > [PATCH] xenbus: Add support for xenbus backend in stub domain > > This is based on the new /dev/xen devices introduced in Linux 3.3. > > Hypervisor changes: > [PATCH 01/18] xen: reinstate previously unused > [PATCH 02/18] xen: allow global VIRQ handlers to be delegated to > [PATCH 03/18] xsm: allow use of XEN_DOMCTL_getdomaininfo by > [PATCH 04/18] xen: Preserve reserved grant entries when switching > > Patch 1 & 4 are required for setting up grant entries in new domains. > Patch 2 & 3 allow xenstored to run in an unprivileged domain. This > currently requires XSM to be enabled to avoid allowing all domUs access > to XEN_DOMCTL_getdomaininfo, so the patch only allows this hypercall if > XSM is being compiled in. > > Toolstack changes: > [PATCH 05/18] tools/libxl: Add xenstore and console backend domain > [PATCH 06/18] lib{xc,xl}: Seed grant tables with xenstore and > > These patches populate two of the eight reserved grant entries in new > domains with the xenstore and console shared pages, which is required > if xenstored is not run in a privileged domain. > > Minios and xenstored: > [PATCH 07/18] mini-os: avoid crash if no console is provided > [PATCH 08/18] mini-os: avoid crash if no xenstore is provided > [PATCH 09/18] mini-os: remove per-fd evtchn limit > [PATCH 10/18] xenstored: use grant references instead of > [PATCH 11/18] xenstored: add NO_SOCKETS compilation option > [PATCH 12/18] xenstored support for in-memory rather than FS based > [PATCH 13/18] xenstored: support running in minios stubdom > [PATCH 14/18] xenstored: always use xc_gnttab_munmap in stubdom > [PATCH 15/18] xenstored: add --event parameter for bootstrapping > [PATCH 16/18] xenstored: pull dom0 event port from shared page > [PATCH 17/18] xenstored: use domain_is_unprivileged instead of > [PATCH 18/18] xenstored: add --priv-domid parameter > > Support for running in a stub domain > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel
Ian Campbell
2012-Jan-12 09:56 UTC
Re: [PATCH 04/18] xen: Preserve reserved grant entries when switching versions
On Thu, 2012-01-12 at 09:49 +0000, Ian Campbell wrote:> On Thu, 2012-01-12 at 08:53 +0000, Jan Beulich wrote: > > >>> On 11.01.12 at 18:21, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > > > --- a/xen/common/grant_table.c > > > +++ b/xen/common/grant_table.c > > > @@ -2106,6 +2106,7 @@ gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) > > > struct domain *d = current->domain; > > > struct grant_table *gt = d->grant_table; > > > struct active_grant_entry *act; > > > + grant_entry_v1_t reserved_entries[GNTTAB_NR_RESERVED_ENTRIES]; > > > > How much stack space does this consume? (Or in other words, where > > does GNTTAB_NR_RESERVED_ENTRIES get defined? It''s not in this > > patch, I can''t find it in earlier patches, and it also doesn''t exist in the > > current staging tree.) > > I could have sworn I''d seen this somewhere but I can''t for the life of > me find it now :-/Found it in patch #6. +/* External tools reserve first few grant table entries. */ +#define GNTTAB_NR_RESERVED_ENTRIES 8 +#define GNTTAB_RESERVED_CONSOLE 0 +#define GNTTAB_RESERVED_XENSTORE 1 +/* (the next 6 are reserved for future use) */ These patches need to be reordered to make sure each step builds. Ian.> > One place I did find it is in the Linux gnttab driver, but that''s not > canonical. > > Ian. > > > > > Jan > > > > > long res; > > > int i; > > > > > > @@ -2126,7 +2127,7 @@ > > > gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) > > > /* (You need to change the version number for e.g. kexec.) */ > > > if ( gt->gt_version != 0 ) > > > { > > > - for ( i = 0; i < nr_grant_entries(gt); i++ ) > > > + for ( i = GNTTAB_NR_RESERVED_ENTRIES; i < nr_grant_entries(gt); i++ > > > ) > > > { > > > act = &active_entry(gt, i); > > > if ( act->pin != 0 ) > > > @@ -2151,15 +2152,38 @@ > > > gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) > > > goto out_unlock; > > > } > > > > > > + /* Preserve the first 8 entries (toolstack reserved grants) */ > > > + if (gt->gt_version == 1) { > > > + memcpy(reserved_entries, gt->shared_v1[0], sizeof(reserved_entries)); > > > + } else if (gt->gt_version == 2) { > > > + for ( i = 0; i < GNTTAB_NR_RESERVED_ENTRIES && i < > > > nr_grant_entries(gt); i++ ) > > > + { > > > + reserved_entries[i].flags = shared_entry_v2(gt, i).hdr.flags; > > > + reserved_entries[i].domid = shared_entry_v2(gt, i).hdr.domid; > > > + reserved_entries[i].frame = shared_entry_v2(gt, > > > i).full_page.frame; > > > + reserved_entries[i].flags |= status_entry(gt, i); > > > + } > > > + } > > > + > > > if ( op.version < 2 && gt->gt_version == 2 ) > > > gnttab_unpopulate_status_frames(d, gt); > > > > > > - if ( op.version != gt->gt_version ) > > > - { > > > - /* Make sure there''s no crud left over in the table from the > > > - old version. */ > > > - for ( i = 0; i < nr_grant_frames(gt); i++ ) > > > - memset(gt->shared_raw[i], 0, PAGE_SIZE); > > > + /* Make sure there''s no crud left over in the table from the > > > + old version. */ > > > + for ( i = 0; i < nr_grant_frames(gt); i++ ) > > > + memset(gt->shared_raw[i], 0, PAGE_SIZE); > > > + > > > + /* Restore the first 8 entries (toolstack reserved grants) */ > > > + if (gt->gt_version != 0 && op.version == 1) { > > > + memcpy(gt->shared_v1[0], reserved_entries, sizeof(reserved_entries)); > > > + } else if (gt->gt_version != 0 && op.version == 2) { > > > + for ( i = 0; i < GNTTAB_NR_RESERVED_ENTRIES; i++ ) > > > + { > > > + status_entry(gt, i) = reserved_entries[i].flags & > > > (GTF_reading|GTF_writing); > > > + shared_entry_v2(gt, i).hdr.flags = reserved_entries[i].flags & > > > ~(GTF_reading|GTF_writing); > > > + shared_entry_v2(gt, i).hdr.domid = reserved_entries[i].domid; > > > + shared_entry_v2(gt, i).full_page.frame = > > > reserved_entries[i].frame; > > > + } > > > } > > > > > > gt->gt_version = op.version; > > > -- > > > 1.7.7.5 > > > > > > > > > _______________________________________________ > > > Xen-devel mailing list > > > Xen-devel@lists.xensource.com > > > http://lists.xensource.com/xen-devel > > > > > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel
On Wed, 2012-01-11 at 17:21 +0000, Daniel De Graaf wrote:> This patch series allows xenstored to run in a stub domian started by > dom0. It is based on a patch series posted by Alex Zeffertt in 2009 - > http://old-list-archives.xen.org/archives/html/xen-devel/2009-03/msg01488.htmlBTW did you come across any patches for stub-oxenstored while you were doing this? I thought there had been such at some point in the past. Ian.> > > A domain configuration for starting xenstored looks like: > > kernel=''/home/daniel/xen/stubdom/mini-os-x86_64-xenstore/mini-os'' > extra='''' > memory=50 > name=''xenstore'' > > Once xenstore is started, "xenstore_dom=1" needs to be added to other > domain''s configurations in order to set up the xenstore connection to > domain 1. > > The following program handles post-creation parts of xenstored. To use > it, run "xl create -p xenstore" and then "init-xenstore $domid". The > running xenstored must be stopped to prevent xl using the UNIX sockets, > and xenconsoled needs to be restarted after switching xenstores. > > /* init-xenstore.c: link with -lxenctrl */ > > #include <fcntl.h> > #include <stdio.h> > #include <string.h> > #include <stdint.h> > #include <stdlib.h> > #include <sys/ioctl.h> > #include <sys/mman.h> > > #define __XEN_TOOLS__ > #include <xen/domctl.h> > #include "xenctrl.h" > > #define IOCTL_XENBUS_BACKEND_SETUP _IOC(_IOC_NONE, ''B'', 1, 0) > #define IOCTL_XENBUS_BACKEND_COMMIT _IOC(_IOC_NONE, ''B'', 2, 0) > > static void set_virq(int domid, int virq) > { > struct xen_domctl command; > xc_interface *xch; > > xch = xc_interface_open(NULL, NULL, 0); > > memset(&command, 0, sizeof(command)); > command.cmd = XEN_DOMCTL_set_virq_handler; > command.interface_version = XEN_DOMCTL_INTERFACE_VERSION; > command.domain = domid; > command.u.set_virq_handler.virq = virq; > xc_domctl(xch, &command); > xc_interface_close(xch); > } > > int main(int argc, char** argv) > { > char buf[512]; > int domid = atoi(argv[1]); > > set_virq(domid, VIRQ_DOM_EXC); > > int fd = open("/dev/xen/xenbus_backend", O_RDWR); > void *map = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0); > int rv = ioctl(fd, IOCTL_XENBUS_BACKEND_SETUP, domid); > *(uint16_t*)(map + 0x810) = rv; > snprintf(buf, 512, "xl unpause %d", domid); > system(buf); > ioctl(fd, IOCTL_XENBUS_BACKEND_COMMIT, 0); > return 0; > } > > ------------------------------------------------- > > Dom0 kernel changes: > [PATCH] xenbus: Add support for xenbus backend in stub domain > > This is based on the new /dev/xen devices introduced in Linux 3.3. > > Hypervisor changes: > [PATCH 01/18] xen: reinstate previously unused > [PATCH 02/18] xen: allow global VIRQ handlers to be delegated to > [PATCH 03/18] xsm: allow use of XEN_DOMCTL_getdomaininfo by > [PATCH 04/18] xen: Preserve reserved grant entries when switching > > Patch 1 & 4 are required for setting up grant entries in new domains. > Patch 2 & 3 allow xenstored to run in an unprivileged domain. This > currently requires XSM to be enabled to avoid allowing all domUs access > to XEN_DOMCTL_getdomaininfo, so the patch only allows this hypercall if > XSM is being compiled in. > > Toolstack changes: > [PATCH 05/18] tools/libxl: Add xenstore and console backend domain > [PATCH 06/18] lib{xc,xl}: Seed grant tables with xenstore and > > These patches populate two of the eight reserved grant entries in new > domains with the xenstore and console shared pages, which is required > if xenstored is not run in a privileged domain. > > Minios and xenstored: > [PATCH 07/18] mini-os: avoid crash if no console is provided > [PATCH 08/18] mini-os: avoid crash if no xenstore is provided > [PATCH 09/18] mini-os: remove per-fd evtchn limit > [PATCH 10/18] xenstored: use grant references instead of > [PATCH 11/18] xenstored: add NO_SOCKETS compilation option > [PATCH 12/18] xenstored support for in-memory rather than FS based > [PATCH 13/18] xenstored: support running in minios stubdom > [PATCH 14/18] xenstored: always use xc_gnttab_munmap in stubdom > [PATCH 15/18] xenstored: add --event parameter for bootstrapping > [PATCH 16/18] xenstored: pull dom0 event port from shared page > [PATCH 17/18] xenstored: use domain_is_unprivileged instead of > [PATCH 18/18] xenstored: add --priv-domid parameter > > Support for running in a stub domain > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel
Ian Campbell
2012-Jan-12 09:59 UTC
Re: [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants
On Wed, 2012-01-11 at 17:21 +0000, Daniel De Graaf wrote:> From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > > This patch claims one reserved grant entry for the console and another > for the xenstore. It modifies the builder to fill in the grant table > entries for the console and the xenstore. > > This has not been tested with any kind of migration. >[...]> diff --git a/tools/libxc/xc_domain_restore.c b/tools/libxc/xc_domain_restore.c > index 3fda6f8..23619da 100644 > --- a/tools/libxc/xc_domain_restore.c > +++ b/tools/libxc/xc_domain_restore.c > @@ -2018,6 +2018,15 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, > memcpy(ctx->live_p2m, ctx->p2m, dinfo->p2m_size * sizeof(xen_pfn_t)); > munmap(ctx->live_p2m, P2M_FL_ENTRIES * PAGE_SIZE); > > +/* TODO don''t hardcode zero here */ > + rc = xc_dom_gnttab_seed(xch, dom, > + *console_mfn, *store_mfn, 0, 0);Presumably this TODO is the source of the comment in the changelog WRT migration. Does it Just Work or is there a legitimate TODO item here? Ian.
Ian Campbell
2012-Jan-12 10:03 UTC
Re: [PATCH 07/18] mini-os: avoid crash if no console is provided
On Wed, 2012-01-11 at 17:21 +0000, Daniel De Graaf wrote: When/why does this happen? I guess it is because when starting the xenstore domain you cannot use xenstore to communicate with xenconsoled (and/or it isn''t even running yet). I wonder if there is a way we can do lazy-setup of the console for just the xenstore domain? Alternatively I seem to recall a little tool which Diego wrote to pull the console ring out of a domain directly as a debuging aid but that relies on us setting up a console ring and evtchn even if xenconsoled cannot be involved which makes this patch unnecessary. Ian.> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > --- > extras/mini-os/console/xencons_ring.c | 7 ++++++- > 1 files changed, 6 insertions(+), 1 deletions(-) > > diff --git a/extras/mini-os/console/xencons_ring.c b/extras/mini-os/console/xencons_ring.c > index 22fd618..14a8bd1 100644 > --- a/extras/mini-os/console/xencons_ring.c > +++ b/extras/mini-os/console/xencons_ring.c > @@ -25,7 +25,10 @@ static inline void notify_daemon(struct consfront_dev *dev) > > static inline struct xencons_interface *xencons_interface(void) > { > - return mfn_to_virt(start_info.console.domU.mfn); > + if (start_info.console.domU.evtchn) > + return mfn_to_virt(start_info.console.domU.mfn); > + else > + return NULL; > } > > int xencons_ring_send_no_notify(struct consfront_dev *dev, const char *data, unsigned len) > @@ -38,6 +41,8 @@ int xencons_ring_send_no_notify(struct consfront_dev *dev, const char *data, uns > intf = xencons_interface(); > else > intf = dev->ring; > + if (!intf) > + return sent; > > cons = intf->out_cons; > prod = intf->out_prod;
Ian Campbell
2012-Jan-12 10:05 UTC
Re: [PATCH 11/18] xenstored: add NO_SOCKETS compilation option
On Wed, 2012-01-11 at 17:21 +0000, Daniel De Graaf wrote:> @@ -308,7 +310,10 @@ static void set_fd(int fd, fd_set *set, int *max) > } > > > -static int initialize_set(fd_set *inset, fd_set *outset, int sock, > int ro_sock, > +static int initialize_set(fd_set *inset, fd_set *outset, > +#ifndef NO_SOCKETS > + int sock, int ro_sock, > +#endif > struct timeval **ptimeout) > { > static struct timeval zero_timeout = { 0 };Rather than removing the argument why not simply accept -1 as a "no sock" value and DTWT with that, this ought to reduce the amount of ifdeffery substantially Ian.
Ian Campbell
2012-Jan-12 10:20 UTC
Re: [PATCH 18/18] xenstored: add --priv-domid parameter
On Wed, 2012-01-11 at 17:21 +0000, Daniel De Graaf wrote:> This parameter identifies an alternative service domain which has > superuser access to the xenstore database, which is currently required > to set up a new domain''s xenstore entries. > > Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > --- > tools/xenstore/xenstored_core.c | 5 +++++ > tools/xenstore/xenstored_core.h | 1 + > tools/xenstore/xenstored_domain.c | 2 +- > 3 files changed, 7 insertions(+), 1 deletions(-) > > diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c > index 65ceb2c..4437c8d 100644 > --- a/tools/xenstore/xenstored_core.c > +++ b/tools/xenstore/xenstored_core.c > @@ -1777,6 +1777,7 @@ static struct option options[] = { > { "event", 1, NULL, ''e'' }, > { "help", 0, NULL, ''H'' }, > { "no-fork", 0, NULL, ''N'' }, > + { "priv-domid", 1, NULL, ''p'' }, > { "output-pid", 0, NULL, ''P'' }, > { "entry-size", 1, NULL, ''S'' }, > { "trace-file", 1, NULL, ''T'' }, > @@ -1789,6 +1790,7 @@ static struct option options[] = { > > extern void dump_conn(struct connection *conn); > int dom0_event = 0; > +int priv_domid = 0; > > int main(int argc, char *argv[]) > { > @@ -1854,6 +1856,9 @@ int main(int argc, char *argv[]) > case ''e'': > dom0_event = strtol(optarg, NULL, 10); > break; > + case ''p'': > + priv_domid = strtol(optarg, NULL, 10); > + break; > } > } > if (optind != argc) > diff --git a/tools/xenstore/xenstored_core.h b/tools/xenstore/xenstored_core.h > index d3040ba..03e2e48 100644 > --- a/tools/xenstore/xenstored_core.h > +++ b/tools/xenstore/xenstored_core.h > @@ -169,6 +169,7 @@ void dtrace_io(const struct connection *conn, const struct buffered_data *data, > > extern int event_fd; > extern int dom0_event; > +extern int priv_domid; > > /* Map the kernel''s xenstore page. */ > void *xenbus_map(void); > diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c > index 5bf16e8..ba9a5ef 100644 > --- a/tools/xenstore/xenstored_domain.c > +++ b/tools/xenstore/xenstored_domain.c > @@ -241,7 +241,7 @@ bool domain_can_read(struct connection *conn) > > bool domain_is_unprivileged(struct connection *conn) > { > - return (conn && conn->domain && conn->domain->domid != 0); > + return (conn && conn->domain && conn->domain->domid != 0 && conn->domain->domid != priv_domid);Is it deliberate / desirable that both dom0 and domPRIV are privileged when this option is used? Or should it be just one or the other, which is equivalent to removing the domid!=0 check since priv_domid defaults to 0. Ian.> } > > bool domain_can_write(struct connection *conn)
On 01/11/12 18:21, Daniel De Graaf wrote:> This patch series allows xenstored to run in a stub domian started by > dom0. It is based on a patch series posted by Alex Zeffertt in 2009 - > http://old-list-archives.xen.org/archives/html/xen-devel/2009-03/msg01488.html >Daniel, Can you explain what is the rationale for moving the xenstored into a stubdom? After all, if an attacker is able to compromise the xenstored, there should be many ways now how to compromise other VMs in the system? And it shouldn''t matter whether the xenstored is in stubdom or whether in Dom0. E.g. the attacker might redirect the block fronts to us some false block backends, so that the VMs get compromised fs. One could probably think of other attacks as well...? joanna. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
At 11:33 +0100 on 12 Jan (1326367997), Joanna Rutkowska wrote:> Daniel, > > Can you explain what is the rationale for moving the xenstored into a > stubdom? After all, if an attacker is able to compromise the xenstored, > there should be many ways now how to compromise other VMs in the system? > And it shouldn''t matter whether the xenstored is in stubdom or whether > in Dom0. E.g. the attacker might redirect the block fronts to us some > false block backends, so that the VMs get compromised fs. One could > probably think of other attacks as well...?I think the point is to protect xenstore from dom0, not dom0 from xenstore. With stub-xenstore and driver domains, only the domain builder and PCIback need to have any privilege, and they can be moved out of dom0 too (e.g., http://dl.acm.org/citation.cfm?id=1346278 , http://tjd.phlegethon.org/words/sosp11-xoar.html) Tim.
On 12/01/2012 10:33, "Joanna Rutkowska" <joanna@invisiblethingslab.com> wrote:> On 01/11/12 18:21, Daniel De Graaf wrote: >> This patch series allows xenstored to run in a stub domian started by >> dom0. It is based on a patch series posted by Alex Zeffertt in 2009 - >>http://old-list-archives.xen.org/archives/html/xen-devel/2009-03/msg01488.htm>> l>> > > Daniel, > > Can you explain what is the rationale for moving the xenstored into a > stubdom? After all, if an attacker is able to compromise the xenstored, > there should be many ways now how to compromise other VMs in the system? > And it shouldn''t matter whether the xenstored is in stubdom or whether > in Dom0. E.g. the attacker might redirect the block fronts to us some > false block backends, so that the VMs get compromised fs. One could > probably think of other attacks as well...?As you point out it''s a critical component in itself, so I suppose this work is mainly about isolating it from the big attack surfaces in dom0. It''s of questionable value unless dom0 itself can be deprivileged, or the big attack surfaces themselves shuffled off into lesser-privileged domains. In a well locked down dom0 I would say that the biggest attack surfaces are via things like domain build, save/restore, and qemu, being intrinsic (ie unavoidable) components of a Xen system which consume complex inputs. We can already do qemu stubdoms, perhaps with some features missing still. Launching isolated, de-privileged (eg can only act on the one specified domain), domain-builder/saver/restorer stubdoms would be an interesting direction imo. It''s easy to be impressed with any disaggregation effort almost for its own sake, and lose sight of the importance of basic security analysis as a starting point. -- Keir> joanna. > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel
Joanna Rutkowska
2012-Jan-12 11:18 UTC
On Dom0 disaggregation (was: Re: [RFC PATCH 0/18] Xenstore stub domain)
On 01/12/12 11:48, Tim Deegan wrote:> At 11:33 +0100 on 12 Jan (1326367997), Joanna Rutkowska wrote: >> Daniel, >> >> Can you explain what is the rationale for moving the xenstored into a >> stubdom? After all, if an attacker is able to compromise the xenstored, >> there should be many ways now how to compromise other VMs in the system? >> And it shouldn''t matter whether the xenstored is in stubdom or whether >> in Dom0. E.g. the attacker might redirect the block fronts to us some >> false block backends, so that the VMs get compromised fs. One could >> probably think of other attacks as well...? > > I think the point is to protect xenstore from dom0, not dom0 from > xenstore. With stub-xenstore and driver domains, only the domain > builder and PCIback need to have any privilege, and they can be moved > out of dom0 too (e.g., http://dl.acm.org/citation.cfm?id=1346278 , > http://tjd.phlegethon.org/words/sosp11-xoar.html) >In order for this to make sense from security point of view, you would need to deprivilige Dom0. When considering this task, one should answer the following questions: 1) Who manages the chipset (MCH)? 2) Who manages the input (keyboard, mouse) 3) Who manages the output (GPU, specifically critical on client systems) From the security point of view there is no point of isolating the entities that manage the above between each other, because a compromise of any of those entities leads to full system compromise (again, #3 applies to client systems). Now, as you pointed out, we shall probably add another bullet to the list, which is: 4) the xenstored (although I''m not 100% if we couldn''t somehow deprivilige xenstored). So, we end up with a conclusion that there is no point separating those 4 functionalists between each other, and so it only make sense to host them in the same domain. How about we call this domain "Dom0", then? ;) (Sure, this "new Dom0" doesn''t include net, USB, and perhaps even SATA drivers, but we already have been doing this on Qubes, except for moving out SATA/blkbackend from Dom0). joanna. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 2012-01-12 at 10:48 +0000, Tim Deegan wrote:> At 11:33 +0100 on 12 Jan (1326367997), Joanna Rutkowska wrote: > > Daniel, > > > > Can you explain what is the rationale for moving the xenstored into a > > stubdom? After all, if an attacker is able to compromise the xenstored, > > there should be many ways now how to compromise other VMs in the system? > > And it shouldn''t matter whether the xenstored is in stubdom or whether > > in Dom0. E.g. the attacker might redirect the block fronts to us some > > false block backends, so that the VMs get compromised fs. One could > > probably think of other attacks as well...? > > I think the point is to protect xenstore from dom0, not dom0 from > xenstore. With stub-xenstore and driver domains, only the domain > builder and PCIback need to have any privilege, and they can be moved > out of dom0 too (e.g., http://dl.acm.org/citation.cfm?id=1346278 , > http://tjd.phlegethon.org/words/sosp11-xoar.html)Also by isolating components you gain the ability to restart them independently. Since xenstored is one of (the only?) dom0 component which cannot be trivially restarted so putting it in a separate domain means you can restart dom0. Ian.
2012/1/12 Ian Campbell <Ian.Campbell@citrix.com>:> On Thu, 2012-01-12 at 10:48 +0000, Tim Deegan wrote: >> At 11:33 +0100 on 12 Jan (1326367997), Joanna Rutkowska wrote: >> > Daniel, >> > >> > Can you explain what is the rationale for moving the xenstored into a >> > stubdom? After all, if an attacker is able to compromise the xenstored, >> > there should be many ways now how to compromise other VMs in the system? >> > And it shouldn't matter whether the xenstored is in stubdom or whether >> > in Dom0. E.g. the attacker might redirect the block fronts to us some >> > false block backends, so that the VMs get compromised fs. One could >> > probably think of other attacks as well...? >> >> I think the point is to protect xenstore from dom0, not dom0 from >> xenstore. Â With stub-xenstore and driver domains, only the domain >> builder and PCIback need to have any privilege, and they can be moved >> out of dom0 too (e.g., http://dl.acm.org/citation.cfm?id=1346278 , >> http://tjd.phlegethon.org/words/sosp11-xoar.html) > > Also by isolating components you gain the ability to restart them > independently. Since xenstored is one of (the only?) dom0 component > which cannot be trivially restarted so putting it in a separate domain > means you can restart dom0. > >Is that possible now (or in some feature) to restart xenstore domain and not loose watches? Now if i save all contents in xenstore and restart xenstored i'm loosing all watches and domU's not able to work correctly... -- Vasiliy Tolstov, Clodo.ru e-mail: v.tolstov@selfip.ru jabber: vase@selfip.ru _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 01/12/12 12:27, Ian Campbell wrote:> On Thu, 2012-01-12 at 10:48 +0000, Tim Deegan wrote: >> At 11:33 +0100 on 12 Jan (1326367997), Joanna Rutkowska wrote: >>> Daniel, >>> >>> Can you explain what is the rationale for moving the xenstored into a >>> stubdom? After all, if an attacker is able to compromise the xenstored, >>> there should be many ways now how to compromise other VMs in the system? >>> And it shouldn''t matter whether the xenstored is in stubdom or whether >>> in Dom0. E.g. the attacker might redirect the block fronts to us some >>> false block backends, so that the VMs get compromised fs. One could >>> probably think of other attacks as well...? >> >> I think the point is to protect xenstore from dom0, not dom0 from >> xenstore. With stub-xenstore and driver domains, only the domain >> builder and PCIback need to have any privilege, and they can be moved >> out of dom0 too (e.g., http://dl.acm.org/citation.cfm?id=1346278 , >> http://tjd.phlegethon.org/words/sosp11-xoar.html) > > Also by isolating components you gain the ability to restart them > independently. Since xenstored is one of (the only?) dom0 component > which cannot be trivially restarted so putting it in a separate domain > means you can restart dom0. >But why would anybody want to restart Dom0, in the first place? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, 2012-01-12 at 11:33 +0000, Vasiliy Tolstov wrote:> 2012/1/12 Ian Campbell <Ian.Campbell@citrix.com>: > > On Thu, 2012-01-12 at 10:48 +0000, Tim Deegan wrote: > >> At 11:33 +0100 on 12 Jan (1326367997), Joanna Rutkowska wrote: > >> > Daniel, > >> > > >> > Can you explain what is the rationale for moving the xenstored into a > >> > stubdom? After all, if an attacker is able to compromise the xenstored, > >> > there should be many ways now how to compromise other VMs in the system? > >> > And it shouldn''t matter whether the xenstored is in stubdom or whether > >> > in Dom0. E.g. the attacker might redirect the block fronts to us some > >> > false block backends, so that the VMs get compromised fs. One could > >> > probably think of other attacks as well...? > >> > >> I think the point is to protect xenstore from dom0, not dom0 from > >> xenstore. With stub-xenstore and driver domains, only the domain > >> builder and PCIback need to have any privilege, and they can be moved > >> out of dom0 too (e.g., http://dl.acm.org/citation.cfm?id=1346278 , > >> http://tjd.phlegethon.org/words/sosp11-xoar.html) > > > > Also by isolating components you gain the ability to restart them > > independently. Since xenstored is one of (the only?) dom0 component > > which cannot be trivially restarted so putting it in a separate domain > > means you can restart dom0. > > > > > > Is that possible now (or in some feature) to restart xenstore domain > and not loose watches? Now if i save all contents in xenstore and > restart xenstored i''m loosing all watches and domU''s not able to work > correctly...I''ve heard that it is possible to do this with the ocaml xenstored. I haven''t tried it myself. Ian.
On Thu, 2012-01-12 at 11:35 +0000, Joanna Rutkowska wrote:> On 01/12/12 12:27, Ian Campbell wrote: > > On Thu, 2012-01-12 at 10:48 +0000, Tim Deegan wrote: > >> At 11:33 +0100 on 12 Jan (1326367997), Joanna Rutkowska wrote: > >>> Daniel, > >>> > >>> Can you explain what is the rationale for moving the xenstored into a > >>> stubdom? After all, if an attacker is able to compromise the xenstored, > >>> there should be many ways now how to compromise other VMs in the system? > >>> And it shouldn''t matter whether the xenstored is in stubdom or whether > >>> in Dom0. E.g. the attacker might redirect the block fronts to us some > >>> false block backends, so that the VMs get compromised fs. One could > >>> probably think of other attacks as well...? > >> > >> I think the point is to protect xenstore from dom0, not dom0 from > >> xenstore. With stub-xenstore and driver domains, only the domain > >> builder and PCIback need to have any privilege, and they can be moved > >> out of dom0 too (e.g., http://dl.acm.org/citation.cfm?id=1346278 , > >> http://tjd.phlegethon.org/words/sosp11-xoar.html) > > > > Also by isolating components you gain the ability to restart them > > independently. Since xenstored is one of (the only?) dom0 component > > which cannot be trivially restarted so putting it in a separate domain > > means you can restart dom0. > > > > But why would anybody want to restart Dom0, in the first place?To patch the kernel? Ian.
Tim Deegan
2012-Jan-12 12:13 UTC
Re: On Dom0 disaggregation (was: Re: [RFC PATCH 0/18] Xenstore stub domain)
At 12:18 +0100 on 12 Jan (1326370728), Joanna Rutkowska wrote:> On 01/12/12 11:48, Tim Deegan wrote: > > I think the point is to protect xenstore from dom0, not dom0 from > > xenstore. With stub-xenstore and driver domains, only the domain > > builder and PCIback need to have any privilege, and they can be moved > > out of dom0 too (e.g., http://dl.acm.org/citation.cfm?id=1346278 , > > http://tjd.phlegethon.org/words/sosp11-xoar.html) > > In order for this to make sense from security point of view, you would > need to deprivilige Dom0.Yep. That''s what Xoar did (or, if you prefer, it moved all dom0''s components out and then got rid of it).> When considering this task, one should answer > the following questions: > > 1) Who manages the chipset (MCH)?A driver domain that contains PCIback. It doesn''t need general privlege (e.g. for scheduler or memory operations).> 2) Who manages the input (keyboard, mouse) > 3) Who manages the output (GPU, specifically critical on client systems) > > From the security point of view there is no point of isolating the > entities that manage the above between each other, because a compromise > of any of those entities leads to full system compromise (again, #3 > applies to client systems).#2 is really a client-only thing as well. Serial console input for servers is owned directly by the hypervisor.> Now, as you pointed out, we shall probably add another bullet to the > list, which is: > > 4) the xenstored > > (although I''m not 100% if we couldn''t somehow deprivilige xenstored).Yes we could (and Xoar did). IIRC the only reason it needs privilege is to map the rings and that can be sorted out by having the builder pre-load a grant entry.> So, we end up with a conclusion that there is no point separating those > 4 functionalists between each other, and so it only make sense to host > them in the same domain. How about we call this domain "Dom0", then? ;)So you''re saying that the PCIback domain (which owns the chipsets) might as well host Xenstore since either of them could hose the system. Maybe so. In Xoar we had two reasons not to do that: - in some configurations you could shut that domain down after boot (though tbh doing that leads to poor error handling!); and - we were doing other hardening of Xenstore that involved breaking it up into more than one VM. Cheers, Tim.
On 01/12/12 13:13, Tim Deegan wrote:> At 12:18 +0100 on 12 Jan (1326370728), Joanna Rutkowska wrote: >> On 01/12/12 11:48, Tim Deegan wrote: >>> I think the point is to protect xenstore from dom0, not dom0 from >>> xenstore. With stub-xenstore and driver domains, only the domain >>> builder and PCIback need to have any privilege, and they can be moved >>> out of dom0 too (e.g., http://dl.acm.org/citation.cfm?id=1346278 , >>> http://tjd.phlegethon.org/words/sosp11-xoar.html) >> >> In order for this to make sense from security point of view, you would >> need to deprivilige Dom0. > > Yep. That''s what Xoar did (or, if you prefer, it moved all dom0''s > components out and then got rid of it). > >> When considering this task, one should answer >> the following questions: >> >> 1) Who manages the chipset (MCH)? > > A driver domain that contains PCIback. It doesn''t need general > privlege (e.g. for scheduler or memory operations). > >> 2) Who manages the input (keyboard, mouse) >> 3) Who manages the output (GPU, specifically critical on client systems) >> >> From the security point of view there is no point of isolating the >> entities that manage the above between each other, because a compromise >> of any of those entities leads to full system compromise (again, #3 >> applies to client systems). > > #2 is really a client-only thing as well. Serial console input for > servers is owned directly by the hypervisor. > >> Now, as you pointed out, we shall probably add another bullet to the >> list, which is: >> >> 4) the xenstored >> >> (although I''m not 100% if we couldn''t somehow deprivilige xenstored). > > Yes we could (and Xoar did). IIRC the only reason it needs privilege is > to map the rings and that can be sorted out by having the builder > pre-load a grant entry. > >> So, we end up with a conclusion that there is no point separating those >> 4 functionalists between each other, and so it only make sense to host >> them in the same domain. How about we call this domain "Dom0", then? ;) > > So you''re saying that the PCIback domain (which owns the chipsets) might > as well host Xenstore since either of them could hose the system. > Maybe so. In Xoar we had two reasons not to do that: > - in some configurations you could shut that domain down after boot > (though tbh doing that leads to poor error handling!); and > - we were doing other hardening of Xenstore that involved breaking it > up into more than one VM. >But the paper says the PCIBack is destroyed after init -- who manages the chipset (MCH, ICH, platform power management), then? Also, if I correctly interpret your architecture, then BlkBack VM must be trusted, as it has access to the disk, and thus can serve compromised fs/kernel/initramfs images to VMs, and also modify boot partition to load compromised Xen on the next boot (unless you use something like Intel TXT for Xen load). So, what''s the reason of separating it from (the trusted) Xenstored VM? Unless the kernel/initramfs images (at least for PV domains) come from some trusted source, not from blkbackend? Then they could contain code to verify integrity of the fs served from blkbackend. Have you considered this in your architecture? Also, if this was a client system, where would you put user input/output handling? joanna. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
At 14:30 +0100 on 12 Jan (1326378636), Joanna Rutkowska wrote:> > So you''re saying that the PCIback domain (which owns the chipsets) might > > as well host Xenstore since either of them could hose the system. > > Maybe so. In Xoar we had two reasons not to do that: > > - in some configurations you could shut that domain down after boot > > (though tbh doing that leads to poor error handling!); and > > - we were doing other hardening of Xenstore that involved breaking it > > up into more than one VM. > > > > But the paper says the PCIBack is destroyed after init -- who manages > the chipset (MCH, ICH, platform power management), then?Nobody. Like I said, it''s not ideal. :) You don''t have to destroy PCIBack -- you can keep it around to do those things. And if you do, it has a _much_ narrower interface to the rest of the world than dom0 does. It has no network access, and no direct interface to customer (i.e. non-driver) VMs. A malicious customer VM must break at least one other system VM before it can attack it.> Also, if I correctly interpret your architecture, then BlkBack VM must > be trusted, as it has access to the disk, and thus can serve compromised > fs/kernel/initramfs images to VMs, and also modify boot partition to > load compromised Xen on the next boot (unless you use something like > Intel TXT for Xen load). So, what''s the reason of separating it from > (the trusted) Xenstored VM?Well, you can have multiple block driver domains, if you have multiple HBAs, and arrange that two customers'' data don''t share a blkback. And there are the usual reasons for driver domains. For example you''re not tied to a particular OS for all your drivers. And device-driver crashes don''t take out xenstore. Trusted boot is orthogonal to the Xoar work. We didn''t go into it in the paper but it''s fairly obvious how it would fit in. And once you''ve got a TPM you could also use vTPMs to protect VMs'' disk images.> Also, if this was a client system, where would you put user input/output > handling?I''d probably make a console driver VM that owned the GPU and the USB controller. It might make sense for the toolstack (or at least the parts that compose the display and demux the inputs) to live in that VM too. Or maybe it would be better to isolate those drivers; I haven''t given it a lot of thought. Cheers, Tim.
On 2012-01-12, at 5:31 AM, Joanna Rutkowska <joanna@invisiblethingslab.com> wrote:> On 01/12/12 13:13, Tim Deegan wrote: >> At 12:18 +0100 on 12 Jan (1326370728), Joanna Rutkowska wrote: >>> On 01/12/12 11:48, Tim Deegan wrote: >>>> I think the point is to protect xenstore from dom0, not dom0 from >>>> xenstore. With stub-xenstore and driver domains, only the domain >>>> builder and PCIback need to have any privilege, and they can be moved >>>> out of dom0 too (e.g., http://dl.acm.org/citation.cfm?id=1346278 , >>>> http://tjd.phlegethon.org/words/sosp11-xoar.html) >>> >>> In order for this to make sense from security point of view, you would >>> need to deprivilige Dom0. >> >> Yep. That''s what Xoar did (or, if you prefer, it moved all dom0''s >> components out and then got rid of it). >> >>> When considering this task, one should answer >>> the following questions: >>> >>> 1) Who manages the chipset (MCH)? >> >> A driver domain that contains PCIback. It doesn''t need general >> privlege (e.g. for scheduler or memory operations). >> >>> 2) Who manages the input (keyboard, mouse) >>> 3) Who manages the output (GPU, specifically critical on client systems) >>> >>> From the security point of view there is no point of isolating the >>> entities that manage the above between each other, because a compromise >>> of any of those entities leads to full system compromise (again, #3 >>> applies to client systems). >> >> #2 is really a client-only thing as well. Serial console input for >> servers is owned directly by the hypervisor. >> >>> Now, as you pointed out, we shall probably add another bullet to the >>> list, which is: >>> >>> 4) the xenstored >>> >>> (although I''m not 100% if we couldn''t somehow deprivilige xenstored). >> >> Yes we could (and Xoar did). IIRC the only reason it needs privilege is >> to map the rings and that can be sorted out by having the builder >> pre-load a grant entry. >> >>> So, we end up with a conclusion that there is no point separating those >>> 4 functionalists between each other, and so it only make sense to host >>> them in the same domain. How about we call this domain "Dom0", then? ;) >> >> So you''re saying that the PCIback domain (which owns the chipsets) might >> as well host Xenstore since either of them could hose the system. >> Maybe so. In Xoar we had two reasons not to do that: >> - in some configurations you could shut that domain down after boot >> (though tbh doing that leads to poor error handling!); and >> - we were doing other hardening of Xenstore that involved breaking it >> up into more than one VM. >> > > But the paper says the PCIBack is destroyed after init -- who manages > the chipset (MCH, ICH, platform power management), then? > > Also, if I correctly interpret your architecture, then BlkBack VM must > be trusted, as it has access to the disk, and thus can serve compromised > fs/kernel/initramfs images to VMs, and also modify boot partition to > load compromised Xen on the next boot (unless you use something like > Intel TXT for Xen load). So, what''s the reason of separating it from > (the trusted) Xenstored VM? > > Unless the kernel/initramfs images (at least for PV domains) come from > some trusted source, not from blkbackend? Then they could contain code > to verify integrity of the fs served from blkbackend. Have you > considered this in your architecture?Indeed. The domain builder only builds known good kernels, currently which are served from its ramdisk. There is an alternative architecture where it''s served by a special storage domain via shared memory too. You''re right about the block backend needing to be trusted to some extent; it''s only a matter of reducing who needs to trust it. On a system with multiple disk controllers, the one serving Xen could be split from the one serving the VMs. With only a single controller, you''d probably use TXT.> > Also, if this was a client system, where would you put user input/output > handling?I''m afraid I''m not sure I know exactly what you mean here... ~M> > joanna. > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel
Daniel De Graaf
2012-Jan-12 15:11 UTC
Re: [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants
On 01/12/2012 04:59 AM, Ian Campbell wrote:> On Wed, 2012-01-11 at 17:21 +0000, Daniel De Graaf wrote: >> From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> >> >> This patch claims one reserved grant entry for the console and another >> for the xenstore. It modifies the builder to fill in the grant table >> entries for the console and the xenstore. >> >> This has not been tested with any kind of migration. >> > [...] >> diff --git a/tools/libxc/xc_domain_restore.c b/tools/libxc/xc_domain_restore.c >> index 3fda6f8..23619da 100644 >> --- a/tools/libxc/xc_domain_restore.c >> +++ b/tools/libxc/xc_domain_restore.c >> @@ -2018,6 +2018,15 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, >> memcpy(ctx->live_p2m, ctx->p2m, dinfo->p2m_size * sizeof(xen_pfn_t)); >> munmap(ctx->live_p2m, P2M_FL_ENTRIES * PAGE_SIZE); >> >> +/* TODO don''t hardcode zero here */ >> + rc = xc_dom_gnttab_seed(xch, dom, >> + *console_mfn, *store_mfn, 0, 0); > > Presumably this TODO is the source of the comment in the changelog WRT > migration. > > Does it Just Work or is there a legitimate TODO item here? > > Ian. > >This causes migration to only work if xenstored/xenconsoled are both in dom0, as the domain ID for both of them are hardcoded to zero. Determining the correct values for these domain IDs is more difficult than the domain build case, because they may not be the same as when the domain was built (especially if we are migrating). The previous patch series used a domid file named similarly to a pid file to identify the location of xenstored and xenconsoled; this method would allow the TODO to be resolved. I think a better solution is to refer to the xenstore/xenconsole domains by name instead of domid, and set the names in a configuration file (/etc/xen/xl.conf?). This would also replace the xenstore_dom/console_dom parameters in patch #5. -- Daniel De Graaf National Security Agency
Daniel De Graaf
2012-Jan-12 15:28 UTC
Re: [PATCH] xenbus: Add support for xenbus backend in stub domain
On 01/12/2012 03:59 AM, Jan Beulich wrote:>>>> On 11.01.12 at 18:22, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >> This adds two ioctls to the /dev/xen/xenbus_backend device allowing the >> xenbus backend to be started after the kernel has booted. This is >> intended to allow dom0 to start another domain to run xenstore. >> >> IOCTL_XENBUS_BACKEND_REMOTE_SETUP requires that the xenstore domain be >> started; the domain ID is passed as the ioctl argument. This sets up a >> listening event channel and grant reference to xenbus, but does not >> start using this interface. It returns the local event channel port. > > Any chance to get at least the setup part matched with the > legacy Linux implementation (defining IOCTL_XENBUS_ALLOC)? > > JanFrom what I can tell, the legacy ioctl cannot restore watches because the return value must be used to set up xenstore communication before they can be used. Or were you just asking if IOCTL_XENBUS_BACKEND_REMOTE_SETUP''s parameter can be changed to xenbus_alloc_t? That structure could be populated in the same way as the legacy implementation, but it wouldn''t give any more information than the current ioctl does.> >> IOCTL_XENBUS_BACKEND_REMOTE_COMMIT causes the kernel to begin using the >> event channel set up in the previous ioctl, reregistering all watches >> and deallocating the previous xenstored event channel. >> >> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> >> ---[...]
Daniel De Graaf
2012-Jan-12 15:37 UTC
Re: [PATCH 18/18] xenstored: add --priv-domid parameter
On 01/12/2012 05:20 AM, Ian Campbell wrote:> On Wed, 2012-01-11 at 17:21 +0000, Daniel De Graaf wrote: >> This parameter identifies an alternative service domain which has >> superuser access to the xenstore database, which is currently required >> to set up a new domain''s xenstore entries. >> >> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> >> --- >> tools/xenstore/xenstored_core.c | 5 +++++ >> tools/xenstore/xenstored_core.h | 1 + >> tools/xenstore/xenstored_domain.c | 2 +- >> 3 files changed, 7 insertions(+), 1 deletions(-) >> >> diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c >> index 65ceb2c..4437c8d 100644 >> --- a/tools/xenstore/xenstored_core.c >> +++ b/tools/xenstore/xenstored_core.c >> @@ -1777,6 +1777,7 @@ static struct option options[] = { >> { "event", 1, NULL, ''e'' }, >> { "help", 0, NULL, ''H'' }, >> { "no-fork", 0, NULL, ''N'' }, >> + { "priv-domid", 1, NULL, ''p'' }, >> { "output-pid", 0, NULL, ''P'' }, >> { "entry-size", 1, NULL, ''S'' }, >> { "trace-file", 1, NULL, ''T'' }, >> @@ -1789,6 +1790,7 @@ static struct option options[] = { >> >> extern void dump_conn(struct connection *conn); >> int dom0_event = 0; >> +int priv_domid = 0; >> >> int main(int argc, char *argv[]) >> { >> @@ -1854,6 +1856,9 @@ int main(int argc, char *argv[]) >> case ''e'': >> dom0_event = strtol(optarg, NULL, 10); >> break; >> + case ''p'': >> + priv_domid = strtol(optarg, NULL, 10); >> + break; >> } >> } >> if (optind != argc) >> diff --git a/tools/xenstore/xenstored_core.h b/tools/xenstore/xenstored_core.h >> index d3040ba..03e2e48 100644 >> --- a/tools/xenstore/xenstored_core.h >> +++ b/tools/xenstore/xenstored_core.h >> @@ -169,6 +169,7 @@ void dtrace_io(const struct connection *conn, const struct buffered_data *data, >> >> extern int event_fd; >> extern int dom0_event; >> +extern int priv_domid; >> >> /* Map the kernel''s xenstore page. */ >> void *xenbus_map(void); >> diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c >> index 5bf16e8..ba9a5ef 100644 >> --- a/tools/xenstore/xenstored_domain.c >> +++ b/tools/xenstore/xenstored_domain.c >> @@ -241,7 +241,7 @@ bool domain_can_read(struct connection *conn) >> >> bool domain_is_unprivileged(struct connection *conn) >> { >> - return (conn && conn->domain && conn->domain->domid != 0); >> + return (conn && conn->domain && conn->domain->domid != 0 && conn->domain->domid != priv_domid); > > Is it deliberate / desirable that both dom0 and domPRIV are privileged > when this option is used? Or should it be just one or the other, which > is equivalent to removing the domid!=0 check since priv_domid defaults > to 0. > > Ian. >Yes. In the cases where I used this, both dom0 and domPRIV are privileged. Since dom0 needs to introduce domPRIV, and introduction is privileged, you can''t just have domPRIV be the only privileged domain. The ideal solution is to have more fine-grained permissions than superuser-or-domU in xenstored, but that is a different topic from moving xenstored into a stubdom. -- Daniel De Graaf National Security Agency
Jan Beulich
2012-Jan-12 15:40 UTC
Re: [PATCH] xenbus: Add support for xenbus backend in stub domain
>>> On 12.01.12 at 16:28, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > On 01/12/2012 03:59 AM, Jan Beulich wrote: >>>>> On 11.01.12 at 18:22, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >>> This adds two ioctls to the /dev/xen/xenbus_backend device allowing the >>> xenbus backend to be started after the kernel has booted. This is >>> intended to allow dom0 to start another domain to run xenstore. >>> >>> IOCTL_XENBUS_BACKEND_REMOTE_SETUP requires that the xenstore domain be >>> started; the domain ID is passed as the ioctl argument. This sets up a >>> listening event channel and grant reference to xenbus, but does not >>> start using this interface. It returns the local event channel port. >> >> Any chance to get at least the setup part matched with the >> legacy Linux implementation (defining IOCTL_XENBUS_ALLOC)? >> >> Jan > > From what I can tell, the legacy ioctl cannot restore watches because the > return value must be used to set up xenstore communication before they can > be used.Then I must have missed something. But this was the only kernel side patch, wasn''t it? Is the intended behavior then to have a Dom0-based xenstored starting first, then do a brain transfer to that running in a stub domain? That certainly wasn''t the behavior of what the legacy Linux implementation was intended for...> Or were you just asking if IOCTL_XENBUS_BACKEND_REMOTE_SETUP''s parameter > can be changed to xenbus_alloc_t? That structure could be populated in the > same way as the legacy implementation, but it wouldn''t give any more > information than the current ioctl does.No, if it functionally differs, then we really should just care that the ioctl numbers don''t collide. Jan
Daniel De Graaf
2012-Jan-12 15:58 UTC
Re: [PATCH] xenbus: Add support for xenbus backend in stub domain
On 01/12/2012 10:40 AM, Jan Beulich wrote:>>>> On 12.01.12 at 16:28, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >> On 01/12/2012 03:59 AM, Jan Beulich wrote: >>>>>> On 11.01.12 at 18:22, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >>>> This adds two ioctls to the /dev/xen/xenbus_backend device allowing the >>>> xenbus backend to be started after the kernel has booted. This is >>>> intended to allow dom0 to start another domain to run xenstore. >>>> >>>> IOCTL_XENBUS_BACKEND_REMOTE_SETUP requires that the xenstore domain be >>>> started; the domain ID is passed as the ioctl argument. This sets up a >>>> listening event channel and grant reference to xenbus, but does not >>>> start using this interface. It returns the local event channel port. >>> >>> Any chance to get at least the setup part matched with the >>> legacy Linux implementation (defining IOCTL_XENBUS_ALLOC)? >>> >>> Jan >> >> From what I can tell, the legacy ioctl cannot restore watches because the >> return value must be used to set up xenstore communication before they can >> be used. > > Then I must have missed something. But this was the only kernel side > patch, wasn''t it? > > Is the intended behavior then to have a Dom0-based xenstored > starting first, then do a brain transfer to that running in a stub > domain? That certainly wasn''t the behavior of what the legacy > Linux implementation was intended for... > >> Or were you just asking if IOCTL_XENBUS_BACKEND_REMOTE_SETUP''s parameter >> can be changed to xenbus_alloc_t? That structure could be populated in the >> same way as the legacy implementation, but it wouldn''t give any more >> information than the current ioctl does. > > No, if it functionally differs, then we really should just care that > the ioctl numbers don''t collide. > > Jan >I think this depends on how the xenstore domain is constructed: if it is started instead of the dom0-based xenstored (not the current method, but converting init-xenstored to create the domain itself should make this possible) then the setup/commit split of the ioctl is not needed. In that case the older ioctl name/interface can be preserved (although the grant field will always be populated with 1). -- Daniel De Graaf National Security Agency
On 01/12/2012 05:33 AM, Joanna Rutkowska wrote:> On 01/11/12 18:21, Daniel De Graaf wrote: >> This patch series allows xenstored to run in a stub domian started by >> dom0. It is based on a patch series posted by Alex Zeffertt in 2009 - >> http://old-list-archives.xen.org/archives/html/xen-devel/2009-03/msg01488.html >> > > Daniel, > > Can you explain what is the rationale for moving the xenstored into a > stubdom? After all, if an attacker is able to compromise the xenstored, > there should be many ways now how to compromise other VMs in the system? > And it shouldn''t matter whether the xenstored is in stubdom or whether > in Dom0. E.g. the attacker might redirect the block fronts to us some > false block backends, so that the VMs get compromised fs. One could > probably think of other attacks as well...? > > joanna. >Splitting xenstored into its own domain (rather than keeping it in dom0) means that it does not need to be privileged, so a compromise of xenstore does not automatically give you full access to all other domains on the system. While it is possible to attack domains by sending them bad commands from xenstore (device unplug/domain changes), it is also possible for a guest VM to detect this and it becomes a denial-of-service instead of a way to compromise the system. This is most easily done if guests use their own full-disk encryption and run integrity checks on the unencrypted parts (kernel/initrd; using a vTPM to unlock the guest-based FDE would work). This split also prevents xenstored from being attacked from dom0, but this is currently not as important security-wise since dom0 has superuser access to the xenstore database. However, it does allow for future changes to xenstore''s security model that do not include a fully-privileged domain. -- Daniel De Graaf National Security Agency
Ian Campbell
2012-Jan-12 16:12 UTC
Re: [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants
On Thu, 2012-01-12 at 15:11 +0000, Daniel De Graaf wrote:> On 01/12/2012 04:59 AM, Ian Campbell wrote: > > On Wed, 2012-01-11 at 17:21 +0000, Daniel De Graaf wrote: > >> From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > >> > >> This patch claims one reserved grant entry for the console and another > >> for the xenstore. It modifies the builder to fill in the grant table > >> entries for the console and the xenstore. > >> > >> This has not been tested with any kind of migration. > >> > > [...] > >> diff --git a/tools/libxc/xc_domain_restore.c b/tools/libxc/xc_domain_restore.c > >> index 3fda6f8..23619da 100644 > >> --- a/tools/libxc/xc_domain_restore.c > >> +++ b/tools/libxc/xc_domain_restore.c > >> @@ -2018,6 +2018,15 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, > >> memcpy(ctx->live_p2m, ctx->p2m, dinfo->p2m_size * sizeof(xen_pfn_t)); > >> munmap(ctx->live_p2m, P2M_FL_ENTRIES * PAGE_SIZE); > >> > >> +/* TODO don''t hardcode zero here */ > >> + rc = xc_dom_gnttab_seed(xch, dom, > >> + *console_mfn, *store_mfn, 0, 0); > > > > Presumably this TODO is the source of the comment in the changelog WRT > > migration. > > > > Does it Just Work or is there a legitimate TODO item here? > > > > Ian. > > > > > > This causes migration to only work if xenstored/xenconsoled are both in > dom0, as the domain ID for both of them are hardcoded to zero. Determining > the correct values for these domain IDs is more difficult than the domain > build case, because they may not be the same as when the domain was built > (especially if we are migrating). > > The previous patch series used a domid file named similarly to a pid file > to identify the location of xenstored and xenconsoled; this method would > allow the TODO to be resolved. I think a better solution is to refer to the > xenstore/xenconsole domains by name instead of domid, and set the names in > a configuration file (/etc/xen/xl.conf?). This would also replace the > xenstore_dom/console_dom parameters in patch #5.That would work. You could also stash the necessary parameters in xenstore from the tool which starts the stub-xenstored such that the toolstack can look them up as needed. That avoids having to have an xl.conf variable. Ian.
Ian Jackson
2012-Jan-12 17:21 UTC
Re: [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants
Daniel De Graaf writes ("Re: [Xen-devel] [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants"):> The previous patch series used a domid file named similarly to a pid file > to identify the location of xenstored and xenconsoled; this method would > allow the TODO to be resolved. I think a better solution is to refer to the > xenstore/xenconsole domains by name instead of domid, and set the names in > a configuration file (/etc/xen/xl.conf?). This would also replace the > xenstore_dom/console_dom parameters in patch #5.Clearly this isn''t a configuration parameter; it''s a runtime value. You don''t store pids in config files. If this can''t go in xenstore itself then a file in /var/run/xen would do just fine. Ian.
Daniel De Graaf
2012-Jan-12 17:32 UTC
Re: [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants
On 01/12/2012 12:21 PM, Ian Jackson wrote:> Daniel De Graaf writes ("Re: [Xen-devel] [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants"): >> The previous patch series used a domid file named similarly to a pid file >> to identify the location of xenstored and xenconsoled; this method would >> allow the TODO to be resolved. I think a better solution is to refer to the >> xenstore/xenconsole domains by name instead of domid, and set the names in >> a configuration file (/etc/xen/xl.conf?). This would also replace the >> xenstore_dom/console_dom parameters in patch #5. > > Clearly this isn''t a configuration parameter; it''s a runtime value. > You don''t store pids in config files.I was suggesting storing the domain name in the config file, which wouldn''t be a runtime value. However, I think it should be possible to place this in xenstore, and that will probably be better as that''s where you would expect things like this to be.> If this can''t go in xenstore itself then a file in /var/run/xen would > do just fine. > > Ian. >
Ian Jackson
2012-Jan-12 17:35 UTC
Re: [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants
Daniel De Graaf writes ("Re: [Xen-devel] [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants"):> I was suggesting storing the domain name in the config file, which wouldn''t > be a runtime value. However, I think it should be possible to place this in > xenstore, and that will probably be better as that''s where you would expect > things like this to be.The registry of domain names is in xenstore, so that''s not going to work :-). Ian.
Ian Campbell
2012-Jan-12 17:38 UTC
Re: [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants
On Thu, 2012-01-12 at 17:35 +0000, Ian Jackson wrote:> Daniel De Graaf writes ("Re: [Xen-devel] [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants"): > > I was suggesting storing the domain name in the config file, which wouldn''t > > be a runtime value. However, I think it should be possible to place this in > > xenstore, and that will probably be better as that''s where you would expect > > things like this to be. > > The registry of domain names is in xenstore, so that''s not going to > work :-).This info is used for starting subsequent domains not the xenstore domain itself so I think it will. Whichever process starts the xenstore stubdom watis for it to initialise and then writes the necessary parameters to xenstore. Future toolstack invocations can then read it back. The toolstack doesn''t need to know where the xenstore is running since it just uses /proc/xen/xenbus. Ian.
Daniel De Graaf
2012-Jan-12 17:47 UTC
Re: [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants
On 01/12/2012 12:35 PM, Ian Jackson wrote:> Daniel De Graaf writes ("Re: [Xen-devel] [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants"): >> I was suggesting storing the domain name in the config file, which wouldn''t >> be a runtime value. However, I think it should be possible to place this in >> xenstore, and that will probably be better as that''s where you would expect >> things like this to be. > > The registry of domain names is in xenstore, so that''s not going to > work :-). > > Ian. >Actually, it''ll work just fine: dom0 has a connection to xenstore, it just can''t (from userspace) tell what domid it''s talking to. It''s really the same thing as just storing the xenstored domid in xenstore; /tool/xenstored/domid is where I plan to put it. -- Daniel De Graaf National Security Agency
Daniel De Graaf
2012-Jan-12 17:56 UTC
Re: [PATCH 07/18] mini-os: avoid crash if no console is provided
On 01/12/2012 05:03 AM, Ian Campbell wrote:> On Wed, 2012-01-11 at 17:21 +0000, Daniel De Graaf wrote: > > When/why does this happen?Initially, when I use a custom domain builder to create the xenstored domain without a xenconsoled running yet (as xenconsoled needs xenstore).> I guess it is because when starting the xenstore domain you cannot use > xenstore to communicate with xenconsoled (and/or it isn''t even running > yet). > > I wonder if there is a way we can do lazy-setup of the console for just > the xenstore domain?Xenstored will produce output on the hypervisor console if Xen is compiled with debugging. If xenstored produces a lot of output, it can block on waiting for xenconsoled, which might be blocking on xenstored - so I don''t think we want to hook it up to the console in the normal case, or at least not the same xenconsoled that talks to domUs.> Alternatively I seem to recall a little tool which Diego wrote to pull > the console ring out of a domain directly as a debuging aid but that > relies on us setting up a console ring and evtchn even if xenconsoled > cannot be involved which makes this patch unnecessary. > > Ian.This runs into the same blocking problem if xenstored produces too much output.
On 01/12/2012 04:57 AM, Ian Campbell wrote:> On Wed, 2012-01-11 at 17:21 +0000, Daniel De Graaf wrote: >> This patch series allows xenstored to run in a stub domian started by >> dom0. It is based on a patch series posted by Alex Zeffertt in 2009 - >> http://old-list-archives.xen.org/archives/html/xen-devel/2009-03/msg01488.html > > BTW did you come across any patches for stub-oxenstored while you were > doing this? I thought there had been such at some point in the past. > > Ian. >I haven''t searched for similar patches to oxenstored. If they exist, they would probably need similar changes to use grants instead of map_foreign_range and support for the bootstrapping. Since I am not fluent in ocaml, I will leave this for other people to implement :)
Changes from v1: - set_virq_handler implemented in libxc - added custom domain builder for xenstored - xenstore/console domain IDs now pulled from xenstore - migration support when using split xenstored (untested, should work) - slightly less intrusive NO_SOCKETS xenstored patch (still has many ifdefs to avoid pulling in socket headers or symbols) - virq handlers removed from dying domain when clearing event channels - dummy XSM module restricts getdomaininfo similar to no-XSM case - formatting/type fixups - partial ioctl compatibility with legacy IOCTL_XENBUS_ALLOC (the new device path uses ''B'' not ''X'' as the base so the ioctl number does not match; otherwise, should be compatible). To start xenstored, run: tools/xenstore/init-xenstore-domain stubdom/mini-os-x86_64-xenstore/mini-os 20 system_u:system_r:domU_t This will populate the xenstore domid key /tool/xenstore/domid ---- [PATCH 01/18] xen: reinstate previously unused hypercall [PATCH 02/18] xen: allow global VIRQ handlers to be delegated to [PATCH 03/18] xen: use XSM instead of IS_PRIV for getdomaininfo [PATCH 04/18] xen: Preserve reserved grant entries when switching [PATCH 05/18] tools/libxl: pull xenstore/console domids from [PATCH 06/18] lib{xc,xl}: Seed grant tables with xenstore and [PATCH 07/18] mini-os: avoid crash if no console is provided [PATCH 08/18] mini-os: avoid crash if no xenstore is provided [PATCH 09/18] mini-os: remove per-fd evtchn limit [PATCH 10/18] xenstored: use grant references instead of [PATCH 11/18] xenstored: add NO_SOCKETS compilation option [PATCH 12/18] xenstored support for in-memory rather than FS based [PATCH 13/18] xenstored: support running in minios stubdom [PATCH 14/18] xenstored: always use xc_gnttab_munmap in stubdom [PATCH 15/18] xenstored: add --event parameter for bootstrapping Removed old #16; shared page is no longer used to pass event. [PATCH 16/18] xenstored: use domain_is_unprivileged instead of [PATCH 17/18] xenstored: add --priv-domid parameter [PATCH 18/18] xenstored: Add stub domain builder
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 01/18] xen: reinstate previously unused XENMEM_remove_from_physmap hypercall
From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> This patch reinstates the XENMEM_remove_from_physmap hypercall which was removed in 19041:ee62aaafff46 because it was not used. However, is now needed in order to support xenstored stub domains. The xenstored stub domain is not priviliged like dom0 and so cannot unilaterally map the xenbus page of other guests into it''s address space. Therefore, before creating a domU the domain builder needs to seed its grant table with a grant ref allowing the xenstored stub domain to access the new domU''s xenbus page. At present domU''s do not start with their grant table mapped. Instead it gets mapped when the guest requests a grant table from the hypervisor. In order to seed the grant table, the domain builder first needs to map it into dom0 address space. But the hypercall to do this requires a gpfn (guest pfn), which is an mfn for PV guest, but a pfn for HVM guests. Therfore, in order to seed the grant table of an HVM guest, dom0 needs to *temporarily* map it into the guest''s "physical" address space. Hence the need to reinstate the XENMEM_remove_from_physmap hypercall. Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- xen/arch/ia64/xen/mm.c | 34 ++++++++++++++++++++++++++++++++++ xen/arch/x86/mm.c | 35 +++++++++++++++++++++++++++++++++++ xen/arch/x86/x86_64/compat/mm.c | 14 ++++++++++++++ xen/include/public/memory.h | 16 ++++++++++++++++ xen/include/xlat.lst | 1 + xen/include/xsm/xsm.h | 6 ++++++ xen/xsm/dummy.c | 6 ++++++ xen/xsm/flask/hooks.c | 6 ++++++ 8 files changed, 118 insertions(+), 0 deletions(-) diff --git a/xen/arch/ia64/xen/mm.c b/xen/arch/ia64/xen/mm.c index 84f6a61..a40f96a 100644 --- a/xen/arch/ia64/xen/mm.c +++ b/xen/arch/ia64/xen/mm.c @@ -3448,6 +3448,40 @@ arch_memory_op(int op, XEN_GUEST_HANDLE(void) arg) break; } + case XENMEM_remove_from_physmap: + { + struct xen_remove_from_physmap xrfp; + unsigned long mfn; + struct domain *d; + + if ( copy_from_guest(&xrfp, arg, 1) ) + return -EFAULT; + + rc = rcu_lock_target_domain_by_id(xrfp.domid, &d); + if ( rc != 0 ) + return rc; + + if ( xsm_remove_from_physmap(current->domain, d) ) + { + rcu_unlock_domain(d); + return -EPERM; + } + + domain_lock(d); + + mfn = gmfn_to_mfn(d, xrfp.gpfn); + + if ( mfn_valid(mfn) ) + guest_physmap_remove_page(d, xrfp.gpfn, mfn, 0); + + domain_unlock(d); + + rcu_unlock_domain(d); + + break; + } + + case XENMEM_machine_memory_map: { struct xen_memory_map memmap; diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 1f996e0..39cc3c0 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -4871,6 +4871,41 @@ long arch_memory_op(int op, XEN_GUEST_HANDLE(void) arg) return rc; } + case XENMEM_remove_from_physmap: + { + struct xen_remove_from_physmap xrfp; + unsigned long mfn; + struct domain *d; + + if ( copy_from_guest(&xrfp, arg, 1) ) + return -EFAULT; + + rc = rcu_lock_target_domain_by_id(xrfp.domid, &d); + if ( rc != 0 ) + return rc; + + if ( xsm_remove_from_physmap(current->domain, d) ) + { + rcu_unlock_domain(d); + return -EPERM; + } + + domain_lock(d); + + mfn = get_gfn_untyped(d, xrfp.gpfn); + + if ( mfn_valid(mfn) ) + guest_physmap_remove_page(d, xrfp.gpfn, mfn, PAGE_ORDER_4K); + + put_gfn(d, xrfp.gpfn); + + domain_unlock(d); + + rcu_unlock_domain(d); + + break; + } + case XENMEM_set_memory_map: { struct xen_foreign_memory_map fmap; diff --git a/xen/arch/x86/x86_64/compat/mm.c b/xen/arch/x86/x86_64/compat/mm.c index bea94fe..dde5430 100644 --- a/xen/arch/x86/x86_64/compat/mm.c +++ b/xen/arch/x86/x86_64/compat/mm.c @@ -82,6 +82,20 @@ int compat_arch_memory_op(int op, XEN_GUEST_HANDLE(void) arg) break; } + case XENMEM_remove_from_physmap: + { + struct compat_remove_from_physmap cmp; + struct xen_remove_from_physmap *nat = (void *)COMPAT_ARG_XLAT_VIRT_BASE; + + if ( copy_from_guest(&cmp, arg, 1) ) + return -EFAULT; + + XLAT_remove_from_physmap(nat, &cmp); + rc = arch_memory_op(op, guest_handle_from_ptr(nat, void)); + + break; + } + case XENMEM_set_memory_map: { struct compat_foreign_memory_map cmp; diff --git a/xen/include/public/memory.h b/xen/include/public/memory.h index c5b78a8..308deff 100644 --- a/xen/include/public/memory.h +++ b/xen/include/public/memory.h @@ -229,6 +229,22 @@ struct xen_add_to_physmap { typedef struct xen_add_to_physmap xen_add_to_physmap_t; DEFINE_XEN_GUEST_HANDLE(xen_add_to_physmap_t); +/* + * Unmaps the page appearing at a particular GPFN from the specified guest''s + * pseudophysical address space. + * arg == addr of xen_remove_from_physmap_t. + */ +#define XENMEM_remove_from_physmap 15 +struct xen_remove_from_physmap { + /* Which domain to change the mapping for. */ + domid_t domid; + + /* GPFN of the current mapping of the page. */ + xen_pfn_t gpfn; +}; +typedef struct xen_remove_from_physmap xen_remove_from_physmap_t; +DEFINE_XEN_GUEST_HANDLE(xen_remove_from_physmap_t); + /*** REMOVED ***/ /*#define XENMEM_translate_gpfn_list 8*/ diff --git a/xen/include/xlat.lst b/xen/include/xlat.lst index 3d92175..ee1772c 100644 --- a/xen/include/xlat.lst +++ b/xen/include/xlat.lst @@ -81,6 +81,7 @@ ! processor_power platform.h ? processor_px platform.h ! psd_package platform.h +! remove_from_physmap memory.h ? xenpf_pcpuinfo platform.h ? xenpf_pcpu_version platform.h ! sched_poll sched.h diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h index df6cec2..566c808 100644 --- a/xen/include/xsm/xsm.h +++ b/xen/include/xsm/xsm.h @@ -169,6 +169,7 @@ struct xsm_operations { int (*update_va_mapping) (struct domain *d, struct domain *f, l1_pgentry_t pte); int (*add_to_physmap) (struct domain *d1, struct domain *d2); + int (*remove_from_physmap) (struct domain *d1, struct domain *d2); int (*sendtrigger) (struct domain *d); int (*bind_pt_irq) (struct domain *d, struct xen_domctl_bind_pt_irq *bind); int (*unbind_pt_irq) (struct domain *d); @@ -738,6 +739,11 @@ static inline int xsm_add_to_physmap(struct domain *d1, struct domain *d2) return xsm_call(add_to_physmap(d1, d2)); } +static inline int xsm_remove_from_physmap(struct domain *d1, struct domain *d2) +{ + return xsm_call(remove_from_physmap(d1, d2)); +} + static inline int xsm_sendtrigger(struct domain *d) { return xsm_call(sendtrigger(d)); diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c index 4bbfbff..65daa4e 100644 --- a/xen/xsm/dummy.c +++ b/xen/xsm/dummy.c @@ -529,6 +529,11 @@ static int dummy_add_to_physmap (struct domain *d1, struct domain *d2) return 0; } +static int dummy_remove_from_physmap (struct domain *d1, struct domain *d2) +{ + return 0; +} + static int dummy_sendtrigger (struct domain *d) { return 0; @@ -690,6 +695,7 @@ void xsm_fixup_ops (struct xsm_operations *ops) set_to_dummy_if_null(ops, mmu_machphys_update); set_to_dummy_if_null(ops, update_va_mapping); set_to_dummy_if_null(ops, add_to_physmap); + set_to_dummy_if_null(ops, remove_from_physmap); set_to_dummy_if_null(ops, sendtrigger); set_to_dummy_if_null(ops, bind_pt_irq); set_to_dummy_if_null(ops, pin_mem_cacheattr); diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c index 0d35767..a2020a9 100644 --- a/xen/xsm/flask/hooks.c +++ b/xen/xsm/flask/hooks.c @@ -1283,6 +1283,11 @@ static int flask_add_to_physmap(struct domain *d1, struct domain *d2) return domain_has_perm(d1, d2, SECCLASS_MMU, MMU__PHYSMAP); } +static int flask_remove_from_physmap(struct domain *d1, struct domain *d2) +{ + return domain_has_perm(d1, d2, SECCLASS_MMU, MMU__PHYSMAP); +} + static int flask_sendtrigger(struct domain *d) { return domain_has_perm(current->domain, d, SECCLASS_DOMAIN, DOMAIN__TRIGGER); @@ -1550,6 +1555,7 @@ static struct xsm_operations flask_ops = { .mmu_machphys_update = flask_mmu_machphys_update, .update_va_mapping = flask_update_va_mapping, .add_to_physmap = flask_add_to_physmap, + .remove_from_physmap = flask_remove_from_physmap, .sendtrigger = flask_sendtrigger, .get_device_group = flask_get_device_group, .test_assign_device = flask_test_assign_device, -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 02/18] xen: allow global VIRQ handlers to be delegated to other domains
This patch sends global VIRQs to a domain designated as the VIRQ handler instead of sending all global VIRQ events to dom0. This is required in order to run xenstored in a stubdom, because VIRQ_DOM_EXC must be sent to xenstored for domain destruction to work properly. This patch was inspired by the xenstored stubdomain patch series sent to xen-devel by Alex Zeffertt in 2009. Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/flask/policy/policy/flask/access_vectors | 1 + tools/libxc/xc_domain.c | 10 ++++ tools/libxc/xenctrl.h | 9 +++ xen/arch/x86/cpu/mcheck/amd_nonfatal.c | 2 +- xen/arch/x86/cpu/mcheck/mce.c | 2 +- xen/arch/x86/cpu/mcheck/mce_intel.c | 6 +- xen/arch/x86/cpu/mcheck/non-fatal.c | 2 +- xen/common/cpu.c | 4 +- xen/common/domain.c | 8 ++-- xen/common/domctl.c | 17 ++++++ xen/common/event_channel.c | 63 +++++++++++++++++++++++- xen/common/trace.c | 2 +- xen/drivers/char/console.c | 4 +- xen/include/public/domctl.h | 8 +++ xen/include/xen/event.h | 12 +++- xen/include/xsm/xsm.h | 6 ++ xen/xsm/dummy.c | 6 ++ xen/xsm/flask/hooks.c | 6 ++ xen/xsm/flask/include/av_perm_to_string.h | 1 + xen/xsm/flask/include/av_permissions.h | 1 + 20 files changed, 151 insertions(+), 19 deletions(-) diff --git a/tools/flask/policy/policy/flask/access_vectors b/tools/flask/policy/policy/flask/access_vectors index 644f2e1..5901911 100644 --- a/tools/flask/policy/policy/flask/access_vectors +++ b/tools/flask/policy/policy/flask/access_vectors @@ -85,6 +85,7 @@ class domain getpodtarget setpodtarget set_misc_info + set_virq_handler } class hvm diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c index ab019b8..d98e68b 100644 --- a/tools/libxc/xc_domain.c +++ b/tools/libxc/xc_domain.c @@ -1504,6 +1504,16 @@ int xc_domain_set_access_required(xc_interface *xch, return do_domctl(xch, &domctl); } +int xc_domain_set_virq_handler(xc_interface *xch, uint32_t domid, int virq) +{ + DECLARE_DOMCTL; + + domctl.cmd = XEN_DOMCTL_set_virq_handler; + domctl.domain = domid; + domctl.u.set_virq_handler.virq = virq; + return do_domctl(xch, &domctl); +} + /* * Local variables: * mode: C diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h index 8b34769..8f3426f 100644 --- a/tools/libxc/xenctrl.h +++ b/tools/libxc/xenctrl.h @@ -747,6 +747,15 @@ int xc_domain_p2m_audit(xc_interface *xch, int xc_domain_set_access_required(xc_interface *xch, uint32_t domid, unsigned int required); +/** + * This function sets the handler of global VIRQs sent by the hypervisor + * + * @parm xch a handle to an open hypervisor interface + * @parm domid the domain id which will handle the VIRQ + * @parm virq the virq number (VIRQ_*) + * return 0 on success, -1 on failure + */ +int xc_domain_set_virq_handler(xc_interface *xch, uint32_t domid, int virq); /* * CPUPOOL MANAGEMENT FUNCTIONS diff --git a/xen/arch/x86/cpu/mcheck/amd_nonfatal.c b/xen/arch/x86/cpu/mcheck/amd_nonfatal.c index 50288bd..9222098 100644 --- a/xen/arch/x86/cpu/mcheck/amd_nonfatal.c +++ b/xen/arch/x86/cpu/mcheck/amd_nonfatal.c @@ -100,7 +100,7 @@ static void mce_amd_checkregs(void *info) if (dom0_vmce_enabled()) { mctelem_commit(mctc); - send_guest_global_virq(dom0, VIRQ_MCA); + send_global_virq(VIRQ_MCA); } else if (++dumpcount >= 10) { x86_mcinfo_dump((struct mc_info *)mctelem_dataptr(mctc)); mctelem_dismiss(mctc); diff --git a/xen/arch/x86/cpu/mcheck/mce.c b/xen/arch/x86/cpu/mcheck/mce.c index b592041..c4e4477 100644 --- a/xen/arch/x86/cpu/mcheck/mce.c +++ b/xen/arch/x86/cpu/mcheck/mce.c @@ -594,7 +594,7 @@ void mcheck_cmn_handler(struct cpu_user_regs *regs, long error_code, if (dom0_vmce_enabled()) { if (mctc != NULL) mctelem_commit(mctc); - send_guest_global_virq(dom0, VIRQ_MCA); + send_global_virq(VIRQ_MCA); } else { x86_mcinfo_dump(mci); if (mctc != NULL) diff --git a/xen/arch/x86/cpu/mcheck/mce_intel.c b/xen/arch/x86/cpu/mcheck/mce_intel.c index 0986025..0894080 100644 --- a/xen/arch/x86/cpu/mcheck/mce_intel.c +++ b/xen/arch/x86/cpu/mcheck/mce_intel.c @@ -354,7 +354,7 @@ static void mce_softirq(void) /* Step2: Send Log to DOM0 through vIRQ */ if (dom0_vmce_enabled()) { mce_printk(MCE_VERBOSE, "MCE: send MCE# to DOM0 through virq\n"); - send_guest_global_virq(dom0, VIRQ_MCA); + send_global_virq(VIRQ_MCA); } } @@ -1085,7 +1085,7 @@ static void cmci_discover(void) if (bs.errcnt && mctc != NULL) { if (dom0_vmce_enabled()) { mctelem_commit(mctc); - send_guest_global_virq(dom0, VIRQ_MCA); + send_global_virq(VIRQ_MCA); } else { x86_mcinfo_dump(mctelem_dataptr(mctc)); mctelem_dismiss(mctc); @@ -1205,7 +1205,7 @@ fastcall void smp_cmci_interrupt(struct cpu_user_regs *regs) if (dom0_vmce_enabled()) { mctelem_commit(mctc); mce_printk(MCE_VERBOSE, "CMCI: send CMCI to DOM0 through virq\n"); - send_guest_global_virq(dom0, VIRQ_MCA); + send_global_virq(VIRQ_MCA); } else { x86_mcinfo_dump(mctelem_dataptr(mctc)); mctelem_dismiss(mctc); diff --git a/xen/arch/x86/cpu/mcheck/non-fatal.c b/xen/arch/x86/cpu/mcheck/non-fatal.c index c57688f..1dded9b 100644 --- a/xen/arch/x86/cpu/mcheck/non-fatal.c +++ b/xen/arch/x86/cpu/mcheck/non-fatal.c @@ -55,7 +55,7 @@ static void mce_checkregs (void *info) if (dom0_vmce_enabled()) { mctelem_commit(mctc); - send_guest_global_virq(dom0, VIRQ_MCA); + send_global_virq(VIRQ_MCA); } else if (++dumpcount >= 10) { x86_mcinfo_dump((struct mc_info *)mctelem_dataptr(mctc)); mctelem_dismiss(mctc); diff --git a/xen/common/cpu.c b/xen/common/cpu.c index 79abdb7..630881e 100644 --- a/xen/common/cpu.c +++ b/xen/common/cpu.c @@ -108,7 +108,7 @@ int cpu_down(unsigned int cpu) notifier_rc = notifier_call_chain(&cpu_chain, CPU_DEAD, hcpu, NULL); BUG_ON(notifier_rc != NOTIFY_DONE); - send_guest_global_virq(dom0, VIRQ_PCPU_STATE); + send_global_virq(VIRQ_PCPU_STATE); cpu_hotplug_done(); return 0; @@ -148,7 +148,7 @@ int cpu_up(unsigned int cpu) notifier_rc = notifier_call_chain(&cpu_chain, CPU_ONLINE, hcpu, NULL); BUG_ON(notifier_rc != NOTIFY_DONE); - send_guest_global_virq(dom0, VIRQ_PCPU_STATE); + send_global_virq(VIRQ_PCPU_STATE); cpu_hotplug_done(); return 0; diff --git a/xen/common/domain.c b/xen/common/domain.c index 52a63ef..f1a7ede 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -116,7 +116,7 @@ static void __domain_finalise_shutdown(struct domain *d) if ( (d->shutdown_code == SHUTDOWN_suspend) && d->suspend_evtchn ) evtchn_send(d, d->suspend_evtchn); else - send_guest_global_virq(dom0, VIRQ_DOM_EXC); + send_global_virq(VIRQ_DOM_EXC); } static void vcpu_check_shutdown(struct vcpu *v) @@ -492,7 +492,7 @@ int domain_kill(struct domain *d) } d->is_dying = DOMDYING_dead; put_domain(d); - send_guest_global_virq(dom0, VIRQ_DOM_EXC); + send_global_virq(VIRQ_DOM_EXC); /* fallthrough */ case DOMDYING_dead: break; @@ -633,7 +633,7 @@ void domain_pause_for_debugger(void) for_each_vcpu ( d, v ) vcpu_sleep_nosync(v); - send_guest_global_virq(dom0, VIRQ_DEBUGGER); + send_global_virq(VIRQ_DEBUGGER); } /* Complete domain destroy after RCU readers are not holding old references. */ @@ -690,7 +690,7 @@ static void complete_domain_destroy(struct rcu_head *head) free_cpumask_var(d->domain_dirty_cpumask); free_domain_struct(d); - send_guest_global_virq(dom0, VIRQ_DOM_EXC); + send_global_virq(VIRQ_DOM_EXC); } /* Release resources belonging to task @p. */ diff --git a/xen/common/domctl.c b/xen/common/domctl.c index d6ae09b..97c7d53 100644 --- a/xen/common/domctl.c +++ b/xen/common/domctl.c @@ -994,6 +994,23 @@ long do_domctl(XEN_GUEST_HANDLE(xen_domctl_t) u_domctl) } break; + case XEN_DOMCTL_set_virq_handler: + { + struct domain *d; + uint32_t virq = op->u.set_virq_handler.virq; + + ret = -ESRCH; + d = rcu_lock_domain_by_id(op->domain); + if ( d != NULL ) + { + ret = xsm_set_virq_handler(d, virq); + if ( !ret ) + ret = set_global_virq_handler(d, virq); + rcu_unlock_domain(d); + } + } + break; + default: ret = arch_do_domctl(op, u_domctl); break; diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c index 9212042..e507481 100644 --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -689,7 +689,7 @@ void send_guest_vcpu_virq(struct vcpu *v, int virq) spin_unlock_irqrestore(&v->virq_lock, flags); } -void send_guest_global_virq(struct domain *d, int virq) +static void send_guest_global_virq(struct domain *d, int virq) { unsigned long flags; int port; @@ -739,6 +739,65 @@ int send_guest_pirq(struct domain *d, const struct pirq *pirq) return evtchn_set_pending(d->vcpu[chn->notify_vcpu_id], port); } +static struct domain *global_virq_handlers[NR_VIRQS] __read_mostly; + +static DEFINE_SPINLOCK(global_virq_handlers_lock); + +void send_global_virq(uint32_t virq) +{ + ASSERT(virq < NR_VIRQS); + ASSERT(virq_is_global(virq)); + + send_guest_global_virq(global_virq_handlers[virq] ?: dom0, virq); +} + +int set_global_virq_handler(struct domain *d, uint32_t virq) +{ + struct domain *old; + + if (virq >= NR_VIRQS) + return -EINVAL; + if (!virq_is_global(virq)) + return -EINVAL; + + if (global_virq_handlers[virq] == d) + return 0; + + if (unlikely(!get_domain(d))) + return -EINVAL; + + spin_lock(&global_virq_handlers_lock); + old = global_virq_handlers[virq]; + global_virq_handlers[virq] = d; + spin_unlock(&global_virq_handlers_lock); + + if (old != NULL) + put_domain(old); + + return 0; +} + +static void clear_global_virq_handlers(struct domain *d) +{ + uint32_t virq; + int put_count = 0; + + spin_lock(&global_virq_handlers_lock); + + for (virq = 0; virq < NR_VIRQS; virq++) { + if (global_virq_handlers[virq] == d) { + global_virq_handlers[virq] = NULL; + put_count++; + } + } + + spin_unlock(&global_virq_handlers_lock); + + while (put_count) { + put_domain(d); + put_count--; + } +} static long evtchn_status(evtchn_status_t *status) { @@ -1160,6 +1219,8 @@ void evtchn_destroy(struct domain *d) d->evtchn[i] = NULL; } spin_unlock(&d->event_lock); + + clear_global_virq_handlers(d); } diff --git a/xen/common/trace.c b/xen/common/trace.c index 5772f24..58cbf39 100644 --- a/xen/common/trace.c +++ b/xen/common/trace.c @@ -661,7 +661,7 @@ static inline void insert_lost_records(struct t_buf *buf) */ static void trace_notify_dom0(unsigned long unused) { - send_guest_global_virq(dom0, VIRQ_TBUF); + send_global_virq(VIRQ_TBUF); } static DECLARE_SOFTIRQ_TASKLET(trace_notify_dom0_tasklet, trace_notify_dom0, 0); diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c index 8a4c684..79b266f 100644 --- a/xen/drivers/char/console.c +++ b/xen/drivers/char/console.c @@ -287,7 +287,7 @@ static void __serial_rx(char c, struct cpu_user_regs *regs) if ( (serial_rx_prod-serial_rx_cons) != SERIAL_RX_SIZE ) serial_rx_ring[SERIAL_RX_MASK(serial_rx_prod++)] = c; /* Always notify the guest: prevents receive path from getting stuck. */ - send_guest_global_virq(dom0, VIRQ_CONSOLE); + send_global_virq(VIRQ_CONSOLE); } static void serial_rx(char c, struct cpu_user_regs *regs) @@ -314,7 +314,7 @@ static void serial_rx(char c, struct cpu_user_regs *regs) static void notify_dom0_con_ring(unsigned long unused) { - send_guest_global_virq(dom0, VIRQ_CON_RING); + send_global_virq(VIRQ_CON_RING); } static DECLARE_SOFTIRQ_TASKLET(notify_dom0_con_ring_tasklet, notify_dom0_con_ring, 0); diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index c7640aa..75be370 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -813,6 +813,12 @@ struct xen_domctl_audit_p2m { typedef struct xen_domctl_audit_p2m xen_domctl_audit_p2m_t; DEFINE_XEN_GUEST_HANDLE(xen_domctl_audit_p2m_t); +struct xen_domctl_set_virq_handler { + uint32_t virq; /* IN */ +}; +typedef struct xen_domctl_set_virq_handler xen_domctl_set_virq_handler_t; +DEFINE_XEN_GUEST_HANDLE(xen_domctl_set_virq_handler_t); + #if defined(__i386__) || defined(__x86_64__) /* XEN_DOMCTL_setvcpuextstate */ /* XEN_DOMCTL_getvcpuextstate */ @@ -912,6 +918,7 @@ struct xen_domctl { #define XEN_DOMCTL_getvcpuextstate 63 #define XEN_DOMCTL_set_access_required 64 #define XEN_DOMCTL_audit_p2m 65 +#define XEN_DOMCTL_set_virq_handler 66 #define XEN_DOMCTL_gdbsx_guestmemio 1000 #define XEN_DOMCTL_gdbsx_pausevcpu 1001 #define XEN_DOMCTL_gdbsx_unpausevcpu 1002 @@ -966,6 +973,7 @@ struct xen_domctl { #endif struct xen_domctl_set_access_required access_required; struct xen_domctl_audit_p2m audit_p2m; + struct xen_domctl_set_virq_handler set_virq_handler; struct xen_domctl_gdbsx_memio gdbsx_guest_memio; struct xen_domctl_gdbsx_pauseunp_vcpu gdbsx_pauseunp_vcpu; struct xen_domctl_gdbsx_domstatus gdbsx_domstatus; diff --git a/xen/include/xen/event.h b/xen/include/xen/event.h index 7e5ad7b..d505ee9 100644 --- a/xen/include/xen/event.h +++ b/xen/include/xen/event.h @@ -24,11 +24,17 @@ void send_guest_vcpu_virq(struct vcpu *v, int virq); /* - * send_guest_global_virq: Notify guest via a global VIRQ. - * @d: Domain to which virtual IRQ should be sent + * send_global_virq: Notify the domain handling a global VIRQ. * @virq: Virtual IRQ number (VIRQ_*) */ -void send_guest_global_virq(struct domain *d, int virq); +void send_global_virq(uint32_t virq); + +/* + * sent_global_virq_handler: Set a global VIRQ handler. + * @d: New target domain for this VIRQ + * @virq: Virtual IRQ number (VIRQ_*), must be global + */ +int set_global_virq_handler(struct domain *d, uint32_t virq); /* * send_guest_pirq: diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h index 566c808..c89c6ed 100644 --- a/xen/include/xsm/xsm.h +++ b/xen/include/xsm/xsm.h @@ -64,6 +64,7 @@ struct xsm_operations { int (*domain_settime) (struct domain *d); int (*set_target) (struct domain *d, struct domain *e); int (*domctl) (struct domain *d, int cmd); + int (*set_virq_handler) (struct domain *d, int virq); int (*tbufcontrol) (void); int (*readconsole) (uint32_t clear); int (*sched_id) (void); @@ -265,6 +266,11 @@ static inline int xsm_domctl (struct domain *d, int cmd) return xsm_call(domctl(d, cmd)); } +static inline int xsm_set_virq_handler (struct domain *d, int virq) +{ + return xsm_call(set_virq_handler(d, virq)); +} + static inline int xsm_tbufcontrol (void) { return xsm_call(tbufcontrol()); diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c index 65daa4e..59db86d 100644 --- a/xen/xsm/dummy.c +++ b/xen/xsm/dummy.c @@ -94,6 +94,11 @@ static int dummy_domctl(struct domain *d, int cmd) return 0; } +static int dummy_set_virq_handler(struct domain *d, int virq) +{ + return 0; +} + static int dummy_tbufcontrol (void) { return 0; @@ -596,6 +601,7 @@ void xsm_fixup_ops (struct xsm_operations *ops) set_to_dummy_if_null(ops, domain_settime); set_to_dummy_if_null(ops, set_target); set_to_dummy_if_null(ops, domctl); + set_to_dummy_if_null(ops, set_virq_handler); set_to_dummy_if_null(ops, tbufcontrol); set_to_dummy_if_null(ops, readconsole); set_to_dummy_if_null(ops, sched_id); diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c index a2020a9..a1feb8d 100644 --- a/xen/xsm/flask/hooks.c +++ b/xen/xsm/flask/hooks.c @@ -597,6 +597,11 @@ static int flask_domctl(struct domain *d, int cmd) return domain_has_perm(current->domain, d, SECCLASS_DOMAIN, DOMAIN__SET_MISC_INFO); } +static int flask_set_virq_handler(struct domain *d, int virq) +{ + return domain_has_perm(current->domain, d, SECCLASS_DOMAIN, DOMAIN__SET_VIRQ_HANDLER); +} + static int flask_tbufcontrol(void) { return domain_has_xen(current->domain, XEN__TBUFCONTROL); @@ -1460,6 +1465,7 @@ static struct xsm_operations flask_ops = { .domain_settime = flask_domain_settime, .set_target = flask_set_target, .domctl = flask_domctl, + .set_virq_handler = flask_set_virq_handler, .tbufcontrol = flask_tbufcontrol, .readconsole = flask_readconsole, .sched_id = flask_sched_id, diff --git a/xen/xsm/flask/include/av_perm_to_string.h b/xen/xsm/flask/include/av_perm_to_string.h index 85cbffc..17a1c36 100644 --- a/xen/xsm/flask/include/av_perm_to_string.h +++ b/xen/xsm/flask/include/av_perm_to_string.h @@ -60,6 +60,7 @@ S_(SECCLASS_DOMAIN, DOMAIN__GETPODTARGET, "getpodtarget") S_(SECCLASS_DOMAIN, DOMAIN__SETPODTARGET, "setpodtarget") S_(SECCLASS_DOMAIN, DOMAIN__SET_MISC_INFO, "set_misc_info") + S_(SECCLASS_DOMAIN, DOMAIN__SET_VIRQ_HANDLER, "set_virq_handler") S_(SECCLASS_HVM, HVM__SETHVMC, "sethvmc") S_(SECCLASS_HVM, HVM__GETHVMC, "gethvmc") S_(SECCLASS_HVM, HVM__SETPARAM, "setparam") diff --git a/xen/xsm/flask/include/av_permissions.h b/xen/xsm/flask/include/av_permissions.h index 9e55a86..42eaf81 100644 --- a/xen/xsm/flask/include/av_permissions.h +++ b/xen/xsm/flask/include/av_permissions.h @@ -61,6 +61,7 @@ #define DOMAIN__GETPODTARGET 0x10000000UL #define DOMAIN__SETPODTARGET 0x20000000UL #define DOMAIN__SET_MISC_INFO 0x40000000UL +#define DOMAIN__SET_VIRQ_HANDLER 0x80000000UL #define HVM__SETHVMC 0x00000001UL #define HVM__GETHVMC 0x00000002UL -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 03/18] xen: use XSM instead of IS_PRIV for getdomaininfo
The XEN_DOMCTL_getdomaininfo domctl does not allow manipulation of domains, only basic information such as size and state, so its use does not fully justify making a domain privileged. XSM modules can also provide fine-grained control over what domains are visible to domains that call getdomaininfo. If XSM is disabled (either at compile time or by using the dummy XSM module) then there is no change in behavior: only IS_PRIV domains can use this domctl. If enabled, the XSM module controls access. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- xen/common/domctl.c | 4 ++++ xen/xsm/dummy.c | 2 ++ 2 files changed, 6 insertions(+), 0 deletions(-) diff --git a/xen/common/domctl.c b/xen/common/domctl.c index 97c7d53..e1bbc9a 100644 --- a/xen/common/domctl.c +++ b/xen/common/domctl.c @@ -263,6 +263,10 @@ long do_domctl(XEN_GUEST_HANDLE(xen_domctl_t) u_domctl) return -EPERM; break; } +#ifdef XSM_ENABLE + case XEN_DOMCTL_getdomaininfo: + break; +#endif default: if ( !IS_PRIV(current->domain) ) return -EPERM; diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c index 59db86d..f94f616 100644 --- a/xen/xsm/dummy.c +++ b/xen/xsm/dummy.c @@ -66,6 +66,8 @@ static int dummy_scheduler (struct domain *d) static int dummy_getdomaininfo (struct domain *d) { + if ( !IS_PRIV(current->domain) ) + return -EPERM; return 0; } -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 04/18] xen: Preserve reserved grant entries when switching versions
In order for the toolstack to use reserved grant table entries, the grant table for a guest must be initialized prior to the guest''s boot. When the guest switches grant table versions (necessary if the guest is using v2 grant tables, or on kexec if switching grant versions), these initial grants will be cleared. Instead of clearing them, preserve the grants across the type change. Attempting to use or preserve v2-only features such as sub-page grants will produce invalid v1 grant entries, which (while they will not work) is not a problem since a guest can always produce such invalid entries. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- xen/common/grant_table.c | 38 +++++++++++++++++++++++++++++++------- xen/include/public/grant_table.h | 6 ++++++ 2 files changed, 37 insertions(+), 7 deletions(-) diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c index 014734d..8d4a4cb 100644 --- a/xen/common/grant_table.c +++ b/xen/common/grant_table.c @@ -2106,6 +2106,7 @@ gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) struct domain *d = current->domain; struct grant_table *gt = d->grant_table; struct active_grant_entry *act; + grant_entry_v1_t reserved_entries[GNTTAB_NR_RESERVED_ENTRIES]; long res; int i; @@ -2126,7 +2127,7 @@ gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) /* (You need to change the version number for e.g. kexec.) */ if ( gt->gt_version != 0 ) { - for ( i = 0; i < nr_grant_entries(gt); i++ ) + for ( i = GNTTAB_NR_RESERVED_ENTRIES; i < nr_grant_entries(gt); i++ ) { act = &active_entry(gt, i); if ( act->pin != 0 ) @@ -2151,15 +2152,38 @@ gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) goto out_unlock; } + /* Preserve the first 8 entries (toolstack reserved grants) */ + if (gt->gt_version == 1) { + memcpy(reserved_entries, gt->shared_v1[0], sizeof(reserved_entries)); + } else if (gt->gt_version == 2) { + for ( i = 0; i < GNTTAB_NR_RESERVED_ENTRIES && i < nr_grant_entries(gt); i++ ) + { + reserved_entries[i].flags = shared_entry_v2(gt, i).hdr.flags; + reserved_entries[i].domid = shared_entry_v2(gt, i).hdr.domid; + reserved_entries[i].frame = shared_entry_v2(gt, i).full_page.frame; + reserved_entries[i].flags |= status_entry(gt, i); + } + } + if ( op.version < 2 && gt->gt_version == 2 ) gnttab_unpopulate_status_frames(d, gt); - if ( op.version != gt->gt_version ) - { - /* Make sure there''s no crud left over in the table from the - old version. */ - for ( i = 0; i < nr_grant_frames(gt); i++ ) - memset(gt->shared_raw[i], 0, PAGE_SIZE); + /* Make sure there''s no crud left over in the table from the + old version. */ + for ( i = 0; i < nr_grant_frames(gt); i++ ) + memset(gt->shared_raw[i], 0, PAGE_SIZE); + + /* Restore the first 8 entries (toolstack reserved grants) */ + if (gt->gt_version != 0 && op.version == 1) { + memcpy(gt->shared_v1[0], reserved_entries, sizeof(reserved_entries)); + } else if (gt->gt_version != 0 && op.version == 2) { + for ( i = 0; i < GNTTAB_NR_RESERVED_ENTRIES; i++ ) + { + status_entry(gt, i) = reserved_entries[i].flags & (GTF_reading|GTF_writing); + shared_entry_v2(gt, i).hdr.flags = reserved_entries[i].flags & ~(GTF_reading|GTF_writing); + shared_entry_v2(gt, i).hdr.domid = reserved_entries[i].domid; + shared_entry_v2(gt, i).full_page.frame = reserved_entries[i].frame; + } } gt->gt_version = op.version; diff --git a/xen/include/public/grant_table.h b/xen/include/public/grant_table.h index 0bf20bc..36d1ac7 100644 --- a/xen/include/public/grant_table.h +++ b/xen/include/public/grant_table.h @@ -117,6 +117,12 @@ struct grant_entry_v1 { }; typedef struct grant_entry_v1 grant_entry_v1_t; +/* External tools reserve first few grant table entries. */ +#define GNTTAB_NR_RESERVED_ENTRIES 8 +#define GNTTAB_RESERVED_CONSOLE 0 +#define GNTTAB_RESERVED_XENSTORE 1 +/* (the next 6 are reserved for future use) */ + /* * Type of grant entry. * GTF_invalid: This grant entry grants no privileges. -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 05/18] tools/libxl: pull xenstore/console domids from xenstore
Instead of assuming that xenstored and xenconsoled are running in dom0, pull the domain IDs from xenstore. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/libxl/libxl_dom.c | 14 ++++++++++++-- tools/libxl/libxl_internal.h | 2 ++ 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c index a4725fe..5e39595 100644 --- a/tools/libxl/libxl_dom.c +++ b/tools/libxl/libxl_dom.c @@ -73,6 +73,7 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid, { libxl_ctx *ctx = libxl__gc_owner(gc); int tsc_mode; + char *xs_domid, *con_domid; xc_domain_max_vcpus(ctx->xch, domid, info->max_vcpus); xc_domain_setmaxmem(ctx->xch, domid, info->target_memkb + LIBXL_MAXMEM_CONSTANT); if (info->type == LIBXL_DOMAIN_TYPE_PV) @@ -104,9 +105,18 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid, xc_shadow_control(ctx->xch, domid, XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION, NULL, 0, &shadow, 0, NULL); } - state->store_port = xc_evtchn_alloc_unbound(ctx->xch, domid, 0); - state->console_port = xc_evtchn_alloc_unbound(ctx->xch, domid, 0); + xs_domid = xs_read(ctx->xsh, XBT_NULL, "/tool/xenstored/domid", NULL); + state->store_domid = xs_domid ? atoi(xs_domid) : 0; + free(xs_domid); + + con_domid = xs_read(ctx->xsh, XBT_NULL, "/tool/xenconsoled/domid", NULL); + state->console_domid = con_domid ? atoi(con_domid) : 0; + free(con_domid); + + state->store_port = xc_evtchn_alloc_unbound(ctx->xch, domid, state->store_domid); + state->console_port = xc_evtchn_alloc_unbound(ctx->xch, domid, state->console_domid); state->vm_generationid_addr = 0; + return 0; } diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index 288c03c..97ead08 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -219,9 +219,11 @@ _hidden int libxl__domain_shutdown_reason(libxl__gc *gc, uint32_t domid); libxl__domain_type((gc), (domid)) == LIBXL_DOMAIN_TYPE_##type typedef struct { uint32_t store_port; + uint32_t store_domid; unsigned long store_mfn; uint32_t console_port; + uint32_t console_domid; unsigned long console_mfn; unsigned long vm_generationid_addr; } libxl__domain_build_state; -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants
From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> This patch claims one reserved grant entry for the console and another for the xenstore. It modifies the builder to fill in the grant table entries for the console and the xenstore. Previous versions of this patch have been sent to xen-devel. See http://lists.xensource.com/archives/html/xen-devel/2008-07/msg00610.html http://lists.xensource.com/archives/html/xen-devel/2009-03/msg01491.html Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/libxc/xc_dom.h | 13 +++ tools/libxc/xc_dom_boot.c | 164 +++++++++++++++++++++++++++++++++++++ tools/libxc/xc_dom_compat_linux.c | 2 + tools/libxc/xc_domain_restore.c | 19 ++++- tools/libxc/xenguest.h | 3 +- tools/libxl/libxl_dom.c | 18 ++++- tools/xcutils/xc_restore.c | 4 +- 7 files changed, 216 insertions(+), 7 deletions(-) diff --git a/tools/libxc/xc_dom.h b/tools/libxc/xc_dom.h index e72f066..6c36403 100644 --- a/tools/libxc/xc_dom.h +++ b/tools/libxc/xc_dom.h @@ -106,7 +106,9 @@ struct xc_dom_image { /* misc xen domain config stuff */ unsigned long flags; unsigned int console_evtchn; + unsigned int console_domid; unsigned int xenstore_evtchn; + unsigned int xenstore_domid; xen_pfn_t shared_info_mfn; xc_interface *xch; @@ -200,6 +202,17 @@ void *xc_dom_boot_domU_map(struct xc_dom_image *dom, xen_pfn_t pfn, xen_pfn_t count); int xc_dom_boot_image(struct xc_dom_image *dom); int xc_dom_compat_check(struct xc_dom_image *dom); +int xc_dom_gnttab_init(struct xc_dom_image *dom); +int xc_dom_gnttab_hvm_seed(xc_interface *xch, uint32_t domid, + unsigned long console_gmfn, + unsigned long xenstore_gmfn, + uint32_t console_domid, + uint32_t xenstore_domid); +int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid, + unsigned long console_gmfn, + unsigned long xenstore_gmfn, + uint32_t console_domid, + uint32_t xenstore_domid); /* --- debugging bits ---------------------------------------------- */ diff --git a/tools/libxc/xc_dom_boot.c b/tools/libxc/xc_dom_boot.c index 65f60df..1a55a6d 100644 --- a/tools/libxc/xc_dom_boot.c +++ b/tools/libxc/xc_dom_boot.c @@ -34,6 +34,7 @@ #include "xg_private.h" #include "xc_dom.h" #include <xen/hvm/params.h> +#include <xen/grant_table.h> /* ------------------------------------------------------------------------ */ @@ -275,6 +276,169 @@ int xc_dom_boot_image(struct xc_dom_image *dom) return rc; } +static unsigned long xc_dom_gnttab_setup(xc_interface *xch, uint32_t domid) +{ + DECLARE_HYPERCALL; + gnttab_setup_table_t setup_table; + DECLARE_HYPERCALL_BUFFER(unsigned long, gmfnp); + int rc; + unsigned long gmfn; + + gmfnp = xc_hypercall_buffer_alloc(xch, gmfnp, sizeof(*gmfnp)); + if (gmfnp == NULL) + return -1; + + setup_table.dom = domid; + setup_table.nr_frames = 1; + set_xen_guest_handle(setup_table.frame_list, gmfnp); + setup_table.status = 0; + + hypercall.op = __HYPERVISOR_grant_table_op; + hypercall.arg[0] = GNTTABOP_setup_table; + hypercall.arg[1] = (unsigned long) &setup_table; + hypercall.arg[2] = 1; + + rc = do_xen_hypercall(xch, &hypercall); + gmfn = *gmfnp; + xc_hypercall_buffer_free(xch, gmfnp); + + if ( rc != 0 || setup_table.status != GNTST_okay ) + { + xc_dom_panic(xch, XC_INTERNAL_ERROR, + "%s: failed to setup domU grant table " + "[errno=%d, status=%" PRId16 "]\n", + __FUNCTION__, rc != 0 ? errno : 0, setup_table.status); + return -1; + } + + return gmfn; +} + +int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid, + unsigned long console_gmfn, + unsigned long xenstore_gmfn, + uint32_t console_domid, + uint32_t xenstore_domid) +{ + + unsigned long gnttab_gmfn; + grant_entry_v1_t *gnttab; + + gnttab_gmfn = xc_dom_gnttab_setup(xch, domid); + if ( gnttab_gmfn == -1 ) + return -1; + + gnttab = xc_map_foreign_range(xch, + domid, + PAGE_SIZE, + PROT_READ|PROT_WRITE, + gnttab_gmfn); + if ( gnttab == NULL ) + { + xc_dom_panic(xch, XC_INTERNAL_ERROR, + "%s: failed to map domU grant table " + "[errno=%d]\n", + __FUNCTION__, errno); + return -1; + } + + if ( domid != console_domid && console_gmfn != -1) + { + gnttab[GNTTAB_RESERVED_CONSOLE].flags = GTF_permit_access; + gnttab[GNTTAB_RESERVED_CONSOLE].domid = console_domid; + gnttab[GNTTAB_RESERVED_CONSOLE].frame = console_gmfn; + } + if ( domid != xenstore_domid && xenstore_gmfn != -1) + { + gnttab[GNTTAB_RESERVED_XENSTORE].flags = GTF_permit_access; + gnttab[GNTTAB_RESERVED_XENSTORE].domid = xenstore_domid; + gnttab[GNTTAB_RESERVED_XENSTORE].frame = xenstore_gmfn; + } + + if ( munmap(gnttab, PAGE_SIZE) == -1 ) + { + xc_dom_panic(xch, XC_INTERNAL_ERROR, + "%s: failed to unmap domU grant table " + "[errno=%d]\n", + __FUNCTION__, errno); + return -1; + } + + return 0; +} + +int xc_dom_gnttab_hvm_seed(xc_interface *xch, uint32_t domid, + unsigned long console_gpfn, + unsigned long xenstore_gpfn, + uint32_t console_domid, + uint32_t xenstore_domid) +{ +#define SCRATCH_PFN_GNTTAB 0xFFFFE + + int rc; + struct xen_add_to_physmap xatp = { + .domid = domid, + .space = XENMAPSPACE_grant_table, + .idx = 0, /* TODO: what''s this? */ + .gpfn = SCRATCH_PFN_GNTTAB + }; + struct xen_remove_from_physmap xrfp = { + .domid = domid, + .gpfn = SCRATCH_PFN_GNTTAB + }; + + rc = do_memory_op(xch, XENMEM_add_to_physmap, &xatp, sizeof(xatp)); + if ( rc != 0 ) + { + xc_dom_panic(xch, XC_INTERNAL_ERROR, + "%s: failed to add gnttab to physmap " + "[errno=%d]\n", + __FUNCTION__, errno); + return -1; + } + + rc = xc_dom_gnttab_seed(xch, domid, + console_gpfn, xenstore_gpfn, + console_domid, xenstore_domid); + if (rc != 0) + { + xc_dom_panic(xch, XC_INTERNAL_ERROR, + "%s: failed to seed gnttab entries\n", + __FUNCTION__); + (void) do_memory_op(xch, XENMEM_remove_from_physmap, &xrfp, sizeof(xrfp)); + return -1; + } + + rc = do_memory_op(xch, XENMEM_remove_from_physmap, &xrfp, sizeof(xrfp)); + if (rc != 0) + { + xc_dom_panic(xch, XC_INTERNAL_ERROR, + "%s: failed to remove gnttab from physmap " + "[errno=%d]\n", + __FUNCTION__, errno); + return -1; + } + + return 0; +} + +int xc_dom_gnttab_init(struct xc_dom_image *dom) +{ + unsigned long console_gmfn; + unsigned long xenstore_gmfn; + int autotranslated; + + autotranslated = xc_dom_feature_translated(dom); + console_gmfn = autotranslated ? + dom->console_pfn : xc_dom_p2m_host(dom, dom->console_pfn); + xenstore_gmfn = autotranslated ? + dom->xenstore_pfn : xc_dom_p2m_host(dom, dom->xenstore_pfn); + + return xc_dom_gnttab_seed(dom->xch, dom->guest_domid, + console_gmfn, xenstore_gmfn, + dom->console_domid, dom->xenstore_domid); +} + /* * Local variables: * mode: C diff --git a/tools/libxc/xc_dom_compat_linux.c b/tools/libxc/xc_dom_compat_linux.c index 0e78842..2183a3b 100644 --- a/tools/libxc/xc_dom_compat_linux.c +++ b/tools/libxc/xc_dom_compat_linux.c @@ -62,6 +62,8 @@ static int xc_linux_build_internal(struct xc_dom_image *dom, goto out; if ( (rc = xc_dom_boot_image(dom)) != 0 ) goto out; + if ( (rc = xc_dom_gnttab_init(dom)) != 0) + goto out; *console_mfn = xc_dom_p2m_host(dom, dom->console_pfn); *store_mfn = xc_dom_p2m_host(dom, dom->xenstore_pfn); diff --git a/tools/libxc/xc_domain_restore.c b/tools/libxc/xc_domain_restore.c index 3fda6f8..8bee684 100644 --- a/tools/libxc/xc_domain_restore.c +++ b/tools/libxc/xc_domain_restore.c @@ -1259,7 +1259,8 @@ static int apply_batch(xc_interface *xch, uint32_t dom, struct restore_ctx *ctx, int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, unsigned int store_evtchn, unsigned long *store_mfn, - unsigned int console_evtchn, unsigned long *console_mfn, + uint32_t store_domid, unsigned int console_evtchn, + unsigned long *console_mfn, uint32_t console_domid, unsigned int hvm, unsigned int pae, int superpages, int no_incr_generationid, unsigned long *vm_generationid_addr) @@ -2018,6 +2019,14 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, memcpy(ctx->live_p2m, ctx->p2m, dinfo->p2m_size * sizeof(xen_pfn_t)); munmap(ctx->live_p2m, P2M_FL_ENTRIES * PAGE_SIZE); + rc = xc_dom_gnttab_seed(xch, dom, *console_mfn, *store_mfn, + console_domid, store_domid); + if (rc != 0) + { + ERROR("error seeding grant table"); + goto out; + } + DPRINTF("Domain ready to be built.\n"); rc = 0; goto out; @@ -2076,6 +2085,14 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, goto out; } + rc = xc_dom_gnttab_hvm_seed(xch, dom, *console_mfn, *store_mfn, + console_domid, store_domid); + if (rc != 0) + { + ERROR("error seeding grant table"); + goto out; + } + /* HVM success! */ rc = 0; diff --git a/tools/libxc/xenguest.h b/tools/libxc/xenguest.h index 6026370..3bd5549 100644 --- a/tools/libxc/xenguest.h +++ b/tools/libxc/xenguest.h @@ -79,7 +79,8 @@ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t max_iter */ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, unsigned int store_evtchn, unsigned long *store_mfn, - unsigned int console_evtchn, unsigned long *console_mfn, + uint32_t store_domid, unsigned int console_evtchn, + unsigned long *console_mfn, uint32_t console_domid, unsigned int hvm, unsigned int pae, int superpages, int no_incr_generationid, unsigned long *vm_generationid_addr); diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c index 5e39595..04db22f 100644 --- a/tools/libxl/libxl_dom.c +++ b/tools/libxl/libxl_dom.c @@ -233,7 +233,9 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid, dom->flags = flags; dom->console_evtchn = state->console_port; + dom->console_domid = state->console_domid; dom->xenstore_evtchn = state->store_port; + dom->xenstore_domid = state->store_domid; if ( (ret = xc_dom_boot_xen_init(dom, ctx->xch, domid)) != 0 ) { LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "xc_dom_boot_xen_init failed"); @@ -259,6 +261,10 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid, LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "xc_dom_boot_image failed"); goto out; } + if ( (ret = xc_dom_gnttab_init(dom)) != 0 ) { + LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "xc_dom_gnttab_init failed"); + goto out; + } state->console_mfn = xc_dom_p2m_host(dom, dom->console_pfn); state->store_mfn = xc_dom_p2m_host(dom, dom->xenstore_pfn); @@ -272,7 +278,8 @@ out: static int hvm_build_set_params(xc_interface *handle, uint32_t domid, libxl_domain_build_info *info, int store_evtchn, unsigned long *store_mfn, - int console_evtchn, unsigned long *console_mfn) + int console_evtchn, unsigned long *console_mfn, + uint32_t store_domid, uint32_t console_domid) { struct hvm_info_table *va_hvm; uint8_t *va_map, sum; @@ -305,6 +312,8 @@ static int hvm_build_set_params(xc_interface *handle, uint32_t domid, xc_set_hvm_param(handle, domid, HVM_PARAM_NESTEDHVM, info->u.hvm.nested_hvm); xc_set_hvm_param(handle, domid, HVM_PARAM_STORE_EVTCHN, store_evtchn); xc_set_hvm_param(handle, domid, HVM_PARAM_CONSOLE_EVTCHN, console_evtchn); + + xc_dom_gnttab_hvm_seed(handle, domid, *console_mfn, *store_mfn, console_domid, store_domid); return 0; } @@ -358,7 +367,9 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid, goto out; } ret = hvm_build_set_params(ctx->xch, domid, info, state->store_port, - &state->store_mfn, state->console_port, &state->console_mfn); + &state->store_mfn, state->console_port, + &state->console_mfn, state->store_domid, + state->console_domid); if (ret) { LIBXL__LOG_ERRNOVAL(ctx, LIBXL__LOG_ERROR, ret, "hvm build set params failed"); goto out; @@ -396,7 +407,8 @@ int libxl__domain_restore_common(libxl__gc *gc, uint32_t domid, } rc = xc_domain_restore(ctx->xch, fd, domid, state->store_port, &state->store_mfn, - state->console_port, &state->console_mfn, + state->store_domid, state->console_port, + &state->console_mfn, state->console_domid, hvm, pae, superpages, no_incr_generationid, &state->vm_generationid_addr); if ( rc ) { diff --git a/tools/xcutils/xc_restore.c b/tools/xcutils/xc_restore.c index 63d53a8..e41a133 100644 --- a/tools/xcutils/xc_restore.c +++ b/tools/xcutils/xc_restore.c @@ -45,8 +45,8 @@ main(int argc, char **argv) else superpages = !!hvm; - ret = xc_domain_restore(xch, io_fd, domid, store_evtchn, &store_mfn, - console_evtchn, &console_mfn, hvm, pae, superpages, + ret = xc_domain_restore(xch, io_fd, domid, store_evtchn, &store_mfn, 0, + console_evtchn, &console_mfn, 0, hvm, pae, superpages, 0, NULL); if ( ret == 0 ) -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 07/18] mini-os: avoid crash if no console is provided
Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- extras/mini-os/console/xencons_ring.c | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/extras/mini-os/console/xencons_ring.c b/extras/mini-os/console/xencons_ring.c index 22fd618..14a8bd1 100644 --- a/extras/mini-os/console/xencons_ring.c +++ b/extras/mini-os/console/xencons_ring.c @@ -25,7 +25,10 @@ static inline void notify_daemon(struct consfront_dev *dev) static inline struct xencons_interface *xencons_interface(void) { - return mfn_to_virt(start_info.console.domU.mfn); + if (start_info.console.domU.evtchn) + return mfn_to_virt(start_info.console.domU.mfn); + else + return NULL; } int xencons_ring_send_no_notify(struct consfront_dev *dev, const char *data, unsigned len) @@ -38,6 +41,8 @@ int xencons_ring_send_no_notify(struct consfront_dev *dev, const char *data, uns intf = xencons_interface(); else intf = dev->ring; + if (!intf) + return sent; cons = intf->out_cons; prod = intf->out_prod; -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 08/18] mini-os: avoid crash if no xenstore is provided
Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- extras/mini-os/xenbus/xenbus.c | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-) diff --git a/extras/mini-os/xenbus/xenbus.c b/extras/mini-os/xenbus/xenbus.c index a8081fd..e475e2c 100644 --- a/extras/mini-os/xenbus/xenbus.c +++ b/extras/mini-os/xenbus/xenbus.c @@ -328,6 +328,10 @@ static int allocate_xenbus_id(void) void init_xenbus(void) { int err; + if (!start_info.store_evtchn) { + printk("Skipping initialization of xenbus\n"); + return; + } printk("Initialising xenbus\n"); DEBUG("init_xenbus called.\n"); xenstore_buf = mfn_to_virt(start_info.store_mfn); @@ -435,6 +439,11 @@ xenbus_msg_reply(int type, DEFINE_WAIT(w); struct xsd_sockmsg *rep; + if (!xenstore_buf) { + printk("xenbus_msg_reply called but no xenstore!\n"); + return NULL; + } + id = allocate_xenbus_id(); add_waiter(w, req_info[id].waitq); -- 1.7.7.5
From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Changes the minios evtchn implementation to use a list instead of an array. This allows it to grow as necessary to support any number of ports. Unfortunately, it''s still limited by NR_EVS in events.c. Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- extras/mini-os/include/lib.h | 16 +++--- tools/libxc/xc_minios.c | 139 ++++++++++++++++++++++-------------------- 2 files changed, 81 insertions(+), 74 deletions(-) diff --git a/extras/mini-os/include/lib.h b/extras/mini-os/include/lib.h index bd3eeaf..12070c3 100644 --- a/extras/mini-os/include/lib.h +++ b/extras/mini-os/include/lib.h @@ -53,6 +53,7 @@ #include <xen/xen.h> #include <xen/event_channel.h> #include "gntmap.h" +#include "list.h" #ifdef HAVE_LIBC #include <stdio.h> @@ -143,7 +144,12 @@ enum fd_type { FTYPE_SAVEFILE, }; -#define MAX_EVTCHN_PORTS 16 +struct evtchn_port_info { + struct minios_list_head list; + evtchn_port_t port; + unsigned long pending; + int bound; +}; extern struct file { enum fd_type type; @@ -158,13 +164,7 @@ extern struct file { off_t offset; } file; struct { - /* To each event channel FD is associated a series of ports which - * wakes select for this FD. */ - struct { - evtchn_port_t port; - unsigned long pending; - int bound; - } ports[MAX_EVTCHN_PORTS]; + struct minios_list_head ports; } evtchn; struct gntmap gntmap; struct { diff --git a/tools/libxc/xc_minios.c b/tools/libxc/xc_minios.c index 8bbfd18..29cce63 100644 --- a/tools/libxc/xc_minios.c +++ b/tools/libxc/xc_minios.c @@ -210,15 +210,34 @@ static struct xc_osdep_ops minios_privcmd_ops = { }, }; + +/* XXX Note: This is not threadsafe */ +static struct evtchn_port_info* port_alloc(int fd) { + struct evtchn_port_info *port_info; + port_info = malloc(sizeof(struct evtchn_port_info)); + if (port_info == NULL) + return NULL; + port_info->pending = 0; + port_info->port = -1; + port_info->bound = 0; + + minios_list_add(&port_info->list, &files[fd].evtchn.ports); + return port_info; +} + +static void port_dealloc(struct evtchn_port_info *port_info) { + if (port_info->bound) + unbind_evtchn(port_info->port); + minios_list_del(&port_info->list); + free(port_info); +} + static xc_osdep_handle minios_evtchn_open(xc_evtchn *xce) { - int fd = alloc_fd(FTYPE_EVTCHN), i; + int fd = alloc_fd(FTYPE_EVTCHN); if ( fd == -1 ) return XC_OSDEP_OPEN_ERROR; - for (i = 0; i < MAX_EVTCHN_PORTS; i++) { - files[fd].evtchn.ports[i].port = -1; - files[fd].evtchn.ports[i].bound = 0; - } + MINIOS_INIT_LIST_HEAD(&files[fd].evtchn.ports); printf("evtchn_open() -> %d\n", fd); return (xc_osdep_handle)fd; } @@ -231,10 +250,10 @@ static int minios_evtchn_close(xc_evtchn *xce, xc_osdep_handle h) void minios_evtchn_close_fd(int fd) { - int i; - for (i = 0; i < MAX_EVTCHN_PORTS; i++) - if (files[fd].evtchn.ports[i].bound) - unbind_evtchn(files[fd].evtchn.ports[i].port); + struct evtchn_port_info *port_info, *tmp; + minios_list_for_each_entry_safe(port_info, tmp, &files[fd].evtchn.ports, list) + port_dealloc(port_info); + files[fd].type = FTYPE_NONE; } @@ -256,35 +275,21 @@ static int minios_evtchn_notify(xc_evtchn *xce, xc_osdep_handle h, evtchn_port_t return ret; } -/* XXX Note: This is not threadsafe */ -static int port_alloc(int fd) { - int i; - for (i= 0; i < MAX_EVTCHN_PORTS; i++) - if (files[fd].evtchn.ports[i].port == -1) - break; - if (i == MAX_EVTCHN_PORTS) { - printf("Too many ports in xc handle\n"); - errno = EMFILE; - return -1; - } - files[fd].evtchn.ports[i].pending = 0; - return i; -} - static void evtchn_handler(evtchn_port_t port, struct pt_regs *regs, void *data) { int fd = (int)(intptr_t)data; - int i; + struct evtchn_port_info *port_info; assert(files[fd].type == FTYPE_EVTCHN); mask_evtchn(port); - for (i= 0; i < MAX_EVTCHN_PORTS; i++) - if (files[fd].evtchn.ports[i].port == port) - break; - if (i == MAX_EVTCHN_PORTS) { - printk("Unknown port for handle %d\n", fd); - return; + minios_list_for_each_entry(port_info, &files[fd].evtchn.ports, list) { + if (port_info->port == port) + goto found; } - files[fd].evtchn.ports[i].pending = 1; + printk("Unknown port for handle %d\n", fd); + return; + + found: + port_info->pending = 1; files[fd].read = 1; wake_up(&event_queue); } @@ -292,12 +297,13 @@ static void evtchn_handler(evtchn_port_t port, struct pt_regs *regs, void *data) static evtchn_port_or_error_t minios_evtchn_bind_unbound_port(xc_evtchn *xce, xc_osdep_handle h, int domid) { int fd = (int)h; - int ret, i; + struct evtchn_port_info *port_info; + int ret; evtchn_port_t port; assert(get_current() == main_thread); - i = port_alloc(fd); - if (i == -1) + port_info = port_alloc(fd); + if (port_info == NULL) return -1; printf("xc_evtchn_bind_unbound_port(%d)", domid); @@ -305,11 +311,12 @@ static evtchn_port_or_error_t minios_evtchn_bind_unbound_port(xc_evtchn *xce, xc printf(" = %d\n", ret); if (ret < 0) { + port_dealloc(port_info); errno = -ret; return -1; } - files[fd].evtchn.ports[i].bound = 1; - files[fd].evtchn.ports[i].port = port; + port_info->bound = 1; + port_info->port = port; unmask_evtchn(port); return port; } @@ -318,12 +325,13 @@ static evtchn_port_or_error_t minios_evtchn_bind_interdomain(xc_evtchn *xce, xc_ evtchn_port_t remote_port) { int fd = (int)h; + struct evtchn_port_info *port_info; evtchn_port_t local_port; - int ret, i; + int ret; assert(get_current() == main_thread); - i = port_alloc(fd); - if (i == -1) + port_info = port_alloc(fd); + if (port_info == NULL) return -1; printf("xc_evtchn_bind_interdomain(%d, %"PRId32")", domid, remote_port); @@ -331,11 +339,12 @@ static evtchn_port_or_error_t minios_evtchn_bind_interdomain(xc_evtchn *xce, xc_ printf(" = %d\n", ret); if (ret < 0) { + port_dealloc(port_info); errno = -ret; return -1; } - files[fd].evtchn.ports[i].bound = 1; - files[fd].evtchn.ports[i].port = local_port; + port_info->bound = 1; + port_info->port = local_port; unmask_evtchn(local_port); return local_port; } @@ -343,42 +352,40 @@ static evtchn_port_or_error_t minios_evtchn_bind_interdomain(xc_evtchn *xce, xc_ static int minios_evtchn_unbind(xc_evtchn *xce, xc_osdep_handle h, evtchn_port_t port) { int fd = (int)h; - int i; - for (i = 0; i < MAX_EVTCHN_PORTS; i++) - if (files[fd].evtchn.ports[i].port == port) { - files[fd].evtchn.ports[i].port = -1; - break; - } - if (i == MAX_EVTCHN_PORTS) { - printf("Warning: couldn''t find port %"PRId32" for xc handle %x\n", port, fd); - errno = -EINVAL; - return -1; + struct evtchn_port_info *port_info; + + minios_list_for_each_entry(port_info, &files[fd].evtchn.ports, list) { + if (port_info->port == port) { + port_dealloc(port_info); + return 0; + } } - files[fd].evtchn.ports[i].bound = 0; - unbind_evtchn(port); - return 0; + printf("Warning: couldn''t find port %"PRId32" for xc handle %x\n", port, fd); + errno = -EINVAL; + return -1; } static evtchn_port_or_error_t minios_evtchn_bind_virq(xc_evtchn *xce, xc_osdep_handle h, unsigned int virq) { int fd = (int)h; + struct evtchn_port_info *port_info; evtchn_port_t port; - int i; assert(get_current() == main_thread); - i = port_alloc(fd); - if (i == -1) + port_info = port_alloc(fd); + if (port_info == NULL) return -1; printf("xc_evtchn_bind_virq(%d)", virq); port = bind_virq(virq, evtchn_handler, (void*)(intptr_t)fd); if (port < 0) { + port_dealloc(port_info); errno = -port; return -1; } - files[fd].evtchn.ports[i].bound = 1; - files[fd].evtchn.ports[i].port = port; + port_info->bound = 1; + port_info->port = port; unmask_evtchn(port); return port; } @@ -386,18 +393,18 @@ static evtchn_port_or_error_t minios_evtchn_bind_virq(xc_evtchn *xce, xc_osdep_h static evtchn_port_or_error_t minios_evtchn_pending(xc_evtchn *xce, xc_osdep_handle h) { int fd = (int)h; - int i; + struct evtchn_port_info *port_info; unsigned long flags; evtchn_port_t ret = -1; local_irq_save(flags); files[fd].read = 0; - for (i = 0; i < MAX_EVTCHN_PORTS; i++) { - evtchn_port_t port = files[fd].evtchn.ports[i].port; - if (port != -1 && files[fd].evtchn.ports[i].pending) { + + minios_list_for_each_entry(port_info, &files[fd].evtchn.ports, list) { + if (port_info->port != -1 && port_info->pending) { if (ret == -1) { - ret = port; - files[fd].evtchn.ports[i].pending = 0; + ret = port_info->port; + port_info->pending = 0; } else { files[fd].read = 1; break; -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 10/18] xenstored: use grant references instead of map_foreign_range
From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> make xenstored use grantref rather than map_foreign_range (which can only be used by privileged domains) This patch modifies the xenstore daemon to use xc_gnttab_map_grant_ref instead of xc_map_foreign_range where available. Previous versions of this patch have been sent to xen-devel. See http://lists.xensource.com/archives/html/xen-devel/2008-07/msg00610.html http://lists.xensource.com/archives/html/xen-devel/2009-03/msg01492.html Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/xenstored_domain.c | 45 ++++++++++++++++++++++++++++++++----- 1 files changed, 39 insertions(+), 6 deletions(-) diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c index 443af82..0b8353b 100644 --- a/tools/xenstore/xenstored_domain.c +++ b/tools/xenstore/xenstored_domain.c @@ -32,8 +32,10 @@ #include "xenstored_watch.h" #include <xenctrl.h> +#include <xen/grant_table.h> static xc_interface **xc_handle; +static xc_gnttab **xcg_handle; static evtchn_port_t virq_port; xc_evtchn *xce_handle = NULL; @@ -174,8 +176,12 @@ static int destroy_domain(void *_domain) eprintf("> Unbinding port %i failed!\n", domain->port); } - if (domain->interface) - munmap(domain->interface, getpagesize()); + if (domain->interface) { + if (*xcg_handle >= 0 && domain->domid != 0) + xc_gnttab_munmap(*xcg_handle, domain->interface, 1); + else + munmap(domain->interface, getpagesize()); + } fire_watches(NULL, "@releaseDomain", false); @@ -344,9 +350,17 @@ void do_introduce(struct connection *conn, struct buffered_data *in) domain = find_domain_by_domid(domid); if (domain == NULL) { - interface = xc_map_foreign_range( - *xc_handle, domid, - getpagesize(), PROT_READ|PROT_WRITE, mfn); + if (*xcg_handle >= 0) { + /* this is the preferred method */ + interface = xc_gnttab_map_grant_ref( + *xcg_handle, domid, + GNTTAB_RESERVED_XENSTORE, + PROT_READ|PROT_WRITE); + } else { + interface = xc_map_foreign_range( + *xc_handle, domid, + getpagesize(), PROT_READ|PROT_WRITE, mfn); + } if (!interface) { send_error(conn, errno); return; @@ -354,7 +368,10 @@ void do_introduce(struct connection *conn, struct buffered_data *in) /* Hang domain off "in" until we''re finished. */ domain = new_domain(in, domid, port); if (!domain) { - munmap(interface, getpagesize()); + if (*xcg_handle >= 0) + xc_gnttab_munmap(*xcg_handle, interface, 1); + else + munmap(interface, getpagesize()); send_error(conn, errno); return; } @@ -552,6 +569,12 @@ static int close_xc_handle(void *_handle) return 0; } +static int close_xcg_handle(void *_handle) +{ + xc_gnttab_close(*(xc_gnttab **)_handle); + return 0; +} + /* Returns the implicit path of a connection (only domains have this) */ const char *get_implicit_path(const struct connection *conn) { @@ -603,6 +626,16 @@ void domain_init(void) talloc_set_destructor(xc_handle, close_xc_handle); + xcg_handle = talloc(talloc_autofree_context(), xc_gnttab*); + if (!xcg_handle) + barf_perror("Failed to allocate domain gnttab handle"); + + *xcg_handle = xc_gnttab_open(NULL, 0); + if (*xcg_handle < 0) + xprintf("WARNING: Failed to open connection to gnttab\n"); + else + talloc_set_destructor(xcg_handle, close_xcg_handle); + xce_handle = xc_evtchn_open(NULL, 0); if (xce_handle == NULL) -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 11/18] xenstored: add NO_SOCKETS compilation option
From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> option for compiling xenstored without unix sockets to support running on mini-OS Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/xenstored_core.c | 22 +++++++++++++++++++++- tools/xenstore/xs.c | 2 ++ tools/xenstore/xs_lib.c | 4 ++++ 3 files changed, 27 insertions(+), 1 deletions(-) diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c index 9e6c2c7..631bfe4 100644 --- a/tools/xenstore/xenstored_core.c +++ b/tools/xenstore/xenstored_core.c @@ -19,9 +19,11 @@ #include <sys/types.h> #include <sys/stat.h> -#include <sys/socket.h> #include <sys/select.h> +#ifndef NO_SOCKETS +#include <sys/socket.h> #include <sys/un.h> +#endif #include <sys/time.h> #include <time.h> #include <unistd.h> @@ -320,8 +322,10 @@ static int initialize_set(fd_set *inset, fd_set *outset, int sock, int ro_sock, FD_ZERO(inset); FD_ZERO(outset); +#ifndef NO_SOCKETS set_fd(sock, inset, &max); set_fd(ro_sock, inset, &max); +#endif set_fd(reopen_log_pipe[0], inset, &max); if (xce_handle != NULL) @@ -343,12 +347,14 @@ static int initialize_set(fd_set *inset, fd_set *outset, int sock, int ro_sock, return max; } +#ifndef NO_SOCKETS static int destroy_fd(void *_fd) { int *fd = _fd; close(*fd); return 0; } +#endif /* Is child a subnode of parent, or equal? */ bool is_child(const char *child, const char *parent) @@ -1352,6 +1358,7 @@ struct connection *new_connection(connwritefn_t *write, connreadfn_t *read) return new; } +#ifndef NO_SOCKETS static int writefd(struct connection *conn, const void *data, unsigned int len) { int rc; @@ -1406,6 +1413,7 @@ static void accept_connection(int sock, bool canwrite) } else close(fd); } +#endif #define TDB_FLAGS 0 @@ -1753,7 +1761,11 @@ extern void dump_conn(struct connection *conn); int main(int argc, char *argv[]) { int opt, *sock, *ro_sock, max; +#ifdef NO_SOCKETS + int minus_one = -1; +#else struct sockaddr_un addr; +#endif fd_set inset, outset; bool dofork = true; bool outputpid = false; @@ -1837,6 +1849,9 @@ int main(int argc, char *argv[]) if (!dofork) talloc_enable_leak_report_full(); +#ifdef NO_SOCKETS + sock = ro_sock = &minus_one; +#else /* Create sockets for them to listen to. */ sock = talloc(talloc_autofree_context(), int); *sock = socket(PF_UNIX, SOCK_STREAM, 0); @@ -1848,10 +1863,12 @@ int main(int argc, char *argv[]) barf_perror("Could not create socket"); talloc_set_destructor(sock, destroy_fd); talloc_set_destructor(ro_sock, destroy_fd); +#endif /* Don''t kill us with SIGPIPE. */ signal(SIGPIPE, SIG_IGN); +#ifndef NO_SOCKETS /* FIXME: Be more sophisticated, don''t mug running daemon. */ unlink(xs_daemon_socket()); unlink(xs_daemon_socket_ro()); @@ -1871,6 +1888,7 @@ int main(int argc, char *argv[]) if (listen(*sock, 1) != 0 || listen(*ro_sock, 1) != 0) barf_perror("Could not listen on sockets"); +#endif if (pipe(reopen_log_pipe)) { barf_perror("pipe"); @@ -1931,11 +1949,13 @@ int main(int argc, char *argv[]) reopen_log(); } +#ifndef NO_SOCKETS if (FD_ISSET(*sock, &inset)) accept_connection(*sock, true); if (FD_ISSET(*ro_sock, &inset)) accept_connection(*ro_sock, false); +#endif if (evtchn_fd != -1 && FD_ISSET(evtchn_fd, &inset)) handle_event(); diff --git a/tools/xenstore/xs.c b/tools/xenstore/xs.c index 8e54fe0..119d945 100644 --- a/tools/xenstore/xs.c +++ b/tools/xenstore/xs.c @@ -271,10 +271,12 @@ struct xs_handle *xs_open(unsigned long flags) { struct xs_handle *xsh = NULL; +#ifndef NO_SOCKETS if (flags & XS_OPEN_READONLY) xsh = get_handle(xs_daemon_socket_ro()); else xsh = get_handle(xs_daemon_socket()); +#endif if (!xsh && !(flags & XS_OPEN_SOCKETONLY)) xsh = get_handle(xs_domain_dev()); diff --git a/tools/xenstore/xs_lib.c b/tools/xenstore/xs_lib.c index 03a9ee4..af3db6b 100644 --- a/tools/xenstore/xs_lib.c +++ b/tools/xenstore/xs_lib.c @@ -39,6 +39,7 @@ const char *xs_daemon_rundir(void) return (s ? s : "/var/run/xenstored"); } +#ifndef NO_SOCKETS static const char *xs_daemon_path(void) { static char buf[PATH_MAX]; @@ -50,6 +51,7 @@ static const char *xs_daemon_path(void) return NULL; return buf; } +#endif const char *xs_daemon_tdb(void) { @@ -58,6 +60,7 @@ const char *xs_daemon_tdb(void) return buf; } +#ifndef NO_SOCKETS const char *xs_daemon_socket(void) { return xs_daemon_path(); @@ -73,6 +76,7 @@ const char *xs_daemon_socket_ro(void) return NULL; return buf; } +#endif const char *xs_domain_dev(void) { -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 12/18] xenstored support for in-memory rather than FS based trivial DB (needed to run on mini-OS)
From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> tdb_copy (a xen modification to tdb?) should honor the TDB_INTERNAL flag for in-memory databases. TODO: leaks memory on error case Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/tdb.c | 36 ++++++++++++++++++++++++++++++++++++ 1 files changed, 36 insertions(+), 0 deletions(-) diff --git a/tools/xenstore/tdb.c b/tools/xenstore/tdb.c index 63205e1..639ce6e 100644 --- a/tools/xenstore/tdb.c +++ b/tools/xenstore/tdb.c @@ -2103,6 +2103,42 @@ TDB_CONTEXT *tdb_copy(TDB_CONTEXT *tdb, const char *outfile) int fd, saved_errno; TDB_CONTEXT *copy; + if (tdb->flags & TDB_INTERNAL) { + struct tdb_header *copydb; + + copy = talloc_zero(outfile, TDB_CONTEXT); + if (copy == NULL) { + errno = ENOMEM; + goto intfail; + } + memcpy(copy, tdb, sizeof(TDB_CONTEXT)); + + if (copy->name || copy->locked || copy->device || copy->inode) { + fprintf(stderr, "tdb_copy assumption(s) failed\n"); + goto intfail; + } + + copydb = talloc_zero_size(copy, copy->map_size); + if (copydb == NULL) { + errno = ENOMEM; + goto intfail; + } + memcpy(copydb, copy->map_ptr, copy->map_size); + copy->map_ptr = (char*) copydb; + + if (tdb_brlock(tdb, GLOBAL_LOCK, F_UNLCK, F_SETLKW, 0) == -1) + goto intfail; + + copy->next = tdbs; + tdbs = copy; + + + return copy; +intfail: + /* TODO (leaking memory is easier) */ + return NULL; + } + fd = open(outfile, O_TRUNC|O_CREAT|O_WRONLY, 0640); if (fd < 0) return NULL; -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 13/18] xenstored: support running in minios stubdom
A previous versions of this patch has been sent to xen-devel. See http://lists.xensource.com/archives/html/xen-devel/2009-03/msg01655.html Originally-by: Diego Ongaro <diego.ongaro@citrix.com> Originally-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- extras/mini-os/include/list.h | 6 ++-- extras/mini-os/main.c | 4 +++ stubdom/Makefile | 29 +++++++++++++++++-- tools/xenstore/Makefile | 9 +++++- tools/xenstore/tdb.c | 6 ++-- tools/xenstore/utils.h | 2 + tools/xenstore/xenstored_core.c | 47 ++++++++++++++++++++++++++++++- tools/xenstore/xenstored_domain.c | 7 +++++ tools/xenstore/xenstored_transaction.c | 2 + 9 files changed, 100 insertions(+), 12 deletions(-) diff --git a/extras/mini-os/include/list.h b/extras/mini-os/include/list.h index a60ae23..4e6a2ac 100644 --- a/extras/mini-os/include/list.h +++ b/extras/mini-os/include/list.h @@ -1,5 +1,5 @@ -#ifndef _LINUX_LIST_H -#define _LINUX_LIST_H +#ifndef _MINIOS_LIST_H +#define _MINIOS_LIST_H /* * Simple doubly linked list implementation. @@ -186,5 +186,5 @@ static __inline__ void minios_list_splice(struct minios_list_head *list, struct n = minios_list_entry(pos->member.next, typeof(*pos), member); \ &pos->member != (head); \ pos = n, n = minios_list_entry(n->member.next, typeof(*n), member)) -#endif /* _LINUX_LIST_H */ +#endif /* _MINIOS_LIST_H */ diff --git a/extras/mini-os/main.c b/extras/mini-os/main.c index b95b889..cd89849 100644 --- a/extras/mini-os/main.c +++ b/extras/mini-os/main.c @@ -60,6 +60,7 @@ static void call_main(void *p) * crashing. */ //sleep(1); +#ifndef CONFIG_XENSTORE #ifndef CONFIG_GRUB sparse((unsigned long) &__app_bss_start, &__app_bss_end - &__app_bss_start); #if defined(HAVE_LWIP) && !defined(CONFIG_QEMU) @@ -67,6 +68,7 @@ static void call_main(void *p) #endif #endif create_thread("pcifront", pcifront_watches, NULL); +#endif #ifdef CONFIG_QEMU /* Fetch argc, argv from XenStore */ @@ -169,9 +171,11 @@ void _exit(int ret) close_all_files(); __libc_fini_array(); printk("main returned %d\n", ret); +#ifndef CONFIG_XENSTORE #ifdef HAVE_LWIP stop_networking(); #endif +#endif stop_kernel(); if (!ret) { /* No problem, just shutdown. */ diff --git a/stubdom/Makefile b/stubdom/Makefile index 3705059..e0a90a9 100644 --- a/stubdom/Makefile +++ b/stubdom/Makefile @@ -74,14 +74,14 @@ TARGET_CPPFLAGS += -I$(XEN_ROOT)/xen/include TARGET_LDFLAGS += -nostdlib -L$(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf/lib -TARGETS=ioemu c caml grub +TARGETS=ioemu c caml grub xenstore CROSS_MAKE := $(MAKE) DESTDIR .PHONY: all all: build ifeq ($(STUBDOM_SUPPORTED),1) -build: genpath ioemu-stubdom c-stubdom pv-grub +build: genpath ioemu-stubdom c-stubdom pv-grub xenstore-stubdom else build: genpath endif @@ -262,6 +262,11 @@ mk-headers-$(XEN_TARGET_ARCH): ioemu/linkfarm.stamp ln -sf $(XEN_ROOT)/tools/libxc/$(XEN_TARGET_ARCH)/*.c . && \ ln -sf $(XEN_ROOT)/tools/libxc/$(XEN_TARGET_ARCH)/*.h . && \ ln -sf $(XEN_ROOT)/tools/libxc/$(XEN_TARGET_ARCH)/Makefile . ) + mkdir -p xenstore + [ -h xenstore/Makefile ] || ( cd xenstore && \ + ln -sf $(XEN_ROOT)/tools/xenstore/*.c . && \ + ln -sf $(XEN_ROOT)/tools/xenstore/*.h . && \ + ln -sf $(XEN_ROOT)/tools/xenstore/Makefile . ) $(CROSS_MAKE) -C $(MINI_OS) links touch mk-headers-$(XEN_TARGET_ARCH) @@ -334,6 +339,14 @@ grub: grub-upstream $(CROSS_ROOT) mkdir -p grub-$(XEN_TARGET_ARCH) CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(CROSS_MAKE) -C $@ OBJ_DIR=$(CURDIR)/grub-$(XEN_TARGET_ARCH) +########## +# xenstore +########## + +.PHONY: xenstore +xenstore: $(CROSS_ROOT) + CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(CROSS_MAKE) -C $@ LWIPDIR=$(CURDIR)/lwip xenstored.a CONFIG_STUBDOM=y + ######## # minios ######## @@ -355,12 +368,16 @@ c-stubdom: mini-os-$(XEN_TARGET_ARCH)-c lwip-$(XEN_TARGET_ARCH) libxc c pv-grub: mini-os-$(XEN_TARGET_ARCH)-grub libxc grub DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="-DCONFIG_GRUB $(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" $(CROSS_MAKE) -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/grub-$(XEN_TARGET_ARCH)/main.a +.PHONY: xenstore-stubdom +xenstore-stubdom: mini-os-$(XEN_TARGET_ARCH)-xenstore libxc xenstore + DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="-DCONFIG_XENSTORE $(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" $(MAKE) -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< LWIPDIR=$(CURDIR)/lwip-$(XEN_TARGET_ARCH) APP_OBJS=$(CURDIR)/xenstore/xenstored.a + ######### # install ######### ifeq ($(STUBDOM_SUPPORTED),1) -install: genpath install-readme install-ioemu install-grub +install: genpath install-readme install-ioemu install-grub install-xenstore else install: genpath endif @@ -379,6 +396,10 @@ install-grub: pv-grub $(INSTALL_DIR) "$(DESTDIR)$(XENFIRMWAREDIR)" $(INSTALL_DATA) mini-os-$(XEN_TARGET_ARCH)-grub/mini-os.gz "$(DESTDIR)$(XENFIRMWAREDIR)/pv-grub-$(XEN_TARGET_ARCH).gz" +install-xenstore: xenstore-stubdom + $(INSTALL_DIR) "$(DESTDIR)/usr/lib/xen/boot" + $(INSTALL_PROG) mini-os-$(XEN_TARGET_ARCH)-xenstore/mini-os.gz "$(DESTDIR)/usr/lib/xen/boot/xenstore-stubdom.gz" + ####### # clean ####### @@ -390,12 +411,14 @@ clean: rm -fr mini-os-$(XEN_TARGET_ARCH)-c rm -fr mini-os-$(XEN_TARGET_ARCH)-caml rm -fr mini-os-$(XEN_TARGET_ARCH)-grub + rm -fr mini-os-$(XEN_TARGET_ARCH)-xenstore $(CROSS_MAKE) -C caml clean $(CROSS_MAKE) -C c clean rm -fr grub-$(XEN_TARGET_ARCH) rm -f $(STUBDOMPATH) [ ! -d libxc-$(XEN_TARGET_ARCH) ] || $(CROSS_MAKE) -C libxc-$(XEN_TARGET_ARCH) clean -[ ! -d ioemu ] || $(CROSS_MAKE) -C ioemu clean + -[ ! -d xenstore ] || $(CROSS_MAKE) -C xenstore clean # clean the cross-compilation result .PHONY: crossclean diff --git a/tools/xenstore/Makefile b/tools/xenstore/Makefile index 4facb62..3a061d6 100644 --- a/tools/xenstore/Makefile +++ b/tools/xenstore/Makefile @@ -28,6 +28,10 @@ endif ALL_TARGETS = libxenstore.so libxenstore.a clients xs_tdb_dump xenstored +ifdef CONFIG_STUBDOM +CFLAGS += -DNO_SOCKETS=1 -DNO_LOCAL_XENBUS=1 -DNO_SYSLOG=1 -DNO_REOPEN_LOG=1 +endif + .PHONY: all all: $(ALL_TARGETS) @@ -45,10 +49,13 @@ xenstored_probes.o: xenstored_solaris.o CFLAGS += -DHAVE_DTRACE=1 endif - + xenstored: $(XENSTORED_OBJS) $(CC) $(LDFLAGS) $^ $(LDLIBS_libxenctrl) $(SOCKET_LIBS) -o $@ $(APPEND_LDFLAGS) +xenstored.a: $(XENSTORED_OBJS) + $(AR) cr $@ $^ + $(CLIENTS): xenstore ln -f xenstore $@ diff --git a/tools/xenstore/tdb.c b/tools/xenstore/tdb.c index 639ce6e..75ffd2a 100644 --- a/tools/xenstore/tdb.c +++ b/tools/xenstore/tdb.c @@ -1334,7 +1334,7 @@ static int tdb_next_lock(TDB_CONTEXT *tdb, struct tdb_traverse_lock *tlock, /* Iterate through chain */ while( tlock->off) { - tdb_off current; + tdb_off mycurrent; if (rec_read(tdb, tlock->off, rec) == -1) goto fail; @@ -1352,10 +1352,10 @@ static int tdb_next_lock(TDB_CONTEXT *tdb, struct tdb_traverse_lock *tlock, } /* Try to clean dead ones from old traverses */ - current = tlock->off; + mycurrent = tlock->off; tlock->off = rec->next; if (!tdb->read_only && - do_delete(tdb, current, rec) != 0) + do_delete(tdb, mycurrent, rec) != 0) goto fail; } tdb_unlock(tdb, tlock->hash, F_WRLCK); diff --git a/tools/xenstore/utils.h b/tools/xenstore/utils.h index f378343..2effd17 100644 --- a/tools/xenstore/utils.h +++ b/tools/xenstore/utils.h @@ -19,7 +19,9 @@ static inline bool strends(const char *a, const char *b) return streq(a + strlen(a) - strlen(b), b); } +#ifndef ARRAY_SIZE #define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0])) +#endif void barf(const char *fmt, ...) __attribute__((noreturn)); void barf_perror(const char *fmt, ...) __attribute__((noreturn)); diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c index 631bfe4..e51f2ad 100644 --- a/tools/xenstore/xenstored_core.c +++ b/tools/xenstore/xenstored_core.c @@ -32,7 +32,9 @@ #include <stdio.h> #include <stdarg.h> #include <stdlib.h> +#ifndef NO_SYSLOG #include <syslog.h> +#endif #include <string.h> #include <errno.h> #include <dirent.h> @@ -61,13 +63,24 @@ LIST_HEAD(connections); static int tracefd = -1; static bool recovery = true; static bool remove_local = true; +#ifndef NO_REOPEN_LOG static int reopen_log_pipe[2]; +#endif static char *tracefile = NULL; static TDB_CONTEXT *tdb_ctx; static void corrupt(struct connection *conn, const char *fmt, ...); static void check_store(void); +#ifdef __MINIOS__ +#define lockf(...) (-ENOSYS) +#endif + +#ifdef NO_SYSLOG +#define openlog(...) ((void) 0) +#define syslog(...) ((void) 0) +#endif + #define log(...) \ do { \ char *s = talloc_asprintf(NULL, __VA_ARGS__); \ @@ -92,8 +105,10 @@ TDB_CONTEXT *tdb_context(struct connection *conn) bool replace_tdb(const char *newname, TDB_CONTEXT *newtdb) { +#ifndef __MINIOS__ if (rename(newname, xs_daemon_tdb()) != 0) return false; +#endif tdb_close(tdb_ctx); tdb_ctx = talloc_steal(talloc_autofree_context(), newtdb); return true; @@ -195,6 +210,7 @@ void trace_destroy(const void *data, const char *type) trace("DESTROY %s %p\n", type, data); } +#ifndef NO_REOPEN_LOG /** * Signal handler for SIGHUP, which requests that the trace log is reopened * (in the main loop). A single byte is written to reopen_log_pipe, to awaken @@ -222,7 +238,7 @@ static void reopen_log(void) trace("\n***\n"); } } - +#endif static bool write_messages(struct connection *conn) { @@ -326,7 +342,9 @@ static int initialize_set(fd_set *inset, fd_set *outset, int sock, int ro_sock, set_fd(sock, inset, &max); set_fd(ro_sock, inset, &max); #endif +#ifndef NO_REOPEN_LOG set_fd(reopen_log_pipe[0], inset, &max); +#endif if (xce_handle != NULL) set_fd(xc_evtchn_fd(xce_handle), inset, &max); @@ -1415,7 +1433,11 @@ static void accept_connection(int sock, bool canwrite) } #endif +#ifdef __MINIOS__ +#define TDB_FLAGS TDB_INTERNAL|TDB_NOLOCK +#else #define TDB_FLAGS 0 +#endif /* We create initial nodes manually. */ static void manual_node(const char *name, const char *child) @@ -1440,7 +1462,11 @@ static void setup_structure(void) { char *tdbname; tdbname = talloc_strdup(talloc_autofree_context(), xs_daemon_tdb()); +#ifdef __MINIOS__ + tdb_ctx = NULL; +#else tdb_ctx = tdb_open(tdbname, 0, TDB_FLAGS, O_RDWR, 0); +#endif if (tdb_ctx) { /* XXX When we make xenstored able to restart, this will have @@ -1666,6 +1692,7 @@ static void corrupt(struct connection *conn, const char *fmt, ...) } +#ifndef __MINIOS__ static void write_pidfile(const char *pidfile) { char buf[100]; @@ -1712,7 +1739,7 @@ static void daemonize(void) /* Discard our parent''s old-fashioned umask prejudices. */ umask(0); } - +#endif static void usage(void) { @@ -1767,7 +1794,11 @@ int main(int argc, char *argv[]) struct sockaddr_un addr; #endif fd_set inset, outset; +#ifdef __MINIOS__ + bool dofork = false; +#else bool dofork = true; +#endif bool outputpid = false; bool no_domain_init = false; const char *pidfile = NULL; @@ -1821,8 +1852,11 @@ int main(int argc, char *argv[]) if (optind != argc) barf("%s: No arguments desired", argv[0]); +#ifndef NO_REOPEN_LOG reopen_log(); +#endif +#ifndef __MINIOS__ /* make sure xenstored directory exists */ if (mkdir(xs_daemon_rundir(), 0755)) { if (errno != EEXIST) { @@ -1844,6 +1878,7 @@ int main(int argc, char *argv[]) } if (pidfile) write_pidfile(pidfile); +#endif /* Talloc leak reports go to stderr, which is closed if we fork. */ if (!dofork) @@ -1890,9 +1925,11 @@ int main(int argc, char *argv[]) barf_perror("Could not listen on sockets"); #endif +#ifndef NO_REOPEN_LOG if (pipe(reopen_log_pipe)) { barf_perror("pipe"); } +#endif /* Setup the database */ setup_structure(); @@ -1921,7 +1958,9 @@ int main(int argc, char *argv[]) xprintf = trace; } +#ifndef NO_REOPEN_LOG signal(SIGHUP, trigger_reopen_log); +#endif if (xce_handle != NULL) evtchn_fd = xc_evtchn_fd(xce_handle); @@ -1929,8 +1968,10 @@ int main(int argc, char *argv[]) /* Get ready to listen to the tools. */ max = initialize_set(&inset, &outset, *sock, *ro_sock, &timeout); +#ifndef __MINIOS__ /* Tell the kernel we''re up and running. */ xenbus_notify_running(); +#endif /* Main loop. */ for (;;) { @@ -1942,12 +1983,14 @@ int main(int argc, char *argv[]) barf_perror("Select failed"); } +#ifndef NO_REOPEN_LOG if (FD_ISSET(reopen_log_pipe[0], &inset)) { char c; if (read(reopen_log_pipe[0], &c, 1) != 1) barf_perror("read failed"); reopen_log(); } +#endif #ifndef NO_SOCKETS if (FD_ISSET(*sock, &inset)) diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c index 0b8353b..811ae3c 100644 --- a/tools/xenstore/xenstored_domain.c +++ b/tools/xenstore/xenstored_domain.c @@ -588,6 +588,12 @@ void restore_existing_connections(void) { } +#ifdef __MINIOS__ +static inline int dom0_init(void) +{ + return 0; +} +#else static int dom0_init(void) { evtchn_port_t port; @@ -611,6 +617,7 @@ static int dom0_init(void) return 0; } +#endif void domain_init(void) { diff --git a/tools/xenstore/xenstored_transaction.c b/tools/xenstore/xenstored_transaction.c index 380c691..c59acfb 100644 --- a/tools/xenstore/xenstored_transaction.c +++ b/tools/xenstore/xenstored_transaction.c @@ -120,7 +120,9 @@ static int destroy_transaction(void *_transaction) trace_destroy(trans, "transaction"); if (trans->tdb) tdb_close(trans->tdb); +#ifndef __MINIOS__ unlink(trans->tdb_name); +#endif return 0; } -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 14/18] xenstored: always use xc_gnttab_munmap in stubdom
Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/xenstored_domain.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c index 811ae3c..aca2149 100644 --- a/tools/xenstore/xenstored_domain.c +++ b/tools/xenstore/xenstored_domain.c @@ -177,10 +177,14 @@ static int destroy_domain(void *_domain) } if (domain->interface) { +#ifdef __MINIOS__ + xc_gnttab_munmap(*xcg_handle, domain->interface, 1); +#else if (*xcg_handle >= 0 && domain->domid != 0) xc_gnttab_munmap(*xcg_handle, domain->interface, 1); else munmap(domain->interface, getpagesize()); +#endif } fire_watches(NULL, "@releaseDomain", false); -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 15/18] xenstored: add --event parameter for bootstrapping
When xenstored is run in a minios domain, it needs a bootstrap connection to dom0 so that additional domain introduce messages can be sent to it. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/xenstored_core.c | 5 +++++ tools/xenstore/xenstored_core.h | 1 + tools/xenstore/xenstored_domain.c | 13 ++++++++++++- 3 files changed, 18 insertions(+), 1 deletions(-) diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c index e51f2ad..4ec63f1 100644 --- a/tools/xenstore/xenstored_core.c +++ b/tools/xenstore/xenstored_core.c @@ -1771,6 +1771,7 @@ static struct option options[] = { { "no-domain-init", 0, NULL, ''D'' }, { "entry-nb", 1, NULL, ''E'' }, { "pid-file", 1, NULL, ''F'' }, + { "event", 1, NULL, ''e'' }, { "help", 0, NULL, ''H'' }, { "no-fork", 0, NULL, ''N'' }, { "output-pid", 0, NULL, ''P'' }, @@ -1784,6 +1785,7 @@ static struct option options[] = { { NULL, 0, NULL, 0 } }; extern void dump_conn(struct connection *conn); +int dom0_event = 0; int main(int argc, char *argv[]) { @@ -1847,6 +1849,9 @@ int main(int argc, char *argv[]) case ''W'': quota_nb_watch_per_domain = strtol(optarg, NULL, 10); break; + case ''e'': + dom0_event = strtol(optarg, NULL, 10); + break; } } if (optind != argc) diff --git a/tools/xenstore/xenstored_core.h b/tools/xenstore/xenstored_core.h index c487089..d3040ba 100644 --- a/tools/xenstore/xenstored_core.h +++ b/tools/xenstore/xenstored_core.h @@ -168,6 +168,7 @@ void trace(const char *fmt, ...); void dtrace_io(const struct connection *conn, const struct buffered_data *data, int out); extern int event_fd; +extern int dom0_event; /* Map the kernel''s xenstore page. */ void *xenbus_map(void); diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c index aca2149..648eb1d 100644 --- a/tools/xenstore/xenstored_domain.c +++ b/tools/xenstore/xenstored_domain.c @@ -593,8 +593,19 @@ void restore_existing_connections(void) } #ifdef __MINIOS__ -static inline int dom0_init(void) +static int dom0_init(void) { + struct domain *domain; + int domid = 0; + evtchn_port_t port = dom0_event; + + domain = new_domain(NULL, domid, port); + domain->interface = xc_gnttab_map_grant_ref(*xcg_handle, domid, + GNTTAB_RESERVED_XENSTORE, PROT_READ|PROT_WRITE); + talloc_steal(domain->conn, domain); + + xc_evtchn_notify(xce_handle, domain->port); + return 0; } #else -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:35 UTC
[PATCH 16/18] xenstored: use domain_is_unprivileged instead of checking conn->id
This centralizes all the permission checking for privileged domains in preparation for allowing domains other than dom0 to be privileged. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/xenstored_core.c | 6 +++--- tools/xenstore/xenstored_domain.c | 8 ++++---- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c index 4ec63f1..eea5fd6 100644 --- a/tools/xenstore/xenstored_core.c +++ b/tools/xenstore/xenstored_core.c @@ -488,7 +488,7 @@ static enum xs_perm_type perm_for_conn(struct connection *conn, mask &= ~XS_PERM_WRITE; /* Owners and tools get it all... */ - if (!conn->id || perms[0].id == conn->id + if (!domain_is_unprivileged(conn) || perms[0].id == conn->id || (conn->target && perms[0].id == conn->target->id)) return (XS_PERM_READ|XS_PERM_WRITE|XS_PERM_OWNER) & mask; @@ -826,11 +826,11 @@ static struct node *construct_node(struct connection *conn, const char *name) node->tdb = tdb_context(conn); node->name = talloc_strdup(node, name); - /* Inherit permissions, except domains own what they create */ + /* Inherit permissions, except unprivileged domains own what they create */ node->num_perms = parent->num_perms; node->perms = talloc_memdup(node, parent->perms, node->num_perms * sizeof(node->perms[0])); - if (conn && conn->id) + if (domain_is_unprivileged(conn)) node->perms[0].id = conn->id; /* No children, no data */ diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c index 648eb1d..5f4a09e 100644 --- a/tools/xenstore/xenstored_domain.c +++ b/tools/xenstore/xenstored_domain.c @@ -336,7 +336,7 @@ void do_introduce(struct connection *conn, struct buffered_data *in) return; } - if (conn->id != 0 || !conn->can_write) { + if (domain_is_unprivileged(conn) || !conn->can_write) { send_error(conn, EACCES); return; } @@ -413,7 +413,7 @@ void do_set_target(struct connection *conn, struct buffered_data *in) return; } - if (conn->id != 0 || !conn->can_write) { + if (domain_is_unprivileged(conn) || !conn->can_write) { send_error(conn, EACCES); return; } @@ -465,7 +465,7 @@ void do_release(struct connection *conn, const char *domid_str) return; } - if (conn->id != 0) { + if (domain_is_unprivileged(conn)) { send_error(conn, EACCES); return; } @@ -502,7 +502,7 @@ void do_resume(struct connection *conn, const char *domid_str) return; } - if (conn->id != 0) { + if (domain_is_unprivileged(conn)) { send_error(conn, EACCES); return; } -- 1.7.7.5
This parameter identifies an alternative service domain which has superuser access to the xenstore database, which is currently required to set up a new domain''s xenstore entries. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/xenstore/xenstored_core.c | 5 +++++ tools/xenstore/xenstored_core.h | 1 + tools/xenstore/xenstored_domain.c | 2 +- 3 files changed, 7 insertions(+), 1 deletions(-) diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c index eea5fd6..9d087de 100644 --- a/tools/xenstore/xenstored_core.c +++ b/tools/xenstore/xenstored_core.c @@ -1774,6 +1774,7 @@ static struct option options[] = { { "event", 1, NULL, ''e'' }, { "help", 0, NULL, ''H'' }, { "no-fork", 0, NULL, ''N'' }, + { "priv-domid", 1, NULL, ''p'' }, { "output-pid", 0, NULL, ''P'' }, { "entry-size", 1, NULL, ''S'' }, { "trace-file", 1, NULL, ''T'' }, @@ -1786,6 +1787,7 @@ static struct option options[] = { extern void dump_conn(struct connection *conn); int dom0_event = 0; +int priv_domid = 0; int main(int argc, char *argv[]) { @@ -1852,6 +1854,9 @@ int main(int argc, char *argv[]) case ''e'': dom0_event = strtol(optarg, NULL, 10); break; + case ''p'': + priv_domid = strtol(optarg, NULL, 10); + break; } } if (optind != argc) diff --git a/tools/xenstore/xenstored_core.h b/tools/xenstore/xenstored_core.h index d3040ba..03e2e48 100644 --- a/tools/xenstore/xenstored_core.h +++ b/tools/xenstore/xenstored_core.h @@ -169,6 +169,7 @@ void dtrace_io(const struct connection *conn, const struct buffered_data *data, extern int event_fd; extern int dom0_event; +extern int priv_domid; /* Map the kernel''s xenstore page. */ void *xenbus_map(void); diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c index 5f4a09e..46bcf3e 100644 --- a/tools/xenstore/xenstored_domain.c +++ b/tools/xenstore/xenstored_domain.c @@ -241,7 +241,7 @@ bool domain_can_read(struct connection *conn) bool domain_is_unprivileged(struct connection *conn) { - return (conn && conn->domain && conn->domain->domid != 0); + return (conn && conn->domain && conn->domain->domid != 0 && conn->domain->domid != priv_domid); } bool domain_can_write(struct connection *conn) -- 1.7.7.5
Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- tools/include/xen-sys/Linux/xenbus_dev.h | 55 +++++++++++++++++ tools/xenstore/Makefile | 9 ++- tools/xenstore/init-xenstore-domain.c | 96 ++++++++++++++++++++++++++++++ 3 files changed, 158 insertions(+), 2 deletions(-) create mode 100644 tools/include/xen-sys/Linux/xenbus_dev.h create mode 100644 tools/xenstore/init-xenstore-domain.c diff --git a/tools/include/xen-sys/Linux/xenbus_dev.h b/tools/include/xen-sys/Linux/xenbus_dev.h new file mode 100644 index 0000000..b107be9 --- /dev/null +++ b/tools/include/xen-sys/Linux/xenbus_dev.h @@ -0,0 +1,55 @@ +/****************************************************************************** + * evtchn.h + * + * Interface to /dev/xen/xenbus_backend. + * + * Copyright (c) 2011 Bastian Blank <waldi@debian.org> + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License version 2 + * as published by the Free Software Foundation; or, when distributed + * separately from the Linux kernel or incorporated into other + * software packages, subject to the following license: + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this source file (the "Software"), to deal in the Software without + * restriction, including without limitation the rights to use, copy, modify, + * merge, publish, distribute, sublicense, and/or sell copies of the Software, + * and to permit persons to whom the Software is furnished to do so, subject to + * the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS + * IN THE SOFTWARE. + */ + +#ifndef __LINUX_XEN_XENBUS_DEV_H__ +#define __LINUX_XEN_XENBUS_DEV_H__ + +#include <linux/ioctl.h> + +#define IOCTL_XENBUS_BACKEND_EVTCHN \ + _IOC(_IOC_NONE, ''B'', 0, 0) + +#define IOCTL_XENBUS_ALLOC \ + _IOC(_IOC_NONE, ''B'', 1, sizeof(struct ioctl_xenbus_alloc)) +struct ioctl_xenbus_alloc { + /* IN */ + /* The domain ID (must exist) for xenstore */ + uint16_t dom; + uint16_t pad; + /* OUT */ + /* The port allocated for xenbus communication */ + uint32_t port; + /* Always set to GNTTAB_RESERVED_XENSTORE */ + uint32_t grant_ref; +}; + +#endif /* __LINUX_XEN_XENBUS_DEV_H__ */ diff --git a/tools/xenstore/Makefile b/tools/xenstore/Makefile index 3a061d6..9b411a5 100644 --- a/tools/xenstore/Makefile +++ b/tools/xenstore/Makefile @@ -26,7 +26,7 @@ LIBXENSTORE := libxenstore.a xenstore xenstore-control: CFLAGS += -static endif -ALL_TARGETS = libxenstore.so libxenstore.a clients xs_tdb_dump xenstored +ALL_TARGETS = libxenstore.so libxenstore.a clients xs_tdb_dump xenstored init-xenstore-domain ifdef CONFIG_STUBDOM CFLAGS += -DNO_SOCKETS=1 -DNO_LOCAL_XENBUS=1 -DNO_SYSLOG=1 -DNO_REOPEN_LOG=1 @@ -50,6 +50,11 @@ xenstored_probes.o: xenstored_solaris.o CFLAGS += -DHAVE_DTRACE=1 endif +init-xenstore-domain.o: CFLAGS += $(CFLAGS_libxenguest) + +init-xenstore-domain: init-xenstore-domain.o $(LIBXENSTORE) + $(CC) $(LDFLAGS) $^ $(LDLIBS_libxenctrl) $(LDLIBS_libxenguest) $(LDLIBS_libxenstore) -o $@ $(APPEND_LDFLAGS) + xenstored: $(XENSTORED_OBJS) $(CC) $(LDFLAGS) $^ $(LDLIBS_libxenctrl) $(SOCKET_LIBS) -o $@ $(APPEND_LDFLAGS) @@ -85,7 +90,7 @@ libxenstore.a: xs.o xs_lib.o clean: rm -f *.a *.o *.opic *.so* xenstored_probes.h rm -f xenstored xs_random xs_stress xs_crashme - rm -f xs_tdb_dump xenstore-control + rm -f xs_tdb_dump xenstore-control init-xenstore-domain rm -f xenstore $(CLIENTS) $(RM) $(DEPS) diff --git a/tools/xenstore/init-xenstore-domain.c b/tools/xenstore/init-xenstore-domain.c new file mode 100644 index 0000000..30f6c9b --- /dev/null +++ b/tools/xenstore/init-xenstore-domain.c @@ -0,0 +1,96 @@ +#include <fcntl.h> +#include <stdio.h> +#include <string.h> +#include <stdint.h> +#include <stdlib.h> +#include <sys/ioctl.h> +#include <sys/mman.h> +#include <xenctrl.h> +#include <xc_dom.h> +#include <xs.h> +#include <xen/sys/xenbus_dev.h> + +static uint32_t domid = -1; + +static int build(xc_interface *xch, char** argv) +{ + char cmdline[512]; + uint32_t ssid; + xen_domain_handle_t handle = { 0 }; + int rv; + int xs_fd = open("/dev/xen/xenbus_backend", O_RDWR); + struct xc_dom_image *dom; + int maxmem = atoi(argv[2]); + int limit_kb = (maxmem + 1)*1024; + struct ioctl_xenbus_alloc alloc; + + rv = xc_flask_context_to_sid(xch, argv[3], strlen(argv[3]), &ssid); + if (rv) return rv; + rv = xc_domain_create(xch, ssid, handle, 0, &domid); + if (rv) return rv; + rv = xc_domain_max_vcpus(xch, domid, 1); + if (rv) return rv; + rv = xc_domain_setmaxmem(xch, domid, limit_kb); + if (rv) return rv; + rv = xc_domain_set_memmap_limit(xch, domid, limit_kb); + if (rv) return rv; + + alloc.dom = domid; + rv = ioctl(xs_fd, IOCTL_XENBUS_ALLOC, &alloc); + if (rv) return rv; + snprintf(cmdline, 512, "--event %d", alloc.port); + + dom = xc_dom_allocate(xch, cmdline, NULL); + rv = xc_dom_kernel_file(dom, argv[1]); + if (rv) return rv; + rv = xc_dom_boot_xen_init(dom, xch, domid); + if (rv) return rv; + rv = xc_dom_parse_image(dom); + if (rv) return rv; + rv = xc_dom_mem_init(dom, maxmem); + if (rv) return rv; + rv = xc_dom_boot_mem_init(dom); + if (rv) return rv; + rv = xc_dom_build_image(dom); + if (rv) return rv; + rv = xc_dom_boot_image(dom); + if (rv) return rv; + + xc_dom_release(dom); + + rv = xc_domain_set_virq_handler(xch, domid, VIRQ_DOM_EXC); + if (rv) return rv; + rv = xc_domain_unpause(xch, domid); + if (rv) return rv; + + return 0; +} + +int main(int argc, char** argv) +{ + xc_interface *xch; + struct xs_handle *xsh; + char buf[16]; + int rv; + + if (argc != 4) { + printf("Use: %s <xenstore-kernel> <memory_mb> <flask-label>\n", argv[0]); + return 2; + } + + xch = xc_interface_open(NULL, NULL, 0); + if (!xch) return 1; + + rv = build(xch, argv); + + xc_interface_close(xch); + + if (rv) return 1; + + xsh = xs_open(0); + rv = snprintf(buf, 16, "%d", domid); + xs_write(xsh, XBT_NULL, "/tool/xenstored/domid", buf, rv); + xs_daemon_close(xsh); + + return 0; +} -- 1.7.7.5
Daniel De Graaf
2012-Jan-12 23:36 UTC
[PATCH] xenbus: Add support for xenbus backend in stub domain
This adds an ioctl to the /dev/xen/xenbus_backend device allowing the xenbus backend to be started after the kernel has booted. This is intended to allow dom0 to start another domain to run xenstore. Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> --- drivers/xen/xenbus/xenbus_comms.c | 6 ++++ drivers/xen/xenbus/xenbus_comms.h | 1 + drivers/xen/xenbus/xenbus_dev_backend.c | 44 +++++++++++++++++++++++++++++++ include/xen/grant_table.h | 2 + include/xen/xenbus_dev.h | 14 ++++++++++ 5 files changed, 67 insertions(+), 0 deletions(-) diff --git a/drivers/xen/xenbus/xenbus_comms.c b/drivers/xen/xenbus/xenbus_comms.c index 2eff7a6..52fe7ad 100644 --- a/drivers/xen/xenbus/xenbus_comms.c +++ b/drivers/xen/xenbus/xenbus_comms.c @@ -234,3 +234,9 @@ int xb_init_comms(void) return 0; } + +void xb_deinit_comms(void) +{ + unbind_from_irqhandler(xenbus_irq, &xb_waitq); + xenbus_irq = 0; +} diff --git a/drivers/xen/xenbus/xenbus_comms.h b/drivers/xen/xenbus/xenbus_comms.h index 6e42800..c8abd3b 100644 --- a/drivers/xen/xenbus/xenbus_comms.h +++ b/drivers/xen/xenbus/xenbus_comms.h @@ -35,6 +35,7 @@ int xs_init(void); int xb_init_comms(void); +void xb_deinit_comms(void); /* Low level routines. */ int xb_write(const void *data, unsigned len); diff --git a/drivers/xen/xenbus/xenbus_dev_backend.c b/drivers/xen/xenbus/xenbus_dev_backend.c index 3d3be78..854315c 100644 --- a/drivers/xen/xenbus/xenbus_dev_backend.c +++ b/drivers/xen/xenbus/xenbus_dev_backend.c @@ -8,7 +8,11 @@ #include <xen/xen.h> #include <xen/page.h> +#include <xen/xenbus.h> #include <xen/xenbus_dev.h> +#include <xen/grant_table.h> +#include <xen/events.h> +#include <asm/xen/hypervisor.h> #include "xenbus_comms.h" @@ -22,6 +26,43 @@ static int xenbus_backend_open(struct inode *inode, struct file *filp) return nonseekable_open(inode, filp); } +static long xenbus_alloc(void __user * data) +{ + struct ioctl_xenbus_alloc op; + struct evtchn_alloc_unbound arg; + int err; + + if (copy_from_user(&op, data, sizeof(op))) + return -EFAULT; + + gnttab_grant_foreign_access_ref(GNTTAB_RESERVED_XENSTORE, op.dom, + virt_to_mfn(xen_store_interface), 0 /* writable */); + + arg.dom = DOMID_SELF; + arg.remote_dom = op.dom; + + err = HYPERVISOR_event_channel_op(EVTCHNOP_alloc_unbound, &arg); + if (err) + return err; + + op.port = arg.port; + op.grant_ref = GNTTAB_RESERVED_XENSTORE; + + xs_suspend(); + + if (xen_store_evtchn > 0) + xb_deinit_comms(); + + xen_store_evtchn = arg.port; + + xs_resume(); + + if (copy_to_user(data, &op, sizeof(op))) + return -EFAULT; + + return 0; +} + static long xenbus_backend_ioctl(struct file *file, unsigned int cmd, unsigned long data) { if (!capable(CAP_SYS_ADMIN)) @@ -33,6 +74,9 @@ static long xenbus_backend_ioctl(struct file *file, unsigned int cmd, unsigned l return xen_store_evtchn; return -ENODEV; + case IOCTL_XENBUS_ALLOC: + return xenbus_alloc((void __user *)data); + default: return -ENOTTY; } diff --git a/include/xen/grant_table.h b/include/xen/grant_table.h index 15f8a00..11e27c3 100644 --- a/include/xen/grant_table.h +++ b/include/xen/grant_table.h @@ -46,6 +46,8 @@ #include <xen/features.h> +#define GNTTAB_RESERVED_XENSTORE 1 + /* NR_GRANT_FRAMES must be less than or equal to that configured in Xen */ #define NR_GRANT_FRAMES 4 diff --git a/include/xen/xenbus_dev.h b/include/xen/xenbus_dev.h index ac5f0fe..b107be9 100644 --- a/include/xen/xenbus_dev.h +++ b/include/xen/xenbus_dev.h @@ -38,4 +38,18 @@ #define IOCTL_XENBUS_BACKEND_EVTCHN \ _IOC(_IOC_NONE, ''B'', 0, 0) +#define IOCTL_XENBUS_ALLOC \ + _IOC(_IOC_NONE, ''B'', 1, sizeof(struct ioctl_xenbus_alloc)) +struct ioctl_xenbus_alloc { + /* IN */ + /* The domain ID (must exist) for xenstore */ + uint16_t dom; + uint16_t pad; + /* OUT */ + /* The port allocated for xenbus communication */ + uint32_t port; + /* Always set to GNTTAB_RESERVED_XENSTORE */ + uint32_t grant_ref; +}; + #endif /* __LINUX_XEN_XENBUS_DEV_H__ */ -- 1.7.7.5
Jan Beulich
2012-Jan-13 07:56 UTC
Re: [PATCH 01/18] xen: reinstate previously unused XENMEM_remove_from_physmap hypercall
>>> On 13.01.12 at 00:35, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > --- a/xen/arch/ia64/xen/mm.c > +++ b/xen/arch/ia64/xen/mm.c > @@ -3448,6 +3448,40 @@ arch_memory_op(int op, XEN_GUEST_HANDLE(void) arg) > break; > } > > + case XENMEM_remove_from_physmap: > + { > + struct xen_remove_from_physmap xrfp; > + unsigned long mfn; > + struct domain *d; > + > + if ( copy_from_guest(&xrfp, arg, 1) ) > + return -EFAULT; > + > + rc = rcu_lock_target_domain_by_id(xrfp.domid, &d); > + if ( rc != 0 ) > + return rc; > + > + if ( xsm_remove_from_physmap(current->domain, d) ) > + { > + rcu_unlock_domain(d); > + return -EPERM; > + } > + > + domain_lock(d); > + > + mfn = gmfn_to_mfn(d, xrfp.gpfn); > + > + if ( mfn_valid(mfn) ) > + guest_physmap_remove_page(d, xrfp.gpfn, mfn, 0);Here and in the other similar paths - is it really to be considered ''success'' if !mfn_valid(mfn)? Further, given that the code looks all the same between ia64 and x86 - is this really arch-specific (the get_gfn_untyped() and put_gfn() used in x86 should really be stubbed out for ia64 anyway)?> + > + domain_unlock(d); > + > + rcu_unlock_domain(d); > + > + break; > + } > + > + > case XENMEM_machine_memory_map: > { > struct xen_memory_map memmap; > --- a/xen/include/xlat.lst > +++ b/xen/include/xlat.lst > @@ -81,6 +81,7 @@ > ! processor_power platform.h > ? processor_px platform.h > ! psd_package platform.h > +! remove_from_physmap memory.hSorry, but this is not how I meant my comment on v1 to be interpreted - I thought that by looking at the rest of the file it would go without saying that the sorting is by header name (alphabetically), then by type name (again alphabetically). (I realize there are other cases where sorting is not fully correct, but as I''m striving to adjust this via secondary changes, I''d like to avoid getting the situation worse with new additions.)> ? xenpf_pcpuinfo platform.h > ? xenpf_pcpu_version platform.h > ! sched_poll sched.hJan
Jan Beulich
2012-Jan-13 08:03 UTC
Re: [PATCH 02/18] xen: allow global VIRQ handlers to be delegated to other domains
>>> On 13.01.12 at 00:35, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > +int set_global_virq_handler(struct domain *d, uint32_t virq) > +{ > + struct domain *old; > + > + if (virq >= NR_VIRQS) > + return -EINVAL; > + if (!virq_is_global(virq)) > + return -EINVAL; > + > + if (global_virq_handlers[virq] == d) > + return 0; > + > + if (unlikely(!get_domain(d))) > + return -EINVAL; > + > + spin_lock(&global_virq_handlers_lock); > + old = global_virq_handlers[virq]; > + global_virq_handlers[virq] = d; > + spin_unlock(&global_virq_handlers_lock); > + > + if (old != NULL) > + put_domain(old); > + > + return 0; > +} > + > +static void clear_global_virq_handlers(struct domain *d) > +{ > + uint32_t virq; > + int put_count = 0; > + > + spin_lock(&global_virq_handlers_lock); > + > + for (virq = 0; virq < NR_VIRQS; virq++) { > + if (global_virq_handlers[virq] == d) { > + global_virq_handlers[virq] = NULL; > + put_count++; > + } > + } > + > + spin_unlock(&global_virq_handlers_lock); > + > + while (put_count) { > + put_domain(d); > + put_count--; > + } > +}Formatting in this entire hunk should be changed to match that of the rest of the file.> --- a/xen/include/xsm/xsm.h > +++ b/xen/include/xsm/xsm.h > @@ -64,6 +64,7 @@ struct xsm_operations { > int (*domain_settime) (struct domain *d); > int (*set_target) (struct domain *d, struct domain *e); > int (*domctl) (struct domain *d, int cmd); > + int (*set_virq_handler) (struct domain *d, int virq);Here and further down, the ''int'' still survived. Jan
Jan Beulich
2012-Jan-13 08:07 UTC
Re: [PATCH 04/18] xen: Preserve reserved grant entries when switching versions
>>> On 13.01.12 at 00:35, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > --- a/xen/include/public/grant_table.h > +++ b/xen/include/public/grant_table.h > @@ -117,6 +117,12 @@ struct grant_entry_v1 { > }; > typedef struct grant_entry_v1 grant_entry_v1_t; > > +/* External tools reserve first few grant table entries. */This reads as if they must not be used (but could without taking extra care) by other than the tools themselves, which would be an incompatible change. As my understanding is that the change is backwards compatible, I would suggest adjusting the comment. Jan> +#define GNTTAB_NR_RESERVED_ENTRIES 8 > +#define GNTTAB_RESERVED_CONSOLE 0 > +#define GNTTAB_RESERVED_XENSTORE 1 > +/* (the next 6 are reserved for future use) */ > + > /* > * Type of grant entry. > * GTF_invalid: This grant entry grants no privileges.
Jan Beulich
2012-Jan-13 08:20 UTC
Re: [PATCH] xenbus: Add support for xenbus backend in stub domain
>>> On 13.01.12 at 00:36, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > --- a/include/xen/xenbus_dev.h > +++ b/include/xen/xenbus_dev.h > @@ -38,4 +38,18 @@ > #define IOCTL_XENBUS_BACKEND_EVTCHN \ > _IOC(_IOC_NONE, ''B'', 0, 0) > > +#define IOCTL_XENBUS_ALLOC \ > + _IOC(_IOC_NONE, ''B'', 1, sizeof(struct ioctl_xenbus_alloc)) > +struct ioctl_xenbus_alloc { > + /* IN */ > + /* The domain ID (must exist) for xenstore */ > + uint16_t dom; > + uint16_t pad; > + /* OUT */ > + /* The port allocated for xenbus communication */ > + uint32_t port; > + /* Always set to GNTTAB_RESERVED_XENSTORE */ > + uint32_t grant_ref; > +};As said in my reply to the previous patch version - if the functionality differs, the number *and* name should be different from the legacy implementation''s. Otherwise, how should compatible user space code be written? Jan> + > #endif /* __LINUX_XEN_XENBUS_DEV_H__ */
Daniel De Graaf
2012-Jan-13 13:58 UTC
Re: [PATCH 02/18] xen: allow global VIRQ handlers to be delegated to other domains
On 01/13/2012 03:03 AM, Jan Beulich wrote:>>>> On 13.01.12 at 00:35, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >> +int set_global_virq_handler(struct domain *d, uint32_t virq) >> +{ >> + struct domain *old; >> + >> + if (virq >= NR_VIRQS) >> + return -EINVAL; >> + if (!virq_is_global(virq)) >> + return -EINVAL; >> + >> + if (global_virq_handlers[virq] == d) >> + return 0; >> + >> + if (unlikely(!get_domain(d))) >> + return -EINVAL; >> + >> + spin_lock(&global_virq_handlers_lock); >> + old = global_virq_handlers[virq]; >> + global_virq_handlers[virq] = d; >> + spin_unlock(&global_virq_handlers_lock); >> + >> + if (old != NULL) >> + put_domain(old); >> + >> + return 0; >> +} >> + >> +static void clear_global_virq_handlers(struct domain *d) >> +{ >> + uint32_t virq; >> + int put_count = 0; >> + >> + spin_lock(&global_virq_handlers_lock); >> + >> + for (virq = 0; virq < NR_VIRQS; virq++) { >> + if (global_virq_handlers[virq] == d) { >> + global_virq_handlers[virq] = NULL; >> + put_count++; >> + } >> + } >> + >> + spin_unlock(&global_virq_handlers_lock); >> + >> + while (put_count) { >> + put_domain(d); >> + put_count--; >> + } >> +} > > Formatting in this entire hunk should be changed to match that of the > rest of the file. > >> --- a/xen/include/xsm/xsm.h >> +++ b/xen/include/xsm/xsm.h >> @@ -64,6 +64,7 @@ struct xsm_operations { >> int (*domain_settime) (struct domain *d); >> int (*set_target) (struct domain *d, struct domain *e); >> int (*domctl) (struct domain *d, int cmd); >> + int (*set_virq_handler) (struct domain *d, int virq); > > Here and further down, the ''int'' still survived. > > Jan >Much of the existing code handling virqs uses int; should I also change these instances to uint32_t?
Daniel De Graaf
2012-Jan-13 14:06 UTC
Re: [PATCH] xenbus: Add support for xenbus backend in stub domain
On 01/13/2012 03:20 AM, Jan Beulich wrote:>>>> On 13.01.12 at 00:36, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >> --- a/include/xen/xenbus_dev.h >> +++ b/include/xen/xenbus_dev.h >> @@ -38,4 +38,18 @@ >> #define IOCTL_XENBUS_BACKEND_EVTCHN \ >> _IOC(_IOC_NONE, ''B'', 0, 0) >> >> +#define IOCTL_XENBUS_ALLOC \ >> + _IOC(_IOC_NONE, ''B'', 1, sizeof(struct ioctl_xenbus_alloc)) >> +struct ioctl_xenbus_alloc { >> + /* IN */ >> + /* The domain ID (must exist) for xenstore */ >> + uint16_t dom; >> + uint16_t pad; >> + /* OUT */ >> + /* The port allocated for xenbus communication */ >> + uint32_t port; >> + /* Always set to GNTTAB_RESERVED_XENSTORE */ >> + uint32_t grant_ref; >> +}; > > As said in my reply to the previous patch version - if the functionality > differs, the number *and* name should be different from the legacy > implementation''s. Otherwise, how should compatible user space code > be written? > > Jan >This implementation has the same functionality as the legacy ioctl, although it has a different number and is performed on a different file, so it is already impossible to make automatically compatible userspace code. The structure name was changed to match other ioctl arguments, but the contents should be the same as in the legacy ioctl. If changing what file the ioctl is performed on justifies changing the ioctl name, then I would prefer the simpler interface where the domain ID is passed in as the ioctl parameter instead of a structure that only has one useful output.
Jan Beulich
2012-Jan-13 15:32 UTC
Re: [PATCH 02/18] xen: allow global VIRQ handlers to be delegated to other domains
>>> On 13.01.12 at 14:58, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > On 01/13/2012 03:03 AM, Jan Beulich wrote: >>>>> On 13.01.12 at 00:35, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >>> +int set_global_virq_handler(struct domain *d, uint32_t virq) >>> +{ >>> + struct domain *old; >>> + >>> + if (virq >= NR_VIRQS) >>> + return -EINVAL; >>> + if (!virq_is_global(virq)) >>> + return -EINVAL; >>> + >>> + if (global_virq_handlers[virq] == d) >>> + return 0; >>> + >>> + if (unlikely(!get_domain(d))) >>> + return -EINVAL; >>> + >>> + spin_lock(&global_virq_handlers_lock); >>> + old = global_virq_handlers[virq]; >>> + global_virq_handlers[virq] = d; >>> + spin_unlock(&global_virq_handlers_lock); >>> + >>> + if (old != NULL) >>> + put_domain(old); >>> + >>> + return 0; >>> +} >>> + >>> +static void clear_global_virq_handlers(struct domain *d) >>> +{ >>> + uint32_t virq; >>> + int put_count = 0; >>> + >>> + spin_lock(&global_virq_handlers_lock); >>> + >>> + for (virq = 0; virq < NR_VIRQS; virq++) { >>> + if (global_virq_handlers[virq] == d) { >>> + global_virq_handlers[virq] = NULL; >>> + put_count++; >>> + } >>> + } >>> + >>> + spin_unlock(&global_virq_handlers_lock); >>> + >>> + while (put_count) { >>> + put_domain(d); >>> + put_count--; >>> + } >>> +} >> >> Formatting in this entire hunk should be changed to match that of the >> rest of the file. >> >>> --- a/xen/include/xsm/xsm.h >>> +++ b/xen/include/xsm/xsm.h >>> @@ -64,6 +64,7 @@ struct xsm_operations { >>> int (*domain_settime) (struct domain *d); >>> int (*set_target) (struct domain *d, struct domain *e); >>> int (*domctl) (struct domain *d, int cmd); >>> + int (*set_virq_handler) (struct domain *d, int virq); >> >> Here and further down, the ''int'' still survived. >> >> Jan >> > > Much of the existing code handling virqs uses int; should I also change > these instances to uint32_t?That would be nice (if you do, making this a separate patch would be desirable). Here I''m just asking to not repeat the mistake. Jan
Jan Beulich
2012-Jan-13 15:37 UTC
Re: [PATCH] xenbus: Add support for xenbus backend in stub domain
>>> On 13.01.12 at 15:06, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > On 01/13/2012 03:20 AM, Jan Beulich wrote: >>>>> On 13.01.12 at 00:36, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >>> --- a/include/xen/xenbus_dev.h >>> +++ b/include/xen/xenbus_dev.h >>> @@ -38,4 +38,18 @@ >>> #define IOCTL_XENBUS_BACKEND_EVTCHN \ >>> _IOC(_IOC_NONE, ''B'', 0, 0) >>> >>> +#define IOCTL_XENBUS_ALLOC \ >>> + _IOC(_IOC_NONE, ''B'', 1, sizeof(struct ioctl_xenbus_alloc)) >>> +struct ioctl_xenbus_alloc { >>> + /* IN */ >>> + /* The domain ID (must exist) for xenstore */ >>> + uint16_t dom; >>> + uint16_t pad; >>> + /* OUT */ >>> + /* The port allocated for xenbus communication */ >>> + uint32_t port; >>> + /* Always set to GNTTAB_RESERVED_XENSTORE */ >>> + uint32_t grant_ref; >>> +}; >> >> As said in my reply to the previous patch version - if the functionality >> differs, the number *and* name should be different from the legacy >> implementation''s. Otherwise, how should compatible user space code >> be written? >> >> Jan >> > > This implementation has the same functionality as the legacy ioctl,It doesn''t - none of the "OUT" fields above exist there.> although it has a different number and is performed on a different file, > so it is already impossible to make automatically compatible userspace > code.That''s not what I was aiming at. The problem you''re introducing is that you name the ioctl IOCTL_XENBUS_ALLOC, identical to what the legacy code named it. Hence one won''t be able to write user mode code to invoke, depending on runtime determination, either the old or the new function. Jan> The structure name was changed to match other ioctl arguments, but > the contents should be the same as in the legacy ioctl. If changing what > file the ioctl is performed on justifies changing the ioctl name, then I > would prefer the simpler interface where the domain ID is passed in as the > ioctl parameter instead of a structure that only has one useful output.
Daniel De Graaf
2012-Jan-13 15:44 UTC
Re: [PATCH] xenbus: Add support for xenbus backend in stub domain
On 01/13/2012 10:37 AM, Jan Beulich wrote:>>>> On 13.01.12 at 15:06, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >> On 01/13/2012 03:20 AM, Jan Beulich wrote: >>>>>> On 13.01.12 at 00:36, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >>>> --- a/include/xen/xenbus_dev.h >>>> +++ b/include/xen/xenbus_dev.h >>>> @@ -38,4 +38,18 @@ >>>> #define IOCTL_XENBUS_BACKEND_EVTCHN \ >>>> _IOC(_IOC_NONE, ''B'', 0, 0) >>>> >>>> +#define IOCTL_XENBUS_ALLOC \ >>>> + _IOC(_IOC_NONE, ''B'', 1, sizeof(struct ioctl_xenbus_alloc)) >>>> +struct ioctl_xenbus_alloc { >>>> + /* IN */ >>>> + /* The domain ID (must exist) for xenstore */ >>>> + uint16_t dom; >>>> + uint16_t pad; >>>> + /* OUT */ >>>> + /* The port allocated for xenbus communication */ >>>> + uint32_t port; >>>> + /* Always set to GNTTAB_RESERVED_XENSTORE */ >>>> + uint32_t grant_ref; >>>> +}; >>> >>> As said in my reply to the previous patch version - if the functionality >>> differs, the number *and* name should be different from the legacy >>> implementation''s. Otherwise, how should compatible user space code >>> be written? >>> >>> Jan >>> >> >> This implementation has the same functionality as the legacy ioctl, > > It doesn''t - none of the "OUT" fields above exist there.Are you looking at the same legacy ioctl as I am? I found it in 2.6.18.hg where the structure was defined as: typedef struct xenbus_alloc { domid_t dom; __u32 port; __u32 grant_ref; } xenbus_alloc_t; The port and grant_ref fields were outputs. I added padding for clarity since domid_t is uint16_t.> >> although it has a different number and is performed on a different file, >> so it is already impossible to make automatically compatible userspace >> code. > > That''s not what I was aiming at. The problem you''re introducing is > that you name the ioctl IOCTL_XENBUS_ALLOC, identical to what > the legacy code named it. Hence one won''t be able to write user > mode code to invoke, depending on runtime determination, either > the old or the new function. > > Jan > >> The structure name was changed to match other ioctl arguments, but >> the contents should be the same as in the legacy ioctl. If changing what >> file the ioctl is performed on justifies changing the ioctl name, then I >> would prefer the simpler interface where the domain ID is passed in as the >> ioctl parameter instead of a structure that only has one useful output. > >
Jan Beulich
2012-Jan-13 16:00 UTC
Re: [PATCH] xenbus: Add support for xenbus backend in stub domain
>>> On 13.01.12 at 16:44, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > On 01/13/2012 10:37 AM, Jan Beulich wrote: >>>>> On 13.01.12 at 15:06, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >>> On 01/13/2012 03:20 AM, Jan Beulich wrote: >>>>>>> On 13.01.12 at 00:36, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >>>>> --- a/include/xen/xenbus_dev.h >>>>> +++ b/include/xen/xenbus_dev.h >>>>> @@ -38,4 +38,18 @@ >>>>> #define IOCTL_XENBUS_BACKEND_EVTCHN \ >>>>> _IOC(_IOC_NONE, ''B'', 0, 0) >>>>> >>>>> +#define IOCTL_XENBUS_ALLOC \ >>>>> + _IOC(_IOC_NONE, ''B'', 1, sizeof(struct ioctl_xenbus_alloc)) >>>>> +struct ioctl_xenbus_alloc { >>>>> + /* IN */ >>>>> + /* The domain ID (must exist) for xenstore */ >>>>> + uint16_t dom; >>>>> + uint16_t pad; >>>>> + /* OUT */ >>>>> + /* The port allocated for xenbus communication */ >>>>> + uint32_t port; >>>>> + /* Always set to GNTTAB_RESERVED_XENSTORE */ >>>>> + uint32_t grant_ref; >>>>> +}; >>>> >>>> As said in my reply to the previous patch version - if the functionality >>>> differs, the number *and* name should be different from the legacy >>>> implementation''s. Otherwise, how should compatible user space code >>>> be written? >>>> >>>> Jan >>>> >>> >>> This implementation has the same functionality as the legacy ioctl, >> >> It doesn''t - none of the "OUT" fields above exist there. > > Are you looking at the same legacy ioctl as I am? I found it in 2.6.18.hg > where the structure was defined as: > > typedef struct xenbus_alloc { > domid_t dom; > __u32 port; > __u32 grant_ref; > } xenbus_alloc_t; > > The port and grant_ref fields were outputs. I added padding for clarity > since domid_t is uint16_t.Ooops, I''m sorry, indeed - I didn''t look at the names and types closely enough. They were too different, so I assumed the whole thing is different. That doesn''t eliminate the difference between the two IOCTL_XENBUS_ALLOC definitions, though [_IOC(_IOC_NONE, ''X'', 0, sizeof(xenbus_alloc_t)) vs _IOC(_IOC_NONE, ''B'', 1, sizeof(struct ioctl_xenbus_alloc))] - one still can''t use both in the same source file. Additionally (forgive if I''m not up to date in this respect) - is it okay these days to use uintX_t in exported Linux headers, and then without including the header that would define the necessary types? Jan>>> although it has a different number and is performed on a different file, >>> so it is already impossible to make automatically compatible userspace >>> code. >> >> That''s not what I was aiming at. The problem you''re introducing is >> that you name the ioctl IOCTL_XENBUS_ALLOC, identical to what >> the legacy code named it. Hence one won''t be able to write user >> mode code to invoke, depending on runtime determination, either >> the old or the new function. >> >> Jan >> >>> The structure name was changed to match other ioctl arguments, but >>> the contents should be the same as in the legacy ioctl. If changing what >>> file the ioctl is performed on justifies changing the ioctl name, then I >>> would prefer the simpler interface where the domain ID is passed in as the >>> ioctl parameter instead of a structure that only has one useful output. >> >>
Daniel De Graaf
2012-Jan-13 17:42 UTC
Re: [PATCH] xenbus: Add support for xenbus backend in stub domain
On 01/13/2012 11:00 AM, Jan Beulich wrote:>>>> On 13.01.12 at 16:44, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >> On 01/13/2012 10:37 AM, Jan Beulich wrote: >>>>>> On 13.01.12 at 15:06, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >>>> On 01/13/2012 03:20 AM, Jan Beulich wrote: >>>>>>>> On 13.01.12 at 00:36, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >>>>>> --- a/include/xen/xenbus_dev.h >>>>>> +++ b/include/xen/xenbus_dev.h >>>>>> @@ -38,4 +38,18 @@ >>>>>> #define IOCTL_XENBUS_BACKEND_EVTCHN \ >>>>>> _IOC(_IOC_NONE, ''B'', 0, 0) >>>>>> >>>>>> +#define IOCTL_XENBUS_ALLOC \ >>>>>> + _IOC(_IOC_NONE, ''B'', 1, sizeof(struct ioctl_xenbus_alloc)) >>>>>> +struct ioctl_xenbus_alloc { >>>>>> + /* IN */ >>>>>> + /* The domain ID (must exist) for xenstore */ >>>>>> + uint16_t dom; >>>>>> + uint16_t pad; >>>>>> + /* OUT */ >>>>>> + /* The port allocated for xenbus communication */ >>>>>> + uint32_t port; >>>>>> + /* Always set to GNTTAB_RESERVED_XENSTORE */ >>>>>> + uint32_t grant_ref; >>>>>> +}; >>>>> >>>>> As said in my reply to the previous patch version - if the functionality >>>>> differs, the number *and* name should be different from the legacy >>>>> implementation''s. Otherwise, how should compatible user space code >>>>> be written? >>>>> >>>>> Jan >>>>> >>>> >>>> This implementation has the same functionality as the legacy ioctl, >>> >>> It doesn''t - none of the "OUT" fields above exist there. >> >> Are you looking at the same legacy ioctl as I am? I found it in 2.6.18.hg >> where the structure was defined as: >> >> typedef struct xenbus_alloc { >> domid_t dom; >> __u32 port; >> __u32 grant_ref; >> } xenbus_alloc_t; >> >> The port and grant_ref fields were outputs. I added padding for clarity >> since domid_t is uint16_t. > > Ooops, I''m sorry, indeed - I didn''t look at the names and types > closely enough. They were too different, so I assumed the whole > thing is different. > > That doesn''t eliminate the difference between the two > IOCTL_XENBUS_ALLOC definitions, though > [_IOC(_IOC_NONE, ''X'', 0, sizeof(xenbus_alloc_t)) vs > _IOC(_IOC_NONE, ''B'', 1, sizeof(struct ioctl_xenbus_alloc))] - one > still can''t use both in the same source file.Is compatibility with 2.6.18-xen more important than keeping the ioctl numbers in a single file consistent? The definition in 2.6.18 is also incorrect (as was my definition) - since the argument is used, it needs to use _IOWR() or _IOC_READ|_IOC_WRITE instead of _IOC_NONE. Since this ioctl must already be performed a different file than the legacy ioctl, a new name seems to be the best solution.> Additionally (forgive if I''m not up to date in this respect) - is it okay > these days to use uintX_t in exported Linux headers, and then > without including the header that would define the necessary > types? > > JanIt''s done in a number of other header files (I was copying from gntdev.h) but that doesn''t mean it''s officially okay. Using __u32 is probably safer.> >>>> although it has a different number and is performed on a different file, >>>> so it is already impossible to make automatically compatible userspace >>>> code. >>> >>> That''s not what I was aiming at. The problem you''re introducing is >>> that you name the ioctl IOCTL_XENBUS_ALLOC, identical to what >>> the legacy code named it. Hence one won''t be able to write user >>> mode code to invoke, depending on runtime determination, either >>> the old or the new function. >>> >>> Jan >>> >>>> The structure name was changed to match other ioctl arguments, but >>>> the contents should be the same as in the legacy ioctl. If changing what >>>> file the ioctl is performed on justifies changing the ioctl name, then I >>>> would prefer the simpler interface where the domain ID is passed in as the >>>> ioctl parameter instead of a structure that only has one useful output. >>>
Jan Beulich
2012-Jan-16 08:19 UTC
Re: [PATCH] xenbus: Add support for xenbus backend in stub domain
>>> On 13.01.12 at 18:42, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: > On 01/13/2012 11:00 AM, Jan Beulich wrote: >>>>> On 13.01.12 at 16:44, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >>> On 01/13/2012 10:37 AM, Jan Beulich wrote: >>>>>>> On 13.01.12 at 15:06, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >>>>> On 01/13/2012 03:20 AM, Jan Beulich wrote: >>>>>>>>> On 13.01.12 at 00:36, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote: >>>>>>> --- a/include/xen/xenbus_dev.h >>>>>>> +++ b/include/xen/xenbus_dev.h >>>>>>> @@ -38,4 +38,18 @@ >>>>>>> #define IOCTL_XENBUS_BACKEND_EVTCHN \ >>>>>>> _IOC(_IOC_NONE, ''B'', 0, 0) >>>>>>> >>>>>>> +#define IOCTL_XENBUS_ALLOC \ >>>>>>> + _IOC(_IOC_NONE, ''B'', 1, sizeof(struct ioctl_xenbus_alloc)) >>>>>>> +struct ioctl_xenbus_alloc { >>>>>>> + /* IN */ >>>>>>> + /* The domain ID (must exist) for xenstore */ >>>>>>> + uint16_t dom; >>>>>>> + uint16_t pad; >>>>>>> + /* OUT */ >>>>>>> + /* The port allocated for xenbus communication */ >>>>>>> + uint32_t port; >>>>>>> + /* Always set to GNTTAB_RESERVED_XENSTORE */ >>>>>>> + uint32_t grant_ref; >>>>>>> +}; >>>>>> >>>>>> As said in my reply to the previous patch version - if the functionality >>>>>> differs, the number *and* name should be different from the legacy >>>>>> implementation''s. Otherwise, how should compatible user space code >>>>>> be written? >>>>>> >>>>>> Jan >>>>>> >>>>> >>>>> This implementation has the same functionality as the legacy ioctl, >>>> >>>> It doesn''t - none of the "OUT" fields above exist there. >>> >>> Are you looking at the same legacy ioctl as I am? I found it in 2.6.18.hg >>> where the structure was defined as: >>> >>> typedef struct xenbus_alloc { >>> domid_t dom; >>> __u32 port; >>> __u32 grant_ref; >>> } xenbus_alloc_t; >>> >>> The port and grant_ref fields were outputs. I added padding for clarity >>> since domid_t is uint16_t. >> >> Ooops, I''m sorry, indeed - I didn''t look at the names and types >> closely enough. They were too different, so I assumed the whole >> thing is different. >> >> That doesn''t eliminate the difference between the two >> IOCTL_XENBUS_ALLOC definitions, though >> [_IOC(_IOC_NONE, ''X'', 0, sizeof(xenbus_alloc_t)) vs >> _IOC(_IOC_NONE, ''B'', 1, sizeof(struct ioctl_xenbus_alloc))] - one >> still can''t use both in the same source file. > > Is compatibility with 2.6.18-xen more important than keeping the ioctl > numbers in a single file consistent? The definition in 2.6.18 is also > incorrect (as was my definition) - since the argument is used, it needs > to use _IOWR() or _IOC_READ|_IOC_WRITE instead of _IOC_NONE.Hmm, that would mean that apart from IOCTL_EVTCHN_RESET *all* current ioctl number definitions under include/xen/ are wrong. Very bad, and ugly to deal with. With generic ioctl handling code not depending on this, I wonder though whether it''s worth fixing at all. Are you aware of any user mode code depending on the correctness of these bits? Which of course isn''t to say that it shouldn''t be done right for new ones...> Since this ioctl must already be performed a different file than the > legacy ioctl, a new name seems to be the best solution.That''s what I was trying to ask you to do. Jan
Ian Campbell
2012-Jan-18 10:21 UTC
Re: [PATCH 07/18] mini-os: avoid crash if no console is provided
On Thu, 2012-01-12 at 17:56 +0000, Daniel De Graaf wrote:> On 01/12/2012 05:03 AM, Ian Campbell wrote: > > On Wed, 2012-01-11 at 17:21 +0000, Daniel De Graaf wrote: > > > > When/why does this happen? > > Initially, when I use a custom domain builder to create the xenstored > domain without a xenconsoled running yet (as xenconsoled needs xenstore).Right, that makes sense.> > I guess it is because when starting the xenstore domain you cannot use > > xenstore to communicate with xenconsoled (and/or it isn''t even running > > yet). > > > > I wonder if there is a way we can do lazy-setup of the console for just > > the xenstore domain? > > Xenstored will produce output on the hypervisor console if Xen is compiled > with debugging. If xenstored produces a lot of output, it can block on > waiting for xenconsoled, which might be blocking on xenstored - so I don''t > think we want to hook it up to the console in the normal case, or at least > not the same xenconsoled that talks to domUs.Hrm, yes.> > Alternatively I seem to recall a little tool which Diego wrote to pull > > the console ring out of a domain directly as a debuging aid but that > > relies on us setting up a console ring and evtchn even if xenconsoled > > cannot be involved which makes this patch unnecessary.> This runs into the same blocking problem if xenstored produces too much > output.What about if the tools/xenstore/init-xenstore-domain tool you posted in your v2 provides console_mfn and console_evtch to the stubdom (via the shared info) and after it has started the domain it would daemonize and sit there pumping stuff out of the stubdom console ring and into /var/log or syslog or something. There should be no interaction with xenstore there, should there? Ian.
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> Changes from v1: > - set_virq_handler implemented in libxc > - added custom domain builder for xenstored > - xenstore/console domain IDs now pulled from xenstore > - migration support when using split xenstored (untested, should work) > - slightly less intrusive NO_SOCKETS xenstored patch > (still has many ifdefs to avoid pulling in socket headers or symbols) > - virq handlers removed from dying domain when clearing event channels > - dummy XSM module restricts getdomaininfo similar to no-XSM case > - formatting/type fixups > - partial ioctl compatibility with legacy IOCTL_XENBUS_ALLOC > (the new device path uses ''B'' not ''X'' as the base so the ioctl number > does not match; otherwise, should be compatible). > > To start xenstored, run: > > tools/xenstore/init-xenstore-domain stubdom/mini-os-x86_64-xenstore/mini-os 20 system_u:system_r:domU_t > > This will populate the xenstore domid key /tool/xenstore/domidI''m just about to go through and review this properly but from the above it''s sounding good, thanks! You have rather undermined my intentions for the next hackathon though ;-) I was planning to investigate and reinvigorate this stuff, I guess there is still stub-oxenstored left for me though! Speaking of which, will you be at the hackathon? http://lists.xen.org/archives/html/xen-devel/2012-01/msg00268.html Ian.> > ---- > > [PATCH 01/18] xen: reinstate previously unused hypercall > [PATCH 02/18] xen: allow global VIRQ handlers to be delegated to > [PATCH 03/18] xen: use XSM instead of IS_PRIV for getdomaininfo > [PATCH 04/18] xen: Preserve reserved grant entries when switching > > [PATCH 05/18] tools/libxl: pull xenstore/console domids from > [PATCH 06/18] lib{xc,xl}: Seed grant tables with xenstore and > > [PATCH 07/18] mini-os: avoid crash if no console is provided > [PATCH 08/18] mini-os: avoid crash if no xenstore is provided > [PATCH 09/18] mini-os: remove per-fd evtchn limit > [PATCH 10/18] xenstored: use grant references instead of > [PATCH 11/18] xenstored: add NO_SOCKETS compilation option > [PATCH 12/18] xenstored support for in-memory rather than FS based > [PATCH 13/18] xenstored: support running in minios stubdom > [PATCH 14/18] xenstored: always use xc_gnttab_munmap in stubdom > [PATCH 15/18] xenstored: add --event parameter for bootstrapping > Removed old #16; shared page is no longer used to pass event. > [PATCH 16/18] xenstored: use domain_is_unprivileged instead of > [PATCH 17/18] xenstored: add --priv-domid parameter > > [PATCH 18/18] xenstored: Add stub domain builder > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel
Ian Campbell
2012-Jan-18 10:36 UTC
Re: [PATCH 01/18] xen: reinstate previously unused XENMEM_remove_from_physmap hypercall
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > > This patch reinstates the XENMEM_remove_from_physmap hypercall > which was removed in 19041:ee62aaafff46 because it was not used. > > However, is now needed in order to support xenstored stub domains. > The xenstored stub domain is not priviliged like dom0 and so cannot > unilaterally map the xenbus page of other guests into it''s address > space. Therefore, before creating a domU the domain builder needs to > seed its grant table with a grant ref allowing the xenstored stub > domain to access the new domU''s xenbus page. > > At present domU''s do not start with their grant table mapped. > Instead it gets mapped when the guest requests a grant table from > the hypervisor. > > In order to seed the grant table, the domain builder first needs to > map it into dom0 address space. But the hypercall to do this > requires a gpfn (guest pfn), which is an mfn for PV guest, but a pfn > for HVM guests. Therfore, in order to seed the grant table of an > HVM guest, dom0 needs to *temporarily* map it into the guest''s > "physical" address space. > > Hence the need to reinstate the XENMEM_remove_from_physmap hypercall. > > Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>Acked-by: Ian Campbell <ian.campbell@citrix.com> (modulo Jan''s comment about ordering in xlat.lst) BTW, since Alex and Diego have subsequently left Citrix you could take my Acked-by''s in this series as Signed-of-by on behalf of Citrix. I''ve no idea if that''s necessary though, I expect not. Ian.> --- > xen/arch/ia64/xen/mm.c | 34 ++++++++++++++++++++++++++++++++++ > xen/arch/x86/mm.c | 35 +++++++++++++++++++++++++++++++++++ > xen/arch/x86/x86_64/compat/mm.c | 14 ++++++++++++++ > xen/include/public/memory.h | 16 ++++++++++++++++ > xen/include/xlat.lst | 1 + > xen/include/xsm/xsm.h | 6 ++++++ > xen/xsm/dummy.c | 6 ++++++ > xen/xsm/flask/hooks.c | 6 ++++++ > 8 files changed, 118 insertions(+), 0 deletions(-) > > diff --git a/xen/arch/ia64/xen/mm.c b/xen/arch/ia64/xen/mm.c > index 84f6a61..a40f96a 100644 > --- a/xen/arch/ia64/xen/mm.c > +++ b/xen/arch/ia64/xen/mm.c > @@ -3448,6 +3448,40 @@ arch_memory_op(int op, XEN_GUEST_HANDLE(void) arg) > break; > } > > + case XENMEM_remove_from_physmap: > + { > + struct xen_remove_from_physmap xrfp; > + unsigned long mfn; > + struct domain *d; > + > + if ( copy_from_guest(&xrfp, arg, 1) ) > + return -EFAULT; > + > + rc = rcu_lock_target_domain_by_id(xrfp.domid, &d); > + if ( rc != 0 ) > + return rc; > + > + if ( xsm_remove_from_physmap(current->domain, d) ) > + { > + rcu_unlock_domain(d); > + return -EPERM; > + } > + > + domain_lock(d); > + > + mfn = gmfn_to_mfn(d, xrfp.gpfn); > + > + if ( mfn_valid(mfn) ) > + guest_physmap_remove_page(d, xrfp.gpfn, mfn, 0); > + > + domain_unlock(d); > + > + rcu_unlock_domain(d); > + > + break; > + } > + > + > case XENMEM_machine_memory_map: > { > struct xen_memory_map memmap; > diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c > index 1f996e0..39cc3c0 100644 > --- a/xen/arch/x86/mm.c > +++ b/xen/arch/x86/mm.c > @@ -4871,6 +4871,41 @@ long arch_memory_op(int op, XEN_GUEST_HANDLE(void) arg) > return rc; > } > > + case XENMEM_remove_from_physmap: > + { > + struct xen_remove_from_physmap xrfp; > + unsigned long mfn; > + struct domain *d; > + > + if ( copy_from_guest(&xrfp, arg, 1) ) > + return -EFAULT; > + > + rc = rcu_lock_target_domain_by_id(xrfp.domid, &d); > + if ( rc != 0 ) > + return rc; > + > + if ( xsm_remove_from_physmap(current->domain, d) ) > + { > + rcu_unlock_domain(d); > + return -EPERM; > + } > + > + domain_lock(d); > + > + mfn = get_gfn_untyped(d, xrfp.gpfn); > + > + if ( mfn_valid(mfn) ) > + guest_physmap_remove_page(d, xrfp.gpfn, mfn, PAGE_ORDER_4K); > + > + put_gfn(d, xrfp.gpfn); > + > + domain_unlock(d); > + > + rcu_unlock_domain(d); > + > + break; > + } > + > case XENMEM_set_memory_map: > { > struct xen_foreign_memory_map fmap; > diff --git a/xen/arch/x86/x86_64/compat/mm.c b/xen/arch/x86/x86_64/compat/mm.c > index bea94fe..dde5430 100644 > --- a/xen/arch/x86/x86_64/compat/mm.c > +++ b/xen/arch/x86/x86_64/compat/mm.c > @@ -82,6 +82,20 @@ int compat_arch_memory_op(int op, XEN_GUEST_HANDLE(void) arg) > break; > } > > + case XENMEM_remove_from_physmap: > + { > + struct compat_remove_from_physmap cmp; > + struct xen_remove_from_physmap *nat = (void *)COMPAT_ARG_XLAT_VIRT_BASE; > + > + if ( copy_from_guest(&cmp, arg, 1) ) > + return -EFAULT; > + > + XLAT_remove_from_physmap(nat, &cmp); > + rc = arch_memory_op(op, guest_handle_from_ptr(nat, void)); > + > + break; > + } > + > case XENMEM_set_memory_map: > { > struct compat_foreign_memory_map cmp; > diff --git a/xen/include/public/memory.h b/xen/include/public/memory.h > index c5b78a8..308deff 100644 > --- a/xen/include/public/memory.h > +++ b/xen/include/public/memory.h > @@ -229,6 +229,22 @@ struct xen_add_to_physmap { > typedef struct xen_add_to_physmap xen_add_to_physmap_t; > DEFINE_XEN_GUEST_HANDLE(xen_add_to_physmap_t); > > +/* > + * Unmaps the page appearing at a particular GPFN from the specified guest''s > + * pseudophysical address space. > + * arg == addr of xen_remove_from_physmap_t. > + */ > +#define XENMEM_remove_from_physmap 15 > +struct xen_remove_from_physmap { > + /* Which domain to change the mapping for. */ > + domid_t domid; > + > + /* GPFN of the current mapping of the page. */ > + xen_pfn_t gpfn; > +}; > +typedef struct xen_remove_from_physmap xen_remove_from_physmap_t; > +DEFINE_XEN_GUEST_HANDLE(xen_remove_from_physmap_t); > + > /*** REMOVED ***/ > /*#define XENMEM_translate_gpfn_list 8*/ > > diff --git a/xen/include/xlat.lst b/xen/include/xlat.lst > index 3d92175..ee1772c 100644 > --- a/xen/include/xlat.lst > +++ b/xen/include/xlat.lst > @@ -81,6 +81,7 @@ > ! processor_power platform.h > ? processor_px platform.h > ! psd_package platform.h > +! remove_from_physmap memory.h > ? xenpf_pcpuinfo platform.h > ? xenpf_pcpu_version platform.h > ! sched_poll sched.h > diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h > index df6cec2..566c808 100644 > --- a/xen/include/xsm/xsm.h > +++ b/xen/include/xsm/xsm.h > @@ -169,6 +169,7 @@ struct xsm_operations { > int (*update_va_mapping) (struct domain *d, struct domain *f, > l1_pgentry_t pte); > int (*add_to_physmap) (struct domain *d1, struct domain *d2); > + int (*remove_from_physmap) (struct domain *d1, struct domain *d2); > int (*sendtrigger) (struct domain *d); > int (*bind_pt_irq) (struct domain *d, struct xen_domctl_bind_pt_irq *bind); > int (*unbind_pt_irq) (struct domain *d); > @@ -738,6 +739,11 @@ static inline int xsm_add_to_physmap(struct domain *d1, struct domain *d2) > return xsm_call(add_to_physmap(d1, d2)); > } > > +static inline int xsm_remove_from_physmap(struct domain *d1, struct domain *d2) > +{ > + return xsm_call(remove_from_physmap(d1, d2)); > +} > + > static inline int xsm_sendtrigger(struct domain *d) > { > return xsm_call(sendtrigger(d)); > diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c > index 4bbfbff..65daa4e 100644 > --- a/xen/xsm/dummy.c > +++ b/xen/xsm/dummy.c > @@ -529,6 +529,11 @@ static int dummy_add_to_physmap (struct domain *d1, struct domain *d2) > return 0; > } > > +static int dummy_remove_from_physmap (struct domain *d1, struct domain *d2) > +{ > + return 0; > +} > + > static int dummy_sendtrigger (struct domain *d) > { > return 0; > @@ -690,6 +695,7 @@ void xsm_fixup_ops (struct xsm_operations *ops) > set_to_dummy_if_null(ops, mmu_machphys_update); > set_to_dummy_if_null(ops, update_va_mapping); > set_to_dummy_if_null(ops, add_to_physmap); > + set_to_dummy_if_null(ops, remove_from_physmap); > set_to_dummy_if_null(ops, sendtrigger); > set_to_dummy_if_null(ops, bind_pt_irq); > set_to_dummy_if_null(ops, pin_mem_cacheattr); > diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c > index 0d35767..a2020a9 100644 > --- a/xen/xsm/flask/hooks.c > +++ b/xen/xsm/flask/hooks.c > @@ -1283,6 +1283,11 @@ static int flask_add_to_physmap(struct domain *d1, struct domain *d2) > return domain_has_perm(d1, d2, SECCLASS_MMU, MMU__PHYSMAP); > } > > +static int flask_remove_from_physmap(struct domain *d1, struct domain *d2) > +{ > + return domain_has_perm(d1, d2, SECCLASS_MMU, MMU__PHYSMAP); > +} > + > static int flask_sendtrigger(struct domain *d) > { > return domain_has_perm(current->domain, d, SECCLASS_DOMAIN, DOMAIN__TRIGGER); > @@ -1550,6 +1555,7 @@ static struct xsm_operations flask_ops = { > .mmu_machphys_update = flask_mmu_machphys_update, > .update_va_mapping = flask_update_va_mapping, > .add_to_physmap = flask_add_to_physmap, > + .remove_from_physmap = flask_remove_from_physmap, > .sendtrigger = flask_sendtrigger, > .get_device_group = flask_get_device_group, > .test_assign_device = flask_test_assign_device,
Ian Campbell
2012-Jan-18 10:39 UTC
Re: [PATCH 02/18] xen: allow global VIRQ handlers to be delegated to other domains
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> > +static void clear_global_virq_handlers(struct domain *d) > +{ > + uint32_t virq; > + int put_count = 0; > + > + spin_lock(&global_virq_handlers_lock); > + > + for (virq = 0; virq < NR_VIRQS; virq++) { > + if (global_virq_handlers[virq] == d) { > + global_virq_handlers[virq] = NULL;I don''t suppose we should rebind to dom0, should we? Seems like we are pretty hosed if this ever happens in a non-controlled manner anyway...> + put_count++; > + } > + } > + > + spin_unlock(&global_virq_handlers_lock); > + > + while (put_count) { > + put_domain(d); > + put_count--; > + } > +}
Ian Campbell
2012-Jan-18 10:43 UTC
Re: [PATCH 04/18] xen: Preserve reserved grant entries when switching versions
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> In order for the toolstack to use reserved grant table entries, the > grant table for a guest must be initialized prior to the guest''s boot. > When the guest switches grant table versions (necessary if the guest is > using v2 grant tables, or on kexec if switching grant versions), these > initial grants will be cleared. Instead of clearing them, preserve > the grants across the type change. > > Attempting to use or preserve v2-only features such as sub-page grants > will produce invalid v1 grant entries, which (while they will not work) > is not a problem since a guest can always produce such invalid entries.Would it be better to explicitly clear such entries? Perhaps we should even warn since it is effectively disallowed for the toolstack to use v2 features in the reserved entries.> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > --- > xen/common/grant_table.c | 38 +++++++++++++++++++++++++++++++------- > xen/include/public/grant_table.h | 6 ++++++ > 2 files changed, 37 insertions(+), 7 deletions(-) > > diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c > index 014734d..8d4a4cb 100644 > --- a/xen/common/grant_table.c > +++ b/xen/common/grant_table.c > @@ -2106,6 +2106,7 @@ gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) > struct domain *d = current->domain; > struct grant_table *gt = d->grant_table; > struct active_grant_entry *act; > + grant_entry_v1_t reserved_entries[GNTTAB_NR_RESERVED_ENTRIES]; > long res; > int i; > > @@ -2126,7 +2127,7 @@ gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) > /* (You need to change the version number for e.g. kexec.) */ > if ( gt->gt_version != 0 ) > { > - for ( i = 0; i < nr_grant_entries(gt); i++ ) > + for ( i = GNTTAB_NR_RESERVED_ENTRIES; i < nr_grant_entries(gt); i++ ) > { > act = &active_entry(gt, i); > if ( act->pin != 0 ) > @@ -2151,15 +2152,38 @@ gnttab_set_version(XEN_GUEST_HANDLE(gnttab_set_version_t uop)) > goto out_unlock; > } > > + /* Preserve the first 8 entries (toolstack reserved grants) */ > + if (gt->gt_version == 1) { > + memcpy(reserved_entries, gt->shared_v1[0], sizeof(reserved_entries)); > + } else if (gt->gt_version == 2) { > + for ( i = 0; i < GNTTAB_NR_RESERVED_ENTRIES && i < nr_grant_entries(gt); i++ ) > + { > + reserved_entries[i].flags = shared_entry_v2(gt, i).hdr.flags; > + reserved_entries[i].domid = shared_entry_v2(gt, i).hdr.domid; > + reserved_entries[i].frame = shared_entry_v2(gt, i).full_page.frame; > + reserved_entries[i].flags |= status_entry(gt, i); > + } > + } > + > if ( op.version < 2 && gt->gt_version == 2 ) > gnttab_unpopulate_status_frames(d, gt); > > - if ( op.version != gt->gt_version ) > - { > - /* Make sure there''s no crud left over in the table from the > - old version. */ > - for ( i = 0; i < nr_grant_frames(gt); i++ ) > - memset(gt->shared_raw[i], 0, PAGE_SIZE); > + /* Make sure there''s no crud left over in the table from the > + old version. */ > + for ( i = 0; i < nr_grant_frames(gt); i++ ) > + memset(gt->shared_raw[i], 0, PAGE_SIZE); > + > + /* Restore the first 8 entries (toolstack reserved grants) */ > + if (gt->gt_version != 0 && op.version == 1) { > + memcpy(gt->shared_v1[0], reserved_entries, sizeof(reserved_entries)); > + } else if (gt->gt_version != 0 && op.version == 2) { > + for ( i = 0; i < GNTTAB_NR_RESERVED_ENTRIES; i++ ) > + { > + status_entry(gt, i) = reserved_entries[i].flags & (GTF_reading|GTF_writing); > + shared_entry_v2(gt, i).hdr.flags = reserved_entries[i].flags & ~(GTF_reading|GTF_writing); > + shared_entry_v2(gt, i).hdr.domid = reserved_entries[i].domid; > + shared_entry_v2(gt, i).full_page.frame = reserved_entries[i].frame; > + } > } > > gt->gt_version = op.version; > diff --git a/xen/include/public/grant_table.h b/xen/include/public/grant_table.h > index 0bf20bc..36d1ac7 100644 > --- a/xen/include/public/grant_table.h > +++ b/xen/include/public/grant_table.h > @@ -117,6 +117,12 @@ struct grant_entry_v1 { > }; > typedef struct grant_entry_v1 grant_entry_v1_t; > > +/* External tools reserve first few grant table entries. */ > +#define GNTTAB_NR_RESERVED_ENTRIES 8 > +#define GNTTAB_RESERVED_CONSOLE 0 > +#define GNTTAB_RESERVED_XENSTORE 1 > +/* (the next 6 are reserved for future use) */ > + > /* > * Type of grant entry. > * GTF_invalid: This grant entry grants no privileges.
Ian Campbell
2012-Jan-18 10:47 UTC
Re: [PATCH 05/18] tools/libxl: pull xenstore/console domids from xenstore
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> Instead of assuming that xenstored and xenconsoled are running in dom0, > pull the domain IDs from xenstore. > > Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>I''m not sure we need to carry these around in the build state, since they can just be looked up (perhaps by helper functions) but nevertheless: Acked-by: Ian Campbell <ian.campbell@citrix.com>> --- > tools/libxl/libxl_dom.c | 14 ++++++++++++-- > tools/libxl/libxl_internal.h | 2 ++ > 2 files changed, 14 insertions(+), 2 deletions(-) > > diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c > index a4725fe..5e39595 100644 > --- a/tools/libxl/libxl_dom.c > +++ b/tools/libxl/libxl_dom.c > @@ -73,6 +73,7 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid, > { > libxl_ctx *ctx = libxl__gc_owner(gc); > int tsc_mode; > + char *xs_domid, *con_domid; > xc_domain_max_vcpus(ctx->xch, domid, info->max_vcpus); > xc_domain_setmaxmem(ctx->xch, domid, info->target_memkb + LIBXL_MAXMEM_CONSTANT); > if (info->type == LIBXL_DOMAIN_TYPE_PV) > @@ -104,9 +105,18 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid, > xc_shadow_control(ctx->xch, domid, XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION, NULL, 0, &shadow, 0, NULL); > } > > - state->store_port = xc_evtchn_alloc_unbound(ctx->xch, domid, 0); > - state->console_port = xc_evtchn_alloc_unbound(ctx->xch, domid, 0); > + xs_domid = xs_read(ctx->xsh, XBT_NULL, "/tool/xenstored/domid", NULL); > + state->store_domid = xs_domid ? atoi(xs_domid) : 0; > + free(xs_domid); > + > + con_domid = xs_read(ctx->xsh, XBT_NULL, "/tool/xenconsoled/domid", NULL); > + state->console_domid = con_domid ? atoi(con_domid) : 0; > + free(con_domid); > + > + state->store_port = xc_evtchn_alloc_unbound(ctx->xch, domid, state->store_domid); > + state->console_port = xc_evtchn_alloc_unbound(ctx->xch, domid, state->console_domid); > state->vm_generationid_addr = 0; > + > return 0; > } > > diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h > index 288c03c..97ead08 100644 > --- a/tools/libxl/libxl_internal.h > +++ b/tools/libxl/libxl_internal.h > @@ -219,9 +219,11 @@ _hidden int libxl__domain_shutdown_reason(libxl__gc *gc, uint32_t domid); > libxl__domain_type((gc), (domid)) == LIBXL_DOMAIN_TYPE_##type > typedef struct { > uint32_t store_port; > + uint32_t store_domid; > unsigned long store_mfn; > > uint32_t console_port; > + uint32_t console_domid; > unsigned long console_mfn; > unsigned long vm_generationid_addr; > } libxl__domain_build_state;
Ian Campbell
2012-Jan-18 11:05 UTC
Re: [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> @@ -275,6 +276,169 @@ int xc_dom_boot_image(struct xc_dom_image *dom) > return rc; > } > > +static unsigned long xc_dom_gnttab_setup(xc_interface *xch, uint32_t domid) > +{ > + DECLARE_HYPERCALL; > + gnttab_setup_table_t setup_table; > + DECLARE_HYPERCALL_BUFFER(unsigned long, gmfnp); > + int rc; > + unsigned long gmfn;There''s a bunch of weird alignment issues in this patch. Presumably some hard tabs have snuck in.> + > + gmfnp = xc_hypercall_buffer_alloc(xch, gmfnp, sizeof(*gmfnp)); > + if (gmfnp == NULL) > + return -1;Alignment.> + > + setup_table.dom = domid; > + setup_table.nr_frames = 1; > + set_xen_guest_handle(setup_table.frame_list, gmfnp); > + setup_table.status = 0; > + > + hypercall.op = __HYPERVISOR_grant_table_op; > + hypercall.arg[0] = GNTTABOP_setup_table; > + hypercall.arg[1] = (unsigned long) &setup_table; > + hypercall.arg[2] = 1; > + > + rc = do_xen_hypercall(xch, &hypercall); > + gmfn = *gmfnp;Alignment. I''m going to stop pointing these out, I guess grep will find them... [...]> +int xc_dom_gnttab_init(struct xc_dom_image *dom) > +{ > + unsigned long console_gmfn; > + unsigned long xenstore_gmfn; > + int autotranslated; > + > + autotranslated = xc_dom_feature_translated(dom); > + console_gmfn = autotranslated ? > + dom->console_pfn : xc_dom_p2m_host(dom, dom->console_pfn); > + xenstore_gmfn = autotranslated ? > + dom->xenstore_pfn : xc_dom_p2m_host(dom, dom->xenstore_pfn); > + > + return xc_dom_gnttab_seed(dom->xch, dom->guest_domid, > + console_gmfn, xenstore_gmfn, > + dom->console_domid, dom->xenstore_domid);So on build we have can do the p2m lookup and hence use the PV version of gnttab_seed, whereas on restore we don''t have a p2m and hence can''t? The internals of xc_domain_restore do have the p2m though, so in principal we could avoid the need for the hvm version and the reintroduction of the remove_from_physmap by making xc_domain_restore output the necessary mfns? I''m not sure it is worth it though.> diff --git a/tools/libxc/xc_domain_restore.c b/tools/libxc/xc_domain_restore.c > index 3fda6f8..8bee684 100644 > --- a/tools/libxc/xc_domain_restore.c > +++ b/tools/libxc/xc_domain_restore.c > @@ -1259,7 +1259,8 @@ static int apply_batch(xc_interface *xch, uint32_t dom, struct restore_ctx *ctx, > > int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, > unsigned int store_evtchn, unsigned long *store_mfn, > - unsigned int console_evtchn, unsigned long *console_mfn, > + uint32_t store_domid, unsigned int console_evtchn, > + unsigned long *console_mfn, uint32_t console_domid, > unsigned int hvm, unsigned int pae, int superpages, > int no_incr_generationid, > unsigned long *vm_generationid_addr) > @@ -2018,6 +2019,14 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, > memcpy(ctx->live_p2m, ctx->p2m, dinfo->p2m_size * sizeof(xen_pfn_t)); > munmap(ctx->live_p2m, P2M_FL_ENTRIES * PAGE_SIZE); > > + rc = xc_dom_gnttab_seed(xch, dom, *console_mfn, *store_mfn, > + console_domid, store_domid); > + if (rc != 0) > + { > + ERROR("error seeding grant table"); > + goto out; > + } > + > DPRINTF("Domain ready to be built.\n"); > rc = 0; > goto out; > @@ -2076,6 +2085,14 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, > goto out; > } > > + rc = xc_dom_gnttab_hvm_seed(xch, dom, *console_mfn, *store_mfn, > + console_domid, store_domid);Oh, we don''t even need to return them from xc_domain_restore since we could just translate from gfn to mfn right here (since we have a p2m in hand) and call xc_dom_gnttab_seed. Or am I missing something?> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c > index 5e39595..04db22f 100644 > --- a/tools/libxl/libxl_dom.c > +++ b/tools/libxl/libxl_dom.c[...]> @@ -305,6 +312,8 @@ static int hvm_build_set_params(xc_interface *handle, uint32_t domid, > xc_set_hvm_param(handle, domid, HVM_PARAM_NESTEDHVM, info->u.hvm.nested_hvm); > xc_set_hvm_param(handle, domid, HVM_PARAM_STORE_EVTCHN, store_evtchn); > xc_set_hvm_param(handle, domid, HVM_PARAM_CONSOLE_EVTCHN, console_evtchn); > + > + xc_dom_gnttab_hvm_seed(handle, domid, *console_mfn, *store_mfn, console_domid, store_domid);Didn''t this already happen in xc_linux_build_internal which was called via xc_linux_build_mem? [...] Ian.
Ian Campbell
2012-Jan-18 11:06 UTC
Re: [PATCH 07/18] mini-os: avoid crash if no console is provided
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > --- > extras/mini-os/console/xencons_ring.c | 7 ++++++- > 1 files changed, 6 insertions(+), 1 deletions(-) > > diff --git a/extras/mini-os/console/xencons_ring.c b/extras/mini-os/console/xencons_ring.c > index 22fd618..14a8bd1 100644 > --- a/extras/mini-os/console/xencons_ring.c > +++ b/extras/mini-os/console/xencons_ring.c > @@ -25,7 +25,10 @@ static inline void notify_daemon(struct consfront_dev *dev) > > static inline struct xencons_interface *xencons_interface(void) > { > - return mfn_to_virt(start_info.console.domU.mfn); > + if (start_info.console.domU.evtchn) > + return mfn_to_virt(start_info.console.domU.mfn); > + else > + return NULL; > } > > int xencons_ring_send_no_notify(struct consfront_dev *dev, const char *data, unsigned len) > @@ -38,6 +41,8 @@ int xencons_ring_send_no_notify(struct consfront_dev *dev, const char *data, uns > intf = xencons_interface(); > else > intf = dev->ring; > + if (!intf) > + return sent;Indentation. Otherwise Acked-by: Ian Campbell <ian.campbell@citrix.com> Although I would still prefer if we could figure out some scheme to get logs out of this domain... Ian.
Ian Campbell
2012-Jan-18 11:08 UTC
Re: [PATCH 08/18] mini-os: avoid crash if no xenstore is provided
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>Acked-by: Ian Campbell <ian.campbell@citrix.com> I suppose we could arrange for xenbus to be omitted entirely but this is easier. Ian.> --- > extras/mini-os/xenbus/xenbus.c | 9 +++++++++ > 1 files changed, 9 insertions(+), 0 deletions(-) > > diff --git a/extras/mini-os/xenbus/xenbus.c b/extras/mini-os/xenbus/xenbus.c > index a8081fd..e475e2c 100644 > --- a/extras/mini-os/xenbus/xenbus.c > +++ b/extras/mini-os/xenbus/xenbus.c > @@ -328,6 +328,10 @@ static int allocate_xenbus_id(void) > void init_xenbus(void) > { > int err; > + if (!start_info.store_evtchn) { > + printk("Skipping initialization of xenbus\n"); > + return; > + } > printk("Initialising xenbus\n"); > DEBUG("init_xenbus called.\n"); > xenstore_buf = mfn_to_virt(start_info.store_mfn); > @@ -435,6 +439,11 @@ xenbus_msg_reply(int type, > DEFINE_WAIT(w); > struct xsd_sockmsg *rep; > > + if (!xenstore_buf) { > + printk("xenbus_msg_reply called but no xenstore!\n"); > + return NULL; > + } > + > id = allocate_xenbus_id(); > add_waiter(w, req_info[id].waitq); >
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > > Changes the minios evtchn implementation to use a list instead of an array. > This allows it to grow as necessary to support any number of ports. > Unfortunately, it''s still limited by NR_EVS in events.c.NR_EVS is 1024 which, IIRC, is the actual ABI limit for a 32 bit domain. In principal we could make this 4096 for a 64 bit stub domain but I guess we only need 1 evtchn per domain plus perhaps a couple of incidental global ones (e.g. console, VIRQ_mumble) and ~1000+ domains is more than enough for any current system.> Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> > Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>Acked-by: Ian Campbell <ian.campbell@citrix.com>> --- > extras/mini-os/include/lib.h | 16 +++--- > tools/libxc/xc_minios.c | 139 ++++++++++++++++++++++-------------------- > 2 files changed, 81 insertions(+), 74 deletions(-) > > diff --git a/extras/mini-os/include/lib.h b/extras/mini-os/include/lib.h > index bd3eeaf..12070c3 100644 > --- a/extras/mini-os/include/lib.h > +++ b/extras/mini-os/include/lib.h > @@ -53,6 +53,7 @@ > #include <xen/xen.h> > #include <xen/event_channel.h> > #include "gntmap.h" > +#include "list.h" > > #ifdef HAVE_LIBC > #include <stdio.h> > @@ -143,7 +144,12 @@ enum fd_type { > FTYPE_SAVEFILE, > }; > > -#define MAX_EVTCHN_PORTS 16 > +struct evtchn_port_info { > + struct minios_list_head list; > + evtchn_port_t port; > + unsigned long pending; > + int bound; > +}; > > extern struct file { > enum fd_type type; > @@ -158,13 +164,7 @@ extern struct file { > off_t offset; > } file; > struct { > - /* To each event channel FD is associated a series of ports which > - * wakes select for this FD. */ > - struct { > - evtchn_port_t port; > - unsigned long pending; > - int bound; > - } ports[MAX_EVTCHN_PORTS]; > + struct minios_list_head ports; > } evtchn; > struct gntmap gntmap; > struct { > diff --git a/tools/libxc/xc_minios.c b/tools/libxc/xc_minios.c > index 8bbfd18..29cce63 100644 > --- a/tools/libxc/xc_minios.c > +++ b/tools/libxc/xc_minios.c > @@ -210,15 +210,34 @@ static struct xc_osdep_ops minios_privcmd_ops = { > }, > }; > > + > +/* XXX Note: This is not threadsafe */ > +static struct evtchn_port_info* port_alloc(int fd) { > + struct evtchn_port_info *port_info; > + port_info = malloc(sizeof(struct evtchn_port_info)); > + if (port_info == NULL) > + return NULL; > + port_info->pending = 0; > + port_info->port = -1; > + port_info->bound = 0; > + > + minios_list_add(&port_info->list, &files[fd].evtchn.ports); > + return port_info; > +} > + > +static void port_dealloc(struct evtchn_port_info *port_info) { > + if (port_info->bound) > + unbind_evtchn(port_info->port); > + minios_list_del(&port_info->list); > + free(port_info); > +} > + > static xc_osdep_handle minios_evtchn_open(xc_evtchn *xce) > { > - int fd = alloc_fd(FTYPE_EVTCHN), i; > + int fd = alloc_fd(FTYPE_EVTCHN); > if ( fd == -1 ) > return XC_OSDEP_OPEN_ERROR; > - for (i = 0; i < MAX_EVTCHN_PORTS; i++) { > - files[fd].evtchn.ports[i].port = -1; > - files[fd].evtchn.ports[i].bound = 0; > - } > + MINIOS_INIT_LIST_HEAD(&files[fd].evtchn.ports); > printf("evtchn_open() -> %d\n", fd); > return (xc_osdep_handle)fd; > } > @@ -231,10 +250,10 @@ static int minios_evtchn_close(xc_evtchn *xce, xc_osdep_handle h) > > void minios_evtchn_close_fd(int fd) > { > - int i; > - for (i = 0; i < MAX_EVTCHN_PORTS; i++) > - if (files[fd].evtchn.ports[i].bound) > - unbind_evtchn(files[fd].evtchn.ports[i].port); > + struct evtchn_port_info *port_info, *tmp; > + minios_list_for_each_entry_safe(port_info, tmp, &files[fd].evtchn.ports, list) > + port_dealloc(port_info); > + > files[fd].type = FTYPE_NONE; > } > > @@ -256,35 +275,21 @@ static int minios_evtchn_notify(xc_evtchn *xce, xc_osdep_handle h, evtchn_port_t > return ret; > } > > -/* XXX Note: This is not threadsafe */ > -static int port_alloc(int fd) { > - int i; > - for (i= 0; i < MAX_EVTCHN_PORTS; i++) > - if (files[fd].evtchn.ports[i].port == -1) > - break; > - if (i == MAX_EVTCHN_PORTS) { > - printf("Too many ports in xc handle\n"); > - errno = EMFILE; > - return -1; > - } > - files[fd].evtchn.ports[i].pending = 0; > - return i; > -} > - > static void evtchn_handler(evtchn_port_t port, struct pt_regs *regs, void *data) > { > int fd = (int)(intptr_t)data; > - int i; > + struct evtchn_port_info *port_info; > assert(files[fd].type == FTYPE_EVTCHN); > mask_evtchn(port); > - for (i= 0; i < MAX_EVTCHN_PORTS; i++) > - if (files[fd].evtchn.ports[i].port == port) > - break; > - if (i == MAX_EVTCHN_PORTS) { > - printk("Unknown port for handle %d\n", fd); > - return; > + minios_list_for_each_entry(port_info, &files[fd].evtchn.ports, list) { > + if (port_info->port == port) > + goto found; > } > - files[fd].evtchn.ports[i].pending = 1; > + printk("Unknown port for handle %d\n", fd); > + return; > + > + found: > + port_info->pending = 1; > files[fd].read = 1; > wake_up(&event_queue); > } > @@ -292,12 +297,13 @@ static void evtchn_handler(evtchn_port_t port, struct pt_regs *regs, void *data) > static evtchn_port_or_error_t minios_evtchn_bind_unbound_port(xc_evtchn *xce, xc_osdep_handle h, int domid) > { > int fd = (int)h; > - int ret, i; > + struct evtchn_port_info *port_info; > + int ret; > evtchn_port_t port; > > assert(get_current() == main_thread); > - i = port_alloc(fd); > - if (i == -1) > + port_info = port_alloc(fd); > + if (port_info == NULL) > return -1; > > printf("xc_evtchn_bind_unbound_port(%d)", domid); > @@ -305,11 +311,12 @@ static evtchn_port_or_error_t minios_evtchn_bind_unbound_port(xc_evtchn *xce, xc > printf(" = %d\n", ret); > > if (ret < 0) { > + port_dealloc(port_info); > errno = -ret; > return -1; > } > - files[fd].evtchn.ports[i].bound = 1; > - files[fd].evtchn.ports[i].port = port; > + port_info->bound = 1; > + port_info->port = port; > unmask_evtchn(port); > return port; > } > @@ -318,12 +325,13 @@ static evtchn_port_or_error_t minios_evtchn_bind_interdomain(xc_evtchn *xce, xc_ > evtchn_port_t remote_port) > { > int fd = (int)h; > + struct evtchn_port_info *port_info; > evtchn_port_t local_port; > - int ret, i; > + int ret; > > assert(get_current() == main_thread); > - i = port_alloc(fd); > - if (i == -1) > + port_info = port_alloc(fd); > + if (port_info == NULL) > return -1; > > printf("xc_evtchn_bind_interdomain(%d, %"PRId32")", domid, remote_port); > @@ -331,11 +339,12 @@ static evtchn_port_or_error_t minios_evtchn_bind_interdomain(xc_evtchn *xce, xc_ > printf(" = %d\n", ret); > > if (ret < 0) { > + port_dealloc(port_info); > errno = -ret; > return -1; > } > - files[fd].evtchn.ports[i].bound = 1; > - files[fd].evtchn.ports[i].port = local_port; > + port_info->bound = 1; > + port_info->port = local_port; > unmask_evtchn(local_port); > return local_port; > } > @@ -343,42 +352,40 @@ static evtchn_port_or_error_t minios_evtchn_bind_interdomain(xc_evtchn *xce, xc_ > static int minios_evtchn_unbind(xc_evtchn *xce, xc_osdep_handle h, evtchn_port_t port) > { > int fd = (int)h; > - int i; > - for (i = 0; i < MAX_EVTCHN_PORTS; i++) > - if (files[fd].evtchn.ports[i].port == port) { > - files[fd].evtchn.ports[i].port = -1; > - break; > - } > - if (i == MAX_EVTCHN_PORTS) { > - printf("Warning: couldn''t find port %"PRId32" for xc handle %x\n", port, fd); > - errno = -EINVAL; > - return -1; > + struct evtchn_port_info *port_info; > + > + minios_list_for_each_entry(port_info, &files[fd].evtchn.ports, list) { > + if (port_info->port == port) { > + port_dealloc(port_info); > + return 0; > + } > } > - files[fd].evtchn.ports[i].bound = 0; > - unbind_evtchn(port); > - return 0; > + printf("Warning: couldn''t find port %"PRId32" for xc handle %x\n", port, fd); > + errno = -EINVAL; > + return -1; > } > > static evtchn_port_or_error_t minios_evtchn_bind_virq(xc_evtchn *xce, xc_osdep_handle h, unsigned int virq) > { > int fd = (int)h; > + struct evtchn_port_info *port_info; > evtchn_port_t port; > - int i; > > assert(get_current() == main_thread); > - i = port_alloc(fd); > - if (i == -1) > + port_info = port_alloc(fd); > + if (port_info == NULL) > return -1; > > printf("xc_evtchn_bind_virq(%d)", virq); > port = bind_virq(virq, evtchn_handler, (void*)(intptr_t)fd); > > if (port < 0) { > + port_dealloc(port_info); > errno = -port; > return -1; > } > - files[fd].evtchn.ports[i].bound = 1; > - files[fd].evtchn.ports[i].port = port; > + port_info->bound = 1; > + port_info->port = port; > unmask_evtchn(port); > return port; > } > @@ -386,18 +393,18 @@ static evtchn_port_or_error_t minios_evtchn_bind_virq(xc_evtchn *xce, xc_osdep_h > static evtchn_port_or_error_t minios_evtchn_pending(xc_evtchn *xce, xc_osdep_handle h) > { > int fd = (int)h; > - int i; > + struct evtchn_port_info *port_info; > unsigned long flags; > evtchn_port_t ret = -1; > > local_irq_save(flags); > files[fd].read = 0; > - for (i = 0; i < MAX_EVTCHN_PORTS; i++) { > - evtchn_port_t port = files[fd].evtchn.ports[i].port; > - if (port != -1 && files[fd].evtchn.ports[i].pending) { > + > + minios_list_for_each_entry(port_info, &files[fd].evtchn.ports, list) { > + if (port_info->port != -1 && port_info->pending) { > if (ret == -1) { > - ret = port; > - files[fd].evtchn.ports[i].pending = 0; > + ret = port_info->port; > + port_info->pending = 0; > } else { > files[fd].read = 1; > break;
Ian Campbell
2012-Jan-18 11:15 UTC
Re: [PATCH 10/18] xenstored: use grant references instead of map_foreign_range
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > > make xenstored use grantref rather than map_foreign_range (which can > only be used by privileged domains) > > This patch modifies the xenstore daemon to use xc_gnttab_map_grant_ref > instead of xc_map_foreign_range where available. > > Previous versions of this patch have been sent to xen-devel. See > http://lists.xensource.com/archives/html/xen-devel/2008-07/msg00610.html > http://lists.xensource.com/archives/html/xen-devel/2009-03/msg01492.html > > Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> > Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > --- > tools/xenstore/xenstored_domain.c | 45 ++++++++++++++++++++++++++++++++----- > 1 files changed, 39 insertions(+), 6 deletions(-) > > diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c > index 443af82..0b8353b 100644 > --- a/tools/xenstore/xenstored_domain.c > +++ b/tools/xenstore/xenstored_domain.c > @@ -32,8 +32,10 @@ > #include "xenstored_watch.h" > > #include <xenctrl.h> > +#include <xen/grant_table.h> > > static xc_interface **xc_handle; > +static xc_gnttab **xcg_handle; > static evtchn_port_t virq_port; > > xc_evtchn *xce_handle = NULL; > @@ -174,8 +176,12 @@ static int destroy_domain(void *_domain) > eprintf("> Unbinding port %i failed!\n", domain->port); > } > > - if (domain->interface) > - munmap(domain->interface, getpagesize()); > + if (domain->interface) { > + if (*xcg_handle >= 0 && domain->domid != 0)Why the special case for domid 0 here? There seems to be no equivalent for the map case, including the one you add in patch 15/18. I think the map and unmap logic could usefully be made into helper functions. Ian.
Ian Campbell
2012-Jan-18 11:23 UTC
Re: [PATCH 11/18] xenstored: add NO_SOCKETS compilation option
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > > option for compiling xenstored without unix sockets to support running on mini-OS > > Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> > Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>I guess this about as good as it gets, you could avoid some of the ifdef''s with "if (*sock != -1)" type constructions but I don''t know that this is necessarily better. Acked-by: Ian Campbell <ian.campbell@citrix.com>> --- > tools/xenstore/xenstored_core.c | 22 +++++++++++++++++++++- > tools/xenstore/xs.c | 2 ++ > tools/xenstore/xs_lib.c | 4 ++++ > 3 files changed, 27 insertions(+), 1 deletions(-) > > diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c > index 9e6c2c7..631bfe4 100644 > --- a/tools/xenstore/xenstored_core.c > +++ b/tools/xenstore/xenstored_core.c > @@ -19,9 +19,11 @@ > > #include <sys/types.h> > #include <sys/stat.h> > -#include <sys/socket.h> > #include <sys/select.h> > +#ifndef NO_SOCKETS > +#include <sys/socket.h> > #include <sys/un.h> > +#endif > #include <sys/time.h> > #include <time.h> > #include <unistd.h> > @@ -320,8 +322,10 @@ static int initialize_set(fd_set *inset, fd_set *outset, int sock, int ro_sock, > FD_ZERO(inset); > FD_ZERO(outset); > > +#ifndef NO_SOCKETS > set_fd(sock, inset, &max); > set_fd(ro_sock, inset, &max); > +#endif > set_fd(reopen_log_pipe[0], inset, &max); > > if (xce_handle != NULL) > @@ -343,12 +347,14 @@ static int initialize_set(fd_set *inset, fd_set *outset, int sock, int ro_sock, > return max; > } > > +#ifndef NO_SOCKETS > static int destroy_fd(void *_fd) > { > int *fd = _fd; > close(*fd); > return 0; > } > +#endif > > /* Is child a subnode of parent, or equal? */ > bool is_child(const char *child, const char *parent) > @@ -1352,6 +1358,7 @@ struct connection *new_connection(connwritefn_t *write, connreadfn_t *read) > return new; > } > > +#ifndef NO_SOCKETS > static int writefd(struct connection *conn, const void *data, unsigned int len) > { > int rc; > @@ -1406,6 +1413,7 @@ static void accept_connection(int sock, bool canwrite) > } else > close(fd); > } > +#endif > > #define TDB_FLAGS 0 > > @@ -1753,7 +1761,11 @@ extern void dump_conn(struct connection *conn); > int main(int argc, char *argv[]) > { > int opt, *sock, *ro_sock, max; > +#ifdef NO_SOCKETS > + int minus_one = -1; > +#else > struct sockaddr_un addr; > +#endif > fd_set inset, outset; > bool dofork = true; > bool outputpid = false; > @@ -1837,6 +1849,9 @@ int main(int argc, char *argv[]) > if (!dofork) > talloc_enable_leak_report_full(); > > +#ifdef NO_SOCKETS > + sock = ro_sock = &minus_one; > +#else > /* Create sockets for them to listen to. */ > sock = talloc(talloc_autofree_context(), int); > *sock = socket(PF_UNIX, SOCK_STREAM, 0); > @@ -1848,10 +1863,12 @@ int main(int argc, char *argv[]) > barf_perror("Could not create socket"); > talloc_set_destructor(sock, destroy_fd); > talloc_set_destructor(ro_sock, destroy_fd); > +#endif > > /* Don''t kill us with SIGPIPE. */ > signal(SIGPIPE, SIG_IGN); > > +#ifndef NO_SOCKETS > /* FIXME: Be more sophisticated, don''t mug running daemon. */ > unlink(xs_daemon_socket()); > unlink(xs_daemon_socket_ro()); > @@ -1871,6 +1888,7 @@ int main(int argc, char *argv[]) > if (listen(*sock, 1) != 0 > || listen(*ro_sock, 1) != 0) > barf_perror("Could not listen on sockets"); > +#endif > > if (pipe(reopen_log_pipe)) { > barf_perror("pipe"); > @@ -1931,11 +1949,13 @@ int main(int argc, char *argv[]) > reopen_log(); > } > > +#ifndef NO_SOCKETS > if (FD_ISSET(*sock, &inset)) > accept_connection(*sock, true); > > if (FD_ISSET(*ro_sock, &inset)) > accept_connection(*ro_sock, false); > +#endif > > if (evtchn_fd != -1 && FD_ISSET(evtchn_fd, &inset)) > handle_event(); > diff --git a/tools/xenstore/xs.c b/tools/xenstore/xs.c > index 8e54fe0..119d945 100644 > --- a/tools/xenstore/xs.c > +++ b/tools/xenstore/xs.c > @@ -271,10 +271,12 @@ struct xs_handle *xs_open(unsigned long flags) > { > struct xs_handle *xsh = NULL; > > +#ifndef NO_SOCKETS > if (flags & XS_OPEN_READONLY) > xsh = get_handle(xs_daemon_socket_ro()); > else > xsh = get_handle(xs_daemon_socket()); > +#endif > > if (!xsh && !(flags & XS_OPEN_SOCKETONLY)) > xsh = get_handle(xs_domain_dev()); > diff --git a/tools/xenstore/xs_lib.c b/tools/xenstore/xs_lib.c > index 03a9ee4..af3db6b 100644 > --- a/tools/xenstore/xs_lib.c > +++ b/tools/xenstore/xs_lib.c > @@ -39,6 +39,7 @@ const char *xs_daemon_rundir(void) > return (s ? s : "/var/run/xenstored"); > } > > +#ifndef NO_SOCKETS > static const char *xs_daemon_path(void) > { > static char buf[PATH_MAX]; > @@ -50,6 +51,7 @@ static const char *xs_daemon_path(void) > return NULL; > return buf; > } > +#endif > > const char *xs_daemon_tdb(void) > { > @@ -58,6 +60,7 @@ const char *xs_daemon_tdb(void) > return buf; > } > > +#ifndef NO_SOCKETS > const char *xs_daemon_socket(void) > { > return xs_daemon_path(); > @@ -73,6 +76,7 @@ const char *xs_daemon_socket_ro(void) > return NULL; > return buf; > } > +#endif > > const char *xs_domain_dev(void) > {
Ian Campbell
2012-Jan-18 11:27 UTC
Re: [PATCH 12/18] xenstored support for in-memory rather than FS based trivial DB (needed to run on mini-OS)
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > > tdb_copy (a xen modification to tdb?)I believe so, the original author of xenstored was also the author of tdb, IIRC.> should honor the TDB_INTERNAL flag > for in-memory databases. > > TODO: leaks memory on error caseDoes this happen on every/any transaction conflict or just if a bad thing has occurred? In any case, is fixing it not just a case of calling talloc_free? FWIW this bit of C xenstored is the main reason I think oxenstored is/would be a better fit for a stub xenstored.> Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> > Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > --- > tools/xenstore/tdb.c | 36 ++++++++++++++++++++++++++++++++++++ > 1 files changed, 36 insertions(+), 0 deletions(-) > > diff --git a/tools/xenstore/tdb.c b/tools/xenstore/tdb.c > index 63205e1..639ce6e 100644 > --- a/tools/xenstore/tdb.c > +++ b/tools/xenstore/tdb.c > @@ -2103,6 +2103,42 @@ TDB_CONTEXT *tdb_copy(TDB_CONTEXT *tdb, const char *outfile) > int fd, saved_errno; > TDB_CONTEXT *copy; > > + if (tdb->flags & TDB_INTERNAL) { > + struct tdb_header *copydb; > + > + copy = talloc_zero(outfile, TDB_CONTEXT); > + if (copy == NULL) { > + errno = ENOMEM; > + goto intfail; > + } > + memcpy(copy, tdb, sizeof(TDB_CONTEXT)); > + > + if (copy->name || copy->locked || copy->device || copy->inode) { > + fprintf(stderr, "tdb_copy assumption(s) failed\n"); > + goto intfail; > + } > + > + copydb = talloc_zero_size(copy, copy->map_size); > + if (copydb == NULL) { > + errno = ENOMEM; > + goto intfail; > + } > + memcpy(copydb, copy->map_ptr, copy->map_size); > + copy->map_ptr = (char*) copydb; > + > + if (tdb_brlock(tdb, GLOBAL_LOCK, F_UNLCK, F_SETLKW, 0) == -1) > + goto intfail; > + > + copy->next = tdbs; > + tdbs = copy; > + > + > + return copy; > +intfail: > + /* TODO (leaking memory is easier) */ > + return NULL; > + } > + > fd = open(outfile, O_TRUNC|O_CREAT|O_WRONLY, 0640); > if (fd < 0) > return NULL;
Jan Beulich
2012-Jan-18 11:28 UTC
Re: [PATCH 02/18] xen: allow global VIRQ handlers to be delegated to other domains
>>> On 18.01.12 at 11:39, Ian Campbell <Ian.Campbell@citrix.com> wrote: > On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote: >> >> +static void clear_global_virq_handlers(struct domain *d) >> +{ >> + uint32_t virq; >> + int put_count = 0; >> + >> + spin_lock(&global_virq_handlers_lock); >> + >> + for (virq = 0; virq < NR_VIRQS; virq++) { >> + if (global_virq_handlers[virq] == d) { >> + global_virq_handlers[virq] = NULL; > > I don''t suppose we should rebind to dom0, should we?Storing NULL here effectively means re-binding to Dom0. Jan
Ian Campbell
2012-Jan-18 11:33 UTC
Re: [PATCH 13/18] xenstored: support running in minios stubdom
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> A previous versions of this patch has been sent to xen-devel. See > http://lists.xensource.com/archives/html/xen-devel/2009-03/msg01655.html > > Originally-by: Diego Ongaro <diego.ongaro@citrix.com> > Originally-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>There''s a lot of ifdefery here. I don''t have any particularly cunning plans to improve it though :-( One thing which might help is to provide nop versions of functions instead of idef''ing both the definition and callsite. e.g. static void write_pidfile(const char *pidfile) +#ifndef __MINIOS__ stuff +#else + nothing +endif then you avoid another ifdef at the place which calls write_pidfile. I''m not sure that pattern helps with many of the instances here though.> --- > extras/mini-os/include/list.h | 6 ++-- > extras/mini-os/main.c | 4 +++ > stubdom/Makefile | 29 +++++++++++++++++-- > tools/xenstore/Makefile | 9 +++++- > tools/xenstore/tdb.c | 6 ++-- > tools/xenstore/utils.h | 2 + > tools/xenstore/xenstored_core.c | 47 ++++++++++++++++++++++++++++++- > tools/xenstore/xenstored_domain.c | 7 +++++ > tools/xenstore/xenstored_transaction.c | 2 + > 9 files changed, 100 insertions(+), 12 deletions(-) > > diff --git a/extras/mini-os/include/list.h b/extras/mini-os/include/list.h > index a60ae23..4e6a2ac 100644 > --- a/extras/mini-os/include/list.h > +++ b/extras/mini-os/include/list.h > @@ -1,5 +1,5 @@ > -#ifndef _LINUX_LIST_H > -#define _LINUX_LIST_H > +#ifndef _MINIOS_LIST_H > +#define _MINIOS_LIST_H > > /* > * Simple doubly linked list implementation. > @@ -186,5 +186,5 @@ static __inline__ void minios_list_splice(struct minios_list_head *list, struct > n = minios_list_entry(pos->member.next, typeof(*pos), member); \ > &pos->member != (head); \ > pos = n, n = minios_list_entry(n->member.next, typeof(*n), member)) > -#endif /* _LINUX_LIST_H */ > +#endif /* _MINIOS_LIST_H */ > > diff --git a/extras/mini-os/main.c b/extras/mini-os/main.c > index b95b889..cd89849 100644 > --- a/extras/mini-os/main.c > +++ b/extras/mini-os/main.c > @@ -60,6 +60,7 @@ static void call_main(void *p) > * crashing. */ > //sleep(1); > > +#ifndef CONFIG_XENSTORE > #ifndef CONFIG_GRUB > sparse((unsigned long) &__app_bss_start, &__app_bss_end - &__app_bss_start); > #if defined(HAVE_LWIP) && !defined(CONFIG_QEMU) > @@ -67,6 +68,7 @@ static void call_main(void *p) > #endif > #endif > create_thread("pcifront", pcifront_watches, NULL); > +#endif > > #ifdef CONFIG_QEMU > /* Fetch argc, argv from XenStore */ > @@ -169,9 +171,11 @@ void _exit(int ret) > close_all_files(); > __libc_fini_array(); > printk("main returned %d\n", ret); > +#ifndef CONFIG_XENSTORE > #ifdef HAVE_LWIP > stop_networking(); > #endif > +#endif > stop_kernel(); > if (!ret) { > /* No problem, just shutdown. */ > diff --git a/stubdom/Makefile b/stubdom/Makefile > index 3705059..e0a90a9 100644 > --- a/stubdom/Makefile > +++ b/stubdom/Makefile > @@ -74,14 +74,14 @@ TARGET_CPPFLAGS += -I$(XEN_ROOT)/xen/include > > TARGET_LDFLAGS += -nostdlib -L$(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf/lib > > -TARGETS=ioemu c caml grub > +TARGETS=ioemu c caml grub xenstore > > CROSS_MAKE := $(MAKE) DESTDIR> > .PHONY: all > all: build > ifeq ($(STUBDOM_SUPPORTED),1) > -build: genpath ioemu-stubdom c-stubdom pv-grub > +build: genpath ioemu-stubdom c-stubdom pv-grub xenstore-stubdom > else > build: genpath > endif > @@ -262,6 +262,11 @@ mk-headers-$(XEN_TARGET_ARCH): ioemu/linkfarm.stamp > ln -sf $(XEN_ROOT)/tools/libxc/$(XEN_TARGET_ARCH)/*.c . && \ > ln -sf $(XEN_ROOT)/tools/libxc/$(XEN_TARGET_ARCH)/*.h . && \ > ln -sf $(XEN_ROOT)/tools/libxc/$(XEN_TARGET_ARCH)/Makefile . ) > + mkdir -p xenstore > + [ -h xenstore/Makefile ] || ( cd xenstore && \ > + ln -sf $(XEN_ROOT)/tools/xenstore/*.c . && \ > + ln -sf $(XEN_ROOT)/tools/xenstore/*.h . && \ > + ln -sf $(XEN_ROOT)/tools/xenstore/Makefile . ) > $(CROSS_MAKE) -C $(MINI_OS) links > touch mk-headers-$(XEN_TARGET_ARCH) > > @@ -334,6 +339,14 @@ grub: grub-upstream $(CROSS_ROOT) > mkdir -p grub-$(XEN_TARGET_ARCH) > CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(CROSS_MAKE) -C $@ OBJ_DIR=$(CURDIR)/grub-$(XEN_TARGET_ARCH) > > +########## > +# xenstore > +########## > + > +.PHONY: xenstore > +xenstore: $(CROSS_ROOT) > + CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(CROSS_MAKE) -C $@ LWIPDIR=$(CURDIR)/lwip xenstored.a CONFIG_STUBDOM=y > + > ######## > # minios > ######## > @@ -355,12 +368,16 @@ c-stubdom: mini-os-$(XEN_TARGET_ARCH)-c lwip-$(XEN_TARGET_ARCH) libxc c > pv-grub: mini-os-$(XEN_TARGET_ARCH)-grub libxc grub > DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="-DCONFIG_GRUB $(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" $(CROSS_MAKE) -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/grub-$(XEN_TARGET_ARCH)/main.a > > +.PHONY: xenstore-stubdom > +xenstore-stubdom: mini-os-$(XEN_TARGET_ARCH)-xenstore libxc xenstore > + DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="-DCONFIG_XENSTORE $(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" $(MAKE) -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< LWIPDIR=$(CURDIR)/lwip-$(XEN_TARGET_ARCH) APP_OBJS=$(CURDIR)/xenstore/xenstored.a > + > ######### > # install > ######### > > ifeq ($(STUBDOM_SUPPORTED),1) > -install: genpath install-readme install-ioemu install-grub > +install: genpath install-readme install-ioemu install-grub install-xenstore > else > install: genpath > endif > @@ -379,6 +396,10 @@ install-grub: pv-grub > $(INSTALL_DIR) "$(DESTDIR)$(XENFIRMWAREDIR)" > $(INSTALL_DATA) mini-os-$(XEN_TARGET_ARCH)-grub/mini-os.gz "$(DESTDIR)$(XENFIRMWAREDIR)/pv-grub-$(XEN_TARGET_ARCH).gz" > > +install-xenstore: xenstore-stubdom > + $(INSTALL_DIR) "$(DESTDIR)/usr/lib/xen/boot" > + $(INSTALL_PROG) mini-os-$(XEN_TARGET_ARCH)-xenstore/mini-os.gz "$(DESTDIR)/usr/lib/xen/boot/xenstore-stubdom.gz" > + > ####### > # clean > ####### > @@ -390,12 +411,14 @@ clean: > rm -fr mini-os-$(XEN_TARGET_ARCH)-c > rm -fr mini-os-$(XEN_TARGET_ARCH)-caml > rm -fr mini-os-$(XEN_TARGET_ARCH)-grub > + rm -fr mini-os-$(XEN_TARGET_ARCH)-xenstore > $(CROSS_MAKE) -C caml clean > $(CROSS_MAKE) -C c clean > rm -fr grub-$(XEN_TARGET_ARCH) > rm -f $(STUBDOMPATH) > [ ! -d libxc-$(XEN_TARGET_ARCH) ] || $(CROSS_MAKE) -C libxc-$(XEN_TARGET_ARCH) clean > -[ ! -d ioemu ] || $(CROSS_MAKE) -C ioemu clean > + -[ ! -d xenstore ] || $(CROSS_MAKE) -C xenstore clean > > # clean the cross-compilation result > .PHONY: crossclean > diff --git a/tools/xenstore/Makefile b/tools/xenstore/Makefile > index 4facb62..3a061d6 100644 > --- a/tools/xenstore/Makefile > +++ b/tools/xenstore/Makefile > @@ -28,6 +28,10 @@ endif > > ALL_TARGETS = libxenstore.so libxenstore.a clients xs_tdb_dump xenstored > > +ifdef CONFIG_STUBDOM > +CFLAGS += -DNO_SOCKETS=1 -DNO_LOCAL_XENBUS=1 -DNO_SYSLOG=1 -DNO_REOPEN_LOG=1 > +endif > + > .PHONY: all > all: $(ALL_TARGETS) > > @@ -45,10 +49,13 @@ xenstored_probes.o: xenstored_solaris.o > > CFLAGS += -DHAVE_DTRACE=1 > endif > - > + > xenstored: $(XENSTORED_OBJS) > $(CC) $(LDFLAGS) $^ $(LDLIBS_libxenctrl) $(SOCKET_LIBS) -o $@ $(APPEND_LDFLAGS) > > +xenstored.a: $(XENSTORED_OBJS) > + $(AR) cr $@ $^ > + > $(CLIENTS): xenstore > ln -f xenstore $@ > > diff --git a/tools/xenstore/tdb.c b/tools/xenstore/tdb.c > index 639ce6e..75ffd2a 100644 > --- a/tools/xenstore/tdb.c > +++ b/tools/xenstore/tdb.c > @@ -1334,7 +1334,7 @@ static int tdb_next_lock(TDB_CONTEXT *tdb, struct tdb_traverse_lock *tlock, > > /* Iterate through chain */ > while( tlock->off) { > - tdb_off current; > + tdb_off mycurrent; > if (rec_read(tdb, tlock->off, rec) == -1) > goto fail; > > @@ -1352,10 +1352,10 @@ static int tdb_next_lock(TDB_CONTEXT *tdb, struct tdb_traverse_lock *tlock, > } > > /* Try to clean dead ones from old traverses */ > - current = tlock->off; > + mycurrent = tlock->off; > tlock->off = rec->next; > if (!tdb->read_only && > - do_delete(tdb, current, rec) != 0) > + do_delete(tdb, mycurrent, rec) != 0) > goto fail; > } > tdb_unlock(tdb, tlock->hash, F_WRLCK); > diff --git a/tools/xenstore/utils.h b/tools/xenstore/utils.h > index f378343..2effd17 100644 > --- a/tools/xenstore/utils.h > +++ b/tools/xenstore/utils.h > @@ -19,7 +19,9 @@ static inline bool strends(const char *a, const char *b) > return streq(a + strlen(a) - strlen(b), b); > } > > +#ifndef ARRAY_SIZE > #define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0])) > +#endif > > void barf(const char *fmt, ...) __attribute__((noreturn)); > void barf_perror(const char *fmt, ...) __attribute__((noreturn)); > diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c > index 631bfe4..e51f2ad 100644 > --- a/tools/xenstore/xenstored_core.c > +++ b/tools/xenstore/xenstored_core.c > @@ -32,7 +32,9 @@ > #include <stdio.h> > #include <stdarg.h> > #include <stdlib.h> > +#ifndef NO_SYSLOG > #include <syslog.h> > +#endif > #include <string.h> > #include <errno.h> > #include <dirent.h> > @@ -61,13 +63,24 @@ LIST_HEAD(connections); > static int tracefd = -1; > static bool recovery = true; > static bool remove_local = true; > +#ifndef NO_REOPEN_LOG > static int reopen_log_pipe[2]; > +#endif > static char *tracefile = NULL; > static TDB_CONTEXT *tdb_ctx; > > static void corrupt(struct connection *conn, const char *fmt, ...); > static void check_store(void); > > +#ifdef __MINIOS__ > +#define lockf(...) (-ENOSYS) > +#endif > + > +#ifdef NO_SYSLOG > +#define openlog(...) ((void) 0) > +#define syslog(...) ((void) 0) > +#endif > + > #define log(...) \ > do { \ > char *s = talloc_asprintf(NULL, __VA_ARGS__); \ > @@ -92,8 +105,10 @@ TDB_CONTEXT *tdb_context(struct connection *conn) > > bool replace_tdb(const char *newname, TDB_CONTEXT *newtdb) > { > +#ifndef __MINIOS__ > if (rename(newname, xs_daemon_tdb()) != 0) > return false; > +#endif > tdb_close(tdb_ctx); > tdb_ctx = talloc_steal(talloc_autofree_context(), newtdb); > return true; > @@ -195,6 +210,7 @@ void trace_destroy(const void *data, const char *type) > trace("DESTROY %s %p\n", type, data); > } > > +#ifndef NO_REOPEN_LOG > /** > * Signal handler for SIGHUP, which requests that the trace log is reopened > * (in the main loop). A single byte is written to reopen_log_pipe, to awaken > @@ -222,7 +238,7 @@ static void reopen_log(void) > trace("\n***\n"); > } > } > - > +#endif > > static bool write_messages(struct connection *conn) > { > @@ -326,7 +342,9 @@ static int initialize_set(fd_set *inset, fd_set *outset, int sock, int ro_sock, > set_fd(sock, inset, &max); > set_fd(ro_sock, inset, &max); > #endif > +#ifndef NO_REOPEN_LOG > set_fd(reopen_log_pipe[0], inset, &max); > +#endif > > if (xce_handle != NULL) > set_fd(xc_evtchn_fd(xce_handle), inset, &max); > @@ -1415,7 +1433,11 @@ static void accept_connection(int sock, bool canwrite) > } > #endif > > +#ifdef __MINIOS__ > +#define TDB_FLAGS TDB_INTERNAL|TDB_NOLOCK > +#else > #define TDB_FLAGS 0 > +#endif > > /* We create initial nodes manually. */ > static void manual_node(const char *name, const char *child) > @@ -1440,7 +1462,11 @@ static void setup_structure(void) > { > char *tdbname; > tdbname = talloc_strdup(talloc_autofree_context(), xs_daemon_tdb()); > +#ifdef __MINIOS__ > + tdb_ctx = NULL; > +#else > tdb_ctx = tdb_open(tdbname, 0, TDB_FLAGS, O_RDWR, 0); > +#endif > > if (tdb_ctx) { > /* XXX When we make xenstored able to restart, this will have > @@ -1666,6 +1692,7 @@ static void corrupt(struct connection *conn, const char *fmt, ...) > } > > > +#ifndef __MINIOS__ > static void write_pidfile(const char *pidfile) > { > char buf[100]; > @@ -1712,7 +1739,7 @@ static void daemonize(void) > /* Discard our parent''s old-fashioned umask prejudices. */ > umask(0); > } > - > +#endif > > static void usage(void) > { > @@ -1767,7 +1794,11 @@ int main(int argc, char *argv[]) > struct sockaddr_un addr; > #endif > fd_set inset, outset; > +#ifdef __MINIOS__ > + bool dofork = false; > +#else > bool dofork = true; > +#endif > bool outputpid = false; > bool no_domain_init = false; > const char *pidfile = NULL; > @@ -1821,8 +1852,11 @@ int main(int argc, char *argv[]) > if (optind != argc) > barf("%s: No arguments desired", argv[0]); > > +#ifndef NO_REOPEN_LOG > reopen_log(); > +#endif > > +#ifndef __MINIOS__ > /* make sure xenstored directory exists */ > if (mkdir(xs_daemon_rundir(), 0755)) { > if (errno != EEXIST) { > @@ -1844,6 +1878,7 @@ int main(int argc, char *argv[]) > } > if (pidfile) > write_pidfile(pidfile); > +#endif > > /* Talloc leak reports go to stderr, which is closed if we fork. */ > if (!dofork) > @@ -1890,9 +1925,11 @@ int main(int argc, char *argv[]) > barf_perror("Could not listen on sockets"); > #endif > > +#ifndef NO_REOPEN_LOG > if (pipe(reopen_log_pipe)) { > barf_perror("pipe"); > } > +#endif > > /* Setup the database */ > setup_structure(); > @@ -1921,7 +1958,9 @@ int main(int argc, char *argv[]) > xprintf = trace; > } > > +#ifndef NO_REOPEN_LOG > signal(SIGHUP, trigger_reopen_log); > +#endif > > if (xce_handle != NULL) > evtchn_fd = xc_evtchn_fd(xce_handle); > @@ -1929,8 +1968,10 @@ int main(int argc, char *argv[]) > /* Get ready to listen to the tools. */ > max = initialize_set(&inset, &outset, *sock, *ro_sock, &timeout); > > +#ifndef __MINIOS__ > /* Tell the kernel we''re up and running. */ > xenbus_notify_running(); > +#endif > > /* Main loop. */ > for (;;) { > @@ -1942,12 +1983,14 @@ int main(int argc, char *argv[]) > barf_perror("Select failed"); > } > > +#ifndef NO_REOPEN_LOG > if (FD_ISSET(reopen_log_pipe[0], &inset)) { > char c; > if (read(reopen_log_pipe[0], &c, 1) != 1) > barf_perror("read failed"); > reopen_log(); > } > +#endif > > #ifndef NO_SOCKETS > if (FD_ISSET(*sock, &inset)) > diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c > index 0b8353b..811ae3c 100644 > --- a/tools/xenstore/xenstored_domain.c > +++ b/tools/xenstore/xenstored_domain.c > @@ -588,6 +588,12 @@ void restore_existing_connections(void) > { > } > > +#ifdef __MINIOS__ > +static inline int dom0_init(void) > +{ > + return 0; > +} > +#else > static int dom0_init(void) > { > evtchn_port_t port; > @@ -611,6 +617,7 @@ static int dom0_init(void) > > return 0; > } > +#endif > > void domain_init(void) > { > diff --git a/tools/xenstore/xenstored_transaction.c b/tools/xenstore/xenstored_transaction.c > index 380c691..c59acfb 100644 > --- a/tools/xenstore/xenstored_transaction.c > +++ b/tools/xenstore/xenstored_transaction.c > @@ -120,7 +120,9 @@ static int destroy_transaction(void *_transaction) > trace_destroy(trans, "transaction"); > if (trans->tdb) > tdb_close(trans->tdb); > +#ifndef __MINIOS__ > unlink(trans->tdb_name); > +#endif > return 0; > } > > -- > 1.7.7.5 > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel
Ian Campbell
2012-Jan-18 11:35 UTC
Re: [PATCH 15/18] xenstored: add --event parameter for bootstrapping
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> When xenstored is run in a minios domain, it needs a bootstrap > connection to dom0 so that additional domain introduce messages can be > sent to it. > > Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>Acked-by: Ian Campbell <ian.campbell@citrix.com>> --- > tools/xenstore/xenstored_core.c | 5 +++++ > tools/xenstore/xenstored_core.h | 1 + > tools/xenstore/xenstored_domain.c | 13 ++++++++++++- > 3 files changed, 18 insertions(+), 1 deletions(-) > > diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c > index e51f2ad..4ec63f1 100644 > --- a/tools/xenstore/xenstored_core.c > +++ b/tools/xenstore/xenstored_core.c > @@ -1771,6 +1771,7 @@ static struct option options[] = { > { "no-domain-init", 0, NULL, ''D'' }, > { "entry-nb", 1, NULL, ''E'' }, > { "pid-file", 1, NULL, ''F'' }, > + { "event", 1, NULL, ''e'' }, > { "help", 0, NULL, ''H'' }, > { "no-fork", 0, NULL, ''N'' }, > { "output-pid", 0, NULL, ''P'' }, > @@ -1784,6 +1785,7 @@ static struct option options[] = { > { NULL, 0, NULL, 0 } }; > > extern void dump_conn(struct connection *conn); > +int dom0_event = 0; > > int main(int argc, char *argv[]) > { > @@ -1847,6 +1849,9 @@ int main(int argc, char *argv[]) > case ''W'': > quota_nb_watch_per_domain = strtol(optarg, NULL, 10); > break; > + case ''e'': > + dom0_event = strtol(optarg, NULL, 10); > + break; > } > } > if (optind != argc) > diff --git a/tools/xenstore/xenstored_core.h b/tools/xenstore/xenstored_core.h > index c487089..d3040ba 100644 > --- a/tools/xenstore/xenstored_core.h > +++ b/tools/xenstore/xenstored_core.h > @@ -168,6 +168,7 @@ void trace(const char *fmt, ...); > void dtrace_io(const struct connection *conn, const struct buffered_data *data, int out); > > extern int event_fd; > +extern int dom0_event; > > /* Map the kernel''s xenstore page. */ > void *xenbus_map(void); > diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c > index aca2149..648eb1d 100644 > --- a/tools/xenstore/xenstored_domain.c > +++ b/tools/xenstore/xenstored_domain.c > @@ -593,8 +593,19 @@ void restore_existing_connections(void) > } > > #ifdef __MINIOS__ > -static inline int dom0_init(void) > +static int dom0_init(void) > { > + struct domain *domain; > + int domid = 0; > + evtchn_port_t port = dom0_event; > + > + domain = new_domain(NULL, domid, port); > + domain->interface = xc_gnttab_map_grant_ref(*xcg_handle, domid, > + GNTTAB_RESERVED_XENSTORE, PROT_READ|PROT_WRITE); > + talloc_steal(domain->conn, domain); > + > + xc_evtchn_notify(xce_handle, domain->port); > + > return 0; > } > #else
Ian Campbell
2012-Jan-18 11:44 UTC
Re: [PATCH 02/18] xen: allow global VIRQ handlers to be delegated to other domains
On Wed, 2012-01-18 at 11:28 +0000, Jan Beulich wrote:> >>> On 18.01.12 at 11:39, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote: > >> > >> +static void clear_global_virq_handlers(struct domain *d) > >> +{ > >> + uint32_t virq; > >> + int put_count = 0; > >> + > >> + spin_lock(&global_virq_handlers_lock); > >> + > >> + for (virq = 0; virq < NR_VIRQS; virq++) { > >> + if (global_virq_handlers[virq] == d) { > >> + global_virq_handlers[virq] = NULL; > > > > I don''t suppose we should rebind to dom0, should we? > > Storing NULL here effectively means re-binding to Dom0.Oh, good, thanks. In that case: Acked-by: Ian Campbell <ian.campbell@citrix.com> Ian.> > Jan >
Ian Campbell
2012-Jan-18 11:44 UTC
Re: [PATCH 16/18] xenstored: use domain_is_unprivileged instead of checking conn->id
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> This centralizes all the permission checking for privileged domains in > preparation for allowing domains other than dom0 to be privileged. > > Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > --- > tools/xenstore/xenstored_core.c | 6 +++--- > tools/xenstore/xenstored_domain.c | 8 ++++---- > 2 files changed, 7 insertions(+), 7 deletions(-) > > diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c > index 4ec63f1..eea5fd6 100644 > --- a/tools/xenstore/xenstored_core.c > +++ b/tools/xenstore/xenstored_core.c > @@ -488,7 +488,7 @@ static enum xs_perm_type perm_for_conn(struct connection *conn, > mask &= ~XS_PERM_WRITE; > > /* Owners and tools get it all... */ > - if (!conn->id || perms[0].id == conn->id > + if (!domain_is_unprivileged(conn) || perms[0].id == conn->iddomain_is_unprivileged is: conn && conn->domain && conn->domain->domid != 0 which isn''t quite the same as the code being replaced. The difference appears to be the conn->id is valid for socket connections as well as domain connections whereas conn->domain is only present for domain connections. Does this change not mean that, for the dom0-process xenstored configuration we now treat socket based connections as unprivileged where previously they would be unprivileged?> || (conn->target && perms[0].id == conn->target->id)) > return (XS_PERM_READ|XS_PERM_WRITE|XS_PERM_OWNER) & mask; > > @@ -826,11 +826,11 @@ static struct node *construct_node(struct connection *conn, const char *name) > node->tdb = tdb_context(conn); > node->name = talloc_strdup(node, name); > > - /* Inherit permissions, except domains own what they create */ > + /* Inherit permissions, except unprivileged domains own what they create */ > node->num_perms = parent->num_perms; > node->perms = talloc_memdup(node, parent->perms, > node->num_perms * sizeof(node->perms[0])); > - if (conn && conn->id) > + if (domain_is_unprivileged(conn)) > node->perms[0].id = conn->id; > > /* No children, no data */ > diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c > index 648eb1d..5f4a09e 100644 > --- a/tools/xenstore/xenstored_domain.c > +++ b/tools/xenstore/xenstored_domain.c > @@ -336,7 +336,7 @@ void do_introduce(struct connection *conn, struct buffered_data *in) > return; > } > > - if (conn->id != 0 || !conn->can_write) { > + if (domain_is_unprivileged(conn) || !conn->can_write) { > send_error(conn, EACCES); > return; > } > @@ -413,7 +413,7 @@ void do_set_target(struct connection *conn, struct buffered_data *in) > return; > } > > - if (conn->id != 0 || !conn->can_write) { > + if (domain_is_unprivileged(conn) || !conn->can_write) { > send_error(conn, EACCES); > return; > } > @@ -465,7 +465,7 @@ void do_release(struct connection *conn, const char *domid_str) > return; > } > > - if (conn->id != 0) { > + if (domain_is_unprivileged(conn)) { > send_error(conn, EACCES); > return; > } > @@ -502,7 +502,7 @@ void do_resume(struct connection *conn, const char *domid_str) > return; > } > > - if (conn->id != 0) { > + if (domain_is_unprivileged(conn)) { > send_error(conn, EACCES); > return; > }
Ian Campbell
2012-Jan-18 11:48 UTC
Re: [PATCH 17/18] xenstored: add --priv-domid parameter
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> This parameter identifies an alternative service domain which has > superuser access to the xenstore database, which is currently required > to set up a new domain''s xenstore entries.Is this equivalent to dom0 adding write permissions to various paths for that domain as it builds it or is there more to it than that. I know that the determination of "various paths" is non-trivial, so I''m not actually suggesting that is a better approach.> > Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > --- > tools/xenstore/xenstored_core.c | 5 +++++ > tools/xenstore/xenstored_core.h | 1 + > tools/xenstore/xenstored_domain.c | 2 +- > 3 files changed, 7 insertions(+), 1 deletions(-) > > diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c > index eea5fd6..9d087de 100644 > --- a/tools/xenstore/xenstored_core.c > +++ b/tools/xenstore/xenstored_core.c > @@ -1774,6 +1774,7 @@ static struct option options[] = { > { "event", 1, NULL, ''e'' }, > { "help", 0, NULL, ''H'' }, > { "no-fork", 0, NULL, ''N'' }, > + { "priv-domid", 1, NULL, ''p'' }, > { "output-pid", 0, NULL, ''P'' }, > { "entry-size", 1, NULL, ''S'' }, > { "trace-file", 1, NULL, ''T'' }, > @@ -1786,6 +1787,7 @@ static struct option options[] = { > > extern void dump_conn(struct connection *conn); > int dom0_event = 0; > +int priv_domid = 0; > > int main(int argc, char *argv[]) > { > @@ -1852,6 +1854,9 @@ int main(int argc, char *argv[]) > case ''e'': > dom0_event = strtol(optarg, NULL, 10); > break; > + case ''p'': > + priv_domid = strtol(optarg, NULL, 10); > + break; > } > } > if (optind != argc) > diff --git a/tools/xenstore/xenstored_core.h b/tools/xenstore/xenstored_core.h > index d3040ba..03e2e48 100644 > --- a/tools/xenstore/xenstored_core.h > +++ b/tools/xenstore/xenstored_core.h > @@ -169,6 +169,7 @@ void dtrace_io(const struct connection *conn, const struct buffered_data *data, > > extern int event_fd; > extern int dom0_event; > +extern int priv_domid; > > /* Map the kernel''s xenstore page. */ > void *xenbus_map(void); > diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c > index 5f4a09e..46bcf3e 100644 > --- a/tools/xenstore/xenstored_domain.c > +++ b/tools/xenstore/xenstored_domain.c > @@ -241,7 +241,7 @@ bool domain_can_read(struct connection *conn) > > bool domain_is_unprivileged(struct connection *conn) > { > - return (conn && conn->domain && conn->domain->domid != 0); > + return (conn && conn->domain && conn->domain->domid != 0 && conn->domain->domid != priv_domid); > } > > bool domain_can_write(struct connection *conn)
On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote:> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>I think eventually we should aim to have this be a rough drop in replacement for /usr/sbin/*xenstored (similar params where it makes sense etc) but this will do for now: Acked-by: Ian Campbell <ian.campbell@citrix.com>> --- > tools/include/xen-sys/Linux/xenbus_dev.h | 55 +++++++++++++++++ > tools/xenstore/Makefile | 9 ++- > tools/xenstore/init-xenstore-domain.c | 96 ++++++++++++++++++++++++++++++ > 3 files changed, 158 insertions(+), 2 deletions(-) > create mode 100644 tools/include/xen-sys/Linux/xenbus_dev.h > create mode 100644 tools/xenstore/init-xenstore-domain.c > > diff --git a/tools/include/xen-sys/Linux/xenbus_dev.h b/tools/include/xen-sys/Linux/xenbus_dev.h > new file mode 100644 > index 0000000..b107be9 > --- /dev/null > +++ b/tools/include/xen-sys/Linux/xenbus_dev.h > @@ -0,0 +1,55 @@ > +/****************************************************************************** > + * evtchn.h > + * > + * Interface to /dev/xen/xenbus_backend. > + * > + * Copyright (c) 2011 Bastian Blank <waldi@debian.org> > + * > + * This program is free software; you can redistribute it and/or > + * modify it under the terms of the GNU General Public License version 2 > + * as published by the Free Software Foundation; or, when distributed > + * separately from the Linux kernel or incorporated into other > + * software packages, subject to the following license: > + * > + * Permission is hereby granted, free of charge, to any person obtaining a copy > + * of this source file (the "Software"), to deal in the Software without > + * restriction, including without limitation the rights to use, copy, modify, > + * merge, publish, distribute, sublicense, and/or sell copies of the Software, > + * and to permit persons to whom the Software is furnished to do so, subject to > + * the following conditions: > + * > + * The above copyright notice and this permission notice shall be included in > + * all copies or substantial portions of the Software. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE > + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING > + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS > + * IN THE SOFTWARE. > + */ > + > +#ifndef __LINUX_XEN_XENBUS_DEV_H__ > +#define __LINUX_XEN_XENBUS_DEV_H__ > + > +#include <linux/ioctl.h> > + > +#define IOCTL_XENBUS_BACKEND_EVTCHN \ > + _IOC(_IOC_NONE, ''B'', 0, 0) > + > +#define IOCTL_XENBUS_ALLOC \ > + _IOC(_IOC_NONE, ''B'', 1, sizeof(struct ioctl_xenbus_alloc)) > +struct ioctl_xenbus_alloc { > + /* IN */ > + /* The domain ID (must exist) for xenstore */ > + uint16_t dom; > + uint16_t pad; > + /* OUT */ > + /* The port allocated for xenbus communication */ > + uint32_t port; > + /* Always set to GNTTAB_RESERVED_XENSTORE */ > + uint32_t grant_ref; > +}; > + > +#endif /* __LINUX_XEN_XENBUS_DEV_H__ */ > diff --git a/tools/xenstore/Makefile b/tools/xenstore/Makefile > index 3a061d6..9b411a5 100644 > --- a/tools/xenstore/Makefile > +++ b/tools/xenstore/Makefile > @@ -26,7 +26,7 @@ LIBXENSTORE := libxenstore.a > xenstore xenstore-control: CFLAGS += -static > endif > > -ALL_TARGETS = libxenstore.so libxenstore.a clients xs_tdb_dump xenstored > +ALL_TARGETS = libxenstore.so libxenstore.a clients xs_tdb_dump xenstored init-xenstore-domain > > ifdef CONFIG_STUBDOM > CFLAGS += -DNO_SOCKETS=1 -DNO_LOCAL_XENBUS=1 -DNO_SYSLOG=1 -DNO_REOPEN_LOG=1 > @@ -50,6 +50,11 @@ xenstored_probes.o: xenstored_solaris.o > CFLAGS += -DHAVE_DTRACE=1 > endif > > +init-xenstore-domain.o: CFLAGS += $(CFLAGS_libxenguest) > + > +init-xenstore-domain: init-xenstore-domain.o $(LIBXENSTORE) > + $(CC) $(LDFLAGS) $^ $(LDLIBS_libxenctrl) $(LDLIBS_libxenguest) $(LDLIBS_libxenstore) -o $@ $(APPEND_LDFLAGS) > + > xenstored: $(XENSTORED_OBJS) > $(CC) $(LDFLAGS) $^ $(LDLIBS_libxenctrl) $(SOCKET_LIBS) -o $@ $(APPEND_LDFLAGS) > > @@ -85,7 +90,7 @@ libxenstore.a: xs.o xs_lib.o > clean: > rm -f *.a *.o *.opic *.so* xenstored_probes.h > rm -f xenstored xs_random xs_stress xs_crashme > - rm -f xs_tdb_dump xenstore-control > + rm -f xs_tdb_dump xenstore-control init-xenstore-domain > rm -f xenstore $(CLIENTS) > $(RM) $(DEPS) > > diff --git a/tools/xenstore/init-xenstore-domain.c b/tools/xenstore/init-xenstore-domain.c > new file mode 100644 > index 0000000..30f6c9b > --- /dev/null > +++ b/tools/xenstore/init-xenstore-domain.c > @@ -0,0 +1,96 @@ > +#include <fcntl.h> > +#include <stdio.h> > +#include <string.h> > +#include <stdint.h> > +#include <stdlib.h> > +#include <sys/ioctl.h> > +#include <sys/mman.h> > +#include <xenctrl.h> > +#include <xc_dom.h> > +#include <xs.h> > +#include <xen/sys/xenbus_dev.h> > + > +static uint32_t domid = -1; > + > +static int build(xc_interface *xch, char** argv) > +{ > + char cmdline[512]; > + uint32_t ssid; > + xen_domain_handle_t handle = { 0 }; > + int rv; > + int xs_fd = open("/dev/xen/xenbus_backend", O_RDWR); > + struct xc_dom_image *dom; > + int maxmem = atoi(argv[2]); > + int limit_kb = (maxmem + 1)*1024; > + struct ioctl_xenbus_alloc alloc; > + > + rv = xc_flask_context_to_sid(xch, argv[3], strlen(argv[3]), &ssid); > + if (rv) return rv; > + rv = xc_domain_create(xch, ssid, handle, 0, &domid); > + if (rv) return rv; > + rv = xc_domain_max_vcpus(xch, domid, 1); > + if (rv) return rv; > + rv = xc_domain_setmaxmem(xch, domid, limit_kb); > + if (rv) return rv; > + rv = xc_domain_set_memmap_limit(xch, domid, limit_kb); > + if (rv) return rv; > + > + alloc.dom = domid; > + rv = ioctl(xs_fd, IOCTL_XENBUS_ALLOC, &alloc); > + if (rv) return rv; > + snprintf(cmdline, 512, "--event %d", alloc.port); > + > + dom = xc_dom_allocate(xch, cmdline, NULL); > + rv = xc_dom_kernel_file(dom, argv[1]); > + if (rv) return rv; > + rv = xc_dom_boot_xen_init(dom, xch, domid); > + if (rv) return rv; > + rv = xc_dom_parse_image(dom); > + if (rv) return rv; > + rv = xc_dom_mem_init(dom, maxmem); > + if (rv) return rv; > + rv = xc_dom_boot_mem_init(dom); > + if (rv) return rv; > + rv = xc_dom_build_image(dom); > + if (rv) return rv; > + rv = xc_dom_boot_image(dom); > + if (rv) return rv; > + > + xc_dom_release(dom); > + > + rv = xc_domain_set_virq_handler(xch, domid, VIRQ_DOM_EXC); > + if (rv) return rv; > + rv = xc_domain_unpause(xch, domid); > + if (rv) return rv; > + > + return 0; > +} > + > +int main(int argc, char** argv) > +{ > + xc_interface *xch; > + struct xs_handle *xsh; > + char buf[16]; > + int rv; > + > + if (argc != 4) { > + printf("Use: %s <xenstore-kernel> <memory_mb> <flask-label>\n", argv[0]); > + return 2; > + } > + > + xch = xc_interface_open(NULL, NULL, 0); > + if (!xch) return 1; > + > + rv = build(xch, argv); > + > + xc_interface_close(xch); > + > + if (rv) return 1; > + > + xsh = xs_open(0); > + rv = snprintf(buf, 16, "%d", domid); > + xs_write(xsh, XBT_NULL, "/tool/xenstored/domid", buf, rv); > + xs_daemon_close(xsh); > + > + return 0; > +}
Ian Campbell
2012-Jan-18 12:07 UTC
Re: [PATCH] xenbus: Add support for xenbus backend in stub domain
On Thu, 2012-01-12 at 23:36 +0000, Daniel De Graaf wrote:> This adds an ioctl to the /dev/xen/xenbus_backend device allowing the > xenbus backend to be started after the kernel has booted. This is > intended to allow dom0 to start another domain to run xenstore.Does this new ioctl need to be made oneshot? What happens if you call it twice? Also it should be disabled once a local xenstored has connected shouldn''t it? Diego''s original patch handled that sort of thing, any reason not to simply forward port it?> > Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > --- > drivers/xen/xenbus/xenbus_comms.c | 6 ++++ > drivers/xen/xenbus/xenbus_comms.h | 1 + > drivers/xen/xenbus/xenbus_dev_backend.c | 44 +++++++++++++++++++++++++++++++ > include/xen/grant_table.h | 2 + > include/xen/xenbus_dev.h | 14 ++++++++++ > 5 files changed, 67 insertions(+), 0 deletions(-) > > diff --git a/drivers/xen/xenbus/xenbus_comms.c b/drivers/xen/xenbus/xenbus_comms.c > index 2eff7a6..52fe7ad 100644 > --- a/drivers/xen/xenbus/xenbus_comms.c > +++ b/drivers/xen/xenbus/xenbus_comms.c > @@ -234,3 +234,9 @@ int xb_init_comms(void) > > return 0; > } > + > +void xb_deinit_comms(void) > +{ > + unbind_from_irqhandler(xenbus_irq, &xb_waitq); > + xenbus_irq = 0; > +} > diff --git a/drivers/xen/xenbus/xenbus_comms.h b/drivers/xen/xenbus/xenbus_comms.h > index 6e42800..c8abd3b 100644 > --- a/drivers/xen/xenbus/xenbus_comms.h > +++ b/drivers/xen/xenbus/xenbus_comms.h > @@ -35,6 +35,7 @@ > > int xs_init(void); > int xb_init_comms(void); > +void xb_deinit_comms(void); > > /* Low level routines. */ > int xb_write(const void *data, unsigned len); > diff --git a/drivers/xen/xenbus/xenbus_dev_backend.c b/drivers/xen/xenbus/xenbus_dev_backend.c > index 3d3be78..854315c 100644 > --- a/drivers/xen/xenbus/xenbus_dev_backend.c > +++ b/drivers/xen/xenbus/xenbus_dev_backend.c > @@ -8,7 +8,11 @@ > > #include <xen/xen.h> > #include <xen/page.h> > +#include <xen/xenbus.h> > #include <xen/xenbus_dev.h> > +#include <xen/grant_table.h> > +#include <xen/events.h> > +#include <asm/xen/hypervisor.h> > > #include "xenbus_comms.h" > > @@ -22,6 +26,43 @@ static int xenbus_backend_open(struct inode *inode, struct file *filp) > return nonseekable_open(inode, filp); > } > > +static long xenbus_alloc(void __user * data) > +{ > + struct ioctl_xenbus_alloc op; > + struct evtchn_alloc_unbound arg; > + int err; > + > + if (copy_from_user(&op, data, sizeof(op))) > + return -EFAULT; > + > + gnttab_grant_foreign_access_ref(GNTTAB_RESERVED_XENSTORE, op.dom, > + virt_to_mfn(xen_store_interface), 0 /* writable */); > + > + arg.dom = DOMID_SELF; > + arg.remote_dom = op.dom; > + > + err = HYPERVISOR_event_channel_op(EVTCHNOP_alloc_unbound, &arg); > + if (err) > + return err; > + > + op.port = arg.port; > + op.grant_ref = GNTTAB_RESERVED_XENSTORE; > + > + xs_suspend(); > + > + if (xen_store_evtchn > 0) > + xb_deinit_comms(); > + > + xen_store_evtchn = arg.port; > + > + xs_resume(); > + > + if (copy_to_user(data, &op, sizeof(op))) > + return -EFAULT; > + > + return 0; > +} > + > static long xenbus_backend_ioctl(struct file *file, unsigned int cmd, unsigned long data) > { > if (!capable(CAP_SYS_ADMIN)) > @@ -33,6 +74,9 @@ static long xenbus_backend_ioctl(struct file *file, unsigned int cmd, unsigned l > return xen_store_evtchn; > return -ENODEV; > > + case IOCTL_XENBUS_ALLOC: > + return xenbus_alloc((void __user *)data); > + > default: > return -ENOTTY; > } > diff --git a/include/xen/grant_table.h b/include/xen/grant_table.h > index 15f8a00..11e27c3 100644 > --- a/include/xen/grant_table.h > +++ b/include/xen/grant_table.h > @@ -46,6 +46,8 @@ > > #include <xen/features.h> > > +#define GNTTAB_RESERVED_XENSTORE 1 > + > /* NR_GRANT_FRAMES must be less than or equal to that configured in Xen */ > #define NR_GRANT_FRAMES 4 > > diff --git a/include/xen/xenbus_dev.h b/include/xen/xenbus_dev.h > index ac5f0fe..b107be9 100644 > --- a/include/xen/xenbus_dev.h > +++ b/include/xen/xenbus_dev.h > @@ -38,4 +38,18 @@ > #define IOCTL_XENBUS_BACKEND_EVTCHN \ > _IOC(_IOC_NONE, ''B'', 0, 0) > > +#define IOCTL_XENBUS_ALLOC \ > + _IOC(_IOC_NONE, ''B'', 1, sizeof(struct ioctl_xenbus_alloc)) > +struct ioctl_xenbus_alloc { > + /* IN */ > + /* The domain ID (must exist) for xenstore */ > + uint16_t dom; > + uint16_t pad; > + /* OUT */ > + /* The port allocated for xenbus communication */ > + uint32_t port; > + /* Always set to GNTTAB_RESERVED_XENSTORE */ > + uint32_t grant_ref; > +}; > + > #endif /* __LINUX_XEN_XENBUS_DEV_H__ */
Daniel De Graaf
2012-Jan-18 14:41 UTC
Re: [PATCH 17/18] xenstored: add --priv-domid parameter
On 01/18/2012 06:48 AM, Ian Campbell wrote:> On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote: >> This parameter identifies an alternative service domain which has >> superuser access to the xenstore database, which is currently required >> to set up a new domain''s xenstore entries. > > Is this equivalent to dom0 adding write permissions to various paths for > that domain as it builds it or is there more to it than that. > > I know that the determination of "various paths" is non-trivial, so I''m > not actually suggesting that is a better approach. >It''s more: the domain builder needs to create entries owned by the new domain, and similar to UNIX chown() can only be called by the superuser. The domain builder also currently relies on the fact that new keys it creates inherit the parent''s ownership instead of being owned by dom0. The introduce operation is also privileged.>> >> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> >> --- >> tools/xenstore/xenstored_core.c | 5 +++++ >> tools/xenstore/xenstored_core.h | 1 + >> tools/xenstore/xenstored_domain.c | 2 +- >> 3 files changed, 7 insertions(+), 1 deletions(-) >> >> diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c >> index eea5fd6..9d087de 100644 >> --- a/tools/xenstore/xenstored_core.c >> +++ b/tools/xenstore/xenstored_core.c >> @@ -1774,6 +1774,7 @@ static struct option options[] = { >> { "event", 1, NULL, ''e'' }, >> { "help", 0, NULL, ''H'' }, >> { "no-fork", 0, NULL, ''N'' }, >> + { "priv-domid", 1, NULL, ''p'' }, >> { "output-pid", 0, NULL, ''P'' }, >> { "entry-size", 1, NULL, ''S'' }, >> { "trace-file", 1, NULL, ''T'' }, >> @@ -1786,6 +1787,7 @@ static struct option options[] = { >> >> extern void dump_conn(struct connection *conn); >> int dom0_event = 0; >> +int priv_domid = 0; >> >> int main(int argc, char *argv[]) >> { >> @@ -1852,6 +1854,9 @@ int main(int argc, char *argv[]) >> case ''e'': >> dom0_event = strtol(optarg, NULL, 10); >> break; >> + case ''p'': >> + priv_domid = strtol(optarg, NULL, 10); >> + break; >> } >> } >> if (optind != argc) >> diff --git a/tools/xenstore/xenstored_core.h b/tools/xenstore/xenstored_core.h >> index d3040ba..03e2e48 100644 >> --- a/tools/xenstore/xenstored_core.h >> +++ b/tools/xenstore/xenstored_core.h >> @@ -169,6 +169,7 @@ void dtrace_io(const struct connection *conn, const struct buffered_data *data, >> >> extern int event_fd; >> extern int dom0_event; >> +extern int priv_domid; >> >> /* Map the kernel''s xenstore page. */ >> void *xenbus_map(void); >> diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c >> index 5f4a09e..46bcf3e 100644 >> --- a/tools/xenstore/xenstored_domain.c >> +++ b/tools/xenstore/xenstored_domain.c >> @@ -241,7 +241,7 @@ bool domain_can_read(struct connection *conn) >> >> bool domain_is_unprivileged(struct connection *conn) >> { >> - return (conn && conn->domain && conn->domain->domid != 0); >> + return (conn && conn->domain && conn->domain->domid != 0 && conn->domain->domid != priv_domid); >> } >> >> bool domain_can_write(struct connection *conn) >-- Daniel De Graaf National Security Agency
Daniel De Graaf
2012-Jan-18 14:44 UTC
Re: [PATCH] xenbus: Add support for xenbus backend in stub domain
On 01/18/2012 07:07 AM, Ian Campbell wrote:> On Thu, 2012-01-12 at 23:36 +0000, Daniel De Graaf wrote: >> This adds an ioctl to the /dev/xen/xenbus_backend device allowing the >> xenbus backend to be started after the kernel has booted. This is >> intended to allow dom0 to start another domain to run xenstore. > > Does this new ioctl need to be made oneshot? What happens if you call it > twice? > > Also it should be disabled once a local xenstored has connected > shouldn''t it? > > Diego''s original patch handled that sort of thing, any reason not to > simply forward port it? >My original patch was trying to be more flexible in allowing xenstored to be created later (migrating watches), or restarted in a different domain. However, since the current version lacks that ability the oneshot safeguards from Diego''s patch should probably be added.
Ian Campbell
2012-Jan-18 14:47 UTC
Re: [PATCH 17/18] xenstored: add --priv-domid parameter
On Wed, 2012-01-18 at 14:41 +0000, Daniel De Graaf wrote:> On 01/18/2012 06:48 AM, Ian Campbell wrote: > > On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote: > >> This parameter identifies an alternative service domain which has > >> superuser access to the xenstore database, which is currently required > >> to set up a new domain''s xenstore entries. > > > > Is this equivalent to dom0 adding write permissions to various paths for > > that domain as it builds it or is there more to it than that. > > > > I know that the determination of "various paths" is non-trivial, so I''m > > not actually suggesting that is a better approach. > > > > It''s more: the domain builder needs to create entries owned by the new > domain, and similar to UNIX chown() can only be called by the superuser. > The domain builder also currently relies on the fact that new keys it > creates inherit the parent''s ownership instead of being owned by dom0. > The introduce operation is also privileged.Thanks for explaining. I wonder if there is somewhere this can be usefully written down so that "privileged" is well defined? docs/misc/xenstore.txt seems to be more about the wire protocol than the underlying semantics. Perhaps someone on list can suggest a suitable place?> > >> > >> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > >> --- > >> tools/xenstore/xenstored_core.c | 5 +++++ > >> tools/xenstore/xenstored_core.h | 1 + > >> tools/xenstore/xenstored_domain.c | 2 +- > >> 3 files changed, 7 insertions(+), 1 deletions(-) > >> > >> diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c > >> index eea5fd6..9d087de 100644 > >> --- a/tools/xenstore/xenstored_core.c > >> +++ b/tools/xenstore/xenstored_core.c > >> @@ -1774,6 +1774,7 @@ static struct option options[] = { > >> { "event", 1, NULL, ''e'' }, > >> { "help", 0, NULL, ''H'' }, > >> { "no-fork", 0, NULL, ''N'' }, > >> + { "priv-domid", 1, NULL, ''p'' }, > >> { "output-pid", 0, NULL, ''P'' }, > >> { "entry-size", 1, NULL, ''S'' }, > >> { "trace-file", 1, NULL, ''T'' }, > >> @@ -1786,6 +1787,7 @@ static struct option options[] = { > >> > >> extern void dump_conn(struct connection *conn); > >> int dom0_event = 0; > >> +int priv_domid = 0; > >> > >> int main(int argc, char *argv[]) > >> { > >> @@ -1852,6 +1854,9 @@ int main(int argc, char *argv[]) > >> case ''e'': > >> dom0_event = strtol(optarg, NULL, 10); > >> break; > >> + case ''p'': > >> + priv_domid = strtol(optarg, NULL, 10); > >> + break; > >> } > >> } > >> if (optind != argc) > >> diff --git a/tools/xenstore/xenstored_core.h b/tools/xenstore/xenstored_core.h > >> index d3040ba..03e2e48 100644 > >> --- a/tools/xenstore/xenstored_core.h > >> +++ b/tools/xenstore/xenstored_core.h > >> @@ -169,6 +169,7 @@ void dtrace_io(const struct connection *conn, const struct buffered_data *data, > >> > >> extern int event_fd; > >> extern int dom0_event; > >> +extern int priv_domid; > >> > >> /* Map the kernel''s xenstore page. */ > >> void *xenbus_map(void); > >> diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c > >> index 5f4a09e..46bcf3e 100644 > >> --- a/tools/xenstore/xenstored_domain.c > >> +++ b/tools/xenstore/xenstored_domain.c > >> @@ -241,7 +241,7 @@ bool domain_can_read(struct connection *conn) > >> > >> bool domain_is_unprivileged(struct connection *conn) > >> { > >> - return (conn && conn->domain && conn->domain->domid != 0); > >> + return (conn && conn->domain && conn->domain->domid != 0 && conn->domain->domid != priv_domid); > >> } > >> > >> bool domain_can_write(struct connection *conn) > > > >
Daniel De Graaf
2012-Jan-18 14:56 UTC
Re: [PATCH 01/18] xen: reinstate previously unused XENMEM_remove_from_physmap hypercall
On 01/18/2012 05:36 AM, Ian Campbell wrote:> On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote: >> From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> >> >> This patch reinstates the XENMEM_remove_from_physmap hypercall >> which was removed in 19041:ee62aaafff46 because it was not used. >> >> However, is now needed in order to support xenstored stub domains. >> The xenstored stub domain is not priviliged like dom0 and so cannot >> unilaterally map the xenbus page of other guests into it''s address >> space. Therefore, before creating a domU the domain builder needs to >> seed its grant table with a grant ref allowing the xenstored stub >> domain to access the new domU''s xenbus page. >> >> At present domU''s do not start with their grant table mapped. >> Instead it gets mapped when the guest requests a grant table from >> the hypervisor. >> >> In order to seed the grant table, the domain builder first needs to >> map it into dom0 address space. But the hypercall to do this >> requires a gpfn (guest pfn), which is an mfn for PV guest, but a pfn >> for HVM guests. Therfore, in order to seed the grant table of an >> HVM guest, dom0 needs to *temporarily* map it into the guest''s >> "physical" address space. >> >> Hence the need to reinstate the XENMEM_remove_from_physmap hypercall. >> >> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> >> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > > Acked-by: Ian Campbell <ian.campbell@citrix.com> (modulo Jan''s comment > about ordering in xlat.lst) > > BTW, since Alex and Diego have subsequently left Citrix you could take > my Acked-by''s in this series as Signed-of-by on behalf of Citrix. I''ve > no idea if that''s necessary though, I expect not. > > Ian. >I''m not an expert in this area, but this is how I read it: the portion of the path authored by Alex/Diego was already signed-off when they were posted, so since the current patches are derived works from them the sign-off may need to stay in order to allow me to sign off because I cannot claim copyright on all of the content. Assuming Citrix actually owns the copyright on the patches, your Ack may suffice to replace the sign-off for this purpose. I guess my real question here would be: should the sign-off from Alex and Diego remain on these patches in addition to your Ack?
Ian Campbell
2012-Jan-18 16:06 UTC
Re: [PATCH 01/18] xen: reinstate previously unused XENMEM_remove_from_physmap hypercall
On Wed, 2012-01-18 at 14:56 +0000, Daniel De Graaf wrote:> On 01/18/2012 05:36 AM, Ian Campbell wrote: > > On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote: > >> From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > >> > >> This patch reinstates the XENMEM_remove_from_physmap hypercall > >> which was removed in 19041:ee62aaafff46 because it was not used. > >> > >> However, is now needed in order to support xenstored stub domains. > >> The xenstored stub domain is not priviliged like dom0 and so cannot > >> unilaterally map the xenbus page of other guests into it''s address > >> space. Therefore, before creating a domU the domain builder needs to > >> seed its grant table with a grant ref allowing the xenstored stub > >> domain to access the new domU''s xenbus page. > >> > >> At present domU''s do not start with their grant table mapped. > >> Instead it gets mapped when the guest requests a grant table from > >> the hypervisor. > >> > >> In order to seed the grant table, the domain builder first needs to > >> map it into dom0 address space. But the hypercall to do this > >> requires a gpfn (guest pfn), which is an mfn for PV guest, but a pfn > >> for HVM guests. Therfore, in order to seed the grant table of an > >> HVM guest, dom0 needs to *temporarily* map it into the guest''s > >> "physical" address space. > >> > >> Hence the need to reinstate the XENMEM_remove_from_physmap hypercall. > >> > >> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > >> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > > > > Acked-by: Ian Campbell <ian.campbell@citrix.com> (modulo Jan''s comment > > about ordering in xlat.lst) > > > > BTW, since Alex and Diego have subsequently left Citrix you could take > > my Acked-by''s in this series as Signed-of-by on behalf of Citrix. I''ve > > no idea if that''s necessary though, I expect not. > > > > Ian. > > > > I''m not an expert in this area,Me neither.> but this is how I read it: the portion of > the path authored by Alex/Diego was already signed-off when they were posted, > so since the current patches are derived works from them the sign-off may > need to stay in order to allow me to sign off because I cannot claim copyright > on all of the content. Assuming Citrix actually owns the copyright on the > patches, your Ack may suffice to replace the sign-off for this purpose.I don''t think an Ack conveys the same meaning (WRT the DCO) as a Signed-off-by.> I guess my real question here would be: should the sign-off from Alex and > Diego remain on these patches in addition to your Ack?I would suggest you keep any signed-off-by they provided and augment it with my ack. I think I saw one or two which said "Originally-by" instead of "Signed-of-by", I guess those were either missing a Signed-off-by in the first place or have been heavily modified? Ian.>
Ian Jackson
2012-Jan-18 17:13 UTC
Re: [PATCH 13/18] xenstored: support running in minios stubdom
Ian Campbell writes ("Re: [Xen-devel] [PATCH 13/18] xenstored: support running in minios stubdom"):> One thing which might help is to provide nop versions of functions > instead of idef''ing both the definition and callsite. e.g. > static void write_pidfile(const char *pidfile) > +#ifndef __MINIOS__ > stuff > +#else > + nothing > +endifI would normally prefer:> +#ifndef __MINIOS__ > static void write_pidfile(const char *pidfile) > stuff > } > +#else > +static void write_pidfile(const char *pidfile) > +} > +endifI think this is fairly easy to read; the only hard part is figuring out which version is being used, which can often be done by putting the relevant bits in a separate file. Ian.
Ian Campbell
2012-Jan-18 17:35 UTC
Re: [PATCH 13/18] xenstored: support running in minios stubdom
On Wed, 2012-01-18 at 17:13 +0000, Ian Jackson wrote:> Ian Campbell writes ("Re: [Xen-devel] [PATCH 13/18] xenstored: support running in minios stubdom"): > > One thing which might help is to provide nop versions of functions > > instead of idef''ing both the definition and callsite. e.g. > > static void write_pidfile(const char *pidfile) > > +#ifndef __MINIOS__ > > stuff > > +#else > > + nothing > > +endif > > I would normally prefer: > > > +#ifndef __MINIOS__ > > static void write_pidfile(const char *pidfile) > > stuff > > } > > +#else > > +static void write_pidfile(const char *pidfile) > > +} > > +endifYes, I''d normally do it this way too, not sure why I wrote the other... Only real difference is that it prevents the prototype getting out of sync and bit-rotting the infrequently used case if there is one. Ian.> I think this is fairly easy to read; the only hard part is figuring > out which version is being used, which can often be done by putting > the relevant bits in a separate file. > > Ian.
Daniel De Graaf
2012-Jan-18 18:18 UTC
Re: [PATCH 10/18] xenstored: use grant references instead of map_foreign_range
On 01/18/2012 06:15 AM, Ian Campbell wrote:> On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote: >> From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> >> >> make xenstored use grantref rather than map_foreign_range (which can >> only be used by privileged domains) >> >> This patch modifies the xenstore daemon to use xc_gnttab_map_grant_ref >> instead of xc_map_foreign_range where available. >> >> Previous versions of this patch have been sent to xen-devel. See >> http://lists.xensource.com/archives/html/xen-devel/2008-07/msg00610.html >> http://lists.xensource.com/archives/html/xen-devel/2009-03/msg01492.html >> >> Signed-off-by: Diego Ongaro <diego.ongaro@citrix.com> >> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> >> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> >> --- >> tools/xenstore/xenstored_domain.c | 45 ++++++++++++++++++++++++++++++++----- >> 1 files changed, 39 insertions(+), 6 deletions(-) >> >> diff --git a/tools/xenstore/xenstored_domain.c b/tools/xenstore/xenstored_domain.c >> index 443af82..0b8353b 100644 >> --- a/tools/xenstore/xenstored_domain.c >> +++ b/tools/xenstore/xenstored_domain.c >> @@ -32,8 +32,10 @@ >> #include "xenstored_watch.h" >> >> #include <xenctrl.h> >> +#include <xen/grant_table.h> >> >> static xc_interface **xc_handle; >> +static xc_gnttab **xcg_handle; >> static evtchn_port_t virq_port; >> >> xc_evtchn *xce_handle = NULL; >> @@ -174,8 +176,12 @@ static int destroy_domain(void *_domain) >> eprintf("> Unbinding port %i failed!\n", domain->port); >> } >> >> - if (domain->interface) >> - munmap(domain->interface, getpagesize()); >> + if (domain->interface) { >> + if (*xcg_handle >= 0 && domain->domid != 0) > > Why the special case for domid 0 here? There seems to be no equivalent > for the map case, including the one you add in patch 15/18.dom0 is mapped by dom0_init()/xenbus_map() and must be unmapped with munmap, which is not guaranteed to be the same as xc_gnttab_munmap. The map case doesn''t need to handle that.> I think the map and unmap logic could usefully be made into helper > functions. > > Ian.-- Daniel De Graaf National Security Agency
Daniel De Graaf
2012-Jan-18 18:31 UTC
Re: [PATCH 16/18] xenstored: use domain_is_unprivileged instead of checking conn->id
On 01/18/2012 06:44 AM, Ian Campbell wrote:> On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote: >> This centralizes all the permission checking for privileged domains in >> preparation for allowing domains other than dom0 to be privileged. >> >> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> >> --- >> tools/xenstore/xenstored_core.c | 6 +++--- >> tools/xenstore/xenstored_domain.c | 8 ++++---- >> 2 files changed, 7 insertions(+), 7 deletions(-) >> >> diff --git a/tools/xenstore/xenstored_core.c b/tools/xenstore/xenstored_core.c >> index 4ec63f1..eea5fd6 100644 >> --- a/tools/xenstore/xenstored_core.c >> +++ b/tools/xenstore/xenstored_core.c >> @@ -488,7 +488,7 @@ static enum xs_perm_type perm_for_conn(struct connection *conn, >> mask &= ~XS_PERM_WRITE; >> >> /* Owners and tools get it all... */ >> - if (!conn->id || perms[0].id == conn->id >> + if (!domain_is_unprivileged(conn) || perms[0].id == conn->id > > domain_is_unprivileged is: > conn && conn->domain && conn->domain->domid != 0 > > which isn''t quite the same as the code being replaced. The difference > appears to be the conn->id is valid for socket connections as well as > domain connections whereas conn->domain is only present for domain > connections. > > Does this change not mean that, for the dom0-process xenstored > configuration we now treat socket based connections as unprivileged > where previously they would be unprivileged?No. For dom0 socket connections, conn->domain is NULL so the connection is not unprivileged (making it privileged). This is why sane people do not make boolean functions test for "un" cases :)
Daniel De Graaf
2012-Jan-18 19:07 UTC
Re: [PATCH 01/18] xen: reinstate previously unused XENMEM_remove_from_physmap hypercall
On 01/18/2012 11:06 AM, Ian Campbell wrote:> On Wed, 2012-01-18 at 14:56 +0000, Daniel De Graaf wrote: >> On 01/18/2012 05:36 AM, Ian Campbell wrote: >>> On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote: >>>> From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> >>>> >>>> This patch reinstates the XENMEM_remove_from_physmap hypercall >>>> which was removed in 19041:ee62aaafff46 because it was not used. >>>> >>>> However, is now needed in order to support xenstored stub domains. >>>> The xenstored stub domain is not priviliged like dom0 and so cannot >>>> unilaterally map the xenbus page of other guests into it''s address >>>> space. Therefore, before creating a domU the domain builder needs to >>>> seed its grant table with a grant ref allowing the xenstored stub >>>> domain to access the new domU''s xenbus page. >>>> >>>> At present domU''s do not start with their grant table mapped. >>>> Instead it gets mapped when the guest requests a grant table from >>>> the hypervisor. >>>> >>>> In order to seed the grant table, the domain builder first needs to >>>> map it into dom0 address space. But the hypercall to do this >>>> requires a gpfn (guest pfn), which is an mfn for PV guest, but a pfn >>>> for HVM guests. Therfore, in order to seed the grant table of an >>>> HVM guest, dom0 needs to *temporarily* map it into the guest''s >>>> "physical" address space. >>>> >>>> Hence the need to reinstate the XENMEM_remove_from_physmap hypercall. >>>> >>>> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> >>>> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> >>> >>> Acked-by: Ian Campbell <ian.campbell@citrix.com> (modulo Jan''s comment >>> about ordering in xlat.lst) >>> >>> BTW, since Alex and Diego have subsequently left Citrix you could take >>> my Acked-by''s in this series as Signed-of-by on behalf of Citrix. I''ve >>> no idea if that''s necessary though, I expect not. >>> >>> Ian. >>> >> >> I''m not an expert in this area, > > Me neither. > >> but this is how I read it: the portion of >> the path authored by Alex/Diego was already signed-off when they were posted, >> so since the current patches are derived works from them the sign-off may >> need to stay in order to allow me to sign off because I cannot claim copyright >> on all of the content. Assuming Citrix actually owns the copyright on the >> patches, your Ack may suffice to replace the sign-off for this purpose. > > I don''t think an Ack conveys the same meaning (WRT the DCO) as a > Signed-off-by. > >> I guess my real question here would be: should the sign-off from Alex and >> Diego remain on these patches in addition to your Ack? > > I would suggest you keep any signed-off-by they provided and augment it > with my ack. > > I think I saw one or two which said "Originally-by" instead of > "Signed-of-by", I guess those were either missing a Signed-off-by in the > first place or have been heavily modified? > > Ian. >I originally replaced all the signed-off-by lines with originally-by and missed one when converting back. When looking at the Linux version of the DCO, it implies (lower down when talking about subsystem maintainers) that if I make changes I need to drop the sign-off and claim clause (b) unless the original author is around to sign-off on the changed patch, or if it is trivial and I note this above my sign-off (not applicable here). This makes me lean toward changing back to "Originally-by" or similar tags. I did keep the From tags for those patches that I did not mostly rewrite, which I assume will be recognized when importing patches. -- Daniel De Graaf National Security Agency
Ian Campbell
2012-Jan-19 10:32 UTC
Re: [PATCH 01/18] xen: reinstate previously unused XENMEM_remove_from_physmap hypercall
On Wed, 2012-01-18 at 19:07 +0000, Daniel De Graaf wrote:> On 01/18/2012 11:06 AM, Ian Campbell wrote: > > On Wed, 2012-01-18 at 14:56 +0000, Daniel De Graaf wrote: > >> On 01/18/2012 05:36 AM, Ian Campbell wrote: > >>> On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote: > >>>> From: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > >>>> > >>>> This patch reinstates the XENMEM_remove_from_physmap hypercall > >>>> which was removed in 19041:ee62aaafff46 because it was not used. > >>>> > >>>> However, is now needed in order to support xenstored stub domains. > >>>> The xenstored stub domain is not priviliged like dom0 and so cannot > >>>> unilaterally map the xenbus page of other guests into it''s address > >>>> space. Therefore, before creating a domU the domain builder needs to > >>>> seed its grant table with a grant ref allowing the xenstored stub > >>>> domain to access the new domU''s xenbus page. > >>>> > >>>> At present domU''s do not start with their grant table mapped. > >>>> Instead it gets mapped when the guest requests a grant table from > >>>> the hypervisor. > >>>> > >>>> In order to seed the grant table, the domain builder first needs to > >>>> map it into dom0 address space. But the hypercall to do this > >>>> requires a gpfn (guest pfn), which is an mfn for PV guest, but a pfn > >>>> for HVM guests. Therfore, in order to seed the grant table of an > >>>> HVM guest, dom0 needs to *temporarily* map it into the guest''s > >>>> "physical" address space. > >>>> > >>>> Hence the need to reinstate the XENMEM_remove_from_physmap hypercall. > >>>> > >>>> Signed-off-by: Alex Zeffertt <alex.zeffertt@eu.citrix.com> > >>>> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> > >>> > >>> Acked-by: Ian Campbell <ian.campbell@citrix.com> (modulo Jan''s comment > >>> about ordering in xlat.lst) > >>> > >>> BTW, since Alex and Diego have subsequently left Citrix you could take > >>> my Acked-by''s in this series as Signed-of-by on behalf of Citrix. I''ve > >>> no idea if that''s necessary though, I expect not. > >>> > >>> Ian. > >>> > >> > >> I''m not an expert in this area, > > > > Me neither. > > > >> but this is how I read it: the portion of > >> the path authored by Alex/Diego was already signed-off when they were posted, > >> so since the current patches are derived works from them the sign-off may > >> need to stay in order to allow me to sign off because I cannot claim copyright > >> on all of the content. Assuming Citrix actually owns the copyright on the > >> patches, your Ack may suffice to replace the sign-off for this purpose. > > > > I don''t think an Ack conveys the same meaning (WRT the DCO) as a > > Signed-off-by. > > > >> I guess my real question here would be: should the sign-off from Alex and > >> Diego remain on these patches in addition to your Ack? > > > > I would suggest you keep any signed-off-by they provided and augment it > > with my ack. > > > > I think I saw one or two which said "Originally-by" instead of > > "Signed-of-by", I guess those were either missing a Signed-off-by in the > > first place or have been heavily modified? > > > > Ian. > > > > I originally replaced all the signed-off-by lines with originally-by and > missed one when converting back. When looking at the Linux version of the > DCO, it implies (lower down when talking about subsystem maintainers) that > if I make changes I need to drop the sign-off and claim clause (b) unless > the original author is around to sign-off on the changed patch, or if it is > trivial and I note this above my sign-off (not applicable here). This makes > me lean toward changing back to "Originally-by" or similar tags. I did keep > the From tags for those patches that I did not mostly rewrite, which I assume > will be recognized when importing patches.The DCO itself isn''t terribly specific about what to do with an existing Signed-off-by if you modify the patch. Common practice appears to be to include both the original and your own and to note what you have changed unless you have done a wholesale rewrite in which case it is "Based-on"/"Originally-by"/etc + your own S-o-b. Ian.>
Daniel De Graaf
2012-Jan-20 20:24 UTC
Re: [PATCH 06/18] lib{xc, xl}: Seed grant tables with xenstore and console grants
On 01/18/2012 06:05 AM, Ian Campbell wrote:> On Thu, 2012-01-12 at 23:35 +0000, Daniel De Graaf wrote: >> @@ -275,6 +276,169 @@ int xc_dom_boot_image(struct xc_dom_image *dom) >> return rc; >> } >> >> +static unsigned long xc_dom_gnttab_setup(xc_interface *xch, uint32_t domid) >> +{ >> + DECLARE_HYPERCALL; >> + gnttab_setup_table_t setup_table; >> + DECLARE_HYPERCALL_BUFFER(unsigned long, gmfnp); >> + int rc; >> + unsigned long gmfn; > > There''s a bunch of weird alignment issues in this patch. Presumably some > hard tabs have snuck in. > >> + >> + gmfnp = xc_hypercall_buffer_alloc(xch, gmfnp, sizeof(*gmfnp)); >> + if (gmfnp == NULL) >> + return -1; > > Alignment. > >> + >> + setup_table.dom = domid; >> + setup_table.nr_frames = 1; >> + set_xen_guest_handle(setup_table.frame_list, gmfnp); >> + setup_table.status = 0; >> + >> + hypercall.op = __HYPERVISOR_grant_table_op; >> + hypercall.arg[0] = GNTTABOP_setup_table; >> + hypercall.arg[1] = (unsigned long) &setup_table; >> + hypercall.arg[2] = 1; >> + >> + rc = do_xen_hypercall(xch, &hypercall); >> + gmfn = *gmfnp; > > Alignment. I''m going to stop pointing these out, I guess grep will find > them... > > [...] >> +int xc_dom_gnttab_init(struct xc_dom_image *dom) >> +{ >> + unsigned long console_gmfn; >> + unsigned long xenstore_gmfn; >> + int autotranslated; >> + >> + autotranslated = xc_dom_feature_translated(dom); >> + console_gmfn = autotranslated ? >> + dom->console_pfn : xc_dom_p2m_host(dom, dom->console_pfn); >> + xenstore_gmfn = autotranslated ? >> + dom->xenstore_pfn : xc_dom_p2m_host(dom, dom->xenstore_pfn); >> + >> + return xc_dom_gnttab_seed(dom->xch, dom->guest_domid, >> + console_gmfn, xenstore_gmfn, >> + dom->console_domid, dom->xenstore_domid); > > So on build we have can do the p2m lookup and hence use the PV version > of gnttab_seed, whereas on restore we don''t have a p2m and hence can''t? > > The internals of xc_domain_restore do have the p2m though, so in > principal we could avoid the need for the hvm version and the > reintroduction of the remove_from_physmap by making xc_domain_restore > output the necessary mfns? I''m not sure it is worth it though. >The reason for needing remove_from_physmap is to temporarily access the grant table, which can be done differently in PV and is not related to p2m translation.> >> diff --git a/tools/libxc/xc_domain_restore.c b/tools/libxc/xc_domain_restore.c >> index 3fda6f8..8bee684 100644 >> --- a/tools/libxc/xc_domain_restore.c >> +++ b/tools/libxc/xc_domain_restore.c >> @@ -1259,7 +1259,8 @@ static int apply_batch(xc_interface *xch, uint32_t dom, struct restore_ctx *ctx, >> >> int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, >> unsigned int store_evtchn, unsigned long *store_mfn, >> - unsigned int console_evtchn, unsigned long *console_mfn, >> + uint32_t store_domid, unsigned int console_evtchn, >> + unsigned long *console_mfn, uint32_t console_domid, >> unsigned int hvm, unsigned int pae, int superpages, >> int no_incr_generationid, >> unsigned long *vm_generationid_addr) >> @@ -2018,6 +2019,14 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, >> memcpy(ctx->live_p2m, ctx->p2m, dinfo->p2m_size * sizeof(xen_pfn_t)); >> munmap(ctx->live_p2m, P2M_FL_ENTRIES * PAGE_SIZE); >> >> + rc = xc_dom_gnttab_seed(xch, dom, *console_mfn, *store_mfn, >> + console_domid, store_domid); >> + if (rc != 0) >> + { >> + ERROR("error seeding grant table"); >> + goto out; >> + } >> + >> DPRINTF("Domain ready to be built.\n"); >> rc = 0; >> goto out; >> @@ -2076,6 +2085,14 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, >> goto out; >> } >> >> + rc = xc_dom_gnttab_hvm_seed(xch, dom, *console_mfn, *store_mfn, >> + console_domid, store_domid); > > Oh, we don''t even need to return them from xc_domain_restore since we > could just translate from gfn to mfn right here (since we have a p2m in > hand) and call xc_dom_gnttab_seed. Or am I missing something?For HVM, we still have to use guest frame numbers because that''s what Xen expects to find in the grant table. Or are you talking about the fact xc_domain_restore returns the mfns? That isn''t added by this patch; they are just used to populate the grant table.>> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c >> index 5e39595..04db22f 100644 >> --- a/tools/libxl/libxl_dom.c >> +++ b/tools/libxl/libxl_dom.c > [...] >> @@ -305,6 +312,8 @@ static int hvm_build_set_params(xc_interface *handle, uint32_t domid, >> xc_set_hvm_param(handle, domid, HVM_PARAM_NESTEDHVM, info->u.hvm.nested_hvm); >> xc_set_hvm_param(handle, domid, HVM_PARAM_STORE_EVTCHN, store_evtchn); >> xc_set_hvm_param(handle, domid, HVM_PARAM_CONSOLE_EVTCHN, console_evtchn); >> + >> + xc_dom_gnttab_hvm_seed(handle, domid, *console_mfn, *store_mfn, console_domid, store_domid); > > Didn''t this already happen in xc_linux_build_internal which was called > via xc_linux_build_mem?No, xl doesn''t call xc_linux_build_mem; that''s only used by xm. -- Daniel De Graaf National Security Agency
Ian Jackson
2012-Jan-24 16:24 UTC
Re: [PATCH 13/18] xenstored: support running in minios stubdom
Ian Campbell writes ("Re: [Xen-devel] [PATCH 13/18] xenstored: support running in minios stubdom"):> On Wed, 2012-01-18 at 17:13 +0000, Ian Jackson wrote: > > I would normally prefer: > > > > > +#ifndef __MINIOS__ > > > static void write_pidfile(const char *pidfile) > > > stuff > > > } > > > +#else > > > +static void write_pidfile(const char *pidfile) > > > +} > > > +endif > > Yes, I''d normally do it this way too, not sure why I wrote the other... > > Only real difference is that it prevents the prototype getting out of > sync and bit-rotting the infrequently used case if there is one.Better that the prototype gets out of sync and you get a compiler error, than that the prototype is updated but the little-used implementations of the body of the function is not adjusted for new semantics implied by new arguments ... Ian.