rusty@rustcorp.com.au
2007-Sep-25 23:57 UTC
[patch 02/43] lguest: fix modules oopsing in lguest guests
The assembly templates for lguest guest patching are in the .init.text section. This means that modules get patched with "cc cc cc cc" or similar junk. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> --- drivers/lguest/lguest_asm.S | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) ==================================================================--- a/drivers/lguest/lguest_asm.S +++ b/drivers/lguest/lguest_asm.S @@ -22,8 +22,9 @@ jmp lguest_init /*G:055 We create a macro which puts the assembler code between lgstart_ and - * lgend_ markers. These templates end up in the .init.text section, so they - * are discarded after boot. */ + * lgend_ markers. These templates are put in the .text section: they can't be + * discarded after boot as we may need to patch modules, too. */ +.text #define LGUEST_PATCH(name, insns...) \ lgstart_##name: insns; lgend_##name:; \ .globl lgstart_##name; .globl lgend_##name @@ -34,7 +35,6 @@ LGUEST_PATCH(pushf, movl lguest_data+LGU LGUEST_PATCH(pushf, movl lguest_data+LGUEST_DATA_irq_enabled, %eax) /*:*/ -.text /* These demark the EIP range where host should never deliver interrupts. */ .global lguest_noirq_start .global lguest_noirq_end -- there are those who do and those who hang on and you don't see too many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-25 23:57 UTC
[patch 04/43] lguest: Consolidate host virtualization support under Virtualization menu
Move lguest under the virtualization menu. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Cc: Avi Kivity <avi@qumranet.com> --- drivers/Kconfig | 2 -- drivers/kvm/Kconfig | 6 +++++- 2 files changed, 5 insertions(+), 3 deletions(-) ==================================================================--- a/drivers/Kconfig +++ b/drivers/Kconfig @@ -87,6 +87,4 @@ source "drivers/kvm/Kconfig" source "drivers/kvm/Kconfig" source "drivers/uio/Kconfig" - -source "drivers/lguest/Kconfig" endmenu ==================================================================--- a/drivers/kvm/Kconfig +++ b/drivers/kvm/Kconfig @@ -45,4 +36,8 @@ config KVM_AMD Provides support for KVM on AMD processors equipped with the AMD-V (SVM) extensions. -endif # VIRTUALIZATION +# OK, it's a little counter-intuitive to do this, but it puts it neatly under +# the virtualization menu. +source drivers/lguest/Kconfig + +endif # VIRTUALIZATION -- there are those who do and those who hang on and you don't see too many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-25 23:57 UTC
[patch 01/43] lguest: lguest example launcher truncates block device file to 0 length on problems
The function should also use ftruncate64() rather than ftruncate() to prevent
files over 4GB (not uncommon for a root filesystem) being zeroed.
Signed-off-by: Chris Malley <mail@chrismalley.co.uk>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
CC: Chris Malley <mail@chrismalley.co.uk>
diff -r bcbb99849fd9 Documentation/lguest/lguest.c
--- a/Documentation/lguest/lguest.c Wed Sep 26 12:21:40 2007 +1000
+++ b/Documentation/lguest/lguest.c Wed Sep 26 14:15:38 2007 +1000
@@ -882,7 +882,7 @@ static u32 handle_block_output(int fd, c
* of the block file (possibly extending it). */
if (off + len > device_len) {
/* Trim it back to the correct length */
- ftruncate(dev->fd, device_len);
+ ftruncate64(dev->fd, device_len);
/* Die, bad Guest, die. */
errx(1, "Write past end %llu+%u", off, len);
}
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-25 23:57 UTC
[patch 00/43] lguest: Patches for 2.6.24 (and patchbomb test)
Hi all, These are the patches I'm planning to submit for 2.6.24. Comments gratefully accepted. Along with the usual cleanups and improvements are Jes' de-i386-ification patches, and a new "virtio" mechanism designed to be shared with KVM (and hopefully other hypervisors). Cheers, Rusty. Documentation/lguest/Makefile | 30 Documentation/lguest/lguest.c | 1745 ++++++++++++----------- Documentation/lguest/lguest.txt | 67 a/drivers/block/lguest_blk.c | 423 ----- a/drivers/char/hvc_lguest.c | 177 -- a/drivers/lguest/io.c | 628 -------- a/drivers/lguest/lguest.c | 1101 -------------- a/drivers/lguest/lguest_asm.S | 94 - a/drivers/lguest/lguest_bus.c | 220 -- a/drivers/lguest/switcher.S | 351 ---- a/drivers/net/lguest_net.c | 552 ------- a/include/linux/lguest_bus.h | 51 arch/i386/Kconfig | 48 arch/i386/Makefile | 3 arch/i386/kernel/asm-offsets.c | 1 arch/i386/kernel/i8259.c | 3 arch/i386/kernel/io_apic.c | 13 arch/i386/kernel/traps.c | 10 arch/i386/lguest/Kconfig | 6 arch/i386/lguest/boot.c | 79 - arch/i386/lguest/head.S | 41 arch/i386/xen/Kconfig | 5 b/arch/i386/lguest/Kconfig | 8 b/arch/i386/lguest/Makefile | 1 b/arch/i386/lguest/boot.c | 1101 ++++++++++++++ b/arch/i386/lguest/head.S | 93 + b/drivers/block/virtio_blk.c | 327 ++++ b/drivers/char/virtio_console.c | 225 ++ b/drivers/lguest/i386_core.c | 476 ++++++ b/drivers/lguest/i386_switcher.S | 350 ++++ b/drivers/lguest/lguest_device.c | 373 ++++ b/drivers/net/virtio_net.c | 438 +++++ b/drivers/virtio/Kconfig | 3 b/drivers/virtio/Makefile | 1 b/drivers/virtio/config.c | 13 b/drivers/virtio/virtio.c | 171 ++ b/drivers/virtio/virtio_ring.c | 313 ++++ b/include/asm-i386/lguest.h | 88 + b/include/linux/virtio.h | 110 + b/include/linux/virtio_blk.h | 52 b/include/linux/virtio_config.h | 112 + b/include/linux/virtio_console.h | 13 b/include/linux/virtio_net.h | 37 b/include/linux/virtio_ring.h | 120 + drivers/Kconfig | 4 drivers/Makefile | 1 drivers/block/Kconfig | 6 drivers/block/Makefile | 2 drivers/char/Kconfig | 8 drivers/char/Makefile | 2 drivers/char/hvc_console.c | 67 drivers/kvm/Kconfig | 7 drivers/lguest/Kconfig | 17 drivers/lguest/Makefile | 20 drivers/lguest/core.c | 588 ------- drivers/lguest/hypercalls.c | 228 +-- drivers/lguest/i386_core.c | 127 + drivers/lguest/i386_switcher.S | 6 drivers/lguest/interrupts_and_traps.c | 131 + drivers/lguest/io.c | 28 drivers/lguest/lg.h | 198 -- drivers/lguest/lguest.c | 1 drivers/lguest/lguest_asm.S | 7 drivers/lguest/lguest_bus.c | 3 drivers/lguest/lguest_user.c | 162 -- drivers/lguest/page_tables.c | 288 ++- drivers/lguest/segments.c | 31 drivers/net/Kconfig | 6 drivers/net/Makefile | 3 drivers/net/lguest_net.c | 7 drivers/virtio/Kconfig | 5 drivers/virtio/Makefile | 1 drivers/virtio/virtio.c | 26 include/asm-i386/irq.h | 4 include/asm-i386/lguest.h | 4 include/asm-i386/lguest_hcall.h | 14 include/linux/Kbuild | 1 include/linux/lguest.h | 88 - include/linux/lguest_launcher.h | 134 - include/linux/mod_devicetable.h | 6 linux-2.6-lguest/include/asm-i386/lguest_hcall.h | 67 scripts/mod/file2alias.c | 19 82 files changed, 6455 insertions(+), 5935 deletions(-) -- there are those who do and those who hang on and you don't see too many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-25 23:57 UTC
[patch 05/43] lguest: Example launcher should include asm/e820.h instead of asm-i386/
Signed-off-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Cc: Steven Rostedt <rostedt@goodmis.org> --- Documentation/lguest/lguest.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) ==================================================================--- a/Documentation/lguest/lguest.c +++ b/Documentation/lguest/lguest.c @@ -46,7 +46,7 @@ typedef uint32_t u32; typedef uint16_t u16; typedef uint8_t u8; #include "../../include/linux/lguest_launcher.h" -#include "../../include/asm-i386/e820.h" +#include "../../include/asm/e820.h" /*:*/ #define PAGE_PRESENT 0x7 /* Present, RW, Execute */ -- 1.4.4.4 -- -- there are those who do and those who hang on and you don't see too many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-25 23:57 UTC
[patch 03/43] lguest: Normalize config options for guest support
1) Group all the "guest OS" support options together, under a PARAVIRT_GUEST menu. 2) Make those options select CONFIG_PARAVIRT, as suggested by Andi. 3) Make kconfig help titles consistent. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Cc: Andi Kleen <ak@suse.de> Cc: Zach Amsden <zamsden@vmware.com> Cc: Jeremy Fitzhardinge <jeremy@goop.org> Cc: Chris Wright <chrisw@sous-sol.org> --- arch/i386/Kconfig | 38 ++++++++++++++++++++++++++++---------- arch/i386/xen/Kconfig | 5 +++-- drivers/lguest/Kconfig | 3 +-- 3 files changed, 32 insertions(+), 14 deletions(-) ==================================================================--- a/arch/i386/Kconfig +++ b/arch/i386/Kconfig @@ -215,27 +215,45 @@ endchoice endchoice config PARAVIRT - bool "Paravirtualization support (EXPERIMENTAL)" - depends on EXPERIMENTAL + bool depends on !(X86_VISWS || X86_VOYAGER) help - Paravirtualization is a way of running multiple instances of - Linux on the same machine, under a hypervisor. This option - changes the kernel so it can modify itself when it is run - under a hypervisor, improving performance significantly. - However, when run without a hypervisor the kernel is - theoretically slower. If in doubt, say N. + This changes the kernel so it can modify itself when it is run + under a hypervisor, potentially improving performance significantly + over full virtualization. However, when run without a hypervisor + the kernel is theoretically slower and slightly larger. + +menuconfig PARAVIRT_GUEST + bool "Paravirtualized guest support" + help + Say Y here to get to see options related to running Linux under + various hypervisors. This option alone does not add any kernel code. + + If you say N, all options in this submenu will be skipped and disabled. + +if PARAVIRT_GUEST source "arch/i386/xen/Kconfig" config VMI - bool "VMI Paravirt-ops support" - depends on PARAVIRT + bool "VMI Guest support" + select PARAVIRT + depends on !(X86_VISWS || X86_VOYAGER) help VMI provides a paravirtualized interface to the VMware ESX server (it could be used by other hypervisors in theory too, but is not at the moment), by linking the kernel to a GPL-ed ROM module provided by the hypervisor. + +config LGUEST_GUEST + bool "Lguest guest support" + select PARAVIRT + depends on !X86_PAE + help + Lguest is a tiny in-kernel hypervisor. Selecting this will + allow your kernel to boot under lguest. This option will increase + your kernel size by about 6k. If in doubt, say N. +endif config ACPI_SRAT bool ==================================================================--- a/arch/i386/xen/Kconfig +++ b/arch/i386/xen/Kconfig @@ -3,8 +3,9 @@ # config XEN - bool "Enable support for Xen hypervisor" - depends on PARAVIRT && X86_CMPXCHG && X86_TSC && !NEED_MULTIPLE_NODES + bool "Xen guest support" + select PARAVIRT + depends on X86_CMPXCHG && X86_TSC && !NEED_MULTIPLE_NODES && !(X86_VISWS || X86_VOYAGER) help This is the Linux Xen port. Enabling this will allow the kernel to boot in a paravirtualized environment under the ==================================================================--- a/drivers/lguest/Kconfig +++ b/drivers/lguest/Kconfig @@ -1,7 +1,6 @@ config LGUEST config LGUEST tristate "Linux hypervisor example code" - depends on X86 && PARAVIRT && EXPERIMENTAL && !X86_PAE && FUTEX - select LGUEST_GUEST + depends on X86 && EXPERIMENTAL && !X86_PAE && FUTEX && !(X86_VISWS || X86_VOYAGER) select HVC_DRIVER ---help--- This is a very simple module which allows you to run -- there are those who do and those who hang on and you don't see too many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-25 23:57 UTC
[patch 08/43] lguest: Lguest currently depends on 32-bit x86, not just x86.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> --- drivers/lguest/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ==================================================================--- a/drivers/lguest/Kconfig +++ b/drivers/lguest/Kconfig @@ -1,6 +1,6 @@ config LGUEST config LGUEST tristate "Linux hypervisor example code" - depends on X86 && EXPERIMENTAL && !X86_PAE && FUTEX && !(X86_VISWS || X86_VOYAGER) + depends on X86_32 && EXPERIMENTAL && !X86_PAE && FUTEX && !(X86_VISWS || X86_VOYAGER) select HVC_DRIVER ---help--- This is a very simple module which allows you to run -- there are those who do and those who hang on and you don't see too many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-25 23:57 UTC
[patch 06/43] lguest: turn err into errx in lguest call sites
These two callsites should really be errx instead of err, since there is
no errno associated with them in the moment they are issued.
Signed-off-by: Glauber de Oliveira Costa <gcosta@redhat.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: Glauber de Oliveira Costa <gcosta@redhat.com>
---
Documentation/lguest/lguest.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
==================================================================---
a/Documentation/lguest/lguest.c
+++ b/Documentation/lguest/lguest.c
@@ -159,7 +159,7 @@ static unsigned long entry_point(void *s
if (memcmp(p, "GenuineLguest", strlen("GenuineLguest"))
== 0)
return (long)p + strlen("GenuineLguest") + page_offset;
- err(1, "Is this image a genuine lguest?");
+ errx(1, "Is this image a genuine lguest?");
}
/* This routine takes an open vmlinux image, which is in ELF, and maps it into
@@ -858,7 +858,7 @@ static u32 handle_block_output(int fd, c
/* We first check that the read or write is within the length of the
* block file. */
if (off >= device_len)
- err(1, "Bad offset %llu vs %llu", off, device_len);
+ errx(1, "Bad offset %llu vs %llu", off, device_len);
/* Move to the right location in the block file. This shouldn't fail,
* but best to check. */
if (lseek64(dev->fd, off, SEEK_SET) != off)
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-25 23:57 UTC
[patch 07/43] lguest: Use copy_to_user() not put_user for struct timespec
Use copy_to_user() when copying a struct timespec to the guest -
put_user() cannot handle two long's in one go on a 64bit arch.
Signed-off-by: Jes Sorensen <jes@sgi.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: Jes Sorensen <jes@sgi.com>
Cc: Al Viro <viro@ftp.linux.org.uk>
---
drivers/lguest/hypercalls.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
==================================================================---
linux-2.6.23-rc4.orig/drivers/lguest/hypercalls.c
+++ linux-2.6.23-rc4/drivers/lguest/hypercalls.c
@@ -243,6 +243,6 @@ void write_timestamp(struct lguest *lg)
{
struct timespec now;
ktime_get_real_ts(&now);
- if (put_user(now, &lg->lguest_data->time))
+ if (copy_to_user(&lg->lguest_data->time, &now, sizeof(struct
timespec)))
kill_guest(lg, "Writing timestamp");
}
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-25 23:57 UTC
[patch 11/43] lguest: lguest_devices belongs in lguest_bus.c: its not i386-specific.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
drivers/lguest/lguest.c | 1 -
drivers/lguest/lguest_bus.c | 2 ++
2 files changed, 2 insertions(+), 1 deletion(-)
==================================================================---
a/drivers/lguest/lguest.c
+++ b/drivers/lguest/lguest.c
@@ -87,7 +87,6 @@ struct lguest_data lguest_data = {
.noirq_end = (u32)lguest_noirq_end,
.blocked_interrupts = { 1 }, /* Block timer interrupts */
};
-struct lguest_device_desc *lguest_devices;
static cycle_t clock_base;
/*G:035 Notice the lazy_hcall() above, rather than hcall(). This is our first
==================================================================---
a/drivers/lguest/lguest_bus.c
+++ b/drivers/lguest/lguest_bus.c
@@ -6,6 +6,8 @@
#include <linux/lguest_bus.h>
#include <asm/io.h>
#include <asm/paravirt.h>
+
+struct lguest_device_desc *lguest_devices;
static ssize_t type_show(struct device *_dev,
struct device_attribute *attr, char *buf)
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
o Describe the new split configurations
o Highlight code documentation in drivers/lguest/README
o Point out necessity of having a getty on /dev/hvc0
o Remove gratuitous "m" in example
o Don't discuss I/O model here, stick to user documentation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
Documentation/lguest/lguest.txt | 66 ++++++++++++++++-----------------------
1 file changed, 28 insertions(+), 38 deletions(-)
==================================================================---
a/Documentation/lguest/lguest.txt
+++ b/Documentation/lguest/lguest.txt
@@ -6,7 +6,7 @@ Linux developers and users to experiment
Linux developers and users to experiment with virtualization with the
minimum of complexity. Nonetheless, it should have sufficient
features to make it useful for specific tasks, and, of course, you are
-encouraged to fork and enhance it.
+encouraged to fork and enhance it (see drivers/lguest/README).
Features:
@@ -23,19 +23,30 @@ Developer features:
Running Lguest:
-- Lguest runs the same kernel as guest and host. You can configure
- them differently, but usually it's easiest not to.
+- The easiest way to run lguest is to use same kernel as guest and host.
+ You can configure them differently, but usually it's easiest not to.
You will need to configure your kernel with the following options:
- CONFIG_HIGHMEM64G=n ("High Memory Support" "64GB")[1]
- CONFIG_TUN=y/m ("Universal TUN/TAP device driver support")
- CONFIG_EXPERIMENTAL=y ("Prompt for development and/or incomplete
code/drivers")
- CONFIG_PARAVIRT=y ("Paravirtualization support (EXPERIMENTAL)")
- CONFIG_LGUEST=y/m ("Linux hypervisor example code")
+ "General setup":
+ "Prompt for development and/or incomplete code/drivers" = Y
+ (CONFIG_EXPERIMENTAL=y)
- and I recommend:
- CONFIG_HZ=100 ("Timer frequency")[2]
+ "Processor type and features":
+ "Paravirtualized guest support" = Y
+ "Lguest guest support" = Y
+ "High Memory Support" = off/4GB
+ "Alignment value to which kernel should be aligned" = 0x100000
+ (CONFIG_PARAVIRT=y, CONFIG_LGUEST_GUEST=y, CONFIG_HIGHMEM64G=n and
+ CONFIG_PHYSICAL_ALIGN=0x100000)
+
+ "Device Drivers":
+ "Network device support"
+ "Universal TUN/TAP device driver support" = M/Y
+ (CONFIG_TUN=m)
+ "Virtualization"
+ "Linux hypervisor example code" = M/Y
+ (CONFIG_LGUEST=m)
- A tool called "lguest" is available in this directory: type
"make"
to build it. If you didn't build your kernel in-tree, use "make
@@ -51,14 +61,17 @@ Running Lguest:
dd if=/dev/zero of=rootfile bs=1M count=2048
qemu -cdrom image.iso -hda rootfile -net user -net nic -boot d
+ Make sure that you install a getty on /dev/hvc0 if you want to log in on the
+ console!
+
- "modprobe lg" if you built it as a module.
- Run an lguest as root:
- Documentation/lguest/lguest 64m vmlinux --tunnet=192.168.19.1
--block=rootfile root=/dev/lgba
+ Documentation/lguest/lguest 64 vmlinux --tunnet=192.168.19.1
--block=rootfile root=/dev/lgba
Explanation:
- 64m: the amount of memory to use.
+ 64: the amount of memory to use, in MB.
vmlinux: the kernel image found in the top of your build directory. You
can also use a standard bzImage.
@@ -99,31 +112,7 @@ Running Lguest:
"--sharenet=<filename>": any two guests using the same file
are on
the same network. This file is created if it does not exist.
-Lguest I/O model:
+There is a helpful mailing list at http://ozlabs.org/mailman/listinfo/lguest
-Lguest uses a simplified DMA model plus shared memory for I/O. Guests
-can communicate with each other if they share underlying memory
-(usually by the lguest program mmaping the same file), but they can
-use any non-shared memory to communicate with the lguest process.
-
-Guests can register DMA buffers at any key (must be a valid physical
-address) using the LHCALL_BIND_DMA(key, dmabufs, num<<8|irq)
-hypercall. "dmabufs" is the physical address of an array of
"num"
-"struct lguest_dma": each contains a used_len, and an array of
-physical addresses and lengths. When a transfer occurs, the
-"used_len" field of one of the buffers which has used_len 0 will be
-set to the length transferred and the irq will fire.
-
-Using an irq value of 0 unbinds the dma buffers.
-
-To send DMA, the LHCALL_SEND_DMA(key, dma_physaddr) hypercall is used,
-and the bytes used is written to the used_len field. This can be 0 if
-noone else has bound a DMA buffer to that key or some other error.
-DMA buffers bound by the same guest are ignored.
-
-Cheers!
+Good luck!
Rusty Russell rusty@rustcorp.com.au.
-
-[1] These are on various places on the TODO list, waiting for you to
- get annoyed enough at the limitation to fix it.
-[2] Lguest is not yet tickless when idle. See [1].
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-25 23:57 UTC
[patch 10/43] lguest: Make lguest_launcher.h types userspace-friendly
lguest_launcher.h uses "u32" not "__u32", which sets a bad
example. Fix that,
and include <linux/types.h>.
This means we need to use -I on the Launcher build line so types.h is found.
(Note: the example launcher also includes asm/e820.h which uses u32/u64,
but at least this cleans up the lguest code).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
Documentation/lguest/Makefile | 2 +-
Documentation/lguest/lguest.c | 4 ++--
include/linux/lguest_launcher.h | 21 +++++++++++----------
3 files changed, 14 insertions(+), 13 deletions(-)
==================================================================---
a/Documentation/lguest/Makefile
+++ b/Documentation/lguest/Makefile
@@ -11,7 +11,7 @@ include $(KBUILD_OUTPUT)/.config
include $(KBUILD_OUTPUT)/.config
LGUEST_GUEST_TOP := ($(CONFIG_PAGE_OFFSET) - 0x08000000)
-CFLAGS:=-Wall -Wmissing-declarations -Wmissing-prototypes -O3 -Wl,-T,lguest.lds
+CFLAGS:=-Wall -Wmissing-declarations -Wmissing-prototypes -O3 -I../../include
-Wl,-T,lguest.lds
LDLIBS:=-lz
# Removing this works for some versions of ld.so (eg. Ubuntu Feisty) and
# not others (eg. FC7).
==================================================================---
a/Documentation/lguest/lguest.c
+++ b/Documentation/lguest/lguest.c
@@ -45,8 +45,8 @@ typedef uint32_t u32;
typedef uint32_t u32;
typedef uint16_t u16;
typedef uint8_t u8;
-#include "../../include/linux/lguest_launcher.h"
-#include "../../include/asm/e820.h"
+#include "linux/lguest_launcher.h"
+#include "asm/e820.h"
/*:*/
#define PAGE_PRESENT 0x7 /* Present, RW, Execute */
==================================================================---
a/include/linux/lguest_launcher.h
+++ b/include/linux/lguest_launcher.h
@@ -1,6 +1,7 @@
#ifndef _ASM_LGUEST_USER
#define _ASM_LGUEST_USER
/* Everything the "lguest" userspace program needs to know. */
+#include <linux/types.h>
/* They can register up to 32 arrays of lguest_dma. */
#define LGUEST_MAX_DMA 32
/* At most we can dma 16 lguest_dma in one op. */
@@ -37,9 +38,9 @@ struct lguest_dma
struct lguest_dma
{
/* 0 if free to be used, filled by the Host. */
- u32 used_len;
+ __u32 used_len;
+ __u16 len[LGUEST_MAX_DMA_SECTIONS];
unsigned long addr[LGUEST_MAX_DMA_SECTIONS];
- u16 len[LGUEST_MAX_DMA_SECTIONS];
};
/*:*/
@@ -52,11 +53,11 @@ struct lguest_block_page
{
/* 0 is a read, 1 is a write. */
int type;
- u32 sector; /* Offset in device = sector * 512. */
- u32 bytes; /* Length expected to be read/written in bytes */
+ __u32 sector; /* Offset in device = sector * 512. */
+ __u32 bytes; /* Length expected to be read/written in bytes */
/* 0 = pending, 1 = done, 2 = done, error */
int result;
- u32 num_sectors; /* Disk length = num_sectors * 512 */
+ __u32 num_sectors; /* Disk length = num_sectors * 512 */
};
/*D:520 The network device is basically a memory page where all the Guests on
@@ -86,21 +87,21 @@ struct lguest_net
*/
struct lguest_device_desc {
/* The device type: console, network, disk etc. */
- u16 type;
+ __u16 type;
#define LGUEST_DEVICE_T_CONSOLE 1
#define LGUEST_DEVICE_T_NET 2
#define LGUEST_DEVICE_T_BLOCK 3
/* The specific features of this device: these depends on device type
* except for LGUEST_DEVICE_F_RANDOMNESS. */
- u16 features;
+ __u16 features;
#define LGUEST_NET_F_NOCSUM 0x4000 /* Don't bother checksumming */
#define LGUEST_DEVICE_F_RANDOMNESS 0x8000 /* IRQ is fairly random */
/* This is how the Guest reports status of the device: the Host can set
* LGUEST_DEVICE_S_REMOVED to indicate removal, but the rest are only
* ever manipulated by the Guest, and only ever set. */
- u16 status;
+ __u16 status;
/* 256 and above are device specific. */
#define LGUEST_DEVICE_S_ACKNOWLEDGE 1 /* We have seen device. */
#define LGUEST_DEVICE_S_DRIVER 2 /* We have found a driver */
@@ -111,8 +112,8 @@ struct lguest_device_desc {
/* Each device exists somewhere in Guest physical memory, over some
* number of pages. */
- u16 num_pages;
- u32 pfn;
+ __u16 num_pages;
+ __u32 pfn;
};
/*:*/
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-25 23:57 UTC
[patch 16/43] lguest: Accept elf files that are valid but have sections that can not be mmaped for some reason.
Signed-off-by: Ronald G. Minnich <rminnich@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
Documentation/lguest/lguest.c | 61 +++++++++++++++++++++--------------------
1 file changed, 32 insertions(+), 29 deletions(-)
==================================================================---
a/Documentation/lguest/lguest.c
+++ b/Documentation/lguest/lguest.c
@@ -15,6 +15,7 @@
#include <stdlib.h>
#include <elf.h>
#include <sys/mman.h>
+#include <sys/param.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
@@ -162,6 +163,30 @@ static unsigned long entry_point(void *s
errx(1, "Is this image a genuine lguest?");
}
+/* This routine is used to load the kernel or initrd. It tries mmap, but if
+ * that fails (Plan 9's kernel file isn't nicely aligned on page
boundaries),
+ * it falls back to reading the memory in. */
+static void map_at(int fd, void *addr, unsigned long offset, unsigned long len)
+{
+ ssize_t r;
+
+ /* We map writable even though for some segments are marked read-only.
+ * The kernel really wants to be writable: it patches its own
+ * instructions.
+ *
+ * MAP_PRIVATE means that the page won't be copied until a write is
+ * done to it. This allows us to share untouched memory between
+ * Guests. */
+ if (mmap(addr, len, PROT_READ|PROT_WRITE|PROT_EXEC,
+ MAP_FIXED|MAP_PRIVATE, fd, offset) != MAP_FAILED)
+ return;
+
+ /* pread does a seek and a read in one shot: saves a few lines. */
+ r = pread(fd, addr, len, offset);
+ if (r != len)
+ err(1, "Reading offset %lu len %lu gave %zi", offset, len, r);
+}
+
/* This routine takes an open vmlinux image, which is in ELF, and maps it into
* the Guest memory. ELF = Embedded Linking Format, which is the format used
* by all modern binaries on Linux including the kernel.
@@ -176,7 +201,6 @@ static unsigned long map_elf(int elf_fd,
static unsigned long map_elf(int elf_fd, const Elf32_Ehdr *ehdr,
unsigned long *page_offset)
{
- void *addr;
Elf32_Phdr phdr[ehdr->e_phnum];
unsigned int i;
unsigned long start = -1UL, end = 0;
@@ -227,23 +251,9 @@ static unsigned long map_elf(int elf_fd,
if (phdr[i].p_paddr + phdr[i].p_filesz > end)
end = phdr[i].p_paddr + phdr[i].p_filesz;
- /* We map this section of the file at its physical address. We
- * map it read & write even if the header says this segment is
- * read-only. The kernel really wants to be writable: it
- * patches its own instructions which would normally be
- * read-only.
- *
- * MAP_PRIVATE means that the page won't be copied until a
- * write is done to it. This allows us to share much of the
- * kernel memory between Guests. */
- addr = mmap((void *)phdr[i].p_paddr,
- phdr[i].p_filesz,
- PROT_READ|PROT_WRITE|PROT_EXEC,
- MAP_FIXED|MAP_PRIVATE,
- elf_fd, phdr[i].p_offset);
- if (addr != (void *)phdr[i].p_paddr)
- err(1, "Mmaping vmlinux seg %i gave %p not %p",
- i, addr, (void *)phdr[i].p_paddr);
+ /* We map this section of the file at its physical address. */
+ map_at(elf_fd, (void *)phdr[i].p_paddr,
+ phdr[i].p_offset, phdr[i].p_filesz);
}
return entry_point((void *)start, (void *)end, *page_offset);
@@ -402,27 +412,20 @@ static unsigned long load_initrd(const c
int ifd;
struct stat st;
unsigned long len;
- void *iaddr;
ifd = open_or_die(name, O_RDONLY);
/* fstat() is needed to get the file size. */
if (fstat(ifd, &st) < 0)
err(1, "fstat() on initrd '%s'", name);
- /* The length needs to be rounded up to a page size: mmap needs the
- * address to be page aligned. */
+ /* We map the initrd at the top of memory, but mmap wants it to be
+ * page-aligned, so we round the size up for that. */
len = page_align(st.st_size);
- /* We map the initrd at the top of memory. */
- iaddr = mmap((void *)mem - len, st.st_size,
- PROT_READ|PROT_EXEC|PROT_WRITE,
- MAP_FIXED|MAP_PRIVATE, ifd, 0);
- if (iaddr != (void *)mem - len)
- err(1, "Mmaping initrd '%s' returned %p not %p",
- name, iaddr, (void *)mem - len);
+ map_at(ifd, (void *)mem - len, 0, st.st_size);
/* Once a file is mapped, you can close the file descriptor. It's a
* little odd, but quite useful. */
close(ifd);
- verbose("mapped initrd %s size=%lu @ %p\n", name, st.st_size,
iaddr);
+ verbose("mapped initrd %s size=%lu @ %p\n", name, len,
(void*)mem-len);
/* We return the initrd size. */
return len;
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-25 23:57 UTC
[patch 15/43] lguest: Rename switcher.S to i386_switcher.S, since its very i386-specific.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> --- drivers/lguest/Makefile | 6 drivers/lguest/i386_switcher.S | 350 ++++++++++++++++++++++++++++++++++++++++ drivers/lguest/switcher.S | 350 ---------------------------------------- 3 files changed, 354 insertions(+), 352 deletions(-) ==================================================================--- a/drivers/lguest/Makefile +++ b/drivers/lguest/Makefile @@ -1,8 +1,10 @@ # Host requires the other files, which can be a module. obj-$(CONFIG_LGUEST) += lg.o -lg-y := core.o hypercalls.o page_tables.o interrupts_and_traps.o \ - segments.o io.o lguest_user.o switcher.o +lg-y = core.o hypercalls.o page_tables.o interrupts_and_traps.o \ + segments.o io.o lguest_user.o + +lg-$(CONFIG_X86_32) += i386_switcher.o Preparation Preparation!: PREFIX=P Guest: PREFIX=G ==================================================================--- /dev/null +++ b/drivers/lguest/i386_switcher.S @@ -0,0 +1,350 @@ +/*P:900 This is the Switcher: code which sits at 0xFFC00000 to do the low-level + * Guest<->Host switch. It is as simple as it can be made, but it's naturally + * very specific to x86. + * + * You have now completed Preparation. If this has whet your appetite; if you + * are feeling invigorated and refreshed then the next, more challenging stage + * can be found in "make Guest". :*/ + +/*S:100 + * Welcome to the Switcher itself! + * + * This file contains the low-level code which changes the CPU to run the Guest + * code, and returns to the Host when something happens. Understand this, and + * you understand the heart of our journey. + * + * Because this is in assembler rather than C, our tale switches from prose to + * verse. First I tried limericks: + * + * There once was an eax reg, + * To which our pointer was fed, + * It needed an add, + * Which asm-offsets.h had + * But this limerick is hurting my head. + * + * Next I tried haikus, but fitting the required reference to the seasons in + * every stanza was quickly becoming tiresome: + * + * The %eax reg + * Holds "struct lguest_pages" now: + * Cherry blossoms fall. + * + * Then I started with Heroic Verse, but the rhyming requirement leeched away + * the content density and led to some uniquely awful oblique rhymes: + * + * These constants are coming from struct offsets + * For use within the asm switcher text. + * + * Finally, I settled for something between heroic hexameter, and normal prose + * with inappropriate linebreaks. Anyway, it aint no Shakespeare. + */ + +// Not all kernel headers work from assembler +// But these ones are needed: the ENTRY() define +// And constants extracted from struct offsets +// To avoid magic numbers and breakage: +// Should they change the compiler can't save us +// Down here in the depths of assembler code. +#include <linux/linkage.h> +#include <asm/asm-offsets.h> +#include <asm/page.h> +#include "lg.h" + +// We mark the start of the code to copy +// It's placed in .text tho it's never run here +// You'll see the trick macro at the end +// Which interleaves data and text to effect. +.text +ENTRY(start_switcher_text) + +// When we reach switch_to_guest we have just left +// The safe and comforting shores of C code +// %eax has the "struct lguest_pages" to use +// Where we save state and still see it from the Guest +// And %ebx holds the Guest shadow pagetable: +// Once set we have truly left Host behind. +ENTRY(switch_to_guest) + // We told gcc all its regs could fade, + // Clobbered by our journey into the Guest + // We could have saved them, if we tried + // But time is our master and cycles count. + + // Segment registers must be saved for the Host + // We push them on the Host stack for later + pushl %es + pushl %ds + pushl %gs + pushl %fs + // But the compiler is fickle, and heeds + // No warning of %ebp clobbers + // When frame pointers are used. That register + // Must be saved and restored or chaos strikes. + pushl %ebp + // The Host's stack is done, now save it away + // In our "struct lguest_pages" at offset + // Distilled into asm-offsets.h + movl %esp, LGUEST_PAGES_host_sp(%eax) + + // All saved and there's now five steps before us: + // Stack, GDT, IDT, TSS + // And last of all the page tables are flipped. + + // Yet beware that our stack pointer must be + // Always valid lest an NMI hits + // %edx does the duty here as we juggle + // %eax is lguest_pages: our stack lies within. + movl %eax, %edx + addl $LGUEST_PAGES_regs, %edx + movl %edx, %esp + + // The Guest's GDT we so carefully + // Placed in the "struct lguest_pages" before + lgdt LGUEST_PAGES_guest_gdt_desc(%eax) + + // The Guest's IDT we did partially + // Move to the "struct lguest_pages" as well. + lidt LGUEST_PAGES_guest_idt_desc(%eax) + + // The TSS entry which controls traps + // Must be loaded up with "ltr" now: + // For after we switch over our page tables + // It (as the rest) will be writable no more. + // (The GDT entry TSS needs + // Changes type when we load it: damn Intel!) + movl $(GDT_ENTRY_TSS*8), %edx + ltr %dx + + // Look back now, before we take this last step! + // The Host's TSS entry was also marked used; + // Let's clear it again, ere we return. + // The GDT descriptor of the Host + // Points to the table after two "size" bytes + movl (LGUEST_PAGES_host_gdt_desc+2)(%eax), %edx + // Clear the type field of "used" (byte 5, bit 2) + andb $0xFD, (GDT_ENTRY_TSS*8 + 5)(%edx) + + // Once our page table's switched, the Guest is live! + // The Host fades as we run this final step. + // Our "struct lguest_pages" is now read-only. + movl %ebx, %cr3 + + // The page table change did one tricky thing: + // The Guest's register page has been mapped + // Writable onto our %esp (stack) -- + // We can simply pop off all Guest regs. + popl %ebx + popl %ecx + popl %edx + popl %esi + popl %edi + popl %ebp + popl %gs + popl %eax + popl %fs + popl %ds + popl %es + + // Near the base of the stack lurk two strange fields + // Which we fill as we exit the Guest + // These are the trap number and its error + // We can simply step past them on our way. + addl $8, %esp + + // The last five stack slots hold return address + // And everything needed to change privilege + // Into the Guest privilege level of 1, + // And the stack where the Guest had last left it. + // Interrupts are turned back on: we are Guest. + iret + +// There are two paths where we switch to the Host +// So we put the routine in a macro. +// We are on our way home, back to the Host +// Interrupted out of the Guest, we come here. +#define SWITCH_TO_HOST \ + /* We save the Guest state: all registers first \ + * Laid out just as "struct lguest_regs" defines */ \ + pushl %es; \ + pushl %ds; \ + pushl %fs; \ + pushl %eax; \ + pushl %gs; \ + pushl %ebp; \ + pushl %edi; \ + pushl %esi; \ + pushl %edx; \ + pushl %ecx; \ + pushl %ebx; \ + /* Our stack and our code are using segments \ + * Set in the TSS and IDT \ + * Yet if we were to touch data we'd use \ + * Whatever data segment the Guest had. \ + * Load the lguest ds segment for now. */ \ + movl $(LGUEST_DS), %eax; \ + movl %eax, %ds; \ + /* So where are we? Which CPU, which struct? \ + * The stack is our clue: our TSS starts \ + * It at the end of "struct lguest_pages". \ + * Or we may have stumbled while restoring \ + * Our Guest segment regs while in switch_to_guest, \ + * The fault pushed atop that part-unwound stack. \ + * If we round the stack down to the page start \ + * We're at the start of "struct lguest_pages". */ \ + movl %esp, %eax; \ + andl $(~(1 << PAGE_SHIFT - 1)), %eax; \ + /* Save our trap number: the switch will obscure it \ + * (The Guest regs are not mapped here in the Host) \ + * %ebx holds it safe for deliver_to_host */ \ + movl LGUEST_PAGES_regs_trapnum(%eax), %ebx; \ + /* The Host GDT, IDT and stack! \ + * All these lie safely hidden from the Guest: \ + * We must return to the Host page tables \ + * (Hence that was saved in struct lguest_pages) */ \ + movl LGUEST_PAGES_host_cr3(%eax), %edx; \ + movl %edx, %cr3; \ + /* As before, when we looked back at the Host \ + * As we left and marked TSS unused \ + * So must we now for the Guest left behind. */ \ + andb $0xFD, (LGUEST_PAGES_guest_gdt+GDT_ENTRY_TSS*8+5)(%eax); \ + /* Switch to Host's GDT, IDT. */ \ + lgdt LGUEST_PAGES_host_gdt_desc(%eax); \ + lidt LGUEST_PAGES_host_idt_desc(%eax); \ + /* Restore the Host's stack where it's saved regs lie */ \ + movl LGUEST_PAGES_host_sp(%eax), %esp; \ + /* Last the TSS: our Host is complete */ \ + movl $(GDT_ENTRY_TSS*8), %edx; \ + ltr %dx; \ + /* Restore now the regs saved right at the first. */ \ + popl %ebp; \ + popl %fs; \ + popl %gs; \ + popl %ds; \ + popl %es + +// Here's where we come when the Guest has just trapped: +// (Which trap we'll see has been pushed on the stack). +// We need only switch back, and the Host will decode +// Why we came home, and what needs to be done. +return_to_host: + SWITCH_TO_HOST + iret + +// An interrupt, with some cause external +// Has ajerked us rudely from the Guest's code +// Again we must return home to the Host +deliver_to_host: + SWITCH_TO_HOST + // But now we must go home via that place + // Where that interrupt was supposed to go + // Had we not been ensconced, running the Guest. + // Here we see the cleverness of our stack: + // The Host stack is formed like an interrupt + // With EIP, CS and EFLAGS layered. + // Interrupt handlers end with "iret" + // And that will take us home at long long last. + + // But first we must find the handler to call! + // The IDT descriptor for the Host + // Has two bytes for size, and four for address: + // %edx will hold it for us for now. + movl (LGUEST_PAGES_host_idt_desc+2)(%eax), %edx + // We now know the table address we need, + // And saved the trap's number inside %ebx. + // Yet the pointer to the handler is smeared + // Across the bits of the table entry. + // What oracle can tell us how to extract + // From such a convoluted encoding? + // I consulted gcc, and it gave + // These instructions, which I gladly credit: + leal (%edx,%ebx,8), %eax + movzwl (%eax),%edx + movl 4(%eax), %eax + xorw %ax, %ax + orl %eax, %edx + // Now the address of the handler's in %edx + // We call it now: its "iret" takes us home. + jmp *%edx + +// Every interrupt can come to us here +// But we must truly tell each apart. +// They number two hundred and fifty six +// And each must land in a different spot, +// Push its number on stack, and join the stream. + +// And worse, a mere six of the traps stand apart +// And push on their stack an addition: +// An error number, thirty two bits long +// So we punish the other two fifty +// And make them push a zero so they match. + +// Yet two fifty six entries is long +// And all will look most the same as the last +// So we create a macro which can make +// As many entries as we need to fill. + +// Note the change to .data then .text: +// We plant the address of each entry +// Into a (data) table for the Host +// To know where each Guest interrupt should go. +.macro IRQ_STUB N TARGET + .data; .long 1f; .text; 1: + // Trap eight, ten through fourteen and seventeen + // Supply an error number. Else zero. + .if (\N <> 8) && (\N < 10 || \N > 14) && (\N <> 17) + pushl $0 + .endif + pushl $\N + jmp \TARGET + ALIGN +.endm + +// This macro creates numerous entries +// Using GAS macros which out-power C's. +.macro IRQ_STUBS FIRST LAST TARGET + irq=\FIRST + .rept \LAST-\FIRST+1 + IRQ_STUB irq \TARGET + irq=irq+1 + .endr +.endm + +// Here's the marker for our pointer table +// Laid in the data section just before +// Each macro places the address of code +// Forming an array: each one points to text +// Which handles interrupt in its turn. +.data +.global default_idt_entries +default_idt_entries: +.text + // The first two traps go straight back to the Host + IRQ_STUBS 0 1 return_to_host + // We'll say nothing, yet, about NMI + IRQ_STUB 2 handle_nmi + // Other traps also return to the Host + IRQ_STUBS 3 31 return_to_host + // All interrupts go via their handlers + IRQ_STUBS 32 127 deliver_to_host + // 'Cept system calls coming from userspace + // Are to go to the Guest, never the Host. + IRQ_STUB 128 return_to_host + IRQ_STUBS 129 255 deliver_to_host + +// The NMI, what a fabulous beast +// Which swoops in and stops us no matter that +// We're suspended between heaven and hell, +// (Or more likely between the Host and Guest) +// When in it comes! We are dazed and confused +// So we do the simplest thing which one can. +// Though we've pushed the trap number and zero +// We discard them, return, and hope we live. +handle_nmi: + addl $8, %esp + iret + +// We are done; all that's left is Mastery +// And "make Mastery" is a journey long +// Designed to make your fingers itch to code. + +// Here ends the text, the file and poem. +ENTRY(end_switcher_text) ==================================================================--- a/drivers/lguest/switcher.S +++ /dev/null @@ -1,350 +0,0 @@ -/*P:900 This is the Switcher: code which sits at 0xFFC00000 to do the low-level - * Guest<->Host switch. It is as simple as it can be made, but it's naturally - * very specific to x86. - * - * You have now completed Preparation. If this has whet your appetite; if you - * are feeling invigorated and refreshed then the next, more challenging stage - * can be found in "make Guest". :*/ - -/*S:100 - * Welcome to the Switcher itself! - * - * This file contains the low-level code which changes the CPU to run the Guest - * code, and returns to the Host when something happens. Understand this, and - * you understand the heart of our journey. - * - * Because this is in assembler rather than C, our tale switches from prose to - * verse. First I tried limericks: - * - * There once was an eax reg, - * To which our pointer was fed, - * It needed an add, - * Which asm-offsets.h had - * But this limerick is hurting my head. - * - * Next I tried haikus, but fitting the required reference to the seasons in - * every stanza was quickly becoming tiresome: - * - * The %eax reg - * Holds "struct lguest_pages" now: - * Cherry blossoms fall. - * - * Then I started with Heroic Verse, but the rhyming requirement leeched away - * the content density and led to some uniquely awful oblique rhymes: - * - * These constants are coming from struct offsets - * For use within the asm switcher text. - * - * Finally, I settled for something between heroic hexameter, and normal prose - * with inappropriate linebreaks. Anyway, it aint no Shakespeare. - */ - -// Not all kernel headers work from assembler -// But these ones are needed: the ENTRY() define -// And constants extracted from struct offsets -// To avoid magic numbers and breakage: -// Should they change the compiler can't save us -// Down here in the depths of assembler code. -#include <linux/linkage.h> -#include <asm/asm-offsets.h> -#include <asm/page.h> -#include "lg.h" - -// We mark the start of the code to copy -// It's placed in .text tho it's never run here -// You'll see the trick macro at the end -// Which interleaves data and text to effect. -.text -ENTRY(start_switcher_text) - -// When we reach switch_to_guest we have just left -// The safe and comforting shores of C code -// %eax has the "struct lguest_pages" to use -// Where we save state and still see it from the Guest -// And %ebx holds the Guest shadow pagetable: -// Once set we have truly left Host behind. -ENTRY(switch_to_guest) - // We told gcc all its regs could fade, - // Clobbered by our journey into the Guest - // We could have saved them, if we tried - // But time is our master and cycles count. - - // Segment registers must be saved for the Host - // We push them on the Host stack for later - pushl %es - pushl %ds - pushl %gs - pushl %fs - // But the compiler is fickle, and heeds - // No warning of %ebp clobbers - // When frame pointers are used. That register - // Must be saved and restored or chaos strikes. - pushl %ebp - // The Host's stack is done, now save it away - // In our "struct lguest_pages" at offset - // Distilled into asm-offsets.h - movl %esp, LGUEST_PAGES_host_sp(%eax) - - // All saved and there's now five steps before us: - // Stack, GDT, IDT, TSS - // And last of all the page tables are flipped. - - // Yet beware that our stack pointer must be - // Always valid lest an NMI hits - // %edx does the duty here as we juggle - // %eax is lguest_pages: our stack lies within. - movl %eax, %edx - addl $LGUEST_PAGES_regs, %edx - movl %edx, %esp - - // The Guest's GDT we so carefully - // Placed in the "struct lguest_pages" before - lgdt LGUEST_PAGES_guest_gdt_desc(%eax) - - // The Guest's IDT we did partially - // Move to the "struct lguest_pages" as well. - lidt LGUEST_PAGES_guest_idt_desc(%eax) - - // The TSS entry which controls traps - // Must be loaded up with "ltr" now: - // For after we switch over our page tables - // It (as the rest) will be writable no more. - // (The GDT entry TSS needs - // Changes type when we load it: damn Intel!) - movl $(GDT_ENTRY_TSS*8), %edx - ltr %dx - - // Look back now, before we take this last step! - // The Host's TSS entry was also marked used; - // Let's clear it again, ere we return. - // The GDT descriptor of the Host - // Points to the table after two "size" bytes - movl (LGUEST_PAGES_host_gdt_desc+2)(%eax), %edx - // Clear the type field of "used" (byte 5, bit 2) - andb $0xFD, (GDT_ENTRY_TSS*8 + 5)(%edx) - - // Once our page table's switched, the Guest is live! - // The Host fades as we run this final step. - // Our "struct lguest_pages" is now read-only. - movl %ebx, %cr3 - - // The page table change did one tricky thing: - // The Guest's register page has been mapped - // Writable onto our %esp (stack) -- - // We can simply pop off all Guest regs. - popl %ebx - popl %ecx - popl %edx - popl %esi - popl %edi - popl %ebp - popl %gs - popl %eax - popl %fs - popl %ds - popl %es - - // Near the base of the stack lurk two strange fields - // Which we fill as we exit the Guest - // These are the trap number and its error - // We can simply step past them on our way. - addl $8, %esp - - // The last five stack slots hold return address - // And everything needed to change privilege - // Into the Guest privilege level of 1, - // And the stack where the Guest had last left it. - // Interrupts are turned back on: we are Guest. - iret - -// There are two paths where we switch to the Host -// So we put the routine in a macro. -// We are on our way home, back to the Host -// Interrupted out of the Guest, we come here. -#define SWITCH_TO_HOST \ - /* We save the Guest state: all registers first \ - * Laid out just as "struct lguest_regs" defines */ \ - pushl %es; \ - pushl %ds; \ - pushl %fs; \ - pushl %eax; \ - pushl %gs; \ - pushl %ebp; \ - pushl %edi; \ - pushl %esi; \ - pushl %edx; \ - pushl %ecx; \ - pushl %ebx; \ - /* Our stack and our code are using segments \ - * Set in the TSS and IDT \ - * Yet if we were to touch data we'd use \ - * Whatever data segment the Guest had. \ - * Load the lguest ds segment for now. */ \ - movl $(LGUEST_DS), %eax; \ - movl %eax, %ds; \ - /* So where are we? Which CPU, which struct? \ - * The stack is our clue: our TSS starts \ - * It at the end of "struct lguest_pages". \ - * Or we may have stumbled while restoring \ - * Our Guest segment regs while in switch_to_guest, \ - * The fault pushed atop that part-unwound stack. \ - * If we round the stack down to the page start \ - * We're at the start of "struct lguest_pages". */ \ - movl %esp, %eax; \ - andl $(~(1 << PAGE_SHIFT - 1)), %eax; \ - /* Save our trap number: the switch will obscure it \ - * (The Guest regs are not mapped here in the Host) \ - * %ebx holds it safe for deliver_to_host */ \ - movl LGUEST_PAGES_regs_trapnum(%eax), %ebx; \ - /* The Host GDT, IDT and stack! \ - * All these lie safely hidden from the Guest: \ - * We must return to the Host page tables \ - * (Hence that was saved in struct lguest_pages) */ \ - movl LGUEST_PAGES_host_cr3(%eax), %edx; \ - movl %edx, %cr3; \ - /* As before, when we looked back at the Host \ - * As we left and marked TSS unused \ - * So must we now for the Guest left behind. */ \ - andb $0xFD, (LGUEST_PAGES_guest_gdt+GDT_ENTRY_TSS*8+5)(%eax); \ - /* Switch to Host's GDT, IDT. */ \ - lgdt LGUEST_PAGES_host_gdt_desc(%eax); \ - lidt LGUEST_PAGES_host_idt_desc(%eax); \ - /* Restore the Host's stack where it's saved regs lie */ \ - movl LGUEST_PAGES_host_sp(%eax), %esp; \ - /* Last the TSS: our Host is complete */ \ - movl $(GDT_ENTRY_TSS*8), %edx; \ - ltr %dx; \ - /* Restore now the regs saved right at the first. */ \ - popl %ebp; \ - popl %fs; \ - popl %gs; \ - popl %ds; \ - popl %es - -// Here's where we come when the Guest has just trapped: -// (Which trap we'll see has been pushed on the stack). -// We need only switch back, and the Host will decode -// Why we came home, and what needs to be done. -return_to_host: - SWITCH_TO_HOST - iret - -// An interrupt, with some cause external -// Has ajerked us rudely from the Guest's code -// Again we must return home to the Host -deliver_to_host: - SWITCH_TO_HOST - // But now we must go home via that place - // Where that interrupt was supposed to go - // Had we not been ensconced, running the Guest. - // Here we see the cleverness of our stack: - // The Host stack is formed like an interrupt - // With EIP, CS and EFLAGS layered. - // Interrupt handlers end with "iret" - // And that will take us home at long long last. - - // But first we must find the handler to call! - // The IDT descriptor for the Host - // Has two bytes for size, and four for address: - // %edx will hold it for us for now. - movl (LGUEST_PAGES_host_idt_desc+2)(%eax), %edx - // We now know the table address we need, - // And saved the trap's number inside %ebx. - // Yet the pointer to the handler is smeared - // Across the bits of the table entry. - // What oracle can tell us how to extract - // From such a convoluted encoding? - // I consulted gcc, and it gave - // These instructions, which I gladly credit: - leal (%edx,%ebx,8), %eax - movzwl (%eax),%edx - movl 4(%eax), %eax - xorw %ax, %ax - orl %eax, %edx - // Now the address of the handler's in %edx - // We call it now: its "iret" takes us home. - jmp *%edx - -// Every interrupt can come to us here -// But we must truly tell each apart. -// They number two hundred and fifty six -// And each must land in a different spot, -// Push its number on stack, and join the stream. - -// And worse, a mere six of the traps stand apart -// And push on their stack an addition: -// An error number, thirty two bits long -// So we punish the other two fifty -// And make them push a zero so they match. - -// Yet two fifty six entries is long -// And all will look most the same as the last -// So we create a macro which can make -// As many entries as we need to fill. - -// Note the change to .data then .text: -// We plant the address of each entry -// Into a (data) table for the Host -// To know where each Guest interrupt should go. -.macro IRQ_STUB N TARGET - .data; .long 1f; .text; 1: - // Trap eight, ten through fourteen and seventeen - // Supply an error number. Else zero. - .if (\N <> 8) && (\N < 10 || \N > 14) && (\N <> 17) - pushl $0 - .endif - pushl $\N - jmp \TARGET - ALIGN -.endm - -// This macro creates numerous entries -// Using GAS macros which out-power C's. -.macro IRQ_STUBS FIRST LAST TARGET - irq=\FIRST - .rept \LAST-\FIRST+1 - IRQ_STUB irq \TARGET - irq=irq+1 - .endr -.endm - -// Here's the marker for our pointer table -// Laid in the data section just before -// Each macro places the address of code -// Forming an array: each one points to text -// Which handles interrupt in its turn. -.data -.global default_idt_entries -default_idt_entries: -.text - // The first two traps go straight back to the Host - IRQ_STUBS 0 1 return_to_host - // We'll say nothing, yet, about NMI - IRQ_STUB 2 handle_nmi - // Other traps also return to the Host - IRQ_STUBS 3 31 return_to_host - // All interrupts go via their handlers - IRQ_STUBS 32 127 deliver_to_host - // 'Cept system calls coming from userspace - // Are to go to the Guest, never the Host. - IRQ_STUB 128 return_to_host - IRQ_STUBS 129 255 deliver_to_host - -// The NMI, what a fabulous beast -// Which swoops in and stops us no matter that -// We're suspended between heaven and hell, -// (Or more likely between the Host and Guest) -// When in it comes! We are dazed and confused -// So we do the simplest thing which one can. -// Though we've pushed the trap number and zero -// We discard them, return, and hope we live. -handle_nmi: - addl $8, %esp - iret - -// We are done; all that's left is Mastery -// And "make Mastery" is a journey long -// Designed to make your fingers itch to code. - -// Here ends the text, the file and poem. -ENTRY(end_switcher_text) -- there are those who do and those who hang on and you don't see too many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-25 23:57 UTC
[patch 13/43] lguest: Move lguest hcalls to arch-specific header
Move architecture specific portion of lg_hcall code to asm-i386/lg_hcall.h
and have it included from linux/lguest.h.
[Changed to asm-i386/lguest_hcall.h so documentation finds it -RR]
Signed-off-by: Jes Sorensen <jes@sgi.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: Jes Sorensen <jes@sgi.com>
---
include/asm-i386/lguest_hcall.h | 67 +++++++++++++++++++++++++++++++++++++
include/linux/lguest.h | 69 ++-------------------------------------
2 files changed, 71 insertions(+), 65 deletions(-)
==================================================================--- /dev/null
+++ linux-2.6-lguest/include/asm-i386/lguest_hcall.h
@@ -0,0 +1,67 @@
+/* Architecture specific portion of the lguest hypercalls */
+#ifndef _I386_LGUEST_HCALL_H
+#define _I386_LGUEST_HCALL_H
+
+#define LHCALL_FLUSH_ASYNC 0
+#define LHCALL_LGUEST_INIT 1
+#define LHCALL_CRASH 2
+#define LHCALL_LOAD_GDT 3
+#define LHCALL_NEW_PGTABLE 4
+#define LHCALL_FLUSH_TLB 5
+#define LHCALL_LOAD_IDT_ENTRY 6
+#define LHCALL_SET_STACK 7
+#define LHCALL_TS 8
+#define LHCALL_SET_CLOCKEVENT 9
+#define LHCALL_HALT 10
+#define LHCALL_BIND_DMA 12
+#define LHCALL_SEND_DMA 13
+#define LHCALL_SET_PTE 14
+#define LHCALL_SET_PMD 15
+#define LHCALL_LOAD_TLS 16
+
+/*G:031 First, how does our Guest contact the Host to ask for privileged
+ * operations? There are two ways: the direct way is to make a
"hypercall",
+ * to make requests of the Host Itself.
+ *
+ * Our hypercall mechanism uses the highest unused trap code (traps 32 and
+ * above are used by real hardware interrupts). Seventeen hypercalls are
+ * available: the hypercall number is put in the %eax register, and the
+ * arguments (when required) are placed in %edx, %ebx and %ecx. If a return
+ * value makes sense, it's returned in %eax.
+ *
+ * Grossly invalid calls result in Sudden Death at the hands of the vengeful
+ * Host, rather than returning failure. This reflects Winston Churchill's
+ * definition of a gentleman: "someone who is only rude
intentionally". */
+#define LGUEST_TRAP_ENTRY 0x1F
+
+static inline unsigned long
+hcall(unsigned long call,
+ unsigned long arg1, unsigned long arg2, unsigned long arg3)
+{
+ /* "int" is the Intel instruction to trigger a trap. */
+ asm volatile("int $" __stringify(LGUEST_TRAP_ENTRY)
+ /* The call is in %eax (aka "a"), and can be replaced */
+ : "=a"(call)
+ /* The other arguments are in %eax, %edx, %ebx & %ecx */
+ : "a"(call), "d"(arg1), "b"(arg2),
"c"(arg3)
+ /* "memory" means this might write somewhere in memory.
+ * This isn't true for all calls, but it's safe to tell
+ * gcc that it might happen so it doesn't get clever. */
+ : "memory");
+ return call;
+}
+/*:*/
+
+void async_hcall(unsigned long call,
+ unsigned long arg1, unsigned long arg2, unsigned long arg3);
+
+/* Can't use our min() macro here: needs to be a constant */
+#define LGUEST_IRQS (NR_IRQS < 32 ? NR_IRQS: 32)
+
+#define LHCALL_RING_SIZE 64
+struct hcall_ring
+{
+ u32 eax, edx, ebx, ecx;
+};
+
+#endif /* _I386_LGUEST_HCALL_H */
==================================================================---
linux-2.6-lguest.orig/include/linux/lguest.h
+++ linux-2.6-lguest/include/linux/lguest.h
@@ -1,76 +1,15 @@
/* Things the lguest guest needs to know. Note: like all lguest interfaces,
* this is subject to wild and random change between versions. */
-#ifndef _ASM_LGUEST_H
-#define _ASM_LGUEST_H
+#ifndef _LINUX_LGUEST_H
+#define _LINUX_LGUEST_H
#ifndef __ASSEMBLY__
#include <asm/irq.h>
-
-#define LHCALL_FLUSH_ASYNC 0
-#define LHCALL_LGUEST_INIT 1
-#define LHCALL_CRASH 2
-#define LHCALL_LOAD_GDT 3
-#define LHCALL_NEW_PGTABLE 4
-#define LHCALL_FLUSH_TLB 5
-#define LHCALL_LOAD_IDT_ENTRY 6
-#define LHCALL_SET_STACK 7
-#define LHCALL_TS 8
-#define LHCALL_SET_CLOCKEVENT 9
-#define LHCALL_HALT 10
-#define LHCALL_BIND_DMA 12
-#define LHCALL_SEND_DMA 13
-#define LHCALL_SET_PTE 14
-#define LHCALL_SET_PMD 15
-#define LHCALL_LOAD_TLS 16
+#include <asm/lguest_hcall.h>
#define LG_CLOCK_MIN_DELTA 100UL
#define LG_CLOCK_MAX_DELTA ULONG_MAX
-/*G:031 First, how does our Guest contact the Host to ask for privileged
- * operations? There are two ways: the direct way is to make a
"hypercall",
- * to make requests of the Host Itself.
- *
- * Our hypercall mechanism uses the highest unused trap code (traps 32 and
- * above are used by real hardware interrupts). Seventeen hypercalls are
- * available: the hypercall number is put in the %eax register, and the
- * arguments (when required) are placed in %edx, %ebx and %ecx. If a return
- * value makes sense, it's returned in %eax.
- *
- * Grossly invalid calls result in Sudden Death at the hands of the vengeful
- * Host, rather than returning failure. This reflects Winston Churchill's
- * definition of a gentleman: "someone who is only rude
intentionally". */
-#define LGUEST_TRAP_ENTRY 0x1F
-
-static inline unsigned long
-hcall(unsigned long call,
- unsigned long arg1, unsigned long arg2, unsigned long arg3)
-{
- /* "int" is the Intel instruction to trigger a trap. */
- asm volatile("int $" __stringify(LGUEST_TRAP_ENTRY)
- /* The call is in %eax (aka "a"), and can be replaced */
- : "=a"(call)
- /* The other arguments are in %eax, %edx, %ebx & %ecx */
- : "a"(call), "d"(arg1), "b"(arg2),
"c"(arg3)
- /* "memory" means this might write somewhere in memory.
- * This isn't true for all calls, but it's safe to tell
- * gcc that it might happen so it doesn't get clever. */
- : "memory");
- return call;
-}
-/*:*/
-
-void async_hcall(unsigned long call,
- unsigned long arg1, unsigned long arg2, unsigned long arg3);
-
-/* Can't use our min() macro here: needs to be a constant */
-#define LGUEST_IRQS (NR_IRQS < 32 ? NR_IRQS: 32)
-
-#define LHCALL_RING_SIZE 64
-struct hcall_ring
-{
- u32 eax, edx, ebx, ecx;
-};
-
/*G:032 The second method of communicating with the Host is to via "struct
* lguest_data". The Guest's very first hypercall is to tell the Host
where
* this is, and then the Guest and Host both publish information in it. :*/
@@ -113,4 +52,4 @@ struct lguest_data
};
extern struct lguest_data lguest_data;
#endif /* __ASSEMBLY__ */
-#endif /* _ASM_LGUEST_H */
+#endif /* _LINUX_LGUEST_H */
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:06 UTC
[patch 18/43] lguest: Remove fixed limit on number of guests, and lguests array.
Back when we had all the Guest state in the switcher, we had a fixed
array of them. This is no longer necessary.
If we switch the network code to using random_ether_addr (46 bits is
enough to avoid clashes), we can get rid of the concept of "guest id"
altogether.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
drivers/lguest/core.c | 14 --------------
drivers/lguest/hypercalls.c | 4 +---
drivers/lguest/io.c | 10 +++++-----
drivers/lguest/lg.h | 5 +----
drivers/lguest/lguest_user.c | 17 ++++++-----------
drivers/net/lguest_net.c | 7 +------
include/linux/lguest.h | 2 --
7 files changed, 14 insertions(+), 45 deletions(-)
==================================================================---
a/drivers/lguest/core.c
+++ b/drivers/lguest/core.c
@@ -46,10 +46,6 @@ static struct {
/* This One Big lock protects all inter-guest data structures. */
DEFINE_MUTEX(lguest_lock);
static DEFINE_PER_CPU(struct lguest *, last_guest);
-
-/* FIXME: Make dynamic. */
-#define MAX_LGUEST_GUESTS 16
-struct lguest lguests[MAX_LGUEST_GUESTS];
/* Offset from where switcher.S was compiled to where we've copied it */
static unsigned long switcher_offset(void)
@@ -660,16 +656,6 @@ int run_guest(struct lguest *lg, unsigne
* deliver_trap() and demand_page(). After all those, we'll be ready to
* examine the Switcher, and our philosophical understanding of the Host/Guest
* duality will be complete. :*/
-
-int find_free_guest(void)
-{
- unsigned int i;
- for (i = 0; i < MAX_LGUEST_GUESTS; i++)
- if (!lguests[i].tsk)
- return i;
- return -1;
-}
-
static void adjust_pge(void *on)
{
if (on)
==================================================================---
a/drivers/lguest/hypercalls.c
+++ b/drivers/lguest/hypercalls.c
@@ -225,9 +225,7 @@ static void initialize(struct lguest *lg
/* We tell the Guest that it can't use the top 4MB of virtual
* addresses used by the Switcher. */
|| put_user(4U*1024*1024, &lg->lguest_data->reserve_mem)
- || put_user(tsc_speed, &lg->lguest_data->tsc_khz)
- /* We also give the Guest a unique id, as used in lguest_net.c. */
- || put_user(lg->guestid, &lg->lguest_data->guestid))
+ || put_user(tsc_speed, &lg->lguest_data->tsc_khz))
kill_guest(lg, "bad guest page %p", lg->lguest_data);
/* We write the current time into the Guest's data page once now. */
==================================================================---
a/drivers/lguest/io.c
+++ b/drivers/lguest/io.c
@@ -212,7 +212,7 @@ int bind_dma(struct lguest *lg,
lg->dma[i].num_dmas = numdmas;
lg->dma[i].next_dma = 0;
lg->dma[i].key = key;
- lg->dma[i].guestid = lg->guestid;
+ lg->dma[i].owner = lg;
lg->dma[i].interrupt = interrupt;
/* Now we add it to the hash table: the position
@@ -412,7 +412,7 @@ static int dma_transfer(struct lguest *s
/* From the "struct lguest_dma_info" we found in the hash, grab the
* Guest. */
- dstlg = &lguests[dst->guestid];
+ dstlg = dst->owner;
/* Read in the source "struct lguest_dma" handed to SEND_DMA. */
lgread(srclg, &src_dma, udma, sizeof(src_dma));
@@ -506,8 +506,8 @@ again:
struct lguest_dma_info *i;
/* Look through the hash for other Guests. */
list_for_each_entry(i, &dma_hash[hash(&key)], list) {
- /* Don't send to ourselves. */
- if (i->guestid == lg->guestid)
+ /* Don't send to ourselves (would deadlock). */
+ if (i->owner->mm == lg->mm)
continue;
if (!key_eq(&key, &i->key))
continue;
@@ -594,7 +594,7 @@ unsigned long get_dma_buffer(struct lgue
* send to its own Guest for the moment, so the entry must be for this
* Guest) */
list_for_each_entry(i, &dma_hash[hash(&key)], list) {
- if (key_eq(&key, &i->key) && i->guestid ==
lg->guestid) {
+ if (key_eq(&key, &i->key) && i->owner == lg) {
unsigned int j;
/* Look through the registered DMA array for an
* available buffer. */
==================================================================---
a/drivers/lguest/lg.h
+++ b/drivers/lguest/lg.h
@@ -52,9 +52,9 @@ struct lguest_dma_info
struct list_head list;
union futex_key key;
unsigned long dmas;
+ struct lguest *owner;
u16 next_dma;
u16 num_dmas;
- u16 guestid;
u8 interrupt; /* 0 when not registered */
};
@@ -141,7 +141,6 @@ struct lguest
struct lguest_data __user *lguest_data;
struct task_struct *tsk;
struct mm_struct *mm; /* == tsk->mm, but that becomes NULL on exit */
- u16 guestid;
u32 pfn_limit;
/* This provides the offset to the base of guest-physical
* memory in the Launcher. */
@@ -196,7 +195,6 @@ struct lguest
DECLARE_BITMAP(irqs_pending, LGUEST_IRQS);
};
-extern struct lguest lguests[];
extern struct mutex lguest_lock;
/* core.c: */
@@ -204,7 +202,6 @@ void lgwrite_u32(struct lguest *lg, unsi
void lgwrite_u32(struct lguest *lg, unsigned long addr, u32 val);
void lgread(struct lguest *lg, void *buf, unsigned long addr, unsigned len);
void lgwrite(struct lguest *lg, unsigned long, const void *buf, unsigned len);
-int find_free_guest(void);
int lguest_address_ok(const struct lguest *lg,
unsigned long addr, unsigned long len);
int run_guest(struct lguest *lg, unsigned long __user *user);
==================================================================---
a/drivers/lguest/lguest_user.c
+++ b/drivers/lguest/lguest_user.c
@@ -167,11 +167,11 @@ static int initialize(struct file *file,
/* "struct lguest" contains everything we (the Host) know about a
* Guest. */
struct lguest *lg;
- int err, i;
+ int err;
u32 args[5];
- /* We grab the Big Lguest lock, which protects the global array
- * "lguests" and multiple simultaneous initializations. */
+ /* We grab the Big Lguest lock, which protects against multiple
+ * simultaneous initializations. */
mutex_lock(&lguest_lock);
/* You can't initialize twice! Close the device and start again... */
if (file->private_data) {
@@ -184,18 +184,13 @@ static int initialize(struct file *file,
goto unlock;
}
- /* Find an unused guest. */
- i = find_free_guest();
- if (i < 0) {
- err = -ENOSPC;
+ lg = kzalloc(sizeof(*lg), GFP_KERNEL);
+ if (!lg) {
+ err = -ENOMEM;
goto unlock;
}
- /* OK, we have an index into the "lguest" array: "lg" is a
convenient
- * pointer. */
- lg = &lguests[i];
/* Populate the easy fields of our "struct lguest" */
- lg->guestid = i;
lg->mem_base = (void __user *)(long)args[0];
lg->pfn_limit = args[1];
lg->page_offset = args[4];
==================================================================---
a/drivers/net/lguest_net.c
+++ b/drivers/net/lguest_net.c
@@ -465,12 +465,7 @@ static int lguestnet_probe(struct lguest
/* Ethernet defaults with some changes */
ether_setup(dev);
dev->set_mac_address = NULL;
-
- dev->dev_addr[0] = 0x02; /* set local assignment bit (IEEE802) */
- dev->dev_addr[1] = 0x00;
- memcpy(&dev->dev_addr[2], &lguest_data.guestid, 2);
- dev->dev_addr[4] = 0x00;
- dev->dev_addr[5] = 0x00;
+ random_ether_addr(dev->dev_addr);
dev->open = lguestnet_open;
dev->stop = lguestnet_close;
==================================================================---
a/include/linux/lguest.h
+++ b/include/linux/lguest.h
@@ -41,8 +41,6 @@ struct lguest_data
/* Fields initialized by the Host at boot: */
/* Memory not to try to access */
unsigned long reserve_mem;
- /* ID of this Guest (used by network driver to set ethernet address) */
- u16 guestid;
/* KHz for the TSC clock. */
u32 tsc_khz;
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
This is an hvc-based virtio console driver. It's suboptimal becuase
hvc expects to have raw access to interrupts and virtio doesn't assume
that, so it currently polls.
There are two solutions: expose hvc's "kick" interface, or wean
off hvc.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
drivers/char/Kconfig | 4
drivers/char/Makefile | 1
drivers/char/virtio_console.c | 225 ++++++++++++++++++++++++++++++++++++++++
include/linux/virtio_console.h | 12 ++
4 files changed, 242 insertions(+)
==================================================================---
a/drivers/char/Kconfig
+++ b/drivers/char/Kconfig
@@ -613,6 +613,10 @@ config HVC_XEN
help
Xen virtual console device driver
+config VIRTIO_CONSOLE
+ bool
+ select HVC_DRIVER
+
config HVCS
tristate "IBM Hypervisor Virtual Console Server support"
depends on PPC_PSERIES
==================================================================---
a/drivers/char/Makefile
+++ b/drivers/char/Makefile
@@ -50,6 +50,7 @@ obj-$(CONFIG_HVC_BEAT) += hvc_beat.o
obj-$(CONFIG_HVC_BEAT) += hvc_beat.o
obj-$(CONFIG_HVC_DRIVER) += hvc_console.o
obj-$(CONFIG_HVC_XEN) += hvc_xen.o
+obj-$(CONFIG_VIRTIO_CONSOLE) += virtio_console.o
obj-$(CONFIG_RAW_DRIVER) += raw.o
obj-$(CONFIG_SGI_SNSC) += snsc.o snsc_event.o
obj-$(CONFIG_MSPEC) += mspec.o
==================================================================--- /dev/null
+++ b/drivers/char/virtio_console.c
@@ -0,0 +1,225 @@
+/*D:300
+ * The Guest console driver
+ *
+ * Writing console drivers is one of the few remaining Dark Arts in Linux.
+ * Fortunately for us, the path of virtual consoles has been well-trodden by
+ * the PowerPC folks, who wrote "hvc_console.c" to generically
support any
+ * virtual console. We use that infrastructure which only requires us to write
+ * the basic put_chars and get_chars functions and call the right register
+ * functions.
+ :*/
+
+/*M:002 The console can be flooded: while the Guest is processing input the
+ * Host can send more. Buffering in the Host could alleviate this, but it is a
+ * difficult problem in general. :*/
+/* Copyright (C) 2006, 2007 Rusty Russell, IBM Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+#include <linux/err.h>
+#include <linux/init.h>
+#include <linux/virtio.h>
+#include <linux/virtio_console.h>
+#include "hvc_console.h"
+
+/*D:340 These represent our input and output console queues, and the virtio
+ * operations for them. */
+static struct virtqueue *in_vq, *out_vq;
+static struct virtio_device *vdev;
+
+/* This is our input buffer, and how much data is left in it. */
+static unsigned int in_len;
+static char *in, *inbuf;
+
+/* The operations for our console. */
+static struct hv_ops virtio_cons;
+
+/*D:310 The put_chars() callback is pretty straightforward.
+ *
+ * We turn the characters into a scatter-gather list, add it to the output
+ * queue and then kick the Host. Then we sit here waiting for it to finish:
+ * inefficient in theory, but in practice implementations will do it
+ * immediately (lguest's Launcher does). */
+static int put_chars(u32 vtermno, const char *buf, int count)
+{
+ struct scatterlist sg[1];
+ unsigned int len;
+
+ /* This is a convenient routine to initialize a single-elem sg list */
+ sg_init_one(sg, buf, count);
+
+ /* add_buf wants a token to identify this buffer: we hand it any
+ * non-NULL pointer, since there's only ever one buffer. */
+ if (out_vq->vq_ops->add_buf(out_vq, sg, 1, 0, (void *)1) == 0) {
+ /* Tell Host to go! */
+ out_vq->vq_ops->kick(out_vq);
+ /* Chill out until it's done with the buffer. */
+ while (!out_vq->vq_ops->get_buf(out_vq, &len))
+ cpu_relax();
+ }
+
+ /* We're expected to return the amount of data we wrote: all of it. */
+ return count;
+}
+
+/* Create a scatter-gather list representing our input buffer and put it in the
+ * queue. */
+static void add_inbuf(void)
+{
+ struct scatterlist sg[1];
+ sg_init_one(sg, inbuf, PAGE_SIZE);
+
+ /* We should always be able to add one buffer to an empty queue. */
+ if (in_vq->vq_ops->add_buf(in_vq, sg, 0, 1, inbuf) != 0)
+ BUG();
+ in_vq->vq_ops->kick(in_vq);
+}
+
+/*D:350 get_chars() is the callback from the hvc_console infrastructure when
+ * an interrupt is received.
+ *
+ * Most of the code deals with the fact that the hvc_console() infrastructure
+ * only asks us for 16 bytes at a time. We keep in_offset and in_used fields
+ * for partially-filled buffers. */
+static int get_chars(u32 vtermno, char *buf, int count)
+{
+ /* If we don't have an input queue yet, we can't get input. */
+ BUG_ON(!in_vq);
+
+ /* No buffer? Try to get one. */
+ if (!in_len) {
+ in = in_vq->vq_ops->get_buf(in_vq, &in_len);
+ if (!in)
+ return 0;
+ }
+
+ /* You want more than we have to give? Well, try wanting less! */
+ if (in_len < count)
+ count = in_len;
+
+ /* Copy across to their buffer and increment offset. */
+ memcpy(buf, in, count);
+ in += count;
+ in_len -= count;
+
+ /* Finished? Re-register buffer so Host will use it again. */
+ if (in_len == 0)
+ add_inbuf();
+
+ return count;
+}
+/*:*/
+
+/*D:320 Console drivers are initialized very early so boot messages can go out,
+ * so we do things slightly differently from the generic virtio initialization
+ * of the net and block drivers.
+ *
+ * At this stage, the console is output-only. It's too early to set up a
+ * virtqueue, so we let the drivers do some boutique early-output thing. */
+int __init virtio_cons_early_init(int (*put_chars)(u32, const char *, int))
+{
+ virtio_cons.put_chars = put_chars;
+ return hvc_instantiate(0, 0, &virtio_cons);
+}
+
+/*D:370 Once we're further in boot, we get probed like any other virtio
device.
+ * At this stage we set up the output virtqueue.
+ *
+ * To set up and manage our virtual console, we call hvc_alloc(). Since we
+ * never remove the console device we never need this pointer again.
+ *
+ * Finally we put our input buffer in the input queue, ready to receive. */
+static int virtcons_probe(struct virtio_device *dev)
+{
+ int err;
+ struct hvc_struct *hvc;
+
+ vdev = dev;
+
+ /* This is the scratch page we use to receive console input */
+ inbuf = kmalloc(PAGE_SIZE, GFP_KERNEL);
+ if (!inbuf) {
+ err = -ENOMEM;
+ goto fail;
+ }
+
+ /* Find the input queue. */
+ /* FIXME: This is why we want to wean off hvc: we do nothing
+ * when input comes in. */
+ in_vq = vdev->config->find_vq(vdev, NULL);
+ if (IS_ERR(in_vq)) {
+ err = PTR_ERR(in_vq);
+ goto free;
+ }
+
+ out_vq = vdev->config->find_vq(vdev, NULL);
+ if (IS_ERR(out_vq)) {
+ err = PTR_ERR(out_vq);
+ goto free_in_vq;
+ }
+
+ /* Start using the new console output. */
+ virtio_cons.get_chars = get_chars;
+ virtio_cons.put_chars = put_chars;
+
+ /* The first argument of hvc_alloc() is the virtual console number, so
+ * we use zero. The second argument is the interrupt number; we
+ * currently leave this as zero: it would be better not to use the
+ * hvc mechanism and fix this (FIXME!).
+ *
+ * The third argument is a "struct hv_ops" containing the
put_chars()
+ * and get_chars() pointers. The final argument is the output buffer
+ * size: we can do any size, so we put PAGE_SIZE here. */
+ hvc = hvc_alloc(0, 0, &virtio_cons, PAGE_SIZE);
+ if (IS_ERR(hvc)) {
+ err = PTR_ERR(hvc);
+ goto free_out_vq;
+ }
+
+ /* Register the input buffer the first time. */
+ add_inbuf();
+ return 0;
+
+free_out_vq:
+ vdev->config->del_vq(out_vq);
+free_in_vq:
+ vdev->config->del_vq(in_vq);
+free:
+ kfree(inbuf);
+fail:
+ return err;
+}
+
+static struct virtio_device_id id_table[] = {
+ { VIRTIO_ID_CONSOLE, VIRTIO_DEV_ANY_ID },
+ { 0 },
+};
+
+static struct virtio_driver virtio_console = {
+ .driver.name = KBUILD_MODNAME,
+ .driver.owner = THIS_MODULE,
+ .id_table = id_table,
+ .probe = virtcons_probe,
+};
+
+static int __init init(void)
+{
+ return register_virtio_driver(&virtio_console);
+}
+module_init(init);
+
+MODULE_DEVICE_TABLE(virtio, id_table);
+MODULE_DESCRIPTION("Virtio console driver");
+MODULE_LICENSE("GPL");
==================================================================--- /dev/null
+++ b/include/linux/virtio_console.h
@@ -0,0 +1,12 @@
+#ifndef _LINUX_VIRTIO_CONSOLE_H
+#define _LINUX_VIRTIO_CONSOLE_H
+#include <linux/virtio_config.h>
+
+/* The ID for virtio console */
+#define VIRTIO_ID_CONSOLE 3
+
+#ifdef __KERNEL__
+int __init virtio_cons_early_init(int (*put_chars)(u32, const char *, int));
+#endif /* __KERNEL__ */
+
+#endif /* _LINUX_VIRTIO_CONSOLE_H */
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:06 UTC
[patch 28/43] lguest: Rename "cr3" to "gpgdir" to avoid x86-specific naming.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
drivers/lguest/lg.h | 6 +++---
drivers/lguest/page_tables.c | 18 +++++++++---------
2 files changed, 12 insertions(+), 12 deletions(-)
==================================================================---
a/drivers/lguest/lg.h
+++ b/drivers/lguest/lg.h
@@ -35,7 +35,7 @@ struct lguest_dma_info
struct pgdir
{
- unsigned long cr3;
+ unsigned long gpgdir;
pgd_t *pgdir;
};
@@ -159,10 +159,10 @@ int init_guest_pagetable(struct lguest *
int init_guest_pagetable(struct lguest *lg, unsigned long pgtable);
void free_guest_pagetable(struct lguest *lg);
void guest_new_pagetable(struct lguest *lg, unsigned long pgtable);
-void guest_set_pmd(struct lguest *lg, unsigned long cr3, u32 i);
+void guest_set_pmd(struct lguest *lg, unsigned long gpgdir, u32 i);
void guest_pagetable_clear_all(struct lguest *lg);
void guest_pagetable_flush_user(struct lguest *lg);
-void guest_set_pte(struct lguest *lg, unsigned long cr3,
+void guest_set_pte(struct lguest *lg, unsigned long gpgdir,
unsigned long vaddr, pte_t val);
void map_switcher_in_guest(struct lguest *lg, struct lguest_pages *pages);
int demand_page(struct lguest *info, unsigned long cr2, int errcode);
==================================================================---
a/drivers/lguest/page_tables.c
+++ b/drivers/lguest/page_tables.c
@@ -96,7 +96,7 @@ static unsigned long gpgd_addr(struct lg
static unsigned long gpgd_addr(struct lguest *lg, unsigned long vaddr)
{
unsigned int index = vaddr >> (PGDIR_SHIFT);
- return lg->pgdirs[lg->pgdidx].cr3 + index * sizeof(pgd_t);
+ return lg->pgdirs[lg->pgdidx].gpgdir + index * sizeof(pgd_t);
}
static unsigned long gpte_addr(struct lguest *lg,
@@ -369,7 +369,7 @@ static unsigned int find_pgdir(struct lg
{
unsigned int i;
for (i = 0; i < ARRAY_SIZE(lg->pgdirs); i++)
- if (lg->pgdirs[i].cr3 == pgtable)
+ if (lg->pgdirs[i].gpgdir == pgtable)
break;
return i;
}
@@ -378,7 +378,7 @@ static unsigned int find_pgdir(struct lg
* allocate a new one (and so the kernel parts are not there), we set
* blank_pgdir. */
static unsigned int new_pgdir(struct lguest *lg,
- unsigned long cr3,
+ unsigned long gpgdir,
int *blank_pgdir)
{
unsigned int next;
@@ -398,7 +398,7 @@ static unsigned int new_pgdir(struct lgu
*blank_pgdir = 1;
}
/* Record which Guest toplevel this shadows. */
- lg->pgdirs[next].cr3 = cr3;
+ lg->pgdirs[next].gpgdir = gpgdir;
/* Release all the non-kernel mappings. */
flush_user_mappings(lg, next);
@@ -499,7 +499,7 @@ static void do_set_pte(struct lguest *lg
* The benefit is that when we have to track a new page table, we can copy keep
* all the kernel mappings. This speeds up context switch immensely. */
void guest_set_pte(struct lguest *lg,
- unsigned long cr3, unsigned long vaddr, pte_t gpte)
+ unsigned long gpgdir, unsigned long vaddr, pte_t gpte)
{
/* Kernel mappings must be changed on all top levels. Slow, but
* doesn't happen often. */
@@ -510,7 +510,7 @@ void guest_set_pte(struct lguest *lg,
do_set_pte(lg, i, vaddr, gpte);
} else {
/* Is this page table one we have a shadow for? */
- int pgdir = find_pgdir(lg, cr3);
+ int pgdir = find_pgdir(lg, gpgdir);
if (pgdir != ARRAY_SIZE(lg->pgdirs))
/* If so, do the update. */
do_set_pte(lg, pgdir, vaddr, gpte);
@@ -531,7 +531,7 @@ void guest_set_pte(struct lguest *lg,
*
* So with that in mind here's our code to to update a (top-level) PGD
entry:
*/
-void guest_set_pmd(struct lguest *lg, unsigned long cr3, u32 idx)
+void guest_set_pmd(struct lguest *lg, unsigned long gpgdir, u32 idx)
{
int pgdir;
@@ -541,7 +541,7 @@ void guest_set_pmd(struct lguest *lg, un
return;
/* If they're talking about a page table we have a shadow for... */
- pgdir = find_pgdir(lg, cr3);
+ pgdir = find_pgdir(lg, gpgdir);
if (pgdir < ARRAY_SIZE(lg->pgdirs))
/* ... throw it away. */
release_pgd(lg, lg->pgdirs[pgdir].pgdir + idx);
@@ -561,7 +561,7 @@ int init_guest_pagetable(struct lguest *
/* We start on the first shadow page table, and give it a blank PGD
* page. */
lg->pgdidx = 0;
- lg->pgdirs[lg->pgdidx].cr3 = pgtable;
+ lg->pgdirs[lg->pgdidx].gpgdir = pgtable;
lg->pgdirs[lg->pgdidx].pgdir = (pgd_t*)get_zeroed_page(GFP_KERNEL);
if (!lg->pgdirs[lg->pgdidx].pgdir)
return -ENOMEM;
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:06 UTC
[patch 43/43] lguest: generalize lgread_u32/lgwrite_u32.
Jes complains that page table code still uses lgread_u32 even though
it now uses general kernel pte types. The best thing to do is to
generalize lgread_u32 and lgwrite_u32.
This means we lose the efficiency of getuser(). We could potentially
regain it if we used __copy_from_user instead of copy_from_user, but
I'm not certain that our range check is equivalent to access_ok() on
all platforms.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: Jes Sorensen <jes@sgi.com>
---
drivers/lguest/core.c | 39 ++++++---------------------------
drivers/lguest/hypercalls.c | 2 -
drivers/lguest/i386_core.c | 4 +--
drivers/lguest/interrupts_and_traps.c | 2 -
drivers/lguest/lg.h | 23 ++++++++++++++++---
drivers/lguest/page_tables.c | 10 ++++----
drivers/lguest/segments.c | 4 +--
7 files changed, 38 insertions(+), 46 deletions(-)
==================================================================---
a/drivers/lguest/core.c
+++ b/drivers/lguest/core.c
@@ -145,33 +145,10 @@ int lguest_address_ok(const struct lgues
return (addr+len) / PAGE_SIZE < lg->pfn_limit && (addr+len >=
addr);
}
-/* This is a convenient routine to get a 32-bit value from the Guest (a very
- * common operation). Here we can see how useful the kill_lguest() routine we
- * met in the Launcher can be: we return a random value (0) instead of needing
- * to return an error. */
-u32 lgread_u32(struct lguest *lg, unsigned long addr)
-{
- u32 val = 0;
-
- /* Don't let them access lguest binary. */
- if (!lguest_address_ok(lg, addr, sizeof(val))
- || get_user(val, (u32 *)(lg->mem_base + addr)) != 0)
- kill_guest(lg, "bad read address %#lx: pfn_limit=%u membase=%p",
addr, lg->pfn_limit, lg->mem_base);
- return val;
-}
-
-/* Same thing for writing a value. */
-void lgwrite_u32(struct lguest *lg, unsigned long addr, u32 val)
-{
- if (!lguest_address_ok(lg, addr, sizeof(val))
- || put_user(val, (u32 *)(lg->mem_base + addr)) != 0)
- kill_guest(lg, "bad write address %#lx", addr);
-}
-
-/* This routine is more generic, and copies a range of Guest bytes into a
- * buffer. If the copy_from_user() fails, we fill the buffer with zeroes, so
- * the caller doesn't end up using uninitialized kernel memory. */
-void lgread(struct lguest *lg, void *b, unsigned long addr, unsigned bytes)
+/* This routine copies memory from the Guest. Here we can see how useful the
+ * kill_lguest() routine we met in the Launcher can be: we return a random
+ * value (all zeroes) instead of needing to return an error. */
+void __lgread(struct lguest *lg, void *b, unsigned long addr, unsigned bytes)
{
if (!lguest_address_ok(lg, addr, bytes)
|| copy_from_user(b, lg->mem_base + addr, bytes) != 0) {
@@ -181,15 +158,15 @@ void lgread(struct lguest *lg, void *b,
}
}
-/* Similarly, our generic routine to copy into a range of Guest bytes. */
-void lgwrite(struct lguest *lg, unsigned long addr, const void *b,
- unsigned bytes)
+/* This is the write (copy into guest) version. */
+void __lgwrite(struct lguest *lg, unsigned long addr, const void *b,
+ unsigned bytes)
{
if (!lguest_address_ok(lg, addr, bytes)
|| copy_to_user(lg->mem_base + addr, b, bytes) != 0)
kill_guest(lg, "bad write address %#lx len %u", addr, bytes);
}
-/* (end of memory access helper routines) :*/
+/*:*/
/*H:030 Let's jump straight to the the main loop which runs the Guest.
* Remember, this is called by the Launcher reading /dev/lguest, and we keep
==================================================================---
a/drivers/lguest/hypercalls.c
+++ b/drivers/lguest/hypercalls.c
@@ -47,7 +47,7 @@ static void do_hcall(struct lguest *lg,
char msg[128];
/* If the lgread fails, it will call kill_guest() itself; the
* kill_guest() with the message will be ignored. */
- lgread(lg, msg, args->arg1, sizeof(msg));
+ __lgread(lg, msg, args->arg1, sizeof(msg));
msg[sizeof(msg)-1] = '\0';
kill_guest(lg, "CRASH: %s", msg);
break;
==================================================================---
a/drivers/lguest/i386_core.c
+++ b/drivers/lguest/i386_core.c
@@ -222,7 +222,7 @@ static int emulate_insn(struct lguest *l
return 0;
/* Decoding x86 instructions is icky. */
- lgread(lg, &insn, physaddr, 1);
+ insn = lgread(lg, &insn, u8);
/* 0x66 is an "operand prefix". It means it's using the upper
16 bits
of the eax register. */
@@ -230,7 +230,7 @@ static int emulate_insn(struct lguest *l
shift = 16;
/* The instruction is 1 byte so far, read the next byte. */
insnlen = 1;
- lgread(lg, &insn, physaddr + insnlen, 1);
+ insn = lgread(lg, physaddr + insnlen, u8);
}
/* We can ignore the lower bit for the moment and decode the 4 opcodes
==================================================================---
a/drivers/lguest/interrupts_and_traps.c
+++ b/drivers/lguest/interrupts_and_traps.c
@@ -45,7 +45,7 @@ static void push_guest_stack(struct lgue
{
/* Stack grows upwards: move stack then write value. */
*gstack -= 4;
- lgwrite_u32(lg, *gstack, val);
+ lgwrite(lg, *gstack, u32, val);
}
/*H:210 The set_guest_interrupt() routine actually delivers the interrupt or
==================================================================---
a/drivers/lguest/lg.h
+++ b/drivers/lguest/lg.h
@@ -99,12 +99,27 @@ extern struct mutex lguest_lock;
extern struct mutex lguest_lock;
/* core.c: */
-u32 lgread_u32(struct lguest *lg, unsigned long addr);
-void lgwrite_u32(struct lguest *lg, unsigned long addr, u32 val);
-void lgread(struct lguest *lg, void *buf, unsigned long addr, unsigned len);
-void lgwrite(struct lguest *lg, unsigned long, const void *buf, unsigned len);
int lguest_address_ok(const struct lguest *lg,
unsigned long addr, unsigned long len);
+void __lgread(struct lguest *, void *, unsigned long, unsigned);
+void __lgwrite(struct lguest *, unsigned long, const void *, unsigned);
+
+/*L:306 Using memory-copy operations like that is usually inconvient, so we
+ * have the following helper macros which read and write a specific type (often
+ * an unsigned long).
+ *
+ * This reads into a variable of the given type then returns that. */
+#define lgread(lg, addr, type) \
+ {( type _v; __lgread((lg), &_v, (addr), sizeof(_v)); _v; )}
+
+/* This checks that the variable is of the given type, then writes it out. */
+#define lgwrite(lg, addr, type, val) \
+ do { \
+ typecheck(type, v); \
+ __lgwrite((lg), &(v), (addr), sizeof(v)); \
+ } while(0)
+/* (end of memory access helper routines) :*/
+
int run_guest(struct lguest *lg, unsigned long __user *user);
/* Helper macros to obtain the first 12 or the last 20 bits, this is only the
==================================================================---
a/drivers/lguest/page_tables.c
+++ b/drivers/lguest/page_tables.c
@@ -209,7 +209,7 @@ int demand_page(struct lguest *lg, unsig
pte_t *spte;
/* First step: get the top-level Guest page table entry. */
- gpgd = __pgd(lgread_u32(lg, gpgd_addr(lg, vaddr)));
+ gpgd = lgread(lg, gpgd_addr(lg, vaddr), pgd_t);
/* Toplevel not present? We can't map it in. */
if (!(pgd_flags(gpgd) & _PAGE_PRESENT))
return 0;
@@ -235,7 +235,7 @@ int demand_page(struct lguest *lg, unsig
/* OK, now we look at the lower level in the Guest page table: keep its
* address, because we might update it later. */
gpte_ptr = gpte_addr(lg, gpgd, vaddr);
- gpte = __pte(lgread_u32(lg, gpte_ptr));
+ gpte = lgread(lg, gpte_ptr, pte_t);
/* If this page isn't in the Guest page tables, we can't page it in.
*/
if (!(pte_flags(gpte) & _PAGE_PRESENT))
@@ -281,7 +281,7 @@ int demand_page(struct lguest *lg, unsig
/* Finally, we write the Guest PTE entry back: we've set the
* _PAGE_ACCESSED and maybe the _PAGE_DIRTY flags. */
- lgwrite_u32(lg, gpte_ptr, pte_val(gpte));
+ lgwrite(lg, gpte_ptr, pte_t, gpte);
/* We succeeded in mapping the page! */
return 1;
@@ -369,12 +369,12 @@ unsigned long guest_pa(struct lguest *lg
pte_t gpte;
/* First step: get the top-level Guest page table entry. */
- gpgd = __pgd(lgread_u32(lg, gpgd_addr(lg, vaddr)));
+ gpgd = lgread(lg, gpgd_addr(lg, vaddr), pgd_t);
/* Toplevel not present? We can't map it in. */
if (!(pgd_flags(gpgd) & _PAGE_PRESENT))
kill_guest(lg, "Bad address %#lx", vaddr);
- gpte = __pte(lgread_u32(lg, gpte_addr(lg, gpgd, vaddr)));
+ gpte = lgread(lg, gpte_addr(lg, gpgd, vaddr), pte_t);
if (!(pte_flags(gpte) & _PAGE_PRESENT))
kill_guest(lg, "Bad address %#lx", vaddr);
==================================================================---
a/drivers/lguest/segments.c
+++ b/drivers/lguest/segments.c
@@ -150,7 +150,7 @@ void load_guest_gdt(struct lguest *lg, u
kill_guest(lg, "too many gdt entries %i", num);
/* We read the whole thing in, then fix it up. */
- lgread(lg, lg->arch.gdt, table, num * sizeof(lg->arch.gdt[0]));
+ __lgread(lg, lg->arch.gdt, table, num * sizeof(lg->arch.gdt[0]));
fixup_gdt_table(lg, 0, ARRAY_SIZE(lg->arch.gdt));
/* Mark that the GDT changed so the core knows it has to copy it again,
* even if the Guest is run on the same CPU. */
@@ -161,7 +161,7 @@ void guest_load_tls(struct lguest *lg, u
{
struct desc_struct *tls = &lg->arch.gdt[GDT_ENTRY_TLS_MIN];
- lgread(lg, tls, gtls, sizeof(*tls)*GDT_ENTRY_TLS_ENTRIES);
+ __lgread(lg, tls, gtls, sizeof(*tls)*GDT_ENTRY_TLS_ENTRIES);
fixup_gdt_table(lg, GDT_ENTRY_TLS_MIN, GDT_ENTRY_TLS_MAX+1);
lg->changed |= CHANGED_GDT_TLS;
}
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:06 UTC
[patch 31/43] lguest: Boot with virtual == physical to get closer to native Linux.
1) This allows us to get alot closer to booting bzImages.
2) It means we don't have to know page_offset.
3) The Guest needs to modify the boot pagetables to create the
PAGE_OFFSET mapping before jumping to C code.
4) guest_pa() walks the page tables rather than using page_offset.
5) We don't use page_offset to figure out whether to emulate: it was
always kinda quesationable, and won't work for instructions done
before remapping (bzImage unpacking in particular).
6) We still want the kernel address for tlb flushing: have the initial
hypercall give us that, too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
Documentation/lguest/lguest.c | 134 +++++++--------------------------
arch/i386/kernel/asm-offsets.c | 1
arch/i386/lguest/boot.c | 7 -
arch/i386/lguest/head.S | 41 ++++++++--
drivers/lguest/hypercalls.c | 8 -
drivers/lguest/i386_core.c | 7 -
drivers/lguest/interrupts_and_traps.c | 13 ++-
drivers/lguest/lg.h | 8 -
drivers/lguest/lguest_user.c | 11 --
drivers/lguest/page_tables.c | 47 +++++++++--
include/asm-i386/lguest_hcall.h | 3
include/linux/lguest.h | 5 -
12 files changed, 139 insertions(+), 146 deletions(-)
==================================================================---
a/Documentation/lguest/lguest.c
+++ b/Documentation/lguest/lguest.c
@@ -178,19 +178,16 @@ static void *get_pages(unsigned int num)
/* To find out where to start we look for the magic Guest string, which marks
* the code we see in lguest_asm.S. This is a hack which we are currently
* plotting to replace with the normal Linux entry point. */
-static unsigned long entry_point(const void *start, const void *end,
- unsigned long page_offset)
+static unsigned long entry_point(const void *start, const void *end)
{
const void *p;
- /* The scan gives us the physical starting address. We want the
- * virtual address in this case, and fortunately, we already figured
- * out the physical-virtual difference and passed it here in
- * "page_offset". */
+ /* The scan gives us the physical starting address. We boot with
+ * pagetables set up with virtual and physical the same, so that's
+ * OK. */
for (p = start; p < end; p++)
if (memcmp(p, "GenuineLguest", strlen("GenuineLguest"))
== 0)
- return to_guest_phys(p + strlen("GenuineLguest"))
- + page_offset;
+ return to_guest_phys(p + strlen("GenuineLguest"));
errx(1, "Is this image a genuine lguest?");
}
@@ -224,14 +221,11 @@ static void map_at(int fd, void *addr, u
* by all modern binaries on Linux including the kernel.
*
* The ELF headers give *two* addresses: a physical address, and a virtual
- * address. The Guest kernel expects to be placed in memory at the physical
- * address, and the page tables set up so it will correspond to that virtual
- * address. We return the difference between the virtual and physical
- * addresses in the "page_offset" pointer.
+ * address. We use the physical address; the Guest will map itself to the
+ * virtual address.
*
* We return the starting address. */
-static unsigned long map_elf(int elf_fd, const Elf32_Ehdr *ehdr,
- unsigned long *page_offset)
+static unsigned long map_elf(int elf_fd, const Elf32_Ehdr *ehdr)
{
void *start = (void *)-1, *end = NULL;
Elf32_Phdr phdr[ehdr->e_phnum];
@@ -255,9 +249,6 @@ static unsigned long map_elf(int elf_fd,
if (read(elf_fd, phdr, sizeof(phdr)) != sizeof(phdr))
err(1, "Reading program headers");
- /* We don't know page_offset yet. */
- *page_offset = 0;
-
/* Try all the headers: there are usually only three. A read-only one,
* a read-write one, and a "note" section which isn't loadable.
*/
for (i = 0; i < ehdr->e_phnum; i++) {
@@ -268,14 +259,6 @@ static unsigned long map_elf(int elf_fd,
verbose("Section %i: size %i addr %p\n",
i, phdr[i].p_memsz, (void *)phdr[i].p_paddr);
- /* We expect a simple linear address space: every segment must
- * have the same difference between virtual (p_vaddr) and
- * physical (p_paddr) address. */
- if (!*page_offset)
- *page_offset = phdr[i].p_vaddr - phdr[i].p_paddr;
- else if (*page_offset != phdr[i].p_vaddr - phdr[i].p_paddr)
- errx(1, "Page offset of section %i different", i);
-
/* We track the first and last address we mapped, so we can
* tell entry_point() where to scan. */
if (from_guest_phys(phdr[i].p_paddr) < start)
@@ -288,50 +271,13 @@ static unsigned long map_elf(int elf_fd,
phdr[i].p_offset, phdr[i].p_filesz);
}
- return entry_point(start, end, *page_offset);
-}
-
-/*L:170 Prepare to be SHOCKED and AMAZED. And possibly a trifle nauseated.
- *
- * We know that CONFIG_PAGE_OFFSET sets what virtual address the kernel expects
- * to be. We don't know what that option was, but we can figure it out
- * approximately by looking at the addresses in the code. I chose the common
- * case of reading a memory location into the %eax register:
- *
- * movl <some-address>, %eax
- *
- * This gets encoded as five bytes: "0xA1 <4-byte-address>".
For example,
- * "0xA1 0x18 0x60 0x47 0xC0" reads the address 0xC0476018 into %eax.
- *
- * In this example can guess that the kernel was compiled with
- * CONFIG_PAGE_OFFSET set to 0xC0000000 (it's always a round number). If
the
- * kernel were larger than 16MB, we might see 0xC1 addresses show up, but our
- * kernel isn't that bloated yet.
- *
- * Unfortunately, x86 has variable-length instructions, so finding this
- * particular instruction properly involves writing a disassembler. Instead,
- * we rely on statistics. We look for "0xA1" and tally the different
bytes
- * which occur 4 bytes later (the "0xC0" in our example above). When
one of
- * those bytes appears three times, we can be reasonably confident that it
- * forms the start of CONFIG_PAGE_OFFSET.
- *
- * This is amazingly reliable. */
-static unsigned long intuit_page_offset(unsigned char *img, unsigned long len)
-{
- unsigned int i, possibilities[256] = { 0 };
-
- for (i = 0; i + 4 < len; i++) {
- /* mov 0xXXXXXXXX,%eax */
- if (img[i] == 0xA1 && ++possibilities[img[i+4]] > 3)
- return (unsigned long)img[i+4] << 24;
- }
- errx(1, "could not determine page offset");
+ return entry_point(start, end);
}
/*L:160 Unfortunately the entire ELF image isn't compressed: the segments
* which need loading are extracted and compressed raw. This denies us the
* information we need to make a fully-general loader. */
-static unsigned long unpack_bzimage(int fd, unsigned long *page_offset)
+static unsigned long unpack_bzimage(int fd)
{
gzFile f;
int ret, len = 0;
@@ -352,12 +298,7 @@ static unsigned long unpack_bzimage(int
verbose("Unpacked size %i addr %p\n", len, img);
- /* Without the ELF header, we can't tell virtual-physical gap. This is
- * CONFIG_PAGE_OFFSET, and people do actually change it. Fortunately,
- * I have a clever way of figuring it out from the code itself. */
- *page_offset = intuit_page_offset(img, len);
-
- return entry_point(img, img + len, *page_offset);
+ return entry_point(img, img + len);
}
/*L:150 A bzImage, unlike an ELF file, is not meant to be loaded. You're
@@ -368,7 +309,7 @@ static unsigned long unpack_bzimage(int
* The bzImage is formed by putting the decompressing code in front of the
* compressed kernel code. So we can simple scan through it looking for the
* first "gzip" header, and start decompressing from there. */
-static unsigned long load_bzimage(int fd, unsigned long *page_offset)
+static unsigned long load_bzimage(int fd)
{
unsigned char c;
int state = 0;
@@ -396,7 +337,7 @@ static unsigned long load_bzimage(int fd
if (c != 0x03)
state = -1;
else
- return unpack_bzimage(fd, page_offset);
+ return unpack_bzimage(fd);
}
}
errx(1, "Could not find kernel in bzImage");
@@ -405,7 +346,7 @@ static unsigned long load_bzimage(int fd
/*L:140 Loading the kernel is easy when it's a "vmlinux", but
most kernels
* come wrapped up in the self-decompressing "bzImage" format. With
some funky
* coding, we can load those, too. */
-static unsigned long load_kernel(int fd, unsigned long *page_offset)
+static unsigned long load_kernel(int fd)
{
Elf32_Ehdr hdr;
@@ -415,10 +356,10 @@ static unsigned long load_kernel(int fd,
/* If it's an ELF file, it starts with "\177ELF" */
if (memcmp(hdr.e_ident, ELFMAG, SELFMAG) == 0)
- return map_elf(fd, &hdr, page_offset);
+ return map_elf(fd, &hdr);
/* Otherwise we assume it's a bzImage, and try to unpack it */
- return load_bzimage(fd, page_offset);
+ return load_bzimage(fd);
}
/* This is a trivial little helper to align pages. Andi Kleen hated it because
@@ -463,27 +404,20 @@ static unsigned long load_initrd(const c
return len;
}
-/* Once we know the address the Guest kernel expects, we can construct simple
- * linear page tables for all of memory which will get the Guest far enough
+/* Once we know how much memory we have, we can construct simple linear page
+ * tables which set virtual == physical which will get the Guest far enough
* into the boot to create its own.
*
* We lay them out of the way, just below the initrd (which is why we need to
* know its size). */
static unsigned long setup_pagetables(unsigned long mem,
- unsigned long initrd_size,
- unsigned long page_offset)
+ unsigned long initrd_size)
{
unsigned long *pgdir, *linear;
unsigned int mapped_pages, i, linear_pages;
unsigned int ptes_per_page = getpagesize()/sizeof(void *);
- /* Ideally we map all physical memory starting at page_offset.
- * However, if page_offset is 0xC0000000 we can only map 1G of physical
- * (0xC0000000 + 1G overflows). */
- if (mem <= -page_offset)
- mapped_pages = mem/getpagesize();
- else
- mapped_pages = -page_offset/getpagesize();
+ mapped_pages = mem/getpagesize();
/* Each PTE page can map ptes_per_page pages: how many do we need? */
linear_pages = (mapped_pages + ptes_per_page-1)/ptes_per_page;
@@ -500,11 +434,9 @@ static unsigned long setup_pagetables(un
for (i = 0; i < mapped_pages; i++)
linear[i] = ((i * getpagesize()) | PAGE_PRESENT);
- /* The top level points to the linear page table pages above. The
- * entry representing page_offset points to the first one, and they
- * continue from there. */
+ /* The top level points to the linear page table pages above. */
for (i = 0; i < mapped_pages; i += ptes_per_page) {
- pgdir[(i + page_offset/getpagesize())/ptes_per_page]
+ pgdir[i/ptes_per_page]
= ((to_guest_phys(linear) + i*sizeof(void *))
| PAGE_PRESENT);
}
@@ -535,15 +467,12 @@ static void concat(char *dst, char *args
/* This is where we actually tell the kernel to initialize the Guest. We saw
* the arguments it expects when we looked at initialize() in lguest_user.c:
* the base of guest "physical" memory, the top physical page to
allow, the
- * top level pagetable, the entry point and the page_offset constant for the
- * Guest. */
-static int tell_kernel(unsigned long pgdir, unsigned long start,
- unsigned long page_offset)
+ * top level pagetable and the entry point for the Guest. */
+static int tell_kernel(unsigned long pgdir, unsigned long start)
{
unsigned long args[] = { LHREQ_INITIALIZE,
(unsigned long)guest_base,
- guest_limit / getpagesize(),
- pgdir, start, page_offset };
+ guest_limit / getpagesize(), pgdir, start };
int fd;
verbose("Guest: %p - %p (%#lx)\n",
@@ -1424,9 +1353,9 @@ static void usage(void)
/*L:105 The main routine is where the real work begins: */
int main(int argc, char *argv[])
{
- /* Memory, top-level pagetable, code startpoint, PAGE_OFFSET and size
- * of the (optional) initrd. */
- unsigned long mem = 0, pgdir, start, page_offset, initrd_size = 0;
+ /* Memory, top-level pagetable, code startpoint and size of the
+ * (optional) initrd. */
+ unsigned long mem = 0, pgdir, start, initrd_size = 0;
/* A temporary and the /dev/lguest file descriptor. */
int i, c, lguest_fd;
/* The list of Guest devices, based on command line arguments. */
@@ -1500,8 +1429,7 @@ int main(int argc, char *argv[])
setup_console(&device_list);
/* Now we load the kernel */
- start = load_kernel(open_or_die(argv[optind+1], O_RDONLY),
- &page_offset);
+ start = load_kernel(open_or_die(argv[optind+1], O_RDONLY));
/* Boot information is stashed at physical address 0 */
boot = from_guest_phys(0);
@@ -1518,7 +1446,7 @@ int main(int argc, char *argv[])
}
/* Set up the initial linear pagetables, starting below the initrd. */
- pgdir = setup_pagetables(mem, initrd_size, page_offset);
+ pgdir = setup_pagetables(mem, initrd_size);
/* The Linux boot header contains an "E820" memory map: ours is a
* simple, single region. */
@@ -1535,7 +1463,7 @@ int main(int argc, char *argv[])
/* We tell the kernel to initialize the Guest: this returns the open
* /dev/lguest file descriptor. */
- lguest_fd = tell_kernel(pgdir, start, page_offset);
+ lguest_fd = tell_kernel(pgdir, start);
/* We fork off a child process, which wakes the Launcher whenever one
* of the input file descriptors needs attention. Otherwise we would
==================================================================---
a/arch/i386/kernel/asm-offsets.c
+++ b/arch/i386/kernel/asm-offsets.c
@@ -133,6 +133,7 @@ void foo(void)
#ifdef CONFIG_LGUEST_GUEST
BLANK();
OFFSET(LGUEST_DATA_irq_enabled, lguest_data, irq_enabled);
+ OFFSET(LGUEST_DATA_pgdir, lguest_data, pgdir);
OFFSET(LGUEST_PAGES_host_gdt_desc, lguest_pages, state.host_gdt_desc);
OFFSET(LGUEST_PAGES_host_idt_desc, lguest_pages, state.host_idt_desc);
OFFSET(LGUEST_PAGES_host_cr3, lguest_pages, state.host_cr3);
==================================================================---
a/arch/i386/lguest/boot.c
+++ b/arch/i386/lguest/boot.c
@@ -86,6 +86,7 @@ struct lguest_data lguest_data = {
.hcall_status = { [0 ... LHCALL_RING_SIZE-1] = 0xFF },
.noirq_start = (u32)lguest_noirq_start,
.noirq_end = (u32)lguest_noirq_end,
+ .kernel_address = PAGE_OFFSET,
.blocked_interrupts = { 1 }, /* Block timer interrupts */
.syscall_vec = SYSCALL_VECTOR,
};
@@ -988,11 +989,7 @@ __init void lguest_init(void *boot)
/*G:070 Now we've seen all the paravirt_ops, we return to
* lguest_init() where the rest of the fairly chaotic boot setup
- * occurs.
- *
- * The Host expects our first hypercall to tell it where our "struct
- * lguest_data" is, so we do that first. */
- hcall(LHCALL_LGUEST_INIT, __pa(&lguest_data), 0, 0);
+ * occurs. */
/* The native boot code sets up initial page tables immediately after
* the kernel itself, and sets init_pg_tables_end so they're not
==================================================================---
a/arch/i386/lguest/head.S
+++ b/arch/i386/lguest/head.S
@@ -1,5 +1,6 @@
#include <linux/linkage.h>
#include <linux/lguest.h>
+#include <asm/lguest_hcall.h>
#include <asm/asm-offsets.h>
#include <asm/thread_info.h>
#include <asm/processor-flags.h>
@@ -8,18 +9,48 @@
* looks for. The plan is that the Linux boot protocol will be extended with a
* "platform type" field which will guide us here from the normal
entry point,
* but for the moment this suffices. The normal boot code uses %esi for the
- * boot header, so we do too. We convert it to a virtual address by adding
- * PAGE_OFFSET, and hand it to lguest_init() as its argument (ie. %eax).
+ * boot header, so we do too.
+ *
+ * WARNING: be very careful here! We're running at addresses equal to
physical
+ * addesses (around 0), not above PAGE_OFFSET as most code expectes
+ * (eg. 0xC0000000). Jumps are relative, so they're OK, but we can't
touch any
+ * data.
*
* The .section line puts this code in .init.text so it will be discarded after
* boot. */
.section .init.text, "ax", @progbits
.ascii "GenuineLguest"
- /* Set up initial stack. */
- movl $(init_thread_union+THREAD_SIZE),%esp
+ /* Make initial hypercall now, so we can set up the pagetables. */
+ movl $LHCALL_LGUEST_INIT, %eax
+ movl $lguest_data - __PAGE_OFFSET, %edx
+ int $LGUEST_TRAP_ENTRY
+
+ /* Set up boot information pointer to hand to lguest_init(): it wants
+ * a virtual address. */
movl %esi, %eax
addl $__PAGE_OFFSET, %eax
- jmp lguest_init
+
+ /* The Host put the toplevel pagetable in lguest_data.pgdir. The movsl
+ * instruction uses %esi, so we needed to save it above. */
+ movl lguest_data - __PAGE_OFFSET + LGUEST_DATA_pgdir, %esi
+
+ /* Copy first 32 entries of page directory to __PAGE_OFFSET entries.
+ * This means the first 128M of kernel memory will be mapped at
+ * PAGE_OFFSET where the kernel expects to run. This will get it far
+ * enough through boot to switch to its own pagetables. */
+ movl $32, %ecx
+ movl %esi, %edi
+ addl $((__PAGE_OFFSET >> 22) * 4), %edi
+ rep
+ movsl
+
+ /* Set up the initial stack so we can run C code. */
+ movl $(init_thread_union+THREAD_SIZE),%esp
+
+
+ /* Jumps are relative, and we're running __PAGE_OFFSET too low at the
+ * moment. */
+ jmp lguest_init+__PAGE_OFFSET
/*G:055 We create a macro which puts the assembler code between lgstart_ and
* lgend_ markers. These templates are put in the .text section: they
can't be
==================================================================---
a/drivers/lguest/hypercalls.c
+++ b/drivers/lguest/hypercalls.c
@@ -181,14 +181,14 @@ static void initialize(struct lguest *lg
/* The Guest tells us where we're not to deliver interrupts by putting
* the range of addresses into "struct lguest_data". */
if (get_user(lg->noirq_start, &lg->lguest_data->noirq_start)
- || get_user(lg->noirq_end, &lg->lguest_data->noirq_end)
- /* We tell the Guest that it can't use the top 4MB of virtual
- * addresses used by the Switcher. */
- || put_user(4U*1024*1024, &lg->lguest_data->reserve_mem))
+ || get_user(lg->noirq_end, &lg->lguest_data->noirq_end))
kill_guest(lg, "bad guest page %p", lg->lguest_data);
/* We write the current time into the Guest's data page once now. */
write_timestamp(lg);
+
+ /* page_tables.c will also do some setup. */
+ page_table_guest_data_init(lg);
/* This is the one case where the above accesses might have been the
* first write to a Guest page. This may have caused a copy-on-write
==================================================================---
a/drivers/lguest/i386_core.c
+++ b/drivers/lguest/i386_core.c
@@ -216,9 +216,10 @@ static int emulate_insn(struct lguest *l
* guest_pa just subtracts the Guest's page_offset. */
unsigned long physaddr = guest_pa(lg, lg->regs->eip);
- /* The guest_pa() function only works for Guest kernel addresses, but
- * that's all we're trying to do anyway. */
- if (lg->regs->eip < lg->page_offset)
+ /* This must be the Guest kernel trying to do something, not userspace!
+ * The bottom two bits of the CS segment register are the privilege
+ * level. */
+ if ((lg->regs->cs & 3) != GUEST_PL)
return 0;
/* Decoding x86 instructions is icky. */
==================================================================---
a/drivers/lguest/interrupts_and_traps.c
+++ b/drivers/lguest/interrupts_and_traps.c
@@ -62,8 +62,9 @@ static void push_guest_stack(struct lgue
* it). */
static void set_guest_interrupt(struct lguest *lg, u32 lo, u32 hi, int has_err)
{
- unsigned long gstack;
+ unsigned long gstack, origstack;
u32 eflags, ss, irq_enable;
+ unsigned long virtstack;
/* There are two cases for interrupts: one where the Guest is already
* in the kernel, and a more complex one where the Guest is in
@@ -71,8 +72,10 @@ static void set_guest_interrupt(struct l
if ((lg->regs->ss&0x3) != GUEST_PL) {
/* The Guest told us their kernel stack with the SET_STACK
* hypercall: both the virtual address and the segment */
- gstack = guest_pa(lg, lg->esp1);
+ virtstack = lg->esp1;
ss = lg->ss1;
+
+ origstack = gstack = guest_pa(lg, virtstack);
/* We push the old stack segment and pointer onto the new
* stack: when the Guest does an "iret" back from the interrupt
* handler the CPU will notice they're dropping privilege
@@ -81,8 +84,10 @@ static void set_guest_interrupt(struct l
push_guest_stack(lg, &gstack, lg->regs->esp);
} else {
/* We're staying on the same Guest (kernel) stack. */
- gstack = guest_pa(lg, lg->regs->esp);
+ virtstack = lg->regs->esp;
ss = lg->regs->ss;
+
+ origstack = gstack = guest_pa(lg, virtstack);
}
/* Remember that we never let the Guest actually disable interrupts, so
@@ -108,7 +113,7 @@ static void set_guest_interrupt(struct l
/* Now we've pushed all the old state, we change the stack, the code
* segment and the address to execute. */
lg->regs->ss = ss;
- lg->regs->esp = gstack + lg->page_offset;
+ lg->regs->esp = virtstack + (gstack - origstack);
lg->regs->cs = (__KERNEL_CS|GUEST_PL);
lg->regs->eip = idt_address(lo, hi);
==================================================================---
a/drivers/lguest/lg.h
+++ b/drivers/lguest/lg.h
@@ -64,7 +64,7 @@ struct lguest
/* This provides the offset to the base of guest-physical
* memory in the Launcher. */
void __user *mem_base;
- u32 page_offset;
+ unsigned long kernel_address;
u32 cr2;
int halted;
int ts;
@@ -166,6 +166,8 @@ void map_switcher_in_guest(struct lguest
void map_switcher_in_guest(struct lguest *lg, struct lguest_pages *pages);
int demand_page(struct lguest *info, unsigned long cr2, int errcode);
void pin_page(struct lguest *lg, unsigned long vaddr);
+unsigned long guest_pa(struct lguest *lg, unsigned long vaddr);
+void page_table_guest_data_init(struct lguest *lg);
/* <arch>_core.c: */
void lguest_arch_host_init(void);
@@ -230,9 +232,5 @@ do { \
} while(0)
/* (End of aside) :*/
-static inline unsigned long guest_pa(struct lguest *lg, unsigned long vaddr)
-{
- return vaddr - lg->page_offset;
-}
#endif /* __ASSEMBLY__ */
#endif /* _LGUEST_H */
==================================================================---
a/drivers/lguest/lguest_user.c
+++ b/drivers/lguest/lguest_user.c
@@ -111,7 +111,7 @@ static ssize_t read(struct file *file, c
return run_guest(lg, (unsigned long __user *)user);
}
-/*L:020 The initialization write supplies 5 pointer sized (32 or 64 bit)
+/*L:020 The initialization write supplies 4 pointer sized (32 or 64 bit)
* values (in addition to the LHREQ_INITIALIZE value). These are:
*
* base: The start of the Guest-physical memory inside the Launcher memory.
@@ -124,12 +124,6 @@ static ssize_t read(struct file *file, c
* pagetables (which are set up by the Launcher).
*
* start: The first instruction to execute ("eip" in x86-speak).
- *
- * page_offset: The PAGE_OFFSET constant in the Guest kernel. We should
- * probably wean the code off this, but it's a very useful constant! Any
- * address above this is within the Guest kernel, and any kernel address can
- * quickly converted from physical to virtual by adding PAGE_OFFSET. It's
- * 0xC0000000 (3G) by default, but it's configurable at kernel build time.
*/
static int initialize(struct file *file, const unsigned long __user *input)
{
@@ -137,7 +131,7 @@ static int initialize(struct file *file,
* Guest. */
struct lguest *lg;
int err;
- unsigned long args[5];
+ unsigned long args[4];
/* We grab the Big Lguest lock, which protects against multiple
* simultaneous initializations. */
@@ -162,7 +156,6 @@ static int initialize(struct file *file,
/* Populate the easy fields of our "struct lguest" */
lg->mem_base = (void __user *)(long)args[0];
lg->pfn_limit = args[1];
- lg->page_offset = args[4];
/* We need a complete page for the Guest registers: they are accessible
* to the Guest and we can only grant it access to whole pages. */
==================================================================---
a/drivers/lguest/page_tables.c
+++ b/drivers/lguest/page_tables.c
@@ -13,6 +13,7 @@
#include <linux/random.h>
#include <linux/percpu.h>
#include <asm/tlbflush.h>
+#include <asm/uaccess.h>
#include "lg.h"
/*M:008 We hold reference to pages, which prevents them from being swapped.
@@ -348,7 +349,7 @@ static void flush_user_mappings(struct l
{
unsigned int i;
/* Release every pgd entry up to the kernel's address. */
- for (i = 0; i < pgd_index(lg->page_offset); i++)
+ for (i = 0; i < pgd_index(lg->kernel_address); i++)
release_pgd(lg, lg->pgdirs[idx].pgdir + i);
}
@@ -360,6 +361,25 @@ void guest_pagetable_flush_user(struct l
flush_user_mappings(lg, lg->pgdidx);
}
/*:*/
+
+/* We walk down the guest page tables to get a guest-physical address */
+unsigned long guest_pa(struct lguest *lg, unsigned long vaddr)
+{
+ pgd_t gpgd;
+ pte_t gpte;
+
+ /* First step: get the top-level Guest page table entry. */
+ gpgd = __pgd(lgread_u32(lg, gpgd_addr(lg, vaddr)));
+ /* Toplevel not present? We can't map it in. */
+ if (!(pgd_flags(gpgd) & _PAGE_PRESENT))
+ kill_guest(lg, "Bad address %#lx", vaddr);
+
+ gpte = __pte(lgread_u32(lg, gpte_addr(lg, gpgd, vaddr)));
+ if (!(pte_flags(gpte) & _PAGE_PRESENT))
+ kill_guest(lg, "Bad address %#lx", vaddr);
+
+ return pte_pfn(gpte) * PAGE_SIZE | (vaddr & ~PAGE_MASK);
+}
/* We keep several page tables. This is a simple routine to find the page
* table (if any) corresponding to this top-level address the Guest has given
@@ -503,7 +523,7 @@ void guest_set_pte(struct lguest *lg,
{
/* Kernel mappings must be changed on all top levels. Slow, but
* doesn't happen often. */
- if (vaddr >= lg->page_offset) {
+ if (vaddr >= lg->kernel_address) {
unsigned int i;
for (i = 0; i < ARRAY_SIZE(lg->pgdirs); i++)
if (lg->pgdirs[i].pgdir)
@@ -553,11 +573,6 @@ void guest_set_pmd(struct lguest *lg, un
* its first page table is. We set some things up here: */
int init_guest_pagetable(struct lguest *lg, unsigned long pgtable)
{
- /* In flush_user_mappings() we loop from 0 to
- * "pgd_index(lg->page_offset)". This assumes it won't hit
- * the Switcher mappings, so check that now. */
- if (pgd_index(lg->page_offset) >= SWITCHER_PGD_INDEX)
- return -EINVAL;
/* We start on the first shadow page table, and give it a blank PGD
* page. */
lg->pgdidx = 0;
@@ -566,6 +581,24 @@ int init_guest_pagetable(struct lguest *
if (!lg->pgdirs[lg->pgdidx].pgdir)
return -ENOMEM;
return 0;
+}
+
+/* When the Guest calls LHCALL_LGUEST_INIT we do more setup. */
+void page_table_guest_data_init(struct lguest *lg)
+{
+ /* We get the kernel address: above this is all kernel memory. */
+ if (get_user(lg->kernel_address,
&lg->lguest_data->kernel_address)
+ /* We tell the Guest that it can't use the top 4MB of virtual
+ * addresses used by the Switcher. */
+ || put_user(4U*1024*1024, &lg->lguest_data->reserve_mem)
+ ||
put_user(lg->pgdirs[lg->pgdidx].gpgdir,&lg->lguest_data->pgdir))
+ kill_guest(lg, "bad guest page %p", lg->lguest_data);
+
+ /* In flush_user_mappings() we loop from 0 to
+ * "pgd_index(lg->kernel_address)". This assumes it won't
hit the
+ * Switcher mappings, so check that now. */
+ if (pgd_index(lg->kernel_address) >= SWITCHER_PGD_INDEX)
+ kill_guest(lg, "bad kernel address %#lx", lg->kernel_address);
}
/* When a Guest dies, our cleanup is fairly simple. */
==================================================================---
a/include/asm-i386/lguest_hcall.h
+++ b/include/asm-i386/lguest_hcall.h
@@ -36,6 +36,7 @@
* definition of a gentleman: "someone who is only rude
intentionally". */
#define LGUEST_TRAP_ENTRY 0x1F
+#ifndef __ASSEMBLY__
static inline unsigned long
hcall(unsigned long call,
unsigned long arg1, unsigned long arg2, unsigned long arg3)
@@ -71,4 +72,6 @@ struct hcall_args
/* These map directly onto eax, ebx, ecx, edx in struct lguest_regs */
unsigned long arg0, arg2, arg3, arg1;
};
+
+#endif /* !__ASSEMBLY__ */
#endif /* _I386_LGUEST_HCALL_H */
==================================================================---
a/include/linux/lguest.h
+++ b/include/linux/lguest.h
@@ -44,11 +44,14 @@ struct lguest_data
unsigned long reserve_mem;
/* KHz for the TSC clock. */
u32 tsc_khz;
+ /* Page where the top-level pagetable is */
+ unsigned long pgdir;
/* Fields initialized by the Guest at boot: */
/* Instruction range to suppress interrupts even if enabled */
unsigned long noirq_start, noirq_end;
-
+ /* Address above which page tables are all identical. */
+ unsigned long kernel_address;
/* The vector to try to use for system calls (0x40 or 0x80). */
unsigned int syscall_vec;
};
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:07 UTC
[patch 12/43] lguest: Only start khvcd when someone uses hvc_console driver.
This changes hvc_init() to be called only when someone actually uses
the hvc_console driver.
hvc_console used to only be for pSeries and iSeries: now lguest and
Xen both want it built-in in case the kernel is a guest under one of
those, even though usually it will be a native boot.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Anton Blanchard <anton@au1.ibm.com>
Cc: paulus <paulus@au1.ibm.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Dave Jones <davej@codemonkey.org.uk>
---
drivers/char/Kconfig | 4 +-
drivers/char/hvc_console.c | 66 ++++++++++++++++++++++++++++++--------------
2 files changed, 47 insertions(+), 23 deletions(-)
diff -r 3c2969068458 drivers/char/Kconfig
--- a/drivers/char/Kconfig Wed Sep 26 14:46:19 2007 +1000
+++ b/drivers/char/Kconfig Wed Sep 26 14:46:50 2007 +1000
@@ -568,8 +568,8 @@ config HVC_DRIVER
config HVC_DRIVER
bool
help
- Users of pSeries machines that want to utilize the hvc console front-end
- module for their backend console driver should select this option.
+ Generic "hypervisor virtual console" infrastructure for various
+ hypervisors (pSeries, iSeries, Xen, lguest).
It will automatically be selected if one of the back-end console drivers
is selected.
diff -r 3c2969068458 drivers/char/hvc_console.c
--- a/drivers/char/hvc_console.c Wed Sep 26 14:46:19 2007 +1000
+++ b/drivers/char/hvc_console.c Wed Sep 26 14:46:21 2007 +1000
@@ -68,6 +68,8 @@ static struct task_struct *hvc_task;
/* Picks up late kicks after list walk but before schedule() */
static int hvc_kicked;
+
+static int hvc_init(void);
#ifdef CONFIG_MAGIC_SYSRQ
static int sysrq_pressed;
@@ -754,6 +756,13 @@ struct hvc_struct __devinit *hvc_alloc(u
struct hvc_struct *hp;
int i;
+ /* We wait until a driver actually comes along */
+ if (!hvc_driver) {
+ int err = hvc_init();
+ if (err)
+ return ERR_PTR(err);
+ }
+
hp = kmalloc(ALIGN(sizeof(*hp), sizeof(long)) + outbuf_size,
GFP_KERNEL);
if (!hp)
@@ -829,16 +838,18 @@ int __devexit hvc_remove(struct hvc_stru
return 0;
}
-/* Driver initialization. Follow console initialization. This is where the
TTY
- * interfaces start to become available. */
-static int __init hvc_init(void)
+/* Driver initialization: called as soon as someone uses hvc_alloc(). */
+static int hvc_init(void)
{
struct tty_driver *drv;
+ int err;
/* We need more than hvc_count adapters due to hotplug additions. */
drv = alloc_tty_driver(HVC_ALLOC_TTY_ADAPTERS);
- if (!drv)
- return -ENOMEM;
+ if (!drv) {
+ err = -ENOMEM;
+ goto out;
+ }
drv->owner = THIS_MODULE;
drv->driver_name = "hvc";
@@ -854,30 +865,43 @@ static int __init hvc_init(void)
* added later. */
hvc_task = kthread_run(khvcd, NULL, "khvcd");
if (IS_ERR(hvc_task)) {
- panic("Couldn't create kthread for console.\n");
- put_tty_driver(drv);
- return -EIO;
- }
-
- if (tty_register_driver(drv))
- panic("Couldn't register hvc console driver\n");
-
+ printk(KERN_ERR "Couldn't create kthread for console.\n");
+ err = PTR_ERR(hvc_task);
+ goto put_tty;
+ }
+
+ err = tty_register_driver(drv);
+ if (err) {
+ printk(KERN_ERR "Couldn't register hvc console driver\n");
+ goto stop_thread;
+ }
+
+ /* FIXME: This mb() seems completely random. Remove it. */
mb();
hvc_driver = drv;
return 0;
-}
-module_init(hvc_init);
+
+put_tty:
+ put_tty_driver(hvc_driver);
+stop_thread:
+ kthread_stop(hvc_task);
+ hvc_task = NULL;
+out:
+ return err;
+}
/* This isn't particularly necessary due to this being a console driver
* but it is nice to be thorough.
*/
static void __exit hvc_exit(void)
{
- kthread_stop(hvc_task);
-
- tty_unregister_driver(hvc_driver);
- /* return tty_struct instances allocated in hvc_init(). */
- put_tty_driver(hvc_driver);
- unregister_console(&hvc_con_driver);
+ if (hvc_driver) {
+ kthread_stop(hvc_task);
+
+ tty_unregister_driver(hvc_driver);
+ /* return tty_struct instances allocated in hvc_init(). */
+ put_tty_driver(hvc_driver);
+ unregister_console(&hvc_con_driver);
+ }
}
module_exit(hvc_exit);
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:07 UTC
[patch 17/43] lguest: Introduce guest mem offset, static link example launcher
In order to avoid problematic special linking of the Launcher, we give
the Host an offset: this means we can use any memory region in the
Launcher as Guest memory rather than insisting on mmap() at 0.
The result is quite pleasing: a number of casts are replaced with
simple additions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
Documentation/lguest/Makefile | 28 -----
Documentation/lguest/lguest.c | 195 +++++++++++++++++++++++------------------
drivers/lguest/core.c | 22 ++--
drivers/lguest/hypercalls.c | 15 +--
drivers/lguest/io.c | 18 ++-
drivers/lguest/lg.h | 3
drivers/lguest/lguest_user.c | 23 ++--
drivers/lguest/page_tables.c | 7 +
8 files changed, 167 insertions(+), 144 deletions(-)
==================================================================---
a/Documentation/lguest/Makefile
+++ b/Documentation/lguest/Makefile
@@ -1,28 +1,8 @@
# This creates the demonstration utility "lguest" which runs a Linux
guest.
+CFLAGS:=-Wall -Wmissing-declarations -Wmissing-prototypes -O3 -I../../include
+LDLIBS:=-lz
-# For those people that have a separate object dir, look there for .config
-KBUILD_OUTPUT := ../..
-ifdef O
- ifeq ("$(origin O)", "command line")
- KBUILD_OUTPUT := $(O)
- endif
-endif
-# We rely on CONFIG_PAGE_OFFSET to know where to put lguest binary.
-include $(KBUILD_OUTPUT)/.config
-LGUEST_GUEST_TOP := ($(CONFIG_PAGE_OFFSET) - 0x08000000)
-
-CFLAGS:=-Wall -Wmissing-declarations -Wmissing-prototypes -O3 -I../../include
-Wl,-T,lguest.lds
-LDLIBS:=-lz
-# Removing this works for some versions of ld.so (eg. Ubuntu Feisty) and
-# not others (eg. FC7).
-LDFLAGS+=-static
-all: lguest.lds lguest
-
-# The linker script on x86 is so complex the only way of creating one
-# which will link our binary in the right place is to mangle the
-# default one.
-lguest.lds:
- $(LD) --verbose | awk '/^==========/ { PRINT=1; next; } /SIZEOF_HEADERS/ {
gsub(/0x[0-9A-F]*/, "$(LGUEST_GUEST_TOP)") } { if (PRINT) print $$0;
}' > $@
+all: lguest
clean:
- rm -f lguest.lds lguest
+ rm -f lguest
==================================================================---
a/Documentation/lguest/lguest.c
+++ b/Documentation/lguest/lguest.c
@@ -1,10 +1,7 @@
/*P:100 This is the Launcher code, a simple program which lays out the
* "physical" memory for the new Guest by mapping the kernel image
and the
* virtual devices, then reads repeatedly from /dev/lguest to run the Guest.
- *
- * The only trick: the Makefile links it at a high address so it will be clear
- * of the guest memory region. It means that each Guest cannot have more than
- * about 2.5G of memory on a normally configured Host. :*/
+:*/
#define _LARGEFILE64_SOURCE
#define _GNU_SOURCE
#include <stdio.h>
@@ -56,6 +53,8 @@ typedef uint8_t u8;
#ifndef SIOCBRADDIF
#define SIOCBRADDIF 0x89a2 /* add interface to bridge */
#endif
+/* We can have up to 256 pages for devices. */
+#define DEVICE_PAGES 256
/*L:120 verbose is both a global flag and a macro. The C preprocessor allows
* this, and although I wouldn't recommend it, it works quite nicely here.
*/
@@ -66,8 +65,10 @@ static bool verbose;
/* The pipe to send commands to the waker process */
static int waker_fd;
-/* The top of guest physical memory. */
-static u32 top;
+/* The pointer to the start of guest memory. */
+static void *guest_base;
+/* The maximum guest physical address allowed, and maximum possible. */
+static unsigned long guest_limit, guest_max;
/* This is our list of devices. */
struct device_list
@@ -111,6 +112,29 @@ struct device
void *priv;
};
+/*L:100 The Launcher code itself takes us out into userspace, that scary place
+ * where pointers run wild and free! Unfortunately, like most userspace
+ * programs, it's quite boring (which is why everyone likes to hack on the
+ * kernel!). Perhaps if you make up an Lguest Drinking Game at this point, it
+ * will get you through this section. Or, maybe not.
+ *
+ * The Launcher sets up a big chunk of memory to be the Guest's
"physical"
+ * memory and stores it in "guest_base". In other words, Guest
physical =+ * Launcher virtual with an offset.
+ *
+ * This can be tough to get your head around, but usually it just means that we
+ * use these trivial conversion functions when the Guest gives us it's
+ * "physical" addresses: */
+static void *from_guest_phys(unsigned long addr)
+{
+ return guest_base + addr;
+}
+
+static unsigned long to_guest_phys(const void *addr)
+{
+ return (addr - guest_base);
+}
+
/*L:130
* Loading the Kernel.
*
@@ -124,33 +148,40 @@ static int open_or_die(const char *name,
return fd;
}
-/* map_zeroed_pages() takes a (page-aligned) address and a number of pages. */
-static void *map_zeroed_pages(unsigned long addr, unsigned int num)
-{
- /* We cache the /dev/zero file-descriptor so we only open it once. */
- static int fd = -1;
-
- if (fd == -1)
- fd = open_or_die("/dev/zero", O_RDONLY);
+/* map_zeroed_pages() takes a number of pages. */
+static void *map_zeroed_pages(unsigned int num)
+{
+ int fd = open_or_die("/dev/zero", O_RDONLY);
+ void *addr;
/* We use a private mapping (ie. if we write to the page, it will be
- * copied), and obviously we insist that it be mapped where we ask. */
- if (mmap((void *)addr, getpagesize() * num,
- PROT_READ|PROT_WRITE|PROT_EXEC, MAP_FIXED|MAP_PRIVATE, fd, 0)
- != (void *)addr)
- err(1, "Mmaping %u pages of /dev/zero @%p", num, (void *)addr);
-
- /* Returning the address is just a courtesy: can simplify callers. */
- return (void *)addr;
+ * copied). */
+ addr = mmap(NULL, getpagesize() * num,
+ PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, fd, 0);
+ if (addr == MAP_FAILED)
+ err(1, "Mmaping %u pages of /dev/zero", num);
+
+ return addr;
+}
+
+/* Get some more pages for a device. */
+static void *get_pages(unsigned int num)
+{
+ void *addr = from_guest_phys(guest_limit);
+
+ guest_limit += num * getpagesize();
+ if (guest_limit > guest_max)
+ errx(1, "Not enough memory for devices");
+ return addr;
}
/* To find out where to start we look for the magic Guest string, which marks
* the code we see in lguest_asm.S. This is a hack which we are currently
* plotting to replace with the normal Linux entry point. */
-static unsigned long entry_point(void *start, void *end,
+static unsigned long entry_point(const void *start, const void *end,
unsigned long page_offset)
{
- void *p;
+ const void *p;
/* The scan gives us the physical starting address. We want the
* virtual address in this case, and fortunately, we already figured
@@ -158,7 +189,8 @@ static unsigned long entry_point(void *s
* "page_offset". */
for (p = start; p < end; p++)
if (memcmp(p, "GenuineLguest", strlen("GenuineLguest"))
== 0)
- return (long)p + strlen("GenuineLguest") + page_offset;
+ return to_guest_phys(p + strlen("GenuineLguest"))
+ + page_offset;
errx(1, "Is this image a genuine lguest?");
}
@@ -201,9 +233,9 @@ static unsigned long map_elf(int elf_fd,
static unsigned long map_elf(int elf_fd, const Elf32_Ehdr *ehdr,
unsigned long *page_offset)
{
+ void *start = (void *)-1, *end = NULL;
Elf32_Phdr phdr[ehdr->e_phnum];
unsigned int i;
- unsigned long start = -1UL, end = 0;
/* Sanity checks on the main ELF header: an x86 executable with a
* reasonable number of correctly-sized program headers. */
@@ -246,17 +278,17 @@ static unsigned long map_elf(int elf_fd,
/* We track the first and last address we mapped, so we can
* tell entry_point() where to scan. */
- if (phdr[i].p_paddr < start)
- start = phdr[i].p_paddr;
- if (phdr[i].p_paddr + phdr[i].p_filesz > end)
- end = phdr[i].p_paddr + phdr[i].p_filesz;
+ if (from_guest_phys(phdr[i].p_paddr) < start)
+ start = from_guest_phys(phdr[i].p_paddr);
+ if (from_guest_phys(phdr[i].p_paddr) + phdr[i].p_filesz > end)
+ end=from_guest_phys(phdr[i].p_paddr)+phdr[i].p_filesz;
/* We map this section of the file at its physical address. */
- map_at(elf_fd, (void *)phdr[i].p_paddr,
+ map_at(elf_fd, from_guest_phys(phdr[i].p_paddr),
phdr[i].p_offset, phdr[i].p_filesz);
}
- return entry_point((void *)start, (void *)end, *page_offset);
+ return entry_point(start, end, *page_offset);
}
/*L:170 Prepare to be SHOCKED and AMAZED. And possibly a trifle nauseated.
@@ -307,7 +339,7 @@ static unsigned long unpack_bzimage(int
* actually configurable as CONFIG_PHYSICAL_START, but as the comment
* there says, "Don't change this unless you know what you are
doing".
* Indeed. */
- void *img = (void *)0x100000;
+ void *img = from_guest_phys(0x100000);
/* gzdopen takes our file descriptor (carefully placed at the start of
* the GZIP header we found) and returns a gzFile. */
@@ -421,7 +453,7 @@ static unsigned long load_initrd(const c
/* We map the initrd at the top of memory, but mmap wants it to be
* page-aligned, so we round the size up for that. */
len = page_align(st.st_size);
- map_at(ifd, (void *)mem - len, 0, st.st_size);
+ map_at(ifd, from_guest_phys(mem - len), 0, st.st_size);
/* Once a file is mapped, you can close the file descriptor. It's a
* little odd, but quite useful. */
close(ifd);
@@ -431,9 +463,9 @@ static unsigned long load_initrd(const c
return len;
}
-/* Once we know how much memory we have, and the address the Guest kernel
- * expects, we can construct simple linear page tables which will get the Guest
- * far enough into the boot to create its own.
+/* Once we know the address the Guest kernel expects, we can construct simple
+ * linear page tables for all of memory which will get the Guest far enough
+ * into the boot to create its own.
*
* We lay them out of the way, just below the initrd (which is why we need to
* know its size). */
@@ -457,7 +489,7 @@ static unsigned long setup_pagetables(un
linear_pages = (mapped_pages + ptes_per_page-1)/ptes_per_page;
/* We put the toplevel page directory page at the top of memory. */
- pgdir = (void *)mem - initrd_size - getpagesize();
+ pgdir = from_guest_phys(mem) - initrd_size - getpagesize();
/* Now we use the next linear_pages pages as pte pages */
linear = (void *)pgdir - linear_pages*getpagesize();
@@ -473,15 +505,16 @@ static unsigned long setup_pagetables(un
* continue from there. */
for (i = 0; i < mapped_pages; i += ptes_per_page) {
pgdir[(i + page_offset/getpagesize())/ptes_per_page]
- = (((u32)linear + i*sizeof(u32)) | PAGE_PRESENT);
- }
-
- verbose("Linear mapping of %u pages in %u pte pages at %p\n",
- mapped_pages, linear_pages, linear);
+ = ((to_guest_phys(linear) + i*sizeof(u32))
+ | PAGE_PRESENT);
+ }
+
+ verbose("Linear mapping of %u pages in %u pte pages at %#lx\n",
+ mapped_pages, linear_pages, to_guest_phys(linear));
/* We return the top level (guest-physical) address: the kernel needs
* to know where it is. */
- return (unsigned long)pgdir;
+ return to_guest_phys(pgdir);
}
/* Simple routine to roll all the commandline arguments together with spaces
@@ -501,14 +534,19 @@ static void concat(char *dst, char *args
/* This is where we actually tell the kernel to initialize the Guest. We saw
* the arguments it expects when we looked at initialize() in lguest_user.c:
- * the top physical page to allow, the top level pagetable, the entry point and
- * the page_offset constant for the Guest. */
+ * the base of guest "physical" memory, the top physical page to
allow, the
+ * top level pagetable, the entry point and the page_offset constant for the
+ * Guest. */
static int tell_kernel(u32 pgdir, u32 start, u32 page_offset)
{
u32 args[] = { LHREQ_INITIALIZE,
- top/getpagesize(), pgdir, start, page_offset };
+ (unsigned long)guest_base,
+ guest_limit / getpagesize(),
+ pgdir, start, page_offset };
int fd;
+ verbose("Guest: %p - %p (%#lx)\n",
+ guest_base, guest_base + guest_limit, guest_limit);
fd = open_or_die("/dev/lguest", O_RDWR);
if (write(fd, args, sizeof(args)) < 0)
err(1, "Writing to /dev/lguest");
@@ -605,11 +643,11 @@ static void *_check_pointer(unsigned lon
{
/* We have to separately check addr and addr+size, because size could
* be huge and addr + size might wrap around. */
- if (addr >= top || addr + size >= top)
+ if (addr >= guest_limit || addr + size >= guest_limit)
errx(1, "%s:%i: Invalid address %li", __FILE__, line, addr);
/* We return a pointer for the caller's convenience, now we know it's
* safe to use. */
- return (void *)addr;
+ return from_guest_phys(addr);
}
/* A macro which transparently hands the line number to the real function. */
#define check_pointer(addr,size) _check_pointer(addr, size, __LINE__)
@@ -646,7 +684,7 @@ static u32 *get_dma_buffer(int fd, void
static u32 *get_dma_buffer(int fd, void *key,
struct iovec iov[], unsigned int *num, u32 *irq)
{
- u32 buf[] = { LHREQ_GETDMA, (u32)key };
+ u32 buf[] = { LHREQ_GETDMA, to_guest_phys(key) };
unsigned long udma;
u32 *res;
@@ -998,11 +1036,11 @@ new_dev_desc(struct lguest_device_desc *
descs[i].features = features;
descs[i].num_pages = num_pages;
/* If they said the device needs memory, we allocate
- * that now, bumping up the top of Guest memory. */
+ * that now. */
if (num_pages) {
- map_zeroed_pages(top, num_pages);
- descs[i].pfn = top/getpagesize();
- top += num_pages*getpagesize();
+ unsigned long pa;
+ pa = to_guest_phys(get_pages(num_pages));
+ descs[i].pfn = pa / getpagesize();
}
return &descs[i];
}
@@ -1040,9 +1078,9 @@ static struct device *new_device(struct
if (handle_input)
set_fd(dev->fd, devices);
dev->desc = new_dev_desc(devices->descs, type, features, num_pages);
- dev->mem = (void *)(dev->desc->pfn * getpagesize());
+ dev->mem = from_guest_phys(dev->desc->pfn * getpagesize());
dev->handle_input = handle_input;
- dev->watch_key = (unsigned long)dev->mem + watch_off;
+ dev->watch_key = to_guest_phys(dev->mem) + watch_off;
dev->handle_output = handle_output;
return dev;
}
@@ -1382,21 +1420,7 @@ static void usage(void)
"<mem-in-mb> vmlinux [args...]");
}
-/*L:100 The Launcher code itself takes us out into userspace, that scary place
- * where pointers run wild and free! Unfortunately, like most userspace
- * programs, it's quite boring (which is why everyone like to hack on the
- * kernel!). Perhaps if you make up an Lguest Drinking Game at this point, it
- * will get you through this section. Or, maybe not.
- *
- * The Launcher binary sits up high, usually starting at address 0xB8000000.
- * Everything below this is the "physical" memory for the Guest. For
example,
- * if the Guest were to write a "1" at physical address 0, we would
see a "1"
- * in the Launcher at "(int *)0". Guest physical == Launcher
virtual.
- *
- * This can be tough to get your head around, but usually it just means that we
- * don't need to do any conversion when the Guest gives us it's
"physical"
- * addresses.
- */
+/*L:105 The main routine is where the real work begins: */
int main(int argc, char *argv[])
{
/* Memory, top-level pagetable, code startpoint, PAGE_OFFSET and size
@@ -1406,8 +1430,8 @@ int main(int argc, char *argv[])
int i, c, lguest_fd;
/* The list of Guest devices, based on command line arguments. */
struct device_list device_list;
- /* The boot information for the Guest: at guest-physical address 0. */
- void *boot = (void *)0;
+ /* The boot information for the Guest. */
+ void *boot;
/* If they specify an initrd file to load. */
const char *initrd_name = NULL;
@@ -1427,9 +1451,16 @@ int main(int argc, char *argv[])
* of memory now. */
for (i = 1; i < argc; i++) {
if (argv[i][0] != '-') {
- mem = top = atoi(argv[i]) * 1024 * 1024;
- device_list.descs = map_zeroed_pages(top, 1);
- top += getpagesize();
+ mem = atoi(argv[i]) * 1024 * 1024;
+ /* We start by mapping anonymous pages over all of
+ * guest-physical memory range. This fills it with 0,
+ * and ensures that the Guest won't be killed when it
+ * tries to access it. */
+ guest_base = map_zeroed_pages(mem / getpagesize()
+ + DEVICE_PAGES);
+ guest_limit = mem;
+ guest_max = mem + DEVICE_PAGES*getpagesize();
+ device_list.descs = get_pages(1);
break;
}
}
@@ -1462,17 +1493,17 @@ int main(int argc, char *argv[])
if (optind + 2 > argc)
usage();
+ verbose("Guest base is at %p\n", guest_base);
+
/* We always have a console device */
setup_console(&device_list);
-
- /* We start by mapping anonymous pages over all of guest-physical
- * memory range. This fills it with 0, and ensures that the Guest
- * won't be killed when it tries to access it. */
- map_zeroed_pages(0, mem / getpagesize());
/* Now we load the kernel */
start = load_kernel(open_or_die(argv[optind+1], O_RDONLY),
&page_offset);
+
+ /* Boot information is stashed at physical address 0 */
+ boot = from_guest_phys(0);
/* Map the initrd image if requested (at top of physical memory) */
if (initrd_name) {
@@ -1495,7 +1526,7 @@ int main(int argc, char *argv[])
= ((struct e820entry) { 0, mem, E820_RAM });
/* The boot header contains a command line pointer: we put the command
* line after the boot header (at address 4096) */
- *(void **)(boot + 0x228) = boot + 4096;
+ *(u32 *)(boot + 0x228) = 4096;
concat(boot + 4096, argv+optind+2);
/* The guest type value of "1" tells the Guest it's under
lguest. */
==================================================================---
a/drivers/lguest/core.c
+++ b/drivers/lguest/core.c
@@ -325,8 +325,8 @@ static int emulate_insn(struct lguest *l
* Dealing With Guest Memory.
*
* When the Guest gives us (what it thinks is) a physical address, we can use
- * the normal copy_from_user() & copy_to_user() on that address: remember,
- * Guest physical == Launcher virtual.
+ * the normal copy_from_user() & copy_to_user() on the corresponding place
in
+ * the memory region allocated by the Launcher.
*
* But we can't trust the Guest: it might be trying to access the Launcher
* code. We have to check that the range is below the pfn_limit the Launcher
@@ -348,8 +348,8 @@ u32 lgread_u32(struct lguest *lg, unsign
/* Don't let them access lguest binary. */
if (!lguest_address_ok(lg, addr, sizeof(val))
- || get_user(val, (u32 __user *)addr) != 0)
- kill_guest(lg, "bad read address %#lx", addr);
+ || get_user(val, (u32 *)(lg->mem_base + addr)) != 0)
+ kill_guest(lg, "bad read address %#lx: pfn_limit=%u membase=%p",
addr, lg->pfn_limit, lg->mem_base);
return val;
}
@@ -357,7 +357,7 @@ void lgwrite_u32(struct lguest *lg, unsi
void lgwrite_u32(struct lguest *lg, unsigned long addr, u32 val)
{
if (!lguest_address_ok(lg, addr, sizeof(val))
- || put_user(val, (u32 __user *)addr) != 0)
+ || put_user(val, (u32 *)(lg->mem_base + addr)) != 0)
kill_guest(lg, "bad write address %#lx", addr);
}
@@ -367,7 +367,7 @@ void lgread(struct lguest *lg, void *b,
void lgread(struct lguest *lg, void *b, unsigned long addr, unsigned bytes)
{
if (!lguest_address_ok(lg, addr, bytes)
- || copy_from_user(b, (void __user *)addr, bytes) != 0) {
+ || copy_from_user(b, lg->mem_base + addr, bytes) != 0) {
/* copy_from_user should do this, but as we rely on it... */
memset(b, 0, bytes);
kill_guest(lg, "bad read address %#lx len %u", addr, bytes);
@@ -379,7 +379,7 @@ void lgwrite(struct lguest *lg, unsigned
unsigned bytes)
{
if (!lguest_address_ok(lg, addr, bytes)
- || copy_to_user((void __user *)addr, b, bytes) != 0)
+ || copy_to_user(lg->mem_base + addr, b, bytes) != 0)
kill_guest(lg, "bad write address %#lx len %u", addr, bytes);
}
/* (end of memory access helper routines) :*/
@@ -616,11 +616,9 @@ int run_guest(struct lguest *lg, unsigne
*
* Note that if the Guest were really messed up, this
* could happen before it's done the INITIALIZE
- * hypercall, so lg->lguest_data will be NULL, so
- * &lg->lguest_data->cr2 will be address 8. Writing
- * into that address won't hurt the Host at all,
- * though. */
- if (put_user(cr2, &lg->lguest_data->cr2))
+ * hypercall, so lg->lguest_data will be NULL */
+ if (lg->lguest_data
+ && put_user(cr2, &lg->lguest_data->cr2))
kill_guest(lg, "Writing cr2");
break;
case 7: /* We've intercepted a Device Not Available fault. */
==================================================================---
a/drivers/lguest/hypercalls.c
+++ b/drivers/lguest/hypercalls.c
@@ -205,16 +205,19 @@ static void initialize(struct lguest *lg
tsc_speed = 0;
/* The pointer to the Guest's "struct lguest_data" is the only
- * argument. */
- lg->lguest_data = (struct lguest_data __user *)lg->regs->edx;
- /* If we check the address they gave is OK now, we can simply
- * copy_to_user/from_user from now on rather than using lgread/lgwrite.
- * I put this in to show that I'm not immune to writing stupid
- * optimizations. */
+ * argument. We check that address now. */
if (!lguest_address_ok(lg, lg->regs->edx, sizeof(*lg->lguest_data)))
{
kill_guest(lg, "bad guest page %p", lg->lguest_data);
return;
}
+
+ /* Having checked it, we simply set lg->lguest_data to point straight
+ * into the Launcher's memory at the right place and then use
+ * copy_to_user/from_user from now on, instead of lgread/write. I put
+ * this in to show that I'm not immune to writing stupid
+ * optimizations. */
+ lg->lguest_data = lg->mem_base + lg->regs->edx;
+
/* The Guest tells us where we're not to deliver interrupts by putting
* the range of addresses into "struct lguest_data". */
if (get_user(lg->noirq_start, &lg->lguest_data->noirq_start)
==================================================================---
a/drivers/lguest/io.c
+++ b/drivers/lguest/io.c
@@ -186,7 +186,7 @@ int bind_dma(struct lguest *lg,
* we're doing this. */
mutex_lock(&lguest_lock);
down_read(fshared);
- if (get_futex_key((u32 __user *)ukey, fshared, &key) != 0) {
+ if (get_futex_key(lg->mem_base + ukey, fshared, &key) != 0) {
kill_guest(lg, "bad dma key %#lx", ukey);
goto unlock;
}
@@ -247,7 +247,8 @@ static int lgread_other(struct lguest *l
void *buf, u32 addr, unsigned bytes)
{
if (!lguest_address_ok(lg, addr, bytes)
- || access_process_vm(lg->tsk, addr, buf, bytes, 0) != bytes) {
+ || access_process_vm(lg->tsk, (unsigned long)lg->mem_base + addr,
+ buf, bytes, 0) != bytes) {
memset(buf, 0, bytes);
kill_guest(lg, "bad address in registered DMA struct");
return 0;
@@ -261,8 +262,8 @@ static int lgwrite_other(struct lguest *
const void *buf, unsigned bytes)
{
if (!lguest_address_ok(lg, addr, bytes)
- || (access_process_vm(lg->tsk, addr, (void *)buf, bytes, 1)
- != bytes)) {
+ || access_process_vm(lg->tsk, (unsigned long)lg->mem_base + addr,
+ (void *)buf, bytes, 1) != bytes) {
kill_guest(lg, "bad address writing to registered DMA");
return 0;
}
@@ -318,7 +319,7 @@ static u32 copy_data(struct lguest *srcl
* copy_to_user_page(), and some arch's seem to need special
* flushes. x86 is fine. */
if (copy_from_user(maddr + (dst->addr[di] + dstoff)%PAGE_SIZE,
- (void __user *)src->addr[si], len) != 0) {
+ srclg->mem_base+src->addr[si], len) != 0) {
/* If a copy failed, it's the source's fault. */
kill_guest(srclg, "bad address in sending DMA");
totlen = 0;
@@ -377,7 +378,8 @@ static u32 do_dma(struct lguest *srclg,
* number of pages. Note that we're holding the destination's
* mmap_sem, as get_user_pages() requires. */
if (get_user_pages(dstlg->tsk, dstlg->mm,
- dst->addr[i], 1, 1, 1, pages+i, NULL)
+ (unsigned long)dstlg->mem_base+dst->addr[i],
+ 1, 1, 1, pages+i, NULL)
!= 1) {
/* This means the destination gave us a bogus buffer */
kill_guest(dstlg, "Error mapping DMA pages");
@@ -493,7 +495,7 @@ again:
mutex_lock(&lguest_lock);
down_read(fshared);
/* Get the futex key for the key the Guest gave us */
- if (get_futex_key((u32 __user *)ukey, fshared, &key) != 0) {
+ if (get_futex_key(lg->mem_base + ukey, fshared, &key) != 0) {
kill_guest(lg, "bad sending DMA key");
goto unlock;
}
@@ -584,7 +586,7 @@ unsigned long get_dma_buffer(struct lgue
/* This can fail if it's not a valid address, or if the address is not
* divisible by 4 (the futex code needs that, we don't really). */
- if (get_futex_key((u32 __user *)ukey, fshared, &key) != 0) {
+ if (get_futex_key(lg->mem_base + ukey, fshared, &key) != 0) {
kill_guest(lg, "bad registered DMA buffer");
goto unlock;
}
==================================================================---
a/drivers/lguest/lg.h
+++ b/drivers/lguest/lg.h
@@ -143,6 +143,9 @@ struct lguest
struct mm_struct *mm; /* == tsk->mm, but that becomes NULL on exit */
u16 guestid;
u32 pfn_limit;
+ /* This provides the offset to the base of guest-physical
+ * memory in the Launcher. */
+ void __user *mem_base;
u32 page_offset;
u32 cr2;
int halted;
==================================================================---
a/drivers/lguest/lguest_user.c
+++ b/drivers/lguest/lguest_user.c
@@ -1,9 +1,9 @@
/*P:200 This contains all the /dev/lguest code, whereby the userspace launcher
* controls and communicates with the Guest. For example, the first write will
- * tell us the memory size, pagetable, entry point and kernel address offset.
- * A read will run the Guest until a signal is pending (-EINTR), or the Guest
- * does a DMA out to the Launcher. Writes are also used to get a DMA buffer
- * registered by the Guest and to send the Guest an interrupt. :*/
+ * tell us the Guest's memory layout, pagetable, entry point and kernel
address
+ * offset. A read will run the Guest until something happens, such as a signal
+ * or the Guest doing a DMA out to the Launcher. Writes are also used to get a
+ * DMA buffer registered by the Guest and to send the Guest an interrupt. :*/
#include <linux/uaccess.h>
#include <linux/miscdevice.h>
#include <linux/fs.h>
@@ -142,8 +142,10 @@ static ssize_t read(struct file *file, c
return run_guest(lg, (unsigned long __user *)user);
}
-/*L:020 The initialization write supplies 4 32-bit values (in addition to the
+/*L:020 The initialization write supplies 5 32-bit values (in addition to the
* 32-bit LHREQ_INITIALIZE value). These are:
+ *
+ * base: The start of the Guest-physical memory inside the Launcher memory.
*
* pfnlimit: The highest (Guest-physical) page number the Guest should be
* allowed to access. The Launcher has to live in Guest memory, so it sets
@@ -166,7 +168,7 @@ static int initialize(struct file *file,
* Guest. */
struct lguest *lg;
int err, i;
- u32 args[4];
+ u32 args[5];
/* We grab the Big Lguest lock, which protects the global array
* "lguests" and multiple simultaneous initializations. */
@@ -194,8 +196,9 @@ static int initialize(struct file *file,
/* Populate the easy fields of our "struct lguest" */
lg->guestid = i;
- lg->pfn_limit = args[0];
- lg->page_offset = args[3];
+ lg->mem_base = (void __user *)(long)args[0];
+ lg->pfn_limit = args[1];
+ lg->page_offset = args[4];
/* We need a complete page for the Guest registers: they are accessible
* to the Guest and we can only grant it access to whole pages. */
@@ -210,13 +213,13 @@ static int initialize(struct file *file,
/* Initialize the Guest's shadow page tables, using the toplevel
* address the Launcher gave us. This allocates memory, so can
* fail. */
- err = init_guest_pagetable(lg, args[1]);
+ err = init_guest_pagetable(lg, args[2]);
if (err)
goto free_regs;
/* Now we initialize the Guest's registers, handing it the start
* address. */
- setup_regs(lg->regs, args[2]);
+ setup_regs(lg->regs, args[3]);
/* There are a couple of GDT entries the Guest expects when first
* booting. */
==================================================================---
a/drivers/lguest/page_tables.c
+++ b/drivers/lguest/page_tables.c
@@ -152,7 +152,7 @@ static spte_t gpte_to_spte(struct lguest
static spte_t gpte_to_spte(struct lguest *lg, gpte_t gpte, int write)
{
spte_t spte;
- unsigned long pfn;
+ unsigned long pfn, base;
/* The Guest sets the global flag, because it thinks that it is using
* PGE. We only told it to use PGE so it would tell us whether it was
@@ -160,11 +160,14 @@ static spte_t gpte_to_spte(struct lguest
* use the global bit, so throw it away. */
spte.flags = (gpte.flags & ~_PAGE_GLOBAL);
+ /* The Guest's pages are offset inside the Launcher. */
+ base = (unsigned long)lg->mem_base / PAGE_SIZE;
+
/* We need a temporary "unsigned long" variable to hold the answer
from
* get_pfn(), because it returns 0xFFFFFFFF on failure, which wouldn't
* fit in spte.pfn. get_pfn() finds the real physical number of the
* page, given the virtual number. */
- pfn = get_pfn(gpte.pfn, write);
+ pfn = get_pfn(base + gpte.pfn, write);
if (pfn == -1UL) {
kill_guest(lg, "failed to get page %u", gpte.pfn);
/* When we destroy the Guest, we'll go through the shadow page
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:07 UTC
[patch 37/43] lguest: Virtio helper routines for a descriptor ringbuffer implementation
These helper routines supply most of the virtqueue_ops for hypervisors
which want to use a ring for virtio. Unlike the previous lguest
implementation:
1) The rings are variable sized (2^n-1 elements).
2) They have an unfortunate limit of 65535 bytes per sg element.
3) The page numbers are always 64 bit (PAE anyone?)
4) They no longer place used[] on a separate page, just a separate
cacheline.
5) We do a modulo on a variable. We could be tricky if we cared.
6) Interrupts and notifies are suppressed using flags within the rings.
Users need only get the ring pages and provide a notify hook (KVM
wants the guest to allocate the rings, lguest does it sanely).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: Dor Laor <dor.laor@qumranet.com>
---
arch/i386/lguest/Kconfig | 1
drivers/virtio/Kconfig | 5
drivers/virtio/Makefile | 1
drivers/virtio/virtio_ring.c | 313 ++++++++++++++++++++++++++++++++++++++++++
include/linux/virtio_ring.h | 119 +++++++++++++++
5 files changed, 439 insertions(+)
==================================================================---
a/arch/i386/lguest/Kconfig
+++ b/arch/i386/lguest/Kconfig
@@ -2,6 +2,7 @@ config LGUEST_GUEST
bool "Lguest guest support"
select PARAVIRT
depends on !X86_PAE
+ select VIRTIO_RING
help
Lguest is a tiny in-kernel hypervisor. Selecting this will
allow your kernel to boot under lguest. This option will increase
==================================================================---
a/drivers/virtio/Kconfig
+++ b/drivers/virtio/Kconfig
@@ -1,3 +1,8 @@
# Virtio always gets selected by whoever wants it.
config VIRTIO
bool
+
+# Similarly the virtio ring implementation.
+config VIRTIO_RING
+ bool
+ depends on VIRTIO
==================================================================---
a/drivers/virtio/Makefile
+++ b/drivers/virtio/Makefile
@@ -1,1 +1,2 @@ obj-$(CONFIG_VIRTIO) += virtio.o
obj-$(CONFIG_VIRTIO) += virtio.o
+obj-$(CONFIG_VIRTIO_RING) += virtio_ring.o
==================================================================--- /dev/null
+++ b/drivers/virtio/virtio_ring.c
@@ -0,0 +1,313 @@
+/* Virtio ring implementation.
+ *
+ * Copyright 2007 Rusty Russell IBM Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include <linux/virtio.h>
+#include <linux/virtio_ring.h>
+#include <linux/device.h>
+
+#ifdef DEBUG
+/* For development, we want to crash whenever the ring is screwed. */
+#define BAD_RING(vq, fmt...) \
+ do { dev_err(&vq->vq.vdev->dev, fmt); BUG(); } while(0)
+#define START_USE(vq) \
+ do { if ((vq)->in_use) panic("in_use = %i\n", (vq)->in_use);
(vq)->in_use = __LINE__; mb(); } while(0)
+#define END_USE(vq) \
+ do { BUG_ON(!(vq)->in_use); (vq)->in_use = 0; mb(); } while(0)
+#else
+#define BAD_RING(vq, fmt...) \
+ do { dev_err(&vq->vq.vdev->dev, fmt); (vq)->broken = true; }
while(0)
+#define START_USE(vq)
+#define END_USE(vq)
+#endif
+
+struct vring_virtqueue
+{
+ struct virtqueue vq;
+
+ /* Actual memory layout for this queue */
+ struct vring vring;
+
+ /* Other side has made a mess, don't try any more. */
+ bool broken;
+
+ /* Number of free buffers */
+ unsigned int num_free;
+ /* Head of free buffer list. */
+ unsigned int free_head;
+ /* Number we've added since last sync. */
+ unsigned int num_added;
+
+ /* Last used index we've seen. */
+ unsigned int last_used_idx;
+
+ /* How to notify other side. FIXME: commonalize hcalls! */
+ void (*notify)(struct virtqueue *vq);
+
+#ifdef DEBUG
+ /* They're supposed to lock for us. */
+ unsigned int in_use;
+#endif
+
+ /* Tokens for callbacks. */
+ void *data[];
+};
+
+#define to_vvq(_vq) container_of(_vq, struct vring_virtqueue, vq)
+
+static int vring_add_buf(struct virtqueue *_vq,
+ struct scatterlist sg[],
+ unsigned int out,
+ unsigned int in,
+ void *data)
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+ unsigned int i, avail, head, uninitialized_var(prev);
+
+ BUG_ON(data == NULL);
+ BUG_ON(out + in > vq->vring.num);
+ BUG_ON(out + in == 0);
+
+ START_USE(vq);
+
+ if (vq->num_free < out + in) {
+ pr_debug("Can't add buf len %i - avail = %i\n",
+ out + in, vq->num_free);
+ END_USE(vq);
+ return -ENOSPC;
+ }
+
+ /* We're about to use some buffers from the free list. */
+ vq->num_free -= out + in;
+
+ head = vq->free_head;
+ for (i = vq->free_head; out; i = vq->vring.desc[i].next, out--) {
+ vq->vring.desc[i].flags = VRING_DESC_F_NEXT;
+ vq->vring.desc[i].addr = (page_to_pfn(sg->page) << PAGE_SHIFT)
+ + sg->offset;
+ vq->vring.desc[i].len = sg->length;
+ prev = i;
+ sg++;
+ }
+ for (; in; i = vq->vring.desc[i].next, in--) {
+ vq->vring.desc[i].flags = VRING_DESC_F_NEXT|VRING_DESC_F_WRITE;
+ vq->vring.desc[i].addr = (page_to_pfn(sg->page) << PAGE_SHIFT)
+ + sg->offset;
+ vq->vring.desc[i].len = sg->length;
+ prev = i;
+ sg++;
+ }
+ /* Last one doesn't continue. */
+ vq->vring.desc[prev].flags &= ~VRING_DESC_F_NEXT;
+
+ /* Update free pointer */
+ vq->free_head = i;
+
+ /* Set token. */
+ vq->data[head] = data;
+
+ /* Put entry in available array (but don't update avail->idx until they
+ * do sync). FIXME: avoid modulus here? */
+ avail = (vq->vring.avail->idx + vq->num_added++) % vq->vring.num;
+ vq->vring.avail->ring[avail] = head;
+
+ pr_debug("Added buffer head %i to %p\n", head, vq);
+ END_USE(vq);
+ return 0;
+}
+
+static void vring_kick(struct virtqueue *_vq)
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+ START_USE(vq);
+ /* Descriptors and available array need to be set before we expose the
+ * new available array entries. */
+ wmb();
+
+ vq->vring.avail->idx += vq->num_added;
+ vq->num_added = 0;
+
+ /* Need to update avail index before checking if we should notify */
+ mb();
+
+ if (!(vq->vring.used->flags & VRING_USED_F_NO_NOTIFY))
+ /* Prod other side to tell it about changes. */
+ vq->notify(&vq->vq);
+
+ END_USE(vq);
+}
+
+static void detach_buf(struct vring_virtqueue *vq, unsigned int head)
+{
+ unsigned int i;
+
+ /* Clear data ptr. */
+ vq->data[head] = NULL;
+
+ /* Put back on free list: find end */
+ i = head;
+ while (vq->vring.desc[i].flags & VRING_DESC_F_NEXT) {
+ i = vq->vring.desc[i].next;
+ vq->num_free++;
+ }
+
+ vq->vring.desc[i].next = vq->free_head;
+ vq->free_head = head;
+ /* Plus final descriptor */
+ vq->num_free++;
+}
+
+/* FIXME: We need to tell other side about removal, to synchronize. */
+static void vring_shutdown(struct virtqueue *_vq)
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+ unsigned int i;
+
+ for (i = 0; i < vq->vring.num; i++)
+ detach_buf(vq, i);
+}
+
+static inline bool more_used(const struct vring_virtqueue *vq)
+{
+ return vq->last_used_idx != vq->vring.used->idx;
+}
+
+static void *vring_get_buf(struct virtqueue *_vq, unsigned int *len)
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+ void *ret;
+ unsigned int i;
+
+ START_USE(vq);
+
+ if (!more_used(vq)) {
+ pr_debug("No more buffers in queue\n");
+ END_USE(vq);
+ return NULL;
+ }
+
+ i = vq->vring.used->ring[vq->last_used_idx%vq->vring.num].id;
+ *len = vq->vring.used->ring[vq->last_used_idx%vq->vring.num].len;
+
+ if (unlikely(i >= vq->vring.num)) {
+ BAD_RING(vq, "id %u out of range\n", i);
+ return NULL;
+ }
+ if (unlikely(!vq->data[i])) {
+ BAD_RING(vq, "id %u is not a head!\n", i);
+ return NULL;
+ }
+
+ /* detach_buf clears data, so grab it now. */
+ ret = vq->data[i];
+ detach_buf(vq, i);
+ vq->last_used_idx++;
+ END_USE(vq);
+ return ret;
+}
+
+static bool vring_restart(struct virtqueue *_vq)
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+
+ START_USE(vq);
+ BUG_ON(!(vq->vring.avail->flags & VRING_AVAIL_F_NO_INTERRUPT));
+
+ /* We optimistically turn back on interrupts, then check if there was
+ * more to do. */
+ vq->vring.avail->flags &= ~VRING_AVAIL_F_NO_INTERRUPT;
+ mb();
+ if (unlikely(more_used(vq))) {
+ vq->vring.avail->flags |= VRING_AVAIL_F_NO_INTERRUPT;
+ END_USE(vq);
+ return false;
+ }
+
+ END_USE(vq);
+ return true;
+}
+
+irqreturn_t vring_interrupt(int irq, void *_vq)
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+
+ if (!more_used(vq)) {
+ pr_debug("virtqueue interrupt with no work for %p\n", vq);
+ return IRQ_NONE;
+ }
+
+ if (unlikely(vq->broken))
+ return IRQ_HANDLED;
+
+ pr_debug("virtqueue callback for %p (%p)\n", vq,
vq->vq.callback);
+ if (vq->vq.callback && !vq->vq.callback(&vq->vq))
+ vq->vring.avail->flags |= VRING_AVAIL_F_NO_INTERRUPT;
+
+ return IRQ_HANDLED;
+}
+
+static struct virtqueue_ops vring_vq_ops = {
+ .add_buf = vring_add_buf,
+ .get_buf = vring_get_buf,
+ .kick = vring_kick,
+ .restart = vring_restart,
+ .shutdown = vring_shutdown,
+};
+
+struct virtqueue *vring_new_virtqueue(unsigned int num,
+ struct virtio_device *vdev,
+ void *pages,
+ void (*notify)(struct virtqueue *),
+ bool (*callback)(struct virtqueue *))
+{
+ struct vring_virtqueue *vq;
+ unsigned int i;
+
+ vq = kmalloc(sizeof(*vq) + sizeof(void *)*num, GFP_KERNEL);
+ if (!vq)
+ return NULL;
+
+ vring_init(&vq->vring, num, pages);
+ vq->vq.callback = callback;
+ vq->vq.vdev = vdev;
+ vq->vq.vq_ops = &vring_vq_ops;
+ vq->notify = notify;
+ vq->broken = false;
+ vq->last_used_idx = 0;
+ vq->num_added = 0;
+#ifdef DEBUG
+ vq->in_use = false;
+#endif
+
+ /* No callback? Tell other side not to bother us. */
+ if (!callback)
+ vq->vring.avail->flags |= VRING_AVAIL_F_NO_INTERRUPT;
+
+ /* Put everything in free lists. */
+ vq->num_free = num;
+ vq->free_head = 0;
+ for (i = 0; i < num-1; i++)
+ vq->vring.desc[i].next = i+1;
+
+ return &vq->vq;
+}
+
+void vring_del_virtqueue(struct virtqueue *vq)
+{
+ kfree(to_vvq(vq));
+}
+
==================================================================--- /dev/null
+++ b/include/linux/virtio_ring.h
@@ -0,0 +1,119 @@
+#ifndef _LINUX_VIRTIO_RING_H
+#define _LINUX_VIRTIO_RING_H
+/* An interface for efficient virtio implementation, currently for use by KVM
+ * and lguest, but hopefully others soon. Do NOT change this since it will
+ * break existing servers and clients.
+ *
+ * This header is BSD licensed so anyone can use the definitions to implement
+ * compatible drivers/servers.
+ *
+ * Copyright Rusty Russell IBM Corporation 2007. */
+#include <linux/types.h>
+
+/* This marks a buffer as continuing via the next field. */
+#define VRING_DESC_F_NEXT 1
+/* This marks a buffer as write-only (otherwise read-only). */
+#define VRING_DESC_F_WRITE 2
+
+/* This means don't notify other side when buffer added. */
+#define VRING_USED_F_NO_NOTIFY 1
+/* This means don't interrupt guest when buffer consumed. */
+#define VRING_AVAIL_F_NO_INTERRUPT 1
+
+/* Virtio ring descriptors: 16 bytes. These can chain together via
"next". */
+struct vring_desc
+{
+ /* Address (guest-physical). */
+ __u64 addr;
+ /* Length. */
+ __u32 len;
+ /* The flags as indicated above. */
+ __u16 flags;
+ /* We chain unused descriptors via this, too */
+ __u16 next;
+};
+
+struct vring_avail
+{
+ __u16 flags;
+ __u16 idx;
+ __u16 ring[];
+};
+
+/* u32 is used here for ids for padding reasons. */
+struct vring_used_elem
+{
+ /* Index of start of used descriptor chain. */
+ __u32 id;
+ /* Total length of the descriptor chain which was used (written to) */
+ __u32 len;
+};
+
+struct vring_used
+{
+ __u16 flags;
+ __u16 idx;
+ struct vring_used_elem ring[];
+};
+
+struct vring {
+ unsigned int num;
+
+ struct vring_desc *desc;
+
+ struct vring_avail *avail;
+
+ struct vring_used *used;
+};
+
+/* The standard layout for the ring is a continuous chunk of memory which looks
+ * like this. The used fields will be aligned to a "num+1" boundary.
+ *
+ * struct vring
+ * {
+ * // The actual descriptors (16 bytes each)
+ * struct vring_desc desc[num];
+ *
+ * // A ring of available descriptor heads with free-running index.
+ * __u16 avail_flags;
+ * __u16 avail_idx;
+ * __u16 available[num];
+ *
+ * // Padding so a correctly-chosen num value will cache-align used_idx.
+ * char pad[sizeof(struct vring_desc) - sizeof(avail_flags)];
+ *
+ * // A ring of used descriptor heads with free-running index.
+ * __u16 used_flags;
+ * __u16 used_idx;
+ * struct vring_used_elem used[num];
+ * };
+ */
+static inline void vring_init(struct vring *vr, unsigned int num, void *p)
+{
+ vr->num = num;
+ vr->desc = p;
+ vr->avail = p + num*sizeof(struct vring);
+ vr->used = p + (num+1)*(sizeof(struct vring) + sizeof(__u16));
+}
+
+static inline unsigned vring_size(unsigned int num)
+{
+ return (num + 1) * (sizeof(struct vring_desc) + sizeof(__u16))
+ + sizeof(__u32) + num * sizeof(struct vring_used_elem);
+}
+
+#ifdef __KERNEL__
+#include <linux/irqreturn.h>
+struct virtio_device;
+struct virtqueue;
+
+struct virtqueue *vring_new_virtqueue(unsigned int num,
+ struct virtio_device *vdev,
+ void *pages,
+ void (*notify)(struct virtqueue *vq),
+ bool (*callback)(struct virtqueue *vq));
+void vring_del_virtqueue(struct virtqueue *vq);
+
+irqreturn_t vring_interrupt(int irq, void *_vq);
+#endif /* __KERNEL__ */
+#endif /* _LINUX_VIRTIO_RING_H */
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:07 UTC
[patch 27/43] lguest: Pagetables to use normal kernel types
This is my first step in the migration of page_tables.c to the kernel
types and functions/macros (2.6.23-rc3). Seems to be working OK.
---
drivers/lguest/hypercalls.c | 2
drivers/lguest/lg.h | 45 +--------
drivers/lguest/page_tables.c | 203 +++++++++++++++++++-----------------------
3 files changed, 105 insertions(+), 145 deletions(-)
==================================================================---
a/drivers/lguest/hypercalls.c
+++ b/drivers/lguest/hypercalls.c
@@ -83,7 +83,7 @@ static void do_hcall(struct lguest *lg,
guest_set_stack(lg, args->arg1, args->arg2, args->arg3);
break;
case LHCALL_SET_PTE:
- guest_set_pte(lg, args->arg1, args->arg2, mkgpte(args->arg3));
+ guest_set_pte(lg, args->arg1, args->arg2, __pte(args->arg3));
break;
case LHCALL_SET_PMD:
guest_set_pmd(lg, args->arg1, args->arg2);
==================================================================---
a/drivers/lguest/lg.h
+++ b/drivers/lguest/lg.h
@@ -29,45 +29,10 @@ struct lguest_dma_info
u8 interrupt; /* 0 when not registered */
};
-/*H:310 The page-table code owes a great debt of gratitude to Andi Kleen. He
- * reviewed the original code which used "u32" for all page table
entries, and
- * insisted that it would be far clearer with explicit typing. I thought it
- * was overkill, but he was right: it is much clearer than it was before.
- *
- * We have separate types for the Guest's ptes & pgds and the shadow
ptes &
- * pgds. There's already a Linux type for these (pte_t and pgd_t) but they
- * change depending on kernel config options (PAE). */
-
-/* Each entry is identical: lower 12 bits of flags and upper 20 bits for the
- * "page frame number" (0 == first physical page, etc). They are
different
- * types so the compiler will warn us if we mix them improperly. */
-typedef union {
- struct { unsigned flags:12, pfn:20; };
- struct { unsigned long val; } raw;
-} spgd_t;
-typedef union {
- struct { unsigned flags:12, pfn:20; };
- struct { unsigned long val; } raw;
-} spte_t;
-typedef union {
- struct { unsigned flags:12, pfn:20; };
- struct { unsigned long val; } raw;
-} gpgd_t;
-typedef union {
- struct { unsigned flags:12, pfn:20; };
- struct { unsigned long val; } raw;
-} gpte_t;
-
-/* We have two convenient macros to convert a "raw" value as handed
to us by
- * the Guest into the correct Guest PGD or PTE type. */
-#define mkgpte(_val) ((gpte_t){.raw.val = _val})
-#define mkgpgd(_val) ((gpgd_t){.raw.val = _val})
-/*:*/
-
struct pgdir
{
unsigned long cr3;
- spgd_t *pgdir;
+ pgd_t *pgdir;
};
/* We have two pages shared with guests, per cpu. */
@@ -158,6 +123,12 @@ int lguest_address_ok(const struct lgues
unsigned long addr, unsigned long len);
int run_guest(struct lguest *lg, unsigned long __user *user);
+/* Helper macros to obtain the first 12 or the last 20 bits, this is only the
+ * first step in the migration to the kernel types. pte_pfn is already defined
+ * in the kernel. */
+#define pgd_flags(x) (pgd_val(x) & ~PAGE_MASK)
+#define pte_flags(x) (pte_val(x) & ~PAGE_MASK)
+#define pgd_pfn(x) (pgd_val(x) >> PAGE_SHIFT)
/* interrupts_and_traps.c: */
void maybe_do_interrupt(struct lguest *lg);
@@ -188,7 +159,7 @@ void guest_pagetable_clear_all(struct lg
void guest_pagetable_clear_all(struct lguest *lg);
void guest_pagetable_flush_user(struct lguest *lg);
void guest_set_pte(struct lguest *lg, unsigned long cr3,
- unsigned long vaddr, gpte_t val);
+ unsigned long vaddr, pte_t val);
void map_switcher_in_guest(struct lguest *lg, struct lguest_pages *pages);
int demand_page(struct lguest *info, unsigned long cr2, int errcode);
void pin_page(struct lguest *lg, unsigned long vaddr);
==================================================================---
a/drivers/lguest/page_tables.c
+++ b/drivers/lguest/page_tables.c
@@ -44,44 +44,32 @@
* (vii) Setting up the page tables initially.
:*/
-/* Pages a 4k long, and each page table entry is 4 bytes long, giving us 1024
- * (or 2^10) entries per page. */
-#define PTES_PER_PAGE_SHIFT 10
-#define PTES_PER_PAGE (1 << PTES_PER_PAGE_SHIFT)
/* 1024 entries in a page table page maps 1024 pages: 4MB. The Switcher is
* conveniently placed at the top 4MB, so it uses a separate, complete PTE
* page. */
-#define SWITCHER_PGD_INDEX (PTES_PER_PAGE - 1)
+#define SWITCHER_PGD_INDEX (PTRS_PER_PGD - 1)
/* We actually need a separate PTE page for each CPU. Remember that after the
* Switcher code itself comes two pages for each CPU, and we don't want
this
* CPU's guest to see the pages of any other CPU. */
-static DEFINE_PER_CPU(spte_t *, switcher_pte_pages);
+static DEFINE_PER_CPU(pte_t *, switcher_pte_pages);
#define switcher_pte_page(cpu) per_cpu(switcher_pte_pages, cpu)
/*H:320 With our shadow and Guest types established, we need to deal with
* them: the page table code is curly enough to need helper functions to keep
* it clear and clean.
*
- * The first helper takes a virtual address, and says which entry in the top
- * level page table deals with that address. Since each top level entry deals
- * with 4M, this effectively divides by 4M. */
-static unsigned vaddr_to_pgd_index(unsigned long vaddr)
-{
- return vaddr >> (PAGE_SHIFT + PTES_PER_PAGE_SHIFT);
-}
-
-/* There are two functions which return pointers to the shadow (aka
"real")
+ * There are two functions which return pointers to the shadow (aka
"real")
* page tables.
*
* spgd_addr() takes the virtual address and returns a pointer to the top-level
* page directory entry for that address. Since we keep track of several page
* tables, the "i" argument tells us which one we're interested
in (it's
* usually the current one). */
-static spgd_t *spgd_addr(struct lguest *lg, u32 i, unsigned long vaddr)
-{
- unsigned int index = vaddr_to_pgd_index(vaddr);
+static pgd_t *spgd_addr(struct lguest *lg, u32 i, unsigned long vaddr)
+{
+ unsigned int index = pgd_index(vaddr);
/* We kill any Guest trying to touch the Switcher addresses. */
if (index >= SWITCHER_PGD_INDEX) {
@@ -95,28 +83,28 @@ static spgd_t *spgd_addr(struct lguest *
/* This routine then takes the PGD entry given above, which contains the
* address of the PTE page. It then returns a pointer to the PTE entry for the
* given address. */
-static spte_t *spte_addr(struct lguest *lg, spgd_t spgd, unsigned long vaddr)
-{
- spte_t *page = __va(spgd.pfn << PAGE_SHIFT);
+static pte_t *spte_addr(struct lguest *lg, pgd_t spgd, unsigned long vaddr)
+{
+ pte_t *page = __va(pgd_pfn(spgd) << PAGE_SHIFT);
/* You should never call this if the PGD entry wasn't valid */
- BUG_ON(!(spgd.flags & _PAGE_PRESENT));
- return &page[(vaddr >> PAGE_SHIFT) % PTES_PER_PAGE];
+ BUG_ON(!(pgd_flags(spgd) & _PAGE_PRESENT));
+ return &page[(vaddr >> PAGE_SHIFT) % PTRS_PER_PTE];
}
/* These two functions just like the above two, except they access the Guest
* page tables. Hence they return a Guest address. */
static unsigned long gpgd_addr(struct lguest *lg, unsigned long vaddr)
{
- unsigned int index = vaddr >> (PAGE_SHIFT + PTES_PER_PAGE_SHIFT);
- return lg->pgdirs[lg->pgdidx].cr3 + index * sizeof(gpgd_t);
+ unsigned int index = vaddr >> (PGDIR_SHIFT);
+ return lg->pgdirs[lg->pgdidx].cr3 + index * sizeof(pgd_t);
}
static unsigned long gpte_addr(struct lguest *lg,
- gpgd_t gpgd, unsigned long vaddr)
-{
- unsigned long gpage = gpgd.pfn << PAGE_SHIFT;
- BUG_ON(!(gpgd.flags & _PAGE_PRESENT));
- return gpage + ((vaddr>>PAGE_SHIFT) % PTES_PER_PAGE) * sizeof(gpte_t);
+ pgd_t gpgd, unsigned long vaddr)
+{
+ unsigned long gpage = pgd_pfn(gpgd) << PAGE_SHIFT;
+ BUG_ON(!(pgd_flags(gpgd) & _PAGE_PRESENT));
+ return gpage + ((vaddr>>PAGE_SHIFT) % PTRS_PER_PTE) * sizeof(pte_t);
}
/*H:350 This routine takes a page number given by the Guest and converts it to
@@ -149,16 +137,15 @@ static unsigned long get_pfn(unsigned lo
* entry can be a little tricky. The flags are (almost) the same, but the
* Guest PTE contains a virtual page number: the CPU needs the real page
* number. */
-static spte_t gpte_to_spte(struct lguest *lg, gpte_t gpte, int write)
-{
- spte_t spte;
- unsigned long pfn, base;
+static pte_t gpte_to_spte(struct lguest *lg, pte_t gpte, int write)
+{
+ unsigned long pfn, base, flags;
/* The Guest sets the global flag, because it thinks that it is using
* PGE. We only told it to use PGE so it would tell us whether it was
* flushing a kernel mapping or a userspace mapping. We don't actually
* use the global bit, so throw it away. */
- spte.flags = (gpte.flags & ~_PAGE_GLOBAL);
+ flags = (pte_flags(gpte) & ~_PAGE_GLOBAL);
/* The Guest's pages are offset inside the Launcher. */
base = (unsigned long)lg->mem_base / PAGE_SIZE;
@@ -167,38 +154,38 @@ static spte_t gpte_to_spte(struct lguest
* get_pfn(), because it returns 0xFFFFFFFF on failure, which wouldn't
* fit in spte.pfn. get_pfn() finds the real physical number of the
* page, given the virtual number. */
- pfn = get_pfn(base + gpte.pfn, write);
+ pfn = get_pfn(base + pte_pfn(gpte), write);
if (pfn == -1UL) {
- kill_guest(lg, "failed to get page %u", gpte.pfn);
+ kill_guest(lg, "failed to get page %lu", pte_pfn(gpte));
/* When we destroy the Guest, we'll go through the shadow page
* tables and release_pte() them. Make sure we don't think
* this one is valid! */
- spte.flags = 0;
- }
- /* Now we assign the page number, and our shadow PTE is complete. */
- spte.pfn = pfn;
- return spte;
+ flags = 0;
+ }
+ /* Now we assemble our shadow PTE from the page number and flags. */
+ return pfn_pte(pfn, __pgprot(flags));
}
/*H:460 And to complete the chain, release_pte() looks like this: */
-static void release_pte(spte_t pte)
+static void release_pte(pte_t pte)
{
/* Remember that get_user_pages() took a reference to the page, in
* get_pfn()? We have to put it back now. */
- if (pte.flags & _PAGE_PRESENT)
- put_page(pfn_to_page(pte.pfn));
+ if (pte_flags(pte) & _PAGE_PRESENT)
+ put_page(pfn_to_page(pte_pfn(pte)));
}
/*:*/
-static void check_gpte(struct lguest *lg, gpte_t gpte)
-{
- if ((gpte.flags & (_PAGE_PWT|_PAGE_PSE)) || gpte.pfn >=
lg->pfn_limit)
+static void check_gpte(struct lguest *lg, pte_t gpte)
+{
+ if ((pte_flags(gpte) & (_PAGE_PWT|_PAGE_PSE))
+ || pte_pfn(gpte) >= lg->pfn_limit)
kill_guest(lg, "bad page table entry");
}
-static void check_gpgd(struct lguest *lg, gpgd_t gpgd)
-{
- if ((gpgd.flags & ~_PAGE_TABLE) || gpgd.pfn >= lg->pfn_limit)
+static void check_gpgd(struct lguest *lg, pgd_t gpgd)
+{
+ if ((pgd_flags(gpgd) & ~_PAGE_TABLE) || pgd_pfn(gpgd) >=
lg->pfn_limit)
kill_guest(lg, "bad page directory entry");
}
@@ -214,21 +201,21 @@ static void check_gpgd(struct lguest *lg
* true. */
int demand_page(struct lguest *lg, unsigned long vaddr, int errcode)
{
- gpgd_t gpgd;
- spgd_t *spgd;
+ pgd_t gpgd;
+ pgd_t *spgd;
unsigned long gpte_ptr;
- gpte_t gpte;
- spte_t *spte;
+ pte_t gpte;
+ pte_t *spte;
/* First step: get the top-level Guest page table entry. */
- gpgd = mkgpgd(lgread_u32(lg, gpgd_addr(lg, vaddr)));
+ gpgd = __pgd(lgread_u32(lg, gpgd_addr(lg, vaddr)));
/* Toplevel not present? We can't map it in. */
- if (!(gpgd.flags & _PAGE_PRESENT))
+ if (!(pgd_flags(gpgd) & _PAGE_PRESENT))
return 0;
/* Now look at the matching shadow entry. */
spgd = spgd_addr(lg, lg->pgdidx, vaddr);
- if (!(spgd->flags & _PAGE_PRESENT)) {
+ if (!(pgd_flags(*spgd) & _PAGE_PRESENT)) {
/* No shadow entry: allocate a new shadow PTE page. */
unsigned long ptepage = get_zeroed_page(GFP_KERNEL);
/* This is not really the Guest's fault, but killing it is
@@ -241,34 +228,35 @@ int demand_page(struct lguest *lg, unsig
check_gpgd(lg, gpgd);
/* And we copy the flags to the shadow PGD entry. The page
* number in the shadow PGD is the page we just allocated. */
- spgd->raw.val = (__pa(ptepage) | gpgd.flags);
+ *spgd = __pgd(__pa(ptepage) | pgd_flags(gpgd));
}
/* OK, now we look at the lower level in the Guest page table: keep its
* address, because we might update it later. */
gpte_ptr = gpte_addr(lg, gpgd, vaddr);
- gpte = mkgpte(lgread_u32(lg, gpte_ptr));
+ gpte = __pte(lgread_u32(lg, gpte_ptr));
/* If this page isn't in the Guest page tables, we can't page it in.
*/
- if (!(gpte.flags & _PAGE_PRESENT))
+ if (!(pte_flags(gpte) & _PAGE_PRESENT))
return 0;
/* Check they're not trying to write to a page the Guest wants
* read-only (bit 2 of errcode == write). */
- if ((errcode & 2) && !(gpte.flags & _PAGE_RW))
+ if ((errcode & 2) && !(pte_flags(gpte) & _PAGE_RW))
return 0;
/* User access to a kernel page? (bit 3 == user access) */
- if ((errcode & 4) && !(gpte.flags & _PAGE_USER))
+ if ((errcode & 4) && !(pte_flags(gpte) & _PAGE_USER))
return 0;
/* Check that the Guest PTE flags are OK, and the page number is below
* the pfn_limit (ie. not mapping the Launcher binary). */
check_gpte(lg, gpte);
/* Add the _PAGE_ACCESSED and (for a write) _PAGE_DIRTY flag */
- gpte.flags |= _PAGE_ACCESSED;
+ gpte = pte_mkyoung(gpte);
+
if (errcode & 2)
- gpte.flags |= _PAGE_DIRTY;
+ gpte = pte_mkdirty(gpte);
/* Get the pointer to the shadow PTE entry we're going to set. */
spte = spte_addr(lg, *spgd, vaddr);
@@ -278,21 +266,21 @@ int demand_page(struct lguest *lg, unsig
/* If this is a write, we insist that the Guest page is writable (the
* final arg to gpte_to_spte()). */
- if (gpte.flags & _PAGE_DIRTY)
+ if (pte_dirty(gpte))
*spte = gpte_to_spte(lg, gpte, 1);
else {
/* If this is a read, don't set the "writable" bit in the page
* table entry, even if the Guest says it's writable. That way
* we come back here when a write does actually ocur, so we can
* update the Guest's _PAGE_DIRTY flag. */
- gpte_t ro_gpte = gpte;
- ro_gpte.flags &= ~_PAGE_RW;
+ pte_t ro_gpte = gpte;
+ pte_wrprotect(ro_gpte);
*spte = gpte_to_spte(lg, ro_gpte, 0);
}
/* Finally, we write the Guest PTE entry back: we've set the
* _PAGE_ACCESSED and maybe the _PAGE_DIRTY flags. */
- lgwrite_u32(lg, gpte_ptr, gpte.raw.val);
+ lgwrite_u32(lg, gpte_ptr, pte_val(gpte));
/* We succeeded in mapping the page! */
return 1;
@@ -308,17 +296,18 @@ int demand_page(struct lguest *lg, unsig
* mapped by the shadow page tables, and is it writable? */
static int page_writable(struct lguest *lg, unsigned long vaddr)
{
- spgd_t *spgd;
+ pgd_t *spgd;
unsigned long flags;
/* Look at the top level entry: is it present? */
spgd = spgd_addr(lg, lg->pgdidx, vaddr);
- if (!(spgd->flags & _PAGE_PRESENT))
+ if (!(pgd_flags(*spgd) & _PAGE_PRESENT))
return 0;
/* Check the flags on the pte entry itself: it must be present and
* writable. */
- flags = spte_addr(lg, *spgd, vaddr)->flags;
+ flags = pte_flags(*(spte_addr(lg, *spgd, vaddr)));
+
return (flags & (_PAGE_PRESENT|_PAGE_RW)) == (_PAGE_PRESENT|_PAGE_RW);
}
@@ -332,22 +321,22 @@ void pin_page(struct lguest *lg, unsigne
}
/*H:450 If we chase down the release_pgd() code, it looks like this: */
-static void release_pgd(struct lguest *lg, spgd_t *spgd)
+static void release_pgd(struct lguest *lg, pgd_t *spgd)
{
/* If the entry's not present, there's nothing to release. */
- if (spgd->flags & _PAGE_PRESENT) {
+ if (pgd_flags(*spgd) & _PAGE_PRESENT) {
unsigned int i;
/* Converting the pfn to find the actual PTE page is easy: turn
* the page number into a physical address, then convert to a
* virtual address (easy for kernel pages like this one). */
- spte_t *ptepage = __va(spgd->pfn << PAGE_SHIFT);
+ pte_t *ptepage = __va(pgd_pfn(*spgd) << PAGE_SHIFT);
/* For each entry in the page, we might need to release it. */
- for (i = 0; i < PTES_PER_PAGE; i++)
+ for (i = 0; i < PTRS_PER_PTE; i++)
release_pte(ptepage[i]);
/* Now we can free the page of PTEs */
free_page((long)ptepage);
/* And zero out the PGD entry we we never release it twice. */
- spgd->raw.val = 0;
+ *spgd = __pgd(0);
}
}
@@ -359,7 +348,7 @@ static void flush_user_mappings(struct l
{
unsigned int i;
/* Release every pgd entry up to the kernel's address. */
- for (i = 0; i < vaddr_to_pgd_index(lg->page_offset); i++)
+ for (i = 0; i < pgd_index(lg->page_offset); i++)
release_pgd(lg, lg->pgdirs[idx].pgdir + i);
}
@@ -398,7 +387,7 @@ static unsigned int new_pgdir(struct lgu
next = random32() % ARRAY_SIZE(lg->pgdirs);
/* If it's never been allocated at all before, try now. */
if (!lg->pgdirs[next].pgdir) {
- lg->pgdirs[next].pgdir = (spgd_t *)get_zeroed_page(GFP_KERNEL);
+ lg->pgdirs[next].pgdir = (pgd_t *)get_zeroed_page(GFP_KERNEL);
/* If the allocation fails, just keep using the one we have */
if (!lg->pgdirs[next].pgdir)
next = lg->pgdidx;
@@ -475,26 +464,27 @@ void guest_pagetable_clear_all(struct lg
* they set _PAGE_DIRTY then we can put a writable PTE entry in immediately.
*/
static void do_set_pte(struct lguest *lg, int idx,
- unsigned long vaddr, gpte_t gpte)
+ unsigned long vaddr, pte_t gpte)
{
/* Look up the matching shadow page directot entry. */
- spgd_t *spgd = spgd_addr(lg, idx, vaddr);
+ pgd_t *spgd = spgd_addr(lg, idx, vaddr);
/* If the top level isn't present, there's no entry to update. */
- if (spgd->flags & _PAGE_PRESENT) {
+ if (pgd_flags(*spgd) & _PAGE_PRESENT) {
/* Otherwise, we start by releasing the existing entry. */
- spte_t *spte = spte_addr(lg, *spgd, vaddr);
+ pte_t *spte = spte_addr(lg, *spgd, vaddr);
release_pte(*spte);
/* If they're setting this entry as dirty or accessed, we might
* as well put that entry they've given us in now. This shaves
* 10% off a copy-on-write micro-benchmark. */
- if (gpte.flags & (_PAGE_DIRTY | _PAGE_ACCESSED)) {
+ if (pte_flags(gpte) & (_PAGE_DIRTY | _PAGE_ACCESSED)) {
check_gpte(lg, gpte);
- *spte = gpte_to_spte(lg, gpte, gpte.flags&_PAGE_DIRTY);
+ *spte = gpte_to_spte(lg, gpte,
+ pte_flags(gpte) & _PAGE_DIRTY);
} else
/* Otherwise we can demand_page() it in later. */
- spte->raw.val = 0;
+ *spte = __pte(0);
}
}
@@ -509,7 +499,7 @@ static void do_set_pte(struct lguest *lg
* The benefit is that when we have to track a new page table, we can copy keep
* all the kernel mappings. This speeds up context switch immensely. */
void guest_set_pte(struct lguest *lg,
- unsigned long cr3, unsigned long vaddr, gpte_t gpte)
+ unsigned long cr3, unsigned long vaddr, pte_t gpte)
{
/* Kernel mappings must be changed on all top levels. Slow, but
* doesn't happen often. */
@@ -564,15 +554,15 @@ int init_guest_pagetable(struct lguest *
int init_guest_pagetable(struct lguest *lg, unsigned long pgtable)
{
/* In flush_user_mappings() we loop from 0 to
- * "vaddr_to_pgd_index(lg->page_offset)". This assumes it
won't hit
+ * "pgd_index(lg->page_offset)". This assumes it won't hit
* the Switcher mappings, so check that now. */
- if (vaddr_to_pgd_index(lg->page_offset) >= SWITCHER_PGD_INDEX)
+ if (pgd_index(lg->page_offset) >= SWITCHER_PGD_INDEX)
return -EINVAL;
/* We start on the first shadow page table, and give it a blank PGD
* page. */
lg->pgdidx = 0;
lg->pgdirs[lg->pgdidx].cr3 = pgtable;
- lg->pgdirs[lg->pgdidx].pgdir = (spgd_t*)get_zeroed_page(GFP_KERNEL);
+ lg->pgdirs[lg->pgdidx].pgdir = (pgd_t*)get_zeroed_page(GFP_KERNEL);
if (!lg->pgdirs[lg->pgdidx].pgdir)
return -ENOMEM;
return 0;
@@ -597,14 +587,14 @@ void free_guest_pagetable(struct lguest
* for each CPU already set up, we just need to hook them in. */
void map_switcher_in_guest(struct lguest *lg, struct lguest_pages *pages)
{
- spte_t *switcher_pte_page = __get_cpu_var(switcher_pte_pages);
- spgd_t switcher_pgd;
- spte_t regs_pte;
+ pte_t *switcher_pte_page = __get_cpu_var(switcher_pte_pages);
+ pgd_t switcher_pgd;
+ pte_t regs_pte;
/* Make the last PGD entry for this Guest point to the Switcher's PTE
* page for this CPU (with appropriate flags). */
- switcher_pgd.pfn = __pa(switcher_pte_page) >> PAGE_SHIFT;
- switcher_pgd.flags = _PAGE_KERNEL;
+ switcher_pgd = __pgd(__pa(switcher_pte_page) | _PAGE_KERNEL);
+
lg->pgdirs[lg->pgdidx].pgdir[SWITCHER_PGD_INDEX] = switcher_pgd;
/* We also change the Switcher PTE page. When we're running the Guest,
@@ -614,10 +604,8 @@ void map_switcher_in_guest(struct lguest
* CPU's "struct lguest_pages": if we make sure the Guest's
register
* page is already mapped there, we don't have to copy them out
* again. */
- regs_pte.pfn = __pa(lg->regs_page) >> PAGE_SHIFT;
- regs_pte.flags = _PAGE_KERNEL;
- switcher_pte_page[(unsigned long)pages/PAGE_SIZE%PTES_PER_PAGE]
- = regs_pte;
+ regs_pte = pfn_pte (__pa(lg->regs_page) >> PAGE_SHIFT,
__pgprot(_PAGE_KERNEL));
+ switcher_pte_page[(unsigned long)pages/PAGE_SIZE%PTRS_PER_PTE] = regs_pte;
}
/*:*/
@@ -638,24 +626,25 @@ static __init void populate_switcher_pte
unsigned int pages)
{
unsigned int i;
- spte_t *pte = switcher_pte_page(cpu);
+ pte_t *pte = switcher_pte_page(cpu);
/* The first entries are easy: they map the Switcher code. */
for (i = 0; i < pages; i++) {
- pte[i].pfn = page_to_pfn(switcher_page[i]);
- pte[i].flags = _PAGE_PRESENT|_PAGE_ACCESSED;
+ pte[i] = mk_pte(switcher_page[i],
+ __pgprot(_PAGE_PRESENT|_PAGE_ACCESSED));
}
/* The only other thing we map is this CPU's pair of pages. */
i = pages + cpu*2;
/* First page (Guest registers) is writable from the Guest */
- pte[i].pfn = page_to_pfn(switcher_page[i]);
- pte[i].flags = _PAGE_PRESENT|_PAGE_ACCESSED|_PAGE_RW;
+ pte[i] = pfn_pte(page_to_pfn(switcher_page[i]),
+ __pgprot(_PAGE_PRESENT|_PAGE_ACCESSED|_PAGE_RW));
+
/* The second page contains the "struct lguest_ro_state", and is
* read-only. */
- pte[i+1].pfn = page_to_pfn(switcher_page[i+1]);
- pte[i+1].flags = _PAGE_PRESENT|_PAGE_ACCESSED;
+ pte[i+1] = pfn_pte(page_to_pfn(switcher_page[i+1]),
+ __pgprot(_PAGE_PRESENT|_PAGE_ACCESSED));
}
/*H:510 At boot or module load time, init_pagetables() allocates and populates
@@ -665,7 +654,7 @@ __init int init_pagetables(struct page *
unsigned int i;
for_each_possible_cpu(i) {
- switcher_pte_page(i) = (spte_t *)get_zeroed_page(GFP_KERNEL);
+ switcher_pte_page(i) = (pte_t *)get_zeroed_page(GFP_KERNEL);
if (!switcher_pte_page(i)) {
free_switcher_pte_pages();
return -ENOMEM;
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:07 UTC
[patch 42/43] lguest: Example launcher handle guests not being ready for input
We currently discard console and network input when the guest has no
input buffers. This patch changes that, so that we simply stop
listening to that fd until the guest refills its input buffers.
This is particularly important because hvc_console without interrupts
does backoff polling and so often lose characters if we discard.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
Documentation/lguest/lguest.c | 107 +++++++++++++++++++++--------------------
1 file changed, 57 insertions(+), 50 deletions(-)
diff -r 7f216dff74d3 Documentation/lguest/lguest.c
--- a/Documentation/lguest/lguest.c Wed Sep 26 14:48:31 2007 +1000
+++ b/Documentation/lguest/lguest.c Wed Sep 26 14:48:31 2007 +1000
@@ -598,15 +598,17 @@ static void wake_parent(int pipefd, int
select(devices.max_infd+1, &rfds, NULL, NULL, NULL);
/* Is it a message from the Launcher? */
if (FD_ISSET(pipefd, &rfds)) {
- int ignorefd;
+ int fd;
/* If read() returns 0, it means the Launcher has
* exited. We silently follow. */
- if (read(pipefd, &ignorefd, sizeof(ignorefd)) == 0)
+ if (read(pipefd, &fd, sizeof(fd)) == 0)
exit(0);
- /* Otherwise it's telling us there's a problem with one
- * of the devices, and we should ignore that file
- * descriptor from now on. */
- FD_CLR(ignorefd, &devices.infds);
+ /* Otherwise it's telling us to change what file
+ * descriptors we're to listen to. */
+ if (fd >= 0)
+ FD_SET(fd, &devices.infds);
+ else
+ FD_CLR(-fd - 1, &devices.infds);
} else /* Send LHREQ_BREAK command. */
write(lguest_fd, args, sizeof(args));
}
@@ -657,18 +659,6 @@ static void *_check_pointer(unsigned lon
}
/* A macro which transparently hands the line number to the real function. */
#define check_pointer(addr,size) _check_pointer(addr, size, __LINE__)
-
-/* This simply sets up an iovec array where we can put data to be discarded.
- * This happens when the Guest doesn't want or can't handle the input:
we have
- * to get rid of it somewhere, and if we bury it in the ceiling space it will
- * start to smell after a week. */
-static void discard_iovec(struct iovec *iov, unsigned int *num)
-{
- static char discard_buf[1024];
- *num = 1;
- iov->iov_base = discard_buf;
- iov->iov_len = sizeof(discard_buf);
-}
/* This function returns the next descriptor in the chain, or vq->vring.num.
*/
static unsigned next_desc(struct virtqueue *vq, unsigned int i)
@@ -812,12 +802,13 @@ static bool handle_console_input(int fd,
/* First we need a console buffer from the Guests's input virtqueue. */
head = get_vq_desc(dev->vq, iov, &out_num, &in_num);
- if (head == dev->vq->vring.num) {
- /* If they're not ready for input, we warn and set up to
- * discard. */
- warnx("console: no dma buffer!");
- discard_iovec(iov, &in_num);
- } else if (out_num)
+
+ /* If they're not ready for input, stop listening to this file
+ * descriptor. We'll start again once they add an input buffer. */
+ if (head == dev->vq->vring.num)
+ return false;
+
+ if (out_num)
errx(1, "Output buffers in console in queue?");
/* This is why we convert to iovecs: the readv() call uses them, and so
@@ -827,15 +818,16 @@ static bool handle_console_input(int fd,
/* This implies that the console is closed, is /dev/null, or
* something went terribly wrong. */
warnx("Failed to get console input, ignoring console.");
- /* Put the input terminal back and return failure (meaning,
- * don't call us again). */
+ /* Put the input terminal back. */
restore_term();
+ /* Remove callback from input vq, so it doesn't restart us. */
+ dev->vq->handle_output = NULL;
+ /* Stop listening to this fd: don't call us again. */
return false;
}
- /* If we actually read the data into the Guest, tell them about it. */
- if (head != dev->vq->vring.num)
- add_used_and_trigger(fd, dev->vq, head, len);
+ /* Tell the Guest about the new input. */
+ add_used_and_trigger(fd, dev->vq, head, len);
/* Three ^C within one second? Exit.
*
@@ -924,7 +916,8 @@ static bool handle_tun_input(int fd, str
/* FIXME: Actually want DRIVER_ACTIVE here. */
if (dev->desc->status & VIRTIO_CONFIG_S_DRIVER_OK)
warn("network: no dma buffer!");
- discard_iovec(iov, &in_num);
+ /* We'll turn this back on if input buffers are registered. */
+ return false;
} else if (out_num)
errx(1, "Output buffers in network recv queue?");
@@ -938,9 +931,8 @@ static bool handle_tun_input(int fd, str
if (len <= 0)
err(1, "reading network");
- /* If we actually read the data into the Guest, tell them about it. */
- if (head != dev->vq->vring.num)
- add_used_and_trigger(fd, dev->vq, head, sizeof(*hdr) + len);
+ /* Tell the Guest about the new packet. */
+ add_used_and_trigger(fd, dev->vq, head, sizeof(*hdr) + len);
verbose("tun input packet len %i [%02x %02x] (%s)\n", len,
((u8 *)iov[1].iov_base)[0], ((u8 *)iov[1].iov_base)[1],
@@ -948,6 +940,15 @@ static bool handle_tun_input(int fd, str
/* All good. */
return true;
+}
+
+/* This callback ensures we try again, in case we stopped console or net
+ * delivery because Guest didn't have any buffers. */
+static void enable_fd(int fd, struct virtqueue *vq)
+{
+ add_device_fd(vq->dev->fd);
+ /* Tell waker to listen to it again */
+ write(waker_fd, &vq->dev->fd, sizeof(vq->dev->fd));
}
/* This is the generic routine we call when the Guest uses LHCALL_NOTIFY. */
@@ -996,17 +997,22 @@ static void handle_input(int fd)
* file descriptors and a method of handling them. */
for (i = devices.dev; i; i = i->next) {
if (i->handle_input && FD_ISSET(i->fd, &fds)) {
+ int dev_fd;
+ if (i->handle_input(fd, i))
+ continue;
+
/* If handle_input() returns false, it means we
- * should no longer service it.
- * handle_console_input() does this. */
- if (!i->handle_input(fd, i)) {
- /* Clear it from the set of input file
- * descriptors kept at the head of the
- * device list. */
- FD_CLR(i->fd, &devices.infds);
- /* Tell waker to ignore it too... */
- write(waker_fd, &i->fd, sizeof(i->fd));
- }
+ * should no longer service it. Networking and
+ * console do this when there's no input
+ * buffers to deliver into. Console also uses
+ * it when it discovers that stdin is
+ * closed. */
+ FD_CLR(i->fd, &devices.infds);
+ /* Tell waker to ignore it too, by sending a
+ * negative fd number (-1, since 0 is a valid
+ * FD number). */
+ dev_fd = -i->fd - 1;
+ write(waker_fd, &dev_fd, sizeof(dev_fd));
}
}
}
@@ -1154,11 +1160,11 @@ static void setup_console(void)
dev->priv = malloc(sizeof(struct console_abort));
((struct console_abort *)dev->priv)->count = 0;
- /* The console needs two virtqueues: the input then the output. We
- * don't care when they refill the input queue, since we don't hold
- * data waiting for them. That's why the input queue's callback is
- * NULL. */
- add_virtqueue(dev, VIRTQUEUE_NUM, NULL);
+ /* The console needs two virtqueues: the input then the output. When
+ * they put something the input queue, we make sure we're listening to
+ * stdin. When they put something in the output queue, we write it to
+ * stdout. */
+ add_virtqueue(dev, VIRTQUEUE_NUM, enable_fd);
add_virtqueue(dev, VIRTQUEUE_NUM, handle_console_output);
verbose("device %u: console\n", devices.device_num++);
@@ -1270,8 +1276,9 @@ static void setup_tun_net(const char *ar
/* First we create a new network device. */
dev = new_device("net", VIRTIO_ID_NET, netfd, handle_tun_input);
- /* Network devices need a receive and a send queue. */
- add_virtqueue(dev, VIRTQUEUE_NUM, NULL);
+ /* Network devices need a receive and a send queue, just like
+ * console. */
+ add_virtqueue(dev, VIRTQUEUE_NUM, enable_fd);
add_virtqueue(dev, VIRTQUEUE_NUM, handle_net_output);
/* We need a socket to perform the magic network ioctls to bring up the
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:07 UTC
[patch 41/43] lguest: Update example launcher for virtio
Implements virtio-based console, network and block servers. The block
server uses a thread so it's async, which is an improvement over the
old synchronous implementation (but a little more complex).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
Documentation/lguest/lguest.c | 1201 ++++++++++++++++++++++++-----------------
1 file changed, 711 insertions(+), 490 deletions(-)
diff -r a7242b81747f Documentation/lguest/lguest.c
--- a/Documentation/lguest/lguest.c Wed Sep 26 14:48:31 2007 +1000
+++ b/Documentation/lguest/lguest.c Wed Sep 26 15:24:32 2007 +1000
@@ -32,7 +32,9 @@
#include <termios.h>
#include <getopt.h>
#include <zlib.h>
-/*L:110 We can ignore the 28 include files we need for this program, but I do
+#include <assert.h>
+#include <sched.h>
+/*L:110 We can ignore the 30 include files we need for this program, but I do
* want to draw attention to the use of kernel-style types.
*
* As Linus said, "C is a Spartan language, and so should your naming
be." I
@@ -44,6 +46,12 @@ typedef uint16_t u16;
typedef uint16_t u16;
typedef uint8_t u8;
#include "linux/lguest_launcher.h"
+#include "linux/pci_ids.h"
+#include "linux/virtio_config.h"
+#include "linux/virtio_net.h"
+#include "linux/virtio_blk.h"
+#include "linux/virtio_console.h"
+#include "linux/virtio_ring.h"
#include "asm/e820.h"
/*:*/
@@ -55,6 +63,8 @@ typedef uint8_t u8;
#endif
/* We can have up to 256 pages for devices. */
#define DEVICE_PAGES 256
+/* This fits nicely in a single 4096-byte page. */
+#define VIRTQUEUE_NUM 127
/*L:120 verbose is both a global flag and a macro. The C preprocessor allows
* this, and although I wouldn't recommend it, it works quite nicely here.
*/
@@ -78,8 +88,17 @@ struct device_list
fd_set infds;
int max_infd;
+ /* Counter to assign interrupt numbers. */
+ unsigned int next_irq;
+
+ /* Counter to print out convenient device numbers. */
+ unsigned int device_num;
+
/* The descriptor page for the devices. */
- struct lguest_device_desc *descs;
+ u8 *descpage;
+
+ /* The tail of the last descriptor. */
+ unsigned int desc_used;
/* A single linked list of devices. */
struct device *dev;
@@ -87,30 +106,87 @@ struct device_list
struct device **lastdev;
};
+/* The list of Guest devices, based on command line arguments. */
+static struct device_list devices;
+
/* The device structure describes a single device. */
struct device
{
/* The linked-list pointer. */
struct device *next;
- /* The descriptor for this device, as mapped into the Guest. */
+
+ /* The this device's descriptor, as mapped into the Guest. */
struct lguest_device_desc *desc;
- /* The memory page(s) of this device, if any. Also mapped in Guest. */
- void *mem;
+
+ /* The name of this device, for --verbose. */
+ const char *name;
/* If handle_input is set, it wants to be called when this file
* descriptor is ready. */
int fd;
bool (*handle_input)(int fd, struct device *me);
- /* If handle_output is set, it wants to be called when the Guest sends
- * DMA to this key. */
- unsigned long watch_key;
- u32 (*handle_output)(int fd, const struct iovec *iov,
- unsigned int num, struct device *me);
+ /* Any queues attached to this device */
+ struct virtqueue *vq;
/* Device-specific data. */
void *priv;
};
+
+/* The virtqueue structure describes a queue attached to a device. */
+struct virtqueue
+{
+ struct virtqueue *next;
+
+ /* Which device owns me. */
+ struct device *dev;
+
+ /* The configuration for this queue. */
+ struct lguest_vqconfig config;
+
+ /* The actual ring of buffers. */
+ struct vring vring;
+
+ /* Last available index we saw. */
+ u16 last_avail_idx;
+
+ /* The routine to call when the Guest pings us. */
+ void (*handle_output)(int fd, struct virtqueue *me);
+};
+
+/* Since guest is UP and we don't run at the same time, we don't need
barriers.
+ * But I include them in the code in case others copy it. */
+#define wmb()
+
+/* Convert an iovec element to the given type.
+ *
+ * This is a fairly ugly trick: we need to know the size of the type and
+ * alignment requirement to check the pointer is kosher. It's also nice to
+ * have the name of the type in case we report failure.
+ *
+ * Typing those three things all the time is cumbersome and error prone, so we
+ * have a macro which sets them all up and passes to the real function. */
+#define convert(iov, type) \
+ ((type *)_convert((iov), sizeof(type), __alignof__(type), #type))
+
+static void *_convert(struct iovec *iov, size_t size, size_t align,
+ const char *name)
+{
+ if (iov->iov_len != size)
+ errx(1, "Bad iovec size %zu for %s", iov->iov_len, name);
+ if ((unsigned long)iov->iov_base % align != 0)
+ errx(1, "Bad alignment %p for %s", iov->iov_base, name);
+ return iov->iov_base;
+}
+
+/* The virtio configuration space is defined to be little-endian. x86 is
+ * little-endian too, but it's nice to be explicit so we have these
helpers. */
+#define cpu_to_le16(v16) (v16)
+#define cpu_to_le32(v32) (v32)
+#define cpu_to_le64(v64) (v64)
+#define le16_to_cpu(v16) (v16)
+#define le32_to_cpu(v32) (v32)
+#define le64_to_cpu(v32) (v64)
/*L:100 The Launcher code itself takes us out into userspace, that scary place
* where pointers run wild and free! Unfortunately, like most userspace
@@ -486,11 +562,11 @@ static int tell_kernel(unsigned long pgd
}
/*:*/
-static void set_fd(int fd, struct device_list *devices)
-{
- FD_SET(fd, &devices->infds);
- if (fd > devices->max_infd)
- devices->max_infd = fd;
+static void add_device_fd(int fd)
+{
+ FD_SET(fd, &devices.infds);
+ if (fd > devices.max_infd)
+ devices.max_infd = fd;
}
/*L:200
@@ -508,18 +584,18 @@ static void set_fd(int fd, struct device
*
* This, of course, is merely a different *kind* of icky.
*/
-static void wake_parent(int pipefd, int lguest_fd, struct device_list *devices)
+static void wake_parent(int pipefd, int lguest_fd)
{
/* Add the pipe from the Launcher to the fdset in the device_list, so
* we watch it, too. */
- set_fd(pipefd, devices);
+ add_device_fd(pipefd);
for (;;) {
- fd_set rfds = devices->infds;
+ fd_set rfds = devices.infds;
unsigned long args[] = { LHREQ_BREAK, 1 };
/* Wait until input is ready from one of the devices. */
- select(devices->max_infd+1, &rfds, NULL, NULL, NULL);
+ select(devices.max_infd+1, &rfds, NULL, NULL, NULL);
/* Is it a message from the Launcher? */
if (FD_ISSET(pipefd, &rfds)) {
int ignorefd;
@@ -530,14 +606,14 @@ static void wake_parent(int pipefd, int
/* Otherwise it's telling us there's a problem with one
* of the devices, and we should ignore that file
* descriptor from now on. */
- FD_CLR(ignorefd, &devices->infds);
+ FD_CLR(ignorefd, &devices.infds);
} else /* Send LHREQ_BREAK command. */
write(lguest_fd, args, sizeof(args));
}
}
/* This routine just sets up a pipe to the Waker process. */
-static int setup_waker(int lguest_fd, struct device_list *device_list)
+static int setup_waker(int lguest_fd)
{
int pipefd[2], child;
@@ -551,7 +627,7 @@ static int setup_waker(int lguest_fd, st
if (child == 0) {
/* Close the "writing" end of our copy of the pipe */
close(pipefd[1]);
- wake_parent(pipefd[0], lguest_fd, device_list);
+ wake_parent(pipefd[0], lguest_fd);
}
/* Close the reading end of our copy of the pipe. */
close(pipefd[0]);
@@ -574,71 +650,13 @@ static void *_check_pointer(unsigned lon
/* We have to separately check addr and addr+size, because size could
* be huge and addr + size might wrap around. */
if (addr >= guest_limit || addr + size >= guest_limit)
- errx(1, "%s:%i: Invalid address %li", __FILE__, line, addr);
+ errx(1, "%s:%i: Invalid address %#lx", __FILE__, line, addr);
/* We return a pointer for the caller's convenience, now we know it's
* safe to use. */
return from_guest_phys(addr);
}
/* A macro which transparently hands the line number to the real function. */
#define check_pointer(addr,size) _check_pointer(addr, size, __LINE__)
-
-/* The Guest has given us the address of a "struct lguest_dma". We
check it's
- * OK and convert it to an iovec (which is a simple array of ptr/size
- * pairs). */
-static u32 *dma2iov(unsigned long dma, struct iovec iov[], unsigned *num)
-{
- unsigned int i;
- struct lguest_dma *udma;
-
- /* First we make sure that the array memory itself is valid. */
- udma = check_pointer(dma, sizeof(*udma));
- /* Now we check each element */
- for (i = 0; i < LGUEST_MAX_DMA_SECTIONS; i++) {
- /* A zero length ends the array. */
- if (!udma->len[i])
- break;
-
- iov[i].iov_base = check_pointer(udma->addr[i], udma->len[i]);
- iov[i].iov_len = udma->len[i];
- }
- *num = i;
-
- /* We return the pointer to where the caller should write the amount of
- * the buffer used. */
- return &udma->used_len;
-}
-
-/* This routine gets a DMA buffer from the Guest for a given key, and converts
- * it to an iovec array. It returns the interrupt the Guest wants when
we're
- * finished, and a pointer to the "used_len" field to fill in. */
-static u32 *get_dma_buffer(int fd, void *key,
- struct iovec iov[], unsigned int *num, u32 *irq)
-{
- unsigned long buf[] = { LHREQ_GETDMA, to_guest_phys(key) };
- unsigned long udma;
- u32 *res;
-
- /* Ask the kernel for a DMA buffer corresponding to this key. */
- udma = write(fd, buf, sizeof(buf));
- /* They haven't registered any, or they're all used? */
- if (udma == (unsigned long)-1)
- return NULL;
-
- /* Convert it into our iovec array */
- res = dma2iov(udma, iov, num);
- /* The kernel stashes irq in ->used_len to get it out to us. */
- *irq = *res;
- /* Return a pointer to ((struct lguest_dma *)udma)->used_len. */
- return res;
-}
-
-/* This is a convenient routine to send the Guest an interrupt. */
-static void trigger_irq(int fd, u32 irq)
-{
- unsigned long buf[] = { LHREQ_IRQ, irq };
- if (write(fd, buf, sizeof(buf)) != 0)
- err(1, "Triggering irq %i", irq);
-}
/* This simply sets up an iovec array where we can put data to be discarded.
* This happens when the Guest doesn't want or can't handle the input:
we have
@@ -652,6 +670,121 @@ static void discard_iovec(struct iovec *
iov->iov_len = sizeof(discard_buf);
}
+/* This function returns the next descriptor in the chain, or vq->vring.num.
*/
+static unsigned next_desc(struct virtqueue *vq, unsigned int i)
+{
+ unsigned int next;
+
+ /* If this descriptor says it doesn't chain, we're done. */
+ if (!(vq->vring.desc[i].flags & VRING_DESC_F_NEXT))
+ return vq->vring.num;
+
+ /* Check they're not leading us off end of descriptors. */
+ next = vq->vring.desc[i].next;
+ /* Make sure compiler knows to grab that: we don't want it changing! */
+ wmb();
+
+ if (next >= vq->vring.num)
+ errx(1, "Desc next is %u", next);
+
+ return next;
+}
+
+/* This looks in the virtqueue and for the first available buffer, and converts
+ * it to an iovec for convenient access. Since descriptors consist of some
+ * number of output then some number of input descriptors, it's actually
two
+ * iovecs, but we pack them into one and note how many of each there were.
+ *
+ * This function returns the descriptor number found, or vq->vring.num
(which
+ * is never a valid descriptor number) if none was found. */
+static unsigned get_vq_desc(struct virtqueue *vq,
+ struct iovec iov[],
+ unsigned int *out_num, unsigned int *in_num)
+{
+ unsigned int i, head;
+
+ /* Check it isn't doing very strange things with descriptor numbers. */
+ if ((u16)(vq->vring.avail->idx - vq->last_avail_idx) >
vq->vring.num)
+ errx(1, "Guest moved used index from %u to %u",
+ vq->last_avail_idx, vq->vring.avail->idx);
+
+ /* If there's nothing new since last we looked, return invalid. */
+ if (vq->vring.avail->idx == vq->last_avail_idx)
+ return vq->vring.num;
+
+ /* Grab the next descriptor number they're advertising, and increment
+ * the index we've seen. */
+ head = vq->vring.avail->ring[vq->last_avail_idx++ %
vq->vring.num];
+
+ /* If their number is silly, that's a fatal mistake. */
+ if (head >= vq->vring.num)
+ errx(1, "Guest says index %u is available", head);
+
+ /* When we start there are none of either input nor output. */
+ *out_num = *in_num = 0;
+
+ i = head;
+ do {
+ /* Grab the first descriptor, and check it's OK. */
+ iov[*out_num + *in_num].iov_len = vq->vring.desc[i].len;
+ iov[*out_num + *in_num].iov_base
+ = check_pointer(vq->vring.desc[i].addr,
+ vq->vring.desc[i].len);
+ /* If this is an input descriptor, increment that count. */
+ if (vq->vring.desc[i].flags & VRING_DESC_F_WRITE)
+ (*in_num)++;
+ else {
+ /* If it's an output descriptor, they're all supposed
+ * to come before any input descriptors. */
+ if (*in_num)
+ errx(1, "Descriptor has out after in");
+ (*out_num)++;
+ }
+
+ /* If we've got too many, that implies a descriptor loop. */
+ if (*out_num + *in_num > vq->vring.num)
+ errx(1, "Looped descriptor");
+ } while ((i = next_desc(vq, i)) != vq->vring.num);
+
+ return head;
+}
+
+/* Once we've used one of their buffers, we tell them about it. We'll
then
+ * want to send them an interrupt, using trigger_irq(). */
+static void add_used(struct virtqueue *vq, unsigned int head, int len)
+{
+ struct vring_used_elem *used;
+
+ /* Get a pointer to the next entry in the used ring. */
+ used = &vq->vring.used->ring[vq->vring.used->idx %
vq->vring.num];
+ used->id = head;
+ used->len = len;
+ /* Make sure buffer is written before we update index. */
+ wmb();
+ vq->vring.used->idx++;
+}
+
+/* This actually sends the interrupt for this virtqueue */
+static void trigger_irq(int fd, struct virtqueue *vq)
+{
+ unsigned long buf[] = { LHREQ_IRQ, vq->config.irq };
+
+ if (vq->vring.avail->flags & VRING_AVAIL_F_NO_INTERRUPT)
+ return;
+
+ /* Send the Guest an interrupt tell them we used something up. */
+ if (write(fd, buf, sizeof(buf)) != 0)
+ err(1, "Triggering irq %i", vq->config.irq);
+}
+
+/* And here's the combo meal deal. Supersize me! */
+static void add_used_and_trigger(int fd, struct virtqueue *vq,
+ unsigned int head, int len)
+{
+ add_used(vq, head, len);
+ trigger_irq(fd, vq);
+}
+
/* Here is the input terminal setting we save, and the routine to restore them
* on exit so the user can see what they type next. */
static struct termios orig_term;
@@ -672,38 +805,37 @@ struct console_abort
/* This is the routine which handles console input (ie. stdin). */
static bool handle_console_input(int fd, struct device *dev)
{
- u32 irq = 0, *lenp;
int len;
- unsigned int num;
- struct iovec iov[LGUEST_MAX_DMA_SECTIONS];
+ unsigned int head, in_num, out_num;
+ struct iovec iov[dev->vq->vring.num];
struct console_abort *abort = dev->priv;
- /* First we get the console buffer from the Guest. The key is dev->mem
- * which was set to 0 in setup_console(). */
- lenp = get_dma_buffer(fd, dev->mem, iov, &num, &irq);
- if (!lenp) {
- /* If it's not ready for input, warn and set up to discard. */
- warn("console: no dma buffer!");
- discard_iovec(iov, &num);
- }
+ /* First we need a console buffer from the Guests's input virtqueue. */
+ head = get_vq_desc(dev->vq, iov, &out_num, &in_num);
+ if (head == dev->vq->vring.num) {
+ /* If they're not ready for input, we warn and set up to
+ * discard. */
+ warnx("console: no dma buffer!");
+ discard_iovec(iov, &in_num);
+ } else if (out_num)
+ errx(1, "Output buffers in console in queue?");
/* This is why we convert to iovecs: the readv() call uses them, and so
* it reads straight into the Guest's buffer. */
- len = readv(dev->fd, iov, num);
+ len = readv(dev->fd, iov, in_num);
if (len <= 0) {
/* This implies that the console is closed, is /dev/null, or
- * something went terribly wrong. We still go through the rest
- * of the logic, though, especially the exit handling below. */
+ * something went terribly wrong. */
warnx("Failed to get console input, ignoring console.");
- len = 0;
- }
-
- /* If we read the data into the Guest, fill in the length and send the
- * interrupt. */
- if (lenp) {
- *lenp = len;
- trigger_irq(fd, irq);
- }
+ /* Put the input terminal back and return failure (meaning,
+ * don't call us again). */
+ restore_term();
+ return false;
+ }
+
+ /* If we actually read the data into the Guest, tell them about it. */
+ if (head != dev->vq->vring.num)
+ add_used_and_trigger(fd, dev->vq, head, len);
/* Three ^C within one second? Exit.
*
@@ -732,202 +864,137 @@ static bool handle_console_input(int fd,
/* Any other key resets the abort counter. */
abort->count = 0;
- /* Now, if we didn't read anything, put the input terminal back and
- * return failure (meaning, don't call us again). */
- if (!len) {
- restore_term();
- return false;
- }
/* Everything went OK! */
return true;
}
-/* Handling console output is much simpler than input. */
-static u32 handle_console_output(int fd, const struct iovec *iov,
- unsigned num, struct device*dev)
-{
- /* Whatever the Guest sends, write it to standard output. Return the
- * number of bytes written. */
- return writev(STDOUT_FILENO, iov, num);
-}
-
-/* Guest->Host network output is also pretty easy. */
-static u32 handle_tun_output(int fd, const struct iovec *iov,
- unsigned num, struct device *dev)
-{
- /* We put a flag in the "priv" pointer of the network device, and
set
- * it as soon as we see output. We'll see why in handle_tun_input() */
- *(bool *)dev->priv = true;
- /* Whatever packet the Guest sent us, write it out to the tun
- * device. */
- return writev(dev->fd, iov, num);
-}
-
-/* This matches the peer_key() in lguest_net.c. The key for any given slot
- * is the address of the network device's page plus 4 * the slot number. */
-static unsigned long peer_offset(unsigned int peernum)
-{
- return 4 * peernum;
-}
-
-/* This is where we handle a packet coming in from the tun device */
+/* Handling output for console is simple: we just get all the output buffers
+ * and write them to stdout. */
+static void handle_console_output(int fd, struct virtqueue *vq)
+{
+ unsigned int head, out, in;
+ int len;
+ struct iovec iov[vq->vring.num];
+
+ /* Keep getting output buffers from the Guest until we run out. */
+ while ((head = get_vq_desc(vq, iov, &out, &in)) != vq->vring.num) {
+ if (in)
+ errx(1, "Input buffers in output queue?");
+ len = writev(STDOUT_FILENO, iov, out);
+ add_used_and_trigger(fd, vq, head, len);
+ }
+}
+
+/* Handling output for network is also simple: we get all the output buffers
+ * and write them (ignoring the first element) to this device's file
descriptor
+ * (stdout). */
+static void handle_net_output(int fd, struct virtqueue *vq)
+{
+ unsigned int head, out, in;
+ int len;
+ struct iovec iov[vq->vring.num];
+
+ /* Keep getting output buffers from the Guest until we run out. */
+ while ((head = get_vq_desc(vq, iov, &out, &in)) != vq->vring.num) {
+ if (in)
+ errx(1, "Input buffers in output queue?");
+ /* Check header, but otherwise ignore it (we said we supported
+ * no features). */
+ (void)convert(&iov[0], struct virtio_net_hdr);
+ len = writev(vq->dev->fd, iov+1, out-1);
+ add_used_and_trigger(fd, vq, head, len);
+ }
+}
+
+/* This is where we handle a packet coming in from the tun device to our
+ * Guest. */
static bool handle_tun_input(int fd, struct device *dev)
{
- u32 irq = 0, *lenp;
+ unsigned int head, in_num, out_num;
int len;
- unsigned num;
- struct iovec iov[LGUEST_MAX_DMA_SECTIONS];
-
- /* First we get a buffer the Guest has bound to its key. */
- lenp = get_dma_buffer(fd, dev->mem+peer_offset(NET_PEERNUM), iov, &num,
- &irq);
- if (!lenp) {
+ struct iovec iov[dev->vq->vring.num];
+ struct virtio_net_hdr *hdr;
+
+ /* First we need a network buffer from the Guests's recv virtqueue. */
+ head = get_vq_desc(dev->vq, iov, &out_num, &in_num);
+ if (head == dev->vq->vring.num) {
/* Now, it's expected that if we try to send a packet too
- * early, the Guest won't be ready yet. This is why we set a
- * flag when the Guest sends its first packet. If it's sent a
- * packet we assume it should be ready to receive them.
- *
- * Actually, this is what the status bits in the descriptor are
- * for: we should *use* them. FIXME! */
- if (*(bool *)dev->priv)
+ * early, the Guest won't be ready yet. Wait until the device
+ * status says it's ready. */
+ /* FIXME: Actually want DRIVER_ACTIVE here. */
+ if (dev->desc->status & VIRTIO_CONFIG_S_DRIVER_OK)
warn("network: no dma buffer!");
- discard_iovec(iov, &num);
- }
+ discard_iovec(iov, &in_num);
+ } else if (out_num)
+ errx(1, "Output buffers in network recv queue?");
+
+ /* First element is the header: we set it to 0 (no features). */
+ hdr = convert(&iov[0], struct virtio_net_hdr);
+ hdr->flags = 0;
+ hdr->gso_type = VIRTIO_NET_HDR_GSO_NONE;
/* Read the packet from the device directly into the Guest's buffer. */
- len = readv(dev->fd, iov, num);
+ len = readv(dev->fd, iov+1, in_num-1);
if (len <= 0)
err(1, "reading network");
- /* Write the used_len, and trigger the interrupt for the Guest */
- if (lenp) {
- *lenp = len;
- trigger_irq(fd, irq);
- }
+ /* If we actually read the data into the Guest, tell them about it. */
+ if (head != dev->vq->vring.num)
+ add_used_and_trigger(fd, dev->vq, head, sizeof(*hdr) + len);
+
verbose("tun input packet len %i [%02x %02x] (%s)\n", len,
- ((u8 *)iov[0].iov_base)[0], ((u8 *)iov[0].iov_base)[1],
- lenp ? "sent" : "discarded");
+ ((u8 *)iov[1].iov_base)[0], ((u8 *)iov[1].iov_base)[1],
+ head != dev->vq->vring.num ? "sent" : "discarded");
+
/* All good. */
return true;
}
-/* The last device handling routine is block output: the Guest has sent a DMA
- * to the block device. It will have placed the command it wants in the
- * "struct lguest_block_page". */
-static u32 handle_block_output(int fd, const struct iovec *iov,
- unsigned num, struct device *dev)
-{
- struct lguest_block_page *p = dev->mem;
- u32 irq, *lenp;
- unsigned int len, reply_num;
- struct iovec reply[LGUEST_MAX_DMA_SECTIONS];
- off64_t device_len, off = (off64_t)p->sector * 512;
-
- /* First we extract the device length from the dev->priv pointer. */
- device_len = *(off64_t *)dev->priv;
-
- /* We first check that the read or write is within the length of the
- * block file. */
- if (off >= device_len)
- errx(1, "Bad offset %llu vs %llu", off, device_len);
- /* Move to the right location in the block file. This shouldn't fail,
- * but best to check. */
- if (lseek64(dev->fd, off, SEEK_SET) != off)
- err(1, "Bad seek to sector %i", p->sector);
-
- verbose("Block: %s at offset %llu\n", p->type ? "WRITE"
: "READ", off);
-
- /* They were supposed to bind a reply buffer at key equal to the start
- * of the block device memory. We need this to tell them when the
- * request is finished. */
- lenp = get_dma_buffer(fd, dev->mem, reply, &reply_num, &irq);
- if (!lenp)
- err(1, "Block request didn't give us a dma buffer");
-
- if (p->type) {
- /* A write request. The DMA they sent contained the data, so
- * write it out. */
- len = writev(dev->fd, iov, num);
- /* Grr... Now we know how long the "struct lguest_dma" they
- * sent was, we make sure they didn't try to write over the end
- * of the block file (possibly extending it). */
- if (off + len > device_len) {
- /* Trim it back to the correct length */
- ftruncate64(dev->fd, device_len);
- /* Die, bad Guest, die. */
- errx(1, "Write past end %llu+%u", off, len);
+/* This is the generic routine we call when the Guest uses LHCALL_NOTIFY. */
+static void handle_output(int fd, unsigned long addr)
+{
+ struct device *i;
+ struct virtqueue *vq;
+
+ /* Check each virtqueue. */
+ for (i = devices.dev; i; i = i->next) {
+ for (vq = i->vq; vq; vq = vq->next) {
+ if (vq->config.pfn == addr/getpagesize()
+ && vq->handle_output) {
+ verbose("Output to %s\n", vq->dev->name);
+ vq->handle_output(fd, vq);
+ return;
+ }
}
- /* The reply length is 0: we just send back an empty DMA to
- * interrupt them and tell them the write is finished. */
- *lenp = 0;
- } else {
- /* A read request. They sent an empty DMA to start the
- * request, and we put the read contents into the reply
- * buffer. */
- len = readv(dev->fd, reply, reply_num);
- *lenp = len;
- }
-
- /* The result is 1 (done), 2 if there was an error (short read or
- * write). */
- p->result = 1 + (p->bytes != len);
- /* Now tell them we've used their reply buffer. */
- trigger_irq(fd, irq);
-
- /* We're supposed to return the number of bytes of the output buffer we
- * used. But the block device uses the "result" field instead, so
we
- * don't bother. */
- return 0;
-}
-
-/* This is the generic routine we call when the Guest sends some DMA out. */
-static void handle_output(int fd, unsigned long dma, unsigned long key,
- struct device_list *devices)
-{
- struct device *i;
- u32 *lenp;
- struct iovec iov[LGUEST_MAX_DMA_SECTIONS];
- unsigned num = 0;
-
- /* Convert the "struct lguest_dma" they're sending to a
"struct
- * iovec". */
- lenp = dma2iov(dma, iov, &num);
-
- /* Check each device: if they expect output to this key, tell them to
- * handle it. */
- for (i = devices->dev; i; i = i->next) {
- if (i->handle_output && key == i->watch_key) {
- /* We write the result straight into the used_len field
- * for them. */
- *lenp = i->handle_output(fd, iov, num, i);
- return;
- }
- }
-
- /* This can happen: the kernel sends any SEND_DMA which doesn't match
- * another Guest to us. It could be that another Guest just left a
- * network, for example. But it's unusual. */
- warnx("Pending dma %p, key %p", (void *)dma, (void *)key);
+ }
+
+ /* Early console write is done using notify on a nul-terminated string
+ * in Guest memory. */
+ if (addr >= guest_limit)
+ errx(1, "Bad NOTIFY %#lx", addr);
+
+ write(STDOUT_FILENO, from_guest_phys(addr),
+ strnlen(from_guest_phys(addr), guest_limit - addr));
}
/* This is called when the waker wakes us up: check for incoming file
* descriptors. */
-static void handle_input(int fd, struct device_list *devices)
+static void handle_input(int fd)
{
/* select() wants a zeroed timeval to mean "don't wait". */
struct timeval poll = { .tv_sec = 0, .tv_usec = 0 };
for (;;) {
struct device *i;
- fd_set fds = devices->infds;
+ fd_set fds = devices.infds;
/* If nothing is ready, we're done. */
- if (select(devices->max_infd+1, &fds, NULL, NULL, &poll) == 0)
+ if (select(devices.max_infd+1, &fds, NULL, NULL, &poll) == 0)
break;
/* Otherwise, call the device(s) which have readable
* file descriptors and a method of handling them. */
- for (i = devices->dev; i; i = i->next) {
+ for (i = devices.dev; i; i = i->next) {
if (i->handle_input && FD_ISSET(i->fd, &fds)) {
/* If handle_input() returns false, it means we
* should no longer service it.
@@ -936,7 +1003,7 @@ static void handle_input(int fd, struct
/* Clear it from the set of input file
* descriptors kept at the head of the
* device list. */
- FD_CLR(i->fd, &devices->infds);
+ FD_CLR(i->fd, &devices.infds);
/* Tell waker to ignore it too... */
write(waker_fd, &i->fd, sizeof(i->fd));
}
@@ -953,43 +1020,93 @@ static void handle_input(int fd, struct
* routines to allocate them.
*
* This routine allocates a new "struct lguest_device_desc" from
descriptor
- * table in the devices array just above the Guest's normal memory. */
-static struct lguest_device_desc *
-new_dev_desc(struct lguest_device_desc *descs,
- u16 type, u16 features, u16 num_pages)
-{
- unsigned int i;
-
- for (i = 0; i < LGUEST_MAX_DEVICES; i++) {
- if (!descs[i].type) {
- descs[i].type = type;
- descs[i].features = features;
- descs[i].num_pages = num_pages;
- /* If they said the device needs memory, we allocate
- * that now. */
- if (num_pages) {
- unsigned long pa;
- pa = to_guest_phys(get_pages(num_pages));
- descs[i].pfn = pa / getpagesize();
- }
- return &descs[i];
- }
- }
- errx(1, "too many devices");
-}
-
-/* This monster routine does all the creation and setup of a new device,
- * including caling new_dev_desc() to allocate the descriptor and device
- * memory. */
-static struct device *new_device(struct device_list *devices,
- u16 type, u16 num_pages, u16 features,
- int fd,
- bool (*handle_input)(int, struct device *),
- unsigned long watch_off,
- u32 (*handle_output)(int,
- const struct iovec *,
- unsigned,
- struct device *))
+ * table just above the Guest's normal memory. It returns a pointer to
that
+ * descriptor. */
+static struct lguest_device_desc *new_dev_desc(u16 type)
+{
+ struct lguest_device_desc *d;
+
+ /* We only have one page for all the descriptors. */
+ if (devices.desc_used + sizeof(*d) > getpagesize())
+ errx(1, "Too many devices");
+
+ /* We don't need to set config_len or status: page is 0 already. */
+ d = (void *)devices.descpage + devices.desc_used;
+ d->type = type;
+ devices.desc_used += sizeof(*d);
+
+ return d;
+}
+
+/* Each device descriptor is followed by some configuration information.
+ * The first byte is a "status" byte for the Guest to report
what's happening.
+ * After that are fields: u8 type, u8 len, [... len bytes...].
+ *
+ * This routine adds a new field to an existing device's descriptor. It
only
+ * works for the last device, but that's OK because that's how we use
it. */
+static void add_desc_field(struct device *dev, u8 type, u8 len, const void *c)
+{
+ /* This is the last descriptor, right? */
+ assert(devices.descpage + devices.desc_used
+ == (u8 *)(dev->desc + 1) + dev->desc->config_len);
+
+ /* We only have one page of device descriptions. */
+ if (devices.desc_used + 2 + len > getpagesize())
+ errx(1, "Too many devices");
+
+ /* Copy in the new config header: type then length. */
+ devices.descpage[devices.desc_used++] = type;
+ devices.descpage[devices.desc_used++] = len;
+ memcpy(devices.descpage + devices.desc_used, c, len);
+ devices.desc_used += len;
+
+ /* Update the device descriptor length: two byte head then data. */
+ dev->desc->config_len += 2 + len;
+}
+
+/* This routine adds a virtqueue to a device. We specify how many descriptors
+ * the virtqueue is to have. */
+static void add_virtqueue(struct device *dev, unsigned int num_descs,
+ void (*handle_output)(int fd, struct virtqueue *me))
+{
+ unsigned int pages;
+ struct virtqueue **i, *vq = malloc(sizeof(*vq));
+ void *p;
+
+ /* First we need some pages for this virtqueue. */
+ pages = (vring_size(num_descs) + getpagesize() - 1) / getpagesize();
+ p = get_pages(pages);
+
+ /* Initialize the configuration. */
+ vq->config.num = num_descs;
+ vq->config.irq = devices.next_irq++;
+ vq->config.pfn = to_guest_phys(p) / getpagesize();
+
+ /* Initialize the vring. */
+ vring_init(&vq->vring, num_descs, p);
+
+ /* Add the configuration information to this device's descriptor. */
+ add_desc_field(dev, VIRTIO_CONFIG_F_VIRTQUEUE,
+ sizeof(vq->config), &vq->config);
+
+ /* Add to tail of list, so dev->vq is first vq, dev->vq->next is
+ * second. */
+ for (i = &dev->vq; *i; i = &(*i)->next);
+ *i = vq;
+
+ /* Link virtqueue back to device. */
+ vq->dev = dev;
+
+ /* Set up handler. */
+ vq->handle_output = handle_output;
+ if (!handle_output)
+ vq->vring.used->flags = VRING_USED_F_NO_NOTIFY;
+}
+
+/* This routine does all the creation and setup of a new device, including
+ * caling new_dev_desc() to allocate the descriptor and device memory. */
+static struct device *new_device(const char *name, u16 type, int fd,
+ bool (*handle_input)(int, struct device *))
{
struct device *dev = malloc(sizeof(*dev));
@@ -997,27 +1114,25 @@ static struct device *new_device(struct
* easier, but the user expects the devices to be arranged on the bus
* in command-line order. The first network device on the command line
* is eth0, the first block device /dev/lgba, etc. */
- *devices->lastdev = dev;
+ *devices.lastdev = dev;
dev->next = NULL;
- devices->lastdev = &dev->next;
+ devices.lastdev = &dev->next;
/* Now we populate the fields one at a time. */
dev->fd = fd;
/* If we have an input handler for this file descriptor, then we add it
* to the device_list's fdset and maxfd. */
if (handle_input)
- set_fd(dev->fd, devices);
- dev->desc = new_dev_desc(devices->descs, type, features, num_pages);
- dev->mem = from_guest_phys(dev->desc->pfn * getpagesize());
+ add_device_fd(dev->fd);
+ dev->desc = new_dev_desc(type);
dev->handle_input = handle_input;
- dev->watch_key = to_guest_phys(dev->mem) + watch_off;
- dev->handle_output = handle_output;
+ dev->name = name;
return dev;
}
/* Our first setup routine is the console. It's a fairly simple device,
but
* UNIX tty handling makes it uglier than it could be. */
-static void setup_console(struct device_list *devices)
+static void setup_console(void)
{
struct device *dev;
@@ -1033,127 +1148,38 @@ static void setup_console(struct device_
atexit(restore_term);
}
- /* We don't currently require any memory for the console, so we ask for
- * 0 pages. */
- dev = new_device(devices, LGUEST_DEVICE_T_CONSOLE, 0, 0,
- STDIN_FILENO, handle_console_input,
- LGUEST_CONSOLE_DMA_KEY, handle_console_output);
+ dev = new_device("console", VIRTIO_ID_CONSOLE,
+ STDIN_FILENO, handle_console_input);
/* We store the console state in dev->priv, and initialize it. */
dev->priv = malloc(sizeof(struct console_abort));
((struct console_abort *)dev->priv)->count = 0;
- verbose("device %p: console\n",
- (void *)(dev->desc->pfn * getpagesize()));
-}
-
-/* Setting up a block file is also fairly straightforward. */
-static void setup_block_file(const char *filename, struct device_list *devices)
-{
- int fd;
- struct device *dev;
- off64_t *device_len;
- struct lguest_block_page *p;
-
- /* We open with O_LARGEFILE because otherwise we get stuck at 2G. We
- * open with O_DIRECT because otherwise our benchmarks go much too
- * fast. */
- fd = open_or_die(filename, O_RDWR|O_LARGEFILE|O_DIRECT);
-
- /* We want one page, and have no input handler (the block file never
- * has anything interesting to say to us). Our timing will be quite
- * random, so it should be a reasonable randomness source. */
- dev = new_device(devices, LGUEST_DEVICE_T_BLOCK, 1,
- LGUEST_DEVICE_F_RANDOMNESS,
- fd, NULL, 0, handle_block_output);
-
- /* We store the device size in the private area */
- device_len = dev->priv = malloc(sizeof(*device_len));
- /* This is the safe way of establishing the size of our device: it
- * might be a normal file or an actual block device like /dev/hdb. */
- *device_len = lseek64(fd, 0, SEEK_END);
-
- /* The device memory is a "struct lguest_block_page". It's
zeroed
- * already, we just need to put in the device size. Block devices
- * think in sectors (ie. 512 byte chunks), so we translate here. */
- p = dev->mem;
- p->num_sectors = *device_len/512;
- verbose("device %p: block %i sectors\n",
- (void *)(dev->desc->pfn * getpagesize()), p->num_sectors);
-}
-
-/*
- * Network Devices.
- *
- * Setting up network devices is quite a pain, because we have three types.
- * First, we have the inter-Guest network. This is a file which is mapped into
- * the address space of the Guests who are on the network. Because it is a
- * shared mapping, the same page underlies all the devices, and they can send
- * DMA to each other.
- *
- * Remember from our network driver, the Guest is told what slot in the page it
- * is to use. We use exclusive fnctl locks to reserve a slot. If another
- * Guest is using a slot, the lock will fail and we try another. Because fnctl
- * locks are cleaned up automatically when we die, this cleverly means that our
- * reservation on the slot will vanish if we crash. */
-static unsigned int find_slot(int netfd, const char *filename)
-{
- struct flock fl;
-
- fl.l_type = F_WRLCK;
- fl.l_whence = SEEK_SET;
- fl.l_len = 1;
- /* Try a 1 byte lock in each possible position number */
- for (fl.l_start = 0;
- fl.l_start < getpagesize()/sizeof(struct lguest_net);
- fl.l_start++) {
- /* If we succeed, return the slot number. */
- if (fcntl(netfd, F_SETLK, &fl) == 0)
- return fl.l_start;
- }
- errx(1, "No free slots in network file %s", filename);
-}
-
-/* This function sets up the network file */
-static void setup_net_file(const char *filename,
- struct device_list *devices)
-{
- int netfd;
- struct device *dev;
-
- /* We don't use open_or_die() here: for friendliness we create the file
- * if it doesn't already exist. */
- netfd = open(filename, O_RDWR, 0);
- if (netfd < 0) {
- if (errno == ENOENT) {
- netfd = open(filename, O_RDWR|O_CREAT, 0600);
- if (netfd >= 0) {
- /* If we succeeded, initialize the file with a
- * blank page. */
- char page[getpagesize()];
- memset(page, 0, sizeof(page));
- write(netfd, page, sizeof(page));
- }
- }
- if (netfd < 0)
- err(1, "cannot open net file '%s'", filename);
- }
-
- /* We need 1 page, and the features indicate the slot to use and that
- * no checksum is needed. We never touch this device again; it's
- * between the Guests on the network, so we don't register input or
- * output handlers. */
- dev = new_device(devices, LGUEST_DEVICE_T_NET, 1,
- find_slot(netfd, filename)|LGUEST_NET_F_NOCSUM,
- -1, NULL, 0, NULL);
-
- /* Map the shared file. */
- if (mmap(dev->mem, getpagesize(), PROT_READ|PROT_WRITE,
- MAP_FIXED|MAP_SHARED, netfd, 0) != dev->mem)
- err(1, "could not mmap '%s'", filename);
- verbose("device %p: shared net %s, peer %i\n",
- (void *)(dev->desc->pfn * getpagesize()), filename,
- dev->desc->features & ~LGUEST_NET_F_NOCSUM);
+
+ /* The console needs two virtqueues: the input then the output. We
+ * don't care when they refill the input queue, since we don't hold
+ * data waiting for them. That's why the input queue's callback is
+ * NULL. */
+ add_virtqueue(dev, VIRTQUEUE_NUM, NULL);
+ add_virtqueue(dev, VIRTQUEUE_NUM, handle_console_output);
+
+ verbose("device %u: console\n", devices.device_num++);
}
/*:*/
+
+/*M:010 Inter-guest networking is an interesting area. Simplest is to have a
+ * --sharenet=<name> option which opens or creates a named pipe. This
can be
+ * used to send packets to another guest in a 1:1 manner.
+ *
+ * More sopisticated is to use one of the tools developed for project like UML
+ * to do networking.
+ *
+ * Faster is to do virtio bonding in kernel. Doing this 1:1 would be
+ * completely generic ("here's my vring, attach to your vring")
and would work
+ * for any traffic. Of course, namespace and permissions issues need to be
+ * dealt with. A more sophisticated "multi-channel" virtio_net.c
could hide
+ * multiple inter-guest channels behind one interface, although it would
+ * require some manner of hotplugging new virtio channels.
+ *
+ * Finally, we could implement a virtio network switch in the kernel. :*/
static u32 str2ip(const char *ipaddr)
{
@@ -1188,7 +1214,7 @@ static void add_to_bridge(int fd, const
/* This sets up the Host end of the network device with an IP address, brings
* it up so packets will flow, the copies the MAC address into the hwaddr
- * pointer (in practice, the Host's slot in the network device's
memory). */
+ * pointer. */
static void configure_device(int fd, const char *devname, u32 ipaddr,
unsigned char hwaddr[6])
{
@@ -1214,18 +1240,18 @@ static void configure_device(int fd, con
memcpy(hwaddr, ifr.ifr_hwaddr.sa_data, 6);
}
-/*L:195 The other kind of network is a Host<->Guest network. This can
either
- * use briding or routing, but the principle is the same: it uses the
"tun"
- * device to inject packets into the Host as if they came in from a normal
- * network card. We just shunt packets between the Guest and the tun
- * device. */
-static void setup_tun_net(const char *arg, struct device_list *devices)
+/*L:195 Our network is a Host<->Guest network. This can either use
bridging or
+ * routing, but the principle is the same: it uses the "tun" device
to inject
+ * packets into the Host as if they came in from a normal network card. We
+ * just shunt packets between the Guest and the tun device. */
+static void setup_tun_net(const char *arg)
{
struct device *dev;
struct ifreq ifr;
int netfd, ipfd;
u32 ip;
const char *br_name = NULL;
+ u8 hwaddr[6];
/* We open the /dev/net/tun device and tell it we want a tap device. A
* tap device is like a tun device, only somehow different. To tell
@@ -1241,21 +1267,12 @@ static void setup_tun_net(const char *ar
* device: trust us! */
ioctl(netfd, TUNSETNOCSUM, 1);
- /* We create the net device with 1 page, using the features field of
- * the descriptor to tell the Guest it is in slot 1 (NET_PEERNUM), and
- * that the device has fairly random timing. We do *not* specify
- * LGUEST_NET_F_NOCSUM: these packets can reach the real world.
- *
- * We will put our MAC address is slot 0 for the Guest to see, so
- * it will send packets to us using the key "peer_offset(0)": */
- dev = new_device(devices, LGUEST_DEVICE_T_NET, 1,
- NET_PEERNUM|LGUEST_DEVICE_F_RANDOMNESS, netfd,
- handle_tun_input, peer_offset(0), handle_tun_output);
-
- /* We keep a flag which says whether we've seen packets come out from
- * this network device. */
- dev->priv = malloc(sizeof(bool));
- *(bool *)dev->priv = false;
+ /* First we create a new network device. */
+ dev = new_device("net", VIRTIO_ID_NET, netfd, handle_tun_input);
+
+ /* Network devices need a receive and a send queue. */
+ add_virtqueue(dev, VIRTQUEUE_NUM, NULL);
+ add_virtqueue(dev, VIRTQUEUE_NUM, handle_net_output);
/* We need a socket to perform the magic network ioctls to bring up the
* tap interface, connect to the bridge etc. Any socket will do! */
@@ -1271,44 +1288,251 @@ static void setup_tun_net(const char *ar
} else /* It is an IP address to set up the device with */
ip = str2ip(arg);
- /* We are peer 0, ie. first slot, so we hand dev->mem to this routine
- * to write the MAC address at the start of the device memory. */
- configure_device(ipfd, ifr.ifr_name, ip, dev->mem);
-
- /* Set "promisc" bit: we want every single packet if we're going
to
- * bridge to other machines (and otherwise it doesn't matter). */
- *((u8 *)dev->mem) |= 0x1;
-
+ /* Set up the tun device, and get the mac address for the interface. */
+ configure_device(ipfd, ifr.ifr_name, ip, hwaddr);
+
+ /* Tell Guest what MAC address to use. */
+ add_desc_field(dev, VIRTIO_CONFIG_NET_MAC_F, sizeof(hwaddr), hwaddr);
+
+ /* We don't seed the socket any more; setup is done. */
close(ipfd);
- verbose("device %p: tun net %u.%u.%u.%u\n",
- (void *)(dev->desc->pfn * getpagesize()),
- (u8)(ip>>24), (u8)(ip>>16), (u8)(ip>>8), (u8)ip);
+ verbose("device %u: tun net %u.%u.%u.%u\n",
+ devices.device_num++,
+ (u8)(ip>>24),(u8)(ip>>16),(u8)(ip>>8),(u8)ip);
if (br_name)
verbose("attached to bridge: %s\n", br_name);
}
+
+
+/*
+ * Block device.
+ *
+ * Serving a block device is really easy: the Guest asks for a block number and
+ * we read or write that position in the file.
+ *
+ * Unfortunately, this is amazingly slow: the Guest waits until the read is
+ * finished before running anything else, even if it could be doing useful
+ * work. We could use async I/O, except it's reputed to suck so hard that
+ * characters actually go missing from your code when you try to use it.
+ *
+ * So we farm the I/O out to thread, and communicate with it via a pipe. */
+
+/* This hangs off device->priv, with the data. */
+struct vblk_info
+{
+ /* The size of the file. */
+ off64_t len;
+
+ /* The file descriptor for the file. */
+ int fd;
+
+ /* IO thread listens on this file descriptor [0]. */
+ int workpipe[2];
+
+ /* IO thread writes to this file descriptor to mark it done, then
+ * Launcher triggers interrupt to Guest. */
+ int done_fd;
+};
+
+/* This is the core of the I/O thread. It returns true if it did something. */
+static bool service_io(struct device *dev)
+{
+ struct vblk_info *vblk = dev->priv;
+ unsigned int head, out_num, in_num, wlen;
+ int ret;
+ struct virtio_blk_inhdr *in;
+ struct virtio_blk_outhdr *out;
+ struct iovec iov[dev->vq->vring.num];
+ off64_t off;
+
+ head = get_vq_desc(dev->vq, iov, &out_num, &in_num);
+ if (head == dev->vq->vring.num)
+ return false;
+
+ if (out_num == 0 || in_num == 0)
+ errx(1, "Bad virtblk cmd %u out=%u in=%u",
+ head, out_num, in_num);
+
+ out = convert(&iov[0], struct virtio_blk_outhdr);
+ in = convert(&iov[out_num+in_num-1], struct virtio_blk_inhdr);
+ off = out->sector * 512;
+
+ /* This is how we implement barriers. Pretty poor, no? */
+ if (out->type & VIRTIO_BLK_T_BARRIER)
+ fdatasync(vblk->fd);
+
+ if (out->type & VIRTIO_BLK_T_SCSI_CMD) {
+ fprintf(stderr, "Scsi commands unsupported\n");
+ in->status = VIRTIO_BLK_S_UNSUPP;
+ wlen = sizeof(in);
+ } else if (out->type & VIRTIO_BLK_T_OUT) {
+ /* Write */
+
+ /* Move to the right location in the block file. This can fail
+ * if they try to write past end. */
+ if (lseek64(vblk->fd, off, SEEK_SET) != off)
+ err(1, "Bad seek to sector %llu", out->sector);
+
+ ret = writev(vblk->fd, iov+1, out_num-1);
+ verbose("WRITE to sector %llu: %i\n", out->sector, ret);
+
+ /* Grr... Now we know how long the descriptor they sent was, we
+ * make sure they didn't try to write over the end of the block
+ * file (possibly extending it). */
+ if (ret > 0 && off + ret > vblk->len) {
+ /* Trim it back to the correct length */
+ ftruncate64(vblk->fd, vblk->len);
+ /* Die, bad Guest, die. */
+ errx(1, "Write past end %llu+%u", off, ret);
+ }
+ wlen = sizeof(in);
+ in->status = (ret >= 0 ? VIRTIO_BLK_S_OK : VIRTIO_BLK_S_IOERR);
+ } else {
+ /* Read */
+
+ /* Move to the right location in the block file. This can fail
+ * if they try to read past end. */
+ if (lseek64(vblk->fd, off, SEEK_SET) != off)
+ err(1, "Bad seek to sector %llu", out->sector);
+
+ ret = readv(vblk->fd, iov+1, in_num-1);
+ verbose("READ from sector %llu: %i\n", out->sector, ret);
+ if (ret >= 0) {
+ wlen = sizeof(in) + ret;
+ in->status = VIRTIO_BLK_S_OK;
+ } else {
+ wlen = sizeof(in);
+ in->status = VIRTIO_BLK_S_IOERR;
+ }
+ }
+
+ /* We can't trigger an IRQ, because we're not the Launcher. It does
+ * that when we tell it we're done. */
+ add_used(dev->vq, head, wlen);
+ return true;
+}
+
+/* This is the thread which actually services the I/O. */
+static int io_thread(void *_dev)
+{
+ struct device *dev = _dev;
+ struct vblk_info *vblk = dev->priv;
+ char c;
+
+ /* Close other side of workpipe so we get 0 read when main dies. */
+ close(vblk->workpipe[1]);
+ /* Close the other side of the done_fd pipe. */
+ close(dev->fd);
+
+ /* When this read fails, it means Launcher died, so we follow. */
+ while (read(vblk->workpipe[0], &c, 1) == 1) {
+ /* We acknowledge each request immediately, to reduce latency,
+ * rather than waiting until we've done them all. I haven't
+ * measured to see if it makes any difference. */
+ while (service_io(dev))
+ write(vblk->done_fd, &c, 1);
+ }
+ return 0;
+}
+
+/* When the thread says some I/O is done, we interrupt the Guest. */
+static bool handle_io_finish(int fd, struct device *dev)
+{
+ char c;
+
+ /* If child died, presumably it printed message. */
+ if (read(dev->fd, &c, 1) != 1)
+ exit(1);
+
+ /* It did some work, so trigger the irq. */
+ trigger_irq(fd, dev->vq);
+ return true;
+}
+
+/* When the Guest submits some I/O, we wake the I/O thread. */
+static void handle_virtblk_output(int fd, struct virtqueue *vq)
+{
+ struct vblk_info *vblk = vq->dev->priv;
+ char c = 0;
+
+ /* Wake up I/O thread and tell it to go to work! */
+ if (write(vblk->workpipe[1], &c, 1) != 1)
+ /* Presumably it indicated why it died. */
+ exit(1);
+}
+
+/* This creates a virtual block device. */
+static void setup_block_file(const char *filename)
+{
+ int p[2];
+ struct device *dev;
+ struct vblk_info *vblk;
+ void *stack;
+ u64 cap;
+ unsigned int val;
+
+ /* This is the pipe the I/O thread will use to tell us I/O is done. */
+ pipe(p);
+
+ /* The device responds to return from I/O thread. */
+ dev = new_device("block", VIRTIO_ID_BLOCK, p[0], handle_io_finish);
+
+ /* The device has a virtqueue. */
+ add_virtqueue(dev, VIRTQUEUE_NUM, handle_virtblk_output);
+
+ /* Allocate the room for our own bookkeeping */
+ vblk = dev->priv = malloc(sizeof(*vblk));
+
+ /* First we open the file and store the length. */
+ vblk->fd = open_or_die(filename, O_RDWR|O_LARGEFILE);
+ vblk->len = lseek64(vblk->fd, 0, SEEK_END);
+
+ /* Tell Guest how many sectors this device has. */
+ cap = cpu_to_le64(vblk->len / 512);
+ add_desc_field(dev, VIRTIO_CONFIG_BLK_F_CAPACITY, sizeof(cap), &cap);
+
+ /* Tell Guest not to put in too many descriptors at once: two are used
+ * for the in and out elements. */
+ val = cpu_to_le32(VIRTQUEUE_NUM - 2);
+ add_desc_field(dev, VIRTIO_CONFIG_BLK_F_SEG_MAX, sizeof(val), &val);
+
+ /* The I/O thread writes to this end of the pipe when done. */
+ vblk->done_fd = p[1];
+
+ /* This is how we tell the I/O thread about more work. */
+ pipe(vblk->workpipe);
+
+ /* Create stack for thread and run it */
+ stack = malloc(32768);
+ if (clone(io_thread, stack + 32768, CLONE_VM, dev) == -1)
+ err(1, "Creating clone");
+
+ /* We don't need to keep the I/O thread's end of the pipes open. */
+ close(vblk->done_fd);
+ close(vblk->workpipe[0]);
+
+ verbose("device %u: virtblock %llu sectors\n",
+ devices.device_num, cap);
+}
/* That's the end of device setup. */
/*L:220 Finally we reach the core of the Launcher, which runs the Guest, serves
* its input and output, and finally, lays it to rest. */
-static void __attribute__((noreturn))
-run_guest(int lguest_fd, struct device_list *device_list)
+static void __attribute__((noreturn)) run_guest(int lguest_fd)
{
for (;;) {
unsigned long args[] = { LHREQ_BREAK, 0 };
- unsigned long arr[2];
+ unsigned long notify_addr;
int readval;
/* We read from the /dev/lguest device to run the Guest. */
- readval = read(lguest_fd, arr, sizeof(arr));
-
- /* The read can only really return sizeof(arr) (the Guest did a
- * SEND_DMA to us), or an error. */
-
- /* For a successful read, arr[0] is the address of the "struct
- * lguest_dma", and arr[1] is the key the Guest sent to. */
- if (readval == sizeof(arr)) {
- handle_output(lguest_fd, arr[0], arr[1], device_list);
+ readval = read(lguest_fd, ¬ify_addr, sizeof(notify_addr));
+
+ /* One unsigned long means the Guest did HCALL_NOTIFY */
+ if (readval == sizeof(notify_addr)) {
+ verbose("Notify on address %#lx\n", notify_addr);
+ handle_output(lguest_fd, notify_addr);
continue;
/* ENOENT means the Guest died. Reading tells us why. */
} else if (errno == ENOENT) {
@@ -1322,7 +1546,7 @@ run_guest(int lguest_fd, struct device_l
/* Service input, then unset the BREAK which releases
* the Waker. */
- handle_input(lguest_fd, device_list);
+ handle_input(lguest_fd);
if (write(lguest_fd, args, sizeof(args)) < 0)
err(1, "Resetting break");
}
@@ -1336,7 +1560,6 @@ run_guest(int lguest_fd, struct device_l
static struct option opts[] = {
{ "verbose", 0, NULL, 'v' },
- { "sharenet", 1, NULL, 's' },
{ "tunnet", 1, NULL, 't' },
{ "block", 1, NULL, 'b' },
{ "initrd", 1, NULL, 'i' },
@@ -1345,7 +1568,7 @@ static void usage(void)
static void usage(void)
{
errx(1, "Usage: lguest [--verbose] "
-
"[--sharenet=<filename>|--tunnet=(<ipaddr>|bridge:<bridgename>)\n"
+ "[--tunnet=(<ipaddr>|bridge:<bridgename>)\n"
"|--block=<filename>|--initrd=<filename>]...\n"
"<mem-in-mb> vmlinux [args...]");
}
@@ -1358,8 +1581,6 @@ int main(int argc, char *argv[])
unsigned long mem = 0, pgdir, start, initrd_size = 0;
/* A temporary and the /dev/lguest file descriptor. */
int i, c, lguest_fd;
- /* The list of Guest devices, based on command line arguments. */
- struct device_list device_list;
/* The boot information for the Guest. */
void *boot;
/* If they specify an initrd file to load. */
@@ -1369,11 +1590,12 @@ int main(int argc, char *argv[])
* device receive input from a file descriptor, we keep an fdset
* (infds) and the maximum fd number (max_infd) with the head of the
* list. We also keep a pointer to the last device, for easy appending
- * to the list. */
- device_list.max_infd = -1;
- device_list.dev = NULL;
- device_list.lastdev = &device_list.dev;
- FD_ZERO(&device_list.infds);
+ * to the list. Finally, we keep the next interrupt number to hand out
+ * (1: remember that 0 is used by the timer). */
+ FD_ZERO(&devices.infds);
+ devices.max_infd = -1;
+ devices.lastdev = &devices.dev;
+ devices.next_irq = 1;
/* We need to know how much memory so we can set up the device
* descriptor and memory pages for the devices as we parse the command
@@ -1390,7 +1612,7 @@ int main(int argc, char *argv[])
+ DEVICE_PAGES);
guest_limit = mem;
guest_max = mem + DEVICE_PAGES*getpagesize();
- device_list.descs = get_pages(1);
+ devices.descpage = get_pages(1);
break;
}
}
@@ -1401,14 +1623,11 @@ int main(int argc, char *argv[])
case 'v':
verbose = true;
break;
- case 's':
- setup_net_file(optarg, &device_list);
- break;
case 't':
- setup_tun_net(optarg, &device_list);
+ setup_tun_net(optarg);
break;
case 'b':
- setup_block_file(optarg, &device_list);
+ setup_block_file(optarg);
break;
case 'i':
initrd_name = optarg;
@@ -1426,7 +1645,7 @@ int main(int argc, char *argv[])
verbose("Guest base is at %p\n", guest_base);
/* We always have a console device */
- setup_console(&device_list);
+ setup_console();
/* Now we load the kernel */
start = load_kernel(open_or_die(argv[optind+1], O_RDONLY));
@@ -1468,10 +1687,10 @@ int main(int argc, char *argv[])
/* We fork off a child process, which wakes the Launcher whenever one
* of the input file descriptors needs attention. Otherwise we would
* run the Guest until it tries to output something. */
- waker_fd = setup_waker(lguest_fd, &device_list);
+ waker_fd = setup_waker(lguest_fd);
/* Finally, run the Guest. This doesn't return. */
- run_guest(lguest_fd, &device_list);
+ run_guest(lguest_fd);
}
/*:*/
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:07 UTC
[patch 36/43] lguest: Module autoprobing support for virtio drivers.
This adds the logic to convert the virtio ids into module aliases, and
includes a modalias entry in sysfs and the env var to make probing work.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
drivers/virtio/virtio.c | 26 ++++++++++++++++++++++++++
scripts/mod/file2alias.c | 18 ++++++++++++++++++
2 files changed, 44 insertions(+)
==================================================================---
a/drivers/virtio/virtio.c
+++ b/drivers/virtio/virtio.c
@@ -20,10 +20,19 @@ static ssize_t status_show(struct device
struct virtio_device *dev = container_of(_d,struct virtio_device,dev);
return sprintf(buf, "0x%08x", dev->config->get_status(dev));
}
+static ssize_t modalias_show(struct device *_d,
+ struct device_attribute *attr, char *buf)
+{
+ struct virtio_device *dev = container_of(_d,struct virtio_device,dev);
+
+ return sprintf(buf, "virtio:d%08Xv%08X\n",
+ dev->id.device, dev->id.vendor);
+}
static struct device_attribute virtio_dev_attrs[] = {
__ATTR_RO(device),
__ATTR_RO(vendor),
__ATTR_RO(status),
+ __ATTR_RO(modalias),
__ATTR_NULL
};
@@ -51,10 +60,27 @@ static int virtio_dev_match(struct devic
return 0;
}
+static int virtio_uevent(struct device *_dv, char **envp,
+ int num_envp, char *buffer, int buffer_size)
+{
+ struct virtio_device *dev = container_of(_dv,struct virtio_device,dev);
+ int i = 0, err;
+ int length = 0;
+
+ err = add_uevent_var(envp, num_envp, &i, buffer, buffer_size, &length,
+ "MODALIAS=virtio:d%08Xv%08X",
+ dev->id.device, dev->id.vendor);
+ if (err)
+ return err;
+ envp[i] = NULL;
+ return 0;
+}
+
static struct bus_type virtio_bus = {
.name = "virtio",
.match = virtio_dev_match,
.dev_attrs = virtio_dev_attrs,
+ .uevent = virtio_uevent,
};
static void add_status(struct virtio_device *dev, unsigned status)
==================================================================---
a/scripts/mod/file2alias.c
+++ b/scripts/mod/file2alias.c
@@ -484,6 +484,20 @@ static int do_parisc_entry(const char *f
return 1;
}
+/* Looks like: virtio:dNvN */
+static int do_virtio_entry(const char *filename, struct virtio_device_id *id,
+ char *alias)
+{
+ id->device = TO_NATIVE(id->device);
+ id->vendor = TO_NATIVE(id->vendor);
+
+ strcpy(alias, "virtio:");
+ ADD(alias, "d", 1, id->device);
+ ADD(alias, "v", id->vendor != VIRTIO_DEV_ANY_ID, id->vendor);
+
+ return 1;
+}
+
/* Ignore any prefix, eg. v850 prepends _ */
static inline int sym_is(const char *symbol, const char *name)
{
@@ -599,6 +613,10 @@ void handle_moddevtable(struct module *m
do_table(symval, sym->st_size,
sizeof(struct parisc_device_id), "parisc",
do_parisc_entry, mod);
+ else if (sym_is(symname, "__mod_virtio_device_table"))
+ do_table(symval, sym->st_size,
+ sizeof(struct virtio_device_id), "virtio",
+ do_virtio_entry, mod);
}
/* Now add out buffered information to the generated C source */
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:07 UTC
[patch 23/43] lguest: Make hypercalls arch-independent.
Clean up the hypercall code to make the code in hypercalls.c
architecture independent. First process the common hypercalls and
then call lguest_arch_do_hcall() if the call hasn't been handled.
Rename struct hcall_ring to hcall_args.
This patch requires the previous patch which reorganize the layout of
struct lguest_regs on i386 so they match the layout of struct
hcall_args.
Signed-off-by: Jes Sorensen <jes@sgi.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
arch/i386/lguest/boot.c | 8 +-
drivers/lguest/hypercalls.c | 120 ++++++++++++---------------------------
drivers/lguest/i386_core.c | 62 +++++++++++++++++++-
drivers/lguest/lg.h | 4 -
include/asm-i386/lguest_hcall.h | 8 +-
include/linux/lguest.h | 2
6 files changed, 112 insertions(+), 92 deletions(-)
diff -r 5da9ffae9f41 arch/i386/lguest/boot.c
--- a/arch/i386/lguest/boot.c Wed Sep 26 14:47:38 2007 +1000
+++ b/arch/i386/lguest/boot.c Wed Sep 26 14:48:14 2007 +1000
@@ -158,10 +158,10 @@ void async_hcall(unsigned long call,
/* Table full, so do normal hcall which will flush table. */
hcall(call, arg1, arg2, arg3);
} else {
- lguest_data.hcalls[next_call].eax = call;
- lguest_data.hcalls[next_call].edx = arg1;
- lguest_data.hcalls[next_call].ebx = arg2;
- lguest_data.hcalls[next_call].ecx = arg3;
+ lguest_data.hcalls[next_call].arg0 = call;
+ lguest_data.hcalls[next_call].arg1 = arg1;
+ lguest_data.hcalls[next_call].arg2 = arg2;
+ lguest_data.hcalls[next_call].arg3 = arg3;
/* Arguments must all be written before we mark it to go */
wmb();
lguest_data.hcall_status[next_call] = 0;
diff -r 5da9ffae9f41 drivers/lguest/hypercalls.c
--- a/drivers/lguest/hypercalls.c Wed Sep 26 14:47:38 2007 +1000
+++ b/drivers/lguest/hypercalls.c Wed Sep 26 14:48:14 2007 +1000
@@ -25,17 +25,13 @@
#include <linux/mm.h>
#include <asm/page.h>
#include <asm/pgtable.h>
-#include <irq_vectors.h>
#include "lg.h"
-/*H:120 This is the core hypercall routine: where the Guest gets what it
- * wants. Or gets killed. Or, in the case of LHCALL_CRASH, both.
- *
- * Remember from the Guest: %eax == which call to make, and the arguments are
- * packed into %edx, %ebx and %ecx if needed. */
-static void do_hcall(struct lguest *lg, struct lguest_regs *regs)
-{
- switch (regs->eax) {
+/*H:120 This is the core hypercall routine: where the Guest gets what it wants.
+ * Or gets killed. Or, in the case of LHCALL_CRASH, both. */
+static void do_hcall(struct lguest *lg, struct hcall_args *args)
+{
+ switch (args->arg0) {
case LHCALL_FLUSH_ASYNC:
/* This call does nothing, except by breaking out of the Guest
* it makes us process all the asynchronous hypercalls. */
@@ -51,7 +47,7 @@ static void do_hcall(struct lguest *lg,
char msg[128];
/* If the lgread fails, it will call kill_guest() itself; the
* kill_guest() with the message will be ignored. */
- lgread(lg, msg, regs->edx, sizeof(msg));
+ lgread(lg, msg, args->arg1, sizeof(msg));
msg[sizeof(msg)-1] = '\0';
kill_guest(lg, "CRASH: %s", msg);
break;
@@ -59,7 +55,7 @@ static void do_hcall(struct lguest *lg,
case LHCALL_FLUSH_TLB:
/* FLUSH_TLB comes in two flavors, depending on the
* argument: */
- if (regs->edx)
+ if (args->arg1)
guest_pagetable_clear_all(lg);
else
guest_pagetable_flush_user(lg);
@@ -71,55 +67,47 @@ static void do_hcall(struct lguest *lg,
* it here. This can legitimately fail, since we currently
* place a limit on the number of DMA pools a Guest can have.
* So we return true or false from this call. */
- regs->eax = bind_dma(lg, regs->edx, regs->ebx,
- regs->ecx >> 8, regs->ecx & 0xFF);
+ args->arg0 = bind_dma(lg, args->arg1, args->arg2,
+ args->arg3 >> 8, args->arg3 & 0xFF);
break;
/* All these calls simply pass the arguments through to the right
* routines. */
case LHCALL_SEND_DMA:
- send_dma(lg, regs->edx, regs->ebx);
- break;
- case LHCALL_LOAD_GDT:
- load_guest_gdt(lg, regs->edx, regs->ebx);
- break;
- case LHCALL_LOAD_IDT_ENTRY:
- load_guest_idt_entry(lg, regs->edx, regs->ebx, regs->ecx);
+ send_dma(lg, args->arg1, args->arg2);
break;
case LHCALL_NEW_PGTABLE:
- guest_new_pagetable(lg, regs->edx);
+ guest_new_pagetable(lg, args->arg1);
break;
case LHCALL_SET_STACK:
- guest_set_stack(lg, regs->edx, regs->ebx, regs->ecx);
+ guest_set_stack(lg, args->arg1, args->arg2, args->arg3);
break;
case LHCALL_SET_PTE:
- guest_set_pte(lg, regs->edx, regs->ebx, mkgpte(regs->ecx));
+ guest_set_pte(lg, args->arg1, args->arg2, mkgpte(args->arg3));
break;
case LHCALL_SET_PMD:
- guest_set_pmd(lg, regs->edx, regs->ebx);
- break;
- case LHCALL_LOAD_TLS:
- guest_load_tls(lg, regs->edx);
+ guest_set_pmd(lg, args->arg1, args->arg2);
break;
case LHCALL_SET_CLOCKEVENT:
- guest_set_clockevent(lg, regs->edx);
- break;
-
+ guest_set_clockevent(lg, args->arg1);
+ break;
case LHCALL_TS:
/* This sets the TS flag, as we saw used in run_guest(). */
- lg->ts = regs->edx;
+ lg->ts = args->arg1;
break;
case LHCALL_HALT:
/* Similarly, this sets the halted flag for run_guest(). */
lg->halted = 1;
break;
default:
- kill_guest(lg, "Bad hypercall %li\n", regs->eax);
- }
-}
-
-/* Asynchronous hypercalls are easy: we just look in the array in the
Guest's
- * "struct lguest_data" and see if there are any new ones marked
"ready".
+ if (lguest_arch_do_hcall(lg, args))
+ kill_guest(lg, "Bad hypercall %li\n", args->arg0);
+ }
+}
+/*:*/
+
+/*H:124 Asynchronous hypercalls are easy: we just look in the array in the
+ * Guest's "struct lguest_data" to see if any new ones are marked
"ready".
*
* We are careful to do these in order: obviously we respect the order the
* Guest put them in the ring, but we also promise the Guest that they will
@@ -134,10 +122,9 @@ static void do_async_hcalls(struct lgues
if (copy_from_user(&st, &lg->lguest_data->hcall_status,
sizeof(st)))
return;
-
/* We process "struct lguest_data"s hcalls[] ring once. */
for (i = 0; i < ARRAY_SIZE(st); i++) {
- struct lguest_regs regs;
+ struct hcall_args args;
/* We remember where we were up to from last time. This makes
* sure that the hypercalls are done in the order the Guest
* places them in the ring. */
@@ -152,18 +139,16 @@ static void do_async_hcalls(struct lgues
if (++lg->next_hcall == LHCALL_RING_SIZE)
lg->next_hcall = 0;
- /* We copy the hypercall arguments into a fake register
- * structure. This makes life simple for do_hcall(). */
- if (get_user(regs.eax, &lg->lguest_data->hcalls[n].eax)
- || get_user(regs.edx, &lg->lguest_data->hcalls[n].edx)
- || get_user(regs.ecx, &lg->lguest_data->hcalls[n].ecx)
- || get_user(regs.ebx, &lg->lguest_data->hcalls[n].ebx)) {
+ /* Copy the hypercall arguments into a local copy of
+ * the hcall_args struct. */
+ if (copy_from_user(&args, &lg->lguest_data->hcalls[n],
+ sizeof(struct hcall_args))) {
kill_guest(lg, "Fetching async hypercalls");
break;
}
/* Do the hypercall, same as a normal one. */
- do_hcall(lg, ®s);
+ do_hcall(lg, &args);
/* Mark the hypercall done. */
if (put_user(0xFF, &lg->lguest_data->hcall_status[n])) {
@@ -182,41 +167,16 @@ static void do_async_hcalls(struct lgues
* Guest makes a hypercall, we end up here to set things up: */
static void initialize(struct lguest *lg)
{
- u32 tsc_speed;
/* You can't do anything until you're initialized. The Guest knows
the
* rules, so we're unforgiving here. */
- if (lg->regs->eax != LHCALL_LGUEST_INIT) {
- kill_guest(lg, "hypercall %li before LGUEST_INIT",
- lg->regs->eax);
- return;
- }
-
- /* We insist that the Time Stamp Counter exist and doesn't change with
- * cpu frequency. Some devious chip manufacturers decided that TSC
- * changes could be handled in software. I decided that time going
- * backwards might be good for benchmarks, but it's bad for users.
- *
- * We also insist that the TSC be stable: the kernel detects unreliable
- * TSCs for its own purposes, and we use that here. */
- if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC) && !check_tsc_unstable())
- tsc_speed = tsc_khz;
- else
- tsc_speed = 0;
-
- /* The pointer to the Guest's "struct lguest_data" is the only
- * argument. We check that address now. */
- if (!lguest_address_ok(lg, lg->regs->edx, sizeof(*lg->lguest_data)))
{
+ if (lg->hcall->arg0 != LHCALL_LGUEST_INIT) {
+ kill_guest(lg, "hypercall %li before INIT", lg->hcall->arg0);
+ return;
+ }
+
+ if (lguest_arch_init_hypercalls(lg))
kill_guest(lg, "bad guest page %p", lg->lguest_data);
- return;
- }
-
- /* Having checked it, we simply set lg->lguest_data to point straight
- * into the Launcher's memory at the right place and then use
- * copy_to_user/from_user from now on, instead of lgread/write. I put
- * this in to show that I'm not immune to writing stupid
- * optimizations. */
- lg->lguest_data = lg->mem_base + lg->regs->edx;
/* The Guest tells us where we're not to deliver interrupts by putting
* the range of addresses into "struct lguest_data". */
@@ -224,8 +184,7 @@ static void initialize(struct lguest *lg
|| get_user(lg->noirq_end, &lg->lguest_data->noirq_end)
/* We tell the Guest that it can't use the top 4MB of virtual
* addresses used by the Switcher. */
- || put_user(4U*1024*1024, &lg->lguest_data->reserve_mem)
- || put_user(tsc_speed, &lg->lguest_data->tsc_khz))
+ || put_user(4U*1024*1024, &lg->lguest_data->reserve_mem))
kill_guest(lg, "bad guest page %p", lg->lguest_data);
/* We write the current time into the Guest's data page once now. */
@@ -237,9 +196,6 @@ static void initialize(struct lguest *lg
* page. */
guest_pagetable_clear_all(lg);
}
-/* Now we've examined the hypercall code; our Guest can make requests.
There
- * is one other way we can do things for the Guest, as we see in
- * emulate_insn(). */
/*H:100
* Hypercalls
diff -r 5da9ffae9f41 drivers/lguest/i386_core.c
--- a/drivers/lguest/i386_core.c Wed Sep 26 14:47:38 2007 +1000
+++ b/drivers/lguest/i386_core.c Wed Sep 26 14:48:14 2007 +1000
@@ -323,7 +323,9 @@ void lguest_arch_handle_trap(struct lgue
cond_resched();
return;
case LGUEST_TRAP_ENTRY:
- lg->hcall = lg->regs;
+ /* Our 'struct hcall_args' maps directly over our regs: we set
+ * up the pointer now to indicate a hypercall is pending. */
+ lg->hcall = (struct hcall_args *)lg->regs;
return;
}
@@ -475,3 +477,61 @@ void __exit lguest_arch_host_fini(void)
}
unlock_cpu_hotplug();
}
+
+
+/*H:122 The i386-specific hypercalls simply farm out to the right functions. */
+int lguest_arch_do_hcall(struct lguest *lg, struct hcall_args *args)
+{
+ switch (args->arg0) {
+ case LHCALL_LOAD_GDT:
+ load_guest_gdt(lg, args->arg1, args->arg2);
+ break;
+ case LHCALL_LOAD_IDT_ENTRY:
+ load_guest_idt_entry(lg, args->arg1, args->arg2, args->arg3);
+ break;
+ case LHCALL_LOAD_TLS:
+ guest_load_tls(lg, args->arg1);
+ break;
+ default:
+ /* Bad Guest. Bad! */
+ return -EIO;
+ }
+ return 0;
+}
+
+/*H:126 i386-specific hypercall initialization: */
+int lguest_arch_init_hypercalls(struct lguest *lg)
+{
+ u32 tsc_speed;
+
+ /* The pointer to the Guest's "struct lguest_data" is the only
+ * argument. We check that address now. */
+ if (!lguest_address_ok(lg, lg->hcall->arg1,
sizeof(*lg->lguest_data)))
+ return -EFAULT;
+
+ /* Having checked it, we simply set lg->lguest_data to point straight
+ * into the Launcher's memory at the right place and then use
+ * copy_to_user/from_user from now on, instead of lgread/write. I put
+ * this in to show that I'm not immune to writing stupid
+ * optimizations. */
+ lg->lguest_data = lg->mem_base + lg->hcall->arg1;
+
+ /* We insist that the Time Stamp Counter exist and doesn't change with
+ * cpu frequency. Some devious chip manufacturers decided that TSC
+ * changes could be handled in software. I decided that time going
+ * backwards might be good for benchmarks, but it's bad for users.
+ *
+ * We also insist that the TSC be stable: the kernel detects unreliable
+ * TSCs for its own purposes, and we use that here. */
+ if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC) && !check_tsc_unstable())
+ tsc_speed = tsc_khz;
+ else
+ tsc_speed = 0;
+ if (put_user(tsc_speed, &lg->lguest_data->tsc_khz))
+ return -EFAULT;
+
+ return 0;
+}
+/* Now we've examined the hypercall code; our Guest can make requests.
There
+ * is one other way we can do things for the Guest, as we see in
+ * emulate_insn(). :*/
diff -r 5da9ffae9f41 drivers/lguest/lg.h
--- a/drivers/lguest/lg.h Wed Sep 26 14:47:38 2007 +1000
+++ b/drivers/lguest/lg.h Wed Sep 26 14:48:14 2007 +1000
@@ -108,7 +108,7 @@ struct lguest
u8 ss1;
/* If a hypercall was asked for, this points to the arguments. */
- struct lguest_regs *hcall;
+ struct hcall_args *hcall;
/* Do we need to stop what we're doing and return to userspace? */
int break_out;
@@ -198,6 +198,8 @@ void lguest_arch_host_fini(void);
void lguest_arch_host_fini(void);
void lguest_arch_run_guest(struct lguest *lg);
void lguest_arch_handle_trap(struct lguest *lg);
+int lguest_arch_init_hypercalls(struct lguest *lg);
+int lguest_arch_do_hcall(struct lguest *lg, struct hcall_args *args);
/* <arch>_switcher.S: */
extern char start_switcher_text[], end_switcher_text[], switch_to_guest[];
diff -r 5da9ffae9f41 include/asm-i386/lguest_hcall.h
--- a/include/asm-i386/lguest_hcall.h Wed Sep 26 14:47:38 2007 +1000
+++ b/include/asm-i386/lguest_hcall.h Wed Sep 26 14:48:14 2007 +1000
@@ -1,6 +1,8 @@
/* Architecture specific portion of the lguest hypercalls */
#ifndef _I386_LGUEST_HCALL_H
#define _I386_LGUEST_HCALL_H
+
+#include <irq_vectors.h>
#define LHCALL_FLUSH_ASYNC 0
#define LHCALL_LGUEST_INIT 1
@@ -59,9 +61,9 @@ void async_hcall(unsigned long call,
#define LGUEST_IRQS (NR_IRQS < 32 ? NR_IRQS: 32)
#define LHCALL_RING_SIZE 64
-struct hcall_ring
+struct hcall_args
{
- u32 eax, edx, ebx, ecx;
+ /* These map directly onto eax, ebx, ecx, edx in struct lguest_regs */
+ unsigned long arg0, arg2, arg3, arg1;
};
-
#endif /* _I386_LGUEST_HCALL_H */
diff -r 5da9ffae9f41 include/linux/lguest.h
--- a/include/linux/lguest.h Wed Sep 26 14:47:38 2007 +1000
+++ b/include/linux/lguest.h Wed Sep 26 14:48:14 2007 +1000
@@ -36,7 +36,7 @@ struct lguest_data
/* 0xFF == done (set by Host), 0 == pending (set by Guest). */
u8 hcall_status[LHCALL_RING_SIZE];
/* The actual registers for the hypercalls. */
- struct hcall_ring hcalls[LHCALL_RING_SIZE];
+ struct hcall_args hcalls[LHCALL_RING_SIZE];
/* Fields initialized by the Host at boot: */
/* Memory not to try to access */
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:07 UTC
[patch 26/43] lguest: guest.h declares a struct timespec, make it include linux/time.h
Signed-off-by: Jes Sorensen <jes@sgi.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> --- include/linux/lguest.h | 1 + 1 file changed, 1 insertion(+) ==================================================================--- linux-2.6.23-rc4.orig/include/linux/lguest.h +++ linux-2.6.23-rc4/include/linux/lguest.h @@ -4,6 +4,7 @@ #define _LINUX_LGUEST_H #ifndef __ASSEMBLY__ +#include <linux/time.h> #include <asm/irq.h> #include <asm/lguest_hcall.h> -- there are those who do and those who hang on and you don't see too many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:07 UTC
[patch 14/43] lguest: Move lguest guest support to arch/i386 where it logically belongs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: Andi Kleen <ak@suse.de>
---
arch/i386/Kconfig | 10
arch/i386/Makefile | 3
arch/i386/lguest/Kconfig | 8
arch/i386/lguest/Makefile | 1
arch/i386/lguest/boot.c | 1101 +++++++++++++++++++++++++++++++++++++++++++
arch/i386/lguest/head.S | 93 +++
drivers/lguest/Makefile | 4
drivers/lguest/lguest.c | 1101 -------------------------------------------
drivers/lguest/lguest_asm.S | 93 ---
9 files changed, 1210 insertions(+), 1204 deletions(-)
==================================================================---
a/arch/i386/Kconfig
+++ b/arch/i386/Kconfig
@@ -245,14 +245,8 @@ config VMI
at the moment), by linking the kernel to a GPL-ed ROM module
provided by the hypervisor.
-config LGUEST_GUEST
- bool "Lguest guest support"
- select PARAVIRT
- depends on !X86_PAE
- help
- Lguest is a tiny in-kernel hypervisor. Selecting this will
- allow your kernel to boot under lguest. This option will increase
- your kernel size by about 6k. If in doubt, say N.
+source "arch/i386/lguest/Kconfig"
+
endif
config ACPI_SRAT
==================================================================---
a/arch/i386/Makefile
+++ b/arch/i386/Makefile
@@ -96,6 +96,9 @@ core-$(CONFIG_X86_ES7000) := arch/i386/m
# Xen paravirtualization support
core-$(CONFIG_XEN) += arch/i386/xen/
+# lguest paravirtualization support
+core-$(CONFIG_LGUEST_GUEST) += arch/i386/lguest/
+
# default subarch .h files
mflags-y += -Iinclude/asm-i386/mach-default
==================================================================--- /dev/null
+++ b/arch/i386/lguest/Kconfig
@@ -0,0 +1,8 @@
+config LGUEST_GUEST
+ bool "Lguest guest support"
+ select PARAVIRT
+ depends on !X86_PAE
+ help
+ Lguest is a tiny in-kernel hypervisor. Selecting this will
+ allow your kernel to boot under lguest. This option will increase
+ your kernel size by about 6k. If in doubt, say N.
==================================================================--- /dev/null
+++ b/arch/i386/lguest/Makefile
@@ -0,0 +1,1 @@
+obj-y := head.o boot.o
==================================================================--- /dev/null
+++ b/arch/i386/lguest/boot.c
@@ -0,0 +1,1101 @@
+/*P:010
+ * A hypervisor allows multiple Operating Systems to run on a single machine.
+ * To quote David Wheeler: "Any problem in computer science can be solved
with
+ * another layer of indirection."
+ *
+ * We keep things simple in two ways. First, we start with a normal Linux
+ * kernel and insert a module (lg.ko) which allows us to run other Linux
+ * kernels the same way we'd run processes. We call the first kernel the
Host,
+ * and the others the Guests. The program which sets up and configures Guests
+ * (such as the example in Documentation/lguest/lguest.c) is called the
+ * Launcher.
+ *
+ * Secondly, we only run specially modified Guests, not normal kernels. When
+ * you set CONFIG_LGUEST to 'y' or 'm', this automatically sets
+ * CONFIG_LGUEST_GUEST=y, which compiles this file into the kernel so it knows
+ * how to be a Guest. This means that you can use the same kernel you boot
+ * normally (ie. as a Host) as a Guest.
+ *
+ * These Guests know that they cannot do privileged operations, such as disable
+ * interrupts, and that they have to ask the Host to do such things explicitly.
+ * This file consists of all the replacements for such low-level native
+ * hardware operations: these special Guest versions call the Host.
+ *
+ * So how does the kernel know it's a Guest? The Guest starts at a special
+ * entry point marked with a magic string, which sets up a few things then
+ * calls here. We replace the native functions in "struct
paravirt_ops"
+ * with our Guest versions, then boot like normal. :*/
+
+/*
+ * Copyright (C) 2006, Rusty Russell <rusty@rustcorp.com.au> IBM
Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or
+ * NON INFRINGEMENT. See the GNU General Public License for more
+ * details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+#include <linux/kernel.h>
+#include <linux/start_kernel.h>
+#include <linux/string.h>
+#include <linux/console.h>
+#include <linux/screen_info.h>
+#include <linux/irq.h>
+#include <linux/interrupt.h>
+#include <linux/clocksource.h>
+#include <linux/clockchips.h>
+#include <linux/lguest.h>
+#include <linux/lguest_launcher.h>
+#include <linux/lguest_bus.h>
+#include <asm/paravirt.h>
+#include <asm/param.h>
+#include <asm/page.h>
+#include <asm/pgtable.h>
+#include <asm/desc.h>
+#include <asm/setup.h>
+#include <asm/e820.h>
+#include <asm/mce.h>
+#include <asm/io.h>
+
+/*G:010 Welcome to the Guest!
+ *
+ * The Guest in our tale is a simple creature: identical to the Host but
+ * behaving in simplified but equivalent ways. In particular, the Guest is the
+ * same kernel as the Host (or at least, built from the same source code). :*/
+
+/* Declarations for definitions in lguest_guest.S */
+extern char lguest_noirq_start[], lguest_noirq_end[];
+extern const char lgstart_cli[], lgend_cli[];
+extern const char lgstart_sti[], lgend_sti[];
+extern const char lgstart_popf[], lgend_popf[];
+extern const char lgstart_pushf[], lgend_pushf[];
+extern const char lgstart_iret[], lgend_iret[];
+extern void lguest_iret(void);
+
+struct lguest_data lguest_data = {
+ .hcall_status = { [0 ... LHCALL_RING_SIZE-1] = 0xFF },
+ .noirq_start = (u32)lguest_noirq_start,
+ .noirq_end = (u32)lguest_noirq_end,
+ .blocked_interrupts = { 1 }, /* Block timer interrupts */
+};
+static cycle_t clock_base;
+
+/*G:035 Notice the lazy_hcall() above, rather than hcall(). This is our first
+ * real optimization trick!
+ *
+ * When lazy_mode is set, it means we're allowed to defer all hypercalls
and do
+ * them as a batch when lazy_mode is eventually turned off. Because hypercalls
+ * are reasonably expensive, batching them up makes sense. For example, a
+ * large mmap might update dozens of page table entries: that code calls
+ * lguest_lazy_mode(PARAVIRT_LAZY_MMU), does the dozen updates, then calls
+ * lguest_lazy_mode(PARAVIRT_LAZY_NONE).
+ *
+ * So, when we're in lazy mode, we call async_hypercall() to store the call
for
+ * future processing. When lazy mode is turned off we issue a hypercall to
+ * flush the stored calls.
+ *
+ * There's also a hack where "mode" is set to
"PARAVIRT_LAZY_FLUSH" which
+ * indicates we're to flush any outstanding calls immediately. This is
used
+ * when an interrupt handler does a kmap_atomic(): the page table changes must
+ * happen immediately even if we're in the middle of a batch. Usually
we're
+ * not, though, so there's nothing to do. */
+static enum paravirt_lazy_mode lazy_mode; /* Note: not SMP-safe! */
+static void lguest_lazy_mode(enum paravirt_lazy_mode mode)
+{
+ if (mode == PARAVIRT_LAZY_FLUSH) {
+ if (unlikely(lazy_mode != PARAVIRT_LAZY_NONE))
+ hcall(LHCALL_FLUSH_ASYNC, 0, 0, 0);
+ } else {
+ lazy_mode = mode;
+ if (mode == PARAVIRT_LAZY_NONE)
+ hcall(LHCALL_FLUSH_ASYNC, 0, 0, 0);
+ }
+}
+
+static void lazy_hcall(unsigned long call,
+ unsigned long arg1,
+ unsigned long arg2,
+ unsigned long arg3)
+{
+ if (lazy_mode == PARAVIRT_LAZY_NONE)
+ hcall(call, arg1, arg2, arg3);
+ else
+ async_hcall(call, arg1, arg2, arg3);
+}
+
+/* async_hcall() is pretty simple: I'm quite proud of it really. We have a
+ * ring buffer of stored hypercalls which the Host will run though next time we
+ * do a normal hypercall. Each entry in the ring has 4 slots for the hypercall
+ * arguments, and a "hcall_status" word which is 0 if the call is
ready to go,
+ * and 255 once the Host has finished with it.
+ *
+ * If we come around to a slot which hasn't been finished, then the table
is
+ * full and we just make the hypercall directly. This has the nice side
+ * effect of causing the Host to run all the stored calls in the ring buffer
+ * which empties it for next time! */
+void async_hcall(unsigned long call,
+ unsigned long arg1, unsigned long arg2, unsigned long arg3)
+{
+ /* Note: This code assumes we're uniprocessor. */
+ static unsigned int next_call;
+ unsigned long flags;
+
+ /* Disable interrupts if not already disabled: we don't want an
+ * interrupt handler making a hypercall while we're already doing
+ * one! */
+ local_irq_save(flags);
+ if (lguest_data.hcall_status[next_call] != 0xFF) {
+ /* Table full, so do normal hcall which will flush table. */
+ hcall(call, arg1, arg2, arg3);
+ } else {
+ lguest_data.hcalls[next_call].eax = call;
+ lguest_data.hcalls[next_call].edx = arg1;
+ lguest_data.hcalls[next_call].ebx = arg2;
+ lguest_data.hcalls[next_call].ecx = arg3;
+ /* Arguments must all be written before we mark it to go */
+ wmb();
+ lguest_data.hcall_status[next_call] = 0;
+ if (++next_call == LHCALL_RING_SIZE)
+ next_call = 0;
+ }
+ local_irq_restore(flags);
+}
+/*:*/
+
+/* Wrappers for the SEND_DMA and BIND_DMA hypercalls. This is mainly because
+ * Jeff Garzik complained that __pa() should never appear in drivers, and this
+ * helps remove most of them. But also, it wraps some ugliness. */
+void lguest_send_dma(unsigned long key, struct lguest_dma *dma)
+{
+ /* The hcall might not write this if something goes wrong */
+ dma->used_len = 0;
+ hcall(LHCALL_SEND_DMA, key, __pa(dma), 0);
+}
+
+int lguest_bind_dma(unsigned long key, struct lguest_dma *dmas,
+ unsigned int num, u8 irq)
+{
+ /* This is the only hypercall which actually wants 5 arguments, and we
+ * only support 4. Fortunately the interrupt number is always less
+ * than 256, so we can pack it with the number of dmas in the final
+ * argument. */
+ if (!hcall(LHCALL_BIND_DMA, key, __pa(dmas), (num << 8) | irq))
+ return -ENOMEM;
+ return 0;
+}
+
+/* Unbinding is the same hypercall as binding, but with 0 num & irq. */
+void lguest_unbind_dma(unsigned long key, struct lguest_dma *dmas)
+{
+ hcall(LHCALL_BIND_DMA, key, __pa(dmas), 0);
+}
+
+/* For guests, device memory can be used as normal memory, so we cast away the
+ * __iomem to quieten sparse. */
+void *lguest_map(unsigned long phys_addr, unsigned long pages)
+{
+ return (__force void *)ioremap(phys_addr, PAGE_SIZE*pages);
+}
+
+void lguest_unmap(void *addr)
+{
+ iounmap((__force void __iomem *)addr);
+}
+
+/*G:033
+ * Here are our first native-instruction replacements: four functions for
+ * interrupt control.
+ *
+ * The simplest way of implementing these would be to have "turn
interrupts
+ * off" and "turn interrupts on" hypercalls. Unfortunately,
this is too slow:
+ * these are by far the most commonly called functions of those we override.
+ *
+ * So instead we keep an "irq_enabled" field inside our "struct
lguest_data",
+ * which the Guest can update with a single instruction. The Host knows to
+ * check there when it wants to deliver an interrupt.
+ */
+
+/* save_flags() is expected to return the processor state (ie.
"eflags"). The
+ * eflags word contains all kind of stuff, but in practice Linux only cares
+ * about the interrupt flag. Our "save_flags()" just returns that.
*/
+static unsigned long save_fl(void)
+{
+ return lguest_data.irq_enabled;
+}
+
+/* "restore_flags" just sets the flags back to the value given. */
+static void restore_fl(unsigned long flags)
+{
+ lguest_data.irq_enabled = flags;
+}
+
+/* Interrupts go off... */
+static void irq_disable(void)
+{
+ lguest_data.irq_enabled = 0;
+}
+
+/* Interrupts go on... */
+static void irq_enable(void)
+{
+ lguest_data.irq_enabled = X86_EFLAGS_IF;
+}
+/*:*/
+/*M:003 Note that we don't check for outstanding interrupts when we
re-enable
+ * them (or when we unmask an interrupt). This seems to work for the moment,
+ * since interrupts are rare and we'll just get the interrupt on the next
timer
+ * tick, but when we turn on CONFIG_NO_HZ, we should revisit this. One way
+ * would be to put the "irq_enabled" field in a page by itself, and
have the
+ * Host write-protect it when an interrupt comes in when irqs are disabled.
+ * There will then be a page fault as soon as interrupts are re-enabled. :*/
+
+/*G:034
+ * The Interrupt Descriptor Table (IDT).
+ *
+ * The IDT tells the processor what to do when an interrupt comes in. Each
+ * entry in the table is a 64-bit descriptor: this holds the privilege level,
+ * address of the handler, and... well, who cares? The Guest just asks the
+ * Host to make the change anyway, because the Host controls the real IDT.
+ */
+static void lguest_write_idt_entry(struct desc_struct *dt,
+ int entrynum, u32 low, u32 high)
+{
+ /* Keep the local copy up to date. */
+ write_dt_entry(dt, entrynum, low, high);
+ /* Tell Host about this new entry. */
+ hcall(LHCALL_LOAD_IDT_ENTRY, entrynum, low, high);
+}
+
+/* Changing to a different IDT is very rare: we keep the IDT up-to-date every
+ * time it is written, so we can simply loop through all entries and tell the
+ * Host about them. */
+static void lguest_load_idt(const struct Xgt_desc_struct *desc)
+{
+ unsigned int i;
+ struct desc_struct *idt = (void *)desc->address;
+
+ for (i = 0; i < (desc->size+1)/8; i++)
+ hcall(LHCALL_LOAD_IDT_ENTRY, i, idt[i].a, idt[i].b);
+}
+
+/*
+ * The Global Descriptor Table.
+ *
+ * The Intel architecture defines another table, called the Global Descriptor
+ * Table (GDT). You tell the CPU where it is (and its size) using the
"lgdt"
+ * instruction, and then several other instructions refer to entries in the
+ * table. There are three entries which the Switcher needs, so the Host simply
+ * controls the entire thing and the Guest asks it to make changes using the
+ * LOAD_GDT hypercall.
+ *
+ * This is the opposite of the IDT code where we have a LOAD_IDT_ENTRY
+ * hypercall and use that repeatedly to load a new IDT. I don't think it
+ * really matters, but wouldn't it be nice if they were the same?
+ */
+static void lguest_load_gdt(const struct Xgt_desc_struct *desc)
+{
+ BUG_ON((desc->size+1)/8 != GDT_ENTRIES);
+ hcall(LHCALL_LOAD_GDT, __pa(desc->address), GDT_ENTRIES, 0);
+}
+
+/* For a single GDT entry which changes, we do the lazy thing: alter our GDT,
+ * then tell the Host to reload the entire thing. This operation is so rare
+ * that this naive implementation is reasonable. */
+static void lguest_write_gdt_entry(struct desc_struct *dt,
+ int entrynum, u32 low, u32 high)
+{
+ write_dt_entry(dt, entrynum, low, high);
+ hcall(LHCALL_LOAD_GDT, __pa(dt), GDT_ENTRIES, 0);
+}
+
+/* OK, I lied. There are three "thread local storage" GDT entries
which change
+ * on every context switch (these three entries are how glibc implements
+ * __thread variables). So we have a hypercall specifically for this case. */
+static void lguest_load_tls(struct thread_struct *t, unsigned int cpu)
+{
+ /* There's one problem which normal hardware doesn't have: the Host
+ * can't handle us removing entries we're currently using. So we
clear
+ * the GS register here: if it's needed it'll be reloaded anyway. */
+ loadsegment(gs, 0);
+ lazy_hcall(LHCALL_LOAD_TLS, __pa(&t->tls_array), cpu, 0);
+}
+
+/*G:038 That's enough excitement for now, back to ploughing through each of
+ * the paravirt_ops (we're about 1/3 of the way through).
+ *
+ * This is the Local Descriptor Table, another weird Intel thingy. Linux only
+ * uses this for some strange applications like Wine. We don't do anything
+ * here, so they'll get an informative and friendly Segmentation Fault. */
+static void lguest_set_ldt(const void *addr, unsigned entries)
+{
+}
+
+/* This loads a GDT entry into the "Task Register": that entry points
to a
+ * structure called the Task State Segment. Some comments scattered though the
+ * kernel code indicate that this used for task switching in ages past, along
+ * with blood sacrifice and astrology.
+ *
+ * Now there's nothing interesting in here that we don't get told
elsewhere.
+ * But the native version uses the "ltr" instruction, which makes the
Host
+ * complain to the Guest about a Segmentation Fault and it'll oops. So we
+ * override the native version with a do-nothing version. */
+static void lguest_load_tr_desc(void)
+{
+}
+
+/* The "cpuid" instruction is a way of querying both the CPU identity
+ * (manufacturer, model, etc) and its features. It was introduced before the
+ * Pentium in 1993 and keeps getting extended by both Intel and AMD. As you
+ * might imagine, after a decade and a half this treatment, it is now a giant
+ * ball of hair. Its entry in the current Intel manual runs to 28 pages.
+ *
+ * This instruction even it has its own Wikipedia entry. The Wikipedia entry
+ * has been translated into 4 languages. I am not making this up!
+ *
+ * We could get funky here and identify ourselves as "GenuineLguest",
but
+ * instead we just use the real "cpuid" instruction. Then I pretty
much turned
+ * off feature bits until the Guest booted. (Don't say that: you'll
damage
+ * lguest sales!) Shut up, inner voice! (Hey, just pointing out that this is
+ * hardly future proof.) Noone's listening! They don't like you
anyway,
+ * parenthetic weirdo!
+ *
+ * Replacing the cpuid so we can turn features off is great for the kernel, but
+ * anyone (including userspace) can just use the raw "cpuid"
instruction and
+ * the Host won't even notice since it isn't privileged. So we try not
to get
+ * too worked up about it. */
+static void lguest_cpuid(unsigned int *eax, unsigned int *ebx,
+ unsigned int *ecx, unsigned int *edx)
+{
+ int function = *eax;
+
+ native_cpuid(eax, ebx, ecx, edx);
+ switch (function) {
+ case 1: /* Basic feature request. */
+ /* We only allow kernel to see SSE3, CMPXCHG16B and SSSE3 */
+ *ecx &= 0x00002201;
+ /* SSE, SSE2, FXSR, MMX, CMOV, CMPXCHG8B, FPU. */
+ *edx &= 0x07808101;
+ /* The Host can do a nice optimization if it knows that the
+ * kernel mappings (addresses above 0xC0000000 or whatever
+ * PAGE_OFFSET is set to) haven't changed. But Linux calls
+ * flush_tlb_user() for both user and kernel mappings unless
+ * the Page Global Enable (PGE) feature bit is set. */
+ *edx |= 0x00002000;
+ break;
+ case 0x80000000:
+ /* Futureproof this a little: if they ask how much extended
+ * processor information there is, limit it to known fields. */
+ if (*eax > 0x80000008)
+ *eax = 0x80000008;
+ break;
+ }
+}
+
+/* Intel has four control registers, imaginatively named cr0, cr2, cr3 and cr4.
+ * I assume there's a cr1, but it hasn't bothered us yet, so we'll
not bother
+ * it. The Host needs to know when the Guest wants to change them, so we have
+ * a whole series of functions like read_cr0() and write_cr0().
+ *
+ * We start with CR0. CR0 allows you to turn on and off all kinds of basic
+ * features, but Linux only really cares about one: the horrifically-named Task
+ * Switched (TS) bit at bit 3 (ie. 8)
+ *
+ * What does the TS bit do? Well, it causes the CPU to trap (interrupt 7) if
+ * the floating point unit is used. Which allows us to restore FPU state
+ * lazily after a task switch, and Linux uses that gratefully, but wouldn't
a
+ * name like "FPUTRAP bit" be a little less cryptic?
+ *
+ * We store cr0 (and cr3) locally, because the Host never changes it. The
+ * Guest sometimes wants to read it and we'd prefer not to bother the Host
+ * unnecessarily. */
+static unsigned long current_cr0, current_cr3;
+static void lguest_write_cr0(unsigned long val)
+{
+ /* 8 == TS bit. */
+ lazy_hcall(LHCALL_TS, val & 8, 0, 0);
+ current_cr0 = val;
+}
+
+static unsigned long lguest_read_cr0(void)
+{
+ return current_cr0;
+}
+
+/* Intel provided a special instruction to clear the TS bit for people too cool
+ * to use write_cr0() to do it. This "clts" instruction is faster,
because all
+ * the vowels have been optimized out. */
+static void lguest_clts(void)
+{
+ lazy_hcall(LHCALL_TS, 0, 0, 0);
+ current_cr0 &= ~8U;
+}
+
+/* CR2 is the virtual address of the last page fault, which the Guest only ever
+ * reads. The Host kindly writes this into our "struct lguest_data",
so we
+ * just read it out of there. */
+static unsigned long lguest_read_cr2(void)
+{
+ return lguest_data.cr2;
+}
+
+/* CR3 is the current toplevel pagetable page: the principle is the same as
+ * cr0. Keep a local copy, and tell the Host when it changes. */
+static void lguest_write_cr3(unsigned long cr3)
+{
+ lazy_hcall(LHCALL_NEW_PGTABLE, cr3, 0, 0);
+ current_cr3 = cr3;
+}
+
+static unsigned long lguest_read_cr3(void)
+{
+ return current_cr3;
+}
+
+/* CR4 is used to enable and disable PGE, but we don't care. */
+static unsigned long lguest_read_cr4(void)
+{
+ return 0;
+}
+
+static void lguest_write_cr4(unsigned long val)
+{
+}
+
+/*
+ * Page Table Handling.
+ *
+ * Now would be a good time to take a rest and grab a coffee or similarly
+ * relaxing stimulant. The easy parts are behind us, and the trek gradually
+ * winds uphill from here.
+ *
+ * Quick refresher: memory is divided into "pages" of 4096 bytes
each. The CPU
+ * maps virtual addresses to physical addresses using "page tables".
We could
+ * use one huge index of 1 million entries: each address is 4 bytes, so
that's
+ * 1024 pages just to hold the page tables. But since most virtual addresses
+ * are unused, we use a two level index which saves space. The CR3 register
+ * contains the physical address of the top level "page directory"
page, which
+ * contains physical addresses of up to 1024 second-level pages. Each of these
+ * second level pages contains up to 1024 physical addresses of actual pages,
+ * or Page Table Entries (PTEs).
+ *
+ * Here's a diagram, where arrows indicate physical addresses:
+ *
+ * CR3 ---> +---------+
+ * | --------->+---------+
+ * | | | PADDR1 |
+ * Top-level | | PADDR2 |
+ * (PMD) page | | |
+ * | | Lower-level |
+ * | | (PTE) page |
+ * | | | |
+ * .... ....
+ *
+ * So to convert a virtual address to a physical address, we look up the top
+ * level, which points us to the second level, which gives us the physical
+ * address of that page. If the top level entry was not present, or the second
+ * level entry was not present, then the virtual address is invalid (we
+ * say "the page was not mapped").
+ *
+ * Put another way, a 32-bit virtual address is divided up like so:
+ *
+ * 1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
+ * |<---- 10 bits ---->|<---- 10 bits ---->|<------ 12 bits
------>|
+ * Index into top Index into second Offset within page
+ * page directory page pagetable page
+ *
+ * The kernel spends a lot of time changing both the top-level page directory
+ * and lower-level pagetable pages. The Guest doesn't know physical
addresses,
+ * so while it maintains these page tables exactly like normal, it also needs
+ * to keep the Host informed whenever it makes a change: the Host will create
+ * the real page tables based on the Guests'.
+ */
+
+/* The Guest calls this to set a second-level entry (pte), ie. to map a page
+ * into a process' address space. We set the entry then tell the Host the
+ * toplevel and address this corresponds to. The Guest uses one pagetable per
+ * process, so we need to tell the Host which one we're changing
(mm->pgd). */
+static void lguest_set_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pteval)
+{
+ *ptep = pteval;
+ lazy_hcall(LHCALL_SET_PTE, __pa(mm->pgd), addr, pteval.pte_low);
+}
+
+/* The Guest calls this to set a top-level entry. Again, we set the entry then
+ * tell the Host which top-level page we changed, and the index of the entry we
+ * changed. */
+static void lguest_set_pmd(pmd_t *pmdp, pmd_t pmdval)
+{
+ *pmdp = pmdval;
+ lazy_hcall(LHCALL_SET_PMD, __pa(pmdp)&PAGE_MASK,
+ (__pa(pmdp)&(PAGE_SIZE-1))/4, 0);
+}
+
+/* There are a couple of legacy places where the kernel sets a PTE, but we
+ * don't know the top level any more. This is useless for us, since we
don't
+ * know which pagetable is changing or what address, so we just tell the Host
+ * to forget all of them. Fortunately, this is very rare.
+ *
+ * ... except in early boot when the kernel sets up the initial pagetables,
+ * which makes booting astonishingly slow. So we don't even tell the Host
+ * anything changed until we've done the first page table switch.
+ */
+static void lguest_set_pte(pte_t *ptep, pte_t pteval)
+{
+ *ptep = pteval;
+ /* Don't bother with hypercall before initial setup. */
+ if (current_cr3)
+ lazy_hcall(LHCALL_FLUSH_TLB, 1, 0, 0);
+}
+
+/* Unfortunately for Lguest, the paravirt_ops for page tables were based on
+ * native page table operations. On native hardware you can set a new page
+ * table entry whenever you want, but if you want to remove one you have to do
+ * a TLB flush (a TLB is a little cache of page table entries kept by the CPU).
+ *
+ * So the lguest_set_pte_at() and lguest_set_pmd() functions above are only
+ * called when a valid entry is written, not when it's removed (ie. marked
not
+ * present). Instead, this is where we come when the Guest wants to remove a
+ * page table entry: we tell the Host to set that entry to 0 (ie. the present
+ * bit is zero). */
+static void lguest_flush_tlb_single(unsigned long addr)
+{
+ /* Simply set it to zero: if it was not, it will fault back in. */
+ lazy_hcall(LHCALL_SET_PTE, current_cr3, addr, 0);
+}
+
+/* This is what happens after the Guest has removed a large number of entries.
+ * This tells the Host that any of the page table entries for userspace might
+ * have changed, ie. virtual addresses below PAGE_OFFSET. */
+static void lguest_flush_tlb_user(void)
+{
+ lazy_hcall(LHCALL_FLUSH_TLB, 0, 0, 0);
+}
+
+/* This is called when the kernel page tables have changed. That's not
very
+ * common (unless the Guest is using highmem, which makes the Guest extremely
+ * slow), so it's worth separating this from the user flushing above. */
+static void lguest_flush_tlb_kernel(void)
+{
+ lazy_hcall(LHCALL_FLUSH_TLB, 1, 0, 0);
+}
+
+/*
+ * The Unadvanced Programmable Interrupt Controller.
+ *
+ * This is an attempt to implement the simplest possible interrupt controller.
+ * I spent some time looking though routines like set_irq_chip_and_handler,
+ * set_irq_chip_and_handler_name, set_irq_chip_data and set_phasers_to_stun and
+ * I *think* this is as simple as it gets.
+ *
+ * We can tell the Host what interrupts we want blocked ready for using the
+ * lguest_data.interrupts bitmap, so disabling (aka "masking") them
is as
+ * simple as setting a bit. We don't actually "ack" interrupts
as such, we
+ * just mask and unmask them. I wonder if we should be cleverer?
+ */
+static void disable_lguest_irq(unsigned int irq)
+{
+ set_bit(irq, lguest_data.blocked_interrupts);
+}
+
+static void enable_lguest_irq(unsigned int irq)
+{
+ clear_bit(irq, lguest_data.blocked_interrupts);
+}
+
+/* This structure describes the lguest IRQ controller. */
+static struct irq_chip lguest_irq_controller = {
+ .name = "lguest",
+ .mask = disable_lguest_irq,
+ .mask_ack = disable_lguest_irq,
+ .unmask = enable_lguest_irq,
+};
+
+/* This sets up the Interrupt Descriptor Table (IDT) entry for each hardware
+ * interrupt (except 128, which is used for system calls), and then tells the
+ * Linux infrastructure that each interrupt is controlled by our level-based
+ * lguest interrupt controller. */
+static void __init lguest_init_IRQ(void)
+{
+ unsigned int i;
+
+ for (i = 0; i < LGUEST_IRQS; i++) {
+ int vector = FIRST_EXTERNAL_VECTOR + i;
+ if (vector != SYSCALL_VECTOR) {
+ set_intr_gate(vector, interrupt[i]);
+ set_irq_chip_and_handler(i, &lguest_irq_controller,
+ handle_level_irq);
+ }
+ }
+ /* This call is required to set up for 4k stacks, where we have
+ * separate stacks for hard and soft interrupts. */
+ irq_ctx_init(smp_processor_id());
+}
+
+/*
+ * Time.
+ *
+ * It would be far better for everyone if the Guest had its own clock, but
+ * until then the Host gives us the time on every interrupt.
+ */
+static unsigned long lguest_get_wallclock(void)
+{
+ return lguest_data.time.tv_sec;
+}
+
+static cycle_t lguest_clock_read(void)
+{
+ unsigned long sec, nsec;
+
+ /* If the Host tells the TSC speed, we can trust that. */
+ if (lguest_data.tsc_khz)
+ return native_read_tsc();
+
+ /* If we can't use the TSC, we read the time value written by the Host.
+ * Since it's in two parts (seconds and nanoseconds), we risk reading
+ * it just as it's changing from 99 & 0.999999999 to 100 and 0, and
+ * getting 99 and 0. As Linux tends to come apart under the stress of
+ * time travel, we must be careful: */
+ do {
+ /* First we read the seconds part. */
+ sec = lguest_data.time.tv_sec;
+ /* This read memory barrier tells the compiler and the CPU that
+ * this can't be reordered: we have to complete the above
+ * before going on. */
+ rmb();
+ /* Now we read the nanoseconds part. */
+ nsec = lguest_data.time.tv_nsec;
+ /* Make sure we've done that. */
+ rmb();
+ /* Now if the seconds part has changed, try again. */
+ } while (unlikely(lguest_data.time.tv_sec != sec));
+
+ /* Our non-TSC clock is in real nanoseconds. */
+ return sec*1000000000ULL + nsec;
+}
+
+/* This is what we tell the kernel is our clocksource. */
+static struct clocksource lguest_clock = {
+ .name = "lguest",
+ .rating = 400,
+ .read = lguest_clock_read,
+ .mask = CLOCKSOURCE_MASK(64),
+ .mult = 1 << 22,
+ .shift = 22,
+};
+
+/* The "scheduler clock" is just our real clock, adjusted to start at
zero */
+static unsigned long long lguest_sched_clock(void)
+{
+ return cyc2ns(&lguest_clock, lguest_clock_read() - clock_base);
+}
+
+/* We also need a "struct clock_event_device": Linux asks us to set
it to go
+ * off some time in the future. Actually, James Morris figured all this out, I
+ * just applied the patch. */
+static int lguest_clockevent_set_next_event(unsigned long delta,
+ struct clock_event_device *evt)
+{
+ if (delta < LG_CLOCK_MIN_DELTA) {
+ if (printk_ratelimit())
+ printk(KERN_DEBUG "%s: small delta %lu ns\n",
+ __FUNCTION__, delta);
+ return -ETIME;
+ }
+ hcall(LHCALL_SET_CLOCKEVENT, delta, 0, 0);
+ return 0;
+}
+
+static void lguest_clockevent_set_mode(enum clock_event_mode mode,
+ struct clock_event_device *evt)
+{
+ switch (mode) {
+ case CLOCK_EVT_MODE_UNUSED:
+ case CLOCK_EVT_MODE_SHUTDOWN:
+ /* A 0 argument shuts the clock down. */
+ hcall(LHCALL_SET_CLOCKEVENT, 0, 0, 0);
+ break;
+ case CLOCK_EVT_MODE_ONESHOT:
+ /* This is what we expect. */
+ break;
+ case CLOCK_EVT_MODE_PERIODIC:
+ BUG();
+ case CLOCK_EVT_MODE_RESUME:
+ break;
+ }
+}
+
+/* This describes our primitive timer chip. */
+static struct clock_event_device lguest_clockevent = {
+ .name = "lguest",
+ .features = CLOCK_EVT_FEAT_ONESHOT,
+ .set_next_event = lguest_clockevent_set_next_event,
+ .set_mode = lguest_clockevent_set_mode,
+ .rating = INT_MAX,
+ .mult = 1,
+ .shift = 0,
+ .min_delta_ns = LG_CLOCK_MIN_DELTA,
+ .max_delta_ns = LG_CLOCK_MAX_DELTA,
+};
+
+/* This is the Guest timer interrupt handler (hardware interrupt 0). We just
+ * call the clockevent infrastructure and it does whatever needs doing. */
+static void lguest_time_irq(unsigned int irq, struct irq_desc *desc)
+{
+ unsigned long flags;
+
+ /* Don't interrupt us while this is running. */
+ local_irq_save(flags);
+ lguest_clockevent.event_handler(&lguest_clockevent);
+ local_irq_restore(flags);
+}
+
+/* At some point in the boot process, we get asked to set up our timing
+ * infrastructure. The kernel doesn't expect timer interrupts before this,
but
+ * we cleverly initialized the "blocked_interrupts" field of
"struct
+ * lguest_data" so that timer interrupts were blocked until now. */
+static void lguest_time_init(void)
+{
+ /* Set up the timer interrupt (0) to go to our simple timer routine */
+ set_irq_handler(0, lguest_time_irq);
+
+ /* Our clock structure look like arch/i386/kernel/tsc.c if we can use
+ * the TSC, otherwise it's a dumb nanosecond-resolution clock. Either
+ * way, the "rating" is initialized so high that it's always
chosen
+ * over any other clocksource. */
+ if (lguest_data.tsc_khz) {
+ lguest_clock.mult = clocksource_khz2mult(lguest_data.tsc_khz,
+ lguest_clock.shift);
+ lguest_clock.flags = CLOCK_SOURCE_IS_CONTINUOUS;
+ }
+ clock_base = lguest_clock_read();
+ clocksource_register(&lguest_clock);
+
+ /* Now we've set up our clock, we can use it as the scheduler clock */
+ paravirt_ops.sched_clock = lguest_sched_clock;
+
+ /* We can't set cpumask in the initializer: damn C limitations! Set it
+ * here and register our timer device. */
+ lguest_clockevent.cpumask = cpumask_of_cpu(0);
+ clockevents_register_device(&lguest_clockevent);
+
+ /* Finally, we unblock the timer interrupt. */
+ enable_lguest_irq(0);
+}
+
+/*
+ * Miscellaneous bits and pieces.
+ *
+ * Here is an oddball collection of functions which the Guest needs for things
+ * to work. They're pretty simple.
+ */
+
+/* The Guest needs to tell the host what stack it expects traps to use. For
+ * native hardware, this is part of the Task State Segment mentioned above in
+ * lguest_load_tr_desc(), but to help hypervisors there's this special
call.
+ *
+ * We tell the Host the segment we want to use (__KERNEL_DS is the kernel data
+ * segment), the privilege level (we're privilege level 1, the Host is 0
and
+ * will not tolerate us trying to use that), the stack pointer, and the number
+ * of pages in the stack. */
+static void lguest_load_esp0(struct tss_struct *tss,
+ struct thread_struct *thread)
+{
+ lazy_hcall(LHCALL_SET_STACK, __KERNEL_DS|0x1, thread->esp0,
+ THREAD_SIZE/PAGE_SIZE);
+}
+
+/* Let's just say, I wouldn't do debugging under a Guest. */
+static void lguest_set_debugreg(int regno, unsigned long value)
+{
+ /* FIXME: Implement */
+}
+
+/* There are times when the kernel wants to make sure that no memory writes are
+ * caught in the cache (that they've all reached real hardware devices).
This
+ * doesn't matter for the Guest which has virtual hardware.
+ *
+ * On the Pentium 4 and above, cpuid() indicates that the Cache Line Flush
+ * (clflush) instruction is available and the kernel uses that. Otherwise, it
+ * uses the older "Write Back and Invalidate Cache" (wbinvd)
instruction.
+ * Unlike clflush, wbinvd can only be run at privilege level 0. So we can
+ * ignore clflush, but replace wbinvd.
+ */
+static void lguest_wbinvd(void)
+{
+}
+
+/* If the Guest expects to have an Advanced Programmable Interrupt Controller,
+ * we play dumb by ignoring writes and returning 0 for reads. So it's no
+ * longer Programmable nor Controlling anything, and I don't think 8 lines
of
+ * code qualifies for Advanced. It will also never interrupt anything. It
+ * does, however, allow us to get through the Linux boot code. */
+#ifdef CONFIG_X86_LOCAL_APIC
+static void lguest_apic_write(unsigned long reg, unsigned long v)
+{
+}
+
+static unsigned long lguest_apic_read(unsigned long reg)
+{
+ return 0;
+}
+#endif
+
+/* STOP! Until an interrupt comes in. */
+static void lguest_safe_halt(void)
+{
+ hcall(LHCALL_HALT, 0, 0, 0);
+}
+
+/* Perhaps CRASH isn't the best name for this hypercall, but we use it to
get a
+ * message out when we're crashing as well as elegant termination like
powering
+ * off.
+ *
+ * Note that the Host always prefers that the Guest speak in physical addresses
+ * rather than virtual addresses, so we use __pa() here. */
+static void lguest_power_off(void)
+{
+ hcall(LHCALL_CRASH, __pa("Power down"), 0, 0);
+}
+
+/*
+ * Panicing.
+ *
+ * Don't. But if you did, this is what happens.
+ */
+static int lguest_panic(struct notifier_block *nb, unsigned long l, void *p)
+{
+ hcall(LHCALL_CRASH, __pa(p), 0, 0);
+ /* The hcall won't return, but to keep gcc happy, we're
"done". */
+ return NOTIFY_DONE;
+}
+
+static struct notifier_block paniced = {
+ .notifier_call = lguest_panic
+};
+
+/* Setting up memory is fairly easy. */
+static __init char *lguest_memory_setup(void)
+{
+ /* We do this here and not earlier because lockcheck barfs if we do it
+ * before start_kernel() */
+ atomic_notifier_chain_register(&panic_notifier_list, &paniced);
+
+ /* The Linux bootloader header contains an "e820" memory map: the
+ * Launcher populated the first entry with our memory limit. */
+ add_memory_region(E820_MAP->addr, E820_MAP->size, E820_MAP->type);
+
+ /* This string is for the boot messages. */
+ return "LGUEST";
+}
+
+/*G:050
+ * Patching (Powerfully Placating Performance Pedants)
+ *
+ * We have already seen that "struct paravirt_ops" lets us replace
simple
+ * native instructions with calls to the appropriate back end all throughout
+ * the kernel. This allows the same kernel to run as a Guest and as a native
+ * kernel, but it's slow because of all the indirect branches.
+ *
+ * Remember that David Wheeler quote about "Any problem in computer
science can
+ * be solved with another layer of indirection"? The rest of that quote
is
+ * "... But that usually will create another problem." This is the
first of
+ * those problems.
+ *
+ * Our current solution is to allow the paravirt back end to optionally patch
+ * over the indirect calls to replace them with something more efficient. We
+ * patch the four most commonly called functions: disable interrupts, enable
+ * interrupts, restore interrupts and save interrupts. We usually have 10
+ * bytes to patch into: the Guest versions of these operations are small enough
+ * that we can fit comfortably.
+ *
+ * First we need assembly templates of each of the patchable Guest operations,
+ * and these are in lguest_asm.S. */
+
+/*G:060 We construct a table from the assembler templates: */
+static const struct lguest_insns
+{
+ const char *start, *end;
+} lguest_insns[] = {
+ [PARAVIRT_PATCH(irq_disable)] = { lgstart_cli, lgend_cli },
+ [PARAVIRT_PATCH(irq_enable)] = { lgstart_sti, lgend_sti },
+ [PARAVIRT_PATCH(restore_fl)] = { lgstart_popf, lgend_popf },
+ [PARAVIRT_PATCH(save_fl)] = { lgstart_pushf, lgend_pushf },
+};
+
+/* Now our patch routine is fairly simple (based on the native one in
+ * paravirt.c). If we have a replacement, we copy it in and return how much of
+ * the available space we used. */
+static unsigned lguest_patch(u8 type, u16 clobber, void *ibuf,
+ unsigned long addr, unsigned len)
+{
+ unsigned int insn_len;
+
+ /* Don't do anything special if we don't have a replacement */
+ if (type >= ARRAY_SIZE(lguest_insns) || !lguest_insns[type].start)
+ return paravirt_patch_default(type, clobber, ibuf, addr, len);
+
+ insn_len = lguest_insns[type].end - lguest_insns[type].start;
+
+ /* Similarly if we can't fit replacement (shouldn't happen, but
let's
+ * be thorough). */
+ if (len < insn_len)
+ return paravirt_patch_default(type, clobber, ibuf, addr, len);
+
+ /* Copy in our instructions. */
+ memcpy(ibuf, lguest_insns[type].start, insn_len);
+ return insn_len;
+}
+
+/*G:030 Once we get to lguest_init(), we know we're a Guest. The
paravirt_ops
+ * structure in the kernel provides a single point for (almost) every routine
+ * we have to override to avoid privileged instructions. */
+__init void lguest_init(void *boot)
+{
+ /* Copy boot parameters first: the Launcher put the physical location
+ * in %esi, and head.S converted that to a virtual address and handed
+ * it to us. We use "__memcpy" because "memcpy" sometimes
tries to do
+ * tricky things to go faster, and we're not ready for that. */
+ __memcpy(&boot_params, boot, PARAM_SIZE);
+ /* The boot parameters also tell us where the command-line is: save
+ * that, too. */
+ __memcpy(boot_command_line, __va(boot_params.hdr.cmd_line_ptr),
+ COMMAND_LINE_SIZE);
+
+ /* We're under lguest, paravirt is enabled, and we're running at
+ * privilege level 1, not 0 as normal. */
+ paravirt_ops.name = "lguest";
+ paravirt_ops.paravirt_enabled = 1;
+ paravirt_ops.kernel_rpl = 1;
+
+ /* We set up all the lguest overrides for sensitive operations. These
+ * are detailed with the operations themselves. */
+ paravirt_ops.save_fl = save_fl;
+ paravirt_ops.restore_fl = restore_fl;
+ paravirt_ops.irq_disable = irq_disable;
+ paravirt_ops.irq_enable = irq_enable;
+ paravirt_ops.load_gdt = lguest_load_gdt;
+ paravirt_ops.memory_setup = lguest_memory_setup;
+ paravirt_ops.cpuid = lguest_cpuid;
+ paravirt_ops.write_cr3 = lguest_write_cr3;
+ paravirt_ops.flush_tlb_user = lguest_flush_tlb_user;
+ paravirt_ops.flush_tlb_single = lguest_flush_tlb_single;
+ paravirt_ops.flush_tlb_kernel = lguest_flush_tlb_kernel;
+ paravirt_ops.set_pte = lguest_set_pte;
+ paravirt_ops.set_pte_at = lguest_set_pte_at;
+ paravirt_ops.set_pmd = lguest_set_pmd;
+#ifdef CONFIG_X86_LOCAL_APIC
+ paravirt_ops.apic_write = lguest_apic_write;
+ paravirt_ops.apic_write_atomic = lguest_apic_write;
+ paravirt_ops.apic_read = lguest_apic_read;
+#endif
+ paravirt_ops.load_idt = lguest_load_idt;
+ paravirt_ops.iret = lguest_iret;
+ paravirt_ops.load_esp0 = lguest_load_esp0;
+ paravirt_ops.load_tr_desc = lguest_load_tr_desc;
+ paravirt_ops.set_ldt = lguest_set_ldt;
+ paravirt_ops.load_tls = lguest_load_tls;
+ paravirt_ops.set_debugreg = lguest_set_debugreg;
+ paravirt_ops.clts = lguest_clts;
+ paravirt_ops.read_cr0 = lguest_read_cr0;
+ paravirt_ops.write_cr0 = lguest_write_cr0;
+ paravirt_ops.init_IRQ = lguest_init_IRQ;
+ paravirt_ops.read_cr2 = lguest_read_cr2;
+ paravirt_ops.read_cr3 = lguest_read_cr3;
+ paravirt_ops.read_cr4 = lguest_read_cr4;
+ paravirt_ops.write_cr4 = lguest_write_cr4;
+ paravirt_ops.write_gdt_entry = lguest_write_gdt_entry;
+ paravirt_ops.write_idt_entry = lguest_write_idt_entry;
+ paravirt_ops.patch = lguest_patch;
+ paravirt_ops.safe_halt = lguest_safe_halt;
+ paravirt_ops.get_wallclock = lguest_get_wallclock;
+ paravirt_ops.time_init = lguest_time_init;
+ paravirt_ops.set_lazy_mode = lguest_lazy_mode;
+ paravirt_ops.wbinvd = lguest_wbinvd;
+ /* Now is a good time to look at the implementations of these functions
+ * before returning to the rest of lguest_init(). */
+
+ /*G:070 Now we've seen all the paravirt_ops, we return to
+ * lguest_init() where the rest of the fairly chaotic boot setup
+ * occurs.
+ *
+ * The Host expects our first hypercall to tell it where our "struct
+ * lguest_data" is, so we do that first. */
+ hcall(LHCALL_LGUEST_INIT, __pa(&lguest_data), 0, 0);
+
+ /* The native boot code sets up initial page tables immediately after
+ * the kernel itself, and sets init_pg_tables_end so they're not
+ * clobbered. The Launcher places our initial pagetables somewhere at
+ * the top of our physical memory, so we don't need extra space: set
+ * init_pg_tables_end to the end of the kernel. */
+ init_pg_tables_end = __pa(pg0);
+
+ /* Load the %fs segment register (the per-cpu segment register) with
+ * the normal data segment to get through booting. */
+ asm volatile ("mov %0, %%fs" : : "r" (__KERNEL_DS) :
"memory");
+
+ /* Clear the part of the kernel data which is expected to be zero.
+ * Normally it will be anyway, but if we're loading from a bzImage with
+ * CONFIG_RELOCATALE=y, the relocations will be sitting here. */
+ memset(__bss_start, 0, __bss_stop - __bss_start);
+
+ /* The Host uses the top of the Guest's virtual address space for the
+ * Host<->Guest Switcher, and it tells us how much it needs in
+ * lguest_data.reserve_mem, set up on the LGUEST_INIT hypercall. */
+ reserve_top_address(lguest_data.reserve_mem);
+
+ /* If we don't initialize the lock dependency checker now, it crashes
+ * paravirt_disable_iospace. */
+ lockdep_init();
+
+ /* The IDE code spends about 3 seconds probing for disks: if we reserve
+ * all the I/O ports up front it can't get them and so doesn't probe.
+ * Other device drivers are similar (but less severe). This cuts the
+ * kernel boot time on my machine from 4.1 seconds to 0.45 seconds. */
+ paravirt_disable_iospace();
+
+ /* This is messy CPU setup stuff which the native boot code does before
+ * start_kernel, so we have to do, too: */
+ cpu_detect(&new_cpu_data);
+ /* head.S usually sets up the first capability word, so do it here. */
+ new_cpu_data.x86_capability[0] = cpuid_edx(1);
+
+ /* Math is always hard! */
+ new_cpu_data.hard_math = 1;
+
+#ifdef CONFIG_X86_MCE
+ mce_disabled = 1;
+#endif
+#ifdef CONFIG_ACPI
+ acpi_disabled = 1;
+ acpi_ht = 0;
+#endif
+
+ /* We set the perferred console to "hvc". This is the
"hypervisor
+ * virtual console" driver written by the PowerPC people, which we also
+ * adapted for lguest's use. */
+ add_preferred_console("hvc", 0, NULL);
+
+ /* Last of all, we set the power management poweroff hook to point to
+ * the Guest routine to power off. */
+ pm_power_off = lguest_power_off;
+
+ /* Now we're set up, call start_kernel() in init/main.c and we proceed
+ * to boot as normal. It never returns. */
+ start_kernel();
+}
+/*
+ * This marks the end of stage II of our journey, The Guest.
+ *
+ * It is now time for us to explore the nooks and crannies of the three Guest
+ * devices and complete our understanding of the Guest in "make
Drivers".
+ */
==================================================================--- /dev/null
+++ b/arch/i386/lguest/head.S
@@ -0,0 +1,93 @@
+#include <linux/linkage.h>
+#include <linux/lguest.h>
+#include <asm/asm-offsets.h>
+#include <asm/thread_info.h>
+#include <asm/processor-flags.h>
+
+/*G:020 This is where we begin: we have a magic signature which the launcher
+ * looks for. The plan is that the Linux boot protocol will be extended with a
+ * "platform type" field which will guide us here from the normal
entry point,
+ * but for the moment this suffices. The normal boot code uses %esi for the
+ * boot header, so we do too. We convert it to a virtual address by adding
+ * PAGE_OFFSET, and hand it to lguest_init() as its argument (ie. %eax).
+ *
+ * The .section line puts this code in .init.text so it will be discarded after
+ * boot. */
+.section .init.text, "ax", @progbits
+.ascii "GenuineLguest"
+ /* Set up initial stack. */
+ movl $(init_thread_union+THREAD_SIZE),%esp
+ movl %esi, %eax
+ addl $__PAGE_OFFSET, %eax
+ jmp lguest_init
+
+/*G:055 We create a macro which puts the assembler code between lgstart_ and
+ * lgend_ markers. These templates are put in the .text section: they
can't be
+ * discarded after boot as we may need to patch modules, too. */
+.text
+#define LGUEST_PATCH(name, insns...) \
+ lgstart_##name: insns; lgend_##name:; \
+ .globl lgstart_##name; .globl lgend_##name
+
+LGUEST_PATCH(cli, movl $0, lguest_data+LGUEST_DATA_irq_enabled)
+LGUEST_PATCH(sti, movl $X86_EFLAGS_IF, lguest_data+LGUEST_DATA_irq_enabled)
+LGUEST_PATCH(popf, movl %eax, lguest_data+LGUEST_DATA_irq_enabled)
+LGUEST_PATCH(pushf, movl lguest_data+LGUEST_DATA_irq_enabled, %eax)
+/*:*/
+
+/* These demark the EIP range where host should never deliver interrupts. */
+.global lguest_noirq_start
+.global lguest_noirq_end
+
+/*M:004 When the Host reflects a trap or injects an interrupt into the Guest,
+ * it sets the eflags interrupt bit on the stack based on
+ * lguest_data.irq_enabled, so the Guest iret logic does the right thing when
+ * restoring it. However, when the Host sets the Guest up for direct traps,
+ * such as system calls, the processor is the one to push eflags onto the
+ * stack, and the interrupt bit will be 1 (in reality, interrupts are always
+ * enabled in the Guest).
+ *
+ * This turns out to be harmless: the only trap which should happen under Linux
+ * with interrupts disabled is Page Fault (due to our lazy mapping of vmalloc
+ * regions), which has to be reflected through the Host anyway. If another
+ * trap *does* go off when interrupts are disabled, the Guest will panic, and
+ * we'll never get to this iret! :*/
+
+/*G:045 There is one final paravirt_op that the Guest implements, and glancing
+ * at it you can see why I left it to last. It's *cool*! It's in
*assembler*!
+ *
+ * The "iret" instruction is used to return from an interrupt or
trap. The
+ * stack looks like this:
+ * old address
+ * old code segment & privilege level
+ * old processor flags ("eflags")
+ *
+ * The "iret" instruction pops those values off the stack and
restores them all
+ * at once. The only problem is that eflags includes the Interrupt Flag which
+ * the Guest can't change: the CPU will simply ignore it when we do an
"iret".
+ * So we have to copy eflags from the stack to lguest_data.irq_enabled before
+ * we do the "iret".
+ *
+ * There are two problems with this: firstly, we need to use a register to do
+ * the copy and secondly, the whole thing needs to be atomic. The first
+ * problem is easy to solve: push %eax on the stack so we can use it, and then
+ * restore it at the end just before the real "iret".
+ *
+ * The second is harder: copying eflags to lguest_data.irq_enabled will turn
+ * interrupts on before we're finished, so we could be interrupted before
we
+ * return to userspace or wherever. Our solution to this is to surround the
+ * code with lguest_noirq_start: and lguest_noirq_end: labels. We tell the
+ * Host that it is *never* to interrupt us there, even if interrupts seem to be
+ * enabled. */
+ENTRY(lguest_iret)
+ pushl %eax
+ movl 12(%esp), %eax
+lguest_noirq_start:
+ /* Note the %ss: segment prefix here. Normal data accesses use the
+ * "ds" segment, but that will have already been restored for
whatever
+ * we're returning to (such as userspace): we can't trust it. The
%ss:
+ * prefix makes sure we use the stack segment, which is still valid. */
+ movl %eax,%ss:lguest_data+LGUEST_DATA_irq_enabled
+ popl %eax
+ iret
+lguest_noirq_end:
==================================================================---
a/drivers/lguest/Makefile
+++ b/drivers/lguest/Makefile
@@ -1,5 +1,5 @@
-# Guest requires the paravirt_ops replacement and the bus driver.
-obj-$(CONFIG_LGUEST_GUEST) += lguest.o lguest_asm.o lguest_bus.o
+# Guest requires the bus driver.
+obj-$(CONFIG_LGUEST_GUEST) += lguest_bus.o
# Host requires the other files, which can be a module.
obj-$(CONFIG_LGUEST) += lg.o
==================================================================---
a/drivers/lguest/lguest.c
+++ /dev/null
@@ -1,1101 +0,0 @@
-/*P:010
- * A hypervisor allows multiple Operating Systems to run on a single machine.
- * To quote David Wheeler: "Any problem in computer science can be solved
with
- * another layer of indirection."
- *
- * We keep things simple in two ways. First, we start with a normal Linux
- * kernel and insert a module (lg.ko) which allows us to run other Linux
- * kernels the same way we'd run processes. We call the first kernel the
Host,
- * and the others the Guests. The program which sets up and configures Guests
- * (such as the example in Documentation/lguest/lguest.c) is called the
- * Launcher.
- *
- * Secondly, we only run specially modified Guests, not normal kernels. When
- * you set CONFIG_LGUEST to 'y' or 'm', this automatically sets
- * CONFIG_LGUEST_GUEST=y, which compiles this file into the kernel so it knows
- * how to be a Guest. This means that you can use the same kernel you boot
- * normally (ie. as a Host) as a Guest.
- *
- * These Guests know that they cannot do privileged operations, such as disable
- * interrupts, and that they have to ask the Host to do such things explicitly.
- * This file consists of all the replacements for such low-level native
- * hardware operations: these special Guest versions call the Host.
- *
- * So how does the kernel know it's a Guest? The Guest starts at a special
- * entry point marked with a magic string, which sets up a few things then
- * calls here. We replace the native functions in "struct
paravirt_ops"
- * with our Guest versions, then boot like normal. :*/
-
-/*
- * Copyright (C) 2006, Rusty Russell <rusty@rustcorp.com.au> IBM
Corporation.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or
- * NON INFRINGEMENT. See the GNU General Public License for more
- * details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
- */
-#include <linux/kernel.h>
-#include <linux/start_kernel.h>
-#include <linux/string.h>
-#include <linux/console.h>
-#include <linux/screen_info.h>
-#include <linux/irq.h>
-#include <linux/interrupt.h>
-#include <linux/clocksource.h>
-#include <linux/clockchips.h>
-#include <linux/lguest.h>
-#include <linux/lguest_launcher.h>
-#include <linux/lguest_bus.h>
-#include <asm/paravirt.h>
-#include <asm/param.h>
-#include <asm/page.h>
-#include <asm/pgtable.h>
-#include <asm/desc.h>
-#include <asm/setup.h>
-#include <asm/e820.h>
-#include <asm/mce.h>
-#include <asm/io.h>
-
-/*G:010 Welcome to the Guest!
- *
- * The Guest in our tale is a simple creature: identical to the Host but
- * behaving in simplified but equivalent ways. In particular, the Guest is the
- * same kernel as the Host (or at least, built from the same source code). :*/
-
-/* Declarations for definitions in lguest_guest.S */
-extern char lguest_noirq_start[], lguest_noirq_end[];
-extern const char lgstart_cli[], lgend_cli[];
-extern const char lgstart_sti[], lgend_sti[];
-extern const char lgstart_popf[], lgend_popf[];
-extern const char lgstart_pushf[], lgend_pushf[];
-extern const char lgstart_iret[], lgend_iret[];
-extern void lguest_iret(void);
-
-struct lguest_data lguest_data = {
- .hcall_status = { [0 ... LHCALL_RING_SIZE-1] = 0xFF },
- .noirq_start = (u32)lguest_noirq_start,
- .noirq_end = (u32)lguest_noirq_end,
- .blocked_interrupts = { 1 }, /* Block timer interrupts */
-};
-static cycle_t clock_base;
-
-/*G:035 Notice the lazy_hcall() above, rather than hcall(). This is our first
- * real optimization trick!
- *
- * When lazy_mode is set, it means we're allowed to defer all hypercalls
and do
- * them as a batch when lazy_mode is eventually turned off. Because hypercalls
- * are reasonably expensive, batching them up makes sense. For example, a
- * large mmap might update dozens of page table entries: that code calls
- * lguest_lazy_mode(PARAVIRT_LAZY_MMU), does the dozen updates, then calls
- * lguest_lazy_mode(PARAVIRT_LAZY_NONE).
- *
- * So, when we're in lazy mode, we call async_hypercall() to store the call
for
- * future processing. When lazy mode is turned off we issue a hypercall to
- * flush the stored calls.
- *
- * There's also a hack where "mode" is set to
"PARAVIRT_LAZY_FLUSH" which
- * indicates we're to flush any outstanding calls immediately. This is
used
- * when an interrupt handler does a kmap_atomic(): the page table changes must
- * happen immediately even if we're in the middle of a batch. Usually
we're
- * not, though, so there's nothing to do. */
-static enum paravirt_lazy_mode lazy_mode; /* Note: not SMP-safe! */
-static void lguest_lazy_mode(enum paravirt_lazy_mode mode)
-{
- if (mode == PARAVIRT_LAZY_FLUSH) {
- if (unlikely(lazy_mode != PARAVIRT_LAZY_NONE))
- hcall(LHCALL_FLUSH_ASYNC, 0, 0, 0);
- } else {
- lazy_mode = mode;
- if (mode == PARAVIRT_LAZY_NONE)
- hcall(LHCALL_FLUSH_ASYNC, 0, 0, 0);
- }
-}
-
-static void lazy_hcall(unsigned long call,
- unsigned long arg1,
- unsigned long arg2,
- unsigned long arg3)
-{
- if (lazy_mode == PARAVIRT_LAZY_NONE)
- hcall(call, arg1, arg2, arg3);
- else
- async_hcall(call, arg1, arg2, arg3);
-}
-
-/* async_hcall() is pretty simple: I'm quite proud of it really. We have a
- * ring buffer of stored hypercalls which the Host will run though next time we
- * do a normal hypercall. Each entry in the ring has 4 slots for the hypercall
- * arguments, and a "hcall_status" word which is 0 if the call is
ready to go,
- * and 255 once the Host has finished with it.
- *
- * If we come around to a slot which hasn't been finished, then the table
is
- * full and we just make the hypercall directly. This has the nice side
- * effect of causing the Host to run all the stored calls in the ring buffer
- * which empties it for next time! */
-void async_hcall(unsigned long call,
- unsigned long arg1, unsigned long arg2, unsigned long arg3)
-{
- /* Note: This code assumes we're uniprocessor. */
- static unsigned int next_call;
- unsigned long flags;
-
- /* Disable interrupts if not already disabled: we don't want an
- * interrupt handler making a hypercall while we're already doing
- * one! */
- local_irq_save(flags);
- if (lguest_data.hcall_status[next_call] != 0xFF) {
- /* Table full, so do normal hcall which will flush table. */
- hcall(call, arg1, arg2, arg3);
- } else {
- lguest_data.hcalls[next_call].eax = call;
- lguest_data.hcalls[next_call].edx = arg1;
- lguest_data.hcalls[next_call].ebx = arg2;
- lguest_data.hcalls[next_call].ecx = arg3;
- /* Arguments must all be written before we mark it to go */
- wmb();
- lguest_data.hcall_status[next_call] = 0;
- if (++next_call == LHCALL_RING_SIZE)
- next_call = 0;
- }
- local_irq_restore(flags);
-}
-/*:*/
-
-/* Wrappers for the SEND_DMA and BIND_DMA hypercalls. This is mainly because
- * Jeff Garzik complained that __pa() should never appear in drivers, and this
- * helps remove most of them. But also, it wraps some ugliness. */
-void lguest_send_dma(unsigned long key, struct lguest_dma *dma)
-{
- /* The hcall might not write this if something goes wrong */
- dma->used_len = 0;
- hcall(LHCALL_SEND_DMA, key, __pa(dma), 0);
-}
-
-int lguest_bind_dma(unsigned long key, struct lguest_dma *dmas,
- unsigned int num, u8 irq)
-{
- /* This is the only hypercall which actually wants 5 arguments, and we
- * only support 4. Fortunately the interrupt number is always less
- * than 256, so we can pack it with the number of dmas in the final
- * argument. */
- if (!hcall(LHCALL_BIND_DMA, key, __pa(dmas), (num << 8) | irq))
- return -ENOMEM;
- return 0;
-}
-
-/* Unbinding is the same hypercall as binding, but with 0 num & irq. */
-void lguest_unbind_dma(unsigned long key, struct lguest_dma *dmas)
-{
- hcall(LHCALL_BIND_DMA, key, __pa(dmas), 0);
-}
-
-/* For guests, device memory can be used as normal memory, so we cast away the
- * __iomem to quieten sparse. */
-void *lguest_map(unsigned long phys_addr, unsigned long pages)
-{
- return (__force void *)ioremap(phys_addr, PAGE_SIZE*pages);
-}
-
-void lguest_unmap(void *addr)
-{
- iounmap((__force void __iomem *)addr);
-}
-
-/*G:033
- * Here are our first native-instruction replacements: four functions for
- * interrupt control.
- *
- * The simplest way of implementing these would be to have "turn
interrupts
- * off" and "turn interrupts on" hypercalls. Unfortunately,
this is too slow:
- * these are by far the most commonly called functions of those we override.
- *
- * So instead we keep an "irq_enabled" field inside our "struct
lguest_data",
- * which the Guest can update with a single instruction. The Host knows to
- * check there when it wants to deliver an interrupt.
- */
-
-/* save_flags() is expected to return the processor state (ie.
"eflags"). The
- * eflags word contains all kind of stuff, but in practice Linux only cares
- * about the interrupt flag. Our "save_flags()" just returns that.
*/
-static unsigned long save_fl(void)
-{
- return lguest_data.irq_enabled;
-}
-
-/* "restore_flags" just sets the flags back to the value given. */
-static void restore_fl(unsigned long flags)
-{
- lguest_data.irq_enabled = flags;
-}
-
-/* Interrupts go off... */
-static void irq_disable(void)
-{
- lguest_data.irq_enabled = 0;
-}
-
-/* Interrupts go on... */
-static void irq_enable(void)
-{
- lguest_data.irq_enabled = X86_EFLAGS_IF;
-}
-/*:*/
-/*M:003 Note that we don't check for outstanding interrupts when we
re-enable
- * them (or when we unmask an interrupt). This seems to work for the moment,
- * since interrupts are rare and we'll just get the interrupt on the next
timer
- * tick, but when we turn on CONFIG_NO_HZ, we should revisit this. One way
- * would be to put the "irq_enabled" field in a page by itself, and
have the
- * Host write-protect it when an interrupt comes in when irqs are disabled.
- * There will then be a page fault as soon as interrupts are re-enabled. :*/
-
-/*G:034
- * The Interrupt Descriptor Table (IDT).
- *
- * The IDT tells the processor what to do when an interrupt comes in. Each
- * entry in the table is a 64-bit descriptor: this holds the privilege level,
- * address of the handler, and... well, who cares? The Guest just asks the
- * Host to make the change anyway, because the Host controls the real IDT.
- */
-static void lguest_write_idt_entry(struct desc_struct *dt,
- int entrynum, u32 low, u32 high)
-{
- /* Keep the local copy up to date. */
- write_dt_entry(dt, entrynum, low, high);
- /* Tell Host about this new entry. */
- hcall(LHCALL_LOAD_IDT_ENTRY, entrynum, low, high);
-}
-
-/* Changing to a different IDT is very rare: we keep the IDT up-to-date every
- * time it is written, so we can simply loop through all entries and tell the
- * Host about them. */
-static void lguest_load_idt(const struct Xgt_desc_struct *desc)
-{
- unsigned int i;
- struct desc_struct *idt = (void *)desc->address;
-
- for (i = 0; i < (desc->size+1)/8; i++)
- hcall(LHCALL_LOAD_IDT_ENTRY, i, idt[i].a, idt[i].b);
-}
-
-/*
- * The Global Descriptor Table.
- *
- * The Intel architecture defines another table, called the Global Descriptor
- * Table (GDT). You tell the CPU where it is (and its size) using the
"lgdt"
- * instruction, and then several other instructions refer to entries in the
- * table. There are three entries which the Switcher needs, so the Host simply
- * controls the entire thing and the Guest asks it to make changes using the
- * LOAD_GDT hypercall.
- *
- * This is the opposite of the IDT code where we have a LOAD_IDT_ENTRY
- * hypercall and use that repeatedly to load a new IDT. I don't think it
- * really matters, but wouldn't it be nice if they were the same?
- */
-static void lguest_load_gdt(const struct Xgt_desc_struct *desc)
-{
- BUG_ON((desc->size+1)/8 != GDT_ENTRIES);
- hcall(LHCALL_LOAD_GDT, __pa(desc->address), GDT_ENTRIES, 0);
-}
-
-/* For a single GDT entry which changes, we do the lazy thing: alter our GDT,
- * then tell the Host to reload the entire thing. This operation is so rare
- * that this naive implementation is reasonable. */
-static void lguest_write_gdt_entry(struct desc_struct *dt,
- int entrynum, u32 low, u32 high)
-{
- write_dt_entry(dt, entrynum, low, high);
- hcall(LHCALL_LOAD_GDT, __pa(dt), GDT_ENTRIES, 0);
-}
-
-/* OK, I lied. There are three "thread local storage" GDT entries
which change
- * on every context switch (these three entries are how glibc implements
- * __thread variables). So we have a hypercall specifically for this case. */
-static void lguest_load_tls(struct thread_struct *t, unsigned int cpu)
-{
- /* There's one problem which normal hardware doesn't have: the Host
- * can't handle us removing entries we're currently using. So we
clear
- * the GS register here: if it's needed it'll be reloaded anyway. */
- loadsegment(gs, 0);
- lazy_hcall(LHCALL_LOAD_TLS, __pa(&t->tls_array), cpu, 0);
-}
-
-/*G:038 That's enough excitement for now, back to ploughing through each of
- * the paravirt_ops (we're about 1/3 of the way through).
- *
- * This is the Local Descriptor Table, another weird Intel thingy. Linux only
- * uses this for some strange applications like Wine. We don't do anything
- * here, so they'll get an informative and friendly Segmentation Fault. */
-static void lguest_set_ldt(const void *addr, unsigned entries)
-{
-}
-
-/* This loads a GDT entry into the "Task Register": that entry points
to a
- * structure called the Task State Segment. Some comments scattered though the
- * kernel code indicate that this used for task switching in ages past, along
- * with blood sacrifice and astrology.
- *
- * Now there's nothing interesting in here that we don't get told
elsewhere.
- * But the native version uses the "ltr" instruction, which makes the
Host
- * complain to the Guest about a Segmentation Fault and it'll oops. So we
- * override the native version with a do-nothing version. */
-static void lguest_load_tr_desc(void)
-{
-}
-
-/* The "cpuid" instruction is a way of querying both the CPU identity
- * (manufacturer, model, etc) and its features. It was introduced before the
- * Pentium in 1993 and keeps getting extended by both Intel and AMD. As you
- * might imagine, after a decade and a half this treatment, it is now a giant
- * ball of hair. Its entry in the current Intel manual runs to 28 pages.
- *
- * This instruction even it has its own Wikipedia entry. The Wikipedia entry
- * has been translated into 4 languages. I am not making this up!
- *
- * We could get funky here and identify ourselves as "GenuineLguest",
but
- * instead we just use the real "cpuid" instruction. Then I pretty
much turned
- * off feature bits until the Guest booted. (Don't say that: you'll
damage
- * lguest sales!) Shut up, inner voice! (Hey, just pointing out that this is
- * hardly future proof.) Noone's listening! They don't like you
anyway,
- * parenthetic weirdo!
- *
- * Replacing the cpuid so we can turn features off is great for the kernel, but
- * anyone (including userspace) can just use the raw "cpuid"
instruction and
- * the Host won't even notice since it isn't privileged. So we try not
to get
- * too worked up about it. */
-static void lguest_cpuid(unsigned int *eax, unsigned int *ebx,
- unsigned int *ecx, unsigned int *edx)
-{
- int function = *eax;
-
- native_cpuid(eax, ebx, ecx, edx);
- switch (function) {
- case 1: /* Basic feature request. */
- /* We only allow kernel to see SSE3, CMPXCHG16B and SSSE3 */
- *ecx &= 0x00002201;
- /* SSE, SSE2, FXSR, MMX, CMOV, CMPXCHG8B, FPU. */
- *edx &= 0x07808101;
- /* The Host can do a nice optimization if it knows that the
- * kernel mappings (addresses above 0xC0000000 or whatever
- * PAGE_OFFSET is set to) haven't changed. But Linux calls
- * flush_tlb_user() for both user and kernel mappings unless
- * the Page Global Enable (PGE) feature bit is set. */
- *edx |= 0x00002000;
- break;
- case 0x80000000:
- /* Futureproof this a little: if they ask how much extended
- * processor information there is, limit it to known fields. */
- if (*eax > 0x80000008)
- *eax = 0x80000008;
- break;
- }
-}
-
-/* Intel has four control registers, imaginatively named cr0, cr2, cr3 and cr4.
- * I assume there's a cr1, but it hasn't bothered us yet, so we'll
not bother
- * it. The Host needs to know when the Guest wants to change them, so we have
- * a whole series of functions like read_cr0() and write_cr0().
- *
- * We start with CR0. CR0 allows you to turn on and off all kinds of basic
- * features, but Linux only really cares about one: the horrifically-named Task
- * Switched (TS) bit at bit 3 (ie. 8)
- *
- * What does the TS bit do? Well, it causes the CPU to trap (interrupt 7) if
- * the floating point unit is used. Which allows us to restore FPU state
- * lazily after a task switch, and Linux uses that gratefully, but wouldn't
a
- * name like "FPUTRAP bit" be a little less cryptic?
- *
- * We store cr0 (and cr3) locally, because the Host never changes it. The
- * Guest sometimes wants to read it and we'd prefer not to bother the Host
- * unnecessarily. */
-static unsigned long current_cr0, current_cr3;
-static void lguest_write_cr0(unsigned long val)
-{
- /* 8 == TS bit. */
- lazy_hcall(LHCALL_TS, val & 8, 0, 0);
- current_cr0 = val;
-}
-
-static unsigned long lguest_read_cr0(void)
-{
- return current_cr0;
-}
-
-/* Intel provided a special instruction to clear the TS bit for people too cool
- * to use write_cr0() to do it. This "clts" instruction is faster,
because all
- * the vowels have been optimized out. */
-static void lguest_clts(void)
-{
- lazy_hcall(LHCALL_TS, 0, 0, 0);
- current_cr0 &= ~8U;
-}
-
-/* CR2 is the virtual address of the last page fault, which the Guest only ever
- * reads. The Host kindly writes this into our "struct lguest_data",
so we
- * just read it out of there. */
-static unsigned long lguest_read_cr2(void)
-{
- return lguest_data.cr2;
-}
-
-/* CR3 is the current toplevel pagetable page: the principle is the same as
- * cr0. Keep a local copy, and tell the Host when it changes. */
-static void lguest_write_cr3(unsigned long cr3)
-{
- lazy_hcall(LHCALL_NEW_PGTABLE, cr3, 0, 0);
- current_cr3 = cr3;
-}
-
-static unsigned long lguest_read_cr3(void)
-{
- return current_cr3;
-}
-
-/* CR4 is used to enable and disable PGE, but we don't care. */
-static unsigned long lguest_read_cr4(void)
-{
- return 0;
-}
-
-static void lguest_write_cr4(unsigned long val)
-{
-}
-
-/*
- * Page Table Handling.
- *
- * Now would be a good time to take a rest and grab a coffee or similarly
- * relaxing stimulant. The easy parts are behind us, and the trek gradually
- * winds uphill from here.
- *
- * Quick refresher: memory is divided into "pages" of 4096 bytes
each. The CPU
- * maps virtual addresses to physical addresses using "page tables".
We could
- * use one huge index of 1 million entries: each address is 4 bytes, so
that's
- * 1024 pages just to hold the page tables. But since most virtual addresses
- * are unused, we use a two level index which saves space. The CR3 register
- * contains the physical address of the top level "page directory"
page, which
- * contains physical addresses of up to 1024 second-level pages. Each of these
- * second level pages contains up to 1024 physical addresses of actual pages,
- * or Page Table Entries (PTEs).
- *
- * Here's a diagram, where arrows indicate physical addresses:
- *
- * CR3 ---> +---------+
- * | --------->+---------+
- * | | | PADDR1 |
- * Top-level | | PADDR2 |
- * (PMD) page | | |
- * | | Lower-level |
- * | | (PTE) page |
- * | | | |
- * .... ....
- *
- * So to convert a virtual address to a physical address, we look up the top
- * level, which points us to the second level, which gives us the physical
- * address of that page. If the top level entry was not present, or the second
- * level entry was not present, then the virtual address is invalid (we
- * say "the page was not mapped").
- *
- * Put another way, a 32-bit virtual address is divided up like so:
- *
- * 1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
- * |<---- 10 bits ---->|<---- 10 bits ---->|<------ 12 bits
------>|
- * Index into top Index into second Offset within page
- * page directory page pagetable page
- *
- * The kernel spends a lot of time changing both the top-level page directory
- * and lower-level pagetable pages. The Guest doesn't know physical
addresses,
- * so while it maintains these page tables exactly like normal, it also needs
- * to keep the Host informed whenever it makes a change: the Host will create
- * the real page tables based on the Guests'.
- */
-
-/* The Guest calls this to set a second-level entry (pte), ie. to map a page
- * into a process' address space. We set the entry then tell the Host the
- * toplevel and address this corresponds to. The Guest uses one pagetable per
- * process, so we need to tell the Host which one we're changing
(mm->pgd). */
-static void lguest_set_pte_at(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t pteval)
-{
- *ptep = pteval;
- lazy_hcall(LHCALL_SET_PTE, __pa(mm->pgd), addr, pteval.pte_low);
-}
-
-/* The Guest calls this to set a top-level entry. Again, we set the entry then
- * tell the Host which top-level page we changed, and the index of the entry we
- * changed. */
-static void lguest_set_pmd(pmd_t *pmdp, pmd_t pmdval)
-{
- *pmdp = pmdval;
- lazy_hcall(LHCALL_SET_PMD, __pa(pmdp)&PAGE_MASK,
- (__pa(pmdp)&(PAGE_SIZE-1))/4, 0);
-}
-
-/* There are a couple of legacy places where the kernel sets a PTE, but we
- * don't know the top level any more. This is useless for us, since we
don't
- * know which pagetable is changing or what address, so we just tell the Host
- * to forget all of them. Fortunately, this is very rare.
- *
- * ... except in early boot when the kernel sets up the initial pagetables,
- * which makes booting astonishingly slow. So we don't even tell the Host
- * anything changed until we've done the first page table switch.
- */
-static void lguest_set_pte(pte_t *ptep, pte_t pteval)
-{
- *ptep = pteval;
- /* Don't bother with hypercall before initial setup. */
- if (current_cr3)
- lazy_hcall(LHCALL_FLUSH_TLB, 1, 0, 0);
-}
-
-/* Unfortunately for Lguest, the paravirt_ops for page tables were based on
- * native page table operations. On native hardware you can set a new page
- * table entry whenever you want, but if you want to remove one you have to do
- * a TLB flush (a TLB is a little cache of page table entries kept by the CPU).
- *
- * So the lguest_set_pte_at() and lguest_set_pmd() functions above are only
- * called when a valid entry is written, not when it's removed (ie. marked
not
- * present). Instead, this is where we come when the Guest wants to remove a
- * page table entry: we tell the Host to set that entry to 0 (ie. the present
- * bit is zero). */
-static void lguest_flush_tlb_single(unsigned long addr)
-{
- /* Simply set it to zero: if it was not, it will fault back in. */
- lazy_hcall(LHCALL_SET_PTE, current_cr3, addr, 0);
-}
-
-/* This is what happens after the Guest has removed a large number of entries.
- * This tells the Host that any of the page table entries for userspace might
- * have changed, ie. virtual addresses below PAGE_OFFSET. */
-static void lguest_flush_tlb_user(void)
-{
- lazy_hcall(LHCALL_FLUSH_TLB, 0, 0, 0);
-}
-
-/* This is called when the kernel page tables have changed. That's not
very
- * common (unless the Guest is using highmem, which makes the Guest extremely
- * slow), so it's worth separating this from the user flushing above. */
-static void lguest_flush_tlb_kernel(void)
-{
- lazy_hcall(LHCALL_FLUSH_TLB, 1, 0, 0);
-}
-
-/*
- * The Unadvanced Programmable Interrupt Controller.
- *
- * This is an attempt to implement the simplest possible interrupt controller.
- * I spent some time looking though routines like set_irq_chip_and_handler,
- * set_irq_chip_and_handler_name, set_irq_chip_data and set_phasers_to_stun and
- * I *think* this is as simple as it gets.
- *
- * We can tell the Host what interrupts we want blocked ready for using the
- * lguest_data.interrupts bitmap, so disabling (aka "masking") them
is as
- * simple as setting a bit. We don't actually "ack" interrupts
as such, we
- * just mask and unmask them. I wonder if we should be cleverer?
- */
-static void disable_lguest_irq(unsigned int irq)
-{
- set_bit(irq, lguest_data.blocked_interrupts);
-}
-
-static void enable_lguest_irq(unsigned int irq)
-{
- clear_bit(irq, lguest_data.blocked_interrupts);
-}
-
-/* This structure describes the lguest IRQ controller. */
-static struct irq_chip lguest_irq_controller = {
- .name = "lguest",
- .mask = disable_lguest_irq,
- .mask_ack = disable_lguest_irq,
- .unmask = enable_lguest_irq,
-};
-
-/* This sets up the Interrupt Descriptor Table (IDT) entry for each hardware
- * interrupt (except 128, which is used for system calls), and then tells the
- * Linux infrastructure that each interrupt is controlled by our level-based
- * lguest interrupt controller. */
-static void __init lguest_init_IRQ(void)
-{
- unsigned int i;
-
- for (i = 0; i < LGUEST_IRQS; i++) {
- int vector = FIRST_EXTERNAL_VECTOR + i;
- if (vector != SYSCALL_VECTOR) {
- set_intr_gate(vector, interrupt[i]);
- set_irq_chip_and_handler(i, &lguest_irq_controller,
- handle_level_irq);
- }
- }
- /* This call is required to set up for 4k stacks, where we have
- * separate stacks for hard and soft interrupts. */
- irq_ctx_init(smp_processor_id());
-}
-
-/*
- * Time.
- *
- * It would be far better for everyone if the Guest had its own clock, but
- * until then the Host gives us the time on every interrupt.
- */
-static unsigned long lguest_get_wallclock(void)
-{
- return lguest_data.time.tv_sec;
-}
-
-static cycle_t lguest_clock_read(void)
-{
- unsigned long sec, nsec;
-
- /* If the Host tells the TSC speed, we can trust that. */
- if (lguest_data.tsc_khz)
- return native_read_tsc();
-
- /* If we can't use the TSC, we read the time value written by the Host.
- * Since it's in two parts (seconds and nanoseconds), we risk reading
- * it just as it's changing from 99 & 0.999999999 to 100 and 0, and
- * getting 99 and 0. As Linux tends to come apart under the stress of
- * time travel, we must be careful: */
- do {
- /* First we read the seconds part. */
- sec = lguest_data.time.tv_sec;
- /* This read memory barrier tells the compiler and the CPU that
- * this can't be reordered: we have to complete the above
- * before going on. */
- rmb();
- /* Now we read the nanoseconds part. */
- nsec = lguest_data.time.tv_nsec;
- /* Make sure we've done that. */
- rmb();
- /* Now if the seconds part has changed, try again. */
- } while (unlikely(lguest_data.time.tv_sec != sec));
-
- /* Our non-TSC clock is in real nanoseconds. */
- return sec*1000000000ULL + nsec;
-}
-
-/* This is what we tell the kernel is our clocksource. */
-static struct clocksource lguest_clock = {
- .name = "lguest",
- .rating = 400,
- .read = lguest_clock_read,
- .mask = CLOCKSOURCE_MASK(64),
- .mult = 1 << 22,
- .shift = 22,
-};
-
-/* The "scheduler clock" is just our real clock, adjusted to start at
zero */
-static unsigned long long lguest_sched_clock(void)
-{
- return cyc2ns(&lguest_clock, lguest_clock_read() - clock_base);
-}
-
-/* We also need a "struct clock_event_device": Linux asks us to set
it to go
- * off some time in the future. Actually, James Morris figured all this out, I
- * just applied the patch. */
-static int lguest_clockevent_set_next_event(unsigned long delta,
- struct clock_event_device *evt)
-{
- if (delta < LG_CLOCK_MIN_DELTA) {
- if (printk_ratelimit())
- printk(KERN_DEBUG "%s: small delta %lu ns\n",
- __FUNCTION__, delta);
- return -ETIME;
- }
- hcall(LHCALL_SET_CLOCKEVENT, delta, 0, 0);
- return 0;
-}
-
-static void lguest_clockevent_set_mode(enum clock_event_mode mode,
- struct clock_event_device *evt)
-{
- switch (mode) {
- case CLOCK_EVT_MODE_UNUSED:
- case CLOCK_EVT_MODE_SHUTDOWN:
- /* A 0 argument shuts the clock down. */
- hcall(LHCALL_SET_CLOCKEVENT, 0, 0, 0);
- break;
- case CLOCK_EVT_MODE_ONESHOT:
- /* This is what we expect. */
- break;
- case CLOCK_EVT_MODE_PERIODIC:
- BUG();
- case CLOCK_EVT_MODE_RESUME:
- break;
- }
-}
-
-/* This describes our primitive timer chip. */
-static struct clock_event_device lguest_clockevent = {
- .name = "lguest",
- .features = CLOCK_EVT_FEAT_ONESHOT,
- .set_next_event = lguest_clockevent_set_next_event,
- .set_mode = lguest_clockevent_set_mode,
- .rating = INT_MAX,
- .mult = 1,
- .shift = 0,
- .min_delta_ns = LG_CLOCK_MIN_DELTA,
- .max_delta_ns = LG_CLOCK_MAX_DELTA,
-};
-
-/* This is the Guest timer interrupt handler (hardware interrupt 0). We just
- * call the clockevent infrastructure and it does whatever needs doing. */
-static void lguest_time_irq(unsigned int irq, struct irq_desc *desc)
-{
- unsigned long flags;
-
- /* Don't interrupt us while this is running. */
- local_irq_save(flags);
- lguest_clockevent.event_handler(&lguest_clockevent);
- local_irq_restore(flags);
-}
-
-/* At some point in the boot process, we get asked to set up our timing
- * infrastructure. The kernel doesn't expect timer interrupts before this,
but
- * we cleverly initialized the "blocked_interrupts" field of
"struct
- * lguest_data" so that timer interrupts were blocked until now. */
-static void lguest_time_init(void)
-{
- /* Set up the timer interrupt (0) to go to our simple timer routine */
- set_irq_handler(0, lguest_time_irq);
-
- /* Our clock structure look like arch/i386/kernel/tsc.c if we can use
- * the TSC, otherwise it's a dumb nanosecond-resolution clock. Either
- * way, the "rating" is initialized so high that it's always
chosen
- * over any other clocksource. */
- if (lguest_data.tsc_khz) {
- lguest_clock.mult = clocksource_khz2mult(lguest_data.tsc_khz,
- lguest_clock.shift);
- lguest_clock.flags = CLOCK_SOURCE_IS_CONTINUOUS;
- }
- clock_base = lguest_clock_read();
- clocksource_register(&lguest_clock);
-
- /* Now we've set up our clock, we can use it as the scheduler clock */
- paravirt_ops.sched_clock = lguest_sched_clock;
-
- /* We can't set cpumask in the initializer: damn C limitations! Set it
- * here and register our timer device. */
- lguest_clockevent.cpumask = cpumask_of_cpu(0);
- clockevents_register_device(&lguest_clockevent);
-
- /* Finally, we unblock the timer interrupt. */
- enable_lguest_irq(0);
-}
-
-/*
- * Miscellaneous bits and pieces.
- *
- * Here is an oddball collection of functions which the Guest needs for things
- * to work. They're pretty simple.
- */
-
-/* The Guest needs to tell the host what stack it expects traps to use. For
- * native hardware, this is part of the Task State Segment mentioned above in
- * lguest_load_tr_desc(), but to help hypervisors there's this special
call.
- *
- * We tell the Host the segment we want to use (__KERNEL_DS is the kernel data
- * segment), the privilege level (we're privilege level 1, the Host is 0
and
- * will not tolerate us trying to use that), the stack pointer, and the number
- * of pages in the stack. */
-static void lguest_load_esp0(struct tss_struct *tss,
- struct thread_struct *thread)
-{
- lazy_hcall(LHCALL_SET_STACK, __KERNEL_DS|0x1, thread->esp0,
- THREAD_SIZE/PAGE_SIZE);
-}
-
-/* Let's just say, I wouldn't do debugging under a Guest. */
-static void lguest_set_debugreg(int regno, unsigned long value)
-{
- /* FIXME: Implement */
-}
-
-/* There are times when the kernel wants to make sure that no memory writes are
- * caught in the cache (that they've all reached real hardware devices).
This
- * doesn't matter for the Guest which has virtual hardware.
- *
- * On the Pentium 4 and above, cpuid() indicates that the Cache Line Flush
- * (clflush) instruction is available and the kernel uses that. Otherwise, it
- * uses the older "Write Back and Invalidate Cache" (wbinvd)
instruction.
- * Unlike clflush, wbinvd can only be run at privilege level 0. So we can
- * ignore clflush, but replace wbinvd.
- */
-static void lguest_wbinvd(void)
-{
-}
-
-/* If the Guest expects to have an Advanced Programmable Interrupt Controller,
- * we play dumb by ignoring writes and returning 0 for reads. So it's no
- * longer Programmable nor Controlling anything, and I don't think 8 lines
of
- * code qualifies for Advanced. It will also never interrupt anything. It
- * does, however, allow us to get through the Linux boot code. */
-#ifdef CONFIG_X86_LOCAL_APIC
-static void lguest_apic_write(unsigned long reg, unsigned long v)
-{
-}
-
-static unsigned long lguest_apic_read(unsigned long reg)
-{
- return 0;
-}
-#endif
-
-/* STOP! Until an interrupt comes in. */
-static void lguest_safe_halt(void)
-{
- hcall(LHCALL_HALT, 0, 0, 0);
-}
-
-/* Perhaps CRASH isn't the best name for this hypercall, but we use it to
get a
- * message out when we're crashing as well as elegant termination like
powering
- * off.
- *
- * Note that the Host always prefers that the Guest speak in physical addresses
- * rather than virtual addresses, so we use __pa() here. */
-static void lguest_power_off(void)
-{
- hcall(LHCALL_CRASH, __pa("Power down"), 0, 0);
-}
-
-/*
- * Panicing.
- *
- * Don't. But if you did, this is what happens.
- */
-static int lguest_panic(struct notifier_block *nb, unsigned long l, void *p)
-{
- hcall(LHCALL_CRASH, __pa(p), 0, 0);
- /* The hcall won't return, but to keep gcc happy, we're
"done". */
- return NOTIFY_DONE;
-}
-
-static struct notifier_block paniced = {
- .notifier_call = lguest_panic
-};
-
-/* Setting up memory is fairly easy. */
-static __init char *lguest_memory_setup(void)
-{
- /* We do this here and not earlier because lockcheck barfs if we do it
- * before start_kernel() */
- atomic_notifier_chain_register(&panic_notifier_list, &paniced);
-
- /* The Linux bootloader header contains an "e820" memory map: the
- * Launcher populated the first entry with our memory limit. */
- add_memory_region(E820_MAP->addr, E820_MAP->size, E820_MAP->type);
-
- /* This string is for the boot messages. */
- return "LGUEST";
-}
-
-/*G:050
- * Patching (Powerfully Placating Performance Pedants)
- *
- * We have already seen that "struct paravirt_ops" lets us replace
simple
- * native instructions with calls to the appropriate back end all throughout
- * the kernel. This allows the same kernel to run as a Guest and as a native
- * kernel, but it's slow because of all the indirect branches.
- *
- * Remember that David Wheeler quote about "Any problem in computer
science can
- * be solved with another layer of indirection"? The rest of that quote
is
- * "... But that usually will create another problem." This is the
first of
- * those problems.
- *
- * Our current solution is to allow the paravirt back end to optionally patch
- * over the indirect calls to replace them with something more efficient. We
- * patch the four most commonly called functions: disable interrupts, enable
- * interrupts, restore interrupts and save interrupts. We usually have 10
- * bytes to patch into: the Guest versions of these operations are small enough
- * that we can fit comfortably.
- *
- * First we need assembly templates of each of the patchable Guest operations,
- * and these are in lguest_asm.S. */
-
-/*G:060 We construct a table from the assembler templates: */
-static const struct lguest_insns
-{
- const char *start, *end;
-} lguest_insns[] = {
- [PARAVIRT_PATCH(irq_disable)] = { lgstart_cli, lgend_cli },
- [PARAVIRT_PATCH(irq_enable)] = { lgstart_sti, lgend_sti },
- [PARAVIRT_PATCH(restore_fl)] = { lgstart_popf, lgend_popf },
- [PARAVIRT_PATCH(save_fl)] = { lgstart_pushf, lgend_pushf },
-};
-
-/* Now our patch routine is fairly simple (based on the native one in
- * paravirt.c). If we have a replacement, we copy it in and return how much of
- * the available space we used. */
-static unsigned lguest_patch(u8 type, u16 clobber, void *ibuf,
- unsigned long addr, unsigned len)
-{
- unsigned int insn_len;
-
- /* Don't do anything special if we don't have a replacement */
- if (type >= ARRAY_SIZE(lguest_insns) || !lguest_insns[type].start)
- return paravirt_patch_default(type, clobber, ibuf, addr, len);
-
- insn_len = lguest_insns[type].end - lguest_insns[type].start;
-
- /* Similarly if we can't fit replacement (shouldn't happen, but
let's
- * be thorough). */
- if (len < insn_len)
- return paravirt_patch_default(type, clobber, ibuf, addr, len);
-
- /* Copy in our instructions. */
- memcpy(ibuf, lguest_insns[type].start, insn_len);
- return insn_len;
-}
-
-/*G:030 Once we get to lguest_init(), we know we're a Guest. The
paravirt_ops
- * structure in the kernel provides a single point for (almost) every routine
- * we have to override to avoid privileged instructions. */
-__init void lguest_init(void *boot)
-{
- /* Copy boot parameters first: the Launcher put the physical location
- * in %esi, and head.S converted that to a virtual address and handed
- * it to us. We use "__memcpy" because "memcpy" sometimes
tries to do
- * tricky things to go faster, and we're not ready for that. */
- __memcpy(&boot_params, boot, PARAM_SIZE);
- /* The boot parameters also tell us where the command-line is: save
- * that, too. */
- __memcpy(boot_command_line, __va(boot_params.hdr.cmd_line_ptr),
- COMMAND_LINE_SIZE);
-
- /* We're under lguest, paravirt is enabled, and we're running at
- * privilege level 1, not 0 as normal. */
- paravirt_ops.name = "lguest";
- paravirt_ops.paravirt_enabled = 1;
- paravirt_ops.kernel_rpl = 1;
-
- /* We set up all the lguest overrides for sensitive operations. These
- * are detailed with the operations themselves. */
- paravirt_ops.save_fl = save_fl;
- paravirt_ops.restore_fl = restore_fl;
- paravirt_ops.irq_disable = irq_disable;
- paravirt_ops.irq_enable = irq_enable;
- paravirt_ops.load_gdt = lguest_load_gdt;
- paravirt_ops.memory_setup = lguest_memory_setup;
- paravirt_ops.cpuid = lguest_cpuid;
- paravirt_ops.write_cr3 = lguest_write_cr3;
- paravirt_ops.flush_tlb_user = lguest_flush_tlb_user;
- paravirt_ops.flush_tlb_single = lguest_flush_tlb_single;
- paravirt_ops.flush_tlb_kernel = lguest_flush_tlb_kernel;
- paravirt_ops.set_pte = lguest_set_pte;
- paravirt_ops.set_pte_at = lguest_set_pte_at;
- paravirt_ops.set_pmd = lguest_set_pmd;
-#ifdef CONFIG_X86_LOCAL_APIC
- paravirt_ops.apic_write = lguest_apic_write;
- paravirt_ops.apic_write_atomic = lguest_apic_write;
- paravirt_ops.apic_read = lguest_apic_read;
-#endif
- paravirt_ops.load_idt = lguest_load_idt;
- paravirt_ops.iret = lguest_iret;
- paravirt_ops.load_esp0 = lguest_load_esp0;
- paravirt_ops.load_tr_desc = lguest_load_tr_desc;
- paravirt_ops.set_ldt = lguest_set_ldt;
- paravirt_ops.load_tls = lguest_load_tls;
- paravirt_ops.set_debugreg = lguest_set_debugreg;
- paravirt_ops.clts = lguest_clts;
- paravirt_ops.read_cr0 = lguest_read_cr0;
- paravirt_ops.write_cr0 = lguest_write_cr0;
- paravirt_ops.init_IRQ = lguest_init_IRQ;
- paravirt_ops.read_cr2 = lguest_read_cr2;
- paravirt_ops.read_cr3 = lguest_read_cr3;
- paravirt_ops.read_cr4 = lguest_read_cr4;
- paravirt_ops.write_cr4 = lguest_write_cr4;
- paravirt_ops.write_gdt_entry = lguest_write_gdt_entry;
- paravirt_ops.write_idt_entry = lguest_write_idt_entry;
- paravirt_ops.patch = lguest_patch;
- paravirt_ops.safe_halt = lguest_safe_halt;
- paravirt_ops.get_wallclock = lguest_get_wallclock;
- paravirt_ops.time_init = lguest_time_init;
- paravirt_ops.set_lazy_mode = lguest_lazy_mode;
- paravirt_ops.wbinvd = lguest_wbinvd;
- /* Now is a good time to look at the implementations of these functions
- * before returning to the rest of lguest_init(). */
-
- /*G:070 Now we've seen all the paravirt_ops, we return to
- * lguest_init() where the rest of the fairly chaotic boot setup
- * occurs.
- *
- * The Host expects our first hypercall to tell it where our "struct
- * lguest_data" is, so we do that first. */
- hcall(LHCALL_LGUEST_INIT, __pa(&lguest_data), 0, 0);
-
- /* The native boot code sets up initial page tables immediately after
- * the kernel itself, and sets init_pg_tables_end so they're not
- * clobbered. The Launcher places our initial pagetables somewhere at
- * the top of our physical memory, so we don't need extra space: set
- * init_pg_tables_end to the end of the kernel. */
- init_pg_tables_end = __pa(pg0);
-
- /* Load the %fs segment register (the per-cpu segment register) with
- * the normal data segment to get through booting. */
- asm volatile ("mov %0, %%fs" : : "r" (__KERNEL_DS) :
"memory");
-
- /* Clear the part of the kernel data which is expected to be zero.
- * Normally it will be anyway, but if we're loading from a bzImage with
- * CONFIG_RELOCATALE=y, the relocations will be sitting here. */
- memset(__bss_start, 0, __bss_stop - __bss_start);
-
- /* The Host uses the top of the Guest's virtual address space for the
- * Host<->Guest Switcher, and it tells us how much it needs in
- * lguest_data.reserve_mem, set up on the LGUEST_INIT hypercall. */
- reserve_top_address(lguest_data.reserve_mem);
-
- /* If we don't initialize the lock dependency checker now, it crashes
- * paravirt_disable_iospace. */
- lockdep_init();
-
- /* The IDE code spends about 3 seconds probing for disks: if we reserve
- * all the I/O ports up front it can't get them and so doesn't probe.
- * Other device drivers are similar (but less severe). This cuts the
- * kernel boot time on my machine from 4.1 seconds to 0.45 seconds. */
- paravirt_disable_iospace();
-
- /* This is messy CPU setup stuff which the native boot code does before
- * start_kernel, so we have to do, too: */
- cpu_detect(&new_cpu_data);
- /* head.S usually sets up the first capability word, so do it here. */
- new_cpu_data.x86_capability[0] = cpuid_edx(1);
-
- /* Math is always hard! */
- new_cpu_data.hard_math = 1;
-
-#ifdef CONFIG_X86_MCE
- mce_disabled = 1;
-#endif
-#ifdef CONFIG_ACPI
- acpi_disabled = 1;
- acpi_ht = 0;
-#endif
-
- /* We set the perferred console to "hvc". This is the
"hypervisor
- * virtual console" driver written by the PowerPC people, which we also
- * adapted for lguest's use. */
- add_preferred_console("hvc", 0, NULL);
-
- /* Last of all, we set the power management poweroff hook to point to
- * the Guest routine to power off. */
- pm_power_off = lguest_power_off;
-
- /* Now we're set up, call start_kernel() in init/main.c and we proceed
- * to boot as normal. It never returns. */
- start_kernel();
-}
-/*
- * This marks the end of stage II of our journey, The Guest.
- *
- * It is now time for us to explore the nooks and crannies of the three Guest
- * devices and complete our understanding of the Guest in "make
Drivers".
- */
==================================================================---
a/drivers/lguest/lguest_asm.S
+++ /dev/null
@@ -1,93 +0,0 @@
-#include <linux/linkage.h>
-#include <linux/lguest.h>
-#include <asm/asm-offsets.h>
-#include <asm/thread_info.h>
-#include <asm/processor-flags.h>
-
-/*G:020 This is where we begin: we have a magic signature which the launcher
- * looks for. The plan is that the Linux boot protocol will be extended with a
- * "platform type" field which will guide us here from the normal
entry point,
- * but for the moment this suffices. The normal boot code uses %esi for the
- * boot header, so we do too. We convert it to a virtual address by adding
- * PAGE_OFFSET, and hand it to lguest_init() as its argument (ie. %eax).
- *
- * The .section line puts this code in .init.text so it will be discarded after
- * boot. */
-.section .init.text, "ax", @progbits
-.ascii "GenuineLguest"
- /* Set up initial stack. */
- movl $(init_thread_union+THREAD_SIZE),%esp
- movl %esi, %eax
- addl $__PAGE_OFFSET, %eax
- jmp lguest_init
-
-/*G:055 We create a macro which puts the assembler code between lgstart_ and
- * lgend_ markers. These templates are put in the .text section: they
can't be
- * discarded after boot as we may need to patch modules, too. */
-.text
-#define LGUEST_PATCH(name, insns...) \
- lgstart_##name: insns; lgend_##name:; \
- .globl lgstart_##name; .globl lgend_##name
-
-LGUEST_PATCH(cli, movl $0, lguest_data+LGUEST_DATA_irq_enabled)
-LGUEST_PATCH(sti, movl $X86_EFLAGS_IF, lguest_data+LGUEST_DATA_irq_enabled)
-LGUEST_PATCH(popf, movl %eax, lguest_data+LGUEST_DATA_irq_enabled)
-LGUEST_PATCH(pushf, movl lguest_data+LGUEST_DATA_irq_enabled, %eax)
-/*:*/
-
-/* These demark the EIP range where host should never deliver interrupts. */
-.global lguest_noirq_start
-.global lguest_noirq_end
-
-/*M:004 When the Host reflects a trap or injects an interrupt into the Guest,
- * it sets the eflags interrupt bit on the stack based on
- * lguest_data.irq_enabled, so the Guest iret logic does the right thing when
- * restoring it. However, when the Host sets the Guest up for direct traps,
- * such as system calls, the processor is the one to push eflags onto the
- * stack, and the interrupt bit will be 1 (in reality, interrupts are always
- * enabled in the Guest).
- *
- * This turns out to be harmless: the only trap which should happen under Linux
- * with interrupts disabled is Page Fault (due to our lazy mapping of vmalloc
- * regions), which has to be reflected through the Host anyway. If another
- * trap *does* go off when interrupts are disabled, the Guest will panic, and
- * we'll never get to this iret! :*/
-
-/*G:045 There is one final paravirt_op that the Guest implements, and glancing
- * at it you can see why I left it to last. It's *cool*! It's in
*assembler*!
- *
- * The "iret" instruction is used to return from an interrupt or
trap. The
- * stack looks like this:
- * old address
- * old code segment & privilege level
- * old processor flags ("eflags")
- *
- * The "iret" instruction pops those values off the stack and
restores them all
- * at once. The only problem is that eflags includes the Interrupt Flag which
- * the Guest can't change: the CPU will simply ignore it when we do an
"iret".
- * So we have to copy eflags from the stack to lguest_data.irq_enabled before
- * we do the "iret".
- *
- * There are two problems with this: firstly, we need to use a register to do
- * the copy and secondly, the whole thing needs to be atomic. The first
- * problem is easy to solve: push %eax on the stack so we can use it, and then
- * restore it at the end just before the real "iret".
- *
- * The second is harder: copying eflags to lguest_data.irq_enabled will turn
- * interrupts on before we're finished, so we could be interrupted before
we
- * return to userspace or wherever. Our solution to this is to surround the
- * code with lguest_noirq_start: and lguest_noirq_end: labels. We tell the
- * Host that it is *never* to interrupt us there, even if interrupts seem to be
- * enabled. */
-ENTRY(lguest_iret)
- pushl %eax
- movl 12(%esp), %eax
-lguest_noirq_start:
- /* Note the %ss: segment prefix here. Normal data accesses use the
- * "ds" segment, but that will have already been restored for
whatever
- * we're returning to (such as userspace): we can't trust it. The
%ss:
- * prefix makes sure we use the stack segment, which is still valid. */
- movl %eax,%ss:lguest_data+LGUEST_DATA_irq_enabled
- popl %eax
- iret
-lguest_noirq_end:
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:20 UTC
[patch 19/43] lguest: Make shadow IDT a complete IDT with 256 entries.
This simplifies the code a little, in preparation for allowing
alternate system call vectors in guests (Plan 9 uses 0x40).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
drivers/lguest/interrupts_and_traps.c | 49 ++++++++++++---------------------
drivers/lguest/lg.h | 3 --
2 files changed, 20 insertions(+), 32 deletions(-)
==================================================================---
a/drivers/lguest/interrupts_and_traps.c
+++ b/drivers/lguest/interrupts_and_traps.c
@@ -215,10 +215,9 @@ int deliver_trap(struct lguest *lg, unsi
* system calls down from 1750ns to 270ns. Plus, if lguest didn't do it,
all
* the other hypervisors would tease it.
*
- * This routine determines if a trap can be delivered directly. */
-static int direct_trap(const struct lguest *lg,
- const struct desc_struct *trap,
- unsigned int num)
+ * This routine indicates if a particular trap number could be delivered
+ * directly. */
+static int direct_trap(unsigned int num)
{
/* Hardware interrupts don't go to the Guest at all (except system
* call). */
@@ -229,14 +228,7 @@ static int direct_trap(const struct lgue
* fault address), general protection faults (in/out emulation) and
* device not available (TS handling), and of course, the hypercall
* trap. */
- if (num == 14 || num == 13 || num == 7 || num == LGUEST_TRAP_ENTRY)
- return 0;
-
- /* Only trap gates (type 15) can go direct to the Guest. Interrupt
- * gates (type 14) disable interrupts as they are entered, which we
- * never let the Guest do. Not present entries (type 0x0) also can't
- * go direct, of course 8) */
- return idt_type(trap->a, trap->b) == 0xF;
+ return num != 14 && num != 13 && num != 7 && num !=
LGUEST_TRAP_ENTRY;
}
/*:*/
@@ -342,15 +334,11 @@ void load_guest_idt_entry(struct lguest
* to copy this again. */
lg->changed |= CHANGED_IDT;
- /* The IDT which we keep in "struct lguest" only contains 32 entries
- * for the traps and LGUEST_IRQS (32) entries for interrupts. We
- * ignore attempts to set handlers for higher interrupt numbers, except
- * for the system call "interrupt" at 128: we have a special IDT
entry
- * for that. */
- if (num < ARRAY_SIZE(lg->idt))
+ /* Check that the Guest doesn't try to step outside the bounds. */
+ if (num >= ARRAY_SIZE(lg->idt))
+ kill_guest(lg, "Setting idt entry %u", num);
+ else
set_trap(lg, &lg->idt[num], num, lo, hi);
- else if (num == SYSCALL_VECTOR)
- set_trap(lg, &lg->syscall_idt, num, lo, hi);
}
/* The default entry for each interrupt points into the Switcher routines which
@@ -393,20 +381,21 @@ void copy_traps(const struct lguest *lg,
/* We can simply copy the direct traps, otherwise we use the default
* ones in the Switcher: they will return to the Host. */
- for (i = 0; i < FIRST_EXTERNAL_VECTOR; i++) {
- if (direct_trap(lg, &lg->idt[i], i))
+ for (i = 0; i < ARRAY_SIZE(lg->idt); i++) {
+ /* If no Guest can ever override this trap, leave it alone. */
+ if (!direct_trap(i))
+ continue;
+
+ /* Only trap gates (type 15) can go direct to the Guest.
+ * Interrupt gates (type 14) disable interrupts as they are
+ * entered, which we never let the Guest do. Not present
+ * entries (type 0x0) also can't go direct, of course. */
+ if (idt_type(lg->idt[i].a, lg->idt[i].b) == 0xF)
idt[i] = lg->idt[i];
else
+ /* Reset it to the default. */
default_idt_entry(&idt[i], i, def[i]);
}
-
- /* Don't forget the system call trap! The IDT entries for other
- * interupts never change, so no need to copy them. */
- i = SYSCALL_VECTOR;
- if (direct_trap(lg, &lg->syscall_idt, i))
- idt[i] = lg->syscall_idt;
- else
- default_idt_entry(&idt[i], i, def[i]);
}
void guest_set_clockevent(struct lguest *lg, unsigned long delta)
==================================================================---
a/drivers/lguest/lg.h
+++ b/drivers/lguest/lg.h
@@ -183,8 +183,7 @@ struct lguest
struct desc_struct gdt[GDT_ENTRIES];
/* The IDT entries: some copied into lguest_ro_state when running. */
- struct desc_struct idt[FIRST_EXTERNAL_VECTOR+LGUEST_IRQS];
- struct desc_struct syscall_idt;
+ struct desc_struct idt[IDT_ENTRIES];
/* Virtual clock device */
struct hrtimer hrt;
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:20 UTC
[patch 30/43] lguest: Allow guest to specify syscall vector to use.
(Based on Ron Minnich's LGUEST_PLAN9_SYSCALL patch).
This patch allows Guests to specify what system call vector they want,
and we try to reserve it. We only allow one non-Linux system call
vector, to try to avoid DoS on the Host.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
arch/i386/lguest/boot.c | 1
drivers/lguest/core.c | 30 +++++++++++++-------
drivers/lguest/i386_core.c | 4 ++
drivers/lguest/interrupts_and_traps.c | 49 ++++++++++++++++++++++++++++++++-
drivers/lguest/lg.h | 3 ++
include/linux/lguest.h | 3 ++
6 files changed, 79 insertions(+), 11 deletions(-)
==================================================================---
a/drivers/lguest/core.c
+++ b/drivers/lguest/core.c
@@ -281,37 +281,47 @@ static int __init init(void)
/* First we put the Switcher up in very high virtual memory. */
err = map_switcher();
if (err)
- return err;
+ goto out;
/* Now we set up the pagetable implementation for the Guests. */
err = init_pagetables(switcher_page, SHARED_SWITCHER_PAGES);
- if (err) {
- unmap_switcher();
- return err;
- }
+ if (err)
+ goto unmap;
/* The I/O subsystem needs some things initialized. */
lguest_io_init();
+ /* We might need to reserve an interrupt vector. */
+ err = init_interrupts();
+ if (err)
+ goto free_pgtables;
+
/* /dev/lguest needs to be registered. */
err = lguest_device_init();
- if (err) {
- free_pagetables();
- unmap_switcher();
- return err;
- }
+ if (err)
+ goto free_interrupts;
/* Finally we do some architecture-specific setup. */
lguest_arch_host_init();
/* All good! */
return 0;
+
+free_interrupts:
+ free_interrupts();
+free_pgtables:
+ free_pagetables();
+unmap:
+ unmap_switcher();
+out:
+ return err;
}
/* Cleaning up is just the same code, backwards. With a little French. */
static void __exit fini(void)
{
lguest_device_remove();
+ free_interrupts();
free_pagetables();
unmap_switcher();
==================================================================---
a/drivers/lguest/i386_core.c
+++ b/drivers/lguest/i386_core.c
@@ -530,6 +530,10 @@ int lguest_arch_init_hypercalls(struct l
if (put_user(tsc_speed, &lg->lguest_data->tsc_khz))
return -EFAULT;
+ /* The interrupt code might not like the system call vector. */
+ if (!check_syscall_vector(lg))
+ kill_guest(lg, "bad syscall vector");
+
return 0;
}
/* Now we've examined the hypercall code; our Guest can make requests.
There
==================================================================---
a/arch/i386/lguest/boot.c
+++ b/arch/i386/lguest/boot.c
@@ -87,6 +87,7 @@ struct lguest_data lguest_data = {
.noirq_start = (u32)lguest_noirq_start,
.noirq_end = (u32)lguest_noirq_end,
.blocked_interrupts = { 1 }, /* Block timer interrupts */
+ .syscall_vec = SYSCALL_VECTOR,
};
static cycle_t clock_base;
==================================================================---
a/drivers/lguest/interrupts_and_traps.c
+++ b/drivers/lguest/interrupts_and_traps.c
@@ -12,7 +12,13 @@
* them first, so we also have a way of "reflecting" them into the
Guest as if
* they had been delivered to it directly. :*/
#include <linux/uaccess.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
#include "lg.h"
+
+/* Allow Guests to use a non-128 (ie. non-Linux) syscall trap. */
+static unsigned int syscall_vector = SYSCALL_VECTOR;
+module_param(syscall_vector, uint, 0444);
/* The address of the interrupt handler is split into two bits: */
static unsigned long idt_address(u32 lo, u32 hi)
@@ -183,6 +189,47 @@ void maybe_do_interrupt(struct lguest *l
* timer interrupt. */
write_timestamp(lg);
}
+/*:*/
+
+/* Linux uses trap 128 for system calls. Plan9 uses 64, and Ron Minnich sent
+ * me a patch, so we support that too. It'd be a big step for lguest if
half
+ * the Plan 9 user base were to start using it.
+ *
+ * Actually now I think of it, it's possible that Ron *is* half the Plan 9
+ * userbase. Oh well. */
+static bool could_be_syscall(unsigned int num)
+{
+ /* Normal Linux SYSCALL_VECTOR or reserved vector? */
+ return num == SYSCALL_VECTOR || num == syscall_vector;
+}
+
+/* The syscall vector it wants must be unused by Host. */
+bool check_syscall_vector(struct lguest *lg)
+{
+ u32 vector;
+
+ if (get_user(vector, &lg->lguest_data->syscall_vec))
+ return false;
+
+ return could_be_syscall(vector);
+}
+
+int init_interrupts(void)
+{
+ /* If they want some strange system call vector, reserve it now */
+ if (syscall_vector != SYSCALL_VECTOR
+ && test_and_set_bit(syscall_vector, used_vectors)) {
+ printk("lg: couldn't reserve syscall %u\n", syscall_vector);
+ return -EBUSY;
+ }
+ return 0;
+}
+
+void free_interrupts(void)
+{
+ if (syscall_vector != SYSCALL_VECTOR)
+ clear_bit(syscall_vector, used_vectors);
+}
/*H:220 Now we've got the routines to deliver interrupts, delivering traps
* like page fault is easy. The only trick is that Intel decided that some
@@ -224,7 +271,7 @@ static int direct_trap(unsigned int num)
{
/* Hardware interrupts don't go to the Guest at all (except system
* call). */
- if (num >= FIRST_EXTERNAL_VECTOR && num != SYSCALL_VECTOR)
+ if (num >= FIRST_EXTERNAL_VECTOR && !could_be_syscall(num))
return 0;
/* The Host needs to see page faults (for shadow paging and to save the
==================================================================---
a/drivers/lguest/lg.h
+++ b/drivers/lguest/lg.h
@@ -142,6 +142,9 @@ void copy_traps(const struct lguest *lg,
const unsigned long *def);
void guest_set_clockevent(struct lguest *lg, unsigned long delta);
void init_clockdev(struct lguest *lg);
+bool check_syscall_vector(struct lguest *lg);
+int init_interrupts(void);
+void free_interrupts(void);
/* segments.c: */
void setup_default_gdt_entries(struct lguest_ro_state *state);
==================================================================---
a/include/linux/lguest.h
+++ b/include/linux/lguest.h
@@ -48,6 +48,9 @@ struct lguest_data
/* Fields initialized by the Guest at boot: */
/* Instruction range to suppress interrupts even if enabled */
unsigned long noirq_start, noirq_end;
+
+ /* The vector to try to use for system calls (0x40 or 0x80). */
+ unsigned int syscall_vec;
};
extern struct lguest_data lguest_data;
#endif /* __ASSEMBLY__ */
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:20 UTC
[patch 21/43] lguest: Reorder guest saved regs to match hyperall order
Move eax next to ebx/ecx/edx in struct lguest_regs on i386, so they
will be located together and allow it to map directly to a struct
hcall_ring entry (which will be renamed struct hcall_args as in a
subsequent patch).
This is in preparation for making the code hcall code architecture
independent.
Signed-off-by: Jes Sorensen <jes@sgi.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
drivers/lguest/i386_switcher.S | 4 ++--
include/asm-i386/lguest.h | 3 +--
2 files changed, 3 insertions(+), 4 deletions(-)
==================================================================---
linux-2.6.23-rc3.orig/drivers/lguest/i386_switcher.S
+++ linux-2.6.23-rc3/drivers/lguest/i386_switcher.S
@@ -134,6 +134,7 @@ ENTRY(switch_to_guest)
// The Guest's register page has been mapped
// Writable onto our %esp (stack) --
// We can simply pop off all Guest regs.
+ popl %eax
popl %ebx
popl %ecx
popl %edx
@@ -141,7 +142,6 @@ ENTRY(switch_to_guest)
popl %edi
popl %ebp
popl %gs
- popl %eax
popl %fs
popl %ds
popl %es
@@ -169,7 +169,6 @@ ENTRY(switch_to_guest)
pushl %es; \
pushl %ds; \
pushl %fs; \
- pushl %eax; \
pushl %gs; \
pushl %ebp; \
pushl %edi; \
@@ -177,6 +176,7 @@ ENTRY(switch_to_guest)
pushl %edx; \
pushl %ecx; \
pushl %ebx; \
+ pushl %eax; \
/* Our stack and our code are using segments \
* Set in the TSS and IDT \
* Yet if we were to touch data we'd use \
==================================================================---
linux-2.6.23-rc3.orig/include/asm-i386/lguest.h
+++ linux-2.6.23-rc3/include/asm-i386/lguest.h
@@ -26,10 +26,9 @@ extern unsigned long default_idt_entries
struct lguest_regs
{
/* Manually saved part. */
- unsigned long ebx, ecx, edx;
+ unsigned long eax, ebx, ecx, edx;
unsigned long esi, edi, ebp;
unsigned long gs;
- unsigned long eax;
unsigned long fs, ds, es;
unsigned long trapnum, errcode;
/* Trap pushed part */
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:20 UTC
[patch 20/43] lguest: Move i386 part of core.c to i386_core.c.
Separate i386 architecture specific from core.c and move it to
i386_core.c and add asm-i386/lguest.h header file to match.
Signed-off-by: Jes Sorensen <jes@sgi.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
diff -r 46a53d30a8c0 arch/i386/lguest/boot.c
--- a/arch/i386/lguest/boot.c Wed Sep 26 14:47:35 2007 +1000
+++ b/arch/i386/lguest/boot.c Wed Sep 26 16:07:52 2007 +1000
@@ -65,6 +65,7 @@
#include <asm/e820.h>
#include <asm/mce.h>
#include <asm/io.h>
+#include <asm/i387.h>
/*G:010 Welcome to the Guest!
*
diff -r 46a53d30a8c0 drivers/lguest/Makefile
--- a/drivers/lguest/Makefile Wed Sep 26 14:47:35 2007 +1000
+++ b/drivers/lguest/Makefile Wed Sep 26 16:07:52 2007 +1000
@@ -6,7 +6,7 @@ lg-y = core.o hypercalls.o page_tables.o
lg-y = core.o hypercalls.o page_tables.o interrupts_and_traps.o \
segments.o io.o lguest_user.o
-lg-$(CONFIG_X86_32) += i386_switcher.o
+lg-$(CONFIG_X86_32) += i386_switcher.o i386_core.o
Preparation Preparation!: PREFIX=P
Guest: PREFIX=G
diff -r 46a53d30a8c0 drivers/lguest/core.c
--- a/drivers/lguest/core.c Wed Sep 26 14:47:35 2007 +1000
+++ b/drivers/lguest/core.c Wed Sep 26 16:07:52 2007 +1000
@@ -11,54 +11,20 @@
#include <linux/vmalloc.h>
#include <linux/cpu.h>
#include <linux/freezer.h>
+#include <linux/highmem.h>
#include <asm/paravirt.h>
-#include <asm/desc.h>
#include <asm/pgtable.h>
#include <asm/uaccess.h>
#include <asm/poll.h>
-#include <asm/highmem.h>
#include <asm/asm-offsets.h>
-#include <asm/i387.h>
#include "lg.h"
-/* Found in switcher.S */
-extern char start_switcher_text[], end_switcher_text[], switch_to_guest[];
-extern unsigned long default_idt_entries[];
-
-/* Every guest maps the core switcher code. */
-#define SHARED_SWITCHER_PAGES \
- DIV_ROUND_UP(end_switcher_text - start_switcher_text, PAGE_SIZE)
-/* Pages for switcher itself, then two pages per cpu */
-#define TOTAL_SWITCHER_PAGES (SHARED_SWITCHER_PAGES + 2 * NR_CPUS)
-
-/* We map at -4M for ease of mapping into the guest (one PTE page). */
-#define SWITCHER_ADDR 0xFFC00000
static struct vm_struct *switcher_vma;
static struct page **switcher_page;
-static int cpu_had_pge;
-static struct {
- unsigned long offset;
- unsigned short segment;
-} lguest_entry;
-
/* This One Big lock protects all inter-guest data structures. */
DEFINE_MUTEX(lguest_lock);
-static DEFINE_PER_CPU(struct lguest *, last_guest);
-
-/* Offset from where switcher.S was compiled to where we've copied it */
-static unsigned long switcher_offset(void)
-{
- return SWITCHER_ADDR - (unsigned long)start_switcher_text;
-}
-
-/* This cpu's struct lguest_pages. */
-static struct lguest_pages *lguest_pages(unsigned int cpu)
-{
- return &(((struct lguest_pages *)
- (SWITCHER_ADDR + SHARED_SWITCHER_PAGES*PAGE_SIZE))[cpu]);
-}
/*H:010 We need to set up the Switcher at a high virtual address. Remember the
* Switcher is a few hundred bytes of assembler code which actually changes the
@@ -69,9 +35,7 @@ static struct lguest_pages *lguest_pages
* Host since it will be running as the switchover occurs.
*
* Trying to map memory at a particular address is an unusual thing to do, so
- * it's not a simple one-liner. We also set up the per-cpu parts of the
- * Switcher here.
- */
+ * it's not a simple one-liner. */
static __init int map_switcher(void)
{
int i, err;
@@ -128,89 +92,10 @@ static __init int map_switcher(void)
goto free_vma;
}
- /* Now the switcher is mapped at the right address, we can't fail!
- * Copy in the compiled-in Switcher code (from switcher.S). */
+ /* Now the Switcher is mapped at the right address, we can't fail!
+ * Copy in the compiled-in Switcher code (from <arch>_switcher.S). */
memcpy(switcher_vma->addr, start_switcher_text,
end_switcher_text - start_switcher_text);
-
- /* Most of the switcher.S doesn't care that it's been moved; on Intel,
- * jumps are relative, and it doesn't access any references to external
- * code or data.
- *
- * The only exception is the interrupt handlers in switcher.S: their
- * addresses are placed in a table (default_idt_entries), so we need to
- * update the table with the new addresses. switcher_offset() is a
- * convenience function which returns the distance between the builtin
- * switcher code and the high-mapped copy we just made. */
- for (i = 0; i < IDT_ENTRIES; i++)
- default_idt_entries[i] += switcher_offset();
-
- /*
- * Set up the Switcher's per-cpu areas.
- *
- * Each CPU gets two pages of its own within the high-mapped region
- * (aka. "struct lguest_pages"). Much of this can be initialized
now,
- * but some depends on what Guest we are running (which is set up in
- * copy_in_guest_info()).
- */
- for_each_possible_cpu(i) {
- /* lguest_pages() returns this CPU's two pages. */
- struct lguest_pages *pages = lguest_pages(i);
- /* This is a convenience pointer to make the code fit one
- * statement to a line. */
- struct lguest_ro_state *state = &pages->state;
-
- /* The Global Descriptor Table: the Host has a different one
- * for each CPU. We keep a descriptor for the GDT which says
- * where it is and how big it is (the size is actually the last
- * byte, not the size, hence the "-1"). */
- state->host_gdt_desc.size = GDT_SIZE-1;
- state->host_gdt_desc.address = (long)get_cpu_gdt_table(i);
-
- /* All CPUs on the Host use the same Interrupt Descriptor
- * Table, so we just use store_idt(), which gets this CPU's IDT
- * descriptor. */
- store_idt(&state->host_idt_desc);
-
- /* The descriptors for the Guest's GDT and IDT can be filled
- * out now, too. We copy the GDT & IDT into ->guest_gdt and
- * ->guest_idt before actually running the Guest. */
- state->guest_idt_desc.size = sizeof(state->guest_idt)-1;
- state->guest_idt_desc.address = (long)&state->guest_idt;
- state->guest_gdt_desc.size = sizeof(state->guest_gdt)-1;
- state->guest_gdt_desc.address = (long)&state->guest_gdt;
-
- /* We know where we want the stack to be when the Guest enters
- * the switcher: in pages->regs. The stack grows upwards, so
- * we start it at the end of that structure. */
- state->guest_tss.esp0 = (long)(&pages->regs + 1);
- /* And this is the GDT entry to use for the stack: we keep a
- * couple of special LGUEST entries. */
- state->guest_tss.ss0 = LGUEST_DS;
-
- /* x86 can have a finegrained bitmap which indicates what I/O
- * ports the process can use. We set it to the end of our
- * structure, meaning "none". */
- state->guest_tss.io_bitmap_base = sizeof(state->guest_tss);
-
- /* Some GDT entries are the same across all Guests, so we can
- * set them up now. */
- setup_default_gdt_entries(state);
- /* Most IDT entries are the same for all Guests, too.*/
- setup_default_idt_entries(state, default_idt_entries);
-
- /* The Host needs to be able to use the LGUEST segments on this
- * CPU, too, so put them in the Host GDT. */
- get_cpu_gdt_table(i)[GDT_ENTRY_LGUEST_CS] = FULL_EXEC_SEGMENT;
- get_cpu_gdt_table(i)[GDT_ENTRY_LGUEST_DS] = FULL_SEGMENT;
- }
-
- /* In the Switcher, we want the %cs segment register to use the
- * LGUEST_CS GDT entry: we've put that in the Host and Guest GDTs, so
- * it will be undisturbed when we switch. To change %cs and jump we
- * need this structure to feed to Intel's "lcall" instruction.
*/
- lguest_entry.offset = (long)switch_to_guest + switcher_offset();
- lguest_entry.segment = LGUEST_CS;
printk(KERN_INFO "lguest: mapped switcher at %p\n",
switcher_vma->addr);
@@ -243,80 +128,6 @@ static void unmap_switcher(void)
__free_pages(switcher_page[i], 0);
}
-/*H:130 Our Guest is usually so well behaved; it never tries to do things it
- * isn't allowed to. Unfortunately, "struct paravirt_ops"
isn't quite
- * complete, because it doesn't contain replacements for the Intel I/O
- * instructions. As a result, the Guest sometimes fumbles across one during
- * the boot process as it probes for various things which are usually attached
- * to a PC.
- *
- * When the Guest uses one of these instructions, we get trap #13 (General
- * Protection Fault) and come here. We see if it's one of those
troublesome
- * instructions and skip over it. We return true if we did. */
-static int emulate_insn(struct lguest *lg)
-{
- u8 insn;
- unsigned int insnlen = 0, in = 0, shift = 0;
- /* The eip contains the *virtual* address of the Guest's instruction:
- * guest_pa just subtracts the Guest's page_offset. */
- unsigned long physaddr = guest_pa(lg, lg->regs->eip);
-
- /* The guest_pa() function only works for Guest kernel addresses, but
- * that's all we're trying to do anyway. */
- if (lg->regs->eip < lg->page_offset)
- return 0;
-
- /* Decoding x86 instructions is icky. */
- lgread(lg, &insn, physaddr, 1);
-
- /* 0x66 is an "operand prefix". It means it's using the upper
16 bits
- of the eax register. */
- if (insn == 0x66) {
- shift = 16;
- /* The instruction is 1 byte so far, read the next byte. */
- insnlen = 1;
- lgread(lg, &insn, physaddr + insnlen, 1);
- }
-
- /* We can ignore the lower bit for the moment and decode the 4 opcodes
- * we need to emulate. */
- switch (insn & 0xFE) {
- case 0xE4: /* in <next byte>,%al */
- insnlen += 2;
- in = 1;
- break;
- case 0xEC: /* in (%dx),%al */
- insnlen += 1;
- in = 1;
- break;
- case 0xE6: /* out %al,<next byte> */
- insnlen += 2;
- break;
- case 0xEE: /* out %al,(%dx) */
- insnlen += 1;
- break;
- default:
- /* OK, we don't know what this is, can't emulate. */
- return 0;
- }
-
- /* If it was an "IN" instruction, they expect the result to be read
- * into %eax, so we change %eax. We always return all-ones, which
- * traditionally means "there's nothing there". */
- if (in) {
- /* Lower bit tells is whether it's a 16 or 32 bit access */
- if (insn & 0x1)
- lg->regs->eax = 0xFFFFFFFF;
- else
- lg->regs->eax |= (0xFFFF << shift);
- }
- /* Finally, we've "done" the instruction, so move past it. */
- lg->regs->eip += insnlen;
- /* Success! */
- return 1;
-}
-/*:*/
-
/*L:305
* Dealing With Guest Memory.
*
@@ -380,104 +191,6 @@ void lgwrite(struct lguest *lg, unsigned
}
/* (end of memory access helper routines) :*/
-static void set_ts(void)
-{
- u32 cr0;
-
- cr0 = read_cr0();
- if (!(cr0 & 8))
- write_cr0(cr0|8);
-}
-
-/*S:010
- * We are getting close to the Switcher.
- *
- * Remember that each CPU has two pages which are visible to the Guest when it
- * runs on that CPU. This has to contain the state for that Guest: we copy the
- * state in just before we run the Guest.
- *
- * Each Guest has "changed" flags which indicate what has changed in
the Guest
- * since it last ran. We saw this set in interrupts_and_traps.c and
- * segments.c.
- */
-static void copy_in_guest_info(struct lguest *lg, struct lguest_pages *pages)
-{
- /* Copying all this data can be quite expensive. We usually run the
- * same Guest we ran last time (and that Guest hasn't run anywhere else
- * meanwhile). If that's not the case, we pretend everything in the
- * Guest has changed. */
- if (__get_cpu_var(last_guest) != lg || lg->last_pages != pages) {
- __get_cpu_var(last_guest) = lg;
- lg->last_pages = pages;
- lg->changed = CHANGED_ALL;
- }
-
- /* These copies are pretty cheap, so we do them unconditionally: */
- /* Save the current Host top-level page directory. */
- pages->state.host_cr3 = __pa(current->mm->pgd);
- /* Set up the Guest's page tables to see this CPU's pages (and no
- * other CPU's pages). */
- map_switcher_in_guest(lg, pages);
- /* Set up the two "TSS" members which tell the CPU what stack to use
- * for traps which do directly into the Guest (ie. traps at privilege
- * level 1). */
- pages->state.guest_tss.esp1 = lg->esp1;
- pages->state.guest_tss.ss1 = lg->ss1;
-
- /* Copy direct-to-Guest trap entries. */
- if (lg->changed & CHANGED_IDT)
- copy_traps(lg, pages->state.guest_idt, default_idt_entries);
-
- /* Copy all GDT entries which the Guest can change. */
- if (lg->changed & CHANGED_GDT)
- copy_gdt(lg, pages->state.guest_gdt);
- /* If only the TLS entries have changed, copy them. */
- else if (lg->changed & CHANGED_GDT_TLS)
- copy_gdt_tls(lg, pages->state.guest_gdt);
-
- /* Mark the Guest as unchanged for next time. */
- lg->changed = 0;
-}
-
-/* Finally: the code to actually call into the Switcher to run the Guest. */
-static void run_guest_once(struct lguest *lg, struct lguest_pages *pages)
-{
- /* This is a dummy value we need for GCC's sake. */
- unsigned int clobber;
-
- /* Copy the guest-specific information into this CPU's "struct
- * lguest_pages". */
- copy_in_guest_info(lg, pages);
-
- /* Set the trap number to 256 (impossible value). If we fault while
- * switching to the Guest (bad segment registers or bug), this will
- * cause us to abort the Guest. */
- lg->regs->trapnum = 256;
-
- /* Now: we push the "eflags" register on the stack, then do an
"lcall".
- * This is how we change from using the kernel code segment to using
- * the dedicated lguest code segment, as well as jumping into the
- * Switcher.
- *
- * The lcall also pushes the old code segment (KERNEL_CS) onto the
- * stack, then the address of this call. This stack layout happens to
- * exactly match the stack of an interrupt... */
- asm volatile("pushf; lcall *lguest_entry"
- /* This is how we tell GCC that %eax ("a") and %ebx
("b")
- * are changed by this routine. The "=" means output. */
- : "=a"(clobber), "=b"(clobber)
- /* %eax contains the pages pointer. ("0" refers to the
- * 0-th argument above, ie "a"). %ebx contains the
- * physical address of the Guest's top-level page
- * directory. */
- : "0"(pages),
"1"(__pa(lg->pgdirs[lg->pgdidx].pgdir))
- /* We tell gcc that all these registers could change,
- * which means we don't have to save and restore them in
- * the Switcher. */
- : "memory", "%edx", "%ecx",
"%edi", "%esi");
-}
-/*:*/
-
/*H:030 Let's jump straight to the the main loop which runs the Guest.
* Remember, this is called by the Launcher reading /dev/lguest, and we keep
* going around and around until something interesting happens. */
@@ -485,11 +198,6 @@ int run_guest(struct lguest *lg, unsigne
{
/* We stop running once the Guest is dead. */
while (!lg->dead) {
- /* We need to initialize this, otherwise gcc complains. It's
- * not (yet) clever enough to see that it's initialized when we
- * need it. */
- unsigned int cr2 = 0; /* Damn gcc */
-
/* First we run any hypercalls the Guest wants done: either in
* the hypercall ring in "struct lguest_data", or directly by
* using int 31 (LGUEST_TRAP_ENTRY). */
@@ -538,130 +246,18 @@ int run_guest(struct lguest *lg, unsigne
* the "Do Not Disturb" sign: */
local_irq_disable();
- /* Remember the awfully-named TS bit? If the Guest has asked
- * to set it we set it now, so we can trap and pass that trap
- * to the Guest if it uses the FPU. */
- if (lg->ts)
- set_ts();
-
- /* SYSENTER is an optimized way of doing system calls. We
- * can't allow it because it always jumps to privilege level 0.
- * A normal Guest won't try it because we don't advertise it in
- * CPUID, but a malicious Guest (or malicious Guest userspace
- * program) could, so we tell the CPU to disable it before
- * running the Guest. */
- if (boot_cpu_has(X86_FEATURE_SEP))
- wrmsr(MSR_IA32_SYSENTER_CS, 0, 0);
-
- /* Now we actually run the Guest. It will pop back out when
- * something interesting happens, and we can examine its
- * registers to see what it was doing. */
- run_guest_once(lg, lguest_pages(raw_smp_processor_id()));
-
- /* The "regs" pointer contains two extra entries which are not
- * really registers: a trap number which says what interrupt or
- * trap made the switcher code come back, and an error code
- * which some traps set. */
-
- /* If the Guest page faulted, then the cr2 register will tell
- * us the bad virtual address. We have to grab this now,
- * because once we re-enable interrupts an interrupt could
- * fault and thus overwrite cr2, or we could even move off to a
- * different CPU. */
- if (lg->regs->trapnum == 14)
- cr2 = read_cr2();
- /* Similarly, if we took a trap because the Guest used the FPU,
- * we have to restore the FPU it expects to see. */
- else if (lg->regs->trapnum == 7)
- math_state_restore();
-
- /* Restore SYSENTER if it's supposed to be on. */
- if (boot_cpu_has(X86_FEATURE_SEP))
- wrmsr(MSR_IA32_SYSENTER_CS, __KERNEL_CS, 0);
+ /* Actually run the Guest until something happens. */
+ lguest_arch_run_guest(lg);
/* Now we're ready to be interrupted or moved to other CPUs */
local_irq_enable();
- /* OK, so what happened? */
- switch (lg->regs->trapnum) {
- case 13: /* We've intercepted a GPF. */
- /* Check if this was one of those annoying IN or OUT
- * instructions which we need to emulate. If so, we
- * just go back into the Guest after we've done it. */
- if (lg->regs->errcode == 0) {
- if (emulate_insn(lg))
- continue;
- }
- break;
- case 14: /* We've intercepted a page fault. */
- /* The Guest accessed a virtual address that wasn't
- * mapped. This happens a lot: we don't actually set
- * up most of the page tables for the Guest at all when
- * we start: as it runs it asks for more and more, and
- * we set them up as required. In this case, we don't
- * even tell the Guest that the fault happened.
- *
- * The errcode tells whether this was a read or a
- * write, and whether kernel or userspace code. */
- if (demand_page(lg, cr2, lg->regs->errcode))
- continue;
-
- /* OK, it's really not there (or not OK): the Guest
- * needs to know. We write out the cr2 value so it
- * knows where the fault occurred.
- *
- * Note that if the Guest were really messed up, this
- * could happen before it's done the INITIALIZE
- * hypercall, so lg->lguest_data will be NULL */
- if (lg->lguest_data
- && put_user(cr2, &lg->lguest_data->cr2))
- kill_guest(lg, "Writing cr2");
- break;
- case 7: /* We've intercepted a Device Not Available fault. */
- /* If the Guest doesn't want to know, we already
- * restored the Floating Point Unit, so we just
- * continue without telling it. */
- if (!lg->ts)
- continue;
- break;
- case 32 ... 255:
- /* These values mean a real interrupt occurred, in
- * which case the Host handler has already been run.
- * We just do a friendly check if another process
- * should now be run, then fall through to loop
- * around: */
- cond_resched();
- case LGUEST_TRAP_ENTRY: /* Handled at top of loop */
- continue;
- }
-
- /* If we get here, it's a trap the Guest wants to know
- * about. */
- if (deliver_trap(lg, lg->regs->trapnum))
- continue;
-
- /* If the Guest doesn't have a handler (either it hasn't
- * registered any yet, or it's one of the faults we don't let
- * it handle), it dies with a cryptic error message. */
- kill_guest(lg, "unhandled trap %li at %#lx (%#lx)",
- lg->regs->trapnum, lg->regs->eip,
- lg->regs->trapnum == 14 ? cr2 : lg->regs->errcode);
- }
+ /* Now we deal with whatever happened to the Guest. */
+ lguest_arch_handle_trap(lg);
+ }
+
/* The Guest is dead => "No such file or directory" */
return -ENOENT;
-}
-
-/* Now we can look at each of the routines this calls, in increasing order of
- * complexity: do_hypercalls(), emulate_insn(), maybe_do_interrupt(),
- * deliver_trap() and demand_page(). After all those, we'll be ready to
- * examine the Switcher, and our philosophical understanding of the Host/Guest
- * duality will be complete. :*/
-static void adjust_pge(void *on)
-{
- if (on)
- write_cr4(read_cr4() | X86_CR4_PGE);
- else
- write_cr4(read_cr4() & ~X86_CR4_PGE);
}
/*H:000
@@ -705,31 +301,8 @@ static int __init init(void)
return err;
}
- /* Finally, we need to turn off "Page Global Enable". PGE is an
- * optimization where page table entries are specially marked to show
- * they never change. The Host kernel marks all the kernel pages this
- * way because it's always present, even when userspace is running.
- *
- * Lguest breaks this: unbeknownst to the rest of the Host kernel, we
- * switch to the Guest kernel. If you don't disable this on all CPUs,
- * you'll get really weird bugs that you'll chase for two days.
- *
- * I used to turn PGE off every time we switched to the Guest and back
- * on when we return, but that slowed the Switcher down noticibly. */
-
- /* We don't need the complexity of CPUs coming and going while we're
- * doing this. */
- lock_cpu_hotplug();
- if (cpu_has_pge) { /* We have a broader idea of "global". */
- /* Remember that this was originally set (for cleanup). */
- cpu_had_pge = 1;
- /* adjust_pge is a helper function which sets or unsets the PGE
- * bit on its CPU, depending on the argument (0 == unset). */
- on_each_cpu(adjust_pge, (void *)0, 0, 1);
- /* Turn off the feature in the global feature set. */
- clear_bit(X86_FEATURE_PGE, boot_cpu_data.x86_capability);
- }
- unlock_cpu_hotplug();
+ /* Finally we do some architecture-specific setup. */
+ lguest_arch_host_init();
/* All good! */
return 0;
@@ -742,15 +315,9 @@ static void __exit fini(void)
free_pagetables();
unmap_switcher();
- /* If we had PGE before we started, turn it back on now. */
- lock_cpu_hotplug();
- if (cpu_had_pge) {
- set_bit(X86_FEATURE_PGE, boot_cpu_data.x86_capability);
- /* adjust_pge's argument "1" means set PGE. */
- on_each_cpu(adjust_pge, (void *)1, 0, 1);
- }
- unlock_cpu_hotplug();
-}
+ lguest_arch_host_fini();
+}
+/*:*/
/* The Host side of lguest can be a module. This is a nice way for people to
* play with it. */
diff -r 46a53d30a8c0 drivers/lguest/i386_core.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/drivers/lguest/i386_core.c Wed Sep 26 16:07:52 2007 +1000
@@ -0,0 +1,476 @@
+/*
+ * Copyright (C) 2006, Rusty Russell <rusty@rustcorp.com.au> IBM
Corporation.
+ * Copyright (C) 2007, Jes Sorensen <jes@sgi.com> SGI.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or
+ * NON INFRINGEMENT. See the GNU General Public License for more
+ * details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+#include <linux/kernel.h>
+#include <linux/start_kernel.h>
+#include <linux/string.h>
+#include <linux/console.h>
+#include <linux/screen_info.h>
+#include <linux/irq.h>
+#include <linux/interrupt.h>
+#include <linux/clocksource.h>
+#include <linux/clockchips.h>
+#include <linux/cpu.h>
+#include <linux/lguest.h>
+#include <linux/lguest_launcher.h>
+#include <linux/lguest_bus.h>
+#include <asm/paravirt.h>
+#include <asm/param.h>
+#include <asm/page.h>
+#include <asm/pgtable.h>
+#include <asm/desc.h>
+#include <asm/setup.h>
+#include <asm/lguest.h>
+#include <asm/uaccess.h>
+#include <asm/i387.h>
+#include "lg.h"
+
+static int cpu_had_pge;
+
+static struct {
+ unsigned long offset;
+ unsigned short segment;
+} lguest_entry;
+
+/* Offset from where switcher.S was compiled to where we've copied it */
+static unsigned long switcher_offset(void)
+{
+ return SWITCHER_ADDR - (unsigned long)start_switcher_text;
+}
+
+/* This cpu's struct lguest_pages. */
+static struct lguest_pages *lguest_pages(unsigned int cpu)
+{
+ return &(((struct lguest_pages *)
+ (SWITCHER_ADDR + SHARED_SWITCHER_PAGES*PAGE_SIZE))[cpu]);
+}
+
+static DEFINE_PER_CPU(struct lguest *, last_guest);
+
+/*S:010
+ * We are getting close to the Switcher.
+ *
+ * Remember that each CPU has two pages which are visible to the Guest when it
+ * runs on that CPU. This has to contain the state for that Guest: we copy the
+ * state in just before we run the Guest.
+ *
+ * Each Guest has "changed" flags which indicate what has changed in
the Guest
+ * since it last ran. We saw this set in interrupts_and_traps.c and
+ * segments.c.
+ */
+static void copy_in_guest_info(struct lguest *lg, struct lguest_pages *pages)
+{
+ /* Copying all this data can be quite expensive. We usually run the
+ * same Guest we ran last time (and that Guest hasn't run anywhere else
+ * meanwhile). If that's not the case, we pretend everything in the
+ * Guest has changed. */
+ if (__get_cpu_var(last_guest) != lg || lg->last_pages != pages) {
+ __get_cpu_var(last_guest) = lg;
+ lg->last_pages = pages;
+ lg->changed = CHANGED_ALL;
+ }
+
+ /* These copies are pretty cheap, so we do them unconditionally: */
+ /* Save the current Host top-level page directory. */
+ pages->state.host_cr3 = __pa(current->mm->pgd);
+ /* Set up the Guest's page tables to see this CPU's pages (and no
+ * other CPU's pages). */
+ map_switcher_in_guest(lg, pages);
+ /* Set up the two "TSS" members which tell the CPU what stack to use
+ * for traps which do directly into the Guest (ie. traps at privilege
+ * level 1). */
+ pages->state.guest_tss.esp1 = lg->esp1;
+ pages->state.guest_tss.ss1 = lg->ss1;
+
+ /* Copy direct-to-Guest trap entries. */
+ if (lg->changed & CHANGED_IDT)
+ copy_traps(lg, pages->state.guest_idt, default_idt_entries);
+
+ /* Copy all GDT entries which the Guest can change. */
+ if (lg->changed & CHANGED_GDT)
+ copy_gdt(lg, pages->state.guest_gdt);
+ /* If only the TLS entries have changed, copy them. */
+ else if (lg->changed & CHANGED_GDT_TLS)
+ copy_gdt_tls(lg, pages->state.guest_gdt);
+
+ /* Mark the Guest as unchanged for next time. */
+ lg->changed = 0;
+}
+
+/* Finally: the code to actually call into the Switcher to run the Guest. */
+static void run_guest_once(struct lguest *lg, struct lguest_pages *pages)
+{
+ /* This is a dummy value we need for GCC's sake. */
+ unsigned int clobber;
+
+ /* Copy the guest-specific information into this CPU's "struct
+ * lguest_pages". */
+ copy_in_guest_info(lg, pages);
+
+ /* Set the trap number to 256 (impossible value). If we fault while
+ * switching to the Guest (bad segment registers or bug), this will
+ * cause us to abort the Guest. */
+ lg->regs->trapnum = 256;
+
+ /* Now: we push the "eflags" register on the stack, then do an
"lcall".
+ * This is how we change from using the kernel code segment to using
+ * the dedicated lguest code segment, as well as jumping into the
+ * Switcher.
+ *
+ * The lcall also pushes the old code segment (KERNEL_CS) onto the
+ * stack, then the address of this call. This stack layout happens to
+ * exactly match the stack of an interrupt... */
+ asm volatile("pushf; lcall *lguest_entry"
+ /* This is how we tell GCC that %eax ("a") and %ebx
("b")
+ * are changed by this routine. The "=" means output. */
+ : "=a"(clobber), "=b"(clobber)
+ /* %eax contains the pages pointer. ("0" refers to the
+ * 0-th argument above, ie "a"). %ebx contains the
+ * physical address of the Guest's top-level page
+ * directory. */
+ : "0"(pages),
"1"(__pa(lg->pgdirs[lg->pgdidx].pgdir))
+ /* We tell gcc that all these registers could change,
+ * which means we don't have to save and restore them in
+ * the Switcher. */
+ : "memory", "%edx", "%ecx",
"%edi", "%esi");
+}
+/*:*/
+
+/*H:040 This is the i386-specific code to setup and run the Guest. Interrupts
+ * are disabled: we own the CPU. */
+void lguest_arch_run_guest(struct lguest *lg)
+{
+ /* Remember the awfully-named TS bit? If the Guest has asked
+ * to set it we set it now, so we can trap and pass that trap
+ * to the Guest if it uses the FPU. */
+ if (lg->ts)
+ lguest_set_ts();
+
+ /* SYSENTER is an optimized way of doing system calls. We
+ * can't allow it because it always jumps to privilege level 0.
+ * A normal Guest won't try it because we don't advertise it in
+ * CPUID, but a malicious Guest (or malicious Guest userspace
+ * program) could, so we tell the CPU to disable it before
+ * running the Guest. */
+ if (boot_cpu_has(X86_FEATURE_SEP))
+ wrmsr(MSR_IA32_SYSENTER_CS, 0, 0);
+
+ /* Now we actually run the Guest. It will pop back out when
+ * something interesting happens, and we can examine its
+ * registers to see what it was doing. */
+ run_guest_once(lg, lguest_pages(raw_smp_processor_id()));
+
+ /* The "regs" pointer contains two extra entries which are not
+ * really registers: a trap number which says what interrupt or
+ * trap made the switcher code come back, and an error code
+ * which some traps set. */
+
+ /* If the Guest page faulted, then the cr2 register will tell
+ * us the bad virtual address. We have to grab this now,
+ * because once we re-enable interrupts an interrupt could
+ * fault and thus overwrite cr2, or we could even move off to a
+ * different CPU. */
+ if (lg->regs->trapnum == 14)
+ lg->arch.last_pagefault = read_cr2();
+ /* Similarly, if we took a trap because the Guest used the FPU,
+ * we have to restore the FPU it expects to see. */
+ else if (lg->regs->trapnum == 7)
+ math_state_restore();
+
+ /* Restore SYSENTER if it's supposed to be on. */
+ if (boot_cpu_has(X86_FEATURE_SEP))
+ wrmsr(MSR_IA32_SYSENTER_CS, __KERNEL_CS, 0);
+}
+
+/*H:130 Our Guest is usually so well behaved; it never tries to do things it
+ * isn't allowed to. Unfortunately, "struct paravirt_ops"
isn't quite
+ * complete, because it doesn't contain replacements for the Intel I/O
+ * instructions. As a result, the Guest sometimes fumbles across one during
+ * the boot process as it probes for various things which are usually attached
+ * to a PC.
+ *
+ * When the Guest uses one of these instructions, we get trap #13 (General
+ * Protection Fault) and come here. We see if it's one of those
troublesome
+ * instructions and skip over it. We return true if we did. */
+static int emulate_insn(struct lguest *lg)
+{
+ u8 insn;
+ unsigned int insnlen = 0, in = 0, shift = 0;
+ /* The eip contains the *virtual* address of the Guest's instruction:
+ * guest_pa just subtracts the Guest's page_offset. */
+ unsigned long physaddr = guest_pa(lg, lg->regs->eip);
+
+ /* The guest_pa() function only works for Guest kernel addresses, but
+ * that's all we're trying to do anyway. */
+ if (lg->regs->eip < lg->page_offset)
+ return 0;
+
+ /* Decoding x86 instructions is icky. */
+ lgread(lg, &insn, physaddr, 1);
+
+ /* 0x66 is an "operand prefix". It means it's using the upper
16 bits
+ of the eax register. */
+ if (insn == 0x66) {
+ shift = 16;
+ /* The instruction is 1 byte so far, read the next byte. */
+ insnlen = 1;
+ lgread(lg, &insn, physaddr + insnlen, 1);
+ }
+
+ /* We can ignore the lower bit for the moment and decode the 4 opcodes
+ * we need to emulate. */
+ switch (insn & 0xFE) {
+ case 0xE4: /* in <next byte>,%al */
+ insnlen += 2;
+ in = 1;
+ break;
+ case 0xEC: /* in (%dx),%al */
+ insnlen += 1;
+ in = 1;
+ break;
+ case 0xE6: /* out %al,<next byte> */
+ insnlen += 2;
+ break;
+ case 0xEE: /* out %al,(%dx) */
+ insnlen += 1;
+ break;
+ default:
+ /* OK, we don't know what this is, can't emulate. */
+ return 0;
+ }
+
+ /* If it was an "IN" instruction, they expect the result to be read
+ * into %eax, so we change %eax. We always return all-ones, which
+ * traditionally means "there's nothing there". */
+ if (in) {
+ /* Lower bit tells is whether it's a 16 or 32 bit access */
+ if (insn & 0x1)
+ lg->regs->eax = 0xFFFFFFFF;
+ else
+ lg->regs->eax |= (0xFFFF << shift);
+ }
+ /* Finally, we've "done" the instruction, so move past it. */
+ lg->regs->eip += insnlen;
+ /* Success! */
+ return 1;
+}
+
+/*H:050 Once we've re-enabled interrupts, we look at why the Guest exited.
*/
+void lguest_arch_handle_trap(struct lguest *lg)
+{
+ switch (lg->regs->trapnum) {
+ case 13: /* We've intercepted a GPF. */
+ /* Check if this was one of those annoying IN or OUT
+ * instructions which we need to emulate. If so, we
+ * just go back into the Guest after we've done it. */
+ if (lg->regs->errcode == 0) {
+ if (emulate_insn(lg))
+ return;
+ }
+ break;
+ case 14: /* We've intercepted a page fault. */
+ /* The Guest accessed a virtual address that wasn't
+ * mapped. This happens a lot: we don't actually set
+ * up most of the page tables for the Guest at all when
+ * we start: as it runs it asks for more and more, and
+ * we set them up as required. In this case, we don't
+ * even tell the Guest that the fault happened.
+ *
+ * The errcode tells whether this was a read or a
+ * write, and whether kernel or userspace code. */
+ if (demand_page(lg, lg->arch.last_pagefault, lg->regs->errcode))
+ return;
+
+ /* OK, it's really not there (or not OK): the Guest
+ * needs to know. We write out the cr2 value so it
+ * knows where the fault occurred.
+ *
+ * Note that if the Guest were really messed up, this
+ * could happen before it's done the INITIALIZE
+ * hypercall, so lg->lguest_data will be NULL */
+ if (lg->lguest_data &&
+ put_user(lg->arch.last_pagefault, &lg->lguest_data->cr2))
+ kill_guest(lg, "Writing cr2");
+ break;
+ case 7: /* We've intercepted a Device Not Available fault. */
+ /* If the Guest doesn't want to know, we already
+ * restored the Floating Point Unit, so we just
+ * continue without telling it. */
+ if (!lg->ts)
+ return;
+ break;
+ case 32 ... 255:
+ /* These values mean a real interrupt occurred, in
+ * which case the Host handler has already been run.
+ * We just do a friendly check if another process
+ * should now be run, then fall through to loop
+ * around: */
+ cond_resched();
+ case LGUEST_TRAP_ENTRY: /* Handled before re-entering Guest */
+ return;
+ }
+
+ /* We didn't handle the trap, so it needs to go to the Guest. */
+ if (!deliver_trap(lg, lg->regs->trapnum))
+ /* If the Guest doesn't have a handler (either it hasn't
+ * registered any yet, or it's one of the faults we don't let
+ * it handle), it dies with a cryptic error message. */
+ kill_guest(lg, "unhandled trap %li at %#lx (%#lx)",
+ lg->regs->trapnum, lg->regs->eip,
+ lg->regs->trapnum == 14 ? lg->arch.last_pagefault
+ : lg->regs->errcode);
+}
+
+/* Now we can look at each of the routines this calls, in increasing order of
+ * complexity: do_hypercalls(), emulate_insn(), maybe_do_interrupt(),
+ * deliver_trap() and demand_page(). After all those, we'll be ready to
+ * examine the Switcher, and our philosophical understanding of the Host/Guest
+ * duality will be complete. :*/
+static void adjust_pge(void *on)
+{
+ if (on)
+ write_cr4(read_cr4() | X86_CR4_PGE);
+ else
+ write_cr4(read_cr4() & ~X86_CR4_PGE);
+}
+
+/*H:020 Now the Switcher is mapped and every thing else is ready, we need to do
+ * some more i386-specific initialization. */
+void __init lguest_arch_host_init(void)
+{
+ int i;
+
+ /* Most of the i386_switcher.S doesn't care that it's been moved; on
+ * Intel, jumps are relative, and it doesn't access any references to
+ * external code or data.
+ *
+ * The only exception is the interrupt handlers in switcher.S: their
+ * addresses are placed in a table (default_idt_entries), so we need to
+ * update the table with the new addresses. switcher_offset() is a
+ * convenience function which returns the distance between the builtin
+ * switcher code and the high-mapped copy we just made. */
+ for (i = 0; i < IDT_ENTRIES; i++)
+ default_idt_entries[i] += switcher_offset();
+
+ /*
+ * Set up the Switcher's per-cpu areas.
+ *
+ * Each CPU gets two pages of its own within the high-mapped region
+ * (aka. "struct lguest_pages"). Much of this can be initialized
now,
+ * but some depends on what Guest we are running (which is set up in
+ * copy_in_guest_info()).
+ */
+ for_each_possible_cpu(i) {
+ /* lguest_pages() returns this CPU's two pages. */
+ struct lguest_pages *pages = lguest_pages(i);
+ /* This is a convenience pointer to make the code fit one
+ * statement to a line. */
+ struct lguest_ro_state *state = &pages->state;
+
+ /* The Global Descriptor Table: the Host has a different one
+ * for each CPU. We keep a descriptor for the GDT which says
+ * where it is and how big it is (the size is actually the last
+ * byte, not the size, hence the "-1"). */
+ state->host_gdt_desc.size = GDT_SIZE-1;
+ state->host_gdt_desc.address = (long)get_cpu_gdt_table(i);
+
+ /* All CPUs on the Host use the same Interrupt Descriptor
+ * Table, so we just use store_idt(), which gets this CPU's IDT
+ * descriptor. */
+ store_idt(&state->host_idt_desc);
+
+ /* The descriptors for the Guest's GDT and IDT can be filled
+ * out now, too. We copy the GDT & IDT into ->guest_gdt and
+ * ->guest_idt before actually running the Guest. */
+ state->guest_idt_desc.size = sizeof(state->guest_idt)-1;
+ state->guest_idt_desc.address = (long)&state->guest_idt;
+ state->guest_gdt_desc.size = sizeof(state->guest_gdt)-1;
+ state->guest_gdt_desc.address = (long)&state->guest_gdt;
+
+ /* We know where we want the stack to be when the Guest enters
+ * the switcher: in pages->regs. The stack grows upwards, so
+ * we start it at the end of that structure. */
+ state->guest_tss.esp0 = (long)(&pages->regs + 1);
+ /* And this is the GDT entry to use for the stack: we keep a
+ * couple of special LGUEST entries. */
+ state->guest_tss.ss0 = LGUEST_DS;
+
+ /* x86 can have a finegrained bitmap which indicates what I/O
+ * ports the process can use. We set it to the end of our
+ * structure, meaning "none". */
+ state->guest_tss.io_bitmap_base = sizeof(state->guest_tss);
+
+ /* Some GDT entries are the same across all Guests, so we can
+ * set them up now. */
+ setup_default_gdt_entries(state);
+ /* Most IDT entries are the same for all Guests, too.*/
+ setup_default_idt_entries(state, default_idt_entries);
+
+ /* The Host needs to be able to use the LGUEST segments on this
+ * CPU, too, so put them in the Host GDT. */
+ get_cpu_gdt_table(i)[GDT_ENTRY_LGUEST_CS] = FULL_EXEC_SEGMENT;
+ get_cpu_gdt_table(i)[GDT_ENTRY_LGUEST_DS] = FULL_SEGMENT;
+ }
+
+ /* In the Switcher, we want the %cs segment register to use the
+ * LGUEST_CS GDT entry: we've put that in the Host and Guest GDTs, so
+ * it will be undisturbed when we switch. To change %cs and jump we
+ * need this structure to feed to Intel's "lcall" instruction.
*/
+ lguest_entry.offset = (long)switch_to_guest + switcher_offset();
+ lguest_entry.segment = LGUEST_CS;
+
+ /* Finally, we need to turn off "Page Global Enable". PGE is an
+ * optimization where page table entries are specially marked to show
+ * they never change. The Host kernel marks all the kernel pages this
+ * way because it's always present, even when userspace is running.
+ *
+ * Lguest breaks this: unbeknownst to the rest of the Host kernel, we
+ * switch to the Guest kernel. If you don't disable this on all CPUs,
+ * you'll get really weird bugs that you'll chase for two days.
+ *
+ * I used to turn PGE off every time we switched to the Guest and back
+ * on when we return, but that slowed the Switcher down noticibly. */
+
+ /* We don't need the complexity of CPUs coming and going while we're
+ * doing this. */
+ lock_cpu_hotplug();
+ if (cpu_has_pge) { /* We have a broader idea of "global". */
+ /* Remember that this was originally set (for cleanup). */
+ cpu_had_pge = 1;
+ /* adjust_pge is a helper function which sets or unsets the PGE
+ * bit on its CPU, depending on the argument (0 == unset). */
+ on_each_cpu(adjust_pge, (void *)0, 0, 1);
+ /* Turn off the feature in the global feature set. */
+ clear_bit(X86_FEATURE_PGE, boot_cpu_data.x86_capability);
+ }
+ unlock_cpu_hotplug();
+};
+/*:*/
+
+void __exit lguest_arch_host_fini(void)
+{
+ /* If we had PGE before we started, turn it back on now. */
+ lock_cpu_hotplug();
+ if (cpu_had_pge) {
+ set_bit(X86_FEATURE_PGE, boot_cpu_data.x86_capability);
+ /* adjust_pge's argument "1" means set PGE. */
+ on_each_cpu(adjust_pge, (void *)1, 0, 1);
+ }
+ unlock_cpu_hotplug();
+}
diff -r 46a53d30a8c0 drivers/lguest/i386_switcher.S
--- a/drivers/lguest/i386_switcher.S Wed Sep 26 14:47:35 2007 +1000
+++ b/drivers/lguest/i386_switcher.S Wed Sep 26 16:07:52 2007 +1000
@@ -48,6 +48,8 @@
#include <linux/linkage.h>
#include <asm/asm-offsets.h>
#include <asm/page.h>
+#include <asm/segment.h>
+#include <asm/lguest.h>
#include "lg.h"
// We mark the start of the code to copy
diff -r 46a53d30a8c0 drivers/lguest/interrupts_and_traps.c
--- a/drivers/lguest/interrupts_and_traps.c Wed Sep 26 14:47:35 2007 +1000
+++ b/drivers/lguest/interrupts_and_traps.c Wed Sep 26 16:07:52 2007 +1000
@@ -165,7 +165,7 @@ void maybe_do_interrupt(struct lguest *l
/* Look at the IDT entry the Guest gave us for this interrupt. The
* first 32 (FIRST_EXTERNAL_VECTOR) entries are for traps, so we skip
* over them. */
- idt = &lg->idt[FIRST_EXTERNAL_VECTOR+irq];
+ idt = &lg->arch.idt[FIRST_EXTERNAL_VECTOR+irq];
/* If they don't have a handler (yet?), we just ignore it */
if (idt_present(idt->a, idt->b)) {
/* OK, mark it no longer pending and deliver it. */
@@ -197,14 +197,14 @@ int deliver_trap(struct lguest *lg, unsi
{
/* Trap numbers are always 8 bit, but we set an impossible trap number
* for traps inside the Switcher, so check that here. */
- if (num >= ARRAY_SIZE(lg->idt))
+ if (num >= ARRAY_SIZE(lg->arch.idt))
return 0;
/* Early on the Guest hasn't set the IDT entries (or maybe it put a
* bogus one in): if we fail here, the Guest will be killed. */
- if (!idt_present(lg->idt[num].a, lg->idt[num].b))
+ if (!idt_present(lg->arch.idt[num].a, lg->arch.idt[num].b))
return 0;
- set_guest_interrupt(lg, lg->idt[num].a, lg->idt[num].b, has_err(num));
+ set_guest_interrupt(lg, lg->arch.idt[num].a, lg->arch.idt[num].b,
has_err(num));
return 1;
}
@@ -341,10 +341,10 @@ void load_guest_idt_entry(struct lguest
lg->changed |= CHANGED_IDT;
/* Check that the Guest doesn't try to step outside the bounds. */
- if (num >= ARRAY_SIZE(lg->idt))
+ if (num >= ARRAY_SIZE(lg->arch.idt))
kill_guest(lg, "Setting idt entry %u", num);
else
- set_trap(lg, &lg->idt[num], num, lo, hi);
+ set_trap(lg, &lg->arch.idt[num], num, lo, hi);
}
/* The default entry for each interrupt points into the Switcher routines which
@@ -387,7 +387,7 @@ void copy_traps(const struct lguest *lg,
/* We can simply copy the direct traps, otherwise we use the default
* ones in the Switcher: they will return to the Host. */
- for (i = 0; i < ARRAY_SIZE(lg->idt); i++) {
+ for (i = 0; i < ARRAY_SIZE(lg->arch.idt); i++) {
/* If no Guest can ever override this trap, leave it alone. */
if (!direct_trap(i))
continue;
@@ -396,8 +396,8 @@ void copy_traps(const struct lguest *lg,
* Interrupt gates (type 14) disable interrupts as they are
* entered, which we never let the Guest do. Not present
* entries (type 0x0) also can't go direct, of course. */
- if (idt_type(lg->idt[i].a, lg->idt[i].b) == 0xF)
- idt[i] = lg->idt[i];
+ if (idt_type(lg->arch.idt[i].a, lg->arch.idt[i].b) == 0xF)
+ idt[i] = lg->arch.idt[i];
else
/* Reset it to the default. */
default_idt_entry(&idt[i], i, def[i]);
diff -r 46a53d30a8c0 drivers/lguest/lg.h
--- a/drivers/lguest/lg.h Wed Sep 26 14:47:35 2007 +1000
+++ b/drivers/lguest/lg.h Wed Sep 26 16:08:02 2007 +1000
@@ -1,12 +1,5 @@
#ifndef _LGUEST_H
#define _LGUEST_H
-
-#include <asm/desc.h>
-
-#define GDT_ENTRY_LGUEST_CS 10
-#define GDT_ENTRY_LGUEST_DS 11
-#define LGUEST_CS (GDT_ENTRY_LGUEST_CS * 8)
-#define LGUEST_DS (GDT_ENTRY_LGUEST_DS * 8)
#ifndef __ASSEMBLY__
#include <linux/types.h>
@@ -19,33 +12,11 @@
#include <linux/wait.h>
#include <linux/err.h>
#include <asm/semaphore.h>
-#include "irq_vectors.h"
-
-#define GUEST_PL 1
-
-struct lguest_regs
-{
- /* Manually saved part. */
- unsigned long ebx, ecx, edx;
- unsigned long esi, edi, ebp;
- unsigned long gs;
- unsigned long eax;
- unsigned long fs, ds, es;
- unsigned long trapnum, errcode;
- /* Trap pushed part */
- unsigned long eip;
- unsigned long cs;
- unsigned long eflags;
- unsigned long esp;
- unsigned long ss;
-};
+
+#include <asm/lguest.h>
void free_pagetables(void);
int init_pagetables(struct page **switcher_page, unsigned int pages);
-
-/* Full 4G segment descriptors, suitable for CS and DS. */
-#define FULL_EXEC_SEGMENT ((struct desc_struct){0x0000ffff, 0x00cf9b00})
-#define FULL_SEGMENT ((struct desc_struct){0x0000ffff, 0x00cf9300})
struct lguest_dma_info
{
@@ -97,23 +68,6 @@ struct pgdir
{
unsigned long cr3;
spgd_t *pgdir;
-};
-
-/* This is a guest-specific page (mapped ro) into the guest. */
-struct lguest_ro_state
-{
- /* Host information we need to restore when we switch back. */
- u32 host_cr3;
- struct Xgt_desc_struct host_idt_desc;
- struct Xgt_desc_struct host_gdt_desc;
- u32 host_sp;
-
- /* Fields which are used when guest is running. */
- struct Xgt_desc_struct guest_idt_desc;
- struct Xgt_desc_struct guest_gdt_desc;
- struct i386_hw_tss guest_tss;
- struct desc_struct guest_idt[IDT_ENTRIES];
- struct desc_struct guest_gdt[GDT_ENTRIES];
};
/* We have two pages shared with guests, per cpu. */
@@ -181,11 +135,7 @@ struct lguest
/* Dead? */
const char *dead;
- /* The GDT entries copied into lguest_ro_state when running. */
- struct desc_struct gdt[GDT_ENTRIES];
-
- /* The IDT entries: some copied into lguest_ro_state when running. */
- struct desc_struct idt[IDT_ENTRIES];
+ struct lguest_arch arch;
/* Virtual clock device */
struct hrtimer hrt;
@@ -240,6 +190,15 @@ int demand_page(struct lguest *info, uns
int demand_page(struct lguest *info, unsigned long cr2, int errcode);
void pin_page(struct lguest *lg, unsigned long vaddr);
+/* <arch>_core.c: */
+void lguest_arch_host_init(void);
+void lguest_arch_host_fini(void);
+void lguest_arch_run_guest(struct lguest *lg);
+void lguest_arch_handle_trap(struct lguest *lg);
+
+/* <arch>_switcher.S: */
+extern char start_switcher_text[], end_switcher_text[], switch_to_guest[];
+
/* lguest_user.c: */
int lguest_device_init(void);
void lguest_device_remove(void);
diff -r 46a53d30a8c0 drivers/lguest/segments.c
--- a/drivers/lguest/segments.c Wed Sep 26 14:47:35 2007 +1000
+++ b/drivers/lguest/segments.c Wed Sep 26 14:48:14 2007 +1000
@@ -73,14 +73,14 @@ static void fixup_gdt_table(struct lgues
/* Segment descriptors contain a privilege level: the Guest is
* sometimes careless and leaves this as 0, even though it's
* running at privilege level 1. If so, we fix it here. */
- if ((lg->gdt[i].b & 0x00006000) == 0)
- lg->gdt[i].b |= (GUEST_PL << 13);
+ if ((lg->arch.gdt[i].b & 0x00006000) == 0)
+ lg->arch.gdt[i].b |= (GUEST_PL << 13);
/* Each descriptor has an "accessed" bit. If we don't set it
* now, the CPU will try to set it when the Guest first loads
* that entry into a segment register. But the GDT isn't
* writable by the Guest, so bad things can happen. */
- lg->gdt[i].b |= 0x00000100;
+ lg->arch.gdt[i].b |= 0x00000100;
}
}
@@ -106,12 +106,12 @@ void setup_guest_gdt(struct lguest *lg)
void setup_guest_gdt(struct lguest *lg)
{
/* Start with full 0-4G segments... */
- lg->gdt[GDT_ENTRY_KERNEL_CS] = FULL_EXEC_SEGMENT;
- lg->gdt[GDT_ENTRY_KERNEL_DS] = FULL_SEGMENT;
+ lg->arch.gdt[GDT_ENTRY_KERNEL_CS] = FULL_EXEC_SEGMENT;
+ lg->arch.gdt[GDT_ENTRY_KERNEL_DS] = FULL_SEGMENT;
/* ...except the Guest is allowed to use them, so set the privilege
* level appropriately in the flags. */
- lg->gdt[GDT_ENTRY_KERNEL_CS].b |= (GUEST_PL << 13);
- lg->gdt[GDT_ENTRY_KERNEL_DS].b |= (GUEST_PL << 13);
+ lg->arch.gdt[GDT_ENTRY_KERNEL_CS].b |= (GUEST_PL << 13);
+ lg->arch.gdt[GDT_ENTRY_KERNEL_DS].b |= (GUEST_PL << 13);
}
/* Like the IDT, we never simply use the GDT the Guest gives us. We set up the
@@ -126,7 +126,7 @@ void copy_gdt_tls(const struct lguest *l
unsigned int i;
for (i = GDT_ENTRY_TLS_MIN; i <= GDT_ENTRY_TLS_MAX; i++)
- gdt[i] = lg->gdt[i];
+ gdt[i] = lg->arch.gdt[i];
}
/* This is the full version */
@@ -138,7 +138,7 @@ void copy_gdt(const struct lguest *lg, s
* replaced. See ignored_gdt() above. */
for (i = 0; i < GDT_ENTRIES; i++)
if (!ignored_gdt(i))
- gdt[i] = lg->gdt[i];
+ gdt[i] = lg->arch.gdt[i];
}
/* This is where the Guest asks us to load a new GDT (LHCALL_LOAD_GDT). */
@@ -146,12 +146,12 @@ void load_guest_gdt(struct lguest *lg, u
{
/* We assume the Guest has the same number of GDT entries as the
* Host, otherwise we'd have to dynamically allocate the Guest GDT. */
- if (num > ARRAY_SIZE(lg->gdt))
+ if (num > ARRAY_SIZE(lg->arch.gdt))
kill_guest(lg, "too many gdt entries %i", num);
/* We read the whole thing in, then fix it up. */
- lgread(lg, lg->gdt, table, num * sizeof(lg->gdt[0]));
- fixup_gdt_table(lg, 0, ARRAY_SIZE(lg->gdt));
+ lgread(lg, lg->arch.gdt, table, num * sizeof(lg->arch.gdt[0]));
+ fixup_gdt_table(lg, 0, ARRAY_SIZE(lg->arch.gdt));
/* Mark that the GDT changed so the core knows it has to copy it again,
* even if the Guest is run on the same CPU. */
lg->changed |= CHANGED_GDT;
@@ -159,7 +159,7 @@ void load_guest_gdt(struct lguest *lg, u
void guest_load_tls(struct lguest *lg, unsigned long gtls)
{
- struct desc_struct *tls = &lg->gdt[GDT_ENTRY_TLS_MIN];
+ struct desc_struct *tls = &lg->arch.gdt[GDT_ENTRY_TLS_MIN];
lgread(lg, tls, gtls, sizeof(*tls)*GDT_ENTRY_TLS_ENTRIES);
fixup_gdt_table(lg, GDT_ENTRY_TLS_MIN, GDT_ENTRY_TLS_MAX+1);
diff -r 46a53d30a8c0 include/asm-i386/lguest.h
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/include/asm-i386/lguest.h Wed Sep 26 16:08:02 2007 +1000
@@ -0,0 +1,87 @@
+#ifndef _I386_LGUEST_H
+#define _I386_LGUEST_H
+
+#define GDT_ENTRY_LGUEST_CS 10
+#define GDT_ENTRY_LGUEST_DS 11
+#define LGUEST_CS (GDT_ENTRY_LGUEST_CS * 8)
+#define LGUEST_DS (GDT_ENTRY_LGUEST_DS * 8)
+
+#ifndef __ASSEMBLY__
+#include <asm/desc.h>
+
+#define GUEST_PL 1
+
+/* Every guest maps the core switcher code. */
+#define SHARED_SWITCHER_PAGES \
+ DIV_ROUND_UP(end_switcher_text - start_switcher_text, PAGE_SIZE)
+/* Pages for switcher itself, then two pages per cpu */
+#define TOTAL_SWITCHER_PAGES (SHARED_SWITCHER_PAGES + 2 * NR_CPUS)
+
+/* We map at -4M for ease of mapping into the guest (one PTE page). */
+#define SWITCHER_ADDR 0xFFC00000
+
+/* Found in switcher.S */
+extern unsigned long default_idt_entries[];
+
+struct lguest_regs
+{
+ /* Manually saved part. */
+ unsigned long ebx, ecx, edx;
+ unsigned long esi, edi, ebp;
+ unsigned long gs;
+ unsigned long eax;
+ unsigned long fs, ds, es;
+ unsigned long trapnum, errcode;
+ /* Trap pushed part */
+ unsigned long eip;
+ unsigned long cs;
+ unsigned long eflags;
+ unsigned long esp;
+ unsigned long ss;
+};
+
+/* This is a guest-specific page (mapped ro) into the guest. */
+struct lguest_ro_state
+{
+ /* Host information we need to restore when we switch back. */
+ u32 host_cr3;
+ struct Xgt_desc_struct host_idt_desc;
+ struct Xgt_desc_struct host_gdt_desc;
+ u32 host_sp;
+
+ /* Fields which are used when guest is running. */
+ struct Xgt_desc_struct guest_idt_desc;
+ struct Xgt_desc_struct guest_gdt_desc;
+ struct i386_hw_tss guest_tss;
+ struct desc_struct guest_idt[IDT_ENTRIES];
+ struct desc_struct guest_gdt[GDT_ENTRIES];
+};
+
+struct lguest_arch
+{
+ /* The GDT entries copied into lguest_ro_state when running. */
+ struct desc_struct gdt[GDT_ENTRIES];
+
+ /* The IDT entries: some copied into lguest_ro_state when running. */
+ struct desc_struct idt[IDT_ENTRIES];
+
+ /* The address of the last guest-visible pagefault (ie. cr2). */
+ unsigned long last_pagefault;
+};
+
+static inline void lguest_set_ts(void)
+{
+ u32 cr0;
+
+ cr0 = read_cr0();
+ if (!(cr0 & 8))
+ write_cr0(cr0|8);
+}
+
+/* Full 4G segment descriptors, suitable for CS and DS. */
+#define FULL_EXEC_SEGMENT ((struct desc_struct){0x0000ffff, 0x00cf9b00})
+#define FULL_SEGMENT ((struct desc_struct){0x0000ffff, 0x00cf9300})
+
+#endif /* __ASSEMBLY__ */
+
+#endif
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:20 UTC
[patch 39/43] lguest: This patch gets rid of the old lguest host I/O infrastructure and replaces it with a single hypercall "LHCALL_NOTIFY" which takes an address.
The main change is the removal of io.c: that mainly did inter-guest
I/O, which virtio doesn't yet support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
drivers/lguest/Makefile | 2
drivers/lguest/core.c | 12
drivers/lguest/hypercalls.c | 26 -
drivers/lguest/io.c | 628 ---------------------------------------
drivers/lguest/lg.h | 27 -
drivers/lguest/lguest_user.c | 39 --
include/asm-i386/lguest_hcall.h | 3
include/linux/lguest_launcher.h | 36 --
8 files changed, 21 insertions(+), 752 deletions(-)
==================================================================---
a/drivers/lguest/Makefile
+++ b/drivers/lguest/Makefile
@@ -1,7 +1,7 @@
# Host requires the other files, which can be a module.
obj-$(CONFIG_LGUEST) += lg.o
lg-y = core.o hypercalls.o page_tables.o interrupts_and_traps.o \
- segments.o io.o lguest_user.o
+ segments.o lguest_user.o
lg-$(CONFIG_X86_32) += i386_switcher.o i386_core.o
==================================================================---
a/drivers/lguest/core.c
+++ b/drivers/lguest/core.c
@@ -202,13 +202,12 @@ int run_guest(struct lguest *lg, unsigne
if (lg->hcall)
do_hypercalls(lg);
- /* It's possible the Guest did a SEND_DMA hypercall to the
+ /* It's possible the Guest did a NOTIFY hypercall to the
* Launcher, in which case we return from the read() now. */
- if (lg->dma_is_pending) {
- if (put_user(lg->pending_dma, user) ||
- put_user(lg->pending_key, user+1))
+ if (lg->pending_notify) {
+ if (put_user(lg->pending_notify, user))
return -EFAULT;
- return sizeof(unsigned long)*2;
+ return sizeof(lg->pending_notify);
}
/* Check for signals */
@@ -287,9 +286,6 @@ static int __init init(void)
err = init_pagetables(switcher_page, SHARED_SWITCHER_PAGES);
if (err)
goto unmap;
-
- /* The I/O subsystem needs some things initialized. */
- lguest_io_init();
/* We might need to reserve an interrupt vector. */
err = init_interrupts();
==================================================================---
a/drivers/lguest/hypercalls.c
+++ b/drivers/lguest/hypercalls.c
@@ -60,22 +60,9 @@ static void do_hcall(struct lguest *lg,
else
guest_pagetable_flush_user(lg);
break;
- case LHCALL_BIND_DMA:
- /* BIND_DMA really wants four arguments, but it's the only call
- * which does. So the Guest packs the number of buffers and
- * the interrupt number into the final argument, and we decode
- * it here. This can legitimately fail, since we currently
- * place a limit on the number of DMA pools a Guest can have.
- * So we return true or false from this call. */
- args->arg0 = bind_dma(lg, args->arg1, args->arg2,
- args->arg3 >> 8, args->arg3 & 0xFF);
- break;
/* All these calls simply pass the arguments through to the right
* routines. */
- case LHCALL_SEND_DMA:
- send_dma(lg, args->arg1, args->arg2);
- break;
case LHCALL_NEW_PGTABLE:
guest_new_pagetable(lg, args->arg1);
break;
@@ -98,6 +85,9 @@ static void do_hcall(struct lguest *lg,
case LHCALL_HALT:
/* Similarly, this sets the halted flag for run_guest(). */
lg->halted = 1;
+ break;
+ case LHCALL_NOTIFY:
+ lg->pending_notify = args->arg1;
break;
default:
if (lguest_arch_do_hcall(lg, args))
@@ -156,9 +146,9 @@ static void do_async_hcalls(struct lgues
break;
}
- /* Stop doing hypercalls if we've just done a DMA to the
- * Launcher: it needs to service this first. */
- if (lg->dma_is_pending)
+ /* Stop doing hypercalls if they want to notify the Launcher:
+ * it needs to service this first. */
+ if (lg->pending_notify)
break;
}
}
@@ -220,9 +210,9 @@ void do_hypercalls(struct lguest *lg)
do_async_hcalls(lg);
/* If we stopped reading the hypercall ring because the Guest did a
- * SEND_DMA to the Launcher, we want to return now. Otherwise we do
+ * NOTIFY to the Launcher, we want to return now. Otherwise we do
* the hypercall. */
- if (!lg->dma_is_pending) {
+ if (!lg->pending_notify) {
do_hcall(lg, lg->hcall);
/* Tricky point: we reset the hcall pointer to mark the
* hypercall as "done". We use the hcall pointer rather than
==================================================================---
a/drivers/lguest/io.c
+++ /dev/null
@@ -1,628 +0,0 @@
-/*P:300 The I/O mechanism in lguest is simple yet flexible, allowing the Guest
- * to talk to the Launcher or directly to another Guest. It uses familiar
- * concepts of DMA and interrupts, plus some neat code stolen from
- * futexes... :*/
-
-/* Copyright (C) 2006 Rusty Russell IBM Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include <linux/types.h>
-#include <linux/futex.h>
-#include <linux/jhash.h>
-#include <linux/mm.h>
-#include <linux/highmem.h>
-#include <linux/uaccess.h>
-#include "lg.h"
-
-/*L:300
- * I/O
- *
- * Getting data in and out of the Guest is quite an art. There are numerous
- * ways to do it, and they all suck differently. We try to keep things fairly
- * close to "real" hardware so our Guest's drivers don't look
like an alien
- * visitation in the middle of the Linux code, and yet make sure that Guests
- * can talk directly to other Guests, not just the Launcher.
- *
- * To do this, the Guest gives us a key when it binds or sends DMA buffers.
- * The key corresponds to a "physical" address inside the Guest (ie.
a virtual
- * address inside the Launcher process). We don't, however, use this key
- * directly.
- *
- * We want Guests which share memory to be able to DMA to each other: two
- * Launchers can mmap memory the same file, then the Guests can communicate.
- * Fortunately, the futex code provides us with a way to get a "union
- * futex_key" corresponding to the memory lying at a virtual address: if
the
- * two processes share memory, the "union futex_key" for that memory
will match
- * even if the memory is mapped at different addresses in each. So we always
- * convert the keys to "union futex_key"s to compare them.
- *
- * Before we dive into this though, we need to look at another set of helper
- * routines used throughout the Host kernel code to access Guest memory.
- :*/
-static struct list_head dma_hash[61];
-
-/* An unfortunate side effect of the Linux double-linked list implementation is
- * that there's no good way to statically initialize an array of linked
- * lists. */
-void lguest_io_init(void)
-{
- unsigned int i;
-
- for (i = 0; i < ARRAY_SIZE(dma_hash); i++)
- INIT_LIST_HEAD(&dma_hash[i]);
-}
-
-/* FIXME: allow multi-page lengths. */
-static int check_dma_list(struct lguest *lg, const struct lguest_dma *dma)
-{
- unsigned int i;
-
- for (i = 0; i < LGUEST_MAX_DMA_SECTIONS; i++) {
- if (!dma->len[i])
- return 1;
- if (!lguest_address_ok(lg, dma->addr[i], dma->len[i]))
- goto kill;
- if (dma->len[i] > PAGE_SIZE)
- goto kill;
- /* We could do over a page, but is it worth it? */
- if ((dma->addr[i] % PAGE_SIZE) + dma->len[i] > PAGE_SIZE)
- goto kill;
- }
- return 1;
-
-kill:
- kill_guest(lg, "bad DMA entry: %u@%#lx", dma->len[i],
dma->addr[i]);
- return 0;
-}
-
-/*L:330 This is our hash function, using the wonderful Jenkins hash.
- *
- * The futex key is a union with three parts: an unsigned long word, a pointer,
- * and an int "offset". We could use jhash_2words() which takes
three u32s.
- * (Ok, the hash functions are great: the naming sucks though).
- *
- * It's nice to be portable to 64-bit platforms, so we use the more generic
- * jhash2(), which takes an array of u32, the number of u32s, and an initial
- * u32 to roll in. This is uglier, but breaks down to almost the same code on
- * 32-bit platforms like this one.
- *
- * We want a position in the array, so we modulo ARRAY_SIZE(dma_hash) (ie. 61).
- */
-static unsigned int hash(const union futex_key *key)
-{
- return jhash2((u32*)&key->both.word,
- (sizeof(key->both.word)+sizeof(key->both.ptr))/4,
- key->both.offset)
- % ARRAY_SIZE(dma_hash);
-}
-
-/* This is a convenience routine to compare two keys. It's a much bemoaned
C
- * weakness that it doesn't allow '==' on structures or unions, so
we have to
- * open-code it like this. */
-static inline int key_eq(const union futex_key *a, const union futex_key *b)
-{
- return (a->both.word == b->both.word
- && a->both.ptr == b->both.ptr
- && a->both.offset == b->both.offset);
-}
-
-/*L:360 OK, when we need to actually free up a Guest's DMA array we do
several
- * things, so we have a convenient function to do it.
- *
- * The caller must hold a read lock on dmainfo owner's
current->mm->mmap_sem
- * for the drop_futex_key_refs(). */
-static void unlink_dma(struct lguest_dma_info *dmainfo)
-{
- /* You locked this too, right? */
- BUG_ON(!mutex_is_locked(&lguest_lock));
- /* This is how we know that the entry is free. */
- dmainfo->interrupt = 0;
- /* Remove it from the hash table. */
- list_del(&dmainfo->list);
- /* Drop the references we were holding (to the inode or mm). */
- drop_futex_key_refs(&dmainfo->key);
-}
-
-/*L:350 This is the routine which we call when the Guest asks to unregister a
- * DMA array attached to a given key. Returns true if the array was found. */
-static int unbind_dma(struct lguest *lg,
- const union futex_key *key,
- unsigned long dmas)
-{
- int i, ret = 0;
-
- /* We don't bother with the hash table, just look through all this
- * Guest's DMA arrays. */
- for (i = 0; i < LGUEST_MAX_DMA; i++) {
- /* In theory it could have more than one array on the same key,
- * or one array on multiple keys, so we check both */
- if (key_eq(key, &lg->dma[i].key) && dmas ==
lg->dma[i].dmas) {
- unlink_dma(&lg->dma[i]);
- ret = 1;
- break;
- }
- }
- return ret;
-}
-
-/*L:340 BIND_DMA: this is the hypercall which sets up an array of "struct
- * lguest_dma" for receiving I/O.
- *
- * The Guest wants to bind an array of "struct lguest_dma"s to a
particular key
- * to receive input. This only happens when the Guest is setting up a new
- * device, so it doesn't have to be very fast.
- *
- * It returns 1 on a successful registration (it can fail if we hit the limit
- * of registrations for this Guest).
- */
-int bind_dma(struct lguest *lg,
- unsigned long ukey, unsigned long dmas, u16 numdmas, u8 interrupt)
-{
- unsigned int i;
- int ret = 0;
- union futex_key key;
- /* Futex code needs the mmap_sem. */
- struct rw_semaphore *fshared = ¤t->mm->mmap_sem;
-
- /* Invalid interrupt? (We could kill the guest here). */
- if (interrupt >= LGUEST_IRQS)
- return 0;
-
- /* We need to grab the Big Lguest Lock, because other Guests may be
- * trying to look through this Guest's DMAs to send something while
- * we're doing this. */
- mutex_lock(&lguest_lock);
- down_read(fshared);
- if (get_futex_key(lg->mem_base + ukey, fshared, &key) != 0) {
- kill_guest(lg, "bad dma key %#lx", ukey);
- goto unlock;
- }
-
- /* We want to keep this key valid once we drop mmap_sem, so we have to
- * hold a reference. */
- get_futex_key_refs(&key);
-
- /* If the Guest specified an interrupt of 0, that means they want to
- * unregister this array of "struct lguest_dma"s. */
- if (interrupt == 0)
- ret = unbind_dma(lg, &key, dmas);
- else {
- /* Look through this Guest's dma array for an unused entry. */
- for (i = 0; i < LGUEST_MAX_DMA; i++) {
- /* If the interrupt is non-zero, the entry is already
- * used. */
- if (lg->dma[i].interrupt)
- continue;
-
- /* OK, a free one! Fill on our details. */
- lg->dma[i].dmas = dmas;
- lg->dma[i].num_dmas = numdmas;
- lg->dma[i].next_dma = 0;
- lg->dma[i].key = key;
- lg->dma[i].owner = lg;
- lg->dma[i].interrupt = interrupt;
-
- /* Now we add it to the hash table: the position
- * depends on the futex key that we got. */
- list_add(&lg->dma[i].list, &dma_hash[hash(&key)]);
- /* Success! */
- ret = 1;
- goto unlock;
- }
- }
- /* If we didn't find a slot to put the key in, drop the reference
- * again. */
- drop_futex_key_refs(&key);
-unlock:
- /* Unlock and out. */
- up_read(fshared);
- mutex_unlock(&lguest_lock);
- return ret;
-}
-
-/*L:385 Note that our routines to access a different Guest's memory are
called
- * lgread_other() and lgwrite_other(): these names emphasize that they are only
- * used when the Guest is *not* the current Guest.
- *
- * The interface for copying from another process's memory is called
- * access_process_vm(), with a final argument of 0 for a read, and 1 for a
- * write.
- *
- * We need lgread_other() to read the destination Guest's "struct
lguest_dma"
- * array. */
-static int lgread_other(struct lguest *lg,
- void *buf, u32 addr, unsigned bytes)
-{
- if (!lguest_address_ok(lg, addr, bytes)
- || access_process_vm(lg->tsk, (unsigned long)lg->mem_base + addr,
- buf, bytes, 0) != bytes) {
- memset(buf, 0, bytes);
- kill_guest(lg, "bad address in registered DMA struct");
- return 0;
- }
- return 1;
-}
-
-/* "lgwrite()" to another Guest: used to update the destination
"used_len" once
- * we've transferred data into the buffer. */
-static int lgwrite_other(struct lguest *lg, u32 addr,
- const void *buf, unsigned bytes)
-{
- if (!lguest_address_ok(lg, addr, bytes)
- || access_process_vm(lg->tsk, (unsigned long)lg->mem_base + addr,
- (void *)buf, bytes, 1) != bytes) {
- kill_guest(lg, "bad address writing to registered DMA");
- return 0;
- }
- return 1;
-}
-
-/*L:400 This is the generic engine which copies from a source "struct
- * lguest_dma" from this Guest into another Guest's "struct
lguest_dma". The
- * destination Guest's pages have already been mapped, as contained in the
- * pages array.
- *
- * If you're wondering if there's a nice "copy from one process to
another"
- * routine, so was I. But Linux isn't really set up to copy between two
- * unrelated processes, so we have to write it ourselves.
- */
-static u32 copy_data(struct lguest *srclg,
- const struct lguest_dma *src,
- const struct lguest_dma *dst,
- struct page *pages[])
-{
- unsigned int totlen, si, di, srcoff, dstoff;
- void *maddr = NULL;
-
- /* We return the total length transferred. */
- totlen = 0;
-
- /* We keep indexes into the source and destination "struct
lguest_dma",
- * and an offset within each region. */
- si = di = 0;
- srcoff = dstoff = 0;
-
- /* We loop until the source or destination is exhausted. */
- while (si < LGUEST_MAX_DMA_SECTIONS && src->len[si]
- && di < LGUEST_MAX_DMA_SECTIONS && dst->len[di])
{
- /* We can only transfer the rest of the src buffer, or as much
- * as will fit into the destination buffer. */
- u32 len = min(src->len[si] - srcoff, dst->len[di] - dstoff);
-
- /* For systems using "highmem" we need to use kmap() to access
- * the page we want. We often use the same page over and over,
- * so rather than kmap() it on every loop, we set the maddr
- * pointer to NULL when we need to move to the next
- * destination page. */
- if (!maddr)
- maddr = kmap(pages[di]);
-
- /* Copy directly from (this Guest's) source address to the
- * destination Guest's kmap()ed buffer. Note that maddr points
- * to the start of the page: we need to add the offset of the
- * destination address and offset within the buffer. */
-
- /* FIXME: This is not completely portable. I looked at
- * copy_to_user_page(), and some arch's seem to need special
- * flushes. x86 is fine. */
- if (copy_from_user(maddr + (dst->addr[di] + dstoff)%PAGE_SIZE,
- srclg->mem_base+src->addr[si], len) != 0) {
- /* If a copy failed, it's the source's fault. */
- kill_guest(srclg, "bad address in sending DMA");
- totlen = 0;
- break;
- }
-
- /* Increment the total and src & dst offsets */
- totlen += len;
- srcoff += len;
- dstoff += len;
-
- /* Presumably we reached the end of the src or dest buffers: */
- if (srcoff == src->len[si]) {
- /* Move to the next buffer at offset 0 */
- si++;
- srcoff = 0;
- }
- if (dstoff == dst->len[di]) {
- /* We need to unmap that destination page and reset
- * maddr ready for the next one. */
- kunmap(pages[di]);
- maddr = NULL;
- di++;
- dstoff = 0;
- }
- }
-
- /* If we still had a page mapped at the end, unmap now. */
- if (maddr)
- kunmap(pages[di]);
-
- return totlen;
-}
-
-/*L:390 This is how we transfer a "struct lguest_dma" from the source
Guest
- * (the current Guest which called SEND_DMA) to another Guest. */
-static u32 do_dma(struct lguest *srclg, const struct lguest_dma *src,
- struct lguest *dstlg, const struct lguest_dma *dst)
-{
- int i;
- u32 ret;
- struct page *pages[LGUEST_MAX_DMA_SECTIONS];
-
- /* We check that both source and destination "struct lguest_dma"s
are
- * within the bounds of the source and destination Guests */
- if (!check_dma_list(dstlg, dst) || !check_dma_list(srclg, src))
- return 0;
-
- /* We need to map the pages which correspond to each parts of
- * destination buffer. */
- for (i = 0; i < LGUEST_MAX_DMA_SECTIONS; i++) {
- if (dst->len[i] == 0)
- break;
- /* get_user_pages() is a complicated function, especially since
- * we only want a single page. But it works, and returns the
- * number of pages. Note that we're holding the destination's
- * mmap_sem, as get_user_pages() requires. */
- if (get_user_pages(dstlg->tsk, dstlg->mm,
- (unsigned long)dstlg->mem_base+dst->addr[i],
- 1, 1, 1, pages+i, NULL)
- != 1) {
- /* This means the destination gave us a bogus buffer */
- kill_guest(dstlg, "Error mapping DMA pages");
- ret = 0;
- goto drop_pages;
- }
- }
-
- /* Now copy the data until we run out of src or dst. */
- ret = copy_data(srclg, src, dst, pages);
-
-drop_pages:
- while (--i >= 0)
- put_page(pages[i]);
- return ret;
-}
-
-/*L:380 Transferring data from one Guest to another is not as simple as I'd
- * like. We've found the "struct lguest_dma_info" bound to the
same address as
- * the send, we need to copy into it.
- *
- * This function returns true if the destination array was empty. */
-static int dma_transfer(struct lguest *srclg,
- unsigned long udma,
- struct lguest_dma_info *dst)
-{
- struct lguest_dma dst_dma, src_dma;
- struct lguest *dstlg;
- u32 i, dma = 0;
-
- /* From the "struct lguest_dma_info" we found in the hash, grab the
- * Guest. */
- dstlg = dst->owner;
- /* Read in the source "struct lguest_dma" handed to SEND_DMA. */
- lgread(srclg, &src_dma, udma, sizeof(src_dma));
-
- /* We need the destination's mmap_sem, and we already hold the
source's
- * mmap_sem for the futex key lookup. Normally this would suggest that
- * we could deadlock if the destination Guest was trying to send to
- * this source Guest at the same time, which is another reason that all
- * I/O is done under the big lguest_lock. */
- down_read(&dstlg->mm->mmap_sem);
-
- /* Look through the destination DMA array for an available buffer. */
- for (i = 0; i < dst->num_dmas; i++) {
- /* We keep a "next_dma" pointer which often helps us avoid
- * looking at lots of previously-filled entries. */
- dma = (dst->next_dma + i) % dst->num_dmas;
- if (!lgread_other(dstlg, &dst_dma,
- dst->dmas + dma * sizeof(struct lguest_dma),
- sizeof(dst_dma))) {
- goto fail;
- }
- if (!dst_dma.used_len)
- break;
- }
-
- /* If we found a buffer, we do the actual data copy. */
- if (i != dst->num_dmas) {
- unsigned long used_lenp;
- unsigned int ret;
-
- ret = do_dma(srclg, &src_dma, dstlg, &dst_dma);
- /* Put used length in the source "struct lguest_dma"'s used_len
- * field. It's a little tricky to figure out where that is,
- * though. */
- lgwrite_u32(srclg,
- udma+offsetof(struct lguest_dma, used_len), ret);
- /* Tranferring 0 bytes is OK if the source buffer was empty. */
- if (ret == 0 && src_dma.len[0] != 0)
- goto fail;
-
- /* The destination Guest might be running on a different CPU:
- * we have to make sure that it will see the "used_len" field
- * change to non-zero *after* it sees the data we copied into
- * the buffer. Hence a write memory barrier. */
- wmb();
- /* Figuring out where the destination's used_len field for this
- * "struct lguest_dma" in the array is also a little ugly. */
- used_lenp = dst->dmas
- + dma * sizeof(struct lguest_dma)
- + offsetof(struct lguest_dma, used_len);
- lgwrite_other(dstlg, used_lenp, &ret, sizeof(ret));
- /* Move the cursor for next time. */
- dst->next_dma++;
- }
- up_read(&dstlg->mm->mmap_sem);
-
- /* We trigger the destination interrupt, even if the destination was
- * empty and we didn't transfer anything: this gives them a chance to
- * wake up and refill. */
- set_bit(dst->interrupt, dstlg->irqs_pending);
- /* Wake up the destination process. */
- wake_up_process(dstlg->tsk);
- /* If we passed the last "struct lguest_dma", the receive had no
- * buffers left. */
- return i == dst->num_dmas;
-
-fail:
- up_read(&dstlg->mm->mmap_sem);
- return 0;
-}
-
-/*L:370 This is the counter-side to the BIND_DMA hypercall; the SEND_DMA
- * hypercall. We find out who's listening, and send to them. */
-void send_dma(struct lguest *lg, unsigned long ukey, unsigned long udma)
-{
- union futex_key key;
- int empty = 0;
- struct rw_semaphore *fshared = ¤t->mm->mmap_sem;
-
-again:
- mutex_lock(&lguest_lock);
- down_read(fshared);
- /* Get the futex key for the key the Guest gave us */
- if (get_futex_key(lg->mem_base + ukey, fshared, &key) != 0) {
- kill_guest(lg, "bad sending DMA key");
- goto unlock;
- }
- /* Since the key must be a multiple of 4, the futex key uses the lower
- * bit of the "offset" field (which would always be 0) to indicate a
- * mapping which is shared with other processes (ie. Guests). */
- if (key.shared.offset & 1) {
- struct lguest_dma_info *i;
- /* Look through the hash for other Guests. */
- list_for_each_entry(i, &dma_hash[hash(&key)], list) {
- /* Don't send to ourselves (would deadlock). */
- if (i->owner->mm == lg->mm)
- continue;
- if (!key_eq(&key, &i->key))
- continue;
-
- /* If dma_transfer() tells us the destination has no
- * available buffers, we increment "empty". */
- empty += dma_transfer(lg, udma, i);
- break;
- }
- /* If the destination is empty, we release our locks and
- * give the destination Guest a brief chance to restock. */
- if (empty == 1) {
- /* Give any recipients one chance to restock. */
- up_read(¤t->mm->mmap_sem);
- mutex_unlock(&lguest_lock);
- /* Next time, we won't try again. */
- empty++;
- goto again;
- }
- } else {
- /* Private mapping: Guest is sending to its Launcher. We set
- * the "dma_is_pending" flag so that the main loop will exit
- * and the Launcher's read() from /dev/lguest will return. */
- lg->dma_is_pending = 1;
- lg->pending_dma = udma;
- lg->pending_key = ukey;
- }
-unlock:
- up_read(fshared);
- mutex_unlock(&lguest_lock);
-}
-/*:*/
-
-void release_all_dma(struct lguest *lg)
-{
- unsigned int i;
-
- BUG_ON(!mutex_is_locked(&lguest_lock));
-
- down_read(&lg->mm->mmap_sem);
- for (i = 0; i < LGUEST_MAX_DMA; i++) {
- if (lg->dma[i].interrupt)
- unlink_dma(&lg->dma[i]);
- }
- up_read(&lg->mm->mmap_sem);
-}
-
-/*M:007 We only return a single DMA buffer to the Launcher, but it would be
- * more efficient to return a pointer to the entire array of DMA buffers, which
- * it can cache and choose one whenever it wants.
- *
- * Currently the Launcher uses a write to /dev/lguest, and the return value is
- * the address of the DMA structure with the interrupt number placed in
- * dma->used_len. If we wanted to return the entire array, we need to
return
- * the address, array size and interrupt number: this seems to require an
- * ioctl(). :*/
-
-/*L:320 This routine looks for a DMA buffer registered by the Guest on the
- * given key (using the BIND_DMA hypercall). */
-unsigned long get_dma_buffer(struct lguest *lg,
- unsigned long ukey, unsigned long *interrupt)
-{
- unsigned long ret = 0;
- union futex_key key;
- struct lguest_dma_info *i;
- struct rw_semaphore *fshared = ¤t->mm->mmap_sem;
-
- /* Take the Big Lguest Lock to stop other Guests sending this Guest DMA
- * at the same time. */
- mutex_lock(&lguest_lock);
- /* To match between Guests sharing the same underlying memory we steal
- * code from the futex infrastructure. This requires that we hold the
- * "mmap_sem" for our process (the Launcher), and pass it to the
futex
- * code. */
- down_read(fshared);
-
- /* This can fail if it's not a valid address, or if the address is not
- * divisible by 4 (the futex code needs that, we don't really). */
- if (get_futex_key(lg->mem_base + ukey, fshared, &key) != 0) {
- kill_guest(lg, "bad registered DMA buffer");
- goto unlock;
- }
- /* Search the hash table for matching entries (the Launcher can only
- * send to its own Guest for the moment, so the entry must be for this
- * Guest) */
- list_for_each_entry(i, &dma_hash[hash(&key)], list) {
- if (key_eq(&key, &i->key) && i->owner == lg) {
- unsigned int j;
- /* Look through the registered DMA array for an
- * available buffer. */
- for (j = 0; j < i->num_dmas; j++) {
- struct lguest_dma dma;
-
- ret = i->dmas + j * sizeof(struct lguest_dma);
- lgread(lg, &dma, ret, sizeof(dma));
- if (dma.used_len == 0)
- break;
- }
- /* Store the interrupt the Guest wants when the buffer
- * is used. */
- *interrupt = i->interrupt;
- break;
- }
- }
-unlock:
- up_read(fshared);
- mutex_unlock(&lguest_lock);
- return ret;
-}
-/*:*/
-
-/*L:410 This really has completed the Launcher. Not only have we now finished
- * the longest chapter in our journey, but this also means we are over halfway
- * through!
- *
- * Enough prevaricating around the bush: it is time for us to dive into the
- * core of the Host, in "make Host".
- */
==================================================================---
a/drivers/lguest/lg.h
+++ b/drivers/lguest/lg.h
@@ -6,7 +6,6 @@
#include <linux/init.h>
#include <linux/stringify.h>
#include <linux/binfmts.h>
-#include <linux/futex.h>
#include <linux/lguest.h>
#include <linux/lguest_launcher.h>
#include <linux/wait.h>
@@ -17,17 +16,6 @@
void free_pagetables(void);
int init_pagetables(struct page **switcher_page, unsigned int pages);
-
-struct lguest_dma_info
-{
- struct list_head list;
- union futex_key key;
- unsigned long dmas;
- struct lguest *owner;
- u16 next_dma;
- u16 num_dmas;
- u8 interrupt; /* 0 when not registered */
-};
struct pgdir
{
@@ -91,14 +79,10 @@ struct lguest
struct task_struct *wake;
unsigned long noirq_start, noirq_end;
- int dma_is_pending;
- unsigned long pending_dma; /* struct lguest_dma */
- unsigned long pending_key; /* address they're sending to */
+ unsigned long pending_notify; /* pfn from LHCALL_NOTIFY */
unsigned int stack_pages;
u32 tsc_khz;
-
- struct lguest_dma_info dma[LGUEST_MAX_DMA];
/* Dead? */
const char *dead;
@@ -185,15 +169,6 @@ int lguest_device_init(void);
int lguest_device_init(void);
void lguest_device_remove(void);
-/* io.c: */
-void lguest_io_init(void);
-int bind_dma(struct lguest *lg,
- unsigned long key, unsigned long udma, u16 numdmas, u8 interrupt);
-void send_dma(struct lguest *info, unsigned long key, unsigned long udma);
-void release_all_dma(struct lguest *lg);
-unsigned long get_dma_buffer(struct lguest *lg, unsigned long key,
- unsigned long *interrupt);
-
/* hypercalls.c: */
void do_hypercalls(struct lguest *lg);
void write_timestamp(struct lguest *lg);
==================================================================---
a/drivers/lguest/lguest_user.c
+++ b/drivers/lguest/lguest_user.c
@@ -2,36 +2,11 @@
* controls and communicates with the Guest. For example, the first write will
* tell us the Guest's memory layout, pagetable, entry point and kernel
address
* offset. A read will run the Guest until something happens, such as a signal
- * or the Guest doing a DMA out to the Launcher. Writes are also used to get a
- * DMA buffer registered by the Guest and to send the Guest an interrupt. :*/
+ * or the Guest doing a NOTIFY out to the Launcher. :*/
#include <linux/uaccess.h>
#include <linux/miscdevice.h>
#include <linux/fs.h>
#include "lg.h"
-
-/*L:310 To send DMA into the Guest, the Launcher needs to be able to ask for a
- * DMA buffer. This is done by writing LHREQ_GETDMA and the key to
- * /dev/lguest. */
-static long user_get_dma(struct lguest *lg, const unsigned long __user *input)
-{
- unsigned long key, udma, irq;
-
- /* Fetch the key they wrote to us. */
- if (get_user(key, input) != 0)
- return -EFAULT;
- /* Look for a free Guest DMA buffer bound to that key. */
- udma = get_dma_buffer(lg, key, &irq);
- if (!udma)
- return -ENOENT;
-
- /* We need to tell the Launcher what interrupt the Guest expects after
- * the buffer is filled. We stash it in udma->used_len. */
- lgwrite_u32(lg, udma + offsetof(struct lguest_dma, used_len), irq);
-
- /* The (guest-physical) address of the DMA buffer is returned from
- * the write(). */
- return udma;
-}
/*L:315 To force the Guest to stop running and return to the Launcher, the
* Waker sets writes LHREQ_BREAK and the value "1" to /dev/lguest.
The
@@ -102,10 +77,10 @@ static ssize_t read(struct file *file, c
return len;
}
- /* If we returned from read() last time because the Guest sent DMA,
+ /* If we returned from read() last time because the Guest notified,
* clear the flag. */
- if (lg->dma_is_pending)
- lg->dma_is_pending = 0;
+ if (lg->pending_notify)
+ lg->pending_notify = 0;
/* Run the Guest until something interesting happens. */
return run_guest(lg, (unsigned long __user *)user);
@@ -216,7 +191,7 @@ unlock:
/*L:010 The first operation the Launcher does must be a write. All writes
* start with a 32 bit number: for the first write this must be
* LHREQ_INITIALIZE to set up the Guest. After that the Launcher can use
- * writes of other values to get DMA buffers and send interrupts. */
+ * writes of other values to send interrupts. */
static ssize_t write(struct file *file, const char __user *in,
size_t size, loff_t *off)
{
@@ -245,8 +220,6 @@ static ssize_t write(struct file *file,
switch (req) {
case LHREQ_INITIALIZE:
return initialize(file, input);
- case LHREQ_GETDMA:
- return user_get_dma(lg, input);
case LHREQ_IRQ:
return user_send_irq(lg, input);
case LHREQ_BREAK:
@@ -276,8 +249,6 @@ static int close(struct inode *inode, st
mutex_lock(&lguest_lock);
/* Cancels the hrtimer set via LHCALL_SET_CLOCKEVENT. */
hrtimer_cancel(&lg->hrt);
- /* Free any DMA buffers the Guest had bound. */
- release_all_dma(lg);
/* Free up the shadow page tables for the Guest. */
free_guest_pagetable(lg);
/* Now all the memory cleanups are done, it's safe to release the
==================================================================---
a/include/asm-i386/lguest_hcall.h
+++ b/include/asm-i386/lguest_hcall.h
@@ -15,11 +15,10 @@
#define LHCALL_TS 8
#define LHCALL_SET_CLOCKEVENT 9
#define LHCALL_HALT 10
-#define LHCALL_BIND_DMA 12
-#define LHCALL_SEND_DMA 13
#define LHCALL_SET_PTE 14
#define LHCALL_SET_PMD 15
#define LHCALL_LOAD_TLS 16
+#define LHCALL_NOTIFY 17
/*G:031 First, how does our Guest contact the Host to ask for privileged
* operations? There are two ways: the direct way is to make a
"hypercall",
==================================================================---
a/include/linux/lguest_launcher.h
+++ b/include/linux/lguest_launcher.h
@@ -9,40 +9,6 @@
/* How many devices? Assume each one wants up to two dma arrays per device. */
#define LGUEST_MAX_DEVICES (LGUEST_MAX_DMA/2)
-
-/*D:200
- * Lguest I/O
- *
- * The lguest I/O mechanism is the only way Guests can talk to devices. There
- * are two hypercalls involved: SEND_DMA for output and BIND_DMA for input. In
- * each case, "struct lguest_dma" describes the buffer: this contains
16
- * addr/len pairs, and if there are fewer buffer elements the len array is
- * terminated with a 0.
- *
- * I/O is organized by keys: BIND_DMA attaches buffers to a particular key, and
- * SEND_DMA transfers to buffers bound to particular key. By convention, keys
- * correspond to a physical address within the device's page. This means
that
- * devices will never accidentally end up with the same keys, and allows the
- * Host use The Futex Trick (as we'll see later in our journey).
- *
- * SEND_DMA simply indicates a key to send to, and the physical address of the
- * "struct lguest_dma" to send. The Host will write the number of
bytes
- * transferred into the "struct lguest_dma"'s used_len member.
- *
- * BIND_DMA indicates a key to bind to, a pointer to an array of "struct
- * lguest_dma"s ready for receiving, the size of that array, and an
interrupt
- * to trigger when data is received. The Host will only allow transfers into
- * buffers with a used_len of zero: it then sets used_len to the number of
- * bytes transferred and triggers the interrupt for the Guest to process the
- * new input. */
-struct lguest_dma
-{
- /* 0 if free to be used, filled by the Host. */
- __u32 used_len;
- __u16 len[LGUEST_MAX_DMA_SECTIONS];
- unsigned long addr[LGUEST_MAX_DMA_SECTIONS];
-};
-/*:*/
/* Where the Host expects the Guest to SEND_DMA console output to. */
#define LGUEST_CONSOLE_DMA_KEY 0
@@ -95,7 +61,7 @@ enum lguest_req
enum lguest_req
{
LHREQ_INITIALIZE, /* + pfnlimit, pgdir, start, pageoffset */
- LHREQ_GETDMA, /* + addr (returns &lguest_dma, irq in ->used_len) */
+ LHREQ_GETDMA, /* No longer used */
LHREQ_IRQ, /* + irq */
LHREQ_BREAK, /* + on/off flag (on blocks until someone does off) */
};
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:20 UTC
[patch 40/43] lguest: Lguest support for Virtio
This makes lguest able to use the virtio devices.
We change the device descriptor page from a simple array to a variable
length "type, config_len, status, config data..." format, and
implement virtio_config_ops to read from that config data.
We use the virtio ring implementation for an efficient Guest <-> Host
virtqueue mechanism, and the new LHCALL_NOTIFY hypercall to kick the
host when it changes.
We also use LHCALL_NOTIFY on kernel addresses for very very early
console output. We could have another hypercall, but this hack works
quite well.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
arch/i386/lguest/Kconfig | 5
arch/i386/lguest/boot.c | 21 ++
drivers/lguest/Makefile | 3
drivers/lguest/lguest_device.c | 373 +++++++++++++++++++++++++++++++++++++++
include/linux/lguest_launcher.h | 47 +---
5 files changed, 421 insertions(+), 28 deletions(-)
==================================================================---
a/arch/i386/lguest/Kconfig
+++ b/arch/i386/lguest/Kconfig
@@ -2,8 +2,13 @@ config LGUEST_GUEST
bool "Lguest guest support"
select PARAVIRT
depends on !X86_PAE
+ select VIRTIO
select VIRTIO_RING
+ select VIRTIO_CONSOLE
help
Lguest is a tiny in-kernel hypervisor. Selecting this will
allow your kernel to boot under lguest. This option will increase
your kernel size by about 6k. If in doubt, say N.
+
+ If you say Y here, make sure you say Y (or M) to the virtio block
+ and net drivers which lguest needs.
==================================================================---
a/arch/i386/lguest/boot.c
+++ b/arch/i386/lguest/boot.c
@@ -55,6 +55,7 @@
#include <linux/clockchips.h>
#include <linux/lguest.h>
#include <linux/lguest_launcher.h>
+#include <linux/virtio_console.h>
#include <asm/paravirt.h>
#include <asm/param.h>
#include <asm/page.h>
@@ -858,6 +859,23 @@ static __init char *lguest_memory_setup(
/* This string is for the boot messages. */
return "LGUEST";
+}
+
+/* Before virtqueues are set up, we use LHCALL_NOTIFY on normal memory to
+ * produce console output. */
+static __init int early_put_chars(u32 vtermno, const char *buf, int count)
+{
+ char scratch[17];
+ unsigned int len = count;
+
+ if (len > sizeof(scratch) - 1)
+ len = sizeof(scratch) - 1;
+ scratch[len] = '\0';
+ memcpy(scratch, buf, len);
+ hcall(LHCALL_NOTIFY, __pa(scratch), 0, 0);
+
+ /* This routine returns the number of bytes actually written. */
+ return len;
}
/*G:050
@@ -1043,6 +1061,9 @@ __init void lguest_init(void *boot)
* adapted for lguest's use. */
add_preferred_console("hvc", 0, NULL);
+ /* Register our very early console. */
+ virtio_cons_early_init(early_put_chars);
+
/* Last of all, we set the power management poweroff hook to point to
* the Guest routine to power off. */
pm_power_off = lguest_power_off;
==================================================================---
a/drivers/lguest/Makefile
+++ b/drivers/lguest/Makefile
@@ -1,3 +1,6 @@
+# Guest requires the device configuration and probing code.
+obj-$(CONFIG_LGUEST_GUEST) += lguest_device.o
+
# Host requires the other files, which can be a module.
obj-$(CONFIG_LGUEST) += lg.o
lg-y = core.o hypercalls.o page_tables.o interrupts_and_traps.o \
==================================================================--- /dev/null
+++ b/drivers/lguest/lguest_device.c
@@ -0,0 +1,373 @@
+/*P:050 Lguest guests use a very simple method to describe devices. It's a
+ * series of device descriptors contained just above the top of normal
+ * memory.
+ *
+ * We use the standard "virtio" device infrastructure, which provides
us with a
+ * console, a network and a block driver. Each one expects some configuration
+ * information and a "virtqueue" mechanism to send and receive data.
:*/
+#include <linux/init.h>
+#include <linux/bootmem.h>
+#include <linux/lguest_launcher.h>
+#include <linux/virtio.h>
+#include <linux/virtio_config.h>
+#include <linux/interrupt.h>
+#include <linux/virtio_ring.h>
+#include <linux/err.h>
+#include <asm/io.h>
+#include <asm/paravirt.h>
+#include <asm/lguest_hcall.h>
+
+/* The pointer to our (page) of device descriptions. */
+static void *lguest_devices;
+
+/* Unique numbering for lguest devices. */
+static unsigned int dev_index;
+
+/* For Guests, device memory can be used as normal memory, so we cast away the
+ * __iomem to quieten sparse. */
+static inline void *lguest_map(unsigned long phys_addr, unsigned long pages)
+{
+ return (__force void *)ioremap(phys_addr, PAGE_SIZE*pages);
+}
+
+static inline void lguest_unmap(void *addr)
+{
+ iounmap((__force void __iomem *)addr);
+}
+
+/*D:100 Each lguest device is just a virtio device plus a pointer to its entry
+ * in the lguest_devices page. */
+struct lguest_device {
+ struct virtio_device vdev;
+
+ /* The entry in the lguest_devices page for this device. */
+ struct lguest_device_desc *desc;
+};
+
+/* Since the virtio infrastructure hands us a pointer to the virtio_device all
+ * the time, it helps to have a curt macro to get a pointer to the struct
+ * lguest_device it's enclosed in. */
+#define to_lgdev(vdev) container_of(vdev, struct lguest_device, vdev)
+
+/*D:130
+ * Device configurations
+ *
+ * The configuration information for a device consists of a series of fields.
+ * The device will look for these fields during setup.
+ *
+ * For us these fields come immediately after that device's descriptor in
the
+ * lguest_devices page.
+ *
+ * Each field starts with a "type" byte, a "length" byte,
then that number of
+ * bytes of configuration information. The device descriptor tells us the
+ * total configuration length so we know when we've reached the last field.
*/
+
+/* type + length bytes */
+#define FHDR_LEN 2
+
+/* This finds the first field of a given type for a device's configuration.
*/
+static void *lg_find(struct virtio_device *vdev, u8 type, unsigned int *len)
+{
+ struct lguest_device_desc *desc = to_lgdev(vdev)->desc;
+ int i;
+
+ for (i = 0; i < desc->config_len; i += FHDR_LEN + desc->config[i+1])
{
+ if (desc->config[i] == type) {
+ /* Mark it used, so Host can know we looked at it, and
+ * also so we won't find the same one twice. */
+ desc->config[i] |= 0x80;
+ /* Remember, the second byte is the length. */
+ *len = desc->config[i+1];
+ /* We return a pointer to the field header. */
+ return desc->config + i;
+ }
+ }
+
+ /* Not found: return NULL for failure. */
+ return NULL;
+}
+
+/* Once they've found a field, getting a copy of it is easy. */
+static void lg_get(struct virtio_device *vdev, void *token,
+ void *buf, unsigned len)
+{
+ /* Check they didn't ask for more than the length of the field! */
+ BUG_ON(len > ((u8 *)token)[1]);
+ memcpy(buf, token + FHDR_LEN, len);
+}
+
+/* Setting the contents is also trivial. */
+static void lg_set(struct virtio_device *vdev, void *token,
+ const void *buf, unsigned len)
+{
+ BUG_ON(len > ((u8 *)token)[1]);
+ memcpy(token + FHDR_LEN, buf, len);
+}
+
+/* The operations to get and set the status word just access the status field
+ * of the device descriptor. */
+static u8 lg_get_status(struct virtio_device *vdev)
+{
+ return to_lgdev(vdev)->desc->status;
+}
+
+static void lg_set_status(struct virtio_device *vdev, u8 status)
+{
+ to_lgdev(vdev)->desc->status = status;
+}
+
+/*
+ * Virtqueues
+ *
+ * The other piece of infrastructure virtio needs is a "virtqueue": a
way of
+ * the Guest device registering buffers for the other side to read from or
+ * write into (ie. send and receive buffers). Each device can have multiple
+ * virtqueues: for example the console has one queue for sending and one for
+ * receiving.
+ *
+ * Fortunately for us, a very fast shared-memory-plus-descriptors virtqueue
+ * already exists in virtio_ring.c. We just need to connect it up.
+ *
+ * We start with the information we need to keep about each virtqueue.
+ */
+
+/*D:140 This is the information we remember about each virtqueue. */
+struct lguest_vq_info
+{
+ /* A copy of the information contained in the device config. */
+ struct lguest_vqconfig config;
+
+ /* The address where we mapped the virtio ring, so we can unmap it. */
+ void *pages;
+};
+
+/* When the virtio_ring code wants to prod the Host, it calls us here and we
+ * make a hypercall. We hand the page number of the virtqueue so the Host
+ * knows which virtqueue we're talking about. */
+static void lg_notify(struct virtqueue *vq)
+{
+ /* We store our virtqueue information in the "priv" pointer of the
+ * virtqueue structure. */
+ struct lguest_vq_info *lvq = vq->priv;
+
+ hcall(LHCALL_NOTIFY, lvq->config.pfn << PAGE_SHIFT, 0, 0);
+}
+
+/* This routine finds the first virtqueue described in the configuration of
+ * this device and sets it up.
+ *
+ * This is kind of an ugly duckling. It'd be nicer to have a standard
+ * representation of a virtqueue in the configuration space, but it seems that
+ * everyone wants to do it differently. The KVM guys want the Guest to
+ * allocate its own pages and tell the Host where they are, but for lguest
it's
+ * simpler for the Host to simply tell us where the pages are.
+ *
+ * So we provide devices with a "find virtqueue and set it up"
function. */
+static struct virtqueue *lg_find_vq(struct virtio_device *vdev,
+ bool (*callback)(struct virtqueue *vq))
+{
+ struct lguest_vq_info *lvq;
+ struct virtqueue *vq;
+ unsigned int len;
+ void *token;
+ int err;
+
+ /* Look for a field of the correct type to mark a virtqueue. Note that
+ * if this succeeds, then the type will be changed so it won't be found
+ * again, and future lg_find_vq() calls will find the next
+ * virtqueue (if any). */
+ token = vdev->config->find(vdev, VIRTIO_CONFIG_F_VIRTQUEUE, &len);
+ if (!token)
+ return ERR_PTR(-ENOENT);
+
+ lvq = kmalloc(sizeof(*lvq), GFP_KERNEL);
+ if (!lvq)
+ return ERR_PTR(-ENOMEM);
+
+ /* Note: we could use a configuration space inside here, just like we
+ * do for the device. This would allow expansion in future, because
+ * our configuration system is designed to be expansible. But this is
+ * way easier. */
+ if (len != sizeof(lvq->config)) {
+ dev_err(&vdev->dev, "Unexpected virtio config len %u\n",
len);
+ err = -EIO;
+ goto free_lvq;
+ }
+ /* Make a copy of the "struct lguest_vqconfig" field. We need a
copy
+ * because the config space might not be aligned correctly. */
+ vdev->config->get(vdev, token, &lvq->config,
sizeof(lvq->config));
+
+ /* Figure out how many pages the ring will take, and map that memory */
+ lvq->pages = lguest_map((unsigned long)lvq->config.pfn <<
PAGE_SHIFT,
+ DIV_ROUND_UP(vring_size(lvq->config.num),
+ PAGE_SIZE));
+ if (!lvq->pages) {
+ err = -ENOMEM;
+ goto free_lvq;
+ }
+
+ /* OK, tell virtio_ring.c to set up a virtqueue now we know its size
+ * and we've got a pointer to its pages. */
+ vq = vring_new_virtqueue(lvq->config.num, vdev, lvq->pages,
+ lg_notify, callback);
+ if (!vq) {
+ err = -ENOMEM;
+ goto unmap;
+ }
+
+ /* Tell the interrupt for this virtqueue to go to the virtio_ring
+ * interrupt handler. */
+ /* FIXME: We used to have a flag for the Host to tell us we could use
+ * the interrupt as a source of randomness: it'd be nice to have that
+ * back.. */
+ err = request_irq(lvq->config.irq, vring_interrupt, IRQF_SHARED,
+ vdev->dev.bus_id, vq);
+ if (err)
+ goto destroy_vring;
+
+ /* Last of all we hook up our 'struct lguest_vq_info" to the
+ * virtqueue's priv pointer. */
+ vq->priv = lvq;
+ return vq;
+
+destroy_vring:
+ vring_del_virtqueue(vq);
+unmap:
+ lguest_unmap(lvq->pages);
+free_lvq:
+ kfree(lvq);
+ return ERR_PTR(err);
+}
+/*:*/
+
+/* Cleaning up a virtqueue is easy */
+static void lg_del_vq(struct virtqueue *vq)
+{
+ struct lguest_vq_info *lvq = vq->priv;
+
+ /* Tell virtio_ring.c to free the virtqueue. */
+ vring_del_virtqueue(vq);
+ /* Unmap the pages containing the ring. */
+ lguest_unmap(lvq->pages);
+ /* Free our own queue information. */
+ kfree(lvq);
+}
+
+/* The ops structure which hooks everything together. */
+static struct virtio_config_ops lguest_config_ops = {
+ .find = lg_find,
+ .get = lg_get,
+ .set = lg_set,
+ .get_status = lg_get_status,
+ .set_status = lg_set_status,
+ .find_vq = lg_find_vq,
+ .del_vq = lg_del_vq,
+};
+
+/* The root device for the lguest virtio devices. This makes them appear as
+ * /sys/devices/lguest/0,1,2 not /sys/devices/0,1,2. */
+static struct device lguest_root = {
+ .parent = NULL,
+ .bus_id = "lguest",
+};
+
+/*D:120 This is the core of the lguest bus: actually adding a new device.
+ * It's a separate function because it's neater that way, and because
an
+ * earlier version of the code supported hotplug and unplug. They were removed
+ * early on because they were never used.
+ *
+ * As Andrew Tridgell says, "Untested code is buggy code".
+ *
+ * It's worth reading this carefully: we start with a pointer to the new
device
+ * descriptor in the "lguest_devices" page. */
+static void add_lguest_device(struct lguest_device_desc *d)
+{
+ struct lguest_device *ldev;
+
+ ldev = kzalloc(sizeof(*ldev), GFP_KERNEL);
+ if (!ldev) {
+ printk(KERN_EMERG "Cannot allocate lguest dev %u\n",
+ dev_index++);
+ return;
+ }
+
+ /* This devices' parent is the lguest/ dir. */
+ ldev->vdev.dev.parent = &lguest_root;
+ /* We have a unique device index thanks to the dev_index counter. */
+ ldev->vdev.index = dev_index++;
+ /* The device type comes straight from the descriptor. There's also a
+ * device vendor field in the virtio_device struct, which we leave as
+ * 0. */
+ ldev->vdev.id.device = d->type;
+ /* We have a simple set of routines for querying the device's
+ * configuration information and setting its status. */
+ ldev->vdev.config = &lguest_config_ops;
+ /* And we remember the device's descriptor for lguest_config_ops. */
+ ldev->desc = d;
+
+ /* register_virtio_device() sets up the generic fields for the struct
+ * virtio_device and calls device_register(). This makes the bus
+ * infrastructure look for a matching driver. */
+ if (register_virtio_device(&ldev->vdev) != 0) {
+ printk(KERN_ERR "Failed to register lguest device %u\n",
+ ldev->vdev.index);
+ kfree(ldev);
+ }
+}
+
+/*D:110 scan_devices() simply iterates through the device page. The type 0 is
+ * reserved to mean "end of devices". */
+static void scan_devices(void)
+{
+ unsigned int i;
+ struct lguest_device_desc *d;
+
+ /* We start at the page beginning, and skip over each entry. */
+ for (i = 0; i < PAGE_SIZE; i += sizeof(*d) + d->config_len) {
+ d = lguest_devices + i;
+
+ /* Once we hit a zero, stop. */
+ if (d->type == 0)
+ break;
+
+ add_lguest_device(d);
+ }
+}
+
+/*D:105 Fairly early in boot, lguest_devices_init() is called to set up the
+ * lguest device infrastructure. We check that we are a Guest by checking
+ * paravirt_ops.name: there are other ways of checking, but this seems most
+ * obvious to me.
+ *
+ * So we can access the "struct lguest_device_desc"s easily, we map
that memory
+ * and store the pointer in the global "lguest_devices". Then we
register a
+ * root device from which all our devices will hang (this seems to be the
+ * correct sysfs incantation).
+ *
+ * Finally we call scan_devices() which adds all the devices found in the
+ * lguest_devices page. */
+static int __init lguest_devices_init(void)
+{
+ if (strcmp(paravirt_ops.name, "lguest") != 0)
+ return 0;
+
+ if (device_register(&lguest_root) != 0)
+ panic("Could not register lguest root");
+
+ /* Devices are in a single page above top of "normal" mem */
+ lguest_devices = lguest_map(max_pfn<<PAGE_SHIFT, 1);
+
+ scan_devices();
+ return 0;
+}
+/* We do this after core stuff, but before the drivers. */
+postcore_initcall(lguest_devices_init);
+
+/*D:150 At this point in the journey we used to now wade through the lguest
+ * devices themselves: net, block and console. Since they're all now
virtio
+ * devices rather than lguest-specific, I've decided to ignore them.
Mostly,
+ * they're kind of boring. But this does mean you'll never experience
the
+ * thrill of reading the forbidden love scene buried deep in the block driver.
+ *
+ * "make Launcher" beckons, where we answer questions like
"Where do Guests
+ * come from?", and "What do you do when someone asks for
optimization?". */
==================================================================---
a/include/linux/lguest_launcher.h
+++ b/include/linux/lguest_launcher.h
@@ -22,37 +22,28 @@
* complex burden for the Host and suboptimal for the Guest, so we have our own
* "lguest" bus and simple drivers.
*
- * Devices are described by an array of LGUEST_MAX_DEVICES of these structs,
- * placed by the Launcher just above the top of physical memory:
+ * Devices are described by a simplified ID, a status byte, and some
"config"
+ * bytes which describe this device's configuration. This is placed by the
+ * Launcher just above the top of physical memory:
*/
struct lguest_device_desc {
- /* The device type: console, network, disk etc. */
- __u16 type;
-#define LGUEST_DEVICE_T_CONSOLE 1
-#define LGUEST_DEVICE_T_NET 2
-#define LGUEST_DEVICE_T_BLOCK 3
+ /* The device type: console, network, disk etc. Type 0 terminates. */
+ __u8 type;
+ /* The number of bytes of the config array. */
+ __u8 config_len;
+ /* A status byte, written by the Guest. */
+ __u8 status;
+ __u8 config[0];
+};
- /* The specific features of this device: these depends on device type
- * except for LGUEST_DEVICE_F_RANDOMNESS. */
- __u16 features;
-#define LGUEST_NET_F_NOCSUM 0x4000 /* Don't bother checksumming */
-#define LGUEST_DEVICE_F_RANDOMNESS 0x8000 /* IRQ is fairly random */
-
- /* This is how the Guest reports status of the device: the Host can set
- * LGUEST_DEVICE_S_REMOVED to indicate removal, but the rest are only
- * ever manipulated by the Guest, and only ever set. */
- __u16 status;
-/* 256 and above are device specific. */
-#define LGUEST_DEVICE_S_ACKNOWLEDGE 1 /* We have seen device. */
-#define LGUEST_DEVICE_S_DRIVER 2 /* We have found a driver */
-#define LGUEST_DEVICE_S_DRIVER_OK 4 /* Driver says OK! */
-#define LGUEST_DEVICE_S_REMOVED 8 /* Device has gone away. */
-#define LGUEST_DEVICE_S_REMOVED_ACK 16 /* Driver has been told. */
-#define LGUEST_DEVICE_S_FAILED 128 /* Something actually failed */
-
- /* Each device exists somewhere in Guest physical memory, over some
- * number of pages. */
- __u16 num_pages;
+/*D:135 This is how we expect the device configuration field for a virtqueue
+ * (type VIRTIO_CONFIG_F_VIRTQUEUE) to be laid out: */
+struct lguest_vqconfig {
+ /* The number of entries in the virtio_ring */
+ __u16 num;
+ /* The interrupt we get when something happens. */
+ __u16 irq;
+ /* The page number of the virtio ring for this device. */
__u32 pfn;
};
/*:*/
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:20 UTC
[patch 29/43] lguest: Introduce "used_vectors" bitmap which can be used to reserve vectors.
This simplifies the io_apic.c __assign_irq_vector() logic and removes
the explicit SYSCALL_VECTOR check, and also allows for vectors to be
reserved by other mechanisms (ie. lguest).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
arch/i386/kernel/i8259.c | 3 ++-
arch/i386/kernel/io_apic.c | 13 ++++++++-----
arch/i386/kernel/traps.c | 10 ++++++++++
include/asm-i386/irq.h | 3 +++
4 files changed, 23 insertions(+), 6 deletions(-)
==================================================================---
a/arch/i386/kernel/i8259.c
+++ b/arch/i386/kernel/i8259.c
@@ -400,7 +400,8 @@ void __init native_init_IRQ(void)
int vector = FIRST_EXTERNAL_VECTOR + i;
if (i >= NR_IRQS)
break;
- if (vector != SYSCALL_VECTOR)
+ /* SYSCALL_VECTOR was reserved in trap_init. */
+ if (!test_bit(vector, used_vectors))
set_intr_gate(vector, interrupt[i]);
}
==================================================================---
a/arch/i386/kernel/io_apic.c
+++ b/arch/i386/kernel/io_apic.c
@@ -1198,7 +1198,7 @@ static int __assign_irq_vector(int irq)
static int __assign_irq_vector(int irq)
{
static int current_vector = FIRST_DEVICE_VECTOR, current_offset = 0;
- int vector, offset, i;
+ int vector, offset;
BUG_ON((unsigned)irq >= NR_IRQ_VECTORS);
@@ -1215,11 +1215,8 @@ next:
}
if (vector == current_vector)
return -ENOSPC;
- if (vector == SYSCALL_VECTOR)
+ if (test_and_set_bit(vector, used_vectors))
goto next;
- for (i = 0; i < NR_IRQ_VECTORS; i++)
- if (irq_vector[i] == vector)
- goto next;
current_vector = vector;
current_offset = offset;
@@ -2290,6 +2287,12 @@ static inline void __init check_timer(vo
void __init setup_IO_APIC(void)
{
+ int i;
+
+ /* Reserve all the system vectors. */
+ for (i = FIRST_SYSTEM_VECTOR; i < NR_VECTORS; i++)
+ set_bit(i, used_vectors);
+
enable_IO_APIC();
if (acpi_ioapic)
==================================================================---
a/arch/i386/kernel/traps.c
+++ b/arch/i386/kernel/traps.c
@@ -64,6 +64,9 @@
#include "mach_traps.h"
int panic_on_unrecovered_nmi;
+
+DECLARE_BITMAP(used_vectors, NR_VECTORS);
+EXPORT_SYMBOL_GPL(used_vectors);
asmlinkage int system_call(void);
@@ -1156,6 +1159,8 @@ static void __init set_task_gate(unsigne
void __init trap_init(void)
{
+ int i;
+
#ifdef CONFIG_EISA
void __iomem *p = ioremap(0x0FFFD9, 4);
if (readl(p) ==
'E'+('I'<<8)+('S'<<16)+('A'<<24))
{
@@ -1215,6 +1220,11 @@ void __init trap_init(void)
set_system_gate(SYSCALL_VECTOR,&system_call);
+ /* Reserve all the builtin and the syscall vector. */
+ for (i = 0; i < FIRST_EXTERNAL_VECTOR; i++)
+ set_bit(i, used_vectors);
+ set_bit(SYSCALL_VECTOR, used_vectors);
+
/*
* Should be a barrier for any external CPU state.
*/
==================================================================---
a/include/asm-i386/irq.h
+++ b/include/asm-i386/irq.h
@@ -45,4 +45,7 @@ void init_IRQ(void);
void init_IRQ(void);
void __init native_init_IRQ(void);
+/* Interrupt vector management */
+extern DECLARE_BITMAP(used_vectors, NR_VECTORS);
+
#endif /* _ASM_IRQ_H */
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
The network driver uses two virtqueues: one for input packets and one
for output packets. This has nice locking properties (ie. we don't do
any for recv vs send).
TODO:
1) Big packets.
2) Multi-client devices (maybe separate driver?).
3) Resolve freeing of old xmit skbs (Christian Borntraeger)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: netdev@vger.kernel.org
---
drivers/net/Kconfig | 6
drivers/net/Makefile | 2
drivers/net/virtio_net.c | 438 ++++++++++++++++++++++++++++++++++++++++++++
include/linux/virtio_net.h | 36 +++
4 files changed, 481 insertions(+), 1 deletion(-)
==================================================================---
a/drivers/net/Kconfig
+++ b/drivers/net/Kconfig
@@ -2998,4 +2998,10 @@ config NET_POLL_CONTROLLER
config NET_POLL_CONTROLLER
def_bool NETPOLL
+config VIRTIO_NET
+ tristate "Virtio network driver (EXPERIMENTAL)"
+ depends on EXPERIMENTAL && VIRTIO
+ ---help---
+ This is the virtual network driver for lguest. Say Y or M.
+
endif # NETDEVICES
==================================================================---
a/drivers/net/Makefile
+++ b/drivers/net/Makefile
@@ -38,7 +38,7 @@ obj-$(CONFIG_SUNVNET) += sunvnet.o
obj-$(CONFIG_MACE) += mace.o
obj-$(CONFIG_BMAC) += bmac.o
-
+obj-$(CONFIG_VIRTIO_NET) += virtio_net.o
obj-$(CONFIG_DGRS) += dgrs.o
obj-$(CONFIG_VORTEX) += 3c59x.o
obj-$(CONFIG_TYPHOON) += typhoon.o
==================================================================--- /dev/null
+++ b/drivers/net/virtio_net.c
@@ -0,0 +1,438 @@
+/* A simple network driver using virtio.
+ *
+ * Copyright 2007 Rusty Russell <rusty@rustcorp.com.au> IBM Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+//#define DEBUG
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/module.h>
+#include <linux/virtio.h>
+#include <linux/virtio_net.h>
+#include <linux/scatterlist.h>
+
+/* FIXME: MTU in config. */
+#define MAX_PACKET_LEN (ETH_HLEN+ETH_DATA_LEN)
+
+struct virtnet_info
+{
+ struct virtio_device *vdev;
+ struct virtqueue *rvq, *svq;
+ struct net_device *dev;
+
+ /* Number of input buffers, and max we've ever had. */
+ unsigned int num, max;
+
+ /* Receive & send queues. */
+ struct sk_buff_head recv;
+ struct sk_buff_head send;
+};
+
+static inline struct virtio_net_hdr *skb_vnet_hdr(struct sk_buff *skb)
+{
+ return (struct virtio_net_hdr *)skb->cb;
+}
+
+static inline void vnet_hdr_to_sg(struct scatterlist *sg, struct sk_buff *skb)
+{
+ sg_init_one(sg, skb_vnet_hdr(skb), sizeof(struct virtio_net_hdr));
+}
+
+static bool skb_xmit_done(struct virtqueue *rvq)
+{
+ struct virtnet_info *vi = rvq->vdev->priv;
+
+ /* In case we were waiting for output buffers. */
+ netif_wake_queue(vi->dev);
+ return true;
+}
+
+static void receive_skb(struct net_device *dev, struct sk_buff *skb,
+ unsigned len)
+{
+ struct virtio_net_hdr *hdr = skb_vnet_hdr(skb);
+
+ if (unlikely(len < sizeof(struct virtio_net_hdr) + ETH_HLEN)) {
+ pr_debug("%s: short packet %i\n", dev->name, len);
+ dev->stats.rx_length_errors++;
+ goto drop;
+ }
+ len -= sizeof(struct virtio_net_hdr);
+ BUG_ON(len > MAX_PACKET_LEN);
+
+ skb_trim(skb, len);
+ skb->protocol = eth_type_trans(skb, dev);
+ pr_debug("Receiving skb proto 0x%04x len %i type %i\n",
+ ntohs(skb->protocol), skb->len, skb->pkt_type);
+ dev->stats.rx_bytes += skb->len;
+ dev->stats.rx_packets++;
+
+ if (hdr->flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) {
+ pr_debug("Needs csum!\n");
+ skb->ip_summed = CHECKSUM_PARTIAL;
+ skb->csum_start = hdr->csum_start;
+ skb->csum_offset = hdr->csum_offset;
+ if (skb->csum_start > skb->len - 2
+ || skb->csum_offset > skb->len - 2) {
+ if (net_ratelimit())
+ printk(KERN_WARNING "%s: csum=%u/%u len=%u\n",
+ dev->name, skb->csum_start,
+ skb->csum_offset, skb->len);
+ goto frame_err;
+ }
+ }
+
+ if (hdr->gso_type != VIRTIO_NET_HDR_GSO_NONE) {
+ pr_debug("GSO!\n");
+ switch (hdr->gso_type) {
+ case VIRTIO_NET_HDR_GSO_TCPV4:
+ skb_shinfo(skb)->gso_type = SKB_GSO_TCPV4;
+ break;
+ case VIRTIO_NET_HDR_GSO_TCPV4_ECN:
+ skb_shinfo(skb)->gso_type = SKB_GSO_TCP_ECN;
+ break;
+ case VIRTIO_NET_HDR_GSO_UDP:
+ skb_shinfo(skb)->gso_type = SKB_GSO_UDP;
+ break;
+ case VIRTIO_NET_HDR_GSO_TCPV6:
+ skb_shinfo(skb)->gso_type = SKB_GSO_TCPV6;
+ break;
+ default:
+ if (net_ratelimit())
+ printk(KERN_WARNING "%s: bad gso type %u.\n",
+ dev->name, hdr->gso_type);
+ goto frame_err;
+ }
+
+ skb_shinfo(skb)->gso_size = hdr->gso_size;
+ if (skb_shinfo(skb)->gso_size == 0) {
+ if (net_ratelimit())
+ printk(KERN_WARNING "%s: zero gso size.\n",
+ dev->name);
+ goto frame_err;
+ }
+
+ /* Header must be checked, and gso_segs computed. */
+ skb_shinfo(skb)->gso_type |= SKB_GSO_DODGY;
+ skb_shinfo(skb)->gso_segs = 0;
+ }
+
+ netif_receive_skb(skb);
+ return;
+
+frame_err:
+ dev->stats.rx_frame_errors++;
+drop:
+ dev_kfree_skb(skb);
+}
+
+static void try_fill_recv(struct virtnet_info *vi)
+{
+ struct sk_buff *skb;
+ struct scatterlist sg[1+MAX_SKB_FRAGS];
+ int num, err;
+
+ for (;;) {
+ skb = netdev_alloc_skb(vi->dev, MAX_PACKET_LEN);
+ if (unlikely(!skb))
+ break;
+
+ skb_put(skb, MAX_PACKET_LEN);
+ vnet_hdr_to_sg(sg, skb);
+ num = skb_to_sgvec(skb, sg+1, 0, skb->len) + 1;
+ skb_queue_head(&vi->recv, skb);
+
+ err = vi->rvq->vq_ops->add_buf(vi->rvq, sg, 0, num, skb);
+ if (err) {
+ skb_unlink(skb, &vi->recv);
+ kfree_skb(skb);
+ break;
+ }
+ vi->num++;
+ }
+ if (unlikely(vi->num > vi->max))
+ vi->max = vi->num;
+ vi->rvq->vq_ops->kick(vi->rvq);
+}
+
+static bool skb_recv_done(struct virtqueue *rvq)
+{
+ struct virtnet_info *vi = rvq->vdev->priv;
+ netif_rx_schedule(vi->dev);
+ /* Suppress further interrupts. */
+ return false;
+}
+
+static int virtnet_poll(struct net_device *dev, int *budget)
+{
+ struct virtnet_info *vi = netdev_priv(dev);
+ struct sk_buff *skb = NULL;
+ unsigned int len, received = 0;
+
+again:
+ while (received < dev->quota &&
+ (skb = vi->rvq->vq_ops->get_buf(vi->rvq, &len)) !=
NULL) {
+ __skb_unlink(skb, &vi->recv);
+ receive_skb(vi->dev, skb, len);
+ vi->num--;
+ received++;
+ }
+
+ dev->quota -= received;
+ *budget -= received;
+
+ /* FIXME: If we oom and completely run out of inbufs, we need
+ * to start a timer trying to fill more. */
+ if (vi->num < vi->max / 2)
+ try_fill_recv(vi);
+
+ /* Still more work to do? */
+ if (skb)
+ return 1; /* not done */
+
+ netif_rx_complete(dev);
+ if (unlikely(!vi->rvq->vq_ops->restart(vi->rvq))
+ && netif_rx_reschedule(dev, received))
+ goto again;
+
+ return 0;
+}
+
+static void free_old_xmit_skbs(struct virtnet_info *vi)
+{
+ struct sk_buff *skb;
+ unsigned int len;
+
+ while ((skb = vi->svq->vq_ops->get_buf(vi->svq, &len)) !=
NULL) {
+ pr_debug("Sent skb %p\n", skb);
+ __skb_unlink(skb, &vi->send);
+ vi->dev->stats.tx_bytes += len;
+ vi->dev->stats.tx_packets++;
+ kfree_skb(skb);
+ }
+}
+
+static int start_xmit(struct sk_buff *skb, struct net_device *dev)
+{
+ struct virtnet_info *vi = netdev_priv(dev);
+ int num, err;
+ struct scatterlist sg[1+MAX_SKB_FRAGS];
+ struct virtio_net_hdr *hdr;
+ const unsigned char *dest = ((struct ethhdr *)skb->data)->h_dest;
+
+ pr_debug("%s: xmit %p %02x:%02x:%02x:%02x:%02x:%02x\n",
+ dev->name, skb,
+ dest[0], dest[1], dest[2], dest[3], dest[4], dest[5]);
+
+ free_old_xmit_skbs(vi);
+
+ /* Encode metadata header at front. */
+ hdr = skb_vnet_hdr(skb);
+ if (skb->ip_summed == CHECKSUM_PARTIAL) {
+ hdr->flags = VIRTIO_NET_HDR_F_NEEDS_CSUM;
+ hdr->csum_start = skb->csum_start - skb_headroom(skb);
+ hdr->csum_offset = skb->csum_offset;
+ } else {
+ hdr->flags = 0;
+ hdr->csum_offset = hdr->csum_start = 0;
+ }
+
+ if (skb_is_gso(skb)) {
+ hdr->gso_size = skb_shinfo(skb)->gso_size;
+ if (skb_shinfo(skb)->gso_type & SKB_GSO_TCP_ECN)
+ hdr->gso_type = VIRTIO_NET_HDR_GSO_TCPV4_ECN;
+ else if (skb_shinfo(skb)->gso_type & SKB_GSO_TCPV4)
+ hdr->gso_type = VIRTIO_NET_HDR_GSO_TCPV4;
+ else if (skb_shinfo(skb)->gso_type & SKB_GSO_TCPV6)
+ hdr->gso_type = VIRTIO_NET_HDR_GSO_TCPV6;
+ else if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP)
+ hdr->gso_type = VIRTIO_NET_HDR_GSO_UDP;
+ else
+ BUG();
+ } else {
+ hdr->gso_type = VIRTIO_NET_HDR_GSO_NONE;
+ hdr->gso_size = 0;
+ }
+
+ vnet_hdr_to_sg(sg, skb);
+ num = skb_to_sgvec(skb, sg+1, 0, skb->len) + 1;
+ __skb_queue_head(&vi->send, skb);
+ err = vi->svq->vq_ops->add_buf(vi->svq, sg, num, 0, skb);
+ if (err) {
+ pr_debug("%s: virtio not prepared to send\n", dev->name);
+ skb_unlink(skb, &vi->send);
+ netif_stop_queue(dev);
+ return NETDEV_TX_BUSY;
+ }
+ vi->svq->vq_ops->kick(vi->svq);
+
+ return 0;
+}
+
+static int virtnet_open(struct net_device *dev)
+{
+ struct virtnet_info *vi = netdev_priv(dev);
+
+ try_fill_recv(vi);
+
+ /* If we didn't even get one input buffer, we're useless. */
+ if (vi->num == 0)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int virtnet_close(struct net_device *dev)
+{
+ struct virtnet_info *vi = netdev_priv(dev);
+ struct sk_buff *skb;
+
+ /* networking core has neutered skb_xmit_done/skb_recv_done, so don't
+ * worry about races vs. get(). */
+ vi->rvq->vq_ops->shutdown(vi->rvq);
+ while ((skb = __skb_dequeue(&vi->recv)) != NULL) {
+ kfree_skb(skb);
+ vi->num--;
+ }
+ vi->svq->vq_ops->shutdown(vi->svq);
+ while ((skb = __skb_dequeue(&vi->send)) != NULL)
+ kfree_skb(skb);
+
+ BUG_ON(vi->num != 0);
+ return 0;
+}
+
+static int virtnet_probe(struct virtio_device *vdev)
+{
+ int err;
+ unsigned int len;
+ struct net_device *dev;
+ struct virtnet_info *vi;
+ void *token;
+
+ /* Allocate ourselves a network device with room for our info */
+ dev = alloc_etherdev(sizeof(struct virtnet_info));
+ if (!dev)
+ return -ENOMEM;
+
+ /* Set up network device as normal. */
+ SET_MODULE_OWNER(dev);
+ ether_setup(dev);
+ dev->open = virtnet_open;
+ dev->stop = virtnet_close;
+ dev->poll = virtnet_poll;
+ dev->hard_start_xmit = start_xmit;
+ dev->weight = 16;
+ dev->features = NETIF_F_HIGHDMA;
+ SET_NETDEV_DEV(dev, &vdev->dev);
+
+ /* Do we support "hardware" checksums? */
+ token = vdev->config->find(vdev, VIRTIO_CONFIG_NET_F, &len);
+ if (virtio_use_bit(vdev, token, len, VIRTIO_NET_F_NO_CSUM)) {
+ /* This opens up the world of extra features. */
+ dev->features |= NETIF_F_HW_CSUM|NETIF_F_SG|NETIF_F_FRAGLIST;
+ if (virtio_use_bit(vdev, token, len, VIRTIO_NET_F_TSO4))
+ dev->features |= NETIF_F_TSO;
+ if (virtio_use_bit(vdev, token, len, VIRTIO_NET_F_UFO))
+ dev->features |= NETIF_F_UFO;
+ if (virtio_use_bit(vdev, token, len, VIRTIO_NET_F_TSO4_ECN))
+ dev->features |= NETIF_F_TSO_ECN;
+ if (virtio_use_bit(vdev, token, len, VIRTIO_NET_F_TSO6))
+ dev->features |= NETIF_F_TSO6;
+ }
+
+ /* Configuration may specify what MAC to use. Otherwise random. */
+ token = vdev->config->find(vdev, VIRTIO_CONFIG_NET_MAC_F, &len);
+ if (token) {
+ dev->addr_len = len;
+ vdev->config->get(vdev, token, dev->dev_addr, len);
+ } else
+ random_ether_addr(dev->dev_addr);
+
+ /* Set up our device-specific information */
+ vi = netdev_priv(dev);
+ vi->dev = dev;
+ vi->vdev = vdev;
+
+ /* We expect two virtqueues, receive then send. */
+ vi->rvq = vdev->config->find_vq(vdev, skb_recv_done);
+ if (IS_ERR(vi->rvq)) {
+ err = PTR_ERR(vi->rvq);
+ goto free;
+ }
+
+ vi->svq = vdev->config->find_vq(vdev, skb_xmit_done);
+ if (IS_ERR(vi->svq)) {
+ err = PTR_ERR(vi->svq);
+ goto free_recv;
+ }
+
+ /* Initialize our empty receive and send queues. */
+ skb_queue_head_init(&vi->recv);
+ skb_queue_head_init(&vi->send);
+
+ err = register_netdev(dev);
+ if (err) {
+ pr_debug("virtio_net: registering device failed\n");
+ goto free_send;
+ }
+ pr_debug("virtnet: registered device %s\n", dev->name);
+ vdev->priv = vi;
+ return 0;
+
+free_send:
+ vdev->config->del_vq(vi->svq);
+free_recv:
+ vdev->config->del_vq(vi->rvq);
+free:
+ free_netdev(dev);
+ return err;
+}
+
+static void virtnet_remove(struct virtio_device *vdev)
+{
+ unregister_netdev(vdev->priv);
+ free_netdev(vdev->priv);
+}
+
+static struct virtio_device_id id_table[] = {
+ { VIRTIO_ID_NET, VIRTIO_DEV_ANY_ID },
+ { 0 },
+};
+
+static struct virtio_driver virtio_net = {
+ .driver.name = KBUILD_MODNAME,
+ .driver.owner = THIS_MODULE,
+ .id_table = id_table,
+ .probe = virtnet_probe,
+ .remove = __devexit_p(virtnet_remove),
+};
+
+static int __init init(void)
+{
+ return register_virtio_driver(&virtio_net);
+}
+
+static void __exit fini(void)
+{
+ unregister_virtio_driver(&virtio_net);
+}
+module_init(init);
+module_exit(fini);
+
+MODULE_DEVICE_TABLE(virtio, id_table);
+MODULE_DESCRIPTION("Virtio network driver");
+MODULE_LICENSE("GPL");
==================================================================--- /dev/null
+++ b/include/linux/virtio_net.h
@@ -0,0 +1,36 @@
+#ifndef _LINUX_VIRTIO_NET_H
+#define _LINUX_VIRTIO_NET_H
+#include <linux/virtio_config.h>
+
+/* The ID for virtio_net */
+#define VIRTIO_ID_NET 1
+
+/* The bitmap of config for virtio net */
+#define VIRTIO_CONFIG_NET_F 0x40
+#define VIRTIO_NET_F_NO_CSUM 0
+#define VIRTIO_NET_F_TSO4 1
+#define VIRTIO_NET_F_UFO 2
+#define VIRTIO_NET_F_TSO4_ECN 3
+#define VIRTIO_NET_F_TSO6 4
+
+/* The config defining mac address. */
+#define VIRTIO_CONFIG_NET_MAC_F 0x41
+
+/* This is the first element of the scatter-gather list. If you don't
+ * specify GSO or CSUM features, you can simply ignore the header. */
+struct virtio_net_hdr
+{
+#define VIRTIO_NET_HDR_F_NEEDS_CSUM 1 // Use csum_start, csum_offset
+ __u8 flags;
+#define VIRTIO_NET_HDR_GSO_NONE 0 // Not a GSO frame
+#define VIRTIO_NET_HDR_GSO_TCPV4 1 // GSO frame, IPv4 TCP (TSO)
+/* FIXME: Do we need this? If they said they can handle ECN, do they care? */
+#define VIRTIO_NET_HDR_GSO_TCPV4_ECN 2 // GSO frame, IPv4 TCP w/ ECN
+#define VIRTIO_NET_HDR_GSO_UDP 3 // GSO frame, IPv4 UDP (UFO)
+#define VIRTIO_NET_HDR_GSO_TCPV6 4 // GSO frame, IPv6 TCP
+ __u8 gso_type;
+ __u16 gso_size;
+ __u16 csum_start;
+ __u16 csum_offset;
+};
+#endif /* _LINUX_VIRTIO_NET_H */
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:21 UTC
[patch 25/43] lguest: Move register setup into i386_core.c
Move setup_regs() to lguest_arch_setup_regs() in i386_core.c given
that this is very architecture specific.
Signed-off-by: Jes Sorensen <jes@sgi.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
drivers/lguest/i386_core.c | 36 ++++++++++++++++++++++++++++++++++++
drivers/lguest/lg.h | 1 +
drivers/lguest/lguest_user.c | 37 +------------------------------------
3 files changed, 38 insertions(+), 36 deletions(-)
==================================================================---
a/drivers/lguest/i386_core.c
+++ b/drivers/lguest/i386_core.c
@@ -535,3 +535,39 @@ int lguest_arch_init_hypercalls(struct l
/* Now we've examined the hypercall code; our Guest can make requests.
There
* is one other way we can do things for the Guest, as we see in
* emulate_insn(). :*/
+
+/*L:030 lguest_arch_setup_regs()
+ *
+ * Most of the Guest's registers are left alone: we used get_zeroed_page()
to
+ * allocate the structure, so they will be 0. */
+void lguest_arch_setup_regs(struct lguest *lg, unsigned long start)
+{
+ struct lguest_regs *regs = lg->regs;
+
+ /* There are four "segment" registers which the Guest needs to boot:
+ * The "code segment" register (cs) refers to the kernel code
segment
+ * __KERNEL_CS, and the "data", "extra" and
"stack" segment registers
+ * refer to the kernel data segment __KERNEL_DS.
+ *
+ * The privilege level is packed into the lower bits. The Guest runs
+ * at privilege level 1 (GUEST_PL).*/
+ regs->ds = regs->es = regs->ss = __KERNEL_DS|GUEST_PL;
+ regs->cs = __KERNEL_CS|GUEST_PL;
+
+ /* The "eflags" register contains miscellaneous flags. Bit 1
(0x002)
+ * is supposed to always be "1". Bit 9 (0x200) controls whether
+ * interrupts are enabled. We always leave interrupts enabled while
+ * running the Guest. */
+ regs->eflags = 0x202;
+
+ /* The "Extended Instruction Pointer" register says where the Guest
is
+ * running. */
+ regs->eip = start;
+
+ /* %esi points to our boot information, at physical address 0, so don't
+ * touch it. */
+ /* There are a couple of GDT entries the Guest expects when first
+ * booting. */
+
+ setup_guest_gdt(lg);
+}
==================================================================---
a/drivers/lguest/lg.h
+++ b/drivers/lguest/lg.h
@@ -204,6 +204,7 @@ void lguest_arch_handle_trap(struct lgue
void lguest_arch_handle_trap(struct lguest *lg);
int lguest_arch_init_hypercalls(struct lguest *lg);
int lguest_arch_do_hcall(struct lguest *lg, struct hcall_args *args);
+void lguest_arch_setup_regs(struct lguest *lg, unsigned long start);
/* <arch>_switcher.S: */
extern char start_switcher_text[], end_switcher_text[], switch_to_guest[];
==================================================================---
a/drivers/lguest/lguest_user.c
+++ b/drivers/lguest/lguest_user.c
@@ -9,37 +9,6 @@
#include <linux/fs.h>
#include "lg.h"
-/*L:030 setup_regs() doesn't really belong in this file, but it gives us an
- * early glimpse deeper into the Host so it's worth having here.
- *
- * Most of the Guest's registers are left alone: we used get_zeroed_page()
to
- * allocate the structure, so they will be 0. */
-static void setup_regs(struct lguest_regs *regs, unsigned long start)
-{
- /* There are four "segment" registers which the Guest needs to boot:
- * The "code segment" register (cs) refers to the kernel code
segment
- * __KERNEL_CS, and the "data", "extra" and
"stack" segment registers
- * refer to the kernel data segment __KERNEL_DS.
- *
- * The privilege level is packed into the lower bits. The Guest runs
- * at privilege level 1 (GUEST_PL).*/
- regs->ds = regs->es = regs->ss = __KERNEL_DS|GUEST_PL;
- regs->cs = __KERNEL_CS|GUEST_PL;
-
- /* The "eflags" register contains miscellaneous flags. Bit 1
(0x002)
- * is supposed to always be "1". Bit 9 (0x200) controls whether
- * interrupts are enabled. We always leave interrupts enabled while
- * running the Guest. */
- regs->eflags = 0x202;
-
- /* The "Extended Instruction Pointer" register says where the Guest
is
- * running. */
- regs->eip = start;
-
- /* %esi points to our boot information, at physical address 0, so don't
- * touch it. */
-}
-
/*L:310 To send DMA into the Guest, the Launcher needs to be able to ask for a
* DMA buffer. This is done by writing LHREQ_GETDMA and the key to
* /dev/lguest. */
@@ -214,11 +183,7 @@ static int initialize(struct file *file,
/* Now we initialize the Guest's registers, handing it the start
* address. */
- setup_regs(lg->regs, args[3]);
-
- /* There are a couple of GDT entries the Guest expects when first
- * booting. */
- setup_guest_gdt(lg);
+ lguest_arch_setup_regs(lg, args[3]);
/* The timer for lguest's clock needs initialization. */
init_clockdev(lg);
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:45 UTC
[patch 24/43] lguest: Change example launcher to use unsigned long not u32
Apply Clue 2x4 to lguest userland<->kernel handling code and the
lguest launcher. Pointers are not to be passed in u32's!
Basic rule of thumb: Anything passing u32's back and forth should be
passing unsigned longs to be portable to 64 bit archs.
For those who forgotten already, I repeat: NO POINTERS IN u32!
Signed-off-by: Jes Sorensen <jes@sgi.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
Documentation/lguest/lguest.c | 29 +++++++++++++++--------------
drivers/lguest/lguest_user.c | 33 +++++++++++++++++----------------
2 files changed, 32 insertions(+), 30 deletions(-)
==================================================================---
a/Documentation/lguest/lguest.c
+++ b/Documentation/lguest/lguest.c
@@ -473,9 +473,9 @@ static unsigned long setup_pagetables(un
unsigned long initrd_size,
unsigned long page_offset)
{
- u32 *pgdir, *linear;
+ unsigned long *pgdir, *linear;
unsigned int mapped_pages, i, linear_pages;
- unsigned int ptes_per_page = getpagesize()/sizeof(u32);
+ unsigned int ptes_per_page = getpagesize()/sizeof(void *);
/* Ideally we map all physical memory starting at page_offset.
* However, if page_offset is 0xC0000000 we can only map 1G of physical
@@ -505,7 +505,7 @@ static unsigned long setup_pagetables(un
* continue from there. */
for (i = 0; i < mapped_pages; i += ptes_per_page) {
pgdir[(i + page_offset/getpagesize())/ptes_per_page]
- = ((to_guest_phys(linear) + i*sizeof(u32))
+ = ((to_guest_phys(linear) + i*sizeof(void *))
| PAGE_PRESENT);
}
@@ -537,12 +537,13 @@ static void concat(char *dst, char *args
* the base of guest "physical" memory, the top physical page to
allow, the
* top level pagetable, the entry point and the page_offset constant for the
* Guest. */
-static int tell_kernel(u32 pgdir, u32 start, u32 page_offset)
-{
- u32 args[] = { LHREQ_INITIALIZE,
- (unsigned long)guest_base,
- guest_limit / getpagesize(),
- pgdir, start, page_offset };
+static int tell_kernel(unsigned long pgdir, unsigned long start,
+ unsigned long page_offset)
+{
+ unsigned long args[] = { LHREQ_INITIALIZE,
+ (unsigned long)guest_base,
+ guest_limit / getpagesize(),
+ pgdir, start, page_offset };
int fd;
verbose("Guest: %p - %p (%#lx)\n",
@@ -586,7 +587,7 @@ static void wake_parent(int pipefd, int
for (;;) {
fd_set rfds = devices->infds;
- u32 args[] = { LHREQ_BREAK, 1 };
+ unsigned long args[] = { LHREQ_BREAK, 1 };
/* Wait until input is ready from one of the devices. */
select(devices->max_infd+1, &rfds, NULL, NULL, NULL);
@@ -684,7 +685,7 @@ static u32 *get_dma_buffer(int fd, void
static u32 *get_dma_buffer(int fd, void *key,
struct iovec iov[], unsigned int *num, u32 *irq)
{
- u32 buf[] = { LHREQ_GETDMA, to_guest_phys(key) };
+ unsigned long buf[] = { LHREQ_GETDMA, to_guest_phys(key) };
unsigned long udma;
u32 *res;
@@ -705,7 +706,7 @@ static u32 *get_dma_buffer(int fd, void
/* This is a convenient routine to send the Guest an interrupt. */
static void trigger_irq(int fd, u32 irq)
{
- u32 buf[] = { LHREQ_IRQ, irq };
+ unsigned long buf[] = { LHREQ_IRQ, irq };
if (write(fd, buf, sizeof(buf)) != 0)
err(1, "Triggering irq %i", irq);
}
@@ -787,7 +788,7 @@ static bool handle_console_input(int fd,
struct timeval now;
gettimeofday(&now, NULL);
if (now.tv_sec <= abort->start.tv_sec+1) {
- u32 args[] = { LHREQ_BREAK, 0 };
+ unsigned long args[] = { LHREQ_BREAK, 0 };
/* Close the fd so Waker will know it has to
* exit. */
close(waker_fd);
@@ -1365,7 +1366,7 @@ run_guest(int lguest_fd, struct device_l
run_guest(int lguest_fd, struct device_list *device_list)
{
for (;;) {
- u32 args[] = { LHREQ_BREAK, 0 };
+ unsigned long args[] = { LHREQ_BREAK, 0 };
unsigned long arr[2];
int readval;
==================================================================---
a/drivers/lguest/lguest_user.c
+++ b/drivers/lguest/lguest_user.c
@@ -43,7 +43,7 @@ static void setup_regs(struct lguest_reg
/*L:310 To send DMA into the Guest, the Launcher needs to be able to ask for a
* DMA buffer. This is done by writing LHREQ_GETDMA and the key to
* /dev/lguest. */
-static long user_get_dma(struct lguest *lg, const u32 __user *input)
+static long user_get_dma(struct lguest *lg, const unsigned long __user *input)
{
unsigned long key, udma, irq;
@@ -67,7 +67,7 @@ static long user_get_dma(struct lguest *
/*L:315 To force the Guest to stop running and return to the Launcher, the
* Waker sets writes LHREQ_BREAK and the value "1" to /dev/lguest.
The
* Launcher then writes LHREQ_BREAK and "0" to release the Waker. */
-static int break_guest_out(struct lguest *lg, const u32 __user *input)
+static int break_guest_out(struct lguest *lg, const unsigned long __user
*input)
{
unsigned long on;
@@ -90,9 +90,9 @@ static int break_guest_out(struct lguest
/*L:050 Sending an interrupt is done by writing LHREQ_IRQ and an interrupt
* number to /dev/lguest. */
-static int user_send_irq(struct lguest *lg, const u32 __user *input)
-{
- u32 irq;
+static int user_send_irq(struct lguest *lg, const unsigned long __user *input)
+{
+ unsigned long irq;
if (get_user(irq, input) != 0)
return -EFAULT;
@@ -142,8 +142,8 @@ static ssize_t read(struct file *file, c
return run_guest(lg, (unsigned long __user *)user);
}
-/*L:020 The initialization write supplies 5 32-bit values (in addition to the
- * 32-bit LHREQ_INITIALIZE value). These are:
+/*L:020 The initialization write supplies 5 pointer sized (32 or 64 bit)
+ * values (in addition to the LHREQ_INITIALIZE value). These are:
*
* base: The start of the Guest-physical memory inside the Launcher memory.
*
@@ -162,13 +162,13 @@ static ssize_t read(struct file *file, c
* quickly converted from physical to virtual by adding PAGE_OFFSET. It's
* 0xC0000000 (3G) by default, but it's configurable at kernel build time.
*/
-static int initialize(struct file *file, const u32 __user *input)
+static int initialize(struct file *file, const unsigned long __user *input)
{
/* "struct lguest" contains everything we (the Host) know about a
* Guest. */
struct lguest *lg;
int err;
- u32 args[5];
+ unsigned long args[5];
/* We grab the Big Lguest lock, which protects against multiple
* simultaneous initializations. */
@@ -259,17 +259,18 @@ unlock:
* start with a 32 bit number: for the first write this must be
* LHREQ_INITIALIZE to set up the Guest. After that the Launcher can use
* writes of other values to get DMA buffers and send interrupts. */
-static ssize_t write(struct file *file, const char __user *input,
+static ssize_t write(struct file *file, const char __user *in,
size_t size, loff_t *off)
{
/* Once the guest is initialized, we hold the "struct lguest" in the
* file private data. */
struct lguest *lg = file->private_data;
- u32 req;
+ const unsigned long __user *input = (const unsigned long __user *)in;
+ unsigned long req;
if (get_user(req, input) != 0)
return -EFAULT;
- input += sizeof(req);
+ input++;
/* If you haven't initialized, you must do that first. */
if (req != LHREQ_INITIALIZE && !lg)
@@ -285,13 +286,13 @@ static ssize_t write(struct file *file,
switch (req) {
case LHREQ_INITIALIZE:
- return initialize(file, (const u32 __user *)input);
+ return initialize(file, input);
case LHREQ_GETDMA:
- return user_get_dma(lg, (const u32 __user *)input);
+ return user_get_dma(lg, input);
case LHREQ_IRQ:
- return user_send_irq(lg, (const u32 __user *)input);
+ return user_send_irq(lg, input);
case LHREQ_BREAK:
- return break_guest_out(lg, (const u32 __user *)input);
+ return break_guest_out(lg, input);
default:
return -EINVAL;
}
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:45 UTC
[patch 34/43] lguest: Block driver using virtio.
The block driver uses scatter-gather lists with sg[0] being the
request information (struct virtio_blk_outhdr) with the type, sector
and inbuf id. The next N sg entries are the bio itself, then the last
sg is the status byte. Whether the N entries are in or out depends on
whether it's a read or a write.
We accept the normal (SCSI) ioctls: they get handed through to the other
side which can then handle it or reply that it's unsupported. It's
not clear that this actually works in general, since I don't know
if blk_pc_request() requests have an accurate rq_data_dir().
Although we try to reply -ENOTTY on unsupported commands, ioctl(fd,
CDROMEJECT) returns success to userspace. This needs a separate
patch.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: Jens Axboe <jens.axboe@oracle.com>
---
drivers/block/Kconfig | 6
drivers/block/Makefile | 1
drivers/block/virtio_blk.c | 327 ++++++++++++++++++++++++++++++++++++++++++++
include/linux/Kbuild | 1
include/linux/virtio_blk.h | 51 ++++++
5 files changed, 386 insertions(+)
==================================================================---
a/drivers/block/Kconfig
+++ b/drivers/block/Kconfig
@@ -443,4 +443,10 @@ config XEN_BLKDEV_FRONTEND
block device driver. It communicates with a back-end driver
in another domain which drives the actual block device.
+config VIRTIO_BLK
+ tristate "Virtio block driver (EXPERIMENTAL)"
+ depends on EXPERIMENTAL && VIRTIO
+ ---help---
+ This is the virtual block driver for lguest. Say Y or M.
+
endif # BLK_DEV
==================================================================---
a/drivers/block/Makefile
+++ b/drivers/block/Makefile
@@ -25,6 +25,7 @@ obj-$(CONFIG_BLK_DEV_UMEM) += umem.o
obj-$(CONFIG_BLK_DEV_UMEM) += umem.o
obj-$(CONFIG_BLK_DEV_NBD) += nbd.o
obj-$(CONFIG_BLK_DEV_CRYPTOLOOP) += cryptoloop.o
+obj-$(CONFIG_VIRTIO_BLK) += virtio_blk.o
obj-$(CONFIG_VIODASD) += viodasd.o
obj-$(CONFIG_BLK_DEV_SX8) += sx8.o
==================================================================--- /dev/null
+++ b/drivers/block/virtio_blk.c
@@ -0,0 +1,327 @@
+//#define DEBUG
+#include <linux/spinlock.h>
+#include <linux/blkdev.h>
+#include <linux/hdreg.h>
+#include <linux/virtio.h>
+#include <linux/virtio_blk.h>
+#include <linux/virtio_blk.h>
+
+static unsigned char virtblk_index = 'a';
+struct virtio_blk
+{
+ spinlock_t lock;
+
+ struct virtio_device *vdev;
+ struct virtqueue *vq;
+
+ /* The disk structure for the kernel. */
+ struct gendisk *disk;
+
+ /* Request tracking. */
+ struct list_head reqs;
+
+ mempool_t *pool;
+
+ /* Scatterlist: can be too big for stack. */
+ struct scatterlist sg[3+MAX_PHYS_SEGMENTS];
+};
+
+struct virtblk_req
+{
+ struct list_head list;
+ struct request *req;
+ struct virtio_blk_outhdr out_hdr;
+ struct virtio_blk_inhdr in_hdr;
+};
+
+static void end_dequeued_request(struct request *req,
+ struct request_queue *q, int uptodate)
+{
+ /* And so the insanity of the block layer infects us here. */
+ int nsectors = req->hard_nr_sectors;
+
+ if (blk_pc_request(req)) {
+ nsectors = (req->data_len + 511) >> 9;
+ if (!nsectors)
+ nsectors = 1;
+ }
+ if (end_that_request_first(req, uptodate, nsectors))
+ BUG();
+ add_disk_randomness(req->rq_disk);
+ end_that_request_last(req, uptodate);
+}
+
+static bool blk_done(struct virtqueue *vq)
+{
+ struct virtio_blk *vblk = vq->vdev->priv;
+ struct virtblk_req *vbr;
+ unsigned int len;
+ unsigned long flags;
+
+ spin_lock_irqsave(&vblk->lock, flags);
+ while ((vbr = vblk->vq->vq_ops->get_buf(vblk->vq, &len)) !=
NULL) {
+ int uptodate;
+ switch (vbr->in_hdr.status) {
+ case VIRTIO_BLK_S_OK:
+ uptodate = 1;
+ break;
+ case VIRTIO_BLK_S_UNSUPP:
+ uptodate = -ENOTTY;
+ break;
+ default:
+ uptodate = 0;
+ break;
+ }
+
+ end_dequeued_request(vbr->req, vblk->disk->queue, uptodate);
+ list_del(&vbr->list);
+ mempool_free(vbr, vblk->pool);
+ }
+ /* In case queue is stopped waiting for more buffers. */
+ blk_start_queue(vblk->disk->queue);
+ spin_unlock_irqrestore(&vblk->lock, flags);
+ return true;
+}
+
+static bool do_req(struct request_queue *q, struct virtio_blk *vblk,
+ struct request *req)
+{
+ unsigned long num, out, in;
+ struct virtblk_req *vbr;
+
+ vbr = mempool_alloc(vblk->pool, GFP_ATOMIC);
+ if (!vbr)
+ /* When another request finishes we'll try again. */
+ return false;
+
+ vbr->req = req;
+ if (blk_fs_request(vbr->req)) {
+ vbr->out_hdr.type = 0;
+ vbr->out_hdr.sector = vbr->req->sector;
+ vbr->out_hdr.ioprio = vbr->req->ioprio;
+ } else if (blk_pc_request(vbr->req)) {
+ vbr->out_hdr.type = VIRTIO_BLK_T_SCSI_CMD;
+ vbr->out_hdr.sector = 0;
+ vbr->out_hdr.ioprio = vbr->req->ioprio;
+ } else {
+ /* We don't put anything else in the queue. */
+ BUG();
+ }
+
+ if (blk_barrier_rq(vbr->req))
+ vbr->out_hdr.type |= VIRTIO_BLK_T_BARRIER;
+
+ vblk->sg[0].page = virt_to_page(&vbr->out_hdr);
+ vblk->sg[0].offset = offset_in_page(&vbr->out_hdr);
+ vblk->sg[0].length = sizeof(vbr->out_hdr);
+ num = blk_rq_map_sg(q, vbr->req, vblk->sg+1);
+ vblk->sg[num+1].page = virt_to_page(&vbr->in_hdr);
+ vblk->sg[num+1].offset = offset_in_page(&vbr->in_hdr);
+ vblk->sg[num+1].length = sizeof(vbr->in_hdr);
+
+ if (rq_data_dir(vbr->req) == WRITE) {
+ vbr->out_hdr.type |= VIRTIO_BLK_T_OUT;
+ out = 1 + num;
+ in = 1;
+ } else {
+ vbr->out_hdr.type |= VIRTIO_BLK_T_IN;
+ out = 1;
+ in = 1 + num;
+ }
+
+ if (vblk->vq->vq_ops->add_buf(vblk->vq, vblk->sg, out, in,
vbr)) {
+ mempool_free(vbr, vblk->pool);
+ return false;
+ }
+
+ list_add_tail(&vbr->list, &vblk->reqs);
+ return true;
+}
+
+static void do_virtblk_request(struct request_queue *q)
+{
+ struct virtio_blk *vblk = NULL;
+ struct request *req;
+ unsigned int issued = 0;
+
+ while ((req = elv_next_request(q)) != NULL) {
+ vblk = req->rq_disk->private_data;
+ BUG_ON(req->nr_phys_segments > ARRAY_SIZE(vblk->sg));
+
+ /* If this request fails, stop queue and wait for something to
+ finish to restart it. */
+ if (!do_req(q, vblk, req)) {
+ blk_stop_queue(q);
+ break;
+ }
+ blkdev_dequeue_request(req);
+ issued++;
+ }
+
+ if (issued)
+ vblk->vq->vq_ops->kick(vblk->vq);
+}
+
+static int virtblk_ioctl(struct inode *inode, struct file *filp,
+ unsigned cmd, unsigned long data)
+{
+ return scsi_cmd_ioctl(filp, inode->i_bdev->bd_disk->queue,
+ inode->i_bdev->bd_disk, cmd,
+ (void __user *)data);
+}
+
+static struct block_device_operations virtblk_fops = {
+ .ioctl = virtblk_ioctl,
+ .owner = THIS_MODULE,
+};
+
+static int virtblk_probe(struct virtio_device *vdev)
+{
+ struct virtio_blk *vblk;
+ int err, major;
+ void *token;
+ unsigned int len;
+ u64 cap;
+ u32 v;
+
+ vdev->priv = vblk = kmalloc(sizeof(*vblk), GFP_KERNEL);
+ if (!vblk) {
+ err = -ENOMEM;
+ goto out;
+ }
+
+ INIT_LIST_HEAD(&vblk->reqs);
+ spin_lock_init(&vblk->lock);
+ vblk->vdev = vdev;
+
+ /* We expect one virtqueue, for output. */
+ vblk->vq = vdev->config->find_vq(vdev, blk_done);
+ if (IS_ERR(vblk->vq)) {
+ err = PTR_ERR(vblk->vq);
+ goto out_free_vblk;
+ }
+
+ vblk->pool = mempool_create_kmalloc_pool(1,sizeof(struct virtblk_req));
+ if (!vblk->pool) {
+ err = -ENOMEM;
+ goto out_free_vq;
+ }
+
+ major = register_blkdev(0, "virtblk");
+ if (major < 0) {
+ err = major;
+ goto out_mempool;
+ }
+
+ /* FIXME: How many partitions? How long is a piece of string? */
+ vblk->disk = alloc_disk(1 << 4);
+ if (!vblk->disk) {
+ err = -ENOMEM;
+ goto out_unregister_blkdev;
+ }
+
+ vblk->disk->queue = blk_init_queue(do_virtblk_request,
&vblk->lock);
+ if (!vblk->disk->queue) {
+ err = -ENOMEM;
+ goto out_put_disk;
+ }
+
+ sprintf(vblk->disk->disk_name, "vd%c", virtblk_index++);
+ vblk->disk->major = major;
+ vblk->disk->first_minor = 0;
+ vblk->disk->private_data = vblk;
+ vblk->disk->fops = &virtblk_fops;
+
+ /* If barriers are supported, tell block layer that queue is ordered */
+ token = vdev->config->find(vdev, VIRTIO_CONFIG_BLK_F, &len);
+ if (virtio_use_bit(vdev, token, len, VIRTIO_BLK_F_BARRIER))
+ blk_queue_ordered(vblk->disk->queue, QUEUE_ORDERED_TAG, NULL);
+
+ err = virtio_config_val(vdev, VIRTIO_CONFIG_BLK_F_CAPACITY, &cap);
+ if (err) {
+ dev_err(&vdev->dev, "Bad/missing capacity in config\n");
+ goto out_put_disk;
+ }
+
+ /* If capacity is too big, truncate with warning. */
+ if ((sector_t)cap != cap) {
+ dev_warn(&vdev->dev, "Capacity %llu too large:
truncating\n",
+ (unsigned long long)cap);
+ cap = (sector_t)-1;
+ }
+ set_capacity(vblk->disk, cap);
+
+ err = virtio_config_val(vdev, VIRTIO_CONFIG_BLK_F_SIZE_MAX, &v);
+ if (!err)
+ blk_queue_max_segment_size(vblk->disk->queue, v);
+ else if (err != -ENOENT) {
+ dev_err(&vdev->dev, "Bad SIZE_MAX in config\n");
+ goto out_put_disk;
+ }
+
+ err = virtio_config_val(vdev, VIRTIO_CONFIG_BLK_F_SEG_MAX, &v);
+ if (!err)
+ blk_queue_max_hw_segments(vblk->disk->queue, v);
+ else if (err != -ENOENT) {
+ dev_err(&vdev->dev, "Bad SEG_MAX in config\n");
+ goto out_put_disk;
+ }
+
+ add_disk(vblk->disk);
+ return 0;
+
+out_put_disk:
+ put_disk(vblk->disk);
+out_unregister_blkdev:
+ unregister_blkdev(major, "virtblk");
+out_mempool:
+ mempool_destroy(vblk->pool);
+out_free_vq:
+ vdev->config->del_vq(vblk->vq);
+out_free_vblk:
+ kfree(vblk);
+out:
+ return err;
+}
+
+static void virtblk_remove(struct virtio_device *vdev)
+{
+ struct virtio_blk *vblk = vdev->priv;
+ int major = vblk->disk->major;
+
+ BUG_ON(!list_empty(&vblk->reqs));
+ blk_cleanup_queue(vblk->disk->queue);
+ put_disk(vblk->disk);
+ unregister_blkdev(major, "virtblk");
+ mempool_destroy(vblk->pool);
+ kfree(vblk);
+}
+
+static struct virtio_device_id id_table[] = {
+ { VIRTIO_ID_BLOCK, VIRTIO_DEV_ANY_ID },
+ { 0 },
+};
+
+static struct virtio_driver virtio_blk = {
+ .driver.name = KBUILD_MODNAME,
+ .driver.owner = THIS_MODULE,
+ .id_table = id_table,
+ .probe = virtblk_probe,
+ .remove = __devexit_p(virtblk_remove),
+};
+
+static int __init init(void)
+{
+ return register_virtio_driver(&virtio_blk);
+}
+
+static void __exit fini(void)
+{
+ unregister_virtio_driver(&virtio_blk);
+}
+module_init(init);
+module_exit(fini);
+
+MODULE_DEVICE_TABLE(virtio, id_table);
+MODULE_DESCRIPTION("Virtio block driver");
+MODULE_LICENSE("GPL");
==================================================================---
a/include/linux/Kbuild
+++ b/include/linux/Kbuild
@@ -343,6 +343,7 @@ unifdef-y += utsname.h
unifdef-y += utsname.h
unifdef-y += videodev2.h
unifdef-y += videodev.h
+unifdef-y += virtio_blk.h
unifdef-y += wait.h
unifdef-y += wanrouter.h
unifdef-y += watchdog.h
==================================================================--- /dev/null
+++ b/include/linux/virtio_blk.h
@@ -0,0 +1,51 @@
+#ifndef _LINUX_VIRTIO_BLK_H
+#define _LINUX_VIRTIO_BLK_H
+#include <linux/virtio_config.h>
+
+/* The ID for virtio_block */
+#define VIRTIO_ID_BLOCK 2
+
+/* Feature bits */
+#define VIRTIO_CONFIG_BLK_F 0x40
+#define VIRTIO_BLK_F_BARRIER 1 /* Does host support barriers? */
+
+/* The capacity (in 512-byte sectors). */
+#define VIRTIO_CONFIG_BLK_F_CAPACITY 0x41
+/* The maximum segment size. */
+#define VIRTIO_CONFIG_BLK_F_SIZE_MAX 0x42
+/* The maximum number of segments. */
+#define VIRTIO_CONFIG_BLK_F_SEG_MAX 0x43
+
+/* These two define direction. */
+#define VIRTIO_BLK_T_IN 0
+#define VIRTIO_BLK_T_OUT 1
+
+/* This bit says it's a scsi command, not an actual read or write. */
+#define VIRTIO_BLK_T_SCSI_CMD 2
+
+/* Barrier before this op. */
+#define VIRTIO_BLK_T_BARRIER 0x80000000
+
+/* This is the first element of the read scatter-gather list. */
+struct virtio_blk_outhdr
+{
+ /* VIRTIO_BLK_T* */
+ __u32 type;
+ /* io priority. */
+ __u32 ioprio;
+ /* Sector (ie. 512 byte offset) */
+ __u64 sector;
+ /* Where to put reply. */
+ __u64 id;
+};
+
+#define VIRTIO_BLK_S_OK 0
+#define VIRTIO_BLK_S_IOERR 1
+#define VIRTIO_BLK_S_UNSUPP 2
+
+/* This is the first element of the write scatter-gather list */
+struct virtio_blk_inhdr
+{
+ unsigned char status;
+};
+#endif /* _LINUX_VIRTIO_BLK_H */
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:45 UTC
[patch 22/43] lguest: Introduce "hcall" pointer to indicate pending hypercall.
Currently we look at the "trapnum" to see if the Guest wants a
hypercall. But once the hypercall is done we have to reset trapnum to
a bogus value, otherwise if we exit to userspace and return, we'd run
the same hypercall twice (that was a nasty bug to find!).
This has two main effects:
1) When Jes's patch changes the hypercall args to be a generic "struct
hcall_args" we simply change the type of "lg->hcall".
It's set by
arch code, so if it has to copy args or something it can do so, and
point "hcall" into lg->arch somewhere.
2) Async hypercalls only get run when an actual hypercall is pending.
This simplfies the code a little and is a more logical semantic.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
drivers/lguest/core.c | 8 +++----
drivers/lguest/hypercalls.c | 48 +++++++++++++++++--------------------------
drivers/lguest/i386_core.c | 13 ++++++-----
drivers/lguest/lg.h | 3 ++
4 files changed, 34 insertions(+), 38 deletions(-)
==================================================================---
a/drivers/lguest/core.c
+++ b/drivers/lguest/core.c
@@ -198,10 +198,10 @@ int run_guest(struct lguest *lg, unsigne
{
/* We stop running once the Guest is dead. */
while (!lg->dead) {
- /* First we run any hypercalls the Guest wants done: either in
- * the hypercall ring in "struct lguest_data", or directly by
- * using int 31 (LGUEST_TRAP_ENTRY). */
- do_hypercalls(lg);
+ /* First we run any hypercalls the Guest wants done. */
+ if (lg->hcall)
+ do_hypercalls(lg);
+
/* It's possible the Guest did a SEND_DMA hypercall to the
* Launcher, in which case we return from the read() now. */
if (lg->dma_is_pending) {
==================================================================---
a/drivers/lguest/hypercalls.c
+++ b/drivers/lguest/hypercalls.c
@@ -246,19 +246,6 @@ static void initialize(struct lguest *lg
* is one other way we can do things for the Guest, as we see in
* emulate_insn(). */
-/*H:110 Tricky point: we mark the hypercall as "done" once we've
done it.
- * Normally we don't need to do this: the Guest will run again and update
the
- * trap number before we come back around the run_guest() loop to
- * do_hypercalls().
- *
- * However, if we are signalled or the Guest sends DMA to the Launcher, that
- * loop will exit without running the Guest. When it comes back it would try
- * to re-run the hypercall. */
-static void clear_hcall(struct lguest *lg)
-{
- lg->regs->trapnum = 255;
-}
-
/*H:100
* Hypercalls
*
@@ -267,16 +254,12 @@ static void clear_hcall(struct lguest *l
*/
void do_hypercalls(struct lguest *lg)
{
- /* Not initialized yet? */
+ /* Not initialized yet? This hypercall must do it. */
if (unlikely(!lg->lguest_data)) {
- /* Did the Guest make a hypercall? We might have come back for
- * some other reason (an interrupt, a different trap). */
- if (lg->regs->trapnum == LGUEST_TRAP_ENTRY) {
- /* Set up the "struct lguest_data" */
- initialize(lg);
- /* The hypercall is done. */
- clear_hcall(lg);
- }
+ /* Set up the "struct lguest_data" */
+ initialize(lg);
+ /* Hcall is done. */
+ lg->hcall = NULL;
return;
}
@@ -286,12 +269,21 @@ void do_hypercalls(struct lguest *lg)
do_async_hcalls(lg);
/* If we stopped reading the hypercall ring because the Guest did a
- * SEND_DMA to the Launcher, we want to return now. Otherwise if the
- * Guest asked us to do a hypercall, we do it. */
- if (!lg->dma_is_pending && lg->regs->trapnum ==
LGUEST_TRAP_ENTRY) {
- do_hcall(lg, lg->regs);
- /* The hypercall is done. */
- clear_hcall(lg);
+ * SEND_DMA to the Launcher, we want to return now. Otherwise we do
+ * the hypercall. */
+ if (!lg->dma_is_pending) {
+ do_hcall(lg, lg->hcall);
+ /* Tricky point: we reset the hcall pointer to mark the
+ * hypercall as "done". We use the hcall pointer rather than
+ * the trap number to indicate a hypercall is pending.
+ * Normally it doesn't matter: the Guest will run again and
+ * update the trap number before we come back here.
+ *
+ * However, if we are signalled or the Guest sends DMA to the
+ * Launcher, the run_guest() loop will exit without running the
+ * Guest. When it comes back it would try to re-run the
+ * hypercall. */
+ lg->hcall = NULL;
}
}
==================================================================---
a/drivers/lguest/i386_core.c
+++ b/drivers/lguest/i386_core.c
@@ -316,13 +316,14 @@ void lguest_arch_handle_trap(struct lgue
return;
break;
case 32 ... 255:
- /* These values mean a real interrupt occurred, in
- * which case the Host handler has already been run.
- * We just do a friendly check if another process
- * should now be run, then fall through to loop
- * around: */
+ /* These values mean a real interrupt occurred, in which case
+ * the Host handler has already been run. We just do a
+ * friendly check if another process should now be run, then
+ * return to run the Guest again */
cond_resched();
- case LGUEST_TRAP_ENTRY: /* Handled before re-entering Guest */
+ return;
+ case LGUEST_TRAP_ENTRY:
+ lg->hcall = lg->regs;
return;
}
==================================================================---
a/drivers/lguest/lg.h
+++ b/drivers/lguest/lg.h
@@ -111,6 +111,9 @@ struct lguest
u32 esp1;
u8 ss1;
+ /* If a hypercall was asked for, this points to the arguments. */
+ struct lguest_regs *hcall;
+
/* Do we need to stop what we're doing and return to userspace? */
int break_out;
wait_queue_head_t break_wq;
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
This attempts to implement a "virtual I/O" layer which should allow
common drivers to be efficiently used across most virtual I/O
mechanisms. It will no-doubt need further enhancement.
The virtio drivers add buffers to virtio queues; as the buffers are consumed
the driver "interrupt" callbacks are invoked.
There is also a generic implementation of config space which drivers can query
to get setup information from the host.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: Dor Laor <dor.laor@qumranet.com>
Cc: Arnd Bergmann <arnd@arndb.de>
---
drivers/Kconfig | 2
drivers/Makefile | 1
drivers/virtio/Kconfig | 3
drivers/virtio/Makefile | 1
drivers/virtio/config.c | 13 ++
drivers/virtio/virtio.c | 171 +++++++++++++++++++++++++++++++++++++++
include/linux/mod_devicetable.h | 6 +
include/linux/virtio.h | 110 +++++++++++++++++++++++++
include/linux/virtio_config.h | 111 +++++++++++++++++++++++++
9 files changed, 418 insertions(+)
==================================================================---
a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -87,4 +87,6 @@ source "drivers/kvm/Kconfig"
source "drivers/kvm/Kconfig"
source "drivers/uio/Kconfig"
+
+source "drivers/virtio/Kconfig"
endmenu
==================================================================---
a/drivers/Makefile
+++ b/drivers/Makefile
@@ -88,3 +88,4 @@ obj-$(CONFIG_HID) += hid/
obj-$(CONFIG_HID) += hid/
obj-$(CONFIG_PPC_PS3) += ps3/
obj-$(CONFIG_OF) += of/
+obj-$(CONFIG_VIRTIO) += virtio/
==================================================================--- /dev/null
+++ b/drivers/virtio/Kconfig
@@ -0,0 +1,3 @@
+# Virtio always gets selected by whoever wants it.
+config VIRTIO
+ bool
==================================================================--- /dev/null
+++ b/drivers/virtio/Makefile
@@ -0,0 +1,1 @@
+obj-$(CONFIG_VIRTIO) += virtio.o
==================================================================--- /dev/null
+++ b/drivers/virtio/config.c
@@ -0,0 +1,13 @@
+/* Configuration space parsing helpers for virtio.
+ *
+ * The configuration is [type][len][... len bytes ...] fields.
+ *
+ * Copyright 2007 Rusty Russell, IBM Corporation.
+ * GPL v2 or later.
+ */
+#include <linux/err.h>
+#include <linux/virtio.h>
+#include <linux/virtio_config.h>
+#include <linux/bug.h>
+#include <asm/system.h>
+
==================================================================--- /dev/null
+++ b/drivers/virtio/virtio.c
@@ -0,0 +1,171 @@
+#include <linux/virtio.h>
+#include <linux/spinlock.h>
+#include <linux/virtio_config.h>
+
+static ssize_t device_show(struct device *_d,
+ struct device_attribute *attr, char *buf)
+{
+ struct virtio_device *dev = container_of(_d,struct virtio_device,dev);
+ return sprintf(buf, "%hu", dev->id.device);
+}
+static ssize_t vendor_show(struct device *_d,
+ struct device_attribute *attr, char *buf)
+{
+ struct virtio_device *dev = container_of(_d,struct virtio_device,dev);
+ return sprintf(buf, "%hu", dev->id.vendor);
+}
+static ssize_t status_show(struct device *_d,
+ struct device_attribute *attr, char *buf)
+{
+ struct virtio_device *dev = container_of(_d,struct virtio_device,dev);
+ return sprintf(buf, "0x%08x", dev->config->get_status(dev));
+}
+static struct device_attribute virtio_dev_attrs[] = {
+ __ATTR_RO(device),
+ __ATTR_RO(vendor),
+ __ATTR_RO(status),
+ __ATTR_NULL
+};
+
+static inline int virtio_id_match(const struct virtio_device *dev,
+ const struct virtio_device_id *id)
+{
+ if (id->device != dev->id.device)
+ return 0;
+
+ return id->vendor == VIRTIO_DEV_ANY_ID || id->vendor !=
dev->id.vendor;
+}
+
+/* This looks through all the IDs a driver claims to support. If any of them
+ * match, we return 1 and the kernel will call virtio_dev_probe(). */
+static int virtio_dev_match(struct device *_dv, struct device_driver *_dr)
+{
+ unsigned int i;
+ struct virtio_device *dev = container_of(_dv,struct virtio_device,dev);
+ const struct virtio_device_id *ids;
+
+ ids = container_of(_dr, struct virtio_driver, driver)->id_table;
+ for (i = 0; ids[i].device; i++)
+ if (virtio_id_match(dev, &ids[i]))
+ return 1;
+ return 0;
+}
+
+static struct bus_type virtio_bus = {
+ .name = "virtio",
+ .match = virtio_dev_match,
+ .dev_attrs = virtio_dev_attrs,
+};
+
+static void add_status(struct virtio_device *dev, unsigned status)
+{
+ dev->config->set_status(dev, dev->config->get_status(dev) |
status);
+}
+
+static int virtio_dev_probe(struct device *_d)
+{
+ int err;
+ struct virtio_device *dev = container_of(_d,struct virtio_device,dev);
+ struct virtio_driver *drv = container_of(dev->dev.driver,
+ struct virtio_driver, driver);
+
+ add_status(dev, VIRTIO_CONFIG_S_DRIVER);
+ err = drv->probe(dev);
+ if (err)
+ add_status(dev, VIRTIO_CONFIG_S_FAILED);
+ else
+ add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK);
+ return err;
+}
+
+int register_virtio_driver(struct virtio_driver *driver)
+{
+ driver->driver.bus = &virtio_bus;
+ driver->driver.probe = virtio_dev_probe;
+ return driver_register(&driver->driver);
+}
+EXPORT_SYMBOL_GPL(register_virtio_driver);
+
+void unregister_virtio_driver(struct virtio_driver *driver)
+{
+ driver_unregister(&driver->driver);
+}
+EXPORT_SYMBOL_GPL(unregister_virtio_driver);
+
+int register_virtio_device(struct virtio_device *dev)
+{
+ int err;
+
+ dev->dev.bus = &virtio_bus;
+ sprintf(dev->dev.bus_id, "%u", dev->index);
+
+ /* Acknowledge that we've seen the device. */
+ add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE);
+
+ /* device_register() causes the bus infrastructure to look for a
+ * matching driver. */
+ err = device_register(&dev->dev);
+ if (err)
+ add_status(dev, VIRTIO_CONFIG_S_FAILED);
+ return err;
+}
+EXPORT_SYMBOL_GPL(register_virtio_device);
+
+void unregister_virtio_device(struct virtio_device *dev)
+{
+ device_unregister(&dev->dev);
+}
+EXPORT_SYMBOL_GPL(unregister_virtio_device);
+
+int __virtio_config_val(struct virtio_device *vdev,
+ u8 type, void *val, size_t size)
+{
+ void *token;
+ unsigned int len;
+
+ token = vdev->config->find(vdev, type, &len);
+ if (!token)
+ return -ENOENT;
+
+ if (len != size)
+ return -EIO;
+
+ vdev->config->get(vdev, token, val, size);
+ return 0;
+}
+EXPORT_SYMBOL_GPL(__virtio_config_val);
+
+int virtio_use_bit(struct virtio_device *vdev,
+ void *token, unsigned int len, unsigned int bitnum)
+{
+ unsigned long bits[16];
+
+ /* This makes it convenient to pass-through find() results. */
+ if (!token)
+ return 0;
+
+ /* bit not in range of this bitfield? */
+ if (bitnum * 8 >= len / 2)
+ return 0;
+
+ /* Giant feature bitfields are silly. */
+ BUG_ON(len > sizeof(bits));
+ vdev->config->get(vdev, token, bits, len);
+
+ if (!test_bit(bitnum, bits))
+ return 0;
+
+ /* Set acknowledge bit, and write it back. */
+ set_bit(bitnum + len * 8 / 2, bits);
+ vdev->config->set(vdev, token, bits, len);
+ return 1;
+}
+EXPORT_SYMBOL_GPL(virtio_use_bit);
+
+static int virtio_init(void)
+{
+ if (bus_register(&virtio_bus) != 0)
+ panic("virtio bus registration failed");
+ return 0;
+}
+core_initcall(virtio_init);
==================================================================---
a/include/linux/mod_devicetable.h
+++ b/include/linux/mod_devicetable.h
@@ -340,4 +340,10 @@ struct parisc_device_id {
#define PA_HVERSION_ANY_ID 0xffff
#define PA_SVERSION_ANY_ID 0xffffffff
+struct virtio_device_id {
+ __u32 device;
+ __u32 vendor;
+};
+#define VIRTIO_DEV_ANY_ID 0xffffffff
+
#endif /* LINUX_MOD_DEVICETABLE_H */
==================================================================--- /dev/null
+++ b/include/linux/virtio.h
@@ -0,0 +1,110 @@
+#ifndef _LINUX_VIRTIO_H
+#define _LINUX_VIRTIO_H
+/* Everything a virtio driver needs to work with any particular virtio
+ * implementation. */
+#include <linux/types.h>
+#include <linux/scatterlist.h>
+#include <linux/spinlock.h>
+#include <linux/device.h>
+#include <linux/mod_devicetable.h>
+
+/**
+ * virtqueue - a queue to register buffers for sending or receiving.
+ * @callback: the function to call when buffers are consumed (can be NULL).
+ * If this returns false, callbacks are suppressed until vq_ops->restart
+ * is called.
+ * @vdev: the virtio device this queue was created for.
+ * @vq_ops: the operations for this virtqueue (see below).
+ * @priv: a pointer for the virtqueue implementation to use.
+ */
+struct virtqueue
+{
+ bool (*callback)(struct virtqueue *vq);
+ struct virtio_device *vdev;
+ struct virtqueue_ops *vq_ops;
+ void *priv;
+};
+
+/**
+ * virtqueue_ops - operations for virtqueue abstraction layer
+ * @add_buf: expose buffer to other end
+ * vq: the struct virtqueue we're talking about.
+ * sg: the description of the buffer(s).
+ * out_num: the number of sg readable by other side
+ * in_num: the number of sg which are writable (after readable ones)
+ * data: the token identifying the buffer.
+ * Returns 0 or an error.
+ * @kick: update after add_buf
+ * vq: the struct virtqueue
+ * After one or more add_buf calls, invoke this to kick the other side.
+ * @get_buf: get the next used buffer
+ * vq: the struct virtqueue we're talking about.
+ * len: the length written into the buffer
+ * Returns NULL or the "data" token handed to add_buf.
+ * @restart: restart callbacks after callback returned false.
+ * vq: the struct virtqueue we're talking about.
+ * This returns "false" (and doesn't re-enable) if there are
pending
+ * buffers in the queue, to avoid a race.
+ * @shutdown: "unadd" all buffers.
+ * vq: the struct virtqueue we're talking about.
+ * Remove everything from the queue.
+ *
+ * Locking rules are straightforward: the driver is responsible for
+ * locking. No two operations may be invoked simultaneously.
+ *
+ * All operations can be called in any context.
+ */
+struct virtqueue_ops {
+ int (*add_buf)(struct virtqueue *vq,
+ struct scatterlist sg[],
+ unsigned int out_num,
+ unsigned int in_num,
+ void *data);
+
+ void (*kick)(struct virtqueue *vq);
+
+ void *(*get_buf)(struct virtqueue *vq, unsigned int *len);
+
+ bool (*restart)(struct virtqueue *vq);
+
+ void (*shutdown)(struct virtqueue *vq);
+};
+
+/**
+ * virtio_device - representation of a device using virtio
+ * @index: unique position on the virtio bus
+ * @dev: underlying device.
+ * @id: the device type identification (used to match it with a driver).
+ * @config: the configuration ops for this device.
+ * @priv: private pointer for the driver's use.
+ */
+struct virtio_device
+{
+ int index;
+ struct device dev;
+ struct virtio_device_id id;
+ struct virtio_config_ops *config;
+ void *priv;
+};
+
+int register_virtio_device(struct virtio_device *dev);
+void unregister_virtio_device(struct virtio_device *dev);
+
+/**
+ * virtio_driver - operations for a virtio I/O driver
+ * @driver: underlying device driver (populate name and owner).
+ * @id_table: the ids serviced by this driver.
+ * @probe: the function to call when a device is found. Returns a token for
+ * remove, or PTR_ERR().
+ * @remove: the function when a device is removed.
+ */
+struct virtio_driver {
+ struct device_driver driver;
+ const struct virtio_device_id *id_table;
+ int (*probe)(struct virtio_device *dev);
+ void (*remove)(struct virtio_device *dev);
+};
+
+int register_virtio_driver(struct virtio_driver *drv);
+void unregister_virtio_driver(struct virtio_driver *drv);
+#endif /* _LINUX_VIRTIO_H */
==================================================================--- /dev/null
+++ b/include/linux/virtio_config.h
@@ -0,0 +1,111 @@
+#ifndef _LINUX_VIRTIO_CONFIG_H
+#define _LINUX_VIRTIO_CONFIG_H
+/* Virtio devices use a standardized configuration space to define their
+ * features and pass configuration information, but each implementation can
+ * store and access that space differently. */
+#include <linux/types.h>
+
+/* Status byte for guest to report progress, and synchronize config. */
+/* We have seen device and processed generic fields (VIRTIO_CONFIG_F_VIRTIO) */
+#define VIRTIO_CONFIG_S_ACKNOWLEDGE 1
+/* We have found a driver for the device. */
+#define VIRTIO_CONFIG_S_DRIVER 2
+/* Driver has used its parts of the config, and is happy */
+#define VIRTIO_CONFIG_S_DRIVER_OK 4
+/* We've given up on this device. */
+#define VIRTIO_CONFIG_S_FAILED 0x80
+
+/* Feature byte (actually 7 bits availabe): */
+/* Requirements/features of the virtio implementation. */
+#define VIRTIO_CONFIG_F_VIRTIO 1
+/* Requirements/features of the virtqueue (may have more than one). */
+#define VIRTIO_CONFIG_F_VIRTQUEUE 2
+
+#ifdef __KERNEL__
+struct virtio_device;
+
+/**
+ * virtio_config_ops - operations for configuring a virtio device
+ * @find: search for the next configuration field of the given type.
+ * vdev: the virtio_device
+ * type: the feature type
+ * len: the (returned) length of the field if found.
+ * Returns a token if found, or NULL. Never returnes the same field twice
+ * (ie. it's used up).
+ * @get: read the value of a configuration field after find().
+ * vdev: the virtio_device
+ * token: the token returned from find().
+ * buf: the buffer to write the field value into.
+ * len: the length of the buffer (given by find()).
+ * Note that contents are conventionally little-endian.
+ * @set: write the value of a configuration field after find().
+ * vdev: the virtio_device
+ * token: the token returned from find().
+ * buf: the buffer to read the field value from.
+ * len: the length of the buffer (given by find()).
+ * Note that contents are conventionally little-endian.
+ * @get_status: read the status byte
+ * vdev: the virtio_device
+ * Returns the status byte
+ * @set_status: write the status byte
+ * vdev: the virtio_device
+ * status: the new status byte
+ * @find_vq: find the first VIRTIO_CONFIG_F_VIRTQUEUE and create a virtqueue.
+ * vdev: the virtio_device
+ * callback: the virqtueue callback
+ * Returns the new virtqueue or ERR_PTR().
+ * @del_vq: free a virtqueue found by find_vq().
+ */
+struct virtio_config_ops
+{
+ void *(*find)(struct virtio_device *vdev, u8 type, unsigned *len);
+ void (*get)(struct virtio_device *vdev, void *token,
+ void *buf, unsigned len);
+ void (*set)(struct virtio_device *vdev, void *token,
+ const void *buf, unsigned len);
+ u8 (*get_status)(struct virtio_device *vdev);
+ void (*set_status)(struct virtio_device *vdev, u8 status);
+ struct virtqueue *(*find_vq)(struct virtio_device *vdev,
+ bool (*callback)(struct virtqueue *));
+ void (*del_vq)(struct virtqueue *vq);
+};
+
+/**
+ * virtio_config_val - get a single virtio config and mark it used.
+ * @config: the virtio config space
+ * @type: the type to search for.
+ * @val: a pointer to the value to fill in.
+ *
+ * Once used, the config type is marked with VIRTIO_CONFIG_F_USED so it
can't
+ * be found again. This version does endian conversion. */
+#define virtio_config_val(vdev, type, v) ({ \
+ int _err = __virtio_config_val((vdev),(type),(v),sizeof(*(v))); \
+ \
+ BUILD_BUG_ON(sizeof(*(v)) != 1 && sizeof(*(v)) != 2 \
+ && sizeof(*(v)) != 4 && sizeof(*(v)) != 8); \
+ if (!_err) { \
+ switch (sizeof(*(v))) { \
+ case 2: le16_to_cpus(v); break; \
+ case 4: le32_to_cpus(v); break; \
+ case 8: le64_to_cpus(v); break; \
+ } \
+ } \
+ _err; \
+})
+
+int __virtio_config_val(struct virtio_device *dev,
+ u8 type, void *val, size_t size);
+
+/**
+ * virtio_use_bit - helper to use a feature bit in a bitfield value.
+ * @dev: the virtio device
+ * @token: the token as returned from vdev->config->find().
+ * @len: the length of the field.
+ * @bitnum: the bit to test.
+ *
+ * If handed a NULL token, it returns false, otherwise returns bit status.
+ * If it's one, it sets the mirroring acknowledgement bit. */
+int virtio_use_bit(struct virtio_device *vdev,
+ void *token, unsigned int len, unsigned int bitnum);
+#endif /* __KERNEL__ */
+#endif /* _LINUX_VIRTIO_CONFIG_H */
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
rusty@rustcorp.com.au
2007-Sep-26 00:45 UTC
[patch 38/43] lguest: This gets rid of the lguest bus, drivers and DMA mechanism, to make way for a generic virtio mechanism.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
arch/i386/lguest/boot.c | 41 --
drivers/block/Makefile | 1
drivers/block/lguest_blk.c | 423 -----------------------------
drivers/char/Makefile | 1
drivers/char/hvc_lguest.c | 177 ------------
drivers/lguest/Kconfig | 10
drivers/lguest/Makefile | 3
drivers/lguest/i386_core.c | 1
drivers/lguest/lguest_bus.c | 220 ---------------
drivers/net/Makefile | 1
drivers/net/lguest_net.c | 552 ---------------------------------------
include/linux/lguest_bus.h | 51 ---
include/linux/lguest_launcher.h | 26 -
13 files changed, 1507 deletions(-)
==================================================================---
a/arch/i386/lguest/boot.c
+++ b/arch/i386/lguest/boot.c
@@ -55,7 +55,6 @@
#include <linux/clockchips.h>
#include <linux/lguest.h>
#include <linux/lguest_launcher.h>
-#include <linux/lguest_bus.h>
#include <asm/paravirt.h>
#include <asm/param.h>
#include <asm/page.h>
@@ -173,46 +172,6 @@ void async_hcall(unsigned long call,
local_irq_restore(flags);
}
/*:*/
-
-/* Wrappers for the SEND_DMA and BIND_DMA hypercalls. This is mainly because
- * Jeff Garzik complained that __pa() should never appear in drivers, and this
- * helps remove most of them. But also, it wraps some ugliness. */
-void lguest_send_dma(unsigned long key, struct lguest_dma *dma)
-{
- /* The hcall might not write this if something goes wrong */
- dma->used_len = 0;
- hcall(LHCALL_SEND_DMA, key, __pa(dma), 0);
-}
-
-int lguest_bind_dma(unsigned long key, struct lguest_dma *dmas,
- unsigned int num, u8 irq)
-{
- /* This is the only hypercall which actually wants 5 arguments, and we
- * only support 4. Fortunately the interrupt number is always less
- * than 256, so we can pack it with the number of dmas in the final
- * argument. */
- if (!hcall(LHCALL_BIND_DMA, key, __pa(dmas), (num << 8) | irq))
- return -ENOMEM;
- return 0;
-}
-
-/* Unbinding is the same hypercall as binding, but with 0 num & irq. */
-void lguest_unbind_dma(unsigned long key, struct lguest_dma *dmas)
-{
- hcall(LHCALL_BIND_DMA, key, __pa(dmas), 0);
-}
-
-/* For guests, device memory can be used as normal memory, so we cast away the
- * __iomem to quieten sparse. */
-void *lguest_map(unsigned long phys_addr, unsigned long pages)
-{
- return (__force void *)ioremap(phys_addr, PAGE_SIZE*pages);
-}
-
-void lguest_unmap(void *addr)
-{
- iounmap((__force void __iomem *)addr);
-}
/*G:033
* Here are our first native-instruction replacements: four functions for
==================================================================---
a/drivers/block/Makefile
+++ b/drivers/block/Makefile
@@ -32,4 +32,3 @@ obj-$(CONFIG_BLK_DEV_UB) += ub.o
obj-$(CONFIG_BLK_DEV_UB) += ub.o
obj-$(CONFIG_XEN_BLKDEV_FRONTEND) += xen-blkfront.o
-obj-$(CONFIG_LGUEST_BLOCK) += lguest_blk.o
==================================================================---
a/drivers/block/lguest_blk.c
+++ /dev/null
@@ -1,423 +0,0 @@
-/*D:400
- * The Guest block driver
- *
- * This is a simple block driver, which appears as /dev/lgba, lgbb, lgbc etc.
- * The mechanism is simple: we place the information about the request in the
- * device page, then use SEND_DMA (containing the data for a write, or an empty
- * "ping" DMA for a read).
- :*/
-/* Copyright 2006 Rusty Russell <rusty@rustcorp.com.au> IBM Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- */
-//#define DEBUG
-#include <linux/init.h>
-#include <linux/types.h>
-#include <linux/blkdev.h>
-#include <linux/interrupt.h>
-#include <linux/lguest_bus.h>
-
-static char next_block_index = 'a';
-
-/*D:420 Here is the structure which holds all the information we need about
- * each Guest block device.
- *
- * I'm sure at this stage, you're wondering "hey, where was the
adventure I was
- * promised?" and thinking "Rusty sucks, I shall say nasty things
about him on
- * my blog". I think Real adventures have boring bits, too, and
you're in the
- * middle of one. But it gets better. Just not quite yet. */
-struct blockdev
-{
- /* The block queue infrastructure wants a spinlock: it is held while it
- * calls our block request function. We grab it in our interrupt
- * handler so the responses don't mess with new requests. */
- spinlock_t lock;
-
- /* The disk structure registered with kernel. */
- struct gendisk *disk;
-
- /* The major device number for this disk, and the interrupt. We only
- * really keep them here for completeness; we'd need them if we
- * supported device unplugging. */
- int major;
- int irq;
-
- /* The physical address of this device's memory page */
- unsigned long phys_addr;
- /* The mapped memory page for convenient acces. */
- struct lguest_block_page *lb_page;
-
- /* We only have a single request outstanding at a time: this is it. */
- struct lguest_dma dma;
- struct request *req;
-};
-
-/*D:495 We originally used end_request() throughout the driver, but it turns
- * out that end_request() is deprecated, and doesn't actually end the
request
- * (which seems like a good reason to deprecate it!). It simply ends the first
- * bio. So if we had 3 bios in a "struct request" we would do all 3,
- * end_request(), do 2, end_request(), do 1 and end_request(): twice as much
- * work as we needed to do.
- *
- * This reinforced to me that I do not understand the block layer.
- *
- * Nonetheless, Jens Axboe gave me this nice helper to end all chunks of a
- * request. This improved disk speed by 130%. */
-static void end_entire_request(struct request *req, int uptodate)
-{
- if (end_that_request_first(req, uptodate, req->hard_nr_sectors))
- BUG();
- add_disk_randomness(req->rq_disk);
- blkdev_dequeue_request(req);
- end_that_request_last(req, uptodate);
-}
-
-/* I'm told there are only two stories in the world worth telling: love and
- * hate. So there used to be a love scene here like this:
- *
- * Launcher: We could make beautiful I/O together, you and I.
- * Guest: My, that's a big disk!
- *
- * Unfortunately, it was just too raunchy for our otherwise-gentle tale. */
-
-/*D:490 This is the interrupt handler, called when a block read or write has
- * been completed for us. */
-static irqreturn_t lgb_irq(int irq, void *_bd)
-{
- /* We handed our "struct blockdev" as the argument to request_irq(),
so
- * it is passed through to us here. This tells us which device we're
- * dealing with in case we have more than one. */
- struct blockdev *bd = _bd;
- unsigned long flags;
-
- /* We weren't doing anything? Strange, but could happen if we shared
- * interrupts (we don't!). */
- if (!bd->req) {
- pr_debug("No work!\n");
- return IRQ_NONE;
- }
-
- /* Not done yet? That's equally strange. */
- if (!bd->lb_page->result) {
- pr_debug("No result!\n");
- return IRQ_NONE;
- }
-
- /* We have to grab the lock before ending the request. */
- spin_lock_irqsave(&bd->lock, flags);
- /* "result" is 1 for success, 2 for failure: end_entire_request()
wants
- * to know whether this succeeded or not. */
- end_entire_request(bd->req, bd->lb_page->result == 1);
- /* Clear out request, it's done. */
- bd->req = NULL;
- /* Reset incoming DMA for next time. */
- bd->dma.used_len = 0;
- /* Ready for more reads or writes */
- blk_start_queue(bd->disk->queue);
- spin_unlock_irqrestore(&bd->lock, flags);
-
- /* The interrupt was for us, we dealt with it. */
- return IRQ_HANDLED;
-}
-
-/*D:480 The block layer's "struct request" contains a number of
"struct bio"s,
- * each of which contains "struct bio_vec"s, each of which contains a
page, an
- * offset and a length.
- *
- * Fortunately there are iterators to help us walk through the "struct
- * request". Even more fortunately, there were plenty of places to steal
the
- * code from. We pack the "struct request" into our "struct
lguest_dma" and
- * return the total length. */
-static unsigned int req_to_dma(struct request *req, struct lguest_dma *dma)
-{
- unsigned int i = 0, idx, len = 0;
- struct bio *bio;
-
- rq_for_each_bio(bio, req) {
- struct bio_vec *bvec;
- bio_for_each_segment(bvec, bio, idx) {
- /* We told the block layer not to give us too many. */
- BUG_ON(i == LGUEST_MAX_DMA_SECTIONS);
- /* If we had a zero-length segment, it would look like
- * the end of the data referred to by the "struct
- * lguest_dma", so make sure that doesn't happen. */
- BUG_ON(!bvec->bv_len);
- /* Convert page & offset to a physical address */
- dma->addr[i] = page_to_phys(bvec->bv_page)
- + bvec->bv_offset;
- dma->len[i] = bvec->bv_len;
- len += bvec->bv_len;
- i++;
- }
- }
- /* If the array isn't full, we mark the end with a 0 length */
- if (i < LGUEST_MAX_DMA_SECTIONS)
- dma->len[i] = 0;
- return len;
-}
-
-/* This creates an empty DMA, useful for prodding the Host without sending data
- * (ie. when we want to do a read) */
-static void empty_dma(struct lguest_dma *dma)
-{
- dma->len[0] = 0;
-}
-
-/*D:470 Setting up a request is fairly easy: */
-static void setup_req(struct blockdev *bd,
- int type, struct request *req, struct lguest_dma *dma)
-{
- /* The type is 1 (write) or 0 (read). */
- bd->lb_page->type = type;
- /* The sector on disk where the read or write starts. */
- bd->lb_page->sector = req->sector;
- /* The result is initialized to 0 (unfinished). */
- bd->lb_page->result = 0;
- /* The current request (so we can end it in the interrupt handler). */
- bd->req = req;
- /* The number of bytes: returned as a side-effect of req_to_dma(),
- * which packs the block layer's "struct request" into our
"struct
- * lguest_dma" */
- bd->lb_page->bytes = req_to_dma(req, dma);
-}
-
-/*D:450 Write is pretty straightforward: we pack the request into a
"struct
- * lguest_dma", then use SEND_DMA to send the request. */
-static void do_write(struct blockdev *bd, struct request *req)
-{
- struct lguest_dma send;
-
- pr_debug("lgb: WRITE sector %li\n", (long)req->sector);
- setup_req(bd, 1, req, &send);
-
- lguest_send_dma(bd->phys_addr, &send);
-}
-
-/* Read is similar to write, except we pack the request into our receive
- * "struct lguest_dma" and send through an empty DMA just to tell the
Host that
- * there's a request pending. */
-static void do_read(struct blockdev *bd, struct request *req)
-{
- struct lguest_dma ping;
-
- pr_debug("lgb: READ sector %li\n", (long)req->sector);
- setup_req(bd, 0, req, &bd->dma);
-
- empty_dma(&ping);
- lguest_send_dma(bd->phys_addr, &ping);
-}
-
-/*D:440 This where requests come in: we get handed the request queue and are
- * expected to pull a "struct request" off it until we've
finished them or
- * we're waiting for a reply: */
-static void do_lgb_request(struct request_queue *q)
-{
- struct blockdev *bd;
- struct request *req;
-
-again:
- /* This sometimes returns NULL even on the very first time around. I
- * wonder if it's something to do with letting elves handle the request
- * queue... */
- req = elv_next_request(q);
- if (!req)
- return;
-
- /* We attached the struct blockdev to the disk: get it back */
- bd = req->rq_disk->private_data;
- /* Sometimes we get repeated requests after blk_stop_queue(), but we
- * can only handle one at a time. */
- if (bd->req)
- return;
-
- /* We only do reads and writes: no tricky business! */
- if (!blk_fs_request(req)) {
- pr_debug("Got non-command 0x%08x\n", req->cmd_type);
- req->errors++;
- end_entire_request(req, 0);
- goto again;
- }
-
- if (rq_data_dir(req) == WRITE)
- do_write(bd, req);
- else
- do_read(bd, req);
-
- /* We've put out the request, so stop any more coming in until we get
- * an interrupt, which takes us to lgb_irq() to re-enable the queue. */
- blk_stop_queue(q);
-}
-
-/*D:430 This is the "struct block_device_operations" we attach to the
disk at
- * the end of lguestblk_probe(). It doesn't seem to want much. */
-static struct block_device_operations lguestblk_fops = {
- .owner = THIS_MODULE,
-};
-
-/*D:425 Setting up a disk device seems to involve a lot of code. I'm not
sure
- * quite why. I do know that the IDE code sent two or three of the maintainers
- * insane, perhaps this is the fringe of the same disease?
- *
- * As in the console code, the probe function gets handed the generic
- * lguest_device from lguest_bus.c: */
-static int lguestblk_probe(struct lguest_device *lgdev)
-{
- struct blockdev *bd;
- int err;
- int irqflags = IRQF_SHARED;
-
- /* First we allocate our own "struct blockdev" and initialize the
easy
- * fields. */
- bd = kmalloc(sizeof(*bd), GFP_KERNEL);
- if (!bd)
- return -ENOMEM;
-
- spin_lock_init(&bd->lock);
- bd->irq = lgdev_irq(lgdev);
- bd->req = NULL;
- bd->dma.used_len = 0;
- bd->dma.len[0] = 0;
- /* The descriptor in the lguest_devices array provided by the Host
- * gives the Guest the physical page number of the device's page. */
- bd->phys_addr = (lguest_devices[lgdev->index].pfn << PAGE_SHIFT);
-
- /* We use lguest_map() to get a pointer to the device page */
- bd->lb_page = lguest_map(bd->phys_addr, 1);
- if (!bd->lb_page) {
- err = -ENOMEM;
- goto out_free_bd;
- }
-
- /* We need a major device number: 0 means "assign one dynamically".
*/
- bd->major = register_blkdev(0, "lguestblk");
- if (bd->major < 0) {
- err = bd->major;
- goto out_unmap;
- }
-
- /* This allocates a "struct gendisk" where we pack all the
information
- * about the disk which the rest of Linux sees. The argument is the
- * number of minor devices desired: we need one minor for the main
- * disk, and one for each partition. Of course, we can't possibly know
- * how many partitions are on the disk (add_disk does that).
- */
- bd->disk = alloc_disk(16);
- if (!bd->disk) {
- err = -ENOMEM;
- goto out_unregister_blkdev;
- }
-
- /* Every disk needs a queue for requests to come in: we set up the
- * queue with a callback function (the core of our driver) and the lock
- * to use. */
- bd->disk->queue = blk_init_queue(do_lgb_request, &bd->lock);
- if (!bd->disk->queue) {
- err = -ENOMEM;
- goto out_put_disk;
- }
-
- /* We can only handle a certain number of pointers in our SEND_DMA
- * call, so we set that with blk_queue_max_hw_segments(). This is not
- * to be confused with blk_queue_max_phys_segments() of course! I
- * know, who could possibly confuse the two?
- *
- * Well, it's simple to tell them apart: this one seems to work and the
- * other one didn't. */
- blk_queue_max_hw_segments(bd->disk->queue, LGUEST_MAX_DMA_SECTIONS);
-
- /* Due to technical limitations of our Host (and simple coding) we
- * can't have a single buffer which crosses a page boundary. Tell it
- * here. This means that our maximum request size is 16
- * (LGUEST_MAX_DMA_SECTIONS) pages. */
- blk_queue_segment_boundary(bd->disk->queue, PAGE_SIZE-1);
-
- /* We name our disk: this becomes the device name when udev does its
- * magic thing and creates the device node, such as /dev/lgba.
- * next_block_index is a global which starts at 'a'. Unfortunately
- * this simple increment logic means that the 27th disk will be called
- * "/dev/lgb{". In that case, I recommend having at least 29 disks,
so
- * your /dev directory will be balanced. */
- sprintf(bd->disk->disk_name, "lgb%c", next_block_index++);
-
- /* We look to the device descriptor again to see if this device's
- * interrupts are expected to be random. If they are, we tell the irq
- * subsystem. At the moment this bit is always set. */
- if (lguest_devices[lgdev->index].features & LGUEST_DEVICE_F_RANDOMNESS)
- irqflags |= IRQF_SAMPLE_RANDOM;
-
- /* Now we have the name and irqflags, we can request the interrupt; we
- * give it the "struct blockdev" we have set up to pass to lgb_irq()
- * when there is an interrupt. */
- err = request_irq(bd->irq, lgb_irq, irqflags, bd->disk->disk_name,
bd);
- if (err)
- goto out_cleanup_queue;
-
- /* We bind our one-entry DMA pool to the key for this block device so
- * the Host can reply to our requests. The key is equal to the
- * physical address of the device's page, which is conveniently
- * unique. */
- err = lguest_bind_dma(bd->phys_addr, &bd->dma, 1, bd->irq);
- if (err)
- goto out_free_irq;
-
- /* We finish our disk initialization and add the disk to the system. */
- bd->disk->major = bd->major;
- bd->disk->first_minor = 0;
- bd->disk->private_data = bd;
- bd->disk->fops = &lguestblk_fops;
- /* This is initialized to the disk size by the Launcher. */
- set_capacity(bd->disk, bd->lb_page->num_sectors);
- add_disk(bd->disk);
-
- printk(KERN_INFO "%s: device %i at major %d\n",
- bd->disk->disk_name, lgdev->index, bd->major);
-
- /* We don't need to keep the "struct blockdev" around, but if we
ever
- * implemented device removal, we'd need this. */
- lgdev->private = bd;
- return 0;
-
-out_free_irq:
- free_irq(bd->irq, bd);
-out_cleanup_queue:
- blk_cleanup_queue(bd->disk->queue);
-out_put_disk:
- put_disk(bd->disk);
-out_unregister_blkdev:
- unregister_blkdev(bd->major, "lguestblk");
-out_unmap:
- lguest_unmap(bd->lb_page);
-out_free_bd:
- kfree(bd);
- return err;
-}
-
-/*D:410 The boilerplate code for registering the lguest block driver is just
- * like the console: */
-static struct lguest_driver lguestblk_drv = {
- .name = "lguestblk",
- .owner = THIS_MODULE,
- .device_type = LGUEST_DEVICE_T_BLOCK,
- .probe = lguestblk_probe,
-};
-
-static __init int lguestblk_init(void)
-{
- return register_lguest_driver(&lguestblk_drv);
-}
-module_init(lguestblk_init);
-
-MODULE_DESCRIPTION("Lguest block driver");
-MODULE_LICENSE("GPL");
==================================================================---
a/drivers/char/Makefile
+++ b/drivers/char/Makefile
@@ -42,7 +42,6 @@ obj-$(CONFIG_N_HDLC) += n_hdlc.o
obj-$(CONFIG_N_HDLC) += n_hdlc.o
obj-$(CONFIG_AMIGA_BUILTIN_SERIAL) += amiserial.o
obj-$(CONFIG_SX) += sx.o generic_serial.o
-obj-$(CONFIG_LGUEST_GUEST) += hvc_lguest.o
obj-$(CONFIG_RIO) += rio/ generic_serial.o
obj-$(CONFIG_HVC_CONSOLE) += hvc_vio.o hvsi.o
obj-$(CONFIG_HVC_ISERIES) += hvc_iseries.o
==================================================================---
a/drivers/char/hvc_lguest.c
+++ /dev/null
@@ -1,177 +0,0 @@
-/*D:300
- * The Guest console driver
- *
- * This is a trivial console driver: we use lguest's DMA mechanism to send
- * bytes out, and register a DMA buffer to receive bytes in. It is assumed to
- * be present and available from the very beginning of boot.
- *
- * Writing console drivers is one of the few remaining Dark Arts in Linux.
- * Fortunately for us, the path of virtual consoles has been well-trodden by
- * the PowerPC folks, who wrote "hvc_console.c" to generically
support any
- * virtual console. We use that infrastructure which only requires us to write
- * the basic put_chars and get_chars functions and call the right register
- * functions.
- :*/
-
-/*M:002 The console can be flooded: while the Guest is processing input the
- * Host can send more. Buffering in the Host could alleviate this, but it is a
- * difficult problem in general. :*/
-/* Copyright (C) 2006 Rusty Russell, IBM Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- */
-#include <linux/err.h>
-#include <linux/init.h>
-#include <linux/lguest_bus.h>
-#include <asm/paravirt.h>
-#include "hvc_console.h"
-
-/*D:340 This is our single console input buffer, with associated "struct
- * lguest_dma" referring to it. Note the 0-terminated length array, and
the
- * use of physical address for the buffer itself. */
-static char inbuf[256];
-static struct lguest_dma cons_input = { .used_len = 0,
- .addr[0] = __pa(inbuf),
- .len[0] = sizeof(inbuf),
- .len[1] = 0 };
-
-/*D:310 The put_chars() callback is pretty straightforward.
- *
- * First we put the pointer and length in a "struct lguest_dma": we
only have
- * one pointer, so we set the second length to 0. Then we use SEND_DMA to send
- * the data to (Host) buffers attached to the console key. Usually a
device's
- * key is a physical address within the device's memory, but because the
- * console device doesn't have any associated physical memory, we use the
- * LGUEST_CONSOLE_DMA_KEY constant (aka 0). */
-static int put_chars(u32 vtermno, const char *buf, int count)
-{
- struct lguest_dma dma;
-
- /* FIXME: DMA buffers in a "struct lguest_dma" are not allowed
- * to go over page boundaries. This never seems to happen,
- * but if it did we'd need to fix this code. */
- dma.len[0] = count;
- dma.len[1] = 0;
- dma.addr[0] = __pa(buf);
-
- lguest_send_dma(LGUEST_CONSOLE_DMA_KEY, &dma);
- /* We're expected to return the amount of data we wrote: all of it. */
- return count;
-}
-
-/*D:350 get_chars() is the callback from the hvc_console infrastructure when
- * an interrupt is received.
- *
- * Firstly we see if our buffer has been filled: if not, we return. The rest
- * of the code deals with the fact that the hvc_console() infrastructure only
- * asks us for 16 bytes at a time. We keep a "cons_offset" variable
for
- * partially-read buffers. */
-static int get_chars(u32 vtermno, char *buf, int count)
-{
- static int cons_offset;
-
- /* Nothing left to see here... */
- if (!cons_input.used_len)
- return 0;
-
- /* You want more than we have to give? Well, try wanting less! */
- if (cons_input.used_len - cons_offset < count)
- count = cons_input.used_len - cons_offset;
-
- /* Copy across to their buffer and increment offset. */
- memcpy(buf, inbuf + cons_offset, count);
- cons_offset += count;
-
- /* Finished? Zero offset, and reset cons_input so Host will use it
- * again. */
- if (cons_offset == cons_input.used_len) {
- cons_offset = 0;
- cons_input.used_len = 0;
- }
- return count;
-}
-/*:*/
-
-static struct hv_ops lguest_cons = {
- .get_chars = get_chars,
- .put_chars = put_chars,
-};
-
-/*D:320 Console drivers are initialized very early so boot messages can go
- * out. At this stage, the console is output-only. Our driver checks
we're a
- * Guest, and if so hands hvc_instantiate() the console number (0), priority
- * (0), and the struct hv_ops containing the put_chars() function. */
-static int __init cons_init(void)
-{
- if (strcmp(paravirt_ops.name, "lguest") != 0)
- return 0;
-
- return hvc_instantiate(0, 0, &lguest_cons);
-}
-console_initcall(cons_init);
-
-/*D:370 To set up and manage our virtual console, we call hvc_alloc() and
- * stash the result in the private pointer of the "struct
lguest_device".
- * Since we never remove the console device we never need this pointer again,
- * but using ->private is considered good form, and you never know who's
going
- * to copy your driver.
- *
- * Once the console is set up, we bind our input buffer ready for input. */
-static int lguestcons_probe(struct lguest_device *lgdev)
-{
- int err;
-
- /* The first argument of hvc_alloc() is the virtual console number, so
- * we use zero. The second argument is the interrupt number.
- *
- * The third argument is a "struct hv_ops" containing the
put_chars()
- * and get_chars() pointers. The final argument is the output buffer
- * size: we use 256 and expect the Host to have room for us to send
- * that much. */
- lgdev->private = hvc_alloc(0, lgdev_irq(lgdev), &lguest_cons, 256);
- if (IS_ERR(lgdev->private))
- return PTR_ERR(lgdev->private);
-
- /* We bind a single DMA buffer at key LGUEST_CONSOLE_DMA_KEY.
- * "cons_input" is that statically-initialized global DMA buffer we
saw
- * above, and we also give the interrupt we want. */
- err = lguest_bind_dma(LGUEST_CONSOLE_DMA_KEY, &cons_input, 1,
- lgdev_irq(lgdev));
- if (err)
- printk("lguest console: failed to bind buffer.\n");
- return err;
-}
-/* Note the use of lgdev_irq() for the interrupt number. We tell hvc_alloc()
- * to expect input when this interrupt is triggered, and then tell
- * lguest_bind_dma() that is the interrupt to send us when input comes in. */
-
-/*D:360 From now on the console driver follows standard Guest driver form:
- * register_lguest_driver() registers the device type and probe function, and
- * the probe function sets up the device.
- *
- * The standard "struct lguest_driver": */
-static struct lguest_driver lguestcons_drv = {
- .name = "lguestcons",
- .owner = THIS_MODULE,
- .device_type = LGUEST_DEVICE_T_CONSOLE,
- .probe = lguestcons_probe,
-};
-
-/* The standard init function */
-static int __init hvc_lguest_init(void)
-{
- return register_lguest_driver(&lguestcons_drv);
-}
-module_init(hvc_lguest_init);
==================================================================---
a/drivers/lguest/Kconfig
+++ b/drivers/lguest/Kconfig
@@ -17,13 +17,3 @@ config LGUEST_GUEST
The guest needs code built-in, even if the host has lguest
support as a module. The drivers are tiny, so we build them
in too.
-
-config LGUEST_NET
- tristate
- default y
- depends on LGUEST_GUEST && NET
-
-config LGUEST_BLOCK
- tristate
- default y
- depends on LGUEST_GUEST && BLOCK
==================================================================---
a/drivers/lguest/Makefile
+++ b/drivers/lguest/Makefile
@@ -1,6 +1,3 @@
-# Guest requires the bus driver.
-obj-$(CONFIG_LGUEST_GUEST) += lguest_bus.o
-
# Host requires the other files, which can be a module.
obj-$(CONFIG_LGUEST) += lg.o
lg-y = core.o hypercalls.o page_tables.o interrupts_and_traps.o \
==================================================================---
a/drivers/lguest/i386_core.c
+++ b/drivers/lguest/i386_core.c
@@ -29,7 +29,6 @@
#include <linux/cpu.h>
#include <linux/lguest.h>
#include <linux/lguest_launcher.h>
-#include <linux/lguest_bus.h>
#include <asm/paravirt.h>
#include <asm/param.h>
#include <asm/page.h>
==================================================================---
a/drivers/lguest/lguest_bus.c
+++ /dev/null
@@ -1,220 +0,0 @@
-/*P:050 Lguest guests use a very simple bus for devices. It's a simple
array
- * of device descriptors contained just above the top of normal memory. The
- * lguest bus is 80% tedious boilerplate code. :*/
-#include <linux/init.h>
-#include <linux/bootmem.h>
-#include <linux/lguest_bus.h>
-#include <asm/io.h>
-#include <asm/paravirt.h>
-
-struct lguest_device_desc *lguest_devices;
-
-static ssize_t type_show(struct device *_dev,
- struct device_attribute *attr, char *buf)
-{
- struct lguest_device *dev = container_of(_dev,struct lguest_device,dev);
- return sprintf(buf, "%hu", lguest_devices[dev->index].type);
-}
-static ssize_t features_show(struct device *_dev,
- struct device_attribute *attr, char *buf)
-{
- struct lguest_device *dev = container_of(_dev,struct lguest_device,dev);
- return sprintf(buf, "%hx", lguest_devices[dev->index].features);
-}
-static ssize_t pfn_show(struct device *_dev,
- struct device_attribute *attr, char *buf)
-{
- struct lguest_device *dev = container_of(_dev,struct lguest_device,dev);
- return sprintf(buf, "%u", lguest_devices[dev->index].pfn);
-}
-static ssize_t status_show(struct device *_dev,
- struct device_attribute *attr, char *buf)
-{
- struct lguest_device *dev = container_of(_dev,struct lguest_device,dev);
- return sprintf(buf, "%hx", lguest_devices[dev->index].status);
-}
-static ssize_t status_store(struct device *_dev, struct device_attribute *attr,
- const char *buf, size_t count)
-{
- struct lguest_device *dev = container_of(_dev,struct lguest_device,dev);
- if (sscanf(buf, "%hi", &lguest_devices[dev->index].status) !=
1)
- return -EINVAL;
- return count;
-}
-static struct device_attribute lguest_dev_attrs[] = {
- __ATTR_RO(type),
- __ATTR_RO(features),
- __ATTR_RO(pfn),
- __ATTR(status, 0644, status_show, status_store),
- __ATTR_NULL
-};
-
-/*D:130 The generic bus infrastructure requires a function which says whether a
- * device matches a driver. For us, it is simple: "struct
lguest_driver"
- * contains a "device_type" field which indicates what type of device
it can
- * handle, so we just cast the args and compare: */
-static int lguest_dev_match(struct device *_dev, struct device_driver *_drv)
-{
- struct lguest_device *dev = container_of(_dev,struct lguest_device,dev);
- struct lguest_driver *drv = container_of(_drv,struct lguest_driver,drv);
-
- return (drv->device_type == lguest_devices[dev->index].type);
-}
-/*:*/
-
-struct lguest_bus {
- struct bus_type bus;
- struct device dev;
-};
-
-static struct lguest_bus lguest_bus = {
- .bus = {
- .name = "lguest",
- .match = lguest_dev_match,
- .dev_attrs = lguest_dev_attrs,
- },
- .dev = {
- .parent = NULL,
- .bus_id = "lguest",
- }
-};
-
-/*D:140 This is the callback which occurs once the bus infrastructure matches
- * up a device and driver, ie. in response to add_lguest_device() calling
- * device_register(), or register_lguest_driver() calling driver_register().
- *
- * At the moment it's always the latter: the devices are added first, since
- * scan_devices() is called from a "core_initcall", and the drivers
themselves
- * called later as a normal "initcall". But it would work the other
way too.
- *
- * So now we have the happy couple, we add the status bit to indicate that we
- * found a driver. If the driver truly loves the device, it will return
- * happiness from its probe function (ok, perhaps this wasn't my greatest
- * analogy), and we set the final "driver ok" bit so the Host sees
it's all
- * green. */
-static int lguest_dev_probe(struct device *_dev)
-{
- int ret;
- struct lguest_device*dev = container_of(_dev,struct lguest_device,dev);
- struct lguest_driver*drv = container_of(dev->dev.driver,
- struct lguest_driver, drv);
-
- lguest_devices[dev->index].status |= LGUEST_DEVICE_S_DRIVER;
- ret = drv->probe(dev);
- if (ret == 0)
- lguest_devices[dev->index].status |= LGUEST_DEVICE_S_DRIVER_OK;
- return ret;
-}
-
-/* The last part of the bus infrastructure is the function lguest drivers use
- * to register themselves. Firstly, we do nothing if there's no lguest bus
- * (ie. this is not a Guest), otherwise we fill in the embedded generic
"struct
- * driver" fields and call the generic driver_register(). */
-int register_lguest_driver(struct lguest_driver *drv)
-{
- if (!lguest_devices)
- return 0;
-
- drv->drv.bus = &lguest_bus.bus;
- drv->drv.name = drv->name;
- drv->drv.owner = drv->owner;
- drv->drv.probe = lguest_dev_probe;
-
- return driver_register(&drv->drv);
-}
-
-/* At the moment we build all the drivers into the kernel because they're
so
- * simple: 8144 bytes for all three of them as I type this. And as the console
- * really needs to be built in, it's actually only 3527 bytes for the
network
- * and block drivers.
- *
- * If they get complex it will make sense for them to be modularized, so we
- * need to explicitly export the symbol.
- *
- * I don't think non-GPL modules make sense, so it's a GPL-only export.
- */
-EXPORT_SYMBOL_GPL(register_lguest_driver);
-
-/*D:120 This is the core of the lguest bus: actually adding a new device.
- * It's a separate function because it's neater that way, and because
an
- * earlier version of the code supported hotplug and unplug. They were removed
- * early on because they were never used.
- *
- * As Andrew Tridgell says, "Untested code is buggy code".
- *
- * It's worth reading this carefully: we start with an index into the array
of
- * "struct lguest_device_desc"s indicating the device which is new:
*/
-static void add_lguest_device(unsigned int index)
-{
- struct lguest_device *new;
-
- /* Each "struct lguest_device_desc" has a "status" field,
which the
- * Guest updates as the device is probed. In the worst case, the Host
- * can look at these bits to tell what part of device setup failed,
- * even if the console isn't available. */
- lguest_devices[index].status |= LGUEST_DEVICE_S_ACKNOWLEDGE;
- new = kmalloc(sizeof(struct lguest_device), GFP_KERNEL);
- if (!new) {
- printk(KERN_EMERG "Cannot allocate lguest device %u\n", index);
- lguest_devices[index].status |= LGUEST_DEVICE_S_FAILED;
- return;
- }
-
- /* The "struct lguest_device" setup is pretty straight-forward
example
- * code. */
- new->index = index;
- new->private = NULL;
- memset(&new->dev, 0, sizeof(new->dev));
- new->dev.parent = &lguest_bus.dev;
- new->dev.bus = &lguest_bus.bus;
- sprintf(new->dev.bus_id, "%u", index);
-
- /* device_register() causes the bus infrastructure to look for a
- * matching driver. */
- if (device_register(&new->dev) != 0) {
- printk(KERN_EMERG "Cannot register lguest device %u\n", index);
- lguest_devices[index].status |= LGUEST_DEVICE_S_FAILED;
- kfree(new);
- }
-}
-
-/*D:110 scan_devices() simply iterates through the device array. The type 0
- * is reserved to mean "no device", and anything else means we have
found a
- * device: add it. */
-static void scan_devices(void)
-{
- unsigned int i;
-
- for (i = 0; i < LGUEST_MAX_DEVICES; i++)
- if (lguest_devices[i].type)
- add_lguest_device(i);
-}
-
-/*D:100 Fairly early in boot, lguest_bus_init() is called to set up the lguest
- * bus. We check that we are a Guest by checking paravirt_ops.name: there are
- * other ways of checking, but this seems most obvious to me.
- *
- * So we can access the array of "struct lguest_device_desc"s easily,
we map
- * that memory and store the pointer in the global "lguest_devices".
Then we
- * register the bus with the core. Doing two registrations seems clunky to me,
- * but it seems to be the correct sysfs incantation.
- *
- * Finally we call scan_devices() which adds all the devices found in the
- * "struct lguest_device_desc" array. */
-static int __init lguest_bus_init(void)
-{
- if (strcmp(paravirt_ops.name, "lguest") != 0)
- return 0;
-
- /* Devices are in a single page above top of "normal" mem */
- lguest_devices = lguest_map(max_pfn<<PAGE_SHIFT, 1);
-
- if (bus_register(&lguest_bus.bus) != 0
- || device_register(&lguest_bus.dev) != 0)
- panic("lguest bus registration failed");
-
- scan_devices();
- return 0;
-}
-/* Do this after core stuff, before devices. */
-postcore_initcall(lguest_bus_init);
==================================================================---
a/drivers/net/Makefile
+++ b/drivers/net/Makefile
@@ -177,7 +177,6 @@ obj-$(CONFIG_HPLANCE) += hplance.o 7990.
obj-$(CONFIG_HPLANCE) += hplance.o 7990.o
obj-$(CONFIG_MVME147_NET) += mvme147.o 7990.o
obj-$(CONFIG_EQUALIZER) += eql.o
-obj-$(CONFIG_LGUEST_NET) += lguest_net.o
obj-$(CONFIG_MIPS_JAZZ_SONIC) += jazzsonic.o
obj-$(CONFIG_MIPS_AU1X00_ENET) += au1000_eth.o
obj-$(CONFIG_MIPS_SIM_NET) += mipsnet.o
==================================================================---
a/drivers/net/lguest_net.c
+++ /dev/null
@@ -1,552 +0,0 @@
-/*D:500
- * The Guest network driver.
- *
- * This is very simple a virtual network driver, and our last Guest driver.
- * The only trick is that it can talk directly to multiple other recipients
- * (ie. other Guests on the same network). It can also be used with only the
- * Host on the network.
- :*/
-
-/* Copyright 2006 Rusty Russell <rusty@rustcorp.com.au> IBM Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- */
-//#define DEBUG
-#include <linux/netdevice.h>
-#include <linux/etherdevice.h>
-#include <linux/module.h>
-#include <linux/mm_types.h>
-#include <linux/io.h>
-#include <linux/lguest_bus.h>
-
-#define SHARED_SIZE PAGE_SIZE
-#define MAX_LANS 4
-#define NUM_SKBS 8
-
-/*M:011 Network code master Jeff Garzik points out numerous shortcomings in
- * this driver if it aspires to greatness.
- *
- * Firstly, it doesn't use "NAPI": the networking's New API,
and is poorer for
- * it. As he says "NAPI means system-wide load leveling, across multiple
- * network interfaces. Lack of NAPI can mean competition at higher
loads."
- *
- * He also points out that we don't implement set_mac_address, so users
cannot
- * change the devices hardware address. When I asked why one would want to:
- * "Bonding, and situations where you /do/ want the MAC address to
"leak" out
- * of the host onto the wider net."
- *
- * Finally, he would like module unloading: "It is not unrealistic to
think of
- * [un|re|]loading the net support module in an lguest guest. And, adding
- * module support makes the programmer more responsible, because they now have
- * to learn to clean up after themselves. Any driver that cannot clean up
- * after itself is an incomplete driver in my book."
- :*/
-
-/*D:530 The "struct lguestnet_info" contains all the information we
need to
- * know about the network device. */
-struct lguestnet_info
-{
- /* The mapped device page(s) (an array of "struct lguest_net"). */
- struct lguest_net *peer;
- /* The physical address of the device page(s) */
- unsigned long peer_phys;
- /* The size of the device page(s). */
- unsigned long mapsize;
-
- /* The lguest_device I come from */
- struct lguest_device *lgdev;
-
- /* My peerid (ie. my slot in the array). */
- unsigned int me;
-
- /* Receive queue: the network packets waiting to be filled. */
- struct sk_buff *skb[NUM_SKBS];
- struct lguest_dma dma[NUM_SKBS];
-};
-/*:*/
-
-/* How many bytes left in this page. */
-static unsigned int rest_of_page(void *data)
-{
- return PAGE_SIZE - ((unsigned long)data % PAGE_SIZE);
-}
-
-/*D:570 Each peer (ie. Guest or Host) on the network binds their receive
- * buffers to a different key: we simply use the physical address of the
- * device's memory page plus the peer number. The Host insists that all
keys
- * be a multiple of 4, so we multiply the peer number by 4. */
-static unsigned long peer_key(struct lguestnet_info *info, unsigned peernum)
-{
- return info->peer_phys + 4 * peernum;
-}
-
-/* This is the routine which sets up a "struct lguest_dma" to point
to a
- * network packet, similar to req_to_dma() in lguest_blk.c. The structure of a
- * "struct sk_buff" has grown complex over the years: it consists of
a "head"
- * linear section pointed to by "skb->data", and possibly an array
of
- * "fragments" in the case of a non-linear packet.
- *
- * Our receive buffers don't use fragments at all but outgoing skbs might,
so
- * we handle it. */
-static void skb_to_dma(const struct sk_buff *skb, unsigned int headlen,
- struct lguest_dma *dma)
-{
- unsigned int i, seg;
-
- /* First, we put the linear region into the "struct lguest_dma".
Each
- * entry can't go over a page boundary, so even though all our packets
- * are 1514 bytes or less, we might need to use two entries here: */
- for (i = seg = 0; i < headlen; seg++, i += rest_of_page(skb->data+i)) {
- dma->addr[seg] = virt_to_phys(skb->data + i);
- dma->len[seg] = min((unsigned)(headlen - i),
- rest_of_page(skb->data + i));
- }
-
- /* Now we handle the fragments: at least they're guaranteed not to go
- * over a page. skb_shinfo(skb) returns a pointer to the structure
- * which tells us about the number of fragments and the fragment
- * array. */
- for (i = 0; i < skb_shinfo(skb)->nr_frags; i++, seg++) {
- const skb_frag_t *f = &skb_shinfo(skb)->frags[i];
- /* Should not happen with MTU less than 64k - 2 * PAGE_SIZE. */
- if (seg == LGUEST_MAX_DMA_SECTIONS) {
- /* We will end up sending a truncated packet should
- * this ever happen. Plus, a cool log message! */
- printk("Woah dude! Megapacket!\n");
- break;
- }
- dma->addr[seg] = page_to_phys(f->page) + f->page_offset;
- dma->len[seg] = f->size;
- }
-
- /* If after all that we didn't use the entire "struct
lguest_dma"
- * array, we terminate it with a 0 length. */
- if (seg < LGUEST_MAX_DMA_SECTIONS)
- dma->len[seg] = 0;
-}
-
-/*
- * Packet transmission.
- *
- * Our packet transmission is a little unusual. A real network card would just
- * send out the packet and leave the receivers to decide if they're
interested.
- * Instead, we look through the network device memory page and see if any of
- * the ethernet addresses match the packet destination, and if so we send it to
- * that Guest.
- *
- * This is made a little more complicated in two cases. The first case is
- * broadcast packets: for that we send the packet to all Guests on the network,
- * one at a time. The second case is "promiscuous" mode, where a
Guest wants
- * to see all the packets on the network. We need a way for the Guest to tell
- * us it wants to see all packets, so it sets the "multicast" bit on
its
- * published MAC address, which is never valid in a real ethernet address.
- */
-#define PROMISC_BIT 0x01
-
-/* This is the callback which is summoned whenever the network device's
- * multicast or promiscuous state changes. If the card is in promiscuous mode,
- * we advertise that in our ethernet address in the device's memory. We do
the
- * same if Linux wants any or all multicast traffic. */
-static void lguestnet_set_multicast(struct net_device *dev)
-{
- struct lguestnet_info *info = netdev_priv(dev);
-
- if ((dev->flags & (IFF_PROMISC|IFF_ALLMULTI)) || dev->mc_count)
- info->peer[info->me].mac[0] |= PROMISC_BIT;
- else
- info->peer[info->me].mac[0] &= ~PROMISC_BIT;
-}
-
-/* A simple test function to see if a peer wants to see all packets.*/
-static int promisc(struct lguestnet_info *info, unsigned int peer)
-{
- return info->peer[peer].mac[0] & PROMISC_BIT;
-}
-
-/* Another simple function to see if a peer's advertised ethernet address
- * matches a packet's destination ethernet address. */
-static int mac_eq(const unsigned char mac[ETH_ALEN],
- struct lguestnet_info *info, unsigned int peer)
-{
- /* Ignore multicast bit, which peer turns on to mean promisc. */
- if ((info->peer[peer].mac[0] & (~PROMISC_BIT)) != mac[0])
- return 0;
- return memcmp(mac+1, info->peer[peer].mac+1, ETH_ALEN-1) == 0;
-}
-
-/* This is the function which actually sends a packet once we've decided a
- * peer wants it: */
-static void transfer_packet(struct net_device *dev,
- struct sk_buff *skb,
- unsigned int peernum)
-{
- struct lguestnet_info *info = netdev_priv(dev);
- struct lguest_dma dma;
-
- /* We use our handy "struct lguest_dma" packing function to prepare
- * the skb for sending. */
- skb_to_dma(skb, skb_headlen(skb), &dma);
- pr_debug("xfer length %04x (%u)\n", htons(skb->len),
skb->len);
-
- /* This is the actual send call which copies the packet. */
- lguest_send_dma(peer_key(info, peernum), &dma);
-
- /* Check that the entire packet was transmitted. If not, it could mean
- * that the other Guest registered a short receive buffer, but this
- * driver should never do that. More likely, the peer is dead. */
- if (dma.used_len != skb->len) {
- dev->stats.tx_carrier_errors++;
- pr_debug("Bad xfer to peer %i: %i of %i (dma %p/%i)\n",
- peernum, dma.used_len, skb->len,
- (void *)dma.addr[0], dma.len[0]);
- } else {
- /* On success we update the stats. */
- dev->stats.tx_bytes += skb->len;
- dev->stats.tx_packets++;
- }
-}
-
-/* Another helper function to tell is if a slot in the device memory is unused.
- * Since we always set the Local Assignment bit in the ethernet address, the
- * first byte can never be 0. */
-static int unused_peer(const struct lguest_net peer[], unsigned int num)
-{
- return peer[num].mac[0] == 0;
-}
-
-/* Finally, here is the routine which handles an outgoing packet. It's
called
- * "start_xmit" for traditional reasons. */
-static int lguestnet_start_xmit(struct sk_buff *skb, struct net_device *dev)
-{
- unsigned int i;
- int broadcast;
- struct lguestnet_info *info = netdev_priv(dev);
- /* Extract the destination ethernet address from the packet. */
- const unsigned char *dest = ((struct ethhdr *)skb->data)->h_dest;
-
- pr_debug("%s: xmit %02x:%02x:%02x:%02x:%02x:%02x\n",
- dev->name, dest[0],dest[1],dest[2],dest[3],dest[4],dest[5]);
-
- /* If it's a multicast packet, we broadcast to everyone. That's not
- * very efficient, but there are very few applications which actually
- * use multicast, which is a shame really.
- *
- * As etherdevice.h points out: "By definition the broadcast address is
- * also a multicast address." So we don't have to test for broadcast
- * packets separately. */
- broadcast = is_multicast_ether_addr(dest);
-
- /* Look through all the published ethernet addresses to see if we
- * should send this packet. */
- for (i = 0; i < info->mapsize/sizeof(struct lguest_net); i++) {
- /* We don't send to ourselves (we actually can't SEND_DMA to
- * ourselves anyway), and don't send to unused slots.*/
- if (i == info->me || unused_peer(info->peer, i))
- continue;
-
- /* If it's broadcast we send it. If they want every packet we
- * send it. If the destination matches their address we send
- * it. Otherwise we go to the next peer. */
- if (!broadcast && !promisc(info, i) && !mac_eq(dest, info,
i))
- continue;
-
- pr_debug("lguestnet %s: sending from %i to %i\n",
- dev->name, info->me, i);
- /* Our routine which actually does the transfer. */
- transfer_packet(dev, skb, i);
- }
-
- /* An xmit routine is expected to dispose of the packet, so we do. */
- dev_kfree_skb(skb);
-
- /* As per kernel convention, 0 means success. This is why I love
- * networking: even if we never sent to anyone, that's still
- * success! */
- return 0;
-}
-
-/*D:560
- * Packet receiving.
- *
- * First, here's a helper routine which fills one of our array of receive
- * buffers: */
-static int fill_slot(struct net_device *dev, unsigned int slot)
-{
- struct lguestnet_info *info = netdev_priv(dev);
-
- /* We can receive ETH_DATA_LEN (1500) byte packets, plus a standard
- * ethernet header of ETH_HLEN (14) bytes. */
- info->skb[slot] = netdev_alloc_skb(dev, ETH_HLEN + ETH_DATA_LEN);
- if (!info->skb[slot]) {
- printk("%s: could not fill slot %i\n", dev->name, slot);
- return -ENOMEM;
- }
-
- /* skb_to_dma() is a helper which sets up the "struct lguest_dma" to
- * point to the data in the skb: we also use it for sending out a
- * packet. */
- skb_to_dma(info->skb[slot], ETH_HLEN + ETH_DATA_LEN,
&info->dma[slot]);
-
- /* This is a Write Memory Barrier: it ensures that the entry in the
- * receive buffer array is written *before* we set the "used_len"
entry
- * to 0. If the Host were looking at the receive buffer array from a
- * different CPU, it could potentially see "used_len = 0" and not
see
- * the updated receive buffer information. This would be a horribly
- * nasty bug, so make sure the compiler and CPU know this has to happen
- * first. */
- wmb();
- /* Writing 0 to "used_len" tells the Host it can use this receive
- * buffer now. */
- info->dma[slot].used_len = 0;
- return 0;
-}
-
-/* This is the actual receive routine. When we receive an interrupt from the
- * Host to tell us a packet has been delivered, we arrive here: */
-static irqreturn_t lguestnet_rcv(int irq, void *dev_id)
-{
- struct net_device *dev = dev_id;
- struct lguestnet_info *info = netdev_priv(dev);
- unsigned int i, done = 0;
-
- /* Look through our entire receive array for an entry which has data
- * in it. */
- for (i = 0; i < ARRAY_SIZE(info->dma); i++) {
- unsigned int length;
- struct sk_buff *skb;
-
- length = info->dma[i].used_len;
- if (length == 0)
- continue;
-
- /* We've found one! Remember the skb (we grabbed the length
- * above), and immediately refill the slot we've taken it
- * from. */
- done++;
- skb = info->skb[i];
- fill_slot(dev, i);
-
- /* This shouldn't happen: micropackets could be sent by a
- * badly-behaved Guest on the network, but the Host will never
- * stuff more data in the buffer than the buffer length. */
- if (length < ETH_HLEN || length > ETH_HLEN + ETH_DATA_LEN) {
- pr_debug(KERN_WARNING "%s: unbelievable skb len: %i\n",
- dev->name, length);
- dev_kfree_skb(skb);
- continue;
- }
-
- /* skb_put(), what a great function! I've ranted about this
- * function before (http://lkml.org/lkml/1999/9/26/24). You
- * call it after you've added data to the end of an skb (in
- * this case, it was the Host which wrote the data). */
- skb_put(skb, length);
-
- /* The ethernet header contains a protocol field: we use the
- * standard helper to extract it, and place the result in
- * skb->protocol. The helper also sets up skb->pkt_type and
- * eats up the ethernet header from the front of the packet. */
- skb->protocol = eth_type_trans(skb, dev);
-
- /* If this device doesn't need checksums for sending, we also
- * don't need to check the packets when they come in. */
- if (dev->features & NETIF_F_NO_CSUM)
- skb->ip_summed = CHECKSUM_UNNECESSARY;
-
- /* As a last resort for debugging the driver or the lguest I/O
- * subsystem, you can uncomment the "#define DEBUG" at the top
- * of this file, which turns all the pr_debug() into printk()
- * and floods the logs. */
- pr_debug("Receiving skb proto 0x%04x len %i type %i\n",
- ntohs(skb->protocol), skb->len, skb->pkt_type);
-
- /* Update the packet and byte counts (visible from ifconfig,
- * and good for debugging). */
- dev->stats.rx_bytes += skb->len;
- dev->stats.rx_packets++;
-
- /* Hand our fresh network packet into the stack's "network
- * interface receive" routine. That will free the packet
- * itself when it's finished. */
- netif_rx(skb);
- }
-
- /* If we found any packets, we assume the interrupt was for us. */
- return done ? IRQ_HANDLED : IRQ_NONE;
-}
-
-/*D:550 This is where we start: when the device is brought up by dhcpd or
- * ifconfig. At this point we advertise our MAC address to the rest of the
- * network, and register receive buffers ready for incoming packets. */
-static int lguestnet_open(struct net_device *dev)
-{
- int i;
- struct lguestnet_info *info = netdev_priv(dev);
-
- /* Copy our MAC address into the device page, so others on the network
- * can find us. */
- memcpy(info->peer[info->me].mac, dev->dev_addr, ETH_ALEN);
-
- /* We might already be in promisc mode (dev->flags & IFF_PROMISC). Our
- * set_multicast callback handles this already, so we call it now. */
- lguestnet_set_multicast(dev);
-
- /* Allocate packets and put them into our "struct lguest_dma" array.
- * If we fail to allocate all the packets we could still limp along,
- * but it's a sign of real stress so we should probably give up now. */
- for (i = 0; i < ARRAY_SIZE(info->dma); i++) {
- if (fill_slot(dev, i) != 0)
- goto cleanup;
- }
-
- /* Finally we tell the Host where our array of "struct lguest_dma"
- * receive buffers is, binding it to the key corresponding to the
- * device's physical memory plus our peerid. */
- if (lguest_bind_dma(peer_key(info,info->me), info->dma,
- NUM_SKBS, lgdev_irq(info->lgdev)) != 0)
- goto cleanup;
- return 0;
-
-cleanup:
- while (--i >= 0)
- dev_kfree_skb(info->skb[i]);
- return -ENOMEM;
-}
-/*:*/
-
-/* The close routine is called when the device is no longer in use: we clean up
- * elegantly. */
-static int lguestnet_close(struct net_device *dev)
-{
- unsigned int i;
- struct lguestnet_info *info = netdev_priv(dev);
-
- /* Clear all trace of our existence out of the device memory by setting
- * the slot which held our MAC address to 0 (unused). */
- memset(&info->peer[info->me], 0,
sizeof(info->peer[info->me]));
-
- /* Unregister our array of receive buffers */
- lguest_unbind_dma(peer_key(info, info->me), info->dma);
- for (i = 0; i < ARRAY_SIZE(info->dma); i++)
- dev_kfree_skb(info->skb[i]);
- return 0;
-}
-
-/*D:510 The network device probe function is basically a standard ethernet
- * device setup. It reads the "struct lguest_device_desc" and sets
the "struct
- * net_device". Oh, the line-by-line excitement! Let's skip over it.
:*/
-static int lguestnet_probe(struct lguest_device *lgdev)
-{
- int err, irqf = IRQF_SHARED;
- struct net_device *dev;
- struct lguestnet_info *info;
- struct lguest_device_desc *desc = &lguest_devices[lgdev->index];
-
- pr_debug("lguest_net: probing for device %i\n", lgdev->index);
-
- dev = alloc_etherdev(sizeof(struct lguestnet_info));
- if (!dev)
- return -ENOMEM;
-
- SET_MODULE_OWNER(dev);
-
- /* Ethernet defaults with some changes */
- ether_setup(dev);
- dev->set_mac_address = NULL;
- random_ether_addr(dev->dev_addr);
-
- dev->open = lguestnet_open;
- dev->stop = lguestnet_close;
- dev->hard_start_xmit = lguestnet_start_xmit;
-
- /* We don't actually support multicast yet, but turning on/off
- * promisc also calls dev->set_multicast_list. */
- dev->set_multicast_list = lguestnet_set_multicast;
- SET_NETDEV_DEV(dev, &lgdev->dev);
-
- /* The network code complains if you have "scatter-gather"
capability
- * if you don't also handle checksums (it seem that would be
- * "illogical"). So we use a lie of omission and don't tell it
that we
- * can handle scattered packets unless we also don't want checksums,
- * even though to us they're completely independent. */
- if (desc->features & LGUEST_NET_F_NOCSUM)
- dev->features = NETIF_F_SG|NETIF_F_NO_CSUM;
-
- info = netdev_priv(dev);
- info->mapsize = PAGE_SIZE * desc->num_pages;
- info->peer_phys = ((unsigned long)desc->pfn << PAGE_SHIFT);
- info->lgdev = lgdev;
- info->peer = lguest_map(info->peer_phys, desc->num_pages);
- if (!info->peer) {
- err = -ENOMEM;
- goto free;
- }
-
- /* This stores our peerid (upper bits reserved for future). */
- info->me = (desc->features & (info->mapsize-1));
-
- err = register_netdev(dev);
- if (err) {
- pr_debug("lguestnet: registering device failed\n");
- goto unmap;
- }
-
- if (lguest_devices[lgdev->index].features & LGUEST_DEVICE_F_RANDOMNESS)
- irqf |= IRQF_SAMPLE_RANDOM;
- if (request_irq(lgdev_irq(lgdev), lguestnet_rcv, irqf, "lguestnet",
- dev) != 0) {
- pr_debug("lguestnet: cannot get irq %i\n", lgdev_irq(lgdev));
- goto unregister;
- }
-
- pr_debug("lguestnet: registered device %s\n", dev->name);
- /* Finally, we put the "struct net_device" in the generic
"struct
- * lguest_device"s private pointer. Again, it's not necessary, but
- * makes sure the cool kernel kids don't tease us. */
- lgdev->private = dev;
- return 0;
-
-unregister:
- unregister_netdev(dev);
-unmap:
- lguest_unmap(info->peer);
-free:
- free_netdev(dev);
- return err;
-}
-
-static struct lguest_driver lguestnet_drv = {
- .name = "lguestnet",
- .owner = THIS_MODULE,
- .device_type = LGUEST_DEVICE_T_NET,
- .probe = lguestnet_probe,
-};
-
-static __init int lguestnet_init(void)
-{
- return register_lguest_driver(&lguestnet_drv);
-}
-module_init(lguestnet_init);
-
-MODULE_DESCRIPTION("Lguest network driver");
-MODULE_LICENSE("GPL");
-
-/*D:580
- * This is the last of the Drivers, and with this we have covered the many and
- * wonderous and fine (and boring) details of the Guest.
- *
- * "make Launcher" beckons, where we answer questions like
"Where do Guests
- * come from?", and "What do you do when someone asks for
optimization?"
- */
==================================================================---
a/include/linux/lguest_bus.h
+++ /dev/null
@@ -1,51 +0,0 @@
-#ifndef _ASM_LGUEST_DEVICE_H
-#define _ASM_LGUEST_DEVICE_H
-/* Everything you need to know about lguest devices. */
-#include <linux/device.h>
-#include <linux/lguest.h>
-#include <linux/lguest_launcher.h>
-
-struct lguest_device {
- /* Unique busid, and index into lguest_page->devices[] */
- unsigned int index;
-
- struct device dev;
-
- /* Driver can hang data off here. */
- void *private;
-};
-
-/*D:380 Since interrupt numbers are arbitrary, we use a convention: each device
- * can use the interrupt number corresponding to its index. The +1 is because
- * interrupt 0 is not usable (it's actually the timer interrupt). */
-static inline int lgdev_irq(const struct lguest_device *dev)
-{
- return dev->index + 1;
-}
-/*:*/
-
-/* dma args must not be vmalloced! */
-void lguest_send_dma(unsigned long key, struct lguest_dma *dma);
-int lguest_bind_dma(unsigned long key, struct lguest_dma *dmas,
- unsigned int num, u8 irq);
-void lguest_unbind_dma(unsigned long key, struct lguest_dma *dmas);
-
-/* Map the virtual device space */
-void *lguest_map(unsigned long phys_addr, unsigned long pages);
-void lguest_unmap(void *);
-
-struct lguest_driver {
- const char *name;
- struct module *owner;
- u16 device_type;
- int (*probe)(struct lguest_device *dev);
- void (*remove)(struct lguest_device *dev);
-
- struct device_driver drv;
-};
-
-extern int register_lguest_driver(struct lguest_driver *drv);
-extern void unregister_lguest_driver(struct lguest_driver *drv);
-
-extern struct lguest_device_desc *lguest_devices; /* Just past max_pfn */
-#endif /* _ASM_LGUEST_DEVICE_H */
==================================================================---
a/include/linux/lguest_launcher.h
+++ b/include/linux/lguest_launcher.h
@@ -41,32 +41,6 @@ struct lguest_dma
__u32 used_len;
__u16 len[LGUEST_MAX_DMA_SECTIONS];
unsigned long addr[LGUEST_MAX_DMA_SECTIONS];
-};
-/*:*/
-
-/*D:460 This is the layout of a block device memory page. The Launcher sets up
- * the num_sectors initially to tell the Guest the size of the disk. The Guest
- * puts the type, sector and length of the request in the first three fields,
- * then DMAs to the Host. The Host processes the request, sets up the result,
- * then DMAs back to the Guest. */
-struct lguest_block_page
-{
- /* 0 is a read, 1 is a write. */
- int type;
- __u32 sector; /* Offset in device = sector * 512. */
- __u32 bytes; /* Length expected to be read/written in bytes */
- /* 0 = pending, 1 = done, 2 = done, error */
- int result;
- __u32 num_sectors; /* Disk length = num_sectors * 512 */
-};
-
-/*D:520 The network device is basically a memory page where all the Guests on
- * the network publish their MAC (ethernet) addresses: it's an array of
"struct
- * lguest_net": */
-struct lguest_net
-{
- /* Simply the mac address (with multicast bit meaning promisc). */
- unsigned char mac[6];
};
/*:*/
--
there are those who do and those who hang on and you don't see too
many doers quoting their contemporaries. -- Larry McVoy
Chris Malley
2007-Sep-28 04:34 UTC
[Lguest] [patch 34/43] lguest: Block driver using virtio.
rusty@rustcorp.com.au wrote:> + sprintf(vblk->disk->disk_name, "vd%c", virtblk_index++);Update lguest documentation to reflect the new virtual block device name. Signed-off-by: Chris Malley <mail@chrismalley.co.uk> --- --- Documentation/lguest/lguest.txt.orig 2007-09-28 12:00:32.000000000 +0100 +++ Documentation/lguest/lguest.txt 2007-09-28 12:04:30.000000000 +0100 @@ -69,7 +69,7 @@ - Run an lguest as root: - Documentation/lguest/lguest 64 vmlinux --tunnet=192.168.19.1 --block=rootfile root=/dev/lgba + Documentation/lguest/lguest 64 vmlinux --tunnet=192.168.19.1 --block=rootfile root=/dev/vda Explanation: 64: the amount of memory to use, in MB. @@ -80,10 +80,10 @@ --tunnet=192.168.19.1: configures a "tap" device for networking with this IP address. - --block=rootfile: a file or block device which becomes /dev/lgba + --block=rootfile: a file or block device which becomes /dev/vda inside the guest. - root=/dev/lgba: this (and anything else on the command line) are + root=/dev/vda: this (and anything else on the command line) are kernel boot parameters. - Configuring networking. I usually have the host masquerade, using
Rusty Russell
2007-Sep-29 06:27 UTC
[Lguest] [patch 34/43] lguest: Block driver using virtio.
On Fri, 2007-09-28 at 12:32 +0100, Chris Malley wrote:> rusty@rustcorp.com.au wrote: > > + sprintf(vblk->disk->disk_name, "vd%c", virtblk_index++); > > Update lguest documentation to reflect the new virtual block device name. > > Signed-off-by: Chris Malley <mail@chrismalley.co.uk>Thanks, applied! Rusty.
Eric Van Hensbergen
2007-Oct-09 13:25 UTC
[Lguest] [patch 00/43] lguest: Patches for 2.6.24 (and patchbomb test)
On 9/26/07, rusty@rustcorp.com.au <rusty@rustcorp.com.au> wrote:> Hi all, > > These are the patches I'm planning to submit for 2.6.24. Comments > gratefully accepted. Along with the usual cleanups and improvements are Jes' > de-i386-ification patches, and a new "virtio" mechanism designed to be shared > with KVM (and hopefully other hypervisors). >I applied the patchbomb to a branch in the v9fs.git tree in prep to write the virtio transport for 9p (it's under lguest-devel branch of http://www.kernel.org/pub/scm/linux/kernel/git/ericvh/v9fs.git -- although for some reason guilt put my name as the Author in place of Rusty everywhere...not sure why...anyways). It built clean, but when I tried to make beer to read the updated docs for virtio, it didn't seem to pick up the documentation for the console, network, or block devices (just the core device support code).... Something stupid I'm doing wrong? -eric
Hi Rusty, I have a late question :-)> There is also a generic implementation of config space which drivers can query > to get setup information from the host.After thinking about that for a while, is there a special reason the config field is defined as little endian? The only reason I can come up with, is to use virtio for cross platform emulators, lets say a ppc emulates an x86 system with virtio devices instead of real ones. Do you have other reasons for a fixed endianess? Christian
On 10/10/07, Christian Borntraeger <borntraeger@de.ibm.com> wrote:> Hi Rusty, > > I have a late question :-) > > > There is also a generic implementation of config space which drivers can query > > to get setup information from the host. > > After thinking about that for a while, is there a special reason the config > field is defined as little endian? The only reason I can come up with, is to > use virtio for cross platform emulators, lets say a ppc emulates an x86 system > with virtio devices instead of real ones.Not really. Some platforms have multi-endianess capabilities. So you may want a virtual machine running in a different endianess than the host machine. -- Glauber de Oliveira Costa. "Free as in Freedom" http://glommer.net "The less confident you are, the more serious you have to act."
On Wednesday 10 October 2007, Glauber de Oliveira Costa wrote:> On 10/10/07, Christian Borntraeger <borntraeger@de.ibm.com> wrote:> > > > After thinking about that for a while, is there a special reason the config > > field is defined as little endian? The only reason I can come up with, is to > > use virtio for cross platform emulators, lets say a ppc emulates an x86 system > > with virtio devices instead of real ones. > > Not really. > > Some platforms have multi-endianess capabilities. So you may want a > virtual machine running in a different endianess than the host > machine.Since virtio is a rather generic concept, there are other things you might want to do with them, like routing virtio data over the network to a machine with different endianess, or actually building hardware that implements some sort of virtio compatible interface. Arnd <><
On 10/10/07, Arnd Bergmann <arnd@arndb.de> wrote:> > Since virtio is a rather generic concept, there are other things you > might want to do with them, like routing virtio data over the network > to a machine with different endianess, or actually building hardware > that implements some sort of virtio compatible interface. >That being the case, why not just use network byte order (big endian)? -eric
Maybe Matching Threads
- [patch 00/43] lguest: Patches for 2.6.24 (and patchbomb test)
- [PATCH 0/7] Modify lguest32 to make room for lguest64
- [PATCH 0/7] Modify lguest32 to make room for lguest64
- [PATCH 0/5 -v2] Modify lguest32 to make room for lguest64 (version 2)
- [PATCH 0/5 -v2] Modify lguest32 to make room for lguest64 (version 2)