Here is some documentation I wrote as part of the doc day yesterday (and this morning if I''m honest). I posted a bunch of it yesterday but this incorporates some additional stuff. * Import PVHVM unplug protocol specification from qemu-xen-unstable.git. * Install markdown documentation as both HTML and TXT. * Updated unmodified devices README based on information on wiki. * Document the xl vif configuration syntax in a manner similar to the existing disk configuration. * Add some basic xl configuration file examples _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-Oct-27 08:58 UTC
[Xen-devel] [PATCH 1 of 6 DOCDAY] docs: import HVM emulated device unplug protocol spec
# HG changeset patch
# User Ian Campbell <ian.campbell@citrix.com>
# Date 1319646021 -3600
# Node ID 03b570c3792bb6abce46de5c9ab560ac169117c3
# Parent c681dd5aecf3da3c6fd0e4d8a760a9cd18617033
docs: import HVM emulated device unplug protocol spec
Convert to markdown as I go.
Currently this lives in qemu-xen.git i386-dm/README.hvm-pv-magic-ioport-disable
and I can never find it when I want it. As we transition to upstream qemu this
location becomes less useful.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
diff -r c681dd5aecf3 -r 03b570c3792b docs/misc/hvm-emulated-unplug.markdown
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/docs/misc/hvm-emulated-unplug.markdown Wed Oct 26 17:20:21 2011 +0100
@@ -0,0 +1,68 @@
+#Xen HVM emulated device unplug protocol
+
+The protocol covers three basic things:
+
+ * Disconnecting emulated devices.
+ * Getting log messages out of the drivers and into dom0.
+ * Allowing dom0 to block the loading of specific drivers. This is
+ intended as a backwards-compatibility thing: if we discover a bug
+ in some old version of the drivers, then rather than working around
+ it in Xen, we have the option of just making those drivers fall
+ back to emulated mode.
+
+The current protocol works like this (from the point of view of
+drivers):
+
+1. When the drivers first come up, they check whether the unplug logic
+ is available by reading a two-byte magic number from IO port `0x10`.
+ These should be `0x49d2`. If the magic number doesn''t match, the
+ drivers don''t do anything.
+
+2. The drivers read a one-byte protocol version from IO port `0x12`. If
+ this is 0, skip to 6.
+
+3. The drivers write a two-byte product number to IO port `0x12`. At
+ the moment, the only drivers using this protocol are our
+ closed-source ones, which use product number 1.
+
+4. The drivers write a four-byte build number to IO port `0x10`.
+
+5. The drivers check the magic number by reading two bytes from `0x10`
+ again. If it''s changed from `0x49d2` to `0xd249`, the drivers are
+ blacklisted and should not load.
+
+6. The drivers write a two-byte bitmask of devices to unplug to IO
+ port `0x10`. The defined fields are:
+
+ * `1` -- All IDE disks (not including CD drives)
+ * `2` -- All emulated NICs
+ * `4` -- All IDE disks except for the primary master (not including CD
+ drives)
+
+ The relevant emulated devices then disappear from the relevant
+ buses. For most guest operating systems, you want to do this
+ before device enumeration happens.
+
+Once the drivers have checked the magic number, they can send log
+messages to qemu which will be logged to wherever qemu''s logs go
+(`/var/log/xen/qemu-dm.log` on normal Xen, dom0 syslog on XenServer).
+These messages are written to IO port `0x12` a byte at a time, and are
+terminated by newlines. There''s a fairly aggressive rate limiter on
+these messages, so they shouldn''t be used for anything even vaguely
+high-volume, but they''re rather useful for debugging and support.
+
+It is still permitted for a driver to use this logging feature if it
+is blacklisted, but *ONLY* if it has checked the magic number and found
+it to be `0x49d2` or `0xd249`.
+
+This isn''t exactly a pretty protocol, but it does solve the problem.
+
+The blacklist is, from qemu''s point of view, handled mostly through
+xenstore. A driver version is considered to be blacklisted if
+`/mh/driver-blacklist/{product_name}/{build_number}` exists and is
+readable, where `{build_number}` is the build number from step 4 as a
+decimal number. `{product_name}` is a string corresponding to the
+product number in step 3.
+
+The master registry of product names and numbers is in
+qemu-xen-unstable''s xenstore.c.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Ian Campbell
2011-Oct-27 08:58 UTC
[Xen-devel] [PATCH 2 of 6 DOCDAY] docs: add some scaffolding for building HTML docs from markdown
# HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1319646217 -3600 # Node ID 52058c9448935818e2a810a438244e06f40b4e0e # Parent 03b570c3792bb6abce46de5c9ab560ac169117c3 docs: add some scaffolding for building HTML docs from markdown Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 03b570c3792b -r 52058c944893 docs/Docs.mk --- a/docs/Docs.mk Wed Oct 26 17:20:21 2011 +0100 +++ b/docs/Docs.mk Wed Oct 26 17:23:37 2011 +0100 @@ -7,3 +7,4 @@ DOXYGEN := doxygen POD2MAN := pod2man DOT := dot NEATO := neato +MARKDOWN := markdown diff -r 03b570c3792b -r 52058c944893 docs/Makefile --- a/docs/Makefile Wed Oct 26 17:20:21 2011 +0100 +++ b/docs/Makefile Wed Oct 26 17:23:37 2011 +0100 @@ -11,10 +11,11 @@ DOC_MAN1SRC := $(wildcard man/*.pod.1) DOC_MAN1 := $(patsubst man/%.pod.1,man1/%.1,$(DOC_MAN1SRC)) DOC_MAN5 := $(patsubst man/%.pod.5,man5/%.5,$(DOC_MAN5SRC)) DOC_TEX := src/user.tex src/interface.tex +DOC_MARKDOWN := $(wildcard misc/*.markdown) DOC_PS := $(patsubst src/%.tex,ps/%.ps,$(DOC_TEX)) DOC_PDF := $(patsubst src/%.tex,pdf/%.pdf,$(DOC_TEX)) -DOC_HTML := $(patsubst src/%.tex,html/%/index.html,$(DOC_TEX)) - +DOC_HTML := $(patsubst src/%.tex,html/%/index.html,$(DOC_TEX)) \ + $(patsubst %.markdown,html/%.html,$(DOC_MARKDOWN)) GFX = $(patsubst %.fig, %.eps, $(wildcard figs/*.fig)) .PHONY: all @@ -37,9 +38,7 @@ ps: $(DOC_PS) pdf: $(DOC_PDF) .PHONY: html -html: - @if which $(LATEX2HTML) 1>/dev/null 2>/dev/null; then \ - $(MAKE) $(DOC_HTML); fi +html: $(DOC_HTML) .PHONY: python-dev-docs python-dev-docs: @@ -110,7 +109,16 @@ ps/%.ps: %.dvi $(FIG2DEV) -L eps $< $@ html/%/index.html: src/%.tex - $(INSTALL_DIR) $(@D) + @$(INSTALL_DIR) $(@D) + @set -e ; if which $(LATEX2HTML) 1>/dev/null 2>/dev/null; then \ + echo "Running latex2html to generate $*/index.html ... "; \ $(LATEX2HTML) -split 0 -show_section_numbers -toc_depth 3 -nonavigation \ -numbered_footnotes -local_icons -noinfo -math -dir $(@D) \ - $< 1>/dev/null 2>/dev/null + $< 1>/dev/null 2>/dev/null ;fi + +html/%.html: %.markdown + @$(INSTALL_DIR) $(@D) + @set -e ; if which $(MARKDOWN) 1>/dev/null 2>/dev/null; then \ + echo "Running markdown to generate $*.html ... "; \ + $(MARKDOWN) $< > $@.tmp ; \ + $(call move-if-changed,$@.tmp,$@) ; fi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-Oct-27 08:58 UTC
[Xen-devel] [PATCH 3 of 6 DOCDAY] unmodified_drivers: update README from http://wiki.xen.org/xenwiki/UnmodifiedDrivers
# HG changeset patch
# User Ian Campbell <ian.campbell@citrix.com>
# Date 1319647443 -3600
# Node ID b49eb99dc11aff809566f4b5bd53053aa5daef7d
# Parent 52058c9448935818e2a810a438244e06f40b4e0e
unmodified_drivers: update README from
http://wiki.xen.org/xenwiki/UnmodifiedDrivers
Add reference to the fact that these drivers are for "classic-Xen"
kernels only
and do not work with PVops but point towards the PVHVM functionality in
mainstream.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
diff -r 52058c944893 -r b49eb99dc11a unmodified_drivers/linux-2.6/README
--- a/unmodified_drivers/linux-2.6/README Wed Oct 26 17:23:37 2011 +0100
+++ b/unmodified_drivers/linux-2.6/README Wed Oct 26 17:44:03 2011 +0100
@@ -1,10 +1,19 @@
-To build:
+These drivers provide paravirtualised drivers for pre-PVops Linux
+kernels which have the classic-Xen patches applied. They do not work
+with kernels which are not patched with the classic-Xen patches. In
+particular this includes all PVops kernels.
-1. ./mkbuildtree [<arch>]
+For 2.6.36 and newer you should use the "PVHVM" functionality which
is
+available in the mainline kernel.
+
+To build do one of:
+
+ ./mkbuildtree [<arch>]
NB. You can override paths to Xen sources and a (stub) XenLinux
build tree via the XEN and XL environment variable.
-2. make -C /path/to/kernel/build M=$PWD modules
+- or -
+ make -C /path/to/kernel/build M=$PWD modules
NB. This is your native kernel build tree (or a distro provided
stub), not the XenLinux sources referred to in step 1.
NB. If you are cross compiling, you need to set ARCH and
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Ian Campbell
2011-Oct-27 08:58 UTC
[Xen-devel] [PATCH 4 of 6 DOCDAY] docs: add a document describing the xl vif syntax
# HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1319705668 -3600 # Node ID 996143b104f0c027ad45b9e04bdccd342e7a1b94 # Parent b49eb99dc11aff809566f4b5bd53053aa5daef7d docs: add a document describing the xl vif syntax Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r b49eb99dc11a -r 996143b104f0 docs/misc/xl-network-configuration.markdown --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/docs/misc/xl-network-configuration.markdown Thu Oct 27 09:54:28 2011 +0100 @@ -0,0 +1,126 @@ +# XL Network Configuration + +## Syntax Overview + +This document specifies the xl config file format vif configuration +option. It has the following form: + + vif = [ ''<vifspec>'', ''<vifspec>'', ... ] + +where each vifspec is in this form: + + [<key>=<value>|<flag>,] + +For example: + + ''mac=00:16:3E:74:3d:76,model=rtl8139,bridge=xenbr0'' + ''mac=00:16:3E:74:34:32'' + '''' # The empty string + +These might be specified in the domain config file like this: + + vif = [ ''mac=00:16:3E:74:34:32'', ''mac=00:16:3e:5f:48:e4,bridge=xenbr1'' ] + +More formally, the string is a series of comma-separated keyword/value +pairs. All keywords are optional. + +Each device has a `DEVID` which is its index within the vif list, starting from 0. + +## Keywords + +### mac + +If specified then this option specifies the MAC address inside the +guest of this VIF device. The value is a 48-bit number represented as +six groups of two hexadecimal digits, separated by colons (:). + +The default if this keyword is not specified is to be automatically +generate a MAC address inside the space assigned to Xen''s +[Organizationally Unique Identifier][oui] (00:16:3e). + +If you are choosing a MAC address then it is strongly recommend to +follow one of the following strategies: + + * Generate a random sequence of 6 byte, set the locally administered + bit (bit 2 of the first byte) and clear the multicast bit (bit 1 + of the first byte). In other words the first byte should have the + bit pattern xxxxxx10 (where x is a randomly generated bit) and the + remaining 5 bytes are randomly generated See + [http://en.wikipedia.org/wiki/MAC_address] for more details the + structure of a MAC address. + * Allocate an address from within the space defined by your + organization''s OUI (if you have one) following your organization''s + procedures for doing so. + * Allocate an address from within the space defined by Xen''s OUI + (00:16:3e). Taking care not to clash with other users of the + physical network segment where this VIF will reside. + +If you have an OUI for your own use then that is the preferred +strategy. Otherwise in general you should prefer to generate a random +MAC and set the locally administered bit since this allows for more +bits of randomness than using the Xen OUI. + +### bridge + +Specifies the name of the network bridge which this VIF should be +added to. The default is `xenbr0`. The bridge must be configured using +your distribution''s network configuration tools. See the [wiki][net] +for guidance and examples. + +### type + +This keyword is valid for HVM guests only. + +Specifies the type of device to valid values are: + + * `ioemu` (default) -- this device will be provided as an emulate + device to the guest and also as a paravirtualised device which the + guest may choose to use instead if it has suitable drivers + available. + * `vif` -- this device will be provided as a paravirtualised device + only. + +### model + +This keyword is valid for HVM guest devices with `type=ioemu` only. + +Specifies the type device to emulated for this guest. Valid values +are: + + * `rtl8139` (default) -- Realtek RTL8139 + * `e1000` -- Intel E1000 + * in principal any device supported by your device model + +### vifname + +This keyword is valid for HVM guest devices with `type=ioemu` only. + +Specifies the backend device name for an emulated device. The default +is `tapDOMID.DEVID` where `DOMID` is the guest domain ID and `DEVID` +is the device number. + +### script + +Specifies the hotplug script to run to configure this device (e.g. to +add it to the relevant bridge). Defaults to +`XEN_SCRIPT_DIR/vif-bridge` but can be set to any script. Some example +scripts are installed in `XEN_SCRIPT_DIR`. + +### ip + +Specifies the IP address for the device, the default is not to +specify an IP address. + +What, if any, effect this has depends on the hotplug script which is +configured. A typically behaviour (exhibited by the example hotplug +scripts) if set might be to configure firewall rules to allow only the +specified IP address to be used by the guest (blocking all others). + +### backend + +Specifies the backend domain which this device should attach to. This +defaults to domain 0. Specifying another domain requires setting up a +driver domain which is outside the scope of this document. + +[oui]: http://en.wikipedia.org/wiki/Organizationally_Unique_Identifier +[net]: http://wiki.xen.org/xenwiki/HostConfiguration/Networking _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-Oct-27 08:58 UTC
[Xen-devel] [PATCH 5 of 6 DOCDAY] xl: add some example configuration files
# HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1319705714 -3600 # Node ID 74e5af602abf03022e8e118bb58876e13812612f # Parent 996143b104f0c027ad45b9e04bdccd342e7a1b94 xl: add some example configuration files These are much more minimal than the equivalent xmexamples but should eventually reference a more complete document describing the available options. These are based on the guest configuration files that I actually use day-to-day. There is an unresolved reference to a document describing the complete syntax. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 996143b104f0 -r 74e5af602abf tools/examples/xlexample.hvm --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/examples/xlexample.hvm Thu Oct 27 09:55:14 2011 +0100 @@ -0,0 +1,39 @@ +# ====================================================================+# Example HVM guest configuration +# ====================================================================+# +# This is a fairly minimal example of what is required for an +# HVM guest. For a more complete guide see <XXX Document TBD> + +# Guest name +name = "example.hvm" + +# 128-bit UUID for the domain as a hexadecimal number. +# Use "uuidgen" to generate one if required. +# The default behavior is to generate a new UUID each time the guest is started. +#uuid = "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" + +# Initial memory allocation (MB) +memory = 128 + +# Maximum memory (MB) +# If this is greater than `memory'' then the slack will start ballooned +# (this assumes guest kernel support for ballooning) +#maxmem = 512 + +# Number of VCPUS +vcpus = 2 + +# Network devices +# A list of ''vifspec'' entries as described in +# docs/misc/xl-network-configuration.markdown +vif = [ '''' ] + +# Disk Devices +# A list of `diskspec'' entries as described in +# docs/misc/xl-disk-configuration.txt +disk = [ ''/dev/vg/guest-volume,raw,xvda,rw'' ] + +# Guest VGA console configuration, either SDL or VNC +sdl = 1 +#vnc = 1 diff -r 996143b104f0 -r 74e5af602abf tools/examples/xlexample.pvlinux --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/examples/xlexample.pvlinux Thu Oct 27 09:55:14 2011 +0100 @@ -0,0 +1,44 @@ +# ====================================================================+# Example PV Linux guest configuration +# ====================================================================+# +# This is a fairly minimal example of what is required for a +# Paravirtualised Linux guest. For a more complete guide see <XXX Document TBD> + +# Guest name +name = "example.pvlinux" + +# 128-bit UUID for the domain as a hexadecimal number. +# Use "uuidgen" to generate one if required. +# The default behavior is to generate a new UUID each time the guest is started. +#uuid = "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" + +# Kernel image to boot +kernel = "/boot/vmlinuz" + +# Ramdisk (optional) +#ramdisk = "/boot/initrd.gz" + +# Kernel command line options +extra = "root=/dev/xvda1" + +# Initial memory allocation (MB) +memory = 128 + +# Maximum memory (MB) +# If this is greater than `memory'' then the slack will start ballooned +# (this assumes guest kernel support for ballooning) +#maxmem = 512 + +# Number of VCPUS +vcpus = 2 + +# Network devices +# A list of ''vifspec'' entries as described in +# docs/misc/xl-network-configuration.markdown +vif = [ '''' ] + +# Disk Devices +# A list of `diskspec'' entries as described in +# docs/misc/xl-disk-configuration.txt +disk = [ ''/dev/vg/guest-volume,raw,xvda,rw'' ] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-Oct-27 08:58 UTC
[Xen-devel] [PATCH 6 of 6 DOCDAY] docs: install text documentation
# HG changeset patch
# User Ian Campbell <ian.campbell@citrix.com>
# Date 1319705722 -3600
# Node ID 8ddcfa3917a695cd6d533376e11e0e6e4da8d673
# Parent 74e5af602abf03022e8e118bb58876e13812612f
docs: install text documentation
Including markdown docs.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
diff -r 74e5af602abf -r 8ddcfa3917a6 .hgignore
--- a/.hgignore Thu Oct 27 09:55:14 2011 +0100
+++ b/.hgignore Thu Oct 27 09:55:22 2011 +0100
@@ -57,6 +57,7 @@
^docs/user/labels\.pl$
^docs/user/user\.css$
^docs/user/user\.html$
+^docs/txt/.*$
^docs/xen-api/vm_lifecycle.eps$
^docs/xen-api/xenapi-datamodel-graph.eps$
^docs/xen-api/xenapi.out$
diff -r 74e5af602abf -r 8ddcfa3917a6 docs/Makefile
--- a/docs/Makefile Thu Oct 27 09:55:14 2011 +0100
+++ b/docs/Makefile Thu Oct 27 09:55:22 2011 +0100
@@ -16,13 +16,16 @@ DOC_PS := $(patsubst src/%.tex,ps/%.ps,
DOC_PDF := $(patsubst src/%.tex,pdf/%.pdf,$(DOC_TEX))
DOC_HTML := $(patsubst src/%.tex,html/%/index.html,$(DOC_TEX)) \
$(patsubst %.markdown,html/%.html,$(DOC_MARKDOWN))
+DOC_TXT := $(patsubst %.txt,txt/%.txt,$(wildcard misc/*.txt)) \
+ $(patsubst %.markdown,txt/%.txt,$(DOC_MARKDOWN))
+
GFX = $(patsubst %.fig, %.eps, $(wildcard figs/*.fig))
.PHONY: all
all: build
.PHONY: build
-build: ps pdf html man-pages
+build: ps pdf html txt man-pages
@if which $(DOT) 1>/dev/null 2>/dev/null ; then \
$(MAKE) -C xen-api build ; else \
echo "Graphviz (dot) not installed; skipping xen-api." ; fi
@@ -40,6 +43,9 @@ pdf: $(DOC_PDF)
.PHONY: html
html: $(DOC_HTML)
+.PHONY: txt
+txt: $(DOC_TXT)
+
.PHONY: python-dev-docs
python-dev-docs:
@mkdir -v -p api/tools/python
@@ -122,3 +128,13 @@ html/%.html: %.markdown
echo "Running markdown to generate $*.html ... "; \
$(MARKDOWN) $< > $@.tmp ; \
$(call move-if-changed,$@.tmp,$@) ; fi
+
+txt/%.txt: %.txt
+ $(INSTALL_DIR) $(@D)
+ cp $< $@.tmp
+ $(call move-if-changed,$@.tmp,$@)
+
+txt/%.txt: %.markdown
+ $(INSTALL_DIR) $(@D)
+ cp $< $@.tmp
+ $(call move-if-changed,$@.tmp,$@)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Ian Jackson
2011-Nov-01 18:41 UTC
Re: [Xen-devel] [PATCH 1 of 6 DOCDAY] docs: import HVM emulated device unplug protocol spec
Ian Campbell writes ("[Xen-devel] [PATCH 1 of 6 DOCDAY] docs: import HVM
emulated device unplug protocol spec"):> docs: import HVM emulated device unplug protocol spec
Applied all six, thanks.
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel