This is the first draft of the use case on "Hotplug for
Virtualization".
Please review and share your comments.
Next week (when Mary is back) we'll post this use case both off the
Hotplug SIG webpage and the use case page (mail will be sent to let you
know when it's posted).
There's also a new terminology page dedicated to virtualization specific
terms that are referred to in the use case, please review that page as
well http://developer.osdl.org/maryedie/HOTPLUG/VirtTerminology.shtml
Thanks for your input.
Martine
HOTPLUG for VIRTUALIZATION USE CASE
---------------------------------
Use Case:
Hotplug for Virtualization
---------------------------------
Version 0.1 (Draft)
Last Modified Date: 08/19/05
Copyright (c) 2005 by The Open Source Development Lab, Inc. Verbatim
copying and distribution of this document is permitted in any medium,
provided this notice is preserved. Draft copies of this document may
not be posted publicly without indicating the draft status.
---------------------------------
Table of Contents
Description
Target Acceptance
Participants/Roles
Scenarios
Dependencies
Implementation Notes
References
---------------------------------
---------------------------------
Description
---------------------------------
Independently of the size or number of actual physical systems used,
(our concept of virtualization includes using multiple separate
systems as the hardware base) virtualization provides an abstraction
layer to give the user access to virtual machines that are independent
of each other. This allows for higher quality of service resource
allocation, increases security through resource isolation, provides
transparent resource redirection, leads to better hardware
consolidation and ultimately also lowers the management cost. The
mapping between physical resource and virtual resource can be achieved
through numerous mechanisms which are not the focus of this use case.
However to keep our focus on the needs of the majority of the open
source community we will limit the scope of this use case to the
virtual machine monitors known as Type-I VMM which include Xen,
VMWare ESX and Virtual Iron (TM) VFe. For further details on the
various types of virtual machine monitors refer to R.P. Goldberg's
article (see reference [Goldberg74]). Within the scope of the Type-I
VMM the virtual layer can be seen as a combination of the virtual
machine monitor and some "special" virtual machines. We will refer
to the kernel associated with the VMM as the "VMM kernel".
The kernel and OS associated with the Virtual Machine will be called
the "guest kernel" and "guest OS" respectively.
Our goal here is
1) to determine the role of hotplug in virtualization and
2) to identify the requirements to support the various operations both
at the physical and virtualization layer that make use of hotplug.
In this document we will use commonly known hotplug specific
terminology defined at
developer.osdl.org/maryedie/HOTPLUG/Terminology.shtml as well as
commonly known virtualization specific terminology defined at
developer.osdl.org/maryedie/HOTPLUG/VirtTerminology.shtml
---------------------------------
Target Acceptance
---------------------------------
When an application is running on a virtual machine it should be able
to expect maximum reliability of the system, have access to any
resources it needs at any given time without disruption to its
execution and be unaffected by any configuration changes happening at
the hardware level.
Replacement of hardware components and addition and/or removal of
physical or virtual components should be totally transparent to the
application. In this context components refer to system hardware
resources like processors, memory, I/O devices, and nodes which is
constituted of any combination of processors, memory and I/O combined
as a hotplugable unit. The requirements for hotplug support of
virtualization should be independent of the choice of virtualization
approach and should be integrated into the mainline kernel.
Virtualization will provide a perfect test environment for different
hotplug features. This use case will help the Hotplug SIG define the
appropriate test scenarios to support virtualization. We can also use
this description to do a gap analysis for the hotplug code that is
either already in the kernel or is currently being developed.
---------------------------------
Participants/Roles
---------------------------------
Systems Administrator
---
Special class of user that has special privileges on a given
system. This is a role held by an individual that acts as
the administrator for a system.
---
* Application Administrator
---
Special class of user that has special privileges for a given
application. This is a role held by an individual that acts as
the administrator for all aspects of an application.
---
* User
---
Any user on the system. This is a role that is held by all individuals
using a system. The user can interact with the system through the
processes associated with applications they are using. Root Users
are users who have root privileges. They are typically the system
administrator. Privileged users have some of the root user
privileges, but not all. They are typically an operations staff
member.
---------------------------------
Scenarios
---------------------------------
There are 4 considered scenarios:
-------------------------
1.Serviceability (hotplug at physical layer)
-------------------------
In this sub-case the System Administrator needs the ability to
remove/replace failing components.
Unfortunately, CPU failures tend to be fatal and usually don't give
any warning. Fortunately, they're also very infrequent. Because they
are usually fatal, it's likely that you won't be looking at a
hot-remove scenario.(Though, if you have a processor failure and
remove it while the system is down, the System Administrator needs
the option to reboot immediately and hot-add the replacement). In
contrast, memory and I/O often give adequate warning, via single-bit
or parity errors, that they're failing; thus providing an opportunity
to have them replaced before they cause a system failure.
-------------------------------------
2.Capacity management (hotplug at physical layer)
-------------------------------------
System Administrators need the option to add more physical resources
to the virtualization layer to create larger virtual machines, or to
relocate physical resources when balancing hardware resources
across multiple virtualization layers to support specific workloads.
------------------------------------------------
3.Migration of the virtualization layer (hotplug at physical layer)
------------------------------------------------
In this sub-case the System Administrator adds some resources
and removes others as a way to migrate the virtualization
layer onto different hardware. "Different hardware" may mean
completely different hardware, or it could simply mean upgrades
of existing hardware. It could also be accomplished on
a system that provides hard partitioning.
------------------------------------------------
4.Virtual resources management (hotplug at the virtualization layer)
------------------------------------------------
This sub-case deals with hot-plug of virtual resources to/from the
OS instances in the virtual machines. The reason to do this is for
capacity management: giving each virtual machine exactly the resources
that it requires to support application workload requirements, and
making those resources apparent to the guest OS,while leaving the
remainder available for other virtual machines. The need
for hot-plug of resources to/from the guest OS(es) depends
on how the virtualization layer and the OS(es) interact. However the
hotplug features required to support either case should be mainstream.
------------------------------------------------
Workflow for Scenario (1) on Serviceability
------------------------------------------------
This scenario covers the expected succession of events when a
component shows signs of failure such as multiple parity errors for
memory. We assume that some sort of event log analyzer will detect
that a component is displaying signs of possible failure and will
either post a message at a predetermined location or send a message to
a dedicated thread to take action. The choice of posting a message or
automatically taking action to isolate the faulty hardware component
from the rest of the system should be done at boot time most likely
through a configuration option. In the case where a message is posted
it is up to the system administrator to take the initiative of
requesting that the component be isolated and eventually replaced
using hotplug functionality. A hotplug event with remove action needs
to be generated to inform the host kernel that the component needs to
be hot-removed.
The hotplug event handler also has to notify the virtualization layer
that a hardware resource will be eliminated so that the virtual
machine(s) that was (were) currently using it get it taken away from
their resources or have it substituted by another available equivalent
physical resource.
------------------------------------------------
Workflow for Scenario (2) on Capacity management
------------------------------------------------
This scenario covers the expected succession of events when a system
administrator decides that the existing physical resources are no
longer sufficient to either cover the needs of the current virtual
machines or to allow for creation of new virtual machines needed to
complete the project. The virtualization layer may or may not provide
a management tool as an aid to the system administrator to highlight
such needs . Such management tool could also provide a means to
communicate with the virtualization layer that a new component needs
to be added and that a hotplug event to add that component needs to
be generated.
------------------------------------------------
Workflow for Scenario (3) on Migration of the virtualization layer
------------------------------------------------
This scenario covers the expected succession of events when a system
administrator decides that a set of existing physical resources must
be replaced either in the context of an upgrade or for full
replacement of one of the platforms that contributes to the hardware
resources. The latter case can of course only occur in a configuration
in which the hardware resources are constituted by multiple separate
platforms. It is the responsibility of the system administrator to
either directly inform the VMM of the request for hot remove of those
components or to provide the management tool with the information
required to handle the hotplug event.
The challenge in this specific scenario is that several components
will be removed at the same time. So if the mechanism used to transfer
the VMM to other resources can be made aware of the fact that multiple
components are being hot-removed the operation may be more efficient
and coherent than if each component is removed individually.
After the hot-remove of all components was successfully accomplished
the physical components are replaced, and the proper actions are
taken to initiate to trigger hot-add of those new components. The
specifics of how the hot-add is triggered is very dependent of the
VMM itself (depends on things such as what the VMM kernel is or if
the VMM has its own embedded management tool).
------------------------------------------------
Workflow for Scenario (4) on Virtual resources management
------------------------------------------------
This scenario covers the expected succession of events when
redistribution of resources is required to address the current needs
of the guest OSs as their workloads vary. An application manager or
a priviledged user knowing their application's workload requirements
may request to the VMM that specific type/amount of resources be
allocated to a given virtual machine. We assume that the guest OS
running on the virtual machine can support all types of hotplug
events. Also for this use case we will assume that any given CPU
that has been hot-added to a virtual machine is fully dedicated to
that VM. From the OS's point-of-view the hotplug event will be handled
identically as when the OS is running directly on the hardware except
that when a resource such as a CPU is removed instead of passing it
down to the PAL it will be passed to the VMM. While each specific
implementation of virtualization may lead to a different interaction
process between the VMM and the OS running on each VM, the mechanism
of redistributing the virtual resources from a hotplug point-of-view
is the same.
---------------------------------
Dependencies
---------------------------------
** An event log analyzer needs to be present in the system to detect
when a component shows signs of failure.
** The guest OS support logical hotplugging of all possible components
** Possibly changes will be needed in the system firmware to support
the various hotplug operations.
---------------------------------
Implementation Notes
---------------------------------
In the previous 3 scenarios one has to take into account the
possibility that the replacement hardware is of a different type than
the original, for example a more recent version of a device or a
different speed CPU. The hot-add operations for each component is
responsible for handling such event.
---------------------------------
References
---------------------------------
[Goldberg74] "Survey of Virtual Machines Research", Robert P.
Goldberg, IEEE Computer, pp. 34-45, June 1974
