Michael S. Tsirkin
2013-May-29 07:49 UTC
[PATCH v2 2/2] virtio-balloon spec: reintroduce "silent deflation" feature
On Tue, May 28, 2013 at 07:40:18PM +0200, Paolo Bonzini wrote:> The original idea of the VIRTIO_BALLOON_F_MUST_TELL_HOST feature was to > let drivers skip usage of the deflate queue when leaking the balloon > ("silent deflation"). Guests may benefit from silent deflate by > aggressively inflating the balloon; they know that they will be able to > use ballooned pages without issuing a (blocking) request to the device. > > The previous patch redefined the feature to ensure correctness of the > operation when drivers do not correctly report deflation. This patch > adds back the optimization. > > The new feature bit is for the host to tell the drivers if silent > deflation is actually supported. The meaning of the feature bit is > reversed compared to the original, because the original meaning was > not safe against migration. > > For features to be safe against migration, they have to be defined as > "this is true if the guest _can_ do X". For such a "positive" feature, > migration is possible if the destination supports it, or the source > didn't set it: > > dest support source set ok? > T T T > T F T > F T F > F F T > > Instead, the old feature was defined as "this is true if the guest > _cannot_ do X". For such a "negative" feature, migration is possible > if the destination supports it, or the source sets it: > > dest support source set ok? > T T T > T F F > F T T > F F T > > However, the negotiated features are supposed to be the AND of the > device- and driver-supported features. In the F/T case, the feature > would be negotiated by the source as T, and become F when negotiated on > the destination. > > Signed-off-by: Paolo Bonzini <pbonzini at redhat.com>Do you have any numbers showing how this new feature improves performance? We are able to batch quite a lot of pages in a single deflate request - is the overhead measureable in practice?> --- > virtio-spec.lyx | 197 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 197 insertions(+) > > diff --git a/virtio-spec.lyx b/virtio-spec.lyx > index 5c76a87..5da4fde 100644 > --- a/virtio-spec.lyx > +++ b/virtio-spec.lyx > @@ -7267,6 +7267,20 @@ VIRTIO_BALLOON_F_STATS_VQ > \end_inset > > (1) A virtqueue for reporting guest memory statistics is present. > +\change_inserted 1531152142 1347020627 > + > +\end_layout > + > +\begin_layout Description > + > +\change_inserted 1531152142 1347020648 > +VIRTIO_BALLOON_F_SILENT_DEFLATE > +\begin_inset space ~ > +\end_inset > + > +(2) Guest need not tell host before pages from the balloon are used. > +\change_unchanged > + > \end_layout > > \end_deeper > @@ -7627,6 +7641,168 @@ VIRTIO_BALLOON_S_MEMFREE The amount of memory not being used for any purpose > VIRTIO_BALLOON_S_MEMTOT The total amount of memory available (in bytes). > \end_layout > > +\begin_layout Description > + > +\change_inserted 1531152142 1368005515 > +Silent > +\begin_inset space ~ > +\end_inset > + > +deflation > +\end_layout > + > +\begin_layout Standard > + > +\change_inserted 1531152142 1368005515 > + > +\series medium > +Some implementation of the balloon device may not require the guest to deflate > + the balloon explicitly; instead, the guest may just take a page from its > + reserve and start using it. > + This is called > +\begin_inset Quotes eld > +\end_inset > + > +silent deflate > +\begin_inset Quotes erd > +\end_inset > + > + > +\end_layout > + > +\begin_layout Standard > + > +\change_inserted 1531152142 1368005515 > +In order to use this feature effectively, both the guest and the host need > + to know how the other part intends to use the balloon. > +\end_layout > + > +\begin_layout Standard > + > +\change_inserted 1531152142 1368005515 > + > +\series medium > +Guests may benefit from silent deflate by aggressively inflating the balloon; > + they know that they will be able to use ballooned pages without issuing > + a (blocking) request to the device. > +\end_layout > + > +\begin_layout Standard > + > +\change_inserted 1531152142 1368005515 > +Knowing that the guest will > +\emph on > +not > +\emph default > + deflate silently also benefits the host. > + For example, if the host is pinning the guest's memory, it may unpin ballooned > + pages and pin them again upon deflation. > + This allows cooperative memory overcommit even if the guest's memory is > + pinned. > +\end_layout > + > +\begin_layout Standard > + > +\change_inserted 1531152142 1368005515 > +Thus, there are two possibilities for the host (either it does not support > + silent deflate, or it does), and three for the guest (it doesn't need silent > + deflate, it may choose to use it if available, it requires it). > + Because there are three possibilities for the guest, support for silent > + deflate is represented by two different feature bits. > +\end_layout > + > +\begin_layout Standard > + > +\change_inserted 1531152142 1368005515 > +The feature bits are used as follows: > +\end_layout > + > +\begin_layout Itemize > + > +\change_inserted 1531152142 1368005671 > +if the host does not support silent deflate, it must propose the > +VIRTIO_\SpecialChar \- > +BALLOON_F_\SpecialChar \- > +CAN_\SpecialChar \- > +TELL_\SpecialChar \- > +HOST feature. > + If the guest does > +\emph on > +not > +\emph default > + negotiate it, the host should assume that the guest will use silent deflate. > + A host that does not support silent deflate then must do nothing on attempts > + of the guest to deflate/inflate the balloon, except put the descriptors on > + the used ring. > +\end_layout > + > +\begin_layout Itemize > + > +\change_inserted 1531152142 1368005515 > +a guest that > +\emph on > +must > +\emph default > + use silent deflation does not need to negotiate any feature. > +\end_layout > + > +\begin_layout Itemize > + > +\change_inserted 1531152142 1368005676 > +a guest that > +\emph on > +will never > +\emph default > + use silent deflation only has to propose VIRTIO_\SpecialChar \- > +BALLOON_F_\SpecialChar \- > +CAN_\SpecialChar \- > +TELL_\SpecialChar \- > +HOST. > + It need not do anything if the host does not negotiate the feature. > +\end_layout > + > +\begin_layout Itemize > + > +\change_inserted 1531152142 1368005686 > +a guest that > +\emph on > +can optionally > +\emph default > + use silent deflation should propose both VIRTIO_\SpecialChar \- > +BALLOON_F_\SpecialChar \- > +CAN_\SpecialChar \- > +TELL_\SpecialChar \- > +HOST > + and VIRTIO_\SpecialChar \- > +BALLOON_F_\SpecialChar \- > +SILENT_\SpecialChar \- > +DEFLATE. > + The guest driver can then use silent deflation if and only if the host > + has negotiated VIRTIO_\SpecialChar \- > +BALLOON_F_\SpecialChar \- > +SILENT_\SpecialChar \- > +DEFLATE too. > +\end_layout > + > +\begin_layout Standard > + > +\change_inserted 1531152142 1368005690 > +Hosts are free to propose the VIRTIO_\SpecialChar \- > +BALLOON_F_\SpecialChar \- > +CAN_\SpecialChar \- > +TELL_\SpecialChar \- > +HOST feature even if they support silent deflation. > + It will > + not affect operation of a well-behaving guest; such a guest > + should never check if the host proposed the feature, and > + should not assume that lack of this feature implies support > + for silent deflation. The only bit to check for this purpose is > + VIRTIO_\SpecialChar \- > +BALLOON_F_\SpecialChar \- > +SILENT_\SpecialChar \- > +DEFLATE. > +\end_layout > + > \begin_layout Chapter* > Appendix H: Rpmsg: Remote Processor Messaging > \end_layout > -- > 1.8.2.1
Paolo Bonzini
2013-May-29 08:47 UTC
[PATCH v2 2/2] virtio-balloon spec: reintroduce "silent deflation" feature
Il 29/05/2013 09:49, Michael S. Tsirkin ha scritto:> On Tue, May 28, 2013 at 07:40:18PM +0200, Paolo Bonzini wrote: >> The original idea of the VIRTIO_BALLOON_F_MUST_TELL_HOST feature was to >> let drivers skip usage of the deflate queue when leaking the balloon >> ("silent deflation"). Guests may benefit from silent deflate by >> aggressively inflating the balloon; they know that they will be able to >> use ballooned pages without issuing a (blocking) request to the device. >> >> The previous patch redefined the feature to ensure correctness of the >> operation when drivers do not correctly report deflation. This patch >> adds back the optimization. >> >> The new feature bit is for the host to tell the drivers if silent >> deflation is actually supported. The meaning of the feature bit is >> reversed compared to the original, because the original meaning was >> not safe against migration. >> >> For features to be safe against migration, they have to be defined as >> "this is true if the guest _can_ do X". For such a "positive" feature, >> migration is possible if the destination supports it, or the source >> didn't set it: >> >> dest support source set ok? >> T T T >> T F T >> F T F >> F F T >> >> Instead, the old feature was defined as "this is true if the guest >> _cannot_ do X". For such a "negative" feature, migration is possible >> if the destination supports it, or the source sets it: >> >> dest support source set ok? >> T T T >> T F F >> F T T >> F F T >> >> However, the negotiated features are supposed to be the AND of the >> device- and driver-supported features. In the F/T case, the feature >> would be negotiated by the source as T, and become F when negotiated on >> the destination. >> >> Signed-off-by: Paolo Bonzini <pbonzini at redhat.com> > > Do you have any numbers showing how this new feature improves > performance? > We are able to batch quite a lot of pages in a single deflate > request - is the overhead measureable in practice?It's not only about better times, but also about better algorithms. I started writing this after seeing the Google fileballoon driver. For that implementation, the deflateq cannot be used at all. Paolo
Michael S. Tsirkin
2013-May-29 09:25 UTC
[PATCH v2 2/2] virtio-balloon spec: reintroduce "silent deflation" feature
On Wed, May 29, 2013 at 10:47:08AM +0200, Paolo Bonzini wrote:> Il 29/05/2013 09:49, Michael S. Tsirkin ha scritto: > > On Tue, May 28, 2013 at 07:40:18PM +0200, Paolo Bonzini wrote: > >> The original idea of the VIRTIO_BALLOON_F_MUST_TELL_HOST feature was to > >> let drivers skip usage of the deflate queue when leaking the balloon > >> ("silent deflation"). Guests may benefit from silent deflate by > >> aggressively inflating the balloon; they know that they will be able to > >> use ballooned pages without issuing a (blocking) request to the device. > >> > >> The previous patch redefined the feature to ensure correctness of the > >> operation when drivers do not correctly report deflation. This patch > >> adds back the optimization. > >> > >> The new feature bit is for the host to tell the drivers if silent > >> deflation is actually supported. The meaning of the feature bit is > >> reversed compared to the original, because the original meaning was > >> not safe against migration. > >> > >> For features to be safe against migration, they have to be defined as > >> "this is true if the guest _can_ do X". For such a "positive" feature, > >> migration is possible if the destination supports it, or the source > >> didn't set it: > >> > >> dest support source set ok? > >> T T T > >> T F T > >> F T F > >> F F T > >> > >> Instead, the old feature was defined as "this is true if the guest > >> _cannot_ do X". For such a "negative" feature, migration is possible > >> if the destination supports it, or the source sets it: > >> > >> dest support source set ok? > >> T T T > >> T F F > >> F T T > >> F F T > >> > >> However, the negotiated features are supposed to be the AND of the > >> device- and driver-supported features. In the F/T case, the feature > >> would be negotiated by the source as T, and become F when negotiated on > >> the destination. > >> > >> Signed-off-by: Paolo Bonzini <pbonzini at redhat.com> > > > > Do you have any numbers showing how this new feature improves > > performance? > > We are able to batch quite a lot of pages in a single deflate > > request - is the overhead measureable in practice? > > It's not only about better times, but also about better algorithms. I > started writing this after seeing the Google fileballoon driver. For > that implementation, the deflateq cannot be used at all. > > PaoloIt seems weak to claim the algorithm is better without trying it out in practice. I asked the fileballoon driver author whether it can use TELL_HOST or if that will be too much overhead. He said don't know. And in particular, I think a feature bit is a bad fit for deciding how page will be reclaimed. Whether you will want a page back soon can change over time depending on load etc. So if we can't just tell host first always, then I think we need to give guest control to do this per inflate command. -- MST
Maybe Matching Threads
- [PATCH v2 2/2] virtio-balloon spec: reintroduce "silent deflation" feature
- [PATCH v2 2/2] virtio-balloon spec: reintroduce "silent deflation" feature
- [PATCH] virtio-balloon spec: rework VIRTIO_BALLOON_F_MUST_TELL_HOST feature, support silent deflation
- [PATCH] virtio-balloon spec: rework VIRTIO_BALLOON_F_MUST_TELL_HOST feature, support silent deflation
- [PATCH] virtio-balloon spec: rework VIRTIO_BALLOON_F_MUST_TELL_HOST feature, support silent deflation