Linux Virtualization - Oct 2016 - [RESEND PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

> On 10/20/2016 11:24 PM, Liang Li wrote:
> > Dave Hansen suggested a new scheme to encode the data structure,
> > because of additional complexity, it's not implemented in v3.
> 
> So, what do you want done with this patch set?  Do you want it applied
as-is
> so that we can introduce a new host/guest ABI that we must support until
> the end of time?  Then, we go back in a year or two and add the newer
> format that addresses the deficiencies that this ABI has with a third
version?
> 
Hi Dave & Michael,

I am working on Dave's new bitmap schema, I have finished the part of
getting the 'hybrid scheme bitmap'
and found the complexity was more than I expected.  The main issue is more
memory is required to
 save the 'hybrid scheme bitmap' beside that used to save the raw page
bitmap, for the worst case, the
memory required is 3 times than that in the previous implementation. 

I am wondering if I should continue, as an alternative solution, how about using
PFNs array when
inflating/deflating only a few pages? Things will be much more simple.


Thanks!
Liang

> > On 10/20/2016 11:24 PM, Liang Li wrote:
> > > Dave Hansen suggested a new scheme to encode the data structure,
> > > because of additional complexity, it's not implemented in v3.
> >
> > So, what do you want done with this patch set?  Do you want it applied
> > as-is so that we can introduce a new host/guest ABI that we must
> > support until the end of time?  Then, we go back in a year or two and
> > add the newer format that addresses the deficiencies that this ABI has
with
> a third version?
> >
> 
> Hi Dave & Michael,
> 
> I am working on Dave's new bitmap schema, I have finished the part of
> getting the 'hybrid scheme bitmap'
> and found the complexity was more than I expected.  The main issue is more
> memory is required to  save the 'hybrid scheme bitmap' beside that
used to
> save the raw page bitmap, for the worst case, the memory required is 3
> times than that in the previous implementation.
> 
3 times memory required is not accurate, please ignore this. sorry ...
The complexity is the point. 
> I am wondering if I should continue, as an alternative solution, how about
> using PFNs array when inflating/deflating only a few pages? Things will be
> much more simple.
> 
> 
> Thanks!
> Liang
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe at lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help at lists.oasis-open.org

On 10/26/2016 03:06 AM, Li, Liang Z wrote:
> I am working on Dave's new bitmap schema, I have finished the part of
> getting the 'hybrid scheme bitmap' and found the complexity was
more
> than I expected. The main issue is more memory is required to save
> the 'hybrid scheme bitmap' beside that used to save the raw page
> bitmap, for the worst case, the memory required is 3 times than that
> in the previous implementation.
Really?  Could you please describe the scenario where this occurs?
> I am wondering if I should continue, as an alternative solution, how about
using PFNs array when
> inflating/deflating only a few pages? Things will be much more
> simple.
Yes, using pfn lists is more efficient than using bitmaps for sparse
bitmaps.  Yes, there will be cases where it is preferable to just use
pfn lists vs. any kind of bitmap.

But, what does it matter?  At least with your current scheme where we go
out and collect get_unused_pages(), we do the allocation up front.  The
space efficiency doesn't matter at all for small sizes since we do the
constant-size allocation *anyway*.

I'm also pretty sure you can pack the pfn and page order into a single
64-bit word and have no bitmap for a given record.  That would make it
pack just as well as the old pfns alone.  Right?

On 10/26/2016 03:13 AM, Li, Liang Z wrote:
> 3 times memory required is not accurate, please ignore this. sorry ...
> The complexity is the point. 
What is making it so complex?  Can you describe the problems?

> On 10/26/2016 03:06 AM, Li, Liang Z wrote:
> > I am working on Dave's new bitmap schema, I have finished the part
of
> > getting the 'hybrid scheme bitmap' and found the complexity
was more
> > than I expected. The main issue is more memory is required to save the
> > 'hybrid scheme bitmap' beside that used to save the raw page
bitmap,
> > for the worst case, the memory required is 3 times than that in the
> > previous implementation.
> 
> Really?  Could you please describe the scenario where this occurs?
> > I am wondering if I should continue, as an alternative solution, how
> > about using PFNs array when inflating/deflating only a few pages?
> > Things will be much more simple.
> 
> Yes, using pfn lists is more efficient than using bitmaps for sparse
bitmaps.
> Yes, there will be cases where it is preferable to just use pfn lists vs.
any kind
> of bitmap.
> 
> But, what does it matter?  At least with your current scheme where we go
> out and collect get_unused_pages(), we do the allocation up front.  The
> space efficiency doesn't matter at all for small sizes since we do the
constant-
> size allocation *anyway*.
> 
> I'm also pretty sure you can pack the pfn and page order into a single
64-bit
> word and have no bitmap for a given record.  That would make it pack just
as
> well as the old pfns alone.  Right?
Yes, thanks for reminding, I am using 128 bit now, I will change it to 64 bit.
Let me finish the v4 first.

Thanks!
Liang