Linux Virtualization - Mar 2016 - [Qemu-devel] [RFC qemu 0/4] A PV solution for live migration optimization

* Paolo Bonzini (pbonzini at redhat.com) wrote:
> 
> 
> On 04/03/2016 15:26, Li, Liang Z wrote:
> >> > 
> >> > The memory usage will keep increasing due to ever growing
caches, etc, so
> >> > you'll be left with very little free memory fairly soon.
> >> > 
> > I don't think so.
> > 
> 
> Roman is right.  For example, here I am looking at a 64 GB (physical)
> machine which was booted about 30 minutes ago, and which is running
> disk-heavy workloads (installing VMs).
> 
> Since I have started writing this email (2 minutes?), the amount of free
> memory has already gone down from 37 GB to 33 GB.  I expect that by the
> time I have finished running the workload, in two hours, it will not
> have any free memory.
But what about a VM sitting idle, or that just has more RAM assigned to it
than is currently using.
 I've got a host here that's been up for 46 days and has been doing some
heavy VM debugging a few days ago, but today:

# free -m
              total        used        free      shared  buff/cache   available
Mem:          96536        1146       44834         184       50555       94735

I very rarely use all it's RAM, so it's got a big chunk of free RAM, and
yes
it's got a big chunk of cache as well.

Dave
> 
> Paolo
--
Dr. David Alan Gilbert / dgilbert at redhat.com / Manchester, UK

> > On 04/03/2016 15:26, Li, Liang Z wrote:
> > >> >
> > >> > The memory usage will keep increasing due to ever
growing caches,
> > >> > etc, so you'll be left with very little free memory
fairly soon.
> > >> >
> > > I don't think so.
> > >
> >
> > Roman is right.  For example, here I am looking at a 64 GB (physical)
> > machine which was booted about 30 minutes ago, and which is running
> > disk-heavy workloads (installing VMs).
> >
> > Since I have started writing this email (2 minutes?), the amount of
> > free memory has already gone down from 37 GB to 33 GB.  I expect that
> > by the time I have finished running the workload, in two hours, it
> > will not have any free memory.
> 
> But what about a VM sitting idle, or that just has more RAM assigned to it
> than is currently using.
>  I've got a host here that's been up for 46 days and has been doing
some
> heavy VM debugging a few days ago, but today:
> 
> # free -m
>               total        used        free      shared  buff/cache  
available
> Mem:          96536        1146       44834         184       50555      
94735
> 
> I very rarely use all it's RAM, so it's got a big chunk of free
RAM, and yes it's
> got a big chunk of cache as well.
> 
> Dave
> 
> >
> > Paolo
I begin to realize Roman's opinions. The PV solution can't handle the
cache memory while inflating balloon could.
Inflating balloon so as to skipping the cache memory is no good for guest's
performance.

How much of the free memory in the guest depends on the workload in the VM  and
the time VM has already run
before live migration. Even the memory usage will keep increasing due to ever
growing caches, but we don't know
when the live migration will happen, assuming there are no or very little free
pages in the guest is not quite right.

The advantage of the pv solution is the smaller performance impact, comparing
with inflating the balloon.

Liang

On Fri, Mar 04, 2016 at 06:51:21PM +0000, Dr. David Alan Gilbert
wrote:
> * Paolo Bonzini (pbonzini at redhat.com) wrote:
> > 
> > 
> > On 04/03/2016 15:26, Li, Liang Z wrote:
> > >> > 
> > >> > The memory usage will keep increasing due to ever
growing caches, etc, so
> > >> > you'll be left with very little free memory fairly
soon.
> > >> > 
> > > I don't think so.
> > > 
> > 
> > Roman is right.  For example, here I am looking at a 64 GB (physical)
> > machine which was booted about 30 minutes ago, and which is running
> > disk-heavy workloads (installing VMs).
> > 
> > Since I have started writing this email (2 minutes?), the amount of
free
> > memory has already gone down from 37 GB to 33 GB.  I expect that by
the
> > time I have finished running the workload, in two hours, it will not
> > have any free memory.
> 
> But what about a VM sitting idle, or that just has more RAM assigned to it
> than is currently using.
>  I've got a host here that's been up for 46 days and has been doing
some
> heavy VM debugging a few days ago, but today:
> 
> # free -m
>               total        used        free      shared  buff/cache  
available
> Mem:          96536        1146       44834         184       50555      
94735
> 
> I very rarely use all it's RAM, so it's got a big chunk of free
RAM, and yes
> it's got a big chunk of cache as well.
One of the promises of virtualization is better resource utilization.
People tend to avoid purchasing VMs so much oversized that they never
touch a significant amount of their RAM.  (Well, at least this is how
things stand in hosting market; I guess enterprize market is similar in
this regard).

That said, I'm not at all opposed to optimizing the migration of free
memory; what I'm trying to say is that creating brand new infrastructure
specifically for that case doesn't look justified when the existing one
can cover it in addition to much more common scenarios.

Roman.

> On Fri, Mar 04, 2016 at 06:51:21PM +0000, Dr. David Alan Gilbert wrote:
> > * Paolo Bonzini (pbonzini at redhat.com) wrote:
> > >
> > >
> > > On 04/03/2016 15:26, Li, Liang Z wrote:
> > > >> >
> > > >> > The memory usage will keep increasing due to ever
growing
> > > >> > caches, etc, so you'll be left with very little
free memory fairly soon.
> > > >> >
> > > > I don't think so.
> > > >
> > >
> > > Roman is right.  For example, here I am looking at a 64 GB
> > > (physical) machine which was booted about 30 minutes ago, and
which
> > > is running disk-heavy workloads (installing VMs).
> > >
> > > Since I have started writing this email (2 minutes?), the amount
of
> > > free memory has already gone down from 37 GB to 33 GB.  I expect
> > > that by the time I have finished running the workload, in two
hours,
> > > it will not have any free memory.
> >
> > But what about a VM sitting idle, or that just has more RAM assigned
> > to it than is currently using.
> >  I've got a host here that's been up for 46 days and has been
doing
> > some heavy VM debugging a few days ago, but today:
> >
> > # free -m
> >               total        used        free      shared  buff/cache  
available
> > Mem:          96536        1146       44834         184       50555   
94735
> >
> > I very rarely use all it's RAM, so it's got a big chunk of
free RAM,
> > and yes it's got a big chunk of cache as well.
> 
> One of the promises of virtualization is better resource utilization.
> People tend to avoid purchasing VMs so much oversized that they never
> touch a significant amount of their RAM.  (Well, at least this is how
things
> stand in hosting market; I guess enterprize market is similar in this
regard).
> 
> That said, I'm not at all opposed to optimizing the migration of free
memory;
> what I'm trying to say is that creating brand new infrastructure
specifically for
> that case doesn't look justified when the existing one can cover it in
addition
> to much more common scenarios.
> 
> Roman.
Even the existing one can cover more common scenarios, but it has performance
issue.
that's why I create a new one.

Liang