Linux Virtualization - Sep 2017 - [Qemu-devel] [RFC] Buffers/caches in VirtIO Balloon driver stats

On Tue, 29 Aug 2017 20:01:53 +0800
Wei Wang <wei.w.wang at intel.com> wrote:
> On 08/29/2017 05:57 PM, Stefan Hajnoczi wrote:
> > On Sun, Aug 27, 2017 at 11:30:33PM +0200, Tom?? Golembiovsk? wrote:  
> >> Hi,  
> > I have CCed the relevant mailing lists and people most recently
involved
> > in virtio-balloon discussions.  Hopefully this will help get the right
> > people to see your questions.
> >  
> >> We'd like to include information about reclaimable memory into
the
> >> statistics in VirtiO Balloon driver. Namely, we'd like to
include
> >> counters for bufferes and caches of Linux kernel. The patch itself
is
> >> pretty trivial -- no problem there. But before we do that I'd
like to
> >> get some input from the QEMU community.
> >>
> >> 1) Is there any reason not to have the stats there?  
> 
> Could you please share the usages of reclaimable memory via the stats?
I'll go ahead then and start sending patches. What would be the proper
course of action here? Send patch for the driver first, or send patch
for QEMU first or send both patches right away?

> >>
> >> 2) Considering the balloon device is multiplatform (Linux, BSD,
> >> Windows), is there a problem with including buffers/caches? These
seem
> >> to be specific to the Linux virtual memory subsystem. Of course,
other
> >> OSes could just report zeros. Are there some internal stats on
those
> >> OSes that could be filled in? I don't now if such or similar
statistic
> >> are available on BSD. On Windows only SystemCache stat looks like
> >> something relevant. Anyone familiar with those OSes has any
suggestions?
> >>
> >>       
> 
> One of the solutions that I'm thinking about is to make virtio 
> platform-ware.
This is not necessary. IIUC the driver does not need to send all the
stats. We can simply treat those stats as specific to Linux driver and
other drivers will not send them. Then QEMU will treat them as if zero
was reported.
> 
> That is, the device by default supports
> VIRTIO_F_LINUX,
> VIRTIO_F_WINDOWS,
> VIRTIO_F_BSD.
> 
> For the Linux driver, only VIRTIO_F_LINUX is supported, then we can
> have Linux specific driver implementations under that feature.

Since there were no suggestions for similar stats on other OSes I'd say
we treat the stats for buffers and caches as Linux specific. If there is
any need to send similar stats for other OSes we will add new stat fields
(specific for that particular OS).


Thanks,

    Tomas

-- 
Tom?? Golembiovsk? <tgolembi at redhat.com>

Resending. It did not reach virtio-dev the first time.

On Tue, 12 Sep 2017 16:29:47 +0200
Tom?? Golembiovsk? <tgolembi at redhat.com> wrote:
> On Tue, 29 Aug 2017 20:01:53 +0800
> Wei Wang <wei.w.wang at intel.com> wrote:
> 
> > On 08/29/2017 05:57 PM, Stefan Hajnoczi wrote:
> > > On Sun, Aug 27, 2017 at 11:30:33PM +0200, Tom?? Golembiovsk?
wrote:
> > >> Hi,  
> > > I have CCed the relevant mailing lists and people most recently
involved
> > > in virtio-balloon discussions.  Hopefully this will help get the
right
> > > people to see your questions.
> > >  
> > >> We'd like to include information about reclaimable memory
into the
> > >> statistics in VirtiO Balloon driver. Namely, we'd like to
include
> > >> counters for bufferes and caches of Linux kernel. The patch
itself is
> > >> pretty trivial -- no problem there. But before we do that
I'd like to
> > >> get some input from the QEMU community.
> > >>
> > >> 1) Is there any reason not to have the stats there?  
> > 
> > Could you please share the usages of reclaimable memory via the stats?
> 
> I'll go ahead then and start sending patches. What would be the proper
> course of action here? Send patch for the driver first, or send patch
> for QEMU first or send both patches right away?
> 
> 
> > >>
> > >> 2) Considering the balloon device is multiplatform (Linux,
BSD,
> > >> Windows), is there a problem with including buffers/caches?
These seem
> > >> to be specific to the Linux virtual memory subsystem. Of
course, other
> > >> OSes could just report zeros. Are there some internal stats
on those
> > >> OSes that could be filled in? I don't now if such or
similar statistic
> > >> are available on BSD. On Windows only SystemCache stat looks
like
> > >> something relevant. Anyone familiar with those OSes has any
suggestions?
> > >>
> > >>       
> > 
> > One of the solutions that I'm thinking about is to make virtio 
> > platform-ware.
> 
> This is not necessary. IIUC the driver does not need to send all the
> stats. We can simply treat those stats as specific to Linux driver and
> other drivers will not send them. Then QEMU will treat them as if zero
> was reported.
> 
> > 
> > That is, the device by default supports
> > VIRTIO_F_LINUX,
> > VIRTIO_F_WINDOWS,
> > VIRTIO_F_BSD.
> > 
> > For the Linux driver, only VIRTIO_F_LINUX is supported, then we can
> > have Linux specific driver implementations under that feature.
> 
> 
> Since there were no suggestions for similar stats on other OSes I'd say
> we treat the stats for buffers and caches as Linux specific. If there is
> any need to send similar stats for other OSes we will add new stat fields
> (specific for that particular OS).
> 
> 
> Thanks,
> 
>     Tomas
> 
> -- 
> Tom?? Golembiovsk? <tgolembi at redhat.com>

-- 
Tom?? Golembiovsk? <tgolembi at redhat.com>

On 09/12/2017 10:29 PM, Tom?? Golembiovsk? wrote:
> On Tue, 29 Aug 2017 20:01:53 +0800
> Wei Wang <wei.w.wang at intel.com> wrote:
>
>> On 08/29/2017 05:57 PM, Stefan Hajnoczi wrote:
>>> On Sun, Aug 27, 2017 at 11:30:33PM +0200, Tom?? Golembiovsk? wrote:
>>>> Hi,
>>> I have CCed the relevant mailing lists and people most recently
involved
>>> in virtio-balloon discussions.  Hopefully this will help get the
right
>>> people to see your questions.
>>>   
>>>> We'd like to include information about reclaimable memory
into the
>>>> statistics in VirtiO Balloon driver. Namely, we'd like to
include
>>>> counters for bufferes and caches of Linux kernel. The patch
itself is
>>>> pretty trivial -- no problem there. But before we do that
I'd like to
>>>> get some input from the QEMU community.
>>>>
>>>> 1) Is there any reason not to have the stats there?
>> Could you please share the usages of reclaimable memory via the stats?
> I'll go ahead then and start sending patches. What would be the proper
> course of action here? Send patch for the driver first, or send patch
> for QEMU first or send both patches right away?
If you have both ready, I think it would be fine to send them all.

>
>
>>>> 2) Considering the balloon device is multiplatform (Linux, BSD,
>>>> Windows), is there a problem with including buffers/caches?
These seem
>>>> to be specific to the Linux virtual memory subsystem. Of
course, other
>>>> OSes could just report zeros. Are there some internal stats on
those
>>>> OSes that could be filled in? I don't now if such or
similar statistic
>>>> are available on BSD. On Windows only SystemCache stat looks
like
>>>> something relevant. Anyone familiar with those OSes has any
suggestions?
>>>>
>>>>        
>> One of the solutions that I'm thinking about is to make virtio
>> platform-ware.
> This is not necessary. IIUC the driver does not need to send all the
> stats. We can simply treat those stats as specific to Linux driver and
> other drivers will not send them. Then QEMU will treat them as if zero
> was reported.
Just had a quick check, I think that should function well too.
QEMU will report -1 for the stats.

Best,
Wei