Xen devel - Sep 2010 - Re: Memory de-duplication (Persistent pages)

Sir,

We are pretty clear with the internal working of tmem.
Now as "ephemeral" pages are already de-duplicated in current
implementation .

1>>So, we propose to do sharing of "persistent" pages also .
2>>We also propose n/w bipass manager for "tmem" as its not
existing currently.

Will it be really useful ?
If yes, it will be implemented within stipulated time frame.

Thank you.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

In my opinion, both could be useful projects.
> -----Original Message-----
> From: Kaustubh Kabra [mailto:kaustubhwise@gmail.com]
> Sent: Saturday, September 04, 2010 1:02 PM
> To: Xen-devel
> Cc: Dan Magenheimer
> Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)
> 
> Sir,
> 
> We are pretty clear with the internal working of tmem.
> Now as "ephemeral" pages are already de-duplicated in current
> implementation .
> 
> 1>>So, we propose to do sharing of "persistent" pages also
.
> 2>>We also propose n/w bipass manager for "tmem" as its not
existing
> currently.
> 
> Will it be really useful ?
> If yes, it will be implemented within stipulated time frame.
> 
> Thank you.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

To solve memory deduplication in Xen using Gpu :

Current problem :
If Memory deduplication is done on Cpu ,a lot of memory is used for data
structures and lot of computation time is wasted in finding duplicate page
and becomes waste, if duplicate page is not found.

Proposed GPU solution: Let Gpu perform seraching for a shared page using
multiple threads in parallel,improving searching operation and save data
structures on GPU device memory ,which is possible. Communication between
Cpu and Gpu can be done using inter processor synchronization which is also
possible. Cpu process will run as a daemon that will invoke gpu process when
ever deduplication is scheduled / if performance is good enough then for
every page written in memory.

Correct me , if anything is wrong.

On Mon, Sep 6, 2010 at 11:05 PM, Dan Magenheimer
<dan.magenheimer@oracle.com
> wrote:
> In my opinion, both could be useful projects.
>
> > -----Original Message-----
> > From: Kaustubh Kabra [mailto:kaustubhwise@gmail.com]
> > Sent: Saturday, September 04, 2010 1:02 PM
> > To: Xen-devel
> > Cc: Dan Magenheimer
> > Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)
> >
> > Sir,
> >
> > We are pretty clear with the internal working of tmem.
> > Now as "ephemeral" pages are already de-duplicated in
current
> > implementation .
> >
> > 1>>So, we propose to do sharing of "persistent" pages
also .
> > 2>>We also propose n/w bipass manager for "tmem" as
its not existing
> > currently.
> >
> > Will it be really useful ?
> > If yes, it will be implemented within stipulated time frame.
> >
> > Thank you.
>


-- 
*Kaustubh Kabra
http://www.kaustubhwise.000a.biz*


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

This is an interesting idea, but  (1) there is no "standard " GPU so I
don''t think it will be possible to write Xen code generically for this;
(2) most of the overhead for searching will be in moving data from RAM to
cache... the CPU overhead is probably a small fraction so a GPU may not help
much (unless it has a much wider bus to RAM); (3) in tmem''s current
deduplication implementation, the search is done synchronously on every
put_page... I don''t know if this will work well asynchronously due to
locking, existing data structures, etc.


Dan

 

From: Kaustubh Kabra [mailto:kaustubhwise@gmail.com] 
Sent: Wednesday, September 15, 2010 1:09 PM
To: Xen-devel
Cc: Dan Magenheimer
Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)

 

To solve memory deduplication in Xen using Gpu :

Current problem :
If Memory deduplication is done on Cpu ,a lot of memory is used for data
structures and lot of computation time is wasted in finding duplicate page and
becomes waste, if duplicate page is not found.

Proposed GPU solution: Let Gpu perform seraching for a shared page using
multiple threads in parallel,improving searching operation and save data
structures on GPU device memory ,which is possible. Communication between Cpu
and Gpu can be done using inter processor synchronization which is also
possible. Cpu process will run as a daemon that will invoke gpu process when
ever deduplication is scheduled / if performance is good enough then for every
page written in memory.

Correct me , if anything is wrong.

On Mon, Sep 6, 2010 at 11:05 PM, Dan Magenheimer <HYPERLINK
"mailto:dan.magenheimer@oracle.com"dan.magenheimer@oracle.com>
wrote:

In my opinion, both could be useful projects.

> -----Original Message-----
> From: Kaustubh Kabra [mailto:HYPERLINK
"mailto:kaustubhwise@gmail.com"kaustubhwise@gmail.com]
> Sent: Saturday, September 04, 2010 1:02 PM
> To: Xen-devel
> Cc: Dan Magenheimer
> Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)
>
> Sir,
>
> We are pretty clear with the internal working of tmem.
> Now as "ephemeral" pages are already de-duplicated in current
> implementation .
>
> 1>>So, we propose to do sharing of "persistent" pages also
.
> 2>>We also propose n/w bipass manager for "tmem" as its not
existing
> currently.
>
> Will it be really useful ?
> If yes, it will be implemented within stipulated time frame.
>
> Thank you.



-- 
Kaustubh Kabra
http://www.kaustubhwise.000a.biz




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, Sep 16, 2010 at 09:22:29AM -0700, Dan Magenheimer
wrote:
>    This is an interesting idea, but  (1) there is no "standard "
GPU so I
>    don''t think it will be possible to write Xen code generically
for this;
>
How about things like OpenCL ? 

-- Pasi
>    (2) most of the overhead for searching will be in moving data from RAM
to
>    cache... the CPU overhead is probably a small fraction so a GPU may not
>    help much (unless it has a much wider bus to RAM); (3) in
tmem''s current
>    deduplication implementation, the search is done synchronously on every
>    put_page... I don''t know if this will work well asynchronously
due to
>    locking, existing data structures, etc.
> 
>    Dan
> 
> 
> 
>    From: Kaustubh Kabra [mailto:kaustubhwise@gmail.com]
>    Sent: Wednesday, September 15, 2010 1:09 PM
>    To: Xen-devel
>    Cc: Dan Magenheimer
>    Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)
> 
> 
> 
>    To solve memory deduplication in Xen using Gpu :
> 
>    Current problem :
>    If Memory deduplication is done on Cpu ,a lot of memory is used for data
>    structures and lot of computation time is wasted in finding duplicate
page
>    and becomes waste, if duplicate page is not found.
> 
>    Proposed GPU solution: Let Gpu perform seraching for a shared page using
>    multiple threads in parallel,improving searching operation and save data
>    structures on GPU device memory ,which is possible. Communication
between
>    Cpu and Gpu can be done using inter processor synchronization which is
>    also possible. Cpu process will run as a daemon that will invoke gpu
>    process when ever deduplication is scheduled / if performance is good
>    enough then for every page written in memory.
> 
>    Correct me , if anything is wrong.
> 
>    On Mon, Sep 6, 2010 at 11:05 PM, Dan Magenheimer
>    <[1]dan.magenheimer@oracle.com> wrote:
> 
>    In my opinion, both could be useful projects.
> 
>    > -----Original Message-----
>    > From: Kaustubh Kabra [mailto:[2]kaustubhwise@gmail.com]
>    > Sent: Saturday, September 04, 2010 1:02 PM
>    > To: Xen-devel
>    > Cc: Dan Magenheimer
>    > Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)
>    >
> 
>    > Sir,
>    >
>    > We are pretty clear with the internal working of tmem.
>    > Now as "ephemeral" pages are already de-duplicated in
current
>    > implementation .
>    >
>    > 1>>So, we propose to do sharing of "persistent"
pages also .
>    > 2>>We also propose n/w bipass manager for "tmem" as
its not existing
>    > currently.
>    >
>    > Will it be really useful ?
>    > If yes, it will be implemented within stipulated time frame.
>    >
>    > Thank you.
> 
>    --
>    Kaustubh Kabra
>    [3]http://www.kaustubhwise.000a.biz
> 
> References
> 
>    Visible links
>    1. mailto:dan.magenheimer@oracle.com
>    2. mailto:kaustubhwise@gmail.com
>    3. http://www.kaustubhwise.000a.biz/
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> From: Pasi Kärkkäinen [mailto:pasik@iki.fi]
> Sent: Thursday, September 16, 2010 12:24 PM
> To: Dan Magenheimer
> Cc: Kaustubh Kabra; Xen-devel
> Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)
> 
> On Thu, Sep 16, 2010 at 09:22:29AM -0700, Dan Magenheimer wrote:
> >    This is an interesting idea, but  (1) there is no "standard
" GPU
> so I
> >    don''t think it will be possible to write Xen code
generically for
> this;
> >
> 
> How about things like OpenCL ?
While I know nothing about OpenCL other than a quick read
in wikipedia, I suspect that putting an OpenCL abstraction
layer in the hypervisor is not going to be popular with
Xen developers. :-)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, Sep 16, 2010 at 12:34:17PM -0700, Dan Magenheimer
wrote:
> > From: Pasi Kärkkäinen [mailto:pasik@iki.fi]
> > Sent: Thursday, September 16, 2010 12:24 PM
> > To: Dan Magenheimer
> > Cc: Kaustubh Kabra; Xen-devel
> > Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)
> > 
> > On Thu, Sep 16, 2010 at 09:22:29AM -0700, Dan Magenheimer wrote:
> > >    This is an interesting idea, but  (1) there is no
"standard " GPU
> > so I
> > >    don''t think it will be possible to write Xen code
generically for
> > this;
> > >
> > 
> > How about things like OpenCL ?
> 
> While I know nothing about OpenCL other than a quick read
> in wikipedia, I suspect that putting an OpenCL abstraction
> layer in the hypervisor is not going to be popular with
> Xen developers. :-)
>
Hehe, yeah, I can believe that :) OpenCL gives GPU abstraction
for non-graphics apps.. that''s why I mentioned it.

"Xen hypervisor 5.0 requires a GPU with at least 4 GB of memory 
with optimized OpenCL drivers to support memory de-duplication". 

Maybe not ;)

-- Pasi


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

At 20:34 +0100 on 16 Sep (1284669257), Dan Magenheimer
wrote:
> > How about things like OpenCL ?
> 
> While I know nothing about OpenCL other than a quick read
> in wikipedia, I suspect that putting an OpenCL abstraction
> layer in the hypervisor is not going to be popular with
> Xen developers. :-)
There''s nothing to say that the GPU code has to live in the hypervisor.
A tools domain that was already scanning for duplicates in other
domains'' memory (or for that matter in block traffic) could use
whatever
GPU resources it liked.

Even in the tmem case a domain could be given read-only access to the
tmem pool (er, if IOMMUs support such a thing, which I don''t recall off
the top of my head) and supply hints to the hypervisor.  That way the
hypervisor would still be scanning with the CPU but only on pages where
it was likely to succeed.

Of course there''s still a question of what the memory traffic does to
performance elsewhere (and if GPU cores are going to share L3 cache with
CPUs that could make a new bottleneck) but that''s not to say it
wouldn''t
be a win.

Cheers,

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> > While I know nothing about OpenCL other than a quick read
> > in wikipedia, I suspect that putting an OpenCL abstraction
> > layer in the hypervisor is not going to be popular with
> > Xen developers. :-)
> 
> There''s nothing to say that the GPU code has to live in the
hypervisor.
> A tools domain that was already scanning for duplicates in other
> domains'' memory (or for that matter in block traffic) could use
> whatever
> GPU resources it liked.
> 
> Even in the tmem case a domain could be given read-only access to the
> tmem pool (er, if IOMMUs support such a thing, which I don''t
recall off
> the top of my head) and supply hints to the hypervisor.  That way the
> hypervisor would still be scanning with the CPU but only on pages where
> it was likely to succeed.
While I agree this is not impossible, tmem is designed to
be highly concurrent and the data structures manage potentially
millions of pages with thousands of pages put/got per vcpu
per domain per second.  I think just trying to deal with the
locking from a tools domain would negate any performance gain
from a GPU.  And, in any case, the overhead of tmem deduplication
is unlikely to be large enough (except possibly in contrived
cases) to be worth the work.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel