Xen devel - Oct 2013 - Improve performance between Network driver domain to guest OS

Hi all,
 I am Shikha Thakur pursuing post graduation from Pune University in
Computer Engineering.Recently, I started my research in Xen hypervisor for
my curriculum project.I am enjoying working with Xen. Being newbie i want
to get more familiar with Xen.

So, I request all please tell me that my idea for this research is valid or
not?

Research Topic::
I am working on "Improving Network I/O virtualization of Xen for Cloud
Computing."



Xen Hypervisor ::
 The driver domain based model of Xen for network I/O virtualization
exhibits poor networking performance.Two Experiments are carried out and
they are:-

1)Network Performance analysis has been carried out to analyze the
virtualization over- head and analyze the baseline reception, transmission
and forwarding performance of the VMs(shown in fig 1).

`[image: Inline image 4]


2)Experiment has carried out to find the hardware bottlenecks and it is
observe that the memory latency is the problem for costly driver domain -
virtual machine communication (shown in fig 2).



[image: Inline image 3]Solution::
  As Memory latency is the bottleneck,Can Packet aggregation be applied to
overcome this?

I would be awaiting for the reply.

Many Thanks.




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Hello, Shikha

On Sun, Oct 06, 2013 at 11:49:13AM +0530, shikha thakur
wrote:
> Hi all,
>  I am Shikha Thakur pursuing post graduation from Pune University in
> Computer Engineering.Recently, I started my research in Xen hypervisor for
> my curriculum project.I am enjoying working with Xen. Being newbie i want
> to get more familiar with Xen.
> 
> So, I request all please tell me that my idea for this research is valid or
> not?
> 
> Research Topic::
> I am working on "Improving Network I/O virtualization of Xen for Cloud
> Computing."
> 
> 
What''s your goal? Higher throughput? Lower latency? Do you have any
merit in mind?
> 
> Xen Hypervisor ::
>  The driver domain based model of Xen for network I/O virtualization
> exhibits poor networking performance.Two Experiments are carried out and
> they are:-
> 
> 1)Network Performance analysis has been carried out to analyze the
> virtualization over- head and analyze the baseline reception, transmission
> and forwarding performance of the VMs(shown in fig 1).
> 
> `[image: Inline image 4]
> 
> 
> 2)Experiment has carried out to find the hardware bottlenecks and it is
> observe that the memory latency is the problem for costly driver domain -
> virtual machine communication (shown in fig 2).
> 
Sorry the image quality is too low I am not able parse those graphs. But
if you have any specific questions regarding Xen network drivers in
Linux I will be happy to answer.
> 
> 
> [image: Inline image 3]Solution::
>   As Memory latency is the bottleneck,Can Packet aggregation be applied to
> overcome this?
> 
It depends. AIUI packet aggregation can introduce latency too. The only
way to prove whether it works is to implement it. :-)

Wei.

On Mon, Oct 7, 2013 at 4:57 PM, Wei Liu <wei.liu2@citrix.com>
wrote:
> Hello, Shikha
>
> On Sun, Oct 06, 2013 at 11:49:13AM +0530, shikha thakur wrote:
>> Hi all,
>>  I am Shikha Thakur pursuing post graduation from Pune University in
>> Computer Engineering.Recently, I started my research in Xen hypervisor
for
>> my curriculum project.I am enjoying working with Xen. Being newbie i
want
>> to get more familiar with Xen.
>>
>> So, I request all please tell me that my idea for this research is
valid or
>> not?
>>
>> Research Topic::
>> I am working on "Improving Network I/O virtualization of Xen for
Cloud
>> Computing."
>>
>>
>
> What''s your goal? Higher throughput? Lower latency? Do you have
any
> merit in mind?
>
>>
>> Xen Hypervisor ::
>>  The driver domain based model of Xen for network I/O virtualization
>> exhibits poor networking performance.Two Experiments are carried out
and
>> they are:-
>>
>> 1)Network Performance analysis has been carried out to analyze the
>> virtualization over- head and analyze the baseline reception,
transmission
>> and forwarding performance of the VMs(shown in fig 1).
>>
>> `[image: Inline image 4]
>>
>>
>> 2)Experiment has carried out to find the hardware bottlenecks and it is
>> observe that the memory latency is the problem for costly driver domain
-
>> virtual machine communication (shown in fig 2).
>>
>
> Sorry the image quality is too low I am not able parse those graphs.
+1.

 -George

Please don''t drop xen-devel.

On Sun, Oct 13, 2013 at 12:12:26PM +0530, shikha thakur
wrote:
> Hello,
> I apologized for the late reply.
> 
> >>What''s your goal? Higher throughput? Lower latency? Do you
have any
> merit in mind?
> 
> *Me: Basically higher throughput is my Aim.
>        **To increase the throughput, the paper i am referencing  analyzed
> the Xen hypervisor and got the memory latency problem.It has been overcome
> by packet aggregation.Thus the higher throughput is achieved.With the
> aggregation mechanism ,the throughput almost increases linearly  with the
> number of VM.(As Shown in Figure below).However, It introduced delay &
> jitter , hence i am trying to either eliminate delay & jitter or find
new
> way of increasing throughput.
> 
> *
> 
I skimmed the paper. "Memory latency" seems to refer to the latency
introduced by memory copy. And the solution in that paper might not work
quite well in the scenario when a VM is expecting low latency with small
packets.
> 
> >>It depends. AIUI packet aggregation can introduce latency too. The
only
> way to prove whether it works is to implement it.
> 
>  *Me:: It has been experimented and it is introducing delay and jitter.
> *
> *First thing i wanna know, Whether the  maximize delay and jitter is really
> a problem for higher throughput?
Sorry I don''t know. you''d better think of the system as a
whole.
Eliminating one single bottleneck might actually decrease performance.
> *
> *
> If yes, there are two method :-
> 1) Packet dimensioning tool which dynamically tune the packet
> aggregation.(Refer attached pdf)**
> 2) What i am planning to do is to imply adaptive jitter buffering algorithm
> to dynamically tune the packet aggregation*.
> 
> *If not of much trouble,Could anyone guide me which one of the above 2
> method is better for minimizing delay and jitter.*
> *
> 
Sorry I cannot give you a definitive answer, that''s beyond my
expertise.
I think the fun of doing research lies in the uncertainty and the thrill
of solving problems. If there''s definitive answer to this question,
somebody would have done it already, right?

The only viable way is to hack a prototype IMHO.

Wei.

Please don''t drop Xen-devel.

On Sat, Oct 26, 2013 at 11:45:45AM +0530, shikha thakur
wrote:
> Hello,
> I am not getting where to do coding for network I/O virtualization. Can you
> please give me some Documentation or any tutorial which can guide me for
> the same?
> 
A starting point would be Linux''s netback / netfront drivers. You can
find them in drivers/net/xen-netback/ and drivers/net/xen-netfront.c of
Linux source tree.

Wei.
> Many thanks,
> Shikha thakur
> 
> 
> On Thu, Oct 17, 2013 at 7:07 PM, shikha thakur <srthakur26@gmail.com>
wrote:
> 
> >
> >
> > ---------- Forwarded message ----------
> > From: Wei Liu <wei.liu2@citrix.com>
> > Date: Mon, Oct 14, 2013 at 3:59 PM
> > Subject: Re: [Xen-devel] Improve performance between Network driver
domain
> > to guest OS
> > To: shikha thakur <srthakur26@gmail.com>
> > Cc: Wei Liu <wei.liu2@citrix.com>, George Dunlap <
> > george.dunlap@eu.citrix.com>, Ritesh Malaiya
<ritesh.opensource@gmail.com>,
> > xen-devel@lists.xen.org
> >
> >
> > Please don''t drop xen-devel.
> >
> > On Sun, Oct 13, 2013 at 12:12:26PM +0530, shikha thakur wrote:
> > > Hello,
> > > I apologized for the late reply.
> > >
> > > >>What''s your goal? Higher throughput? Lower
latency? Do you have any
> > > merit in mind?
> > >
> > > *Me: Basically higher throughput is my Aim.
> > >        **To increase the throughput, the paper i am referencing 
analyzed
> > > the Xen hypervisor and got the memory latency problem.It has been
> > overcome
> > > by packet aggregation.Thus the higher throughput is achieved.With
the
> > > aggregation mechanism ,the throughput almost increases linearly 
with the
> > > number of VM.(As Shown in Figure below).However, It introduced
delay &
> > > jitter , hence i am trying to either eliminate delay & jitter
or find new
> > > way of increasing throughput.
> > >
> > > *
> > >
> >
> > I skimmed the paper. "Memory latency" seems to refer to the
latency
> > introduced by memory copy. And the solution in that paper might not
work
> > quite well in the scenario when a VM is expecting low latency with
small
> > packets.
> >
> > >
> > > >>It depends. AIUI packet aggregation can introduce latency
too. The only
> > > way to prove whether it works is to implement it.
> > >
> > >  *Me:: It has been experimented and it is introducing delay and
jitter.
> > > *
> > > *First thing i wanna know, Whether the  maximize delay and jitter
is
> > really
> > > a problem for higher throughput?
> >
> > Sorry I don''t know. you''d better think of the system
as a whole.
> > Eliminating one single bottleneck might actually decrease performance.
> >
> > > *
> > > *
> > > If yes, there are two method :-
> > > 1) Packet dimensioning tool which dynamically tune the packet
> > > aggregation.(Refer attached pdf)**
> > > 2) What i am planning to do is to imply adaptive jitter buffering
> > algorithm
> > > to dynamically tune the packet aggregation*.
> > >
> > > *If not of much trouble,Could anyone guide me which one of the
above 2
> > > method is better for minimizing delay and jitter.*
> > > *
> > >
> >
> > Sorry I cannot give you a definitive answer, that''s beyond my
expertise.
> > I think the fun of doing research lies in the uncertainty and the
thrill
> > of solving problems. If there''s definitive answer to this
question,
> > somebody would have done it already, right?
> >
> > The only viable way is to hack a prototype IMHO.
> >
> > Wei.
> >
> >