Xen devel - Feb 2007 - [Xen-ia64-devel] [PATCH 0/3][IA64] Accelerate IDE PIO on HVM/IA64

Hi,

This set of patches remarkably accelerates IDE PIO on HVM/IA64.
I got a throughput of 2.11MB/sec in disk read performance.
Without it, it was only 64kB/sec.

I posted the prototype once. 
http://lists.xensource.com/archives/html/xen-devel/2006-12/msg00077.html

The basic idea is to add a buffering mechanism in a hypervisor.
I know this approach is not sophisticated. But there is no other
good way in IA64 which has no string instructions like x86''s.

This patchset is indispensable to support windows/ia64 on HVM
since installing windows and crash dumping is terribly slow.

There is no effect on x86 side. Please apply to xen-unstable/3.0.5.

Thanks,
Kouya


_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel

1/3 add a bufferring mechanism for IDE PIO in qemu.

Signed-off-by: Kouya Shimura <kouya@jp.fujitsu.com>



_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel

2/3 add one more page to the guest for IDE PIO.

Signed-off-by: Kouya Shimura <kouya@jp.fujitsu.com>



_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel

3/3 add a bufferring mechanism for IDE PIO in a hypervisor.

Signed-off-by: Kouya Shimura <kouya@jp.fujitsu.com>



_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel

On 27/2/07 09:34, "Kouya SHIMURA" <kouya@jp.fujitsu.com> wrote:
> The basic idea is to add a buffering mechanism in a hypervisor.
> I know this approach is not sophisticated. But there is no other
> good way in IA64 which has no string instructions like x86''s.
> 
> This patchset is indispensable to support windows/ia64 on HVM
> since installing windows and crash dumping is terribly slow.
Can you explain how this new code works? As I understand it the problem is
that each PIO instruction decoded by Xen and propagated to qemu only
transfers a single word of data. How does this new buffering mechanism work
around this?

 -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Keir,

For disk read, new buffering mechanism works as follows:

 1. Xen traps a single load instruction which performs PIO. 
    FYI, IA64 doesn''t have IN/OUT instructions and io ports are memory
    mapped.
+2. Xen checks whether the data is already buffered. 
+3. If the data is buffered and not the last one, 
    xen just returns the data to a guest.
 4. Otherwise, xen make an i/o request to qemu as usual.
    And the guest blocks.
 5. Qemu receives the i/o request and prepares the block of data that
    is buffered in qemu.
+6. Qemu copies the block to a shared page which I add newly as
    "buffered_pio_page". That is, it exposes the qemu''s
buffering
    data to xen.
 7. Qemu returns the single data on i/o request to xen as usual.
 8. Xen resumes the guest.

The above lines beginning with + are newly added mechanism. 
1,2,3 are almost repeated and transactions between xen and qemu
are drastically reduced.

For disk write, it is almost the same as the above explanation.
The difference is a direction of copy. And what I have to say is
that qemu does nothing until the qemu''s buffer becomes full. So
the transaction to qemu can be deferred.

Thanks,
Kouya

Keir Fraser writes:
 > On 27/2/07 09:34, "Kouya SHIMURA" <kouya@jp.fujitsu.com>
wrote:
 > 
 > > The basic idea is to add a buffering mechanism in a hypervisor.
 > > I know this approach is not sophisticated. But there is no other
 > > good way in IA64 which has no string instructions like
x86''s.
 > > 
 > > This patchset is indispensable to support windows/ia64 on HVM
 > > since installing windows and crash dumping is terribly slow.
 > 
 > Can you explain how this new code works? As I understand it the problem is
 > that each PIO instruction decoded by Xen and propagated to qemu only
 > transfers a single word of data. How does this new buffering mechanism
work
 > around this?
 > 
 >  -- Keir


_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel

On Tue, 2007-02-27 at 18:34 +0900, Kouya SHIMURA wrote:
> Hi,
> 
> This set of patches remarkably accelerates IDE PIO on HVM/IA64.
> I got a throughput of 2.11MB/sec in disk read performance.
> Without it, it was only 64kB/sec.
> 
> I posted the prototype once. 
> http://lists.xensource.com/archives/html/xen-devel/2006-12/msg00077.html
> 
> The basic idea is to add a buffering mechanism in a hypervisor.
> I know this approach is not sophisticated. But there is no other
> good way in IA64 which has no string instructions like x86''s.
   This seems like a pretty good performance increase for only a single
page of domain overhead in Xen.  My main concern would be that we
maintain correctness.  It is writing out the dirty buffer before the
domain shuts down, right?  I''m guessing the flush probably does that.
Looks like a bug here though:

+static inline void
+buffered_pio_write(IDEState *s, uint32_t addr, int size)
+{
+    struct pio_buffer *piobuf = piobuf_by_addr(addr);
+    uint32_t data_end;
+
+    if (!piobuf)
+        return;
+    buffered_pio_flush(piobuf);
+    data_end = s->data_end - s->data_ptr - size;
+    if (data_end <= 0)
+        data_end = 0;

   data_end is unsigned, so it will never be < 0.  Same problem on the
read func.  Thanks,

	Alex

-- 
Alex Williamson                             HP Open Source & Linux Org.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Alex,

Thanks for pointing out bugs. I''ll check the whole code again.

For domain shutting down, my patch does nothing.  But the original
qemu (or real IDE controller chip also?) is involved in the same
problem. The dirty data will be missing by sudden death of a guest. 
A guest OS has the responsibility to flush the dirty buffer.

I might have to consider save/restore of hvm. That is a future work.

Thanks,
Kouya

Alex Williamson writes:
 > On Tue, 2007-02-27 at 18:34 +0900, Kouya SHIMURA wrote:
 > > Hi,
 > > 
 > > This set of patches remarkably accelerates IDE PIO on HVM/IA64.
 > > I got a throughput of 2.11MB/sec in disk read performance.
 > > Without it, it was only 64kB/sec.
 > > 
 > > I posted the prototype once. 
 > >
http://lists.xensource.com/archives/html/xen-devel/2006-12/msg00077.html
 > > 
 > > The basic idea is to add a buffering mechanism in a hypervisor.
 > > I know this approach is not sophisticated. But there is no other
 > > good way in IA64 which has no string instructions like
x86''s.
 > 
 >    This seems like a pretty good performance increase for only a single
 > page of domain overhead in Xen.  My main concern would be that we
 > maintain correctness.  It is writing out the dirty buffer before the
 > domain shuts down, right?  I''m guessing the flush probably does
that.
 > Looks like a bug here though:
 > 
 > +static inline void
 > +buffered_pio_write(IDEState *s, uint32_t addr, int size)
 > +{
 > +    struct pio_buffer *piobuf = piobuf_by_addr(addr);
 > +    uint32_t data_end;
 > +
 > +    if (!piobuf)
 > +        return;
 > +    buffered_pio_flush(piobuf);
 > +    data_end = s->data_end - s->data_ptr - size;
 > +    if (data_end <= 0)
 > +        data_end = 0;
 > 
 >    data_end is unsigned, so it will never be < 0.  Same problem on the
 > read func.  Thanks,
 > 
 > 	Alex
 > 
 > -- 
 > Alex Williamson                             HP Open Source & Linux
Org.
 > 
 > 
 > _______________________________________________
 > Xen-devel mailing list
 > Xen-devel@lists.xensource.com
 > http://lists.xensource.com/xen-devel


_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel