Xen devel - Mar 2008 - Status of "Out of SW-IOMMU space" problem?

Is the "dma: avoid unnecessarily SWIOTLB bounce buffering" patch
likely
to fix the "Out of SW-IOMMU space" issue with 3ware cards and Xen, or 
any other patches for that matter?

I know some people have had success updating their 3ware card''s
firmware
to latest, but that''s not easily achievable for our situation.  So, 
we''re stuck on 2.6.16.x for dom0 :(

Thanks,
-Chris


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

I''d be interested to know if it does. Unfortunately I have my doubts
since
the patch fixes a page-boundary straddling issue, and if that were the root
problem it''s unlikely to be affected by controller firmware updates.

 -- Keir

On 31/3/08 18:23, "Christopher S. Aker" <caker@theshore.net>
wrote:
> Is the "dma: avoid unnecessarily SWIOTLB bounce buffering" patch
likely
> to fix the "Out of SW-IOMMU space" issue with 3ware cards and
Xen, or
> any other patches for that matter?
> 
> I know some people have had success updating their 3ware card''s
firmware
> to latest, but that''s not easily achievable for our situation. 
So,
> we''re stuck on 2.6.16.x for dom0 :(
> 
> Thanks,
> -Chris
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

No, firmware updates doesn''t solve this issue.
At least not for us. I updated different 7000 and 8000 series controller
without any success.
But I''m also interested if there''s some patch around for this
''out of sw-iommu''
error ;)

Stephan


Keir Fraser schrieb:
> I''d be interested to know if it does. Unfortunately I have my
doubts since
> the patch fixes a page-boundary straddling issue, and if that were the root
> problem it''s unlikely to be affected by controller firmware
updates.
> 
>  -- Keir
> 
> On 31/3/08 18:23, "Christopher S. Aker"
<caker@theshore.net> wrote:
> 
>> Is the "dma: avoid unnecessarily SWIOTLB bounce buffering"
patch likely
>> to fix the "Out of SW-IOMMU space" issue with 3ware cards and
Xen, or
>> any other patches for that matter?
>>
>> I know some people have had success updating their 3ware
card''s firmware
>> to latest, but that''s not easily achievable for our situation.
So,
>> we''re stuck on 2.6.16.x for dom0 :(
>>
>> Thanks,
>> -Chris
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

-- 
Stephan Seitz
Senior System Administrator

*netz-haut* e.K.
multimediale kommunikation

zweierweg 22
97074 würzburg

fon: +49 931 2876247
fax: +49 931 2876248

web: www.netz-haut.de <http://www.netz-haut.de/>

registriergericht: amtsgericht würzburg, hra 5054



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Well then, you should try the patch you originally posted about. :-)

 -- Keir

On 1/4/08 22:04, "Stephan Seitz" <s.seitz@netz-haut.de> wrote:
> No, firmware updates doesn''t solve this issue.
> At least not for us. I updated different 7000 and 8000 series controller
> without any success.
> But I''m also interested if there''s some patch around for
this ''out of
> sw-iommu''
> error ;)
> 
> Stephan
> 
> 
> Keir Fraser schrieb:
>> I''d be interested to know if it does. Unfortunately I have my
doubts since
>> the patch fixes a page-boundary straddling issue, and if that were the
root
>> problem it''s unlikely to be affected by controller firmware
updates.
>> 
>>  -- Keir
>> 
>> On 31/3/08 18:23, "Christopher S. Aker"
<caker@theshore.net> wrote:
>> 
>>> Is the "dma: avoid unnecessarily SWIOTLB bounce
buffering" patch likely
>>> to fix the "Out of SW-IOMMU space" issue with 3ware cards
and Xen, or
>>> any other patches for that matter?
>>> 
>>> I know some people have had success updating their 3ware
card''s firmware
>>> to latest, but that''s not easily achievable for our
situation.  So,
>>> we''re stuck on 2.6.16.x for dom0 :(
>>> 
>>> Thanks,
>>> -Chris
>>> 
>>> 
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>> 
>> 
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Well, I have to add 2 eurocents on this issue ;)

I am struggling with this since months, but I could not do too much, 
having all servers in production. I eventually managed to setup a test 
server, so I hope I can progress on this.

As you say, latest firmware update does not help (to be correct, i have 
the *feeling* it took me more time to reproduce, but it crashed anyway).

So far, I have the following observations:
- Xen 3.1.0 and the associated kernel (2.6.18) works (this is what we 
have on all our production servers)
- Any newer version starting from Xen 3.1.1 (again each time with the 
matching kernel) show the "Out of SW-IOMMU space" problem

In all cases, I am using the same 3ware 3w-9xxx driver from 3ware, as 
the 9650SE is not supported in the stock kernel. So the issue is coming 
from ''elsewhere''.

I had a look in what happened between 3.1.0 and 3.1.1, and we have a 
handful of changes in linux-2.6-xen-sparse/arch/i386/kernel/swiotlb.c (5 
changesets to be precise)

I''ll try to spend some time in reverting these changes to see if it
helps...

--
Philippe

Stephan Seitz wrote:
> No, firmware updates doesn''t solve this issue.
> At least not for us. I updated different 7000 and 8000 series controller
> without any success.
> But I''m also interested if there''s some patch around for
this ''out of
> sw-iommu''
> error ;)
>
> Stephan
>
>
> Keir Fraser schrieb:
>> I''d be interested to know if it does. Unfortunately I have my
doubts
>> since
>> the patch fixes a page-boundary straddling issue, and if that were 
>> the root
>> problem it''s unlikely to be affected by controller firmware
updates.
>>
>>  -- Keir
>>
>> On 31/3/08 18:23, "Christopher S. Aker"
<caker@theshore.net> wrote:
>>
>>> Is the "dma: avoid unnecessarily SWIOTLB bounce
buffering" patch likely
>>> to fix the "Out of SW-IOMMU space" issue with 3ware cards
and Xen, or
>>> any other patches for that matter?
>>>
>>> I know some people have had success updating their 3ware
card''s
>>> firmware
>>> to latest, but that''s not easily achievable for our
situation.  So,
>>> we''re stuck on 2.6.16.x for dom0 :(
>>>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

I didn''t knew either -- thnx for this.

Unfortunately, it does not solve the SW-IOMMU space issue here -- ran my 
test suite and was able to reproduce.

--
Phil.

Stephan Seitz wrote:
> Hi,
>
> maybe everyone except me knows about this. I found a patch to the 3w-9xxx
> driver adressing this issue in the fedora 2.6.21-xen src.rpm
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Summary from the previous episodes: ;)

A) Xen 3.1.0 runs fine
B) Any newer version generates SW-IOMMU problem when Dom0 is under I/O 
stress.

C) Starting from there, I decided to build a Xen 3.1.3 / Linux 2.6.18.8 
kernel excluding all changes related to swiotlb.c.
That means: backout of changsets 15412, 15409, 15408, 15407 and 15405
This configuration runs fine as well!

Then, I re-applied changeset 15405, and the problem re-appears. So the 
issue is really linked to this changeset.
Looking at it more in detail, there is only one ''intrusive''
change there:

swiotlb.c

    575         if (address_needs_mapping(hwdev, dev_addr)) {

becomes

    575         if
    (range_straddles_page_boundary(page_to_pseudophys(sg->page)
    576                           + sg->offset, sg->length)
    577             || address_needs_mapping(hwdev, dev_addr)) {

And indeed, if I remove the test on
''range_straddles_page_boundary'', it
runs fine, if I leave it, I get the SW-IOMMU problem.

To understand how big the impact of the
''range_straddles_page_boundary''
test is, I added an else/printk and it gives:

    Apr 21 09:07:45 xenon kernel: SWIOTLB -
    range_straddles_page_boundary test bypassed
    Apr 21 09:07:58 xenon last message repeated 3629 times
    ...

so the impact is rather big: about 300 times per second...

This is almost as far as I can go on the topic. My knowledge on what''s 
really happening in that part of the code is somewhat limited.
So I appreciate any help on how to progress this further.

(Side note: not sure what''s happening here, looks like the mailing
lists
are broken...)

--
Phil.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 21/4/08 09:02, "Philippe Vanhaesendonck" <xen@e-bulles.be>
wrote:
> C) Starting from there, I decided to build a Xen 3.1.3 / Linux 2.6.18.8
> kernel excluding all changes related to swiotlb.c.
> That means: backout of changsets 15412, 15409, 15408, 15407 and 15405
> This configuration runs fine as well!
> 
> Then, I re-applied changeset 15405, and the problem re-appears. So the
> issue is really linked to this changeset.
> Looking at it more in detail, there is only one
''intrusive'' change there:
Can you please try the repository at
http://xenbits.xensource.com/xen-3.1-testing.hg ?

The changeset 15705 probably fixes the issue.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Please try with changeset 15705 from
http://xenbits.xensource.com/xen-3.1-testing.hg

 -- Keir

On 18/4/08 09:57, "Philippe Vanhaesendonck" <xen@e-bulles.be>
wrote:
> I didn''t knew either -- thnx for this.
> 
> Unfortunately, it does not solve the SW-IOMMU space issue here -- ran my
> test suite and was able to reproduce.
> 
> --
> Phil.
> 
> Stephan Seitz wrote:
>> Hi,
>> 
>> maybe everyone except me knows about this. I found a patch to the
3w-9xxx
>> driver adressing this issue in the fedora 2.6.21-xen src.rpm
>> 
>> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser wrote:
> Can you please try the repository at
> http://xenbits.xensource.com/xen-3.1-testing.hg ?
>
> The changeset 15705 probably fixes the issue.
>   
Keir,

I merged the 3.1.4-RC5 in my tree, recompiled the kernel and launched my 
test suite.
Tests are running for more than 18 hours now, without any problems (it 
usually crashes within the first 2 hours).

So I believe this problem is indeed fixed by changeset 15705 which is 
has a ''softer'' approach for the range_straddles_page_boundary
routine.

Than you very much

--
Philippe

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel