Chris Joelly
2008-Aug-04 12:07 UTC
[Xen-users] Out of SW-IOMMU space for 7222 bytes at device ...: swiotlb or dom0_mem?
Hello, i am experiencing some strange errors from the xen kernel, all related to SATA and SAS raid controllers, 3ware and LSI in my setup: 03:00.0 RAID bus controller: 3ware Inc 9650SE SATA-II RAID (rev 01) 08:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1064ET PCI-Express Fusion-MPT SAS (rev 04) Erros: Out of SW-IOMMU space for 7222 bytes at device ... i found somwhere on the internet that increasing dom0_mem and swiotlb should do the trick, but thats unfortunately not true for my setup ... can anybody tell whats this swiotlb value and how can i calculate a appropriate value? Chris -- "The greatest proof that intelligent life other that humans exists in the universe is that none of it has tried to contact us!" _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Markus Schuster
2008-Aug-04 14:15 UTC
[Xen-users] Re: Out of SW-IOMMU space for 7222 bytes at device ...: swiotlb or dom0_mem?
Chris Joelly wrote:> i am experiencing some strange errors from the xen kernel, all related > to SATA and SAS raid controllers, 3ware and LSI in my setup: > > 03:00.0 RAID bus controller: 3ware Inc 9650SE SATA-II RAID (rev 01) > 08:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1064ET > PCI-Express Fusion-MPT SAS (rev 04) > > Erros: > Out of SW-IOMMU space for 7222 bytes at device ...I had the same problem with a 3ware SATA raid controller and for me the trick was to use the kernel source available from http://xenbits.xensource.com/linux-2.6.18-xen.hg. Search xen-devel for my post from 2008-06-10. I give some more details there. Greetings, Markus _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Scott Garron
2008-Aug-04 16:44 UTC
Re: [Xen-users] Out of SW-IOMMU space for 7222 bytes at device ...: swiotlb or dom0_mem?
Chris Joelly wrote:> Erros: > Out of SW-IOMMU space for 7222 bytes at device ... > > i found somwhere on the internet that increasing dom0_mem and swiotlb > should do the trick, but thats unfortunately not true for my setup ...What version of Xen and the kernel are you running? This bug was supposedly fixed in April: http://lists.xensource.com/archives/html/xen-changelog/2008-04/msg00008.html ... but a similar (or identical) one appears to still have "NEW" status, posted on April 25th. http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1227 Regardless of how you set swiotlb, under high I/O load, this bug has the potential to completely eat your storage array (I found out the hard way). As far as I know, it only seems to affect the 3Ware cards on 64-bit systems, running a 64-bit kernel. 3Ware''s knowledge base says this about it: http://www.3ware.com/kb/article.aspx?id=15345 -- Scott Garron _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Stephan Seitz
2008-Aug-04 17:30 UTC
Re: [Xen-users] Out of SW-IOMMU space for 7222 bytes at device ...: swiotlb or dom0_mem?
Hi Chris, you might want to try the drivers mentioned here http://www.3ware.com/KB/article.aspx?id=15243&cNode=6I1C6S This *should* fix the issue for dom0. Anyway, as well as these drivers fixed it for me (3.2.1, 2.6.18.8-xen, 64bit w/ plenty of RAM), I''ld never ever take this productive. Continous kernel msgs kept me from trusting that beast. If you count manpower it'' might be cheaper to buy some areca controller instead. (http://www.areca.com.tw/products/pcix.htm) Sorry for this ad, but I found them as reliable as 3ware. Cheers, Stephan Chris Joelly schrieb:> Hello, > > i am experiencing some strange errors from the xen kernel, all related > to SATA and SAS raid controllers, 3ware and LSI in my setup: > > 03:00.0 RAID bus controller: 3ware Inc 9650SE SATA-II RAID (rev 01) > 08:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1064ET PCI-Express Fusion-MPT SAS (rev 04) > > Erros: > Out of SW-IOMMU space for 7222 bytes at device ... > > i found somwhere on the internet that increasing dom0_mem and swiotlb > should do the trick, but thats unfortunately not true for my setup ... > > can anybody tell whats this swiotlb value and how can i calculate a > appropriate value? > > Chris >-- Stephan Seitz Senior System Administrator *netz-haut* e.K. multimediale kommunikation zweierweg 22 97074 würzburg fon: +49 931 2876247 fax: +49 931 2876248 web: www.netz-haut.de <http://www.netz-haut.de/> registriergericht: amtsgericht würzburg, hra 5054 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Chiu, PCM (Peter)
2008-Aug-05 08:37 UTC
RE: [Xen-users] Re: Out of SW-IOMMU space for 7222 bytes at device ...:swiotlb or dom0_mem?
Interesting comments. Can you elaborate a bit more as to why the 2.6.18 kernel works? We have been hitting the same problem on the OpenSuse 10.3 with 2.6.22.17-0.1 (.x86_64-xen) as well as OpenSuse 11.0 with 2.6.25.5-1.1 (.x86_64-xen). Going back to an older release would mean reversing the OS installation... Peter -----Original Message----- From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Markus Schuster Sent: 04 August 2008 15:15 To: xen-users@lists.xensource.com Subject: [Xen-users] Re: Out of SW-IOMMU space for 7222 bytes at device ...:swiotlb or dom0_mem? Chris Joelly wrote:> i am experiencing some strange errors from the xen kernel, all related > to SATA and SAS raid controllers, 3ware and LSI in my setup: > > 03:00.0 RAID bus controller: 3ware Inc 9650SE SATA-II RAID (rev 01) > 08:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1064ET > PCI-Express Fusion-MPT SAS (rev 04) > > Erros: > Out of SW-IOMMU space for 7222 bytes at device ...I had the same problem with a 3ware SATA raid controller and for me the trick was to use the kernel source available from http://xenbits.xensource.com/linux-2.6.18-xen.hg. Search xen-devel for my post from 2008-06-10. I give some more details there. Greetings, Markus _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Markus Schuster
2008-Aug-05 09:26 UTC
[Xen-users] RE: Re: Out of SW-IOMMU space for 7222 bytes at device ...:swiotlb or dom0_mem?
Chiu, PCM (Peter) wrote:> Can you elaborate a bit more as to why the 2.6.18 kernel works?Well, I don''t know exactly why it works as I haven''t tried to understand the source code. I think this question should go to the developers. But as far as I know changeset 501 (http://xenbits.xensource.com/linux-2.6.18-xen.hg?rev/5486a234923d) contains the relevant change/fix to the source code. Maybe it''s possible to generarte a patch for your running kernel version out of this mercurial changeset? Greetings, Markus _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Chris Joelly
2008-Aug-25 21:12 UTC
Re: [Xen-users] Out of SW-IOMMU space for 7222 bytes at device ...: swiotlb or dom0_mem?
On Mon, Aug 04, 2008, Scott Garron wrote:> Chris Joelly wrote: >> Erros: >> Out of SW-IOMMU space for 7222 bytes at device ... >> >> i found somwhere on the internet that increasing dom0_mem and swiotlb >> should do the trick, but thats unfortunately not true for my setup ... > > What version of Xen and the kernel are you running? This bug was > supposedly fixed in April: > > http://lists.xensource.com/archives/html/xen-changelog/2008-04/msg00008.html > > ... but a similar (or identical) one appears to still have "NEW" status, > posted on April 25th. > > http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1227 > > Regardless of how you set swiotlb, under high I/O load, this bug > has the potential to completely eat your storage array (I found out the > hard way). > > As far as I know, it only seems to affect the 3Ware cards on 64-bit > systems, running a 64-bit kernel. > > 3Ware''s knowledge base says this about it: > > http://www.3ware.com/kb/article.aspx?id=15345i tried the following and got some success, at least till now no errors occurred: get the tarball for xen 3.1.4, compiled and installed it on Ubuntu server 8.04 64bit and configured Ubuntu''s dom0 kernel 2.6.24 to be run via 3.1.4 hypervisor. thanks, Chris _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users