Are the testing kernels (kernel-4.9.37-29.el7 and kernel-4.9.37-29.el6, with the one config file change) working for everyone: (turn off: CONFIG_IO_STRICT_DEVMEM) If we don't hear any negative comments by Wednesday July 19th, 2017 then we are going to push those to updates as they solve iscsi issues with some hardware and don't seem to impact anything else based on limited testing. BTW .. to test, edit the xen repo config file in /etc/yum.repos.d/ and turn on the testing repository .. or yum --enablerepo=centos-virt-xen-testing upgrade kernel* Thanks, Johnny Hughes -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: OpenPGP digital signature URL: <http://lists.centos.org/pipermail/centos-virt/attachments/20170717/395b2a99/attachment-0002.sig>
Hello, is this kernel usable also for KVM or is only for XEN? S pozdravem Kristi?n Feldsam Tel.: +420 773 303 353, +421 944 137 535 E-mail.: support at feldhost.cz www.feldhost.cz - FeldHost? ? profesion?ln? hostingov? a serverov? slu?by za adekv?tn? ceny. FELDSAM s.r.o. V rohu 434/3 Praha 4 ? Libu?, PS? 142 00 I?: 290 60 958, DI?: CZ290 60 958 C 200350 veden? u M?stsk?ho soudu v Praze Banka: Fio banka a.s. ??slo ??tu: 2400330446/2010 BIC: FIOBCZPPXX IBAN: CZ82 2010 0000 0024 0033 0446> On 17 Jul 2017, at 16:45, Johnny Hughes <johnny at centos.org> wrote: > > Are the testing kernels (kernel-4.9.37-29.el7 and kernel-4.9.37-29.el6, > with the one config file change) working for everyone: > > (turn off: CONFIG_IO_STRICT_DEVMEM) > > If we don't hear any negative comments by Wednesday July 19th, 2017 then > we are going to push those to updates as they solve iscsi issues with > some hardware and don't seem to impact anything else based on limited > testing. > > BTW .. to test, edit the xen repo config file in /etc/yum.repos.d/ and > turn on the testing repository .. or > > yum --enablerepo=centos-virt-xen-testing upgrade kernel* > > Thanks, > Johnny Hughes > > _______________________________________________ > CentOS-virt mailing list > CentOS-virt at centos.org > https://lists.centos.org/mailman/listinfo/centos-virt-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.centos.org/pipermail/centos-virt/attachments/20170717/d35c7251/attachment-0002.html>
On 07/17/2017 09:47 AM, Kristi?n Feldsam wrote:> Hello, is this kernel usable also for KVM or is only for XEN?This kernel is intended for the Xen repos. None of us are testing it with KVM to my knowledge, but it may work. The KVM-related virt SIG repos don't include a custom kernel. This kernel is tracking an upstream LTS kernel and building for Xen specific functionality. Personally, I would stick with the base kernels for CentOS as they're intended to run KVM and are maintained longer than upstream LTS kernels. -- Kevin Stange Chief Technology Officer Steadfast | Managed Infrastructure, Datacenter and Cloud Services 800 S Wells, Suite 190 | Chicago, IL 60607 312.602.2689 X203 | Fax: 312.602.2688 kevin at steadfast.net | www.steadfast.net
On 07/17/2017 09:47 AM, Kristi?n Feldsam wrote:> Hello, is this kernel usable also for KVM or is only for XEN? > >It certainly will work with KVM (I use it on my KVM test server and my xen test server. But a better kernel probably for KVm is here: http://mirror.centos.org/altarch/7/experimental/x86_64/>> On 17 Jul 2017, at 16:45, Johnny Hughes <johnny at centos.org >> <mailto:johnny at centos.org>> wrote: >> >> Are the testing kernels (kernel-4.9.37-29.el7 and kernel-4.9.37-29.el6, >> with the one config file change) working for everyone: >> >> (turn off: CONFIG_IO_STRICT_DEVMEM) >> >> If we don't hear any negative comments by Wednesday July 19th, 2017 then >> we are going to push those to updates as they solve iscsi issues with >> some hardware and don't seem to impact anything else based on limited >> testing. >> >> BTW .. to test, edit the xen repo config file in /etc/yum.repos.d/ and >> turn on the testing repository .. or >> >> yum --enablerepo=centos-virt-xen-testing upgrade kernel* >>-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: OpenPGP digital signature URL: <http://lists.centos.org/pipermail/centos-virt/attachments/20170717/9e503bc7/attachment-0002.sig>
On Mon, 17 Jul 2017, Johnny Hughes wrote:> Are the testing kernels (kernel-4.9.37-29.el7 and kernel-4.9.37-29.el6, > with the one config file change) working for everyone: > > (turn off: CONFIG_IO_STRICT_DEVMEM)Hello. Maybe it's not the most appropriate thread or time, but I have been signalling it before: 4.9.* kernels do not work well for me any more (and for other people neither, as I know). Last stable kernel was 4.9.13-22. Since 4.9.25-26 I do often get: on 3 supermicro servers (different generations): - memory allocation errors on Dom0 and corresponding lost lost page writes due to buffer I/O error on PV guests - after such memory allocation error od dom0 I have spotted also: - NFS client hangups on guests (server not responding, still trying => server OK) - iptables lockups on PV guest reboot on 1 supermicro server: - memory allocation errors on Dom0 and SATA lockups (many, if not SATA channels at - once): exception Emask 0x0 SAct 0x20 SErr 0x0 action 0x6 frozen hard resetting link failed to IDENTIFY (I/O error, err_mask=0x4) then: blk_update_request: I/O error, dev sd., sector .... All of these machines have been tested with memtest, no detected memory problems. No such things occur, when I boot 4.9.13-22 Most of my guests are centos 6 x86_64, bridged. Do anyone had such problems, dealt with it somehow? Since spotting these errors I have done many tests, compiled and tested to point out single code change (kernel version, patch) - no conclusions yet. But one has changed much between 4.9.13 and 4.9.25: kernel size and configuration. 4.9.13 size was 6MB and 4.9.24 is 7.1MB. Many modules have been compiled into kernel, here is shortened, but significant list: - iptables (NETFILTER_XTABLES, IP_NF_FILTER, IP_NF_TARGET_REJECT) - SATA_AHCI - ATA_AHCI (PATA, what a heck?) - FBDEV_FRONTEND - HID_MAGICKMOUSE - HID_NTRIG - USB_XHCI - INTEL_SMARTCONNECT Do we really need these compiled into dom0 kernel? I assume, that the biggest change in size is due to yama and CRYPTO_*, and it is not going to change. Regards, -- Piotr Gackiewicz Intertele S.A. - operator system?w ITL.PL i DOMENY.ITL.PL al. T. Rejtana 10, 35-310 Rzesz?w TEL: +48 17 8507580, FAX: +48 17 8520275 http://www.itl.pl - niezawodne us?ugi hostingowe http://domeny.itl.pl - tanie domeny internetowe http://www.intertele.pl
On 07/19/2017 04:27 AM, Piotr Gackiewicz wrote:> On Mon, 17 Jul 2017, Johnny Hughes wrote: > >> Are the testing kernels (kernel-4.9.37-29.el7 and kernel-4.9.37-29.el6, >> with the one config file change) working for everyone: >> >> (turn off: CONFIG_IO_STRICT_DEVMEM) > > Hello. > Maybe it's not the most appropriate thread or time, but I have been > signalling it before: > > 4.9.* kernels do not work well for me any more (and for other people > neither, as I know). Last stable kernel was 4.9.13-22. > > Since 4.9.25-26 I do often get: > on 3 supermicro servers (different generations): > - memory allocation errors on Dom0 and corresponding lost lost page writes > due to buffer I/O error on PV guests > - after such memory allocation error od dom0 I have spotted also: > - NFS client hangups on guests (server not responding, still trying > => server OK) > - iptables lockups on PV guest reboot > > on 1 supermicro server: > - memory allocation errors on Dom0 and SATA lockups (many, if not SATA > channels at > - once): > exception Emask 0x0 SAct 0x20 SErr 0x0 action 0x6 frozen > hard resetting link > failed to IDENTIFY (I/O error, err_mask=0x4) > then: blk_update_request: I/O error, dev sd., sector .... > > > All of these machines have been tested with memtest, no detected memory > problems. > No such things occur, when I boot 4.9.13-22 > Most of my guests are centos 6 x86_64, bridged. > > Do anyone had such problems, dealt with it somehow? > > > Since spotting these errors I have done many tests, compiled and tested to > point out single code change (kernel version, patch) - no conclusions yet. > > But one has changed much between 4.9.13 and 4.9.25: kernel size and > configuration. > 4.9.13 size was 6MB and 4.9.24 is 7.1MB. Many modules have been > compiled into kernel, here is shortened, but significant list: > - iptables (NETFILTER_XTABLES, IP_NF_FILTER, IP_NF_TARGET_REJECT) > - SATA_AHCI > - ATA_AHCI (PATA, what a heck?) > - FBDEV_FRONTEND > - HID_MAGICKMOUSE > - HID_NTRIG > - USB_XHCI > - INTEL_SMARTCONNECT >Modules that are not loaded are not used. It has no impact at all on performance or compatibility unless it is used. If you take an lsmod of the kernel that works and one of the kernel with issues, we can see if there are LOADED modules that might cause issues. The modules that are built are the same as Fedora and if in the RHEL 7 kernel, RHEL 7. We did troubleshoot and turn off some things recently, one thing in particular was CONFIG_IO_STRICT_DEVMEM , which is on in fedora, but which is off in some other distros and causes issues with ISCSI and some other things. We also added some specific xen patches, one for netback queue, one for apic, one for nested dom0. Also upstream has added in several xen patches since 4.9.13. And yes, we did change the kernel configs specifically to add in iptables as many people want them. If you can point to problems with a specific module, we can discuss it here and turn it off if necessary. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: OpenPGP digital signature URL: <http://lists.centos.org/pipermail/centos-virt/attachments/20170719/6b1190c1/attachment-0002.sig>
On 2017-07-19 02:27 AM, Piotr Gackiewicz wrote:> On Mon, 17 Jul 2017, Johnny Hughes wrote: > >> Are the testing kernels (kernel-4.9.37-29.el7 and kernel-4.9.37-29.el6, >> with the one config file change) working for everyone: >> >> (turn off: CONFIG_IO_STRICT_DEVMEM) > > Hello. > Maybe it's not the most appropriate thread or time, but I have been > signalling it before: > > 4.9.* kernels do not work well for me any more (and for other people > neither, as I know). Last stable kernel was 4.9.13-22. > > Since 4.9.25-26 I do often get: > on 3 supermicro servers (different generations): > - memory allocation errors on Dom0 and corresponding lost lost page > writes > due to buffer I/O error on PV guests > - after such memory allocation error od dom0 I have spotted also: > - NFS client hangups on guests (server not responding, still > trying => server OK) > - iptables lockups on PV guest reboot > > on 1 supermicro server: > - memory allocation errors on Dom0 and SATA lockups (many, if not SATA > channels at > - once): > exception Emask 0x0 SAct 0x20 SErr 0x0 action 0x6 frozen > hard resetting link > failed to IDENTIFY (I/O error, err_mask=0x4) > then: blk_update_request: I/O error, dev sd., sector .... > > > All of these machines have been tested with memtest, no detected > memory problems. > No such things occur, when I boot 4.9.13-22 > Most of my guests are centos 6 x86_64, bridged. > > Do anyone had such problems, dealt with it somehow? > > > Since spotting these errors I have done many tests, compiled and > tested to > point out single code change (kernel version, patch) - no conclusions > yet. > > But one has changed much between 4.9.13 and 4.9.25: kernel size and > configuration. > 4.9.13 size was 6MB and 4.9.24 is 7.1MB. Many modules have been > compiled into kernel, here is shortened, but significant list: > - iptables (NETFILTER_XTABLES, IP_NF_FILTER, IP_NF_TARGET_REJECT) > - SATA_AHCI > - ATA_AHCI (PATA, what a heck?) > - FBDEV_FRONTEND > - HID_MAGICKMOUSE > - HID_NTRIG > - USB_XHCI > - INTEL_SMARTCONNECT > > Do we really need these compiled into dom0 kernel? > > I assume, that the biggest change in size is due to yama and CRYPTO_*, > and > it is not going to change. > > Regards, > > > > _______________________________________________ > CentOS-virt mailing list > CentOS-virt at centos.org > https://lists.centos.org/mailman/listinfo/centos-virtI have not done any deep digging on this, but we had xen on centos 7 on a couple servers for some experimental vm's. AMD, with Tyan S2882. (Other then the kernel, all the packages are up to date). The CPU's do not support HVM. We host paravirtualized centos 7 instances (stock centos 7 kernels), but in our testing it was rebooting even with no vm's running. With the 4.9.23-26 (I think... would rather retest that before saying for sure) and 4.9.25-27 (I can confirm this) (haven't tested 4.9.34-29), We get periodic reboots and kernel panics. * by I think, we tested 2 kernels since 4.9.13-22 which had troubles so far, and I recall us not yet testing 4.9.34-29). Temporary solution for us is just holding onto the 4.9.13-22 kernel. Anyway, I suppose a simple "me too", for now. Have not brought it up prior as we haven't done any of the legwork to narrow it down. -- Nathan Coulson