The Mellanox ConnectX-4 does not seem to load any firmware looking at dmesg.? I upgraded the firmware on the card on Server2 (that is what I'll call it) so that it matches what was on Server1.? Server2 multichannel still works.? I removed the 802.3ad LACP on Server1 (and removed the config in the switch) and set up both NICs to have an IP on the same subnet.? Multichannel still does not work.? I went into the BIOS on Server1 and nothing stood out as a problem preventing multichannel.? I upgraded Server2 to 4.15rc2 so everything matches as closely as possible to Server1. Does Samba do any kind of logging for multichannel?? I have logging turned up to 10 and all I see is is the option " oing parameter server multi channel support = yes" and nothing else.? How does one debug why Samba is not using multichannel when it's enabled? I'm actually having to work on my vacation right now to try and solve this as we need it working this Tuesday for our project.? I'm frustrated right now as I cannot see why this is not working. BTW, even though I'm frustrated, I very much appreciate the help being provided. ??? Chris On 8/18/21 7:20 AM, Chris Dos via samba wrote:> I'll check to make sure the card is not loading any firmware and if so, I > normally update to the latest firmware packages provided by kernel.org. > > The server that is working only seems to be working on the 100G interface and > not on a 1G on a different network.? So something is still not right.? It > almost seems like it's voodoo right now when it does work.? :) > > ??? Chris > > > On 8/18/21 6:45 AM, L.P.H. van Belle via samba wrote: >> Hai Chris, >> >>> -----Oorspronkelijk bericht----- >>> Van: samba [mailto:samba-bounces at lists.samba.org] Namens >>> Chris Dos via samba >>> Verzonden: woensdag 18 augustus 2021 14:17 >>> Aan: samba at lists.samba.org >>> Onderwerp: Re: [Samba] Multichannel and RSS Not Detected >>> >>> I was digging into the difference between the servers again >>> and I noticed some other difference.? >>> >>> I had upgraded the firmware a couple of months ago on the >>> Supermicro Mellanox card (had to get the firmware from Supermicro) that is in the >>> server that is not working.? I'm also running the Mellanox driver itself on >>> the server that is not working.? >> Well, after you did this, did you check for new firmware packages for debian also. >> These might be out of sync. >> >> dpkg -l |grep firmware >> I dont know if linux is also loading firmware, but check that, should be somewhere in the bootlog/dmesg output. >> If thats the case, i would look in : packages.debian.org if the current firmware is available in newer packages. >> >>> Both of these servers are multi-homed and are connected to three different >>> networks.? I discovered this morning that when looking at the server that is >>> working on the storage interface, that multichannel is not working on the >>> internal interface which is using a different NIC.? So there is another piece >>> added to this puzzle.? You gave an example to disable offloading of the NICs.? >>> Is that required for multichannel/RSS to work? >> I dont think its required. >> >> I would start here with newer kernel and linux firmwares. >> https://packages.debian.org/search?suite=buster-backports§ion=all&arch=any&searchon=names&keywords=linux-image-5.10 >> >> https://packages.debian.org/search?suite=buster§ion=all&arch=any&searchon=names&keywords=firmware-linux >> And possible the "specific firmwares" >> ( like i use : firmware-bnx2 ) >> >> Cant help much beside above, i hope i gave you some ideas where to look at. >> >> >> Greetz, >> >> Louis >> >> >>> ??? Chris >>> >>> On 8/17/21 8:28 AM, L.P.H. van Belle via samba wrote: >>>> On the virtual hosts and possible guest.. >>>> Did you check if there not any problem with drivers and >>> possible offloading of the nics.. >>>> I use XenServer, so thats the example i can give. >>>> >>>> >>> https://github.com/cloudnull/XenServer-Offloading-Off/blob/mas >> ter/offloadingoff.sh >>>> But which one does work now? >>>> You didnt tell that. :-( >>>> >>>> Greetz, >>>> >>>> Louis >>> -- >>> To unsubscribe from this list go to the following URL and read the >>> instructions: https://lists.samba.org/mailman/options/samba >>> >>>
I'm still getting nowhere on this. Is there a recommended vendor for commercial support for my use case?? Which support vendors actually have developers working on Samba??? I need to get this working in the next couple of days. ??? Chris On 8/21/21 3:45 PM, Chris Dos via samba wrote:> The Mellanox ConnectX-4 does not seem to load any firmware looking at dmesg.? > I upgraded the firmware on the card on Server2 (that is what I'll call it) so > that it matches what was on Server1.? Server2 multichannel still works.? I > removed the 802.3ad LACP on Server1 (and removed the config in the switch) and > set up both NICs to have an IP on the same subnet.? Multichannel still does > not work.? I went into the BIOS on Server1 and nothing stood out as a problem > preventing multichannel.? I upgraded Server2 to 4.15rc2 so everything matches > as closely as possible to Server1. > > Does Samba do any kind of logging for multichannel?? I have logging turned up > to 10 and all I see is is the option " oing parameter server multi channel > support = yes" and nothing else.? How does one debug why Samba is not using > multichannel when it's enabled? > > I'm actually having to work on my vacation right now to try and solve this as > we need it working this Tuesday for our project.? I'm frustrated right now as > I cannot see why this is not working. > > BTW, even though I'm frustrated, I very much appreciate the help being provided. > > ??? Chris > > > On 8/18/21 7:20 AM, Chris Dos via samba wrote: >> I'll check to make sure the card is not loading any firmware and if so, I >> normally update to the latest firmware packages provided by kernel.org. >> >> The server that is working only seems to be working on the 100G interface and >> not on a 1G on a different network.? So something is still not right.? It >> almost seems like it's voodoo right now when it does work.? :) >> >> ??? Chris >> >> >> On 8/18/21 6:45 AM, L.P.H. van Belle via samba wrote: >>> Hai Chris, >>> >>>> -----Oorspronkelijk bericht----- >>>> Van: samba [mailto:samba-bounces at lists.samba.org] Namens >>>> Chris Dos via samba >>>> Verzonden: woensdag 18 augustus 2021 14:17 >>>> Aan: samba at lists.samba.org >>>> Onderwerp: Re: [Samba] Multichannel and RSS Not Detected >>>> >>>> I was digging into the difference between the servers again >>>> and I noticed some other difference.? >>>> >>>> I had upgraded the firmware a couple of months ago on the >>>> Supermicro Mellanox card (had to get the firmware from Supermicro) that is in the >>>> server that is not working.? I'm also running the Mellanox driver itself on >>>> the server that is not working.? >>> Well, after you did this, did you check for new firmware packages for debian also. >>> These might be out of sync. >>> >>> dpkg -l |grep firmware >>> I dont know if linux is also loading firmware, but check that, should be somewhere in the bootlog/dmesg output. >>> If thats the case, i would look in : packages.debian.org if the current firmware is available in newer packages. >>> >>>> Both of these servers are multi-homed and are connected to three different >>>> networks.? I discovered this morning that when looking at the server that is >>>> working on the storage interface, that multichannel is not working on the >>>> internal interface which is using a different NIC.? So there is another piece >>>> added to this puzzle.? You gave an example to disable offloading of the NICs.? >>>> Is that required for multichannel/RSS to work? >>> I dont think its required. >>> >>> I would start here with newer kernel and linux firmwares. >>> https://packages.debian.org/search?suite=buster-backports§ion=all&arch=any&searchon=names&keywords=linux-image-5.10 >>> >>> https://packages.debian.org/search?suite=buster§ion=all&arch=any&searchon=names&keywords=firmware-linux >>> And possible the "specific firmwares" >>> ( like i use : firmware-bnx2 ) >>> >>> Cant help much beside above, i hope i gave you some ideas where to look at. >>> >>> >>> Greetz, >>> >>> Louis >>> >>> >>>> ??? Chris >>>> >>>> On 8/17/21 8:28 AM, L.P.H. van Belle via samba wrote: >>>>> On the virtual hosts and possible guest.. >>>>> Did you check if there not any problem with drivers and >>>> possible offloading of the nics.. >>>>> I use XenServer, so thats the example i can give. >>>>> >>>>> >>>> https://github.com/cloudnull/XenServer-Offloading-Off/blob/mas >>> ter/offloadingoff.sh >>>>> But which one does work now? >>>>> You didnt tell that. :-( >>>>> >>>>> Greetz, >>>>> >>>>> Louis >>>> -- >>>> To unsubscribe from this list go to the following URL and read the >>>> instructions: https://lists.samba.org/mailman/options/samba >>>> >>>>
Ten months later and it is time to work on this again.? We used local storage to do the job we had to get done last year.? We have other projects coming up it will be nice to get this working. I am at the point again where googling is not providing any answers.? Essentially I have a server that RSS Multichannel does not seem to work on, but another one with the same NIC and a bit older hardware that works fine.? I'm probably going to have to take a packet capture to provide help debugging this problem so what would the ports need to be or should I just dump everything on the interface?? Is there any debug logging that can be enabled for Multichannel? We are running version 4.15.7 using apt.van-belle.nl Debian packages even though we are running Devuan (Debian Bullseye minus SystemD) Here is part of my smb.conf file: [global] ?? ?netbios name = SB1 ?? ?interfaces = "172.28.202.10;capability=RSS,speed=1000000000" "172.28.90.10;capability=RSS,speed=10000000000" "172.28.133.10;capability=RSS,speed=100000000000" ??? server multi channel support = yes ?? ?workgroup = GIANT ?? ?wins server = 172.28.202.5 ?? ?os level = 20 ??? log level = 3 ?? ?client min protocol = SMB3 ??? os level = 8 ??? local master = No ??? domain master = No ?? ?ntlm auth = yes ?? ?log file = /var/log/samba/log.%m ??? max log size = 1000 ?? ?server role = standalone server ?? ?obey pam restrictions = No ?? ?unix password sync = no ?? ?passwd program = /usr/bin/passwd %u ?? ?passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* ?? ?invalid users = root ?? ?usershare path ?? ?usershare allow guests = yes ?? ?guest account = nas ?? ?aio read size = 1 ?? ?aio write size = 1 ?? ?vfs objects = catia ??? mangled names = no ??? catia:mappings 0x22:0xa8,0x2a:0xa4,0x2f:0xf8,0x3a:0xf7,0x3c:0xab,0x3e:0xbb,0x3f:0xbf,0x5c:0xff,0x7c:0xa6 ?? ?posix locking = no ?? ?security = user ?? ?map to guest = bad user??? ?? ?store dos attributes = yes ?? ?dos filetimes = yes [production] ?? ?path = /netshares/production ?? ?force user = nas ?? ?force group = nas ??? create mask = 770 ??? force create mode = 770 ??? directory mask = 2770 ??? force directory mode = 2770 ?? ?guest ok = yes ?? ?read only = no ?? ?hosts allow = 172.28.133. 172.28.90. 172.28.202.5 Any help would be appreciated.? I reached out to SerNet to see if Stefan Metzmacher (who was the main author of the Multichannel code) was available and received a support quote for them to help, but out management did not want to spend the money at the time.? So I'm back to using the mailing list. I have found others that have not gotten Multichannel working for some reason or another and they just gave up.? I do have Multichannel working on three other servers running Devuan with a bit different hardware.? So why it is not working on the main server I need it working on I have no idea. ??? Chris On 8/21/21 15:45, Chris Dos via samba wrote:> Does Samba do any kind of logging for multichannel?? I have logging turned up > to 10 and all I see is is the option " oing parameter server multi channel > support = yes" and nothing else.? How does one debug why Samba is not using > multichannel when it's enabled? > > I'm actually having to work on my vacation right now to try and solve this as > we need it working this Tuesday for our project.? I'm frustrated right now as > I cannot see why this is not working. > > BTW, even though I'm frustrated, I very much appreciate the help being provided. > > ??? Chris >