Kristiansen Morten
2011-Mar-22 08:52 UTC
[Ocfs2-users] Different disksizes on same disks between cluster nodes
Hi, Resently we resinstalled our four node cluster and tried to upgrade ocfs2 from 1.2.9 to 1.4.7. When two nodes was reinstalled, I stopped ocfs2, deinstalled ocfs2 version 1.2.9 and installed 1.4.7. The last two nodes was down for reinstall. When I mounted node 2 I discovered a mismatch between disksizes on node 1 and node2: [root at node1 ocfs2]# df -Ph Filesystem Size Used Avail Use% Mounted on /dev/sda11 965M 623M 293M 69% / /dev/sda10 449G 848M 425G 1% /disk00 /dev/sda9 2.0G 60M 1.8G 4% /tmp /dev/sda7 9.7G 151M 9.1G 2% /home /dev/sda6 9.7G 1.8G 7.5G 19% /usr /dev/sda5 9.7G 416M 8.8G 5% /var /dev/sda8 4.5G 171M 4.1G 4% /var/log /dev/sda3 20G 5.6G 13G 31% /opt/oracle /dev/sda1 190M 12M 169M 7% /boot tmpfs 16G 0 16G 0% /dev/shm /dev/mapper/mpath8 2.0T 1.9T 158G 93% /disk01 /dev/mapper/mpath9 700G 26G 675G 4% /disk02 /dev/mapper/mpath10 550G 162G 389G 30% /disk03 /dev/mapper/mpath0 850G 652G 199G 77% /disk04 /dev/mapper/mpath6 200G 18G 183G 9% /disk05 /dev/mapper/mpath5 450G 240G 211G 54% /disk06 /dev/mapper/mpath3 750G 2.6G 748G 1% /disk07 /dev/mapper/mpath2 3.6T 2.2T 1.5T 60% /disk08 /dev/mapper/mpath4 4.0G 392M 3.7G 10% /disk09 /dev/mapper/mpath1 316G 5.1G 311G 2% /disk10 /dev/mapper/mpath7 4.1G 392M 3.7G 10% /disk11 [root at node2 ocfs2]# df -PH Filesystem Size Used Avail Use% Mounted on /dev/sda11 1.1G 654M 307M 69% / /dev/sda10 482G 890M 457G 1% /disk00 /dev/sda9 2.1G 63M 2.0G 4% /tmp /dev/sda7 11G 159M 9.8G 2% /home /dev/sda6 11G 1.9G 8.0G 19% /usr /dev/sda5 11G 436M 9.5G 5% /var /dev/sda8 4.8G 175M 4.4G 4% /var/log /dev/sda3 21G 6.0G 14G 31% /opt/oracle /dev/sda1 200M 13M 177M 7% /boot tmpfs 17G 0 17G 0% /dev/shm /dev/mapper/mpath0 2.2T 2.0T 170G 93% /disk01 /dev/mapper/mpath1 752G 28G 725G 4% /disk02 /dev/mapper/mpath2 591G 173G 418G 30% /disk03 /dev/mapper/mpath3 913G 700G 213G 77% /disk04 /dev/mapper/mpath9 215G 19G 197G 9% /disk05 /dev/mapper/mpath8 484G 257G 227G 54% /disk06 /dev/mapper/mpath6 806G 2.8G 803G 1% /disk07 /dev/mapper/mpath5 4.0T 2.4T 1.6T 60% /disk08 /dev/mapper/mpath7 4.3G 411M 3.9G 10% /disk09 /dev/mapper/mpath4 339G 5.5G 334G 2% /disk10 /dev/mapper/mpath10 4.4G 411M 4.0G 10% /disk11 When I discovered the mismatch, I stopped ocfs2, deinstalled version 1.4.7 and reinstalled 1.2.9 on both nodes. When mounting the disks again, the sizes was the same and the same as node1 when running 1.4.7. When I'm writing this, I can see that not only the ocfs2 disks are different. I see that also the OS disks on ext3 file system is different. When I check them against the nodes now, it's all equal and no differences. Has anybody experienced something like this? There was no reboot between the deinstallations and installations of ocfs2. Mvh Morten Kristiansen -------------- next part -------------- An HTML attachment was scrubbed... URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20110322/a9b12e5a/attachment.html
Sunil Mushran
2011-Mar-22 20:13 UTC
[Ocfs2-users] Different disksizes on same disks between cluster nodes
Can you run df without any arguments? Could be a rounding issue in df. Still unsure how ocfs2 install/deinstall could trigger that. On 03/22/2011 01:52 AM, Kristiansen Morten wrote:> > Hi, > > Resently we resinstalled our four node cluster and tried to upgrade ocfs2 from 1.2.9 to 1.4.7. When two nodes was reinstalled, I stopped ocfs2, deinstalled ocfs2 version 1.2.9 and installed 1.4.7. The last two nodes was down for reinstall. When I mounted node 2 I discovered a mismatch between disksizes on node 1 and node2: > > [root at node1 ocfs2]# df -Ph > > Filesystem Size Used Avail Use% Mounted on > > /dev/sda11 965M 623M 293M 69% / > > /dev/sda10 449G 848M 425G 1% /disk00 > > /dev/sda9 2.0G 60M 1.8G 4% /tmp > > /dev/sda7 9.7G 151M 9.1G 2% /home > > /dev/sda6 9.7G 1.8G 7.5G 19% /usr > > /dev/sda5 9.7G 416M 8.8G 5% /var > > /dev/sda8 4.5G 171M 4.1G 4% /var/log > > /dev/sda3 20G 5.6G 13G 31% /opt/oracle > > /dev/sda1 190M 12M 169M 7% /boot > > tmpfs 16G 0 16G 0% /dev/shm > > /dev/mapper/mpath8 2.0T 1.9T 158G 93% /disk01 > > /dev/mapper/mpath9 700G 26G 675G 4% /disk02 > > /dev/mapper/mpath10 550G 162G 389G 30% /disk03 > > /dev/mapper/mpath0 850G 652G 199G 77% /disk04 > > /dev/mapper/mpath6 200G 18G 183G 9% /disk05 > > /dev/mapper/mpath5 450G 240G 211G 54% /disk06 > > /dev/mapper/mpath3 750G 2.6G 748G 1% /disk07 > > /dev/mapper/mpath2 3.6T 2.2T 1.5T 60% /disk08 > > /dev/mapper/mpath4 4.0G 392M 3.7G 10% /disk09 > > /dev/mapper/mpath1 316G 5.1G 311G 2% /disk10 > > /dev/mapper/mpath7 4.1G 392M 3.7G 10% /disk11 > > [root at node2 ocfs2]# df -PH > > Filesystem Size Used Avail Use% Mounted on > > /dev/sda11 1.1G 654M 307M 69% / > > /dev/sda10 482G 890M 457G 1% /disk00 > > /dev/sda9 2.1G 63M 2.0G 4% /tmp > > /dev/sda7 11G 159M 9.8G 2% /home > > /dev/sda6 11G 1.9G 8.0G 19% /usr > > /dev/sda5 11G 436M 9.5G 5% /var > > /dev/sda8 4.8G 175M 4.4G 4% /var/log > > /dev/sda3 21G 6.0G 14G 31% /opt/oracle > > /dev/sda1 200M 13M 177M 7% /boot > > tmpfs 17G 0 17G 0% /dev/shm > > /dev/mapper/mpath0 2.2T 2.0T 170G 93% /disk01 > > /dev/mapper/mpath1 752G 28G 725G 4% /disk02 > > /dev/mapper/mpath2 591G 173G 418G 30% /disk03 > > /dev/mapper/mpath3 913G 700G 213G 77% /disk04 > > /dev/mapper/mpath9 215G 19G 197G 9% /disk05 > > /dev/mapper/mpath8 484G 257G 227G 54% /disk06 > > /dev/mapper/mpath6 806G 2.8G 803G 1% /disk07 > > /dev/mapper/mpath5 4.0T 2.4T 1.6T 60% /disk08 > > /dev/mapper/mpath7 4.3G 411M 3.9G 10% /disk09 > > /dev/mapper/mpath4 339G 5.5G 334G 2% /disk10 > > /dev/mapper/mpath10 4.4G 411M 4.0G 10% /disk11 > > When I discovered the mismatch, I stopped ocfs2, deinstalled version 1.4.7 and reinstalled 1.2.9 on both nodes. When mounting the disks again, the sizes was the same and the same as node1 when running 1.4.7. When I'm writing this, I can see that not only the ocfs2 disks are different. I see that also the OS disks on ext3 file system is different. When I check them against the nodes now, it's all equal and no differences. Has anybody experienced something like this? There was no reboot between the deinstallations and installations of ocfs2. > > Mvh > > Morten Kristiansen > > > _______________________________________________ > Ocfs2-users mailing list > Ocfs2-users at oss.oracle.com > http://oss.oracle.com/mailman/listinfo/ocfs2-users-------------- next part -------------- An HTML attachment was scrubbed... URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20110322/8c73a99c/attachment-0001.html
Herbert van den Bergh
2011-Mar-23 00:01 UTC
[Ocfs2-users] Different disksizes on same disks between cluster nodes
Before anyone else wastes any time looking into this "mystery": one df uses -h and the other uses -H Thanks, Herbert. On 03/22/2011 01:13 PM, Sunil Mushran wrote:> Can you run df without any arguments? Could be a rounding issue in df. > Still unsure how ocfs2 install/deinstall could trigger that. > > On 03/22/2011 01:52 AM, Kristiansen Morten wrote: >> >> Hi, >> >> Resently we resinstalled our four node cluster and tried to upgrade >> ocfs2 from 1.2.9 to 1.4.7. When two nodes was reinstalled, I stopped >> ocfs2, deinstalled ocfs2 version 1.2.9 and installed 1.4.7. The last >> two nodes was down for reinstall. When I mounted node 2 I discovered a >> mismatch between disksizes on node 1 and node2: >> >> [root at node1 ocfs2]# df -Ph >> >> Filesystem Size Used Avail Use% Mounted on >> >> /dev/sda11 965M 623M 293M 69% / >> >> /dev/sda10 449G 848M 425G 1% /disk00 >> >> /dev/sda9 2.0G 60M 1.8G 4% /tmp >> >> /dev/sda7 9.7G 151M 9.1G 2% /home >> >> /dev/sda6 9.7G 1.8G 7.5G 19% /usr >> >> /dev/sda5 9.7G 416M 8.8G 5% /var >> >> /dev/sda8 4.5G 171M 4.1G 4% /var/log >> >> /dev/sda3 20G 5.6G 13G 31% /opt/oracle >> >> /dev/sda1 190M 12M 169M 7% /boot >> >> tmpfs 16G 0 16G 0% /dev/shm >> >> /dev/mapper/mpath8 2.0T 1.9T 158G 93% /disk01 >> >> /dev/mapper/mpath9 700G 26G 675G 4% /disk02 >> >> /dev/mapper/mpath10 550G 162G 389G 30% /disk03 >> >> /dev/mapper/mpath0 850G 652G 199G 77% /disk04 >> >> /dev/mapper/mpath6 200G 18G 183G 9% /disk05 >> >> /dev/mapper/mpath5 450G 240G 211G 54% /disk06 >> >> /dev/mapper/mpath3 750G 2.6G 748G 1% /disk07 >> >> /dev/mapper/mpath2 3.6T 2.2T 1.5T 60% /disk08 >> >> /dev/mapper/mpath4 4.0G 392M 3.7G 10% /disk09 >> >> /dev/mapper/mpath1 316G 5.1G 311G 2% /disk10 >> >> /dev/mapper/mpath7 4.1G 392M 3.7G 10% /disk11 >> >> [root at node2 ocfs2]# df -PH >> >> Filesystem Size Used Avail Use% Mounted on >> >> /dev/sda11 1.1G 654M 307M 69% / >> >> /dev/sda10 482G 890M 457G 1% /disk00 >> >> /dev/sda9 2.1G 63M 2.0G 4% /tmp >> >> /dev/sda7 11G 159M 9.8G 2% /home >> >> /dev/sda6 11G 1.9G 8.0G 19% /usr >> >> /dev/sda5 11G 436M 9.5G 5% /var >> >> /dev/sda8 4.8G 175M 4.4G 4% /var/log >> >> /dev/sda3 21G 6.0G 14G 31% /opt/oracle >> >> /dev/sda1 200M 13M 177M 7% /boot >> >> tmpfs 17G 0 17G 0% /dev/shm >> >> /dev/mapper/mpath0 2.2T 2.0T 170G 93% /disk01 >> >> /dev/mapper/mpath1 752G 28G 725G 4% /disk02 >> >> /dev/mapper/mpath2 591G 173G 418G 30% /disk03 >> >> /dev/mapper/mpath3 913G 700G 213G 77% /disk04 >> >> /dev/mapper/mpath9 215G 19G 197G 9% /disk05 >> >> /dev/mapper/mpath8 484G 257G 227G 54% /disk06 >> >> /dev/mapper/mpath6 806G 2.8G 803G 1% /disk07 >> >> /dev/mapper/mpath5 4.0T 2.4T 1.6T 60% /disk08 >> >> /dev/mapper/mpath7 4.3G 411M 3.9G 10% /disk09 >> >> /dev/mapper/mpath4 339G 5.5G 334G 2% /disk10 >> >> /dev/mapper/mpath10 4.4G 411M 4.0G 10% /disk11 >> >> When I discovered the mismatch, I stopped ocfs2, deinstalled version >> 1.4.7 and reinstalled 1.2.9 on both nodes. When mounting the disks >> again, the sizes was the same and the same as node1 when running >> 1.4.7. When I?m writing this, I can see that not only the ocfs2 disks >> are different. I see that also the OS disks on ext3 file system is >> different. When I check them against the nodes now, it?s all equal and >> no differences. Has anybody experienced something like this? There was >> no reboot between the deinstallations and installations of ocfs2. >> >> Mvh >> >> Morten Kristiansen >> >> >> _______________________________________________ >> Ocfs2-users mailing list >> Ocfs2-users at oss.oracle.com >> http://oss.oracle.com/mailman/listinfo/ocfs2-users > > > > _______________________________________________ > Ocfs2-users mailing list > Ocfs2-users at oss.oracle.com > http://oss.oracle.com/mailman/listinfo/ocfs2-users