Jon;
Found this one today and noticed no response so I thought I would give it a go.?
From the manual it states to use zero partitions when possible and to deal
entirely with the block device itself.? The manual also states that the creation
of an external journal is recommended.? I used a separate 1TB disk for the
creation of the external journal and in some instances swap space.?
Here I outline a small capable robust Configuration for under $40000.00:
[root at ns1 ~]# cat /proc/mdstat???????????????????????????????????
Personalities : [raid1] [raid10]?????????????????????????????????
md3 : active raid10 sdc[0] sdh[5] sdg[4] sdf[3] sde[2] sdd[1]????
????? 2930287488 blocks 64K chunks 2 near-copies [6/6] [UUUUUU]??
?????????????????????????????????????????????????????????????????
md1 : active raid1 sda3[0] sdb3[1]???????????????????????????????
????? 891520 blocks [2/2] [UU]???????????????????????????????????
?????????????????????????????????????????????????????????????????
md0 : active raid1 sdb1[1] sda1[0]???????????????????????????????
????? 965634880 blocks [2/2] [UU]????????????????????????????????
?????????????????????????????????????????????????????????????????
unused devices: <none>???????????????????????????????????????????
[root at ns1 ~]# lctl dl????????????????????????????????????????????
? 0 UP mgc MGC192.168.0.10 at tcp f038d71d-792a-16cd-9909-e24dbe6dcfd2 5
? 1 UP ost OSS OSS_uuid 3???????????????????????????????????????????
? 2 UP obdfilter ioio-OST0002 ioio-OST0002_UUID 13??????????????????
[root at ns1 ~]# cat /etc/fstab????????????????????????????????????????
/dev/md0??????????????? /?????????????????????? ext3??? defaults??????? 1 1
tmpfs?????????????????? /dev/shm??????????????? tmpfs?? defaults??????? 0 0
devpts????????????????? /dev/pts??????????????? devpts? gid=5,mode=620? 0 0
sysfs?????????????????? /sys??????????????????? sysfs?? defaults??????? 0 0
proc??????????????????? /proc?????????????????? proc??? defaults??????? 0 0
LABEL=SWAP-sdb2???????? swap??????????????????? swap??? defaults??????? 0 0
LABEL=SWAP-sda2???????? swap??????????????????? swap??? defaults??????? 0 0
/dev/md3??????????????? /mnt/data/ost02???????? lustre? defaults,_netdev 0 0
#/dev/md4?????????????? /mnt/flash/ost03??????? lustre? defaults,_netdev 0 0
[root at ns1 ~]# cat /proc/net/bonding
cat: /proc/net/bonding: Is a directory
[root at ns1 ~]# cat /proc/net/bond
cat: /proc/net/bond: No such file or directory
[root at ns1 ~]# cat /proc/net/bonding/bond0????
Ethernet Channel Bonding Driver: v3.4.0-1 (October 7, 2008)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)????????????????
MII Status: up????????????????????????????????????
MII Polling Interval (ms): 10?????????????????????
Up Delay (ms): 0??????????????????????????????????
Down Delay (ms): 0????????????????????????????????
802.3ad info
LACP rate: slow
Active Aggregator Info:
??????? Aggregator ID: 1
??????? Number of ports: 5
??????? Actor Key: 17????
??????? Partner Key: 37823
??????? Partner Mac Address: 00:22:57:9a:ea:00
Slave Interface: eth0
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:1b:21:28:74:d8
Aggregator ID: 1???????????????????
Slave Interface: eth1
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 1
Permanent HW addr: 00:1b:21:28:76:d2
Aggregator ID: 1???????????????????
Slave Interface: eth2
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:39:2f
Aggregator ID: 1???????????????????
Slave Interface: eth3
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:39:30
Aggregator ID: 1???????????????????
Slave Interface: eth4
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:39:31
Aggregator ID: 1???????????????????
[root at ns1 ~]# ifconfig?????????????
bond0???? Link encap:Ethernet? HWaddr 00:1B:21:28:74:D8?
????????? inet addr:192.168.0.19? Bcast:192.168.0.255? Mask:255.255.255.0
????????? UP BROADCAST RUNNING MASTER MULTICAST? MTU:1500? Metric:1?????
????????? RX packets:89680747 errors:0 dropped:493712 overruns:0 frame:0
????????? TX packets:54703009 errors:0 dropped:0 overruns:0 carrier:0???
????????? collisions:0 txqueuelen:0?????????????????????????????????????
????????? RX bytes:87798998004 (81.7 GiB)? TX bytes:137015921248 (127.6 GiB)
eth0????? Link encap:Ethernet? HWaddr 00:1B:21:28:74:D8?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:6158033 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:4754443 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000?????????????????????????????
????????? RX bytes:2572939264 (2.3 GiB)? TX bytes:6124172876 (5.7 GiB)
eth1????? Link encap:Ethernet? HWaddr 00:1B:21:28:74:D8?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:1715604 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:3837069 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000?????????????????????????????
????????? RX bytes:105078923 (100.2 MiB)? TX bytes:2128255017 (1.9 GiB)
eth2????? Link encap:Ethernet? HWaddr 00:1B:21:28:74:D8?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:141703 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:3999225 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000?????????????????????????????
????????? RX bytes:14455587 (13.7 MiB)? TX bytes:2440673445 (2.2 GiB)
????????? Interrupt:74??????????????????????????????????????????????
eth3????? Link encap:Ethernet? HWaddr 00:1B:21:28:74:D8?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:63575447 errors:0 dropped:493696 overruns:0 frame:0
????????? TX packets:14850445 errors:0 dropped:0 overruns:0 carrier:0??
????????? collisions:0 txqueuelen:1000?????????????????????????????????
????????? RX bytes:83036911085 (77.3 GiB)? TX bytes:122278775783 (113.8 GiB)
????????? Interrupt:98?????????????????????????????????????????????????????
eth4????? Link encap:Ethernet? HWaddr 00:1B:21:28:74:D8?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:18089968 errors:0 dropped:16 overruns:0 frame:0
????????? TX packets:27261838 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000
????????? RX bytes:2069613561 (1.9 GiB)? TX bytes:4044047045 (3.7 GiB)
????????? Interrupt:169
eth5????? Link encap:Ethernet? HWaddr 00:22:15:06:39:32
????????? inet addr:XX.XX.XXX.XXX? Bcast:96.53.120.255? Mask:255.255.255.0
????????? UP BROADCAST RUNNING MULTICAST? MTU:1500? Metric:1
????????? RX packets:8650228 errors:0 dropped:4716 overruns:0 frame:0
????????? TX packets:4278085 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000
????????? RX bytes:7047416207 (6.5 GiB)? TX bytes:1721168584 (1.6 GiB)
????????? Interrupt:177
lo??????? Link encap:Local Loopback
????????? inet addr:127.0.0.1? Mask:255.0.0.0
????????? UP LOOPBACK RUNNING? MTU:16436? Metric:1
????????? RX packets:211949 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:211949 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:0
????????? RX bytes:48836875 (46.5 MiB)? TX bytes:48836875 (46.5 MiB)
[root at ns1 ~]# uptime
?12:25:59 up 7 days, 16:49,? 5 users,? load average: 0.00, 0.02, 0.00]#
**Firewall/DNS External IP eth5 OSS/OST
[root at ns2 ~]# cat /proc/mdstat???????????????????????????
Personalities : [raid1] [raid10]?????????????????????????
md3 : active raid10 sdc[0] sdh[5] sdg[4] sdf[3] sde[2] sdd[1]
????? 2930287488 blocks 64K chunks 2 near-copies [6/6] [UUUUUU]
??????????????????????????????????????????????????????????????
md1 : active raid1 sda3[0] sdb3[1]????????????????????????????
????? 891520 blocks [2/2] [UU]????????????????????????????????
??????????????????????????????????????????????????????????????
md0 : active raid1 sdb1[1] sda1[0]????????????????????????????
????? 965634880 blocks [2/2] [UU]?????????????????????????????
??????????????????????????????????????????????????????????????
unused devices: <none>????????????????????????????????????????
[root at ns2 ~]# lctl dl?????????????????????????????????????????
? 0 UP mgc MGC192.168.0.10 at tcp 0e72db5f-9495-e31e-85c2-8d31a009d130 5
? 1 UP ost OSS OSS_uuid 3???????????????????????????????????????????
? 2 UP obdfilter ioio-OST0001 ioio-OST0001_UUID 13??????????????????
[root at ns2 ~]# cat /etc/fstab????????????????????????????????????????
/dev/md0??????????????? /?????????????????????? ext3??? defaults??????? 1 1
tmpfs?????????????????? /dev/shm??????????????? tmpfs?? defaults??????? 0 0
devpts????????????????? /dev/pts??????????????? devpts? gid=5,mode=620? 0 0
sysfs?????????????????? /sys??????????????????? sysfs?? defaults??????? 0 0
proc??????????????????? /proc?????????????????? proc??? defaults??????? 0 0
LABEL=SWAP-sdb2???????? swap??????????????????? swap??? defaults??????? 0 0
LABEL=SWAP-sda2???????? swap??????????????????? swap??? defaults??????? 0 0
/dev/md3??????????????? /mnt/data/ost01???????? lustre? defaults,_netdev 0 0
#/dev/md4?????????????? /mnt/flash/ost02??????? lustre? defaults,_netdev 0 0
[root at ns2 ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.4.0-1 (October 7, 2008)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)????????????????
MII Status: up????????????????????????????????????
MII Polling Interval (ms): 10?????????????????????
Up Delay (ms): 0??????????????????????????????????
Down Delay (ms): 0????????????????????????????????
802.3ad info
LACP rate: slow
Active Aggregator Info:
??????? Aggregator ID: 1
??????? Number of ports: 5
??????? Actor Key: 17????
??????? Partner Key: 37823
??????? Partner Mac Address: 00:22:57:9a:ea:00
Slave Interface: eth0
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:1b:21:2a:1c:3c
Aggregator ID: 1???????????????????
Slave Interface: eth1
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:1b:21:2a:1c:34
Aggregator ID: 1???????????????????
Slave Interface: eth2
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:38:b3
Aggregator ID: 1???????????????????
Slave Interface: eth3
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:38:b4
Aggregator ID: 1???????????????????
Slave Interface: eth4
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:38:b5
Aggregator ID: 1???????????????????
[root at ns2 ~]# ifconfig?????????????
bond0???? Link encap:Ethernet? HWaddr 00:1B:21:2A:1C:3C?
????????? inet addr:192.168.0.13? Bcast:192.168.0.255? Mask:255.255.255.0
????????? UP BROADCAST RUNNING MASTER MULTICAST? MTU:1500? Metric:1?????
????????? RX packets:73883791 errors:0 dropped:405040 overruns:0 frame:0
????????? TX packets:89400059 errors:0 dropped:0 overruns:0 carrier:0???
????????? collisions:0 txqueuelen:0?????????????????????????????????????
????????? RX bytes:74258769521 (69.1 GiB)? TX bytes:116179989862 (108.2 GiB)
eth0????? Link encap:Ethernet? HWaddr 00:1B:21:2A:1C:3C?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:53224496 errors:0 dropped:2 overruns:0 frame:0
????????? TX packets:37354638 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000??????????????????????????????
????????? RX bytes:70370076932 (65.5 GiB)? TX bytes:51859648442 (48.2 GiB)
eth1????? Link encap:Ethernet? HWaddr 00:1B:21:2A:1C:3C?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:20567731 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:41842603 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000??????????????????????????????
????????? RX bytes:3880185151 (3.6 GiB)? TX bytes:33224132469 (30.9 GiB)
eth2????? Link encap:Ethernet? HWaddr 00:1B:21:2A:1C:3C?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:46143 errors:0 dropped:0 overruns:0 frame:0?
????????? TX packets:2055392 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000?????????????????????????????
????????? RX bytes:3573296 (3.4 MiB)? TX bytes:487616806 (465.0 MiB)
????????? Interrupt:74?????????????????????????????????????????????
eth3????? Link encap:Ethernet? HWaddr 00:1B:21:2A:1C:3C?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:22228 errors:0 dropped:0 overruns:0 frame:0?
????????? TX packets:2406691 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000?????????????????????????????
????????? RX bytes:2445080 (2.3 MiB)? TX bytes:675387313 (644.0 MiB)
????????? Interrupt:98?????????????????????????????????????????????
eth4????? Link encap:Ethernet? HWaddr 00:1B:21:2A:1C:3C?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:23207 errors:0 dropped:405038 overruns:0 frame:0
????????? TX packets:5740755 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000
????????? RX bytes:2489986 (2.3 MiB)? TX bytes:29933208120 (27.8 GiB)
????????? Interrupt:169
eth5????? Link encap:Ethernet? HWaddr 00:22:15:06:38:B6
????????? inet addr:XX.XX.XXX.XXX? Bcast:96.53.120.255? Mask:255.255.255.0
????????? UP BROADCAST RUNNING MULTICAST? MTU:1500? Metric:1
????????? RX packets:5679458 errors:0 dropped:124836 overruns:0 frame:0
????????? TX packets:1640529 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000
????????? RX bytes:329507734 (314.2 MiB)? TX bytes:1288265410 (1.1 GiB)
????????? Interrupt:177
lo??????? Link encap:Local Loopback
????????? inet addr:127.0.0.1? Mask:255.0.0.0
????????? UP LOOPBACK RUNNING? MTU:16436? Metric:1
????????? RX packets:33230 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:33230 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:0
????????? RX bytes:24813551 (23.6 MiB)? TX bytes:24813551 (23.6 MiB)
[root at ns2 ~]# uptime
?12:28:08 up 7 days, 16:52,? 1 user,? load average: 0.37, 0.25, 0.19
** Firewall/DNS External IP OSS/OST
[root at ns3 ~]# cat /proc/mdstat???????????????????????????????????
Personalities : [raid1] [raid10]?????????????????????????????????
md3 : active raid10 sdc[0] sdh[5] sdg[4] sdf[3] sde[2] sdd[1]????
????? 2930287488 blocks 64K chunks 2 near-copies [6/6] [UUUUUU]??
?????????????????????????????????????????????????????????????????
md1 : active raid1 sda3[0] sdb3[1]???????????????????????????????
????? 891520 blocks [2/2] [UU]???????????????????????????????????
?????????????????????????????????????????????????????????????????
md0 : active raid1 sdb1[1] sda1[0]???????????????????????????????
????? 965634880 blocks [2/2] [UU]????????????????????????????????
?????????????????????????????????????????????????????????????????
unused devices: <none>???????????????????????????????????????????
[root at ns3 ~]# lctl dl????????????????????????????????????????????
? 0 UP mgc MGC192.168.0.10 at tcp 75c57524-4f99-6424-f78a-eaff14d115bd 5
? 1 UP ost OSS OSS_uuid 3???????????????????????????????????????????
? 2 UP obdfilter ioio-OST0000 ioio-OST0000_UUID 13??????????????????
[root at ns3 ~]# cat /etc/fstab????????????????????????????????????????
/dev/md0??????????????? /?????????????????????? ext3??? defaults??????? 1 1
tmpfs?????????????????? /dev/shm??????????????? tmpfs?? defaults??????? 0 0
devpts????????????????? /dev/pts??????????????? devpts? gid=5,mode=620? 0 0
sysfs?????????????????? /sys??????????????????? sysfs?? defaults??????? 0 0
proc??????????????????? /proc?????????????????? proc??? defaults??????? 0 0
LABEL=SWAP-sdb2???????? swap??????????????????? swap??? defaults??????? 0 0
LABEL=SWAP-sda2???????? swap??????????????????? swap??? defaults??????? 0 0
/dev/md3??????????????? /mnt/data/ost00???????? lustre? defaults,_netdev 0 0
#/dev/md4?????????????? /mnt/flash/ost01??????? lustre? defaults,_netdev 0 0
[root at ns3 ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.4.0-1 (October 7, 2008)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)????????????????
MII Status: up????????????????????????????????????
MII Polling Interval (ms): 10?????????????????????
Up Delay (ms): 0??????????????????????????????????
Down Delay (ms): 0????????????????????????????????
802.3ad info
LACP rate: slow
Active Aggregator Info:
??????? Aggregator ID: 5
??????? Number of ports: 6
??????? Actor Key: 17????
??????? Partner Key: 37823
??????? Partner Mac Address: 00:22:57:9a:ea:00
Slave Interface: eth0
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:3a:0f
Aggregator ID: 5???????????????????
Slave Interface: eth1
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:3a:10
Aggregator ID: 5???????????????????
Slave Interface: eth2
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:3a:11
Aggregator ID: 5???????????????????
Slave Interface: eth3
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:3a:12
Aggregator ID: 5???????????????????
Slave Interface: eth4
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:1b:21:2a:17:76
Aggregator ID: 5???????????????????
Slave Interface: eth5
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:1b:21:2a:1c:dc
Aggregator ID: 5???????????????????
[root at ns3 ~]# ifconfig?????????????
bond0???? Link encap:Ethernet? HWaddr 00:22:15:06:3A:0F?
????????? inet addr:192.168.0.7? Bcast:192.168.0.255? Mask:255.255.255.0
????????? inet6 addr: fe80::222:15ff:fe06:3a0f/64 Scope:Link???????????
????????? UP BROADCAST RUNNING MASTER MULTICAST? MTU:1500? Metric:1????
????????? RX packets:91510283 errors:0 dropped:364706 overruns:0 frame:0
????????? TX packets:69302068 errors:0 dropped:0 overruns:0 carrier:0??
????????? collisions:0 txqueuelen:0????????????????????????????????????
????????? RX bytes:55958008995 (52.1 GiB)? TX bytes:223016561848 (207.7 GiB)
eth0????? Link encap:Ethernet? HWaddr 00:22:15:06:3A:0F?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:42433 errors:0 dropped:2 overruns:0 frame:0?
????????? TX packets:9926913 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000?????????????????????????????
????????? RX bytes:3374702 (3.2 MiB)? TX bytes:851799191 (812.3 MiB)
????????? Interrupt:74?????????????????????????????????????????????
eth1????? Link encap:Ethernet? HWaddr 00:22:15:06:3A:0F?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:1712310 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:266655 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000????????????????????????????
????????? RX bytes:80908257 (77.1 MiB)? TX bytes:208988501 (199.3 MiB)
????????? Interrupt:98???????????????????????????????????????????????
eth2????? Link encap:Ethernet? HWaddr 00:22:15:06:3A:0F?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:8953421 errors:0 dropped:364702 overruns:0 frame:0
????????? TX packets:27700345 errors:0 dropped:0 overruns:0 carrier:0?
????????? collisions:0 txqueuelen:1000????????????????????????????????
????????? RX bytes:952280437 (908.1 MiB)? TX bytes:182598635220 (170.0 GiB)
????????? Interrupt:169???????????????????????????????????????????????????
eth3????? Link encap:Ethernet? HWaddr 00:22:15:06:3A:0F?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:23273 errors:0 dropped:2 overruns:0 frame:0?
????????? TX packets:5192617 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000?????????????????????????????
????????? RX bytes:2493150 (2.3 MiB)? TX bytes:1880125062 (1.7 GiB)
????????? Interrupt:177????????????????????????????????????????????
eth4????? Link encap:Ethernet? HWaddr 00:22:15:06:3A:0F?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:38058348 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:1653728 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000
????????? RX bytes:3642127587 (3.3 GiB)? TX bytes:759108694 (723.9 MiB)
eth5????? Link encap:Ethernet? HWaddr 00:22:15:06:3A:0F
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:42720504 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:24561812 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000
????????? RX bytes:51276825174 (47.7 GiB)? TX bytes:36717905520 (34.1 GiB)
lo??????? Link encap:Local Loopback
????????? inet addr:127.0.0.1? Mask:255.0.0.0
????????? inet6 addr: ::1/128 Scope:Host
????????? UP LOOPBACK RUNNING? MTU:16436? Metric:1
????????? RX packets:2798 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:2798 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:0
????????? RX bytes:4431535 (4.2 MiB)? TX bytes:4431535 (4.2 MiB)
[root at ns3 ~]# uptime
?12:30:58 up 7 days, 16:55,? 1 user,? load average: 0.00, 0.00, 0.00
** OSS/OST LAMP webroot=/var/www/html ie: not Lustre mounted
[root at ten ~]# cat /proc/mdstat???????????????????????????????????
Personalities : [raid1]??????????????????????????????????????????
md0 : active raid1 sdc[0] sdb[1]?????????????????????????????????
????? 976762496 blocks [2/2] [UU]????????????????????????????????
?????????????????????????????????????????????????????????????????
unused devices: <none>???????????????????????????????????????????
[root at ten ~]# lctl dl????????????????????????????????????????????
? 0 UP mgs MGS MGS 17????????????????????????????????????????????
? 1 UP mgc MGC192.168.0.10 at tcp 24fb4ce0-ea3f-1324-5d61-ea9f45f0baa8 5
? 2 UP mdt MDS MDS_uuid 3???????????????????????????????????????????
? 3 UP lov ioio-mdtlov ioio-mdtlov_UUID 4???????????????????????????
? 4 UP mds ioio-MDT0000 ioio-MDT0000_UUID 11????????????????????????
? 5 UP osc ioio-OST0000-osc ioio-mdtlov_UUID 5??????????????????????
? 6 UP osc ioio-OST0001-osc ioio-mdtlov_UUID 5??????????????????????
? 7 UP osc ioio-OST0002-osc ioio-mdtlov_UUID 5??????????????????????
? 8 UP lov ioio-clilov-ffff8102179aa000 b47f5fbf-f9dc-7660-3488-6a0d3420f386 4
? 9 UP mdc ioio-MDT0000-mdc-ffff8102179aa000
b47f5fbf-f9dc-7660-3488-6a0d3420f386 5
?10 UP osc ioio-OST0000-osc-ffff8102179aa000
b47f5fbf-f9dc-7660-3488-6a0d3420f386 5
?11 UP osc ioio-OST0001-osc-ffff8102179aa000
b47f5fbf-f9dc-7660-3488-6a0d3420f386 5
?12 UP osc ioio-OST0002-osc-ffff8102179aa000
b47f5fbf-f9dc-7660-3488-6a0d3420f386 5
[root at ten ~]# cat
/etc/fstab??????????????????????????????????????????????????????
LABEL=/???????????????? /?????????????????????? ext3??? defaults??????? 1
1???????
tmpfs?????????????????? /dev/shm??????????????? tmpfs?? defaults??????? 0
0???????
devpts????????????????? /dev/pts??????????????? devpts? gid=5,mode=620? 0
0???????
sysfs?????????????????? /sys??????????????????? sysfs?? defaults??????? 0
0???????
proc??????????????????? /proc?????????????????? proc??? defaults??????? 0
0???????
LABEL=SWAP-sda2???????? swap??????????????????? swap??? defaults??????? 0
0???????
/dev/md0??????????????? /mnt/data/mdt?????????? lustre? defaults,_netdev 0
0??????
#/dev/sdd1????????????? /mnt/data/flash???????? lustre? defaults,_netdev 0
0??????
[root at ten ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.4.0-1 (October 7, 2008)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)????????????????
MII Status: up????????????????????????????????????
MII Polling Interval (ms): 10?????????????????????
Up Delay (ms): 0??????????????????????????????????
Down Delay (ms): 0????????????????????????????????
802.3ad info
LACP rate: slow
Active Aggregator Info:
??????? Aggregator ID: 5
??????? Number of ports: 6
??????? Actor Key: 17????
??????? Partner Key: 37823
??????? Partner Mac Address: 00:22:57:9a:ea:00
Slave Interface: eth0
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:3a:93
Aggregator ID: 5???????????????????
Slave Interface: eth1
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:3a:94
Aggregator ID: 5???????????????????
Slave Interface: eth2
MII Status: up??????
Speed: 1000 Mbps????
Duplex: full????????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:3a:95
Aggregator ID: 5???????????????????
Slave Interface: eth3
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:00:00:00:00:00
Aggregator ID: 5
Slave Interface: eth4
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:1b:21:28:77:db
Aggregator ID: 5
Slave Interface: eth5
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 1
Permanent HW addr: 00:1b:21:28:77:6c
Aggregator ID: 5
[root at ten ~]# uptime
?12:36:03 up 7 days, 16:59,? 1 user,? load average: 0.00, 0.01, 0.00
[root at ten ~]# lfs df -i
UUID????????????????????? Inodes?????? IUsed?????? IFree IUse% Mounted on
ioio-MDT0000_UUID????? 215600857???? 2046153?? 213554704?? 1% /mnt/ioio[MDT:0]
ioio-OST0000_UUID????? 183148544????? 656907?? 182491637?? 0% /mnt/ioio[OST:0]
ioio-OST0001_UUID????? 183148544????? 657442?? 182491102?? 0% /mnt/ioio[OST:1]
ioio-OST0002_UUID????? 183148544????? 668586?? 182479958?? 0% /mnt/ioio[OST:2]
filesystem summary:??? 215600857???? 2046153?? 213554704?? 1% /mnt/ioio
** MGS/MDT Machine.? Redundant Lustre Client.? Redundant LAMP Server.
Last login: Mon Jan 21 21:37:31 2013????????????
[root at three ~]# cat /proc/mdstat????????????????
Personalities : [raid6] [raid5] [raid4]?????????
md0 : active raid5 sdb[1] sdd[2] sdc[0] sde[4]??
????? 5860538880 blocks super 1.1 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]
????? bitmap: 0/15 pages [0KB], 65536KB chunk?????????????????????????????????
unused devices: <none>
[root at three ~]# df -h
Filesystem??????????? Size? Used Avail Use% Mounted on
/dev/sda1???????????? 459G?? 18G? 418G?? 5% /????????
tmpfs??????????????? 1004M?? 37M? 968M?? 4% /dev/shm?
/dev/md0????????????? 5.4T? 3.2T? 2.0T? 62% /mnt/rv??
192.168.0.10 at tcp0:/ioio??????????????????????????????
????????????????????? 8.1T? 4.8T? 3.0T? 62% /mnt/ioio
[root at three ~]# lctl dl??????????????????????????????
? 0 UP mgc MGC192.168.0.10 at tcp 4aee8c8e-ab6f-d288-9cff-7953dff76905 5
? 1 UP lov ioio-clilov-ffff880075023000 06e9543a-5540-afe2-8e6a-5f21d8085076 4
? 2 UP mdc ioio-MDT0000-mdc-ffff880075023000
06e9543a-5540-afe2-8e6a-5f21d8085076 5
? 3 UP osc ioio-OST0000-osc-ffff880075023000
06e9543a-5540-afe2-8e6a-5f21d8085076 5
? 4 UP osc ioio-OST0001-osc-ffff880075023000
06e9543a-5540-afe2-8e6a-5f21d8085076 5
? 5 UP osc ioio-OST0002-osc-ffff880075023000
06e9543a-5540-afe2-8e6a-5f21d8085076 5
[root at three ~]# cat
/etc/fstab????????????????????????????????????????????????????
#
# /etc/fstab
# Created by anaconda on Mon Sep? 3 07:44:32 2012
#???????????????????????????????????????????????
# Accessible filesystems, by reference, are maintained under
''/dev/disk''
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#?????????????????????????????????????????????????????????????????????????
UUID=06e1bb0d-d7c8-42a4-bad1-6d12f931b77a /???? ext4??? defaults??????? 1 1
tmpfs?????????????????? /dev/shm??????????????? tmpfs?? defaults??????? 0 0
devpts????????????????? /dev/pts??????????????? devpts? gid=5,mode=620? 0 0
sysfs?????????????????? /sys??????????????????? sysfs?? defaults??????? 0 0
proc??????????????????? /proc?????????????????? proc??? defaults??????? 0 0
/dev/md0??????????????? /mnt/rv???????????????? ext4??? defaults??????? 0 0
[root at three ~]# ifconfig
bond0???? Link encap:Ethernet? HWaddr 00:22:15:06:3B:73?
????????? inet addr:192.168.0.3? Bcast:192.168.0.255? Mask:255.255.255.0
????????? inet6 addr: fe80::222:15ff:fe06:3b73/64 Scope:Link???????????
????????? UP BROADCAST RUNNING MASTER MULTICAST? MTU:1500? Metric:1????
????????? RX packets:2084539 errors:0 dropped:0 overruns:0 frame:0?????
????????? TX packets:1840604 errors:0 dropped:0 overruns:0 carrier:0???
????????? collisions:0 txqueuelen:0????????????????????????????????????
????????? RX bytes:1353887185 (1.2 GiB)? TX bytes:990151888 (944.2 MiB)
eth0????? Link encap:Ethernet? HWaddr 00:22:15:06:3B:73?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:186644 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:1611744 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000?????????????????????????????
????????? RX bytes:165778866 (158.0 MiB)? TX bytes:950467922 (906.4 MiB)
????????? Interrupt:18?????????????????????????????????????????????????
eth1????? Link encap:Ethernet? HWaddr 00:22:15:06:3B:73?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:1896003 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:34762 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000???????????????????????????
????????? RX bytes:1187880303 (1.1 GiB)? TX bytes:14394977 (13.7 MiB)
????????? Interrupt:19??????????????????????????????????????????????
eth2????? Link encap:Ethernet? HWaddr 00:22:15:06:3B:73?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:1892 errors:0 dropped:0 overruns:0 frame:0??
????????? TX packets:194098 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000????????????????????????????
????????? RX bytes:228016 (222.6 KiB)? TX bytes:25288989 (24.1 MiB)
????????? Interrupt:16????????????????????????????????????????????
eth3????? Link encap:Ethernet? HWaddr 00:22:15:06:3B:73?
????????? UP BROADCAST SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:0 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000???????????????????????
????????? RX bytes:0 (0.0 b)? TX bytes:0 (0.0 b)?????????????
????????? Interrupt:17???????????????????????????????????????
lo??????? Link encap:Local Loopback?
????????? inet addr:127.0.0.1? Mask:255.0.0.0
????????? inet6 addr: ::1/128 Scope:Host????
????????? UP LOOPBACK RUNNING? MTU:16436? Metric:1
????????? RX packets:6544 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:6544 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:0?????????????????????????????
????????? RX bytes:1409100 (1.3 MiB)? TX bytes:1409100 (1.3 MiB)
[root at three ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.5.0 (November 4, 2008)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)????????????????
MII Status: up????????????????????????????????????
MII Polling Interval (ms): 10?????????????????????
Up Delay (ms): 0??????????????????????????????????
Down Delay (ms): 0????????????????????????????????
802.3ad info
LACP rate: slow
Aggregator selection policy (ad_select): stable
Active Aggregator Info:???????????????????????
??????? Aggregator ID: 1??????????????????????
??????? Number of ports: 3????????????????????
??????? Actor Key: 17?????????????????????????
??????? Partner Key: 37823????????????????????
??????? Partner Mac Address: 00:22:57:9a:ea:00
Slave Interface: eth0
MII Status: up??????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:3b:73
Aggregator ID: 1???????????????????
Slave Interface: eth1
MII Status: up??????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:3b:74
Aggregator ID: 1???????????????????
Slave Interface: eth2
MII Status: up??????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:3b:75
Aggregator ID: 1???????????????????
Slave Interface: eth3
MII Status: down????
Link Failure Count: 0
Permanent HW addr: 00:22:15:06:3b:76
Aggregator ID: 4???????????????????
[root at three ~]# ifconfig???????????
bond0???? Link encap:Ethernet? HWaddr 00:22:15:06:3B:73?
????????? inet addr:192.168.0.3? Bcast:192.168.0.255? Mask:255.255.255.0
????????? inet6 addr: fe80::222:15ff:fe06:3b73/64 Scope:Link???????????
????????? UP BROADCAST RUNNING MASTER MULTICAST? MTU:1500? Metric:1????
????????? RX packets:2085357 errors:0 dropped:0 overruns:0 frame:0?????
????????? TX packets:1841322 errors:0 dropped:0 overruns:0 carrier:0???
????????? collisions:0 txqueuelen:0????????????????????????????????????
????????? RX bytes:1354333514 (1.2 GiB)? TX bytes:990593774 (944.7 MiB)
eth0????? Link encap:Ethernet? HWaddr 00:22:15:06:3B:73?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:186719 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:1612447 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000?????????????????????????????
????????? RX bytes:165785794 (158.1 MiB)? TX bytes:950893930 (906.8 MiB)
????????? Interrupt:18?????????????????????????????????????????????????
eth1????? Link encap:Ethernet? HWaddr 00:22:15:06:3B:73?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:1896745 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:34765 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000???????????????????????????
????????? RX bytes:1188319580 (1.1 GiB)? TX bytes:14395233 (13.7 MiB)
????????? Interrupt:19??????????????????????????????????????????????
eth2????? Link encap:Ethernet? HWaddr 00:22:15:06:3B:73?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:1893 errors:0 dropped:0 overruns:0 frame:0??
????????? TX packets:194110 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000
????????? RX bytes:228140 (222.7 KiB)? TX bytes:25304611 (24.1 MiB)
????????? Interrupt:16
eth3????? Link encap:Ethernet? HWaddr 00:22:15:06:3B:73
????????? UP BROADCAST SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:0 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000
????????? RX bytes:0 (0.0 b)? TX bytes:0 (0.0 b)
????????? Interrupt:17
lo??????? Link encap:Local Loopback
????????? inet addr:127.0.0.1? Mask:255.0.0.0
????????? inet6 addr: ::1/128 Scope:Host
????????? UP LOOPBACK RUNNING? MTU:16436? Metric:1
????????? RX packets:6544 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:6544 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:0
????????? RX bytes:1409100 (1.3 MiB)? TX bytes:1409100 (1.3 MiB)
[root at three ~]# uptime
?12:37:32 up 15:06,? 7 users,? load average: 0.00, 0.02, 0.00
** Mobile Mythtv backend.? Raid Storage/Lustre/Mythtv Backup.? Zoneminder
/dev/video0-3 /mnt/ioio/html/zm Lustre Mounted? LAMP webroot=/mnt/ioio/html
Lustre mounted.?
[mythtv at twentythree bin]$ df -h
Filesystem??????????? Size? Used Avail Use% Mounted on
/dev/sda1????????????? 35G? 9.2G?? 24G? 28% /????????
tmpfs??????????????? 1004M?? 12K 1004M?? 1% /dev/shm?
192.168.0.10 at tcp0:/ioio??????????????????????????????
????????????????????? 8.1T? 4.8T? 3.0T? 62% /mnt/ioio
[mythtv at twentythree bin]$ cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.5.0 (November 4, 2008)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)????????????????
MII Status: up????????????????????????????????????
MII Polling Interval (ms): 10?????????????????????
Up Delay (ms): 0??????????????????????????????????
Down Delay (ms): 0????????????????????????????????
802.3ad info
LACP rate: slow
Aggregator selection policy (ad_select): stable
Active Aggregator Info:???????????????????????
??????? Aggregator ID: 1??????????????????????
??????? Number of ports: 2????????????????????
??????? Actor Key: 17?????????????????????????
??????? Partner Key: 37823????????????????????
??????? Partner Mac Address: 00:22:57:9a:ea:00
Slave Interface: eth0
MII Status: up??????
Link Failure Count: 0
Permanent HW addr: 00:22:15:95:2d:23
Aggregator ID: 1???????????????????
Slave Interface: eth1
MII Status: up??????
Link Failure Count: 0
Permanent HW addr: 00:04:e2:d7:33:c3
Aggregator ID: 1???????????????????
[mythtv at twentythree bin]$ ifconfig?
bond0???? Link encap:Ethernet? HWaddr 00:22:15:95:2D:23?
????????? inet addr:192.168.0.23? Bcast:192.168.0.255? Mask:255.255.255.0
????????? inet6 addr: fe80::222:15ff:fe95:2d23/64 Scope:Link????????????
????????? UP BROADCAST RUNNING MASTER MULTICAST? MTU:1500? Metric:1?????
????????? RX packets:591449 errors:0 dropped:0 overruns:0 frame:0???????
????????? TX packets:238413 errors:0 dropped:0 overruns:0 carrier:2?????
????????? collisions:0 txqueuelen:0?????????????????????????????????????
????????? RX bytes:324872352 (309.8 MiB)? TX bytes:33161781 (31.6 MiB)??
eth0????? Link encap:Ethernet? HWaddr 00:22:15:95:2D:23?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:135740 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:130875 errors:0 dropped:0 overruns:0 carrier:2
????????? collisions:0 txqueuelen:1000
????????? RX bytes:8407741 (8.0 MiB)? TX bytes:13738521 (13.1 MiB)
????????? Interrupt:28
eth1????? Link encap:Ethernet? HWaddr 00:22:15:95:2D:23
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:455709 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:107538 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000
????????? RX bytes:316464611 (301.8 MiB)? TX bytes:19423260 (18.5 MiB)
????????? Interrupt:17
lo??????? Link encap:Local Loopback
????????? inet addr:127.0.0.1? Mask:255.0.0.0
????????? inet6 addr: ::1/128 Scope:Host
????????? UP LOOPBACK RUNNING? MTU:16436? Metric:1
????????? RX packets:262 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:262 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:0
????????? RX bytes:15508 (15.1 KiB)? TX bytes:15508 (15.1 KiB)
[mythtv at twentythree bin]$ uptime
?12:47:20 up 13:43, 14 users,? load average: 1.12, 1.14, 1.11
[mythtv at twentythree bin]$ lfs df -h
UUID?????????????????????? bytes??????? Used?? Available Use% Mounted on
ioio-MDT0000_UUID???????? 815.0G????? 374.0M????? 768.1G?? 0% /mnt/ioio[MDT:0]
ioio-OST0000_UUID?????????? 2.7T??????? 1.7T????? 915.7G? 65% /mnt/ioio[OST:0]
ioio-OST0001_UUID?????????? 2.7T??????? 1.8T????? 796.3G? 70% /mnt/ioio[OST:1]
ioio-OST0002_UUID?????????? 2.7T??????? 1.3T??????? 1.3T? 50% /mnt/ioio[OST:2]
filesystem summary:???????? 8.1T??????? 4.7T??????? 2.9T? 62% /mnt/ioio
** Lustre Client.? Mythtv Frontend.? BzFlag 2.4.2 Enabled.? Can be e17 driven. ?
[root at sixteen ~]# df -h
Filesystem??????????? Size? Used Avail Use% Mounted on
/dev/sda1???????????? 143G? 9.0G? 127G?? 7% /????????
tmpfs???????????????? 1.3G?? 88K? 1.3G?? 1% /dev/shm?
192.168.0.10 at tcp0:/ioio??????????????????????????????
????????????????????? 8.1T? 4.8T? 3.0T? 62% /mnt/ioio
[root at sixteen ~]# cat /proc/net/bonding/bond0????????
Ethernet Channel Bonding Driver: v3.5.0 (November 4, 2008)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)????????????????
MII Status: up????????????????????????????????????
MII Polling Interval (ms): 10?????????????????????
Up Delay (ms): 0??????????????????????????????????
Down Delay (ms): 0????????????????????????????????
802.3ad info
LACP rate: slow
Aggregator selection policy (ad_select): stable
Active Aggregator Info:???????????????????????
??????? Aggregator ID: 1??????????????????????
??????? Number of ports: 1????????????????????
??????? Actor Key: 17?????????????????????????
??????? Partner Key: 1????????????????????????
??????? Partner Mac Address: 00:00:00:00:00:00
Slave Interface: eth0
MII Status: up??????
Link Failure Count: 0
Permanent HW addr: 00:14:85:18:83:7a
Aggregator ID: 1???????????????????
Slave Interface: eth1
MII Status: up??????
Link Failure Count: 0
Permanent HW addr: 00:0e:0c:dd:3a:c7
Aggregator ID: 2???????????????????
[root at sixteen ~]# ifconfig?????????
bond0???? Link encap:Ethernet? HWaddr 00:14:85:18:83:7A?
????????? inet addr:192.168.0.16? Bcast:192.168.0.255? Mask:255.255.255.0
????????? inet6 addr: fe80::214:85ff:fe18:837a/64 Scope:Link????????????
????????? UP BROADCAST RUNNING MASTER MULTICAST? MTU:1500? Metric:1?????
????????? RX packets:531666451 errors:42 dropped:0 overruns:0 frame:42??
????????? TX packets:155532069 errors:0 dropped:0 overruns:0 carrier:0??
????????? collisions:0 txqueuelen:0?????????????????????????????????????
????????? RX bytes:561475010002 (522.9 GiB)? TX bytes:650261730391 (605.6 GiB)
eth0????? Link encap:Ethernet? HWaddr 00:14:85:18:83:7A?
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:525163826 errors:42 dropped:0 overruns:0 frame:42
????????? TX packets:155449633 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000
????????? RX bytes:561052441951 (522.5 GiB)? TX bytes:650251508327 (605.5 GiB)
????????? Interrupt:22
eth1????? Link encap:Ethernet? HWaddr 00:14:85:18:83:7A
????????? UP BROADCAST RUNNING SLAVE MULTICAST? MTU:1500? Metric:1
????????? RX packets:6502625 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:82436 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:1000
????????? RX bytes:422568051 (402.9 MiB)? TX bytes:10222064 (9.7 MiB)
lo??????? Link encap:Local Loopback
????????? inet addr:127.0.0.1? Mask:255.0.0.0
????????? inet6 addr: ::1/128 Scope:Host
????????? UP LOOPBACK RUNNING? MTU:16436? Metric:1
????????? RX packets:205748 errors:0 dropped:0 overruns:0 frame:0
????????? TX packets:205748 errors:0 dropped:0 overruns:0 carrier:0
????????? collisions:0 txqueuelen:0
????????? RX bytes:56913227 (54.2 MiB)? TX bytes:56913227 (54.2 MiB)
[root at sixteen ~]# uptime
?12:49:51 up 28 days, 14:59,? 1 user,? load average: 0.08, 0.05, 0.01
[root at sixteen ~]# lfs df -h
UUID?????????????????????? bytes??????? Used?? Available Use% Mounted on
ioio-MDT0000_UUID???????? 815.0G????? 374.0M????? 768.1G?? 0% /mnt/ioio[MDT:0]
ioio-OST0000_UUID?????????? 2.7T??????? 1.7T????? 915.7G? 65% /mnt/ioio[OST:0]
ioio-OST0001_UUID?????????? 2.7T??????? 1.8T????? 796.3G? 70% /mnt/ioio[OST:1]
ioio-OST0002_UUID?????????? 2.7T??????? 1.3T??????? 1.3T? 50% /mnt/ioio[OST:2]
filesystem summary:???????? 8.1T??????? 4.7T??????? 2.9T? 62% /mnt/ioio
** Lustre Client.? Mythtv Backend.
So there are reasons for using raid10 on specific parts of your Lustre
deployment.? I had a few bad drives/sata errors on one box and raid10 saved
everything.? I was able to watch the drive fail after time and finally when it
died I was able to replace while still having Lustre interoperablility.? Still
observing for that same box with another bad drive.? Procurring 1TB 32mb sata II
drives market driven.?
Rebuild time was tolerable but from recollection less than a couple of hours.?
Just one of those tasks every raided machine does once a week anyhow via the
cron resync.
I haven''t lost data with this layout for years and have been able to
adapt, maintain and implement different technologies and updates along the way.?
https://groups.google.com/forum/?fromgroups#!msg/lustre-discuss-list/NfNilGsVYxA/KVCR8nixXbMJ
Would smaller disks be better?? Have you ever tried to make a large raid 5 array
out of 2+TB Sata III drives?? There are some definate changes regarding the
partitioning structure.? I eventually had to just let the installer do all the
work for me when I made a 6TB raid 5 Lustre Backup array.? Smaller disks are
easier from this perspective.?
From this base I attached all HDMI cables.? My LM7600 deploys 3d stereoscopic
images and I''ve even take to researching the production and conversion
of that medium.? Beamsplitter on the RV, lol.? The HDMI is upscaled to 4K by the
Pioneer 1527K and it''s all fed into a dozen KEF 101''s
with 3 subs for 9.1 Stereo Sound utilizing neoX soundtracks from the newest blu
rays.? Waiting on 4K OLED sets to become available.
I routinely get 90+ days of uptime on my Lustre machines and find I''m
more like the Maytag guy when it comes to maintaining the network.? It is just
there and for the most part enjoyable to use.? Not saying that there
haven''t been some challenges when everything hangs in the balance of a
few keystrokes but for the most part it works period.
I would like to expand it all again with Sata III 4TB storage disks and 120 MB
SSD boot drives.? Grab some 10GigE interconnect cards and plum some Bell Axia
into the? house but time will tell.? Having a purpose for the storage is the
best question to ask.? 4x8=32TB per OST x 3 OST = 96TB linear raid.? That would
be personal use future proofed with room for offering storage services.?
Perhaps the smallest clients of all are rooted androids and tablets accessing
the network via apps to the SMB/NMB shared Lustre Mount.? This ties the entire
configuration together even driving down the road rebooting servers or
restarting apache.? Some apps like mythfrontend and mythmote are far superior to
anything produced by commercial vendors for products.? Most commercial apps are
chalk full of thoughtless design much like smart TV''s and for the most
part commercialy available PVRs.?
>________________________________
> From: Jon Yeargers <yeargers at ohsu.edu>
>To: "lustre-discuss at lists.lustre.org" <lustre-discuss at
lists.lustre.org>
>Sent: Friday, December 28, 2012 11:37:06 AM
>Subject: [Lustre-discuss] RAID usage?
>
>
>Im curious to hear what RAID configurations people are using in existing
Lustre installations.
>?
>I?ve been collecting bids to build an array and assuming RAID6/1Tb but the
more I read dire warnings about losing data and ?days to rebuild an array? the
more I?m questioning my approach.
>?
>Would RAID 10 be better? Smaller disks?
>_______________________________________________
>Lustre-discuss mailing list
>Lustre-discuss at lists.lustre.org
>http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20130122/66945790/attachment-0001.html