This is a lightly loaded v20z but it has zfs across its two disks.. its hung (requiring a power cycle) twice since running 5.11 opensol-20060904 the last time I had a `vmstat 1` running... nice page rates right before death :-) kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr f0 m0 s0 s1 in sy cs us sy id 0 0 0 1788296 50360 0 3 4 0 0 0 0 0 1 0 63 1962 273 5435 0 2 97 0 0 0 1788296 50428 0 3 0 0 0 0 0 0 0 0 0 1075 323 446 1 1 98 0 0 0 1788296 50504 0 3 4 0 0 0 0 0 1 0 0 929 435 385 0 1 99 0 0 0 1788296 50432 0 2 0 0 0 0 0 0 0 0 0 921 472 517 1 0 99 0 0 0 1788296 50436 0 1 0 0 0 0 0 0 0 0 0 900 426 361 0 1 98 20 0 0 1788296 50000 0 3 0 0 0 0 0 0 0 0 9 864 230 236 0 49 51 0 0 0 1788296 50272 0 5 8 0 0 0 0 0 0 0 45 2460 373 4372 1 20 79 0 0 0 1788296 50252 2 5 0 0 0 0 0 0 0 0 0 1066 467 469 1 1 98 0 0 0 1788040 50304 0 29 0 0 0 0 0 0 0 0 0 931 355 353 0 1 99 0 0 0 1788040 50120 0 0 0 0 0 0 0 0 0 0 0 693 348 426 0 1 99 0 0 0 1787784 49972 0 83 4 0 0 0 0 0 1 0 0 1048 376 476 0 1 98 0 0 0 1787784 49836 0 17 0 0 0 0 0 0 0 0 39 1676 377 4254 0 2 98 0 0 0 1787528 49760 2 18 0 0 0 0 0 0 0 0 2 790 313 462 0 0 99 0 0 0 1787528 49748 1 26 0 0 0 0 0 0 0 0 0 833 432 513 1 1 98 0 0 0 1787528 49444 0 66 0 0 0 0 0 0 0 0 0 1176 467 563 0 2 98 0 0 0 1787016 49148 0 87 0 0 0 0 0 0 0 0 0 1239 422 460 0 1 99 0 0 0 1786760 49128 0 27 0 252 256 0 1437 0 5 0 89 2005 386 5124 1 5 94 0 0 0 1786760 49148 0 87 0 214 214 0 1900 0 4 0 0 1136 490 505 1 1 98 0 0 0 1786504 49148 0 32 0 166 166 0 952 0 3 0 0 789 336 397 0 1 99 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr f0 m0 s0 s1 in sy cs us sy id 0 0 0 1786504 49148 0 1 0 0 0 0 0 0 0 0 0 675 263 405 28 1 71 0 0 0 1786504 49148 0 0 0 0 0 0 0 0 0 0 40 845 221 2499 50 1 49 0 0 0 1786504 49148 1 50 0 333 337 0 2762 0 6 0 51 1584 479 4976 43 4 53 0 0 0 1739020 49092 0 4 0 1935 1978 0 13663 0 36 0 98 9032 1183 28470 1 11 87 0 0 0 1738764 49796 0 92 0 8264 8724 0 80894 0 152 0 188 13118 1118 36769 0 14 86 0 0 0 1738764 49368 0 0 0 4198 4364 0 34772 0 77 0 135 10556 1592 31932 0 12 87 1 0 0 1738764 47252 19 28 32 5395 5675 0 56668 0 105 0 150 11534 1135 34519 1 17 82 0 0 0 1738764 49032 3 17 4 6933 7399 0 173424 0 129 0 339 16913 1046 40602 0 24 75 0 0 0 1738764 52536 17 23 43 4717 4812 0 32283 0 93 0 265 10603 3460 31909 1 20 79 0 0 0 1738764 48520 70 81 28 4820 5110 0 64402 0 96 0 57 4233 12106 16099 2 28 69 0 0 0 1738764 48556 257 290 134 3838 3965 0 31038 0 104 0 55 5518 8696 19797 6 22 72 0 0 0 1738764 48868 118 137 36 5349 5500 0 43605 0 109 0 123 12934 786 32379 7 14 79 0 0 0 1738764 50016 196 258 115 3909 4063 0 39666 0 104 0 292 18954 873 49225 1 20 79 0 0 0 1738764 49380 27 38 20 8906 9125 0 77911 0 172 0 144 17529 856 41523 1 16 83 1 0 0 1738764 48120 3 56 16 5288 5296 0 80894 0 103 0 131 13210 2877 39454 0 19 80 0 0 0 1738508 48992 41 71 24 7543 7606 0 80807 0 147 0 161 18161 1453 55461 0 20 79 1 0 0 1738508 47964 24 55 8 7873 8079 0 72299 0 151 0 199 11502 1815 51196 1 20 79 0 0 0 1738508 49316 128 208 206 6420 6780 0 92682 0 178 0 170 12950 1692 46450 1 19 80 0 0 0 1738252 48916 84 204 259 7582 7976 0 125476 0 201 0 106 8189 3275 51597 1 24 75 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr f0 m0 s0 s1 in sy cs us sy id 0 0 0 1737996 51224 54 222 670 3283 3283 0 0 0 229 0 39 3070 6590 11314 1 15 84 0 0 0 1737996 49616 128 347 855 3160 3410 0 44589 0 270 0 50 4663 5857 10050 2 18 80 0 0 0 1737996 52088 64 155 318 0 0 0 0 0 78 0 163 7183 3732 42740 3 19 78 0 0 0 1737996 50912 186 289 390 3038 3633 0 79700 0 155 0 159 15334 2993 35202 2 17 81 0 0 0 1737996 49272 276 459 724 1120 1508 0 85377 0 195 0 36 3901 11036 10635 2 28 70 0 0 0 1737996 48100 49 178 506 6678 6933 0 288123 0 253 0 16 2761 12678 7663 3 37 59 0 0 0 1737996 46004 240 446 808 11869 14016 0 560661 0 420 0 7 2597 8195 3651 2 26 72 0 0 0 1737996 47200 338 527 732 3268 3419 0 243514 0 255 0 106 7373 1339 17765 0 13 87 0 0 0 1737996 44468 172 344 654 2187 2542 0 308709 0 207 0 257 15433 750 43955 0 24 75 0 0 0 1737996 46464 108 329 866 2419 3188 0 383872 0 276 0 106 11104 2767 29523 1 18 81 0 0 0 1737996 43472 468 859 1574 2305 3138 0 603123 0 432 0 25 4609 1526 11965 1 16 83 0 0 0 1737996 39192 804 1215 1574 4183 5446 0 1069089 0 481 0 26 3628 1930 8346 1 22 77 0 0 0 1737996 41196 589 1007 1647 4338 6138 0 882406 0 456 0 115 4310 1449 11585 1 17 82 0 0 0 1737996 41508 458 951 1900 2299 3409 0 752009 0 491 0 58 5949 2451 9800 7 19 75 0 0 0 1737996 39720 1253 1725 1850 2903 4484 0 861155 0 520 0 18 2956 170 4665 26 13 61 0 0 0 1737996 35812 2204 2735 1988 4067 7070 0 1260373 0 585 0 8 3351 330 2350 36 19 45 0 0 0 1737996 36568 1664 2094 1676 4901 8843 0 1144375 0 495 0 107 3586 247 7572 38 18 44 0 0 0 1737996 35688 1304 1754 1820 4298 7738 0 1159977 0 516 0 16 2747 381 2056 45 15 40 0 0 0 1737996 38288 1459 1946 1928 5235 7414 0 1032865 0 529 0 51 4145 1783 10283 5 20 75 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr f0 m0 s0 s1 in sy cs us sy id 0 0 0 1737996 37016 1583 1904 1225 4375 7532 0 1176681 0 358 0 59 7499 1266 14909 1 21 78 0 0 0 1737996 37984 2012 2275 1131 7362 10489 0 1024621 0 429 0 77 6145 283 11650 1 18 81 0 0 0 1737996 37012 1404 1709 1209 2992 5163 0 936718 0 352 0 46 6200 214 11146 1 18 81 0 0 0 1737996 41396 1967 2225 1000 4808 8102 0 743902 0 349 0 163 8749 1480 18514 1 23 76 0 0 0 1737996 36576 1429 1825 1502 4996 8438 0 1047181 0 481 0 20 3686 5974 4079 1 25 74 11 0 0 1737996 41344 752 1004 942 3215 5617 0 833705 0 282 0 81 3183 1684 5604 6 51 43 0 0 0 1737996 38268 1762 2086 1293 4368 6994 0 632236 0 412 0 49 5125 803 10426 1 26 73 0 0 0 1737996 40552 1621 2059 1675 2318 3639 0 819196 0 469 0 13 3153 1840 5763 3 16 81 0 0 0 1737996 39592 678 959 1076 3786 5592 0 1057687 0 358 0 41 4615 4122 11673 6 23 70 0 0 0 1737996 42072 701 927 889 3076 4568 0 581368 0 256 0 148 5916 1137 17732 1 16 83 0 0 0 1737996 39456 1308 1668 1474 3074 4800 0 776042 0 391 0 87 5668 2964 12825 2 20 78 0 0 0 1737996 42340 994 1358 1433 4538 7405 0 851064 0 468 0 50 4849 982 12537 10 19 70 0 0 0 1737996 39028 985 1316 1237 3779 6558 0 919575 0 292 0 159 9563 2600 24289 8 25 67 0 0 0 1737996 36284 1378 1689 1197 3085 6085 0 1083825 0 428 0 52 8116 1926 17117 1 28 71 0 0 0 1737996 38084 1109 1425 1231 4184 7403 0 1067642 0 379 0 58 5828 1989 13336 1 22 77 0 0 0 1737996 39340 1435 1785 1382 2784 5314 0 1104102 0 351 0 169 5820 2588 15140 3 25 72 0 0 0 1737996 42520 998 1329 1206 2927 5915 0 813373 0 364 0 52 5372 1554 10111 20 18 62 0 0 0 1737996 42308 686 989 1181 1804 3123 0 689227 0 335 0 68 4869 3065 14458 0 20 79 0 0 0 1737996 36700 1557 2013 1732 3646 5381 0 947088 0 488 0 24 3313 3525 8203 1 20 78 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr f0 m0 s0 s1 in sy cs us sy id 0 0 0 1737996 41624 1345 1720 1385 4139 7430 0 851778 0 420 0 126 5335 854 15992 2 20 79 0 0 0 1737996 41164 1090 1457 1325 3546 5616 0 792982 0 392 0 40 4399 1760 11303 3 18 80 0 0 0 1737996 39340 1413 1805 1563 3237 5512 0 932877 0 434 0 26 3023 1278 5994 1 16 83 0 0 0 1737996 38536 1432 1831 1690 3518 6445 0 962633 0 458 0 62 5221 2475 13104 1 20 79 0 0 0 1737996 38192 1332 1660 1271 4506 7457 0 1053449 0 397 0 117 5203 1407 17779 4 20 75 0 0 0 1737996 41408 570 918 1368 3302 6720 0 828377 0 390 0 43 4654 1523 12792 14 19 68 0 0 0 1737996 41216 810 1196 1549 1968 3104 0 801174 0 395 0 43 3677 3860 8482 1 19 80 0 0 0 1737996 38820 1027 1382 1406 3338 6048 0 984334 0 418 0 72 5297 2362 17174 2 23 75 0 0 0 1737996 35888 1618 1965 1345 4156 6235 0 930311 0 392 0 71 7234 376 12623 1 17 82 0 0 0 1737996 37564 1117 1428 1195 5001 8170 0 1078505 0 367 0 138 6595 1017 14795 1 21 79 0 0 0 1737996 39908 587 958 1393 2323 4184 0 806301 0 379 0 46 4427 836 14156 14 16 70 0 0 0 1737996 38792 730 1119 1540 1888 2901 0 942703 0 417 0 13 3172 779 12516 23 16 61 0 0 0 1737996 40740 937 1213 1071 2638 4158 0 904782 0 320 0 135 7463 1092 19341 6 20 74 0 0 0 1737996 36316 1245 1533 1150 2563 3735 0 895317 0 328 0 64 6120 2348 17895 2 22 76 0 0 0 1737996 29432 1120 1393 1069 5901 8907 0 1473395 0 380 0 71 5981 3922 22592 2 38 60 0 0 0 1737996 29768 1295 1550 902 6573 9050 0 1667068 0 351 0 63 6893 234 14724 1 24 76 0 0 0 1737996 31792 1629 1994 1154 3355 6920 0 1862622 0 362 0 142 6872 639 17371 0 28 71 0 0 0 1737996 31664 1408 1774 1332 4955 7260 0 1424338 0 437 0 30 4402 2179 8581 1 30 69 0 0 0 1737996 31628 1641 2062 1607 2954 5499 0 1436376 0 464 0 20 3940 2732 8890 2 27 71 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr f0 m0 s0 s1 in sy cs us sy id 0 0 0 1737996 29876 1503 1909 1577 4670 7943 0 1625249 0 473 0 44 4686 4239 10704 1 31 67 0 0 0 1737988 31280 1195 1476 1069 5244 9263 0 1580549 0 382 0 259 7899 1440 16329 0 31 68 0 0 0 1737984 35064 2334 2835 1774 4485 6970 0 1294728 0 530 0 19 3506 2042 8447 1 26 73 0 0 0 1737980 32196 1862 2246 1497 4827 7029 0 1273657 0 472 0 35 3896 669 7950 1 21 78 0 0 0 1737980 31132 1699 2016 1160 5787 9794 0 1728667 0 413 0 75 6999 427 17545 1 28 71 0 0 0 1737980 30248 1678 1910 939 6128 9670 0 1436419 0 352 0 156 11027 1323 40795 1 34 64 1 0 0 1737980 26000 857 1145 1040 2914 5581 0 2062464 0 341 0 117 5911 1766 37645 1 37 62 0 0 0 1737980 25696 1566 2023 1686 3900 6118 0 1919406 0 497 0 38 3829 2202 11821 1 34 65 0 0 0 1737980 26100 1693 2202 1900 4946 8432 0 2213808 0 554 0 32 3455 1889 5308 0 32 67 0 0 0 1737980 25064 1851 2427 2146 4694 9030 0 1924543 0 583 0 42 3140 1531 5200 1 28 70 0 0 0 1737980 29152 1392 1747 1317 5160 8035 0 1927262 0 436 0 110 4511 2861 11445 0 34 65 0 0 0 1737980 26296 2114 2559 1714 6214 8830 0 1794665 0 562 0 3 2902 3232 1775 1 29 69 0 0 0 1737980 29704 1374 1658 1108 6694 8768 0 1851994 0 418 0 95 8884 1157 20949 0 34 66 0 0 0 1737980 27588 886 1179 1132 3351 5420 0 1782578 0 346 0 73 7990 177 18377 0 27 72 0 0 0 1737980 25380 942 1185 868 2891 5270 0 1873857 0 290 0 150 8230 416 24604 0 30 70 7 0 0 1737980 26548 554 700 526 1895 3708 0 2051318 0 191 0 45 2955 348 6739 0 73 27 0 0 0 1737980 26972 1416 1735 1248 2409 4162 0 1971194 0 365 0 76 7122 624 16685 0 38 61 5 0 0 1737980 27632 815 1088 903 2485 5854 0 1930024 0 288 0 95 6784 2129 22576 0 37 63 0 0 0 1737980 25920 1104 1605 1853 2905 5252 0 1952915 0 513 0 52 3612 3477 7675 0 36 64 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr f0 m0 s0 s1 in sy cs us sy id 0 0 0 1737980 26484 1627 2071 1738 4272 6662 0 1924110 0 554 0 3 2827 2278 4347 1 27 71 0 0 0 1737980 26936 1295 1809 1930 5095 7790 0 1980392 0 587 0 4 2672 3429 1329 0 27 72 0 0 0 1737980 26232 1556 2035 1792 3434 5347 0 1853738 0 526 0 5 2517 3681 1935 1 32 67 0 0 0 1737980 23496 1827 2282 1746 5985 9650 0 2021765 0 564 0 12 2901 5937 2928 1 36 63 0 0 0 1737980 24192 1958 2414 1691 6457 11474 0 2038953 0 542 0 11 2905 5439 2346 1 34 65 0 0 0 1737980 28808 2466 2872 1476 5040 7992 0 1739312 0 472 0 98 4261 5124 5642 1 35 63 0 0 0 1737980 25880 2516 3023 1827 6086 9784 0 1629361 0 573 0 6 2899 4008 1967 1 30 69 10 0 0 1737980 25228 2616 3051 1589 5928 10362 0 1894884 0 519 0 10 3013 3370 2690 2 36 62 1 0 0 1737980 24380 2089 2516 1690 7173 10993 0 1816530 0 554 0 2 2813 3675 1446 1 38 60 0 0 0 1737980 26312 2608 2957 1378 7560 10943 0 1838124 0 490 0 129 4461 4404 8988 1 35 63 0 0 0 1737980 27472 2245 2640 1480 6299 9950 0 1808604 0 500 0 10 3067 5691 3397 2 33 65 0 0 0 1737980 26656 2562 2939 1395 7041 11909 0 1946556 0 481 0 23 3862 5447 5017 2 36 62 0 0 0 1737980 29244 2081 2432 1217 8256 13807 0 1656034 0 461 0 10 2815 5462 3262 1 33 66 0 0 0 1737980 27512 1613 1981 1384 4941 7528 0 1715102 0 441 0 16 2944 4432 3396 1 30 69 0 0 0 1737980 23156 1432 1780 1300 4984 8804 0 2179026 0 446 0 149 3794 3933 10033 0 38 62 0 0 0 1737980 22820 1660 2106 1567 3777 8171 0 2300020 0 480 0 146 3351 3966 5686 1 36 62 6 0 0 1737980 18184 1543 2156 2188 5310 11018 0 2567140 0 651 0 265 4641 5886 4403 1 43 56 0 0 71 1737980 15544 1252 1806 2049 4574 8285 0 3177604 0 621 0 329 5630 4118 10038 1 46 53 7 0 71 1737980 15012 1236 1831 2148 5191 7949 0 3053756 0 640 0 469 5459 6214 9385 1 54 45 kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr f0 m0 s0 s1 in sy cs us sy id 0 0 72 1737980 13668 1199 1905 2700 4986 7886 0 3547972 0 802 0 442 4178 5700 4247 1 53 46 0 0 73 1737980 12268 777 1349 2067 3434 5514 0 3349511 0 591 0 446 4937 4168 6505 1 50 49 0 0 73 1737980 12220 831 1272 1505 2891 5547 0 4674204 0 411 0 383 3869 2056 6338 1 53 46 0 0 73 1737980 12884 1091 1589 1660 2892 5672 0 4420706 0 522 0 479 5359 3064 9363 1 51 48 0 0 73 1737980 12248 1060 1595 1754 3332 6287 0 5512981 0 475 0 438 4336 2680 6287 0 61 38 0 0 73 1737980 12276 768 1025 829 1708 3704 0 5378965 0 292 0 340 2876 927 6153 0 55 45 0 0 73 1737980 12204 773 1147 1382 2794 5208 0 5604091 0 255 0 225 2827 1838 3338 1 59 41 0 0 73 1737980 12544 638 946 1183 1600 3198 0 4989934 0 710 0 557 2928 2039 4436 0 52 48 Read from remote host w2: Connection timed out Connection to w2 closed. rob at logan[5:32am]~