Hi, I want to know all files that are located on a specific OST, but neither ''lfs find ...'' nor ''lfs getstripe ...'' can give me an answer. I do not use striping. This is what I tried: gnode30 ~ # lfs df -h UUID bytes Used Available Use% Mounted on scratch-MDT0000_UUID 710.0G 41.0G 668.9G 5% /mnt/ lustre[MDT:0] scratch-OST0001_UUID 6.3T 327.3G 5.9T 5% /mnt/ lustre[OST:1] scratch-OST0002_UUID 6.3T 327.3G 5.9T 5% /mnt/ lustre[OST:2] scratch-OST0003_UUID 6.3T 328.3G 5.9T 5% /mnt/ lustre[OST:3] scratch-OST0004_UUID 6.3T 327.3G 5.9T 5% /mnt/ lustre[OST:4] scratch-OST0006_UUID 6.3T 327.3G 5.9T 5% /mnt/ lustre[OST:6] scratch-OST0007_UUID 6.3T 326.3G 5.9T 5% /mnt/ lustre[OST:7] filesystem summary: 37.6T 1.9T 35.7T 5% /mnt/lustre gnode30 ~ # lctl dl 0 UP mgc MGC10.1.140.1 at tcp 5c3be07b-fab6-390e-0fe6-044a5dd0b5c9 5 1 UP lov scratch-clilov-ffff8100de423000 9ad18aa7- e0e4-9766-0f26-9cabac1f9a28 4 2 UP mdc scratch-MDT0000-mdc-ffff8100de423000 9ad18aa7- e0e4-9766-0f26-9cabac1f9a28 5 3 UP osc scratch-OST0001-osc-ffff8100de423000 9ad18aa7- e0e4-9766-0f26-9cabac1f9a28 5 4 UP osc scratch-OST0002-osc-ffff8100de423000 9ad18aa7- e0e4-9766-0f26-9cabac1f9a28 5 5 UP osc scratch-OST0003-osc-ffff8100de423000 9ad18aa7- e0e4-9766-0f26-9cabac1f9a28 5 6 UP osc scratch-OST0004-osc-ffff8100de423000 9ad18aa7- e0e4-9766-0f26-9cabac1f9a28 5 7 UP osc scratch-OST0006-osc-ffff8100de423000 9ad18aa7- e0e4-9766-0f26-9cabac1f9a28 5 8 UP osc scratch-OST0007-osc-ffff8100de423000 9ad18aa7- e0e4-9766-0f26-9cabac1f9a28 5 gnode30 ~ # cd /mnt/lustre/ gnode30 lustre # ls -lh total 5.9G -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero1 -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero2 -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero3 -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero4 -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero5 -rw-r--r-- 1 root root 1000M Jan 29 19:42 zero6 gnode30 lustre # lfs find -O scratch-OST0003_UUID /mnt/lustre gnode30 lustre # lfs find -O 7 /mnt/lustre gnode30 lustre # lfs getstripe -O scratch-OST0003_UUID /mnt/lustre error: setup_obd_uuid: unknown obduuid: scratch-OST0003_UUID /mnt/lustre/zero1 /mnt/lustre/zero2 /mnt/lustre/zero3 /mnt/lustre/zero4 /mnt/lustre/zero5 /mnt/lustre/zero6 gnode30 lustre # Am I missing the right syntax? Is this only possible if I use striping? Do I use the wrong UUID, how to determine the right UUID? Thanks, Anselm Strauss
On Jan 29, 2008 19:57 +0100, Anselm Strauss wrote:> I want to know all files that are located on a specific OST, but > neither ''lfs find ...'' nor ''lfs getstripe ...'' can give me an answer. > I do not use striping. This is what I tried: > > gnode30 ~ # lfs df -h > UUID bytes Used Available Use% Mounted on > scratch-MDT0000_UUID 710.0G 41.0G 668.9G 5% /mnt/ > lustre[MDT:0] > scratch-OST0001_UUID 6.3T 327.3G 5.9T 5% /mnt/ > lustre[OST:1] > scratch-OST0002_UUID 6.3T 327.3G 5.9T 5% /mnt/ > lustre[OST:2] > scratch-OST0003_UUID 6.3T 328.3G 5.9T 5% /mnt/ > lustre[OST:3] > scratch-OST0004_UUID 6.3T 327.3G 5.9T 5% /mnt/ > lustre[OST:4] > scratch-OST0006_UUID 6.3T 327.3G 5.9T 5% /mnt/ > lustre[OST:6] > scratch-OST0007_UUID 6.3T 326.3G 5.9T 5% /mnt/ > lustre[OST:7] > > filesystem summary: 37.6T 1.9T 35.7T 5% /mnt/lustre > > gnode30 lustre # lfs find -O scratch-OST0003_UUID /mnt/lustre > gnode30 lustre # lfs find -O 7 /mnt/lustre > gnode30 lustre # lfs getstripe -O scratch-OST0003_UUID /mnt/lustre > error: setup_obd_uuid: unknown obduuid: scratch-OST0003_UUID > /mnt/lustre/zero1 > /mnt/lustre/zero2 > /mnt/lustre/zero3 > /mnt/lustre/zero4 > /mnt/lustre/zero5 > /mnt/lustre/zero6Try using just "lfs getstripe /mnt/lustre" as a starter. It will print out the list of OST UUIDs at the start. It would appear that your syntax is correct, so I''m not sure what is going wrong... $ lfs df UUID 1K-blocks Used Available Use% Mounted on mds-myth-0_UUID 9174328 677728 8496600 7% /myth[MDT:0] ost-myth-0_UUID 292223856 247877024 44346832 84% /myth[OST:0] ost-myth-1_UUID 94442984 91277176 3165808 96% /myth[OST:1] ost-myth-2_UUID 487388376 332816808 154571568 68% /myth[OST:2] ost-myth-3_UUID 487865304 347373400 140491904 71% /myth[OST:3] filesystem summary: 1361920520 1019344408 342576112 74% /myth $ cat /proc/fs/lustre/version lustre: 1.6.4.2 kernel: 47 build: b_release_1_6_4-20080123034511-CHANGED-.usr.src.linux-2.6.16.46-0.14 $ lfs getstripe /myth/tv/1017_20080129100000.mpg OBDS: 0: ost-myth-0_UUID ACTIVE 1: ost-myth-1_UUID ACTIVE 2: ost-myth-2_UUID ACTIVE 3: ost-myth-3_UUID ACTIVE /myth/tv/1017_20080129100000.mpg obdidx objid objid group 2 527023 0x80aaf 0 1 531093 0x81a95 0 $ lfs getstripe -O ost-myth-1_UUID /myth/tv | head /myth/tv/1017_20080129100000.mpg /myth/tv/1017_20071101143000.mpg.png /myth/tv/1037_20080119013000.mpg.png /myth/tv/1017_20071219103000.mpg /myth/tv/1008_20080102200000.mpg /myth/tv/1039_20080123131300.mpg.png /myth/tv/1014_20070905170000.mpg.png /myth/tv/1039_20070914160000.mpg.png /myth/tv/1014_20071009090000.mpg.png /myth/tv/1017_20070917180000.mpg Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
Anselm Strauss wrote:> Hi, > > I want to know all files that are located on a specific OST, but > neither ''lfs find ...'' nor ''lfs getstripe ...'' can give me an answer. > I do not use striping. This is what I tried: > > gnode30 ~ # lfs df -h > UUID bytes Used Available Use% Mounted on > scratch-MDT0000_UUID 710.0G 41.0G 668.9G 5% /mnt/ > lustre[MDT:0] > scratch-OST0001_UUID 6.3T 327.3G 5.9T 5% /mnt/ > lustre[OST:1] > scratch-OST0002_UUID 6.3T 327.3G 5.9T 5% /mnt/ > lustre[OST:2] > scratch-OST0003_UUID 6.3T 328.3G 5.9T 5% /mnt/ > lustre[OST:3] > scratch-OST0004_UUID 6.3T 327.3G 5.9T 5% /mnt/ > lustre[OST:4] > scratch-OST0006_UUID 6.3T 327.3G 5.9T 5% /mnt/ > lustre[OST:6] > scratch-OST0007_UUID 6.3T 326.3G 5.9T 5% /mnt/ > lustre[OST:7] > > filesystem summary: 37.6T 1.9T 35.7T 5% /mnt/lustre > > gnode30 ~ # lctl dl > 0 UP mgc MGC10.1.140.1 at tcp 5c3be07b-fab6-390e-0fe6-044a5dd0b5c9 5 > 1 UP lov scratch-clilov-ffff8100de423000 9ad18aa7- > e0e4-9766-0f26-9cabac1f9a28 4 > 2 UP mdc scratch-MDT0000-mdc-ffff8100de423000 9ad18aa7- > e0e4-9766-0f26-9cabac1f9a28 5 > 3 UP osc scratch-OST0001-osc-ffff8100de423000 9ad18aa7- > e0e4-9766-0f26-9cabac1f9a28 5 > 4 UP osc scratch-OST0002-osc-ffff8100de423000 9ad18aa7- > e0e4-9766-0f26-9cabac1f9a28 5 > 5 UP osc scratch-OST0003-osc-ffff8100de423000 9ad18aa7- > e0e4-9766-0f26-9cabac1f9a28 5 > 6 UP osc scratch-OST0004-osc-ffff8100de423000 9ad18aa7- > e0e4-9766-0f26-9cabac1f9a28 5 > 7 UP osc scratch-OST0006-osc-ffff8100de423000 9ad18aa7- > e0e4-9766-0f26-9cabac1f9a28 5 > 8 UP osc scratch-OST0007-osc-ffff8100de423000 9ad18aa7- > e0e4-9766-0f26-9cabac1f9a28 5 > gnode30 ~ # cd /mnt/lustre/ > gnode30 lustre # ls -lh > total 5.9G > -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero1 > -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero2 > -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero3 > -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero4 > -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero5 > -rw-r--r-- 1 root root 1000M Jan 29 19:42 zero6 > gnode30 lustre # lfs find -O scratch-OST0003_UUID /mnt/lustre > gnode30 lustre # lfs find -O 7 /mnt/lustre > gnode30 lustre # lfs getstripe -O scratch-OST0003_UUID /mnt/lustre > error: setup_obd_uuid: unknown obduuid: scratch-OST0003_UUID > /mnt/lustre/zero1 > /mnt/lustre/zero2 > /mnt/lustre/zero3 > /mnt/lustre/zero4 > /mnt/lustre/zero5 > /mnt/lustre/zero6 > gnode30 lustre # > > Am I missing the right syntax? Is this only possible if I use > striping? Do I use the wrong UUID, how to determine the right UUID?Use the fourth field in ''lctl dl'' lfs getstripe -r --obd scratch-OST0003-osc-ffff8100de423000 /mnt/lustre should work. cliffw> > Thanks, > Anselm Strauss > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss
On 30.01.2008, at 02:09, Cliff White wrote:> Anselm Strauss wrote: >> Hi, >> I want to know all files that are located on a specific OST, but >> neither ''lfs find ...'' nor ''lfs getstripe ...'' can give me an >> answer. I do not use striping. This is what I tried: >> gnode30 ~ # lfs df -h >> UUID bytes Used Available Use% Mounted on >> scratch-MDT0000_UUID 710.0G 41.0G 668.9G 5% /mnt/ >> lustre[MDT:0] >> scratch-OST0001_UUID 6.3T 327.3G 5.9T 5% /mnt/ >> lustre[OST:1] >> scratch-OST0002_UUID 6.3T 327.3G 5.9T 5% /mnt/ >> lustre[OST:2] >> scratch-OST0003_UUID 6.3T 328.3G 5.9T 5% /mnt/ >> lustre[OST:3] >> scratch-OST0004_UUID 6.3T 327.3G 5.9T 5% /mnt/ >> lustre[OST:4] >> scratch-OST0006_UUID 6.3T 327.3G 5.9T 5% /mnt/ >> lustre[OST:6] >> scratch-OST0007_UUID 6.3T 326.3G 5.9T 5% /mnt/ >> lustre[OST:7] >> filesystem summary: 37.6T 1.9T 35.7T 5% /mnt/lustre >> gnode30 ~ # lctl dl >> 0 UP mgc MGC10.1.140.1 at tcp 5c3be07b-fab6-390e-0fe6-044a5dd0b5c9 5 >> 1 UP lov scratch-clilov-ffff8100de423000 9ad18aa7- >> e0e4-9766-0f26-9cabac1f9a28 4 >> 2 UP mdc scratch-MDT0000-mdc-ffff8100de423000 9ad18aa7- >> e0e4-9766-0f26-9cabac1f9a28 5 >> 3 UP osc scratch-OST0001-osc-ffff8100de423000 9ad18aa7- >> e0e4-9766-0f26-9cabac1f9a28 5 >> 4 UP osc scratch-OST0002-osc-ffff8100de423000 9ad18aa7- >> e0e4-9766-0f26-9cabac1f9a28 5 >> 5 UP osc scratch-OST0003-osc-ffff8100de423000 9ad18aa7- >> e0e4-9766-0f26-9cabac1f9a28 5 >> 6 UP osc scratch-OST0004-osc-ffff8100de423000 9ad18aa7- >> e0e4-9766-0f26-9cabac1f9a28 5 >> 7 UP osc scratch-OST0006-osc-ffff8100de423000 9ad18aa7- >> e0e4-9766-0f26-9cabac1f9a28 5 >> 8 UP osc scratch-OST0007-osc-ffff8100de423000 9ad18aa7- >> e0e4-9766-0f26-9cabac1f9a28 5 >> gnode30 ~ # cd /mnt/lustre/ >> gnode30 lustre # ls -lh >> total 5.9G >> -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero1 >> -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero2 >> -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero3 >> -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero4 >> -rw-r--r-- 1 root root 1000M Jan 29 19:41 zero5 >> -rw-r--r-- 1 root root 1000M Jan 29 19:42 zero6 >> gnode30 lustre # lfs find -O scratch-OST0003_UUID /mnt/lustre >> gnode30 lustre # lfs find -O 7 /mnt/lustre >> gnode30 lustre # lfs getstripe -O scratch-OST0003_UUID /mnt/lustre >> error: setup_obd_uuid: unknown obduuid: scratch-OST0003_UUID >> /mnt/lustre/zero1 >> /mnt/lustre/zero2 >> /mnt/lustre/zero3 >> /mnt/lustre/zero4 >> /mnt/lustre/zero5 >> /mnt/lustre/zero6 >> gnode30 lustre # >> Am I missing the right syntax? Is this only possible if I use >> striping? Do I use the wrong UUID, how to determine the right UUID? > > Use the fourth field in ''lctl dl'' > lfs getstripe -r --obd scratch-OST0003-osc-ffff8100de423000 /mnt/ > lustre > > should work.This didn''t help either, the UUID is still unknown: gnode30 ~ # lfs getstripe -r --obd scratch-OST0003-osc- ffff8100de423000 /mnt/lustre error: setup_obd_uuid: unknown obduuid: scratch-OST0003-osc- ffff8100de423000 /mnt/lustre default stripe_count: 6 stripe_size: 0 stripe_offset: 1 /mnt/lustre/zero2 /mnt/lustre/zero gnode30 ~ #> cliffw > >> Thanks, >> Anselm StraussThanks for your help, Anselm Strauss
On 30.01.2008, at 00:03, Andreas Dilger wrote:> On Jan 29, 2008 19:57 +0100, Anselm Strauss wrote: >> I want to know all files that are located on a specific OST, but >> neither ''lfs find ...'' nor ''lfs getstripe ...'' can give me an answer. >> I do not use striping. This is what I tried: >> >> gnode30 ~ # lfs df -h >> UUID bytes Used Available Use% Mounted on >> scratch-MDT0000_UUID 710.0G 41.0G 668.9G 5% /mnt/ >> lustre[MDT:0] >> scratch-OST0001_UUID 6.3T 327.3G 5.9T 5% /mnt/ >> lustre[OST:1] >> scratch-OST0002_UUID 6.3T 327.3G 5.9T 5% /mnt/ >> lustre[OST:2] >> scratch-OST0003_UUID 6.3T 328.3G 5.9T 5% /mnt/ >> lustre[OST:3] >> scratch-OST0004_UUID 6.3T 327.3G 5.9T 5% /mnt/ >> lustre[OST:4] >> scratch-OST0006_UUID 6.3T 327.3G 5.9T 5% /mnt/ >> lustre[OST:6] >> scratch-OST0007_UUID 6.3T 326.3G 5.9T 5% /mnt/ >> lustre[OST:7] >> >> filesystem summary: 37.6T 1.9T 35.7T 5% /mnt/lustre >> >> gnode30 lustre # lfs find -O scratch-OST0003_UUID /mnt/lustre >> gnode30 lustre # lfs find -O 7 /mnt/lustre >> gnode30 lustre # lfs getstripe -O scratch-OST0003_UUID /mnt/lustre >> error: setup_obd_uuid: unknown obduuid: scratch-OST0003_UUID >> /mnt/lustre/zero1 >> /mnt/lustre/zero2 >> /mnt/lustre/zero3 >> /mnt/lustre/zero4 >> /mnt/lustre/zero5 >> /mnt/lustre/zero6 > > Try using just "lfs getstripe /mnt/lustre" as a starter. It will > print > out the list of OST UUIDs at the start. It would appear that your > syntax > is correct, so I''m not sure what is going wrong... > > $ lfs df > UUID 1K-blocks Used Available Use% Mounted on > mds-myth-0_UUID 9174328 677728 8496600 7% /myth[MDT:0] > ost-myth-0_UUID 292223856 247877024 44346832 84% /myth[OST:0] > ost-myth-1_UUID 94442984 91277176 3165808 96% /myth[OST:1] > ost-myth-2_UUID 487388376 332816808 154571568 68% /myth[OST:2] > ost-myth-3_UUID 487865304 347373400 140491904 71% /myth[OST:3] > > filesystem summary: 1361920520 1019344408 342576112 74% /mythgnode30 ~ # lfs df UUID 1K-blocks Used Available Use% Mounted on scratch-MDT0000_UUID 744455808 43024252 701431556 5% /mnt/ lustre[MDT:0] scratch-OST0001_UUID 6726440480 342319020 6384121460 5% /mnt/ lustre[OST:1] scratch-OST0002_UUID 6726440480 342319020 6384121460 5% /mnt/ lustre[OST:2] scratch-OST0003_UUID 6726440480 342319020 6384121460 5% /mnt/ lustre[OST:3] scratch-OST0004_UUID 6726440480 343343024 6383097456 5% /mnt/ lustre[OST:4] scratch-OST0006_UUID 6726440480 342317996 6384122484 5% /mnt/ lustre[OST:6] scratch-OST0007_UUID 6726440480 342317996 6384122484 5% /mnt/ lustre[OST:7] filesystem summary: 40358642880 2054936076 38303706804 5% /mnt/ lustre> $ cat /proc/fs/lustre/version > lustre: 1.6.4.2 > kernel: 47 > build: b_release_1_6_4-20080123034511- > CHANGED-.usr.src.linux-2.6.16.46-0.14gnode30 ~ # cat /proc/fs/lustre/version lustre: 1.6.4.2 kernel: patchless build: 1.6.4.2-19700101010000-PRISTINE-.usr.src.linux-2.6.22-gentoo- r9-ubelix-node-2007.1> $ lfs getstripe /myth/tv/1017_20080129100000.mpg > OBDS: > 0: ost-myth-0_UUID ACTIVE > 1: ost-myth-1_UUID ACTIVE > 2: ost-myth-2_UUID ACTIVE > 3: ost-myth-3_UUID ACTIVE > /myth/tv/1017_20080129100000.mpg > obdidx objid objid group > 2 527023 0x80aaf 0 > 1 531093 0x81a95 0gnode30 ~ # lfs getstripe /mnt/lustre OBDS: /mnt/lustre default stripe_count: 6 stripe_size: 0 stripe_offset: 1 /mnt/lustre/zero2 obdidx objid objid group 4 7 0x7 0 /mnt/lustre/zero obdidx objid objid group 1 75 0x4b 0 2 12 0xc 0 3 73 0x49 0 4 10 0xa 0 6 9 0x9 0 7 70 0x46 0 I''m not seeing an OBDS list here, I assume this is not good ...> $ lfs getstripe -O ost-myth-1_UUID /myth/tv | head > /myth/tv/1017_20080129100000.mpg > /myth/tv/1017_20071101143000.mpg.png > /myth/tv/1037_20080119013000.mpg.png > /myth/tv/1017_20071219103000.mpg > /myth/tv/1008_20080102200000.mpg > /myth/tv/1039_20080123131300.mpg.png > /myth/tv/1014_20070905170000.mpg.png > /myth/tv/1039_20070914160000.mpg.png > /myth/tv/1014_20071009090000.mpg.png > /myth/tv/1017_20070917180000.mpggnode30 ~ # lfs getstripe -O scratch-OST0001_UUID /mnt/lustre error: setup_obd_uuid: unknown obduuid: scratch-OST0001_UUID /mnt/lustre default stripe_count: 6 stripe_size: 0 stripe_offset: 1 /mnt/lustre/zero2 /mnt/lustre/zero By the way, is the ''lfs getstripe'' command supposed to work if striping is disabled with ''lfs setstripe -d /mnt/lustre''?> Cheers, AndreasThanks for your help, Anselm
Anselm Strauss wrote:> On 30.01.2008, at 00:03, Andreas Dilger wrote: > >> On Jan 29, 2008 19:57 +0100, Anselm Strauss wrote: >>> I want to know all files that are located on a specific OST, but >>> neither ''lfs find ...'' nor ''lfs getstripe ...'' can give me an answer. >>> I do not use striping. This is what I tried: >>> >>> gnode30 ~ # lfs df -h >>> UUID bytes Used Available Use% Mounted on >>> scratch-MDT0000_UUID 710.0G 41.0G 668.9G 5% /mnt/ >>> lustre[MDT:0] >>> scratch-OST0001_UUID 6.3T 327.3G 5.9T 5% /mnt/ >>> lustre[OST:1] >>> scratch-OST0002_UUID 6.3T 327.3G 5.9T 5% /mnt/ >>> lustre[OST:2] >>> scratch-OST0003_UUID 6.3T 328.3G 5.9T 5% /mnt/ >>> lustre[OST:3] >>> scratch-OST0004_UUID 6.3T 327.3G 5.9T 5% /mnt/ >>> lustre[OST:4] >>> scratch-OST0006_UUID 6.3T 327.3G 5.9T 5% /mnt/ >>> lustre[OST:6] >>> scratch-OST0007_UUID 6.3T 326.3G 5.9T 5% /mnt/ >>> lustre[OST:7] >>> >>> filesystem summary: 37.6T 1.9T 35.7T 5% /mnt/lustre >>> >>> gnode30 lustre # lfs find -O scratch-OST0003_UUID /mnt/lustre >>> gnode30 lustre # lfs find -O 7 /mnt/lustre >>> gnode30 lustre # lfs getstripe -O scratch-OST0003_UUID /mnt/lustre >>> error: setup_obd_uuid: unknown obduuid: scratch-OST0003_UUID >>> /mnt/lustre/zero1 >>> /mnt/lustre/zero2 >>> /mnt/lustre/zero3 >>> /mnt/lustre/zero4 >>> /mnt/lustre/zero5 >>> /mnt/lustre/zero6 >> Try using just "lfs getstripe /mnt/lustre" as a starter. It will >> print >> out the list of OST UUIDs at the start. It would appear that your >> syntax >> is correct, so I''m not sure what is going wrong... >> >> $ lfs df >> UUID 1K-blocks Used Available Use% Mounted on >> mds-myth-0_UUID 9174328 677728 8496600 7% /myth[MDT:0] >> ost-myth-0_UUID 292223856 247877024 44346832 84% /myth[OST:0] >> ost-myth-1_UUID 94442984 91277176 3165808 96% /myth[OST:1] >> ost-myth-2_UUID 487388376 332816808 154571568 68% /myth[OST:2] >> ost-myth-3_UUID 487865304 347373400 140491904 71% /myth[OST:3] >> >> filesystem summary: 1361920520 1019344408 342576112 74% /myth > > gnode30 ~ # lfs df > UUID 1K-blocks Used Available Use% Mounted on > scratch-MDT0000_UUID 744455808 43024252 701431556 5% /mnt/ > lustre[MDT:0] > scratch-OST0001_UUID 6726440480 342319020 6384121460 5% /mnt/ > lustre[OST:1] > scratch-OST0002_UUID 6726440480 342319020 6384121460 5% /mnt/ > lustre[OST:2] > scratch-OST0003_UUID 6726440480 342319020 6384121460 5% /mnt/ > lustre[OST:3] > scratch-OST0004_UUID 6726440480 343343024 6383097456 5% /mnt/ > lustre[OST:4] > scratch-OST0006_UUID 6726440480 342317996 6384122484 5% /mnt/ > lustre[OST:6] > scratch-OST0007_UUID 6726440480 342317996 6384122484 5% /mnt/ > lustre[OST:7] > > filesystem summary: 40358642880 2054936076 38303706804 5% /mnt/ > lustre > >> $ cat /proc/fs/lustre/version >> lustre: 1.6.4.2 >> kernel: 47 >> build: b_release_1_6_4-20080123034511- >> CHANGED-.usr.src.linux-2.6.16.46-0.14 > > gnode30 ~ # cat /proc/fs/lustre/version > lustre: 1.6.4.2 > kernel: patchless > build: 1.6.4.2-19700101010000-PRISTINE-.usr.src.linux-2.6.22-gentoo- > r9-ubelix-node-2007.1 > > >> $ lfs getstripe /myth/tv/1017_20080129100000.mpg >> OBDS: >> 0: ost-myth-0_UUID ACTIVE >> 1: ost-myth-1_UUID ACTIVE >> 2: ost-myth-2_UUID ACTIVE >> 3: ost-myth-3_UUID ACTIVE >> /myth/tv/1017_20080129100000.mpg >> obdidx objid objid group >> 2 527023 0x80aaf 0 >> 1 531093 0x81a95 0 > > gnode30 ~ # lfs getstripe /mnt/lustre > OBDS: > /mnt/lustre > default stripe_count: 6 stripe_size: 0 stripe_offset: 1 > /mnt/lustre/zero2 > obdidx objid objid group > 4 7 0x7 0 > > /mnt/lustre/zero > obdidx objid objid group > 1 75 0x4b 0 > 2 12 0xc 0 > 3 73 0x49 0 > 4 10 0xa 0 > 6 9 0x9 0 > 7 70 0x46 0 > > I''m not seeing an OBDS list here, I assume this is not good ... > >> $ lfs getstripe -O ost-myth-1_UUID /myth/tv | head >> /myth/tv/1017_20080129100000.mpg >> /myth/tv/1017_20071101143000.mpg.png >> /myth/tv/1037_20080119013000.mpg.png >> /myth/tv/1017_20071219103000.mpg >> /myth/tv/1008_20080102200000.mpg >> /myth/tv/1039_20080123131300.mpg.png >> /myth/tv/1014_20070905170000.mpg.png >> /myth/tv/1039_20070914160000.mpg.png >> /myth/tv/1014_20071009090000.mpg.png >> /myth/tv/1017_20070917180000.mpg > > gnode30 ~ # lfs getstripe -O scratch-OST0001_UUID /mnt/lustre > error: setup_obd_uuid: unknown obduuid: scratch-OST0001_UUID > /mnt/lustre > default stripe_count: 6 stripe_size: 0 stripe_offset: 1 > /mnt/lustre/zero2 > /mnt/lustre/zero > > By the way, is the ''lfs getstripe'' command supposed to work if > striping is disabled with ''lfs setstripe -d /mnt/lustre''?Yes, it should. Are you running this from a client? cliffw> >> Cheers, Andreas > > Thanks for your help, > Anselm > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss
On 30.01.2008, at 18:46, Cliff White wrote:> Anselm Strauss wrote: >> On 30.01.2008, at 00:03, Andreas Dilger wrote: >>> On Jan 29, 2008 19:57 +0100, Anselm Strauss wrote: >>>> I want to know all files that are located on a specific OST, but >>>> neither ''lfs find ...'' nor ''lfs getstripe ...'' can give me an >>>> answer. >>>> I do not use striping. This is what I tried: >>>> >>>> gnode30 ~ # lfs df -h >>>> UUID bytes Used Available Use% Mounted on >>>> scratch-MDT0000_UUID 710.0G 41.0G 668.9G 5% /mnt/ >>>> lustre[MDT:0] >>>> scratch-OST0001_UUID 6.3T 327.3G 5.9T 5% /mnt/ >>>> lustre[OST:1] >>>> scratch-OST0002_UUID 6.3T 327.3G 5.9T 5% /mnt/ >>>> lustre[OST:2] >>>> scratch-OST0003_UUID 6.3T 328.3G 5.9T 5% /mnt/ >>>> lustre[OST:3] >>>> scratch-OST0004_UUID 6.3T 327.3G 5.9T 5% /mnt/ >>>> lustre[OST:4] >>>> scratch-OST0006_UUID 6.3T 327.3G 5.9T 5% /mnt/ >>>> lustre[OST:6] >>>> scratch-OST0007_UUID 6.3T 326.3G 5.9T 5% /mnt/ >>>> lustre[OST:7] >>>> >>>> filesystem summary: 37.6T 1.9T 35.7T 5% /mnt/ >>>> lustre >>>> >>>> gnode30 lustre # lfs find -O scratch-OST0003_UUID /mnt/lustre >>>> gnode30 lustre # lfs find -O 7 /mnt/lustre >>>> gnode30 lustre # lfs getstripe -O scratch-OST0003_UUID /mnt/lustre >>>> error: setup_obd_uuid: unknown obduuid: scratch-OST0003_UUID >>>> /mnt/lustre/zero1 >>>> /mnt/lustre/zero2 >>>> /mnt/lustre/zero3 >>>> /mnt/lustre/zero4 >>>> /mnt/lustre/zero5 >>>> /mnt/lustre/zero6 >>> Try using just "lfs getstripe /mnt/lustre" as a starter. It will >>> print >>> out the list of OST UUIDs at the start. It would appear that >>> your syntax >>> is correct, so I''m not sure what is going wrong... >>> >>> $ lfs df >>> UUID 1K-blocks Used Available Use% Mounted on >>> mds-myth-0_UUID 9174328 677728 8496600 7% /myth[MDT: >>> 0] >>> ost-myth-0_UUID 292223856 247877024 44346832 84% /myth[OST: >>> 0] >>> ost-myth-1_UUID 94442984 91277176 3165808 96% /myth[OST: >>> 1] >>> ost-myth-2_UUID 487388376 332816808 154571568 68% /myth[OST: >>> 2] >>> ost-myth-3_UUID 487865304 347373400 140491904 71% /myth[OST: >>> 3] >>> >>> filesystem summary: 1361920520 1019344408 342576112 74% /myth >> gnode30 ~ # lfs df >> UUID 1K-blocks Used Available Use% Mounted on >> scratch-MDT0000_UUID 744455808 43024252 701431556 5% /mnt/ >> lustre[MDT:0] >> scratch-OST0001_UUID 6726440480 342319020 6384121460 5% /mnt/ >> lustre[OST:1] >> scratch-OST0002_UUID 6726440480 342319020 6384121460 5% /mnt/ >> lustre[OST:2] >> scratch-OST0003_UUID 6726440480 342319020 6384121460 5% /mnt/ >> lustre[OST:3] >> scratch-OST0004_UUID 6726440480 343343024 6383097456 5% /mnt/ >> lustre[OST:4] >> scratch-OST0006_UUID 6726440480 342317996 6384122484 5% /mnt/ >> lustre[OST:6] >> scratch-OST0007_UUID 6726440480 342317996 6384122484 5% /mnt/ >> lustre[OST:7] >> filesystem summary: 40358642880 2054936076 38303706804 5% /mnt/ >> lustre >>> $ cat /proc/fs/lustre/version >>> lustre: 1.6.4.2 >>> kernel: 47 >>> build: b_release_1_6_4-20080123034511- >>> CHANGED-.usr.src.linux-2.6.16.46-0.14 >> gnode30 ~ # cat /proc/fs/lustre/version >> lustre: 1.6.4.2 >> kernel: patchless >> build: 1.6.4.2-19700101010000-PRISTINE-.usr.src.linux-2.6.22- >> gentoo- r9-ubelix-node-2007.1 >>> $ lfs getstripe /myth/tv/1017_20080129100000.mpg >>> OBDS: >>> 0: ost-myth-0_UUID ACTIVE >>> 1: ost-myth-1_UUID ACTIVE >>> 2: ost-myth-2_UUID ACTIVE >>> 3: ost-myth-3_UUID ACTIVE >>> /myth/tv/1017_20080129100000.mpg >>> obdidx objid objid group >>> 2 527023 0x80aaf 0 >>> 1 531093 0x81a95 0 >> gnode30 ~ # lfs getstripe /mnt/lustre >> OBDS: >> /mnt/lustre >> default stripe_count: 6 stripe_size: 0 stripe_offset: 1 >> /mnt/lustre/zero2 >> obdidx objid objid group >> 4 7 0x7 0 >> /mnt/lustre/zero >> obdidx objid objid group >> 1 75 0x4b 0 >> 2 12 0xc 0 >> 3 73 0x49 0 >> 4 10 0xa 0 >> 6 9 0x9 0 >> 7 70 0x46 0 >> I''m not seeing an OBDS list here, I assume this is not good ... >>> $ lfs getstripe -O ost-myth-1_UUID /myth/tv | head >>> /myth/tv/1017_20080129100000.mpg >>> /myth/tv/1017_20071101143000.mpg.png >>> /myth/tv/1037_20080119013000.mpg.png >>> /myth/tv/1017_20071219103000.mpg >>> /myth/tv/1008_20080102200000.mpg >>> /myth/tv/1039_20080123131300.mpg.png >>> /myth/tv/1014_20070905170000.mpg.png >>> /myth/tv/1039_20070914160000.mpg.png >>> /myth/tv/1014_20071009090000.mpg.png >>> /myth/tv/1017_20070917180000.mpg >> gnode30 ~ # lfs getstripe -O scratch-OST0001_UUID /mnt/lustre >> error: setup_obd_uuid: unknown obduuid: scratch-OST0001_UUID >> /mnt/lustre >> default stripe_count: 6 stripe_size: 0 stripe_offset: 1 >> /mnt/lustre/zero2 >> /mnt/lustre/zero >> By the way, is the ''lfs getstripe'' command supposed to work if >> striping is disabled with ''lfs setstripe -d /mnt/lustre''? > > Yes, it should. Are you running this from a client? > cliffwYes, although it''s a patchless client running on a 2.6.22 kernel. I also tried it on the failover server that was not mounting any lustre devices while I mounted the filesystem from the first server. It runs a Vanilla 2.6.18.8 kernel with lustre patches and lustre version 1.6.4.2. I got the same error. I also did an strace for the command on my patchless client: ... open("/proc/fs/lustre/lov/scratch-clilov-ffff8100f64bd000/target_obd", O_RDONLY) = 4 fstat(4, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b163b43a000 read(4, "", 1024) = 0 close(4) = 0 munmap(0x2b163b43a000, 4096) = 0 write(2, "error: setup_obd_uuid: unknown o"..., 77error: setup_obd_uuid: unknown obduuid: scratch-OST0001-osc-ffff8100f64bd000 ) = 77 ... Should there be any content in /proc/fs/lustre/lov/.../target_obd, mine is empty? Anselm>>> Cheers, Andreas >> Thanks for your help, >> Anselm
On Jan 30, 2008 20:48 +0100, Anselm Strauss wrote:> Yes, although it''s a patchless client running on a 2.6.22 kernel. I > also tried it on the failover server that was not mounting any lustre > devices while I mounted the filesystem from the first server. It runs > a Vanilla 2.6.18.8 kernel with lustre patches and lustre version > 1.6.4.2. I got the same error. > > I also did an strace for the command on my patchless client: > > ... > open("/proc/fs/lustre/lov/scratch-clilov-ffff8100f64bd000/target_obd", > O_RDONLY) = 4 > fstat(4, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x2b163b43a000 > read(4, "", 1024) = 0 > close(4) = 0 > munmap(0x2b163b43a000, 4096) = 0 > write(2, "error: setup_obd_uuid: unknown o"..., 77error: > setup_obd_uuid: unknown obduuid: scratch-OST0001-osc-ffff8100f64bd000 > ) = 77 > ... > > Should there be any content in /proc/fs/lustre/lov/.../target_obd, > mine is empty?Yes, this is definitely the source of the problem. I do recall there were some changes in the 2.6.22 kernel related to /proc/sys (not allowing proc entries to be created under /proc/sys) but this may have been more extensive... Of course the release notes for the 2.6.22 kernel explained this all clearly :-). Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.