Aaron Everett
2009-Mar-06 01:01 UTC
[Lustre-discuss] Permission denied error when mv files
Hello all, I am running Lustre 1.6.6 on 3 OST''s and 1 MDS with 1.6.6 clients. I am also running Sun Grid Engine and have approximately 70 client nodes, all multiple processors. The Lustre machines are running RHEL5 2.6.18-92.1.10.el5_lustre.1.6.6smp and clients are running RHEL4 2.6.9-67.0.22.EL_lustre.1.6.6smp. I am using autofs on the clients to mount the Lustre filesystem. My /etc/auto.lustre file contains: work -fstype=lustre -flock lustrefs:/lustre We are seeing a reproducible error if we try to mv a file from /tmp to the lustre filesystem. If we cp the same file, there is no error. [aeverett at rdlogin02 ~]$ [aeverett at rdlogin02 ~]$ touch /tmp/ae.test [aeverett at rdlogin02 ~]$ mv /tmp/ae.test /lustre/work/aeverett/ae.file mv: cannot create regular file `/lustre/work/aeverett/ae.file'': Permission denied [aeverett at rdlogin02 ~]$ cp /tmp/ae.test /lustre/work/aeverett/ae.file [aeverett at rdlogin02 ~]$ ls -al /lustre/work/aeverett/ae.file -rw-r--r-- 1 aeverett users 0 Mar 5 19:57 /lustre/work/aeverett/ae.file [aeverett at rdlogin02 ~]$ How does writing the file during a mv differ from a cp? Should I change my flags for the automount? Any help is appreciated! Regards, Aaron -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090305/97bc655e/attachment-0001.html
Andreas Dilger
2009-Mar-06 11:04 UTC
[Lustre-discuss] Permission denied error when mv files
On Mar 05, 2009 20:01 -0500, Aaron Everett wrote:> We are seeing a reproducible error if we try to mv a file from /tmp to > the lustre filesystem. If we cp the same file, there is no error. > > [aeverett at rdlogin02 ~]$ touch /tmp/ae.test > > [aeverett at rdlogin02 ~]$ mv /tmp/ae.test /lustre/work/aeverett/ae.file > > mv: cannot create regular file `/lustre/work/aeverett/ae.file'': > Permission deniedRun strace on the "mv" and see what syscall is returning this error.> [aeverett at rdlogin02 ~]$ cp /tmp/ae.test /lustre/work/aeverett/ae.file > > [aeverett at rdlogin02 ~]$ ls -al /lustre/work/aeverett/ae.file > > -rw-r--r-- 1 aeverett users 0 Mar 5 19:57 > /lustre/work/aeverett/ae.fileCheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
Aaron Everett
2009-Mar-06 16:30 UTC
[Lustre-discuss] Permission denied error when mv files
One more interesting tidbit. Even through the permission denied error is
given, the file is actually written. Looking at this, it looks like the
stat after the write fails. Am I reading that correctly?
Here is the result of strace:
[aeverett at rdlogin02 ~]$ touch /tmp/ae.test
[aeverett at rdlogin02 ~]$ strace mv /tmp/ae.test
/lustre/work/aeverett/ae.lustremvtest
execve("/bin/mv", ["mv", "/tmp/ae.test",
"/lustre/work/aeverett/ae.lustrem"...], [/* 51 vars */]) = 0
uname({sys="Linux", node="rdlogin02.forteds.com", ...}) = 0
brk(0) = 0x8094000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or
directory)
open("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/tls/i686/sse2/l
ibacl.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/tls/i686/sse2
", 0xbfe78920) = -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/tls/i686/libacl
.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/tls/i686",
0xbfe78920) = -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/tls/sse2/libacl
.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/tls/sse2",
0xbfe78920) = -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/tls/libacl.so.1
", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/tls",
0xbfe78920) = -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/i686/sse2/libac
l.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/i686/sse2",
0xbfe78920) = -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/i686/libacl.so.
1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/i686",
0xbfe78920) = -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/sse2/libacl.so.
1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/sse2",
0xbfe78920) = -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/libacl.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1",
{st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/tls/i686/sse2/libacl
.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/tls/i686/sse2",
0xbfe78920) = -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/tls/i686/libacl.so.1
", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/tls/i686",
0xbfe78920) = -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/tls/sse2/libacl.so.1
", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/tls/sse2",
0xbfe78920) = -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/tls/libacl.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/tls",
0xbfe78920)
= -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/i686/sse2/libacl.so.
1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/i686/sse2",
0xbfe78920) = -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/i686/libacl.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/i686",
0xbfe78920)
= -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/sse2/libacl.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/sse2",
0xbfe78920)
= -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/libacl.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX",
{st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/tools/sge/lib/lx24-x86/tls/i686/sse2/libacl.so.1", O_RDONLY) =
-1
ENOENT (No such file or directory)
stat64("/tools/sge/lib/lx24-x86/tls/i686/sse2", 0xbfe78920) = -1
ENOENT
(No such file or directory)
open("/tools/sge/lib/lx24-x86/tls/i686/libacl.so.1", O_RDONLY) = -1
ENOENT (No such file or directory)
stat64("/tools/sge/lib/lx24-x86/tls/i686", 0xbfe78920) = -1 ENOENT (No
such file or directory)
open("/tools/sge/lib/lx24-x86/tls/sse2/libacl.so.1", O_RDONLY) = -1
ENOENT (No such file or directory)
stat64("/tools/sge/lib/lx24-x86/tls/sse2", 0xbfe78920) = -1 ENOENT (No
such file or directory)
open("/tools/sge/lib/lx24-x86/tls/libacl.so.1", O_RDONLY) = -1 ENOENT
(No such file or directory)
stat64("/tools/sge/lib/lx24-x86/tls", 0xbfe78920) = -1 ENOENT (No such
file or directory)
open("/tools/sge/lib/lx24-x86/i686/sse2/libacl.so.1", O_RDONLY) = -1
ENOENT (No such file or directory)
stat64("/tools/sge/lib/lx24-x86/i686/sse2", 0xbfe78920) = -1 ENOENT
(No
such file or directory)
open("/tools/sge/lib/lx24-x86/i686/libacl.so.1", O_RDONLY) = -1 ENOENT
(No such file or directory)
stat64("/tools/sge/lib/lx24-x86/i686", 0xbfe78920) = -1 ENOENT (No
such
file or directory)
open("/tools/sge/lib/lx24-x86/sse2/libacl.so.1", O_RDONLY) = -1 ENOENT
(No such file or directory)
stat64("/tools/sge/lib/lx24-x86/sse2", 0xbfe78920) = -1 ENOENT (No
such
file or directory)
open("/tools/sge/lib/lx24-x86/libacl.so.1", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/tools/sge/lib/lx24-x86", {st_mode=S_IFDIR|0755, st_size=4096,
...}) = 0
open("/tools/gnu/x86-linux/lib/tls/i686/sse2/libacl.so.1", O_RDONLY)
-1 ENOENT (No such file or directory)
stat64("/tools/gnu/x86-linux/lib/tls/i686/sse2", 0xbfe78920) = -1
ENOENT
(No such file or directory)
open("/tools/gnu/x86-linux/lib/tls/i686/libacl.so.1", O_RDONLY) = -1
ENOENT (No such file or directory)
stat64("/tools/gnu/x86-linux/lib/tls/i686", 0xbfe78920) = -1 ENOENT
(No
such file or directory)
open("/tools/gnu/x86-linux/lib/tls/sse2/libacl.so.1", O_RDONLY) = -1
ENOENT (No such file or directory)
stat64("/tools/gnu/x86-linux/lib/tls/sse2", 0xbfe78920) = -1 ENOENT
(No
such file or directory)
open("/tools/gnu/x86-linux/lib/tls/libacl.so.1", O_RDONLY) = -1 ENOENT
(No such file or directory)
stat64("/tools/gnu/x86-linux/lib/tls", 0xbfe78920) = -1 ENOENT (No
such
file or directory)
open("/tools/gnu/x86-linux/lib/i686/sse2/libacl.so.1", O_RDONLY) = -1
ENOENT (No such file or directory)
stat64("/tools/gnu/x86-linux/lib/i686/sse2", 0xbfe78920) = -1 ENOENT
(No
such file or directory)
open("/tools/gnu/x86-linux/lib/i686/libacl.so.1", O_RDONLY) = -1
ENOENT
(No such file or directory)
stat64("/tools/gnu/x86-linux/lib/i686", 0xbfe78920) = -1 ENOENT (No
such
file or directory)
open("/tools/gnu/x86-linux/lib/sse2/libacl.so.1", O_RDONLY) = -1
ENOENT
(No such file or directory)
stat64("/tools/gnu/x86-linux/lib/sse2", 0xbfe78920) = -1 ENOENT (No
such
file or directory)
open("/tools/gnu/x86-linux/lib/libacl.so.1", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/tools/gnu/x86-linux/lib", {st_mode=S_IFDIR|0755,
st_size=4096,
...}) = 0
open("/tools/cadence/x86-linux/spr4.0/tools/lib/tls/i686/sse2/libacl.so.
1", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/cadence/x86-linux/spr4.0/tools/lib/tls/i686/sse2",
0xbfe78920) = -1 ENOENT (No such file or directory)
open("/tools/cadence/x86-linux/spr4.0/tools/lib/tls/i686/libacl.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/cadence/x86-linux/spr4.0/tools/lib/tls/i686",
0xbfe78920)
= -1 ENOENT (No such file or directory)
open("/tools/cadence/x86-linux/spr4.0/tools/lib/tls/sse2/libacl.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/cadence/x86-linux/spr4.0/tools/lib/tls/sse2",
0xbfe78920)
= -1 ENOENT (No such file or directory)
open("/tools/cadence/x86-linux/spr4.0/tools/lib/tls/libacl.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/cadence/x86-linux/spr4.0/tools/lib/tls", 0xbfe78920) =
-1
ENOENT (No such file or directory)
open("/tools/cadence/x86-linux/spr4.0/tools/lib/i686/sse2/libacl.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/cadence/x86-linux/spr4.0/tools/lib/i686/sse2",
0xbfe78920) = -1 ENOENT (No such file or directory)
open("/tools/cadence/x86-linux/spr4.0/tools/lib/i686/libacl.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/cadence/x86-linux/spr4.0/tools/lib/i686", 0xbfe78920)
-1 ENOENT (No such file or directory)
open("/tools/cadence/x86-linux/spr4.0/tools/lib/sse2/libacl.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/tools/cadence/x86-linux/spr4.0/tools/lib/sse2", 0xbfe78920)
-1 ENOENT (No such file or directory)
open("/tools/cadence/x86-linux/spr4.0/tools/lib/libacl.so.1",
O_RDONLY)
= -1 ENOENT (No such file or directory)
stat64("/tools/cadence/x86-linux/spr4.0/tools/lib",
{st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=88116, ...}) = 0
old_mmap(NULL, 88116, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fae000
close(3) = 0
open("/lib/libacl.so.1", O_RDONLY) = 3
read(3,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\22\0\0004\0\0\0"...
, 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=22164, ...}) = 0
old_mmap(NULL, 20948, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x3f8000
old_mmap(0x3fd000, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5000) = 0x3fd000
close(3) = 0
open("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/libselinux.so.1
", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/libselinux.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/tools/sge/lib/lx24-x86/libselinux.so.1", O_RDONLY) = -1 ENOENT
(No such file or directory)
open("/tools/gnu/x86-linux/lib/libselinux.so.1", O_RDONLY) = -1 ENOENT
(No such file or directory)
open("/tools/cadence/x86-linux/spr4.0/tools/lib/libselinux.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib/libselinux.so.1", O_RDONLY) = 3
read(3,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\220\303\0004\0\0\0"
..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=56336, ...}) = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
-1, 0) = 0xb7fad000
old_mmap(0xc37000, 56176, PROT_READ|PROT_EXEC,
MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xc37000
old_mmap(0xc44000, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xd000) = 0xc44000
close(3) = 0
open("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/libc.so.6",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/libc.so.6",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/tools/sge/lib/lx24-x86/libc.so.6", O_RDONLY) = -1 ENOENT (No
such
file or directory)
open("/tools/gnu/x86-linux/lib/libc.so.6", O_RDONLY) = -1 ENOENT (No
such file or directory)
open("/tools/cadence/x86-linux/spr4.0/tools/lib/libc.so.6", O_RDONLY)
-1 ENOENT (No such file or directory)
open("/lib/tls/libc.so.6", O_RDONLY) = 3
read(3,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\276\212\0004\0\0\0"
..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=1529136, ...}) = 0
old_mmap(0x897000, 1227964, PROT_READ|PROT_EXEC,
MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x897000
old_mmap(0x9bd000, 16384, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x125000) = 0x9bd000
old_mmap(0x9c1000, 7356, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x9c1000
close(3) = 0
open("/tools/novas/5.5/share/PLI/nc_xl/LINUX/xl_loadpli1/libattr.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/tools/novas/5.5/share/PLI/modelsim_pli/LINUX/libattr.so.1",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/tools/sge/lib/lx24-x86/libattr.so.1", O_RDONLY) = -1 ENOENT (No
such file or directory)
open("/tools/gnu/x86-linux/lib/libattr.so.1", O_RDONLY) = -1 ENOENT
(No
such file or directory)
open("/tools/cadence/x86-linux/spr4.0/tools/lib/libattr.so.1",
O_RDONLY)
= -1 ENOENT (No such file or directory)
open("/lib/libattr.so.1", O_RDONLY) = 3
read(3,
"\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0000\v\0\0004\0\0\0"...,
512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=30899, ...}) = 0
old_mmap(NULL, 13508, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) = 0x37a000
old_mmap(0x37d000, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x37d000
close(3) = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
-1, 0) = 0xb7fac000
mprotect(0x9bd000, 8192, PROT_READ) = 0
mprotect(0x893000, 4096, PROT_READ) = 0
set_thread_area({entry_number:-1 -> 6, base_addr:0xb7fac6c0,
limit:1048575, seg_32bit:1, contents:0, read_exec_only:0,
limit_in_pages:1, seg_not_p
resent:0, useable:1}) = 0
munmap(0xb7fae000, 88116) = 0
access("/etc/selinux/", F_OK) = 0
brk(0) = 0x8094000
brk(0x80b5000) = 0x80b5000
open("/etc/selinux/config", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=448, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7fc3000
read(3, "# This file controls the state o"..., 4096) = 448
close(3) = 0
munmap(0xb7fc3000, 4096) = 0
open("/proc/mounts", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7fc3000
read(3, "rootfs / rootfs rw 0 0\n/proc /pr"..., 1024) = 1024
close(3) = 0
munmap(0xb7fc3000, 4096) = 0
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=48508720, ...}) = 0
mmap2(NULL, 2097152, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7dac000
close(3) = 0
geteuid32() = 680
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo
...}) = 0
open("/proc/filesystems", O_RDONLY) = 3
read(3, "nodev\tsysfs\nnodev\trootfs\nnodev\tb"..., 4095) = 326
open("/proc/self/attr/current", O_RDONLY) = 4
read(4, "user_u:system_r:unconfined_t\0", 4095) = 29
close(4) = 0
close(3) = 0
stat64("/lustre/work/aeverett/ae.lustremvtest", 0xbfe79170) = -1
ENOENT
(No such file or directory)
lstat64("/tmp/ae.test", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
lstat64("/lustre/work/aeverett/ae.lustremvtest", 0xbfe79060) = -1
ENOENT
(No such file or directory)
rename("/tmp/ae.test",
"/lustre/work/aeverett/ae.lustremvtest") = -1
EXDEV (Invalid cross-device link)
unlink("/lustre/work/aeverett/ae.lustremvtest") = -1 ENOENT (No such
file or directory)
lgetxattr("/tmp/ae.test", "security.selinux"...,
"user_u:object_r:tmp_t", 255) = 22
open("/proc/self/attr/fscreate", O_RDWR) = 3
write(3, "user_u:object_r:tmp_t\0", 22) = 22
close(3) = 0
open("/tmp/ae.test", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
open("/lustre/work/aeverett/ae.lustremvtest",
O_WRONLY|O_CREAT|O_LARGEFILE, 0100644) = -1 EACCES (Permission denied)
open("/usr/share/locale/locale.alias", O_RDONLY) = 4
fstat64(4, {st_mode=S_IFREG|0644, st_size=2528, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7dab000
read(4, "# Locale name alias data base.\n#"..., 4096) = 2528
read(4, "", 4096) = 0
close(4) = 0
munmap(0xb7dab000, 4096) = 0
open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo",
O_RDONLY)
= -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo",
O_RDONLY)
= -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) =
-1
ENOENT (No such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY)
-1 ENOENT (No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY)
-1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
write(2, "mv: ", 4mv: ) = 4
write(2, "cannot create regular file `/lus"..., 66cannot create
regular
file `/lustre/work/aeverett/ae.lustremvtest'') = 66
open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) =
-1
ENOENT (No such file or directory)
open("/usr/share/locale/en_US.utf8/LC_MESSAGES/libc.mo", O_RDONLY) =
-1
ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)
write(2, ": Permission denied", 19: Permission denied) = 19
write(2, "\n", 1
) = 1
close(3) = 0
open("/proc/self/attr/fscreate", O_RDWR) = 3
write(3, NULL, 0) = 0
close(3) = 0
exit_group(1) = ?
[aeverett at rdlogin02 ~]$ ls /tmp
-----Original Message-----
From: Andreas.Dilger at sun.com [mailto:Andreas.Dilger at sun.com] On Behalf
Of Andreas Dilger
Sent: Friday, March 06, 2009 6:05 AM
To: Aaron Everett
Cc: lustre-discuss at lists.lustre.org
Subject: Re: [Lustre-discuss] Permission denied error when mv files
On Mar 05, 2009 20:01 -0500, Aaron Everett wrote:> We are seeing a reproducible error if we try to mv a file from /tmp to
> the lustre filesystem. If we cp the same file, there is no error.
>
> [aeverett at rdlogin02 ~]$ touch /tmp/ae.test
>
> [aeverett at rdlogin02 ~]$ mv /tmp/ae.test /lustre/work/aeverett/ae.file
>
> mv: cannot create regular file `/lustre/work/aeverett/ae.file'':
> Permission denied
Run strace on the "mv" and see what syscall is returning this error.
> [aeverett at rdlogin02 ~]$ cp /tmp/ae.test /lustre/work/aeverett/ae.file
>
> [aeverett at rdlogin02 ~]$ ls -al /lustre/work/aeverett/ae.file
>
> -rw-r--r-- 1 aeverett users 0 Mar 5 19:57
> /lustre/work/aeverett/ae.file
Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.
Kevin Van Maren
2009-Mar-06 19:00 UTC
[Lustre-discuss] Permission denied error when mv files
Aaron Everett wrote:> One more interesting tidbit. Even through the permission denied error is > given, the file is actually written. Looking at this, it looks like the > stat after the write fails. Am I reading that correctly? >It looks like the file was only written by the "cp", not the "mv": open("/lustre/work/aeverett/ae.lustremvtest", O_WRONLY|O_CREAT|O_LARGEFILE, 0100644) = -1 EACCES (Permission denied) The similar trace around the open "/lustre/work/aeverett/ae.lustremvtest" in "cp" could be useful. selinux is enabled, and is my first guess as to the problem -- have you tried disabling selinux? This is the relevant section of the trace (at the end). geteuid32() = 680 ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 open("/proc/filesystems", O_RDONLY) = 3 read(3, "nodev\tsysfs\nnodev\trootfs\nnodev\tb"..., 4095) = 326 open("/proc/self/attr/current", O_RDONLY) = 4 read(4, "user_u:system_r:unconfined_t\0", 4095) = 29 close(4) = 0 close(3) = 0 stat64("/lustre/work/aeverett/ae.lustremvtest", 0xbfe79170) = -1 ENOENT (No such file or directory) lstat64("/tmp/ae.test", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 lstat64("/lustre/work/aeverett/ae.lustremvtest", 0xbfe79060) = -1 ENOENT (No such file or directory) rename("/tmp/ae.test", "/lustre/work/aeverett/ae.lustremvtest") = -1 EXDEV (Invalid cross-device link) unlink("/lustre/work/aeverett/ae.lustremvtest") = -1 ENOENT (No such file or directory) lgetxattr("/tmp/ae.test", "security.selinux"..., "user_u:object_r:tmp_t", 255) = 22 open("/proc/self/attr/fscreate", O_RDWR) = 3 write(3, "user_u:object_r:tmp_t\0", 22) = 22 close(3) = 0 open("/tmp/ae.test", O_RDONLY|O_LARGEFILE) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 open("/lustre/work/aeverett/ae.lustremvtest", O_WRONLY|O_CREAT|O_LARGEFILE, 0100644) = -1 EACCES (Permission denied) write(2, "mv: ", 4mv: ) = 4 write(2, "cannot create regular file `/lus"..., 66 cannot create regular file `/lustre/work/aeverett/ae.lustremvtest'') = 66 write(2, ": Permission denied", 19: Permission denied) = 19 Kevin