zfs discuss - Aug 2009 - Failure of Quicktime *.mov files after move to zfs disk

During the course of backup I had occassion to copy a number of
quicktime video (*.mov) files to zfs server disk.  

Once there... navigating to them with quicktime player and opening
results in a failure that (From windows Vista laptop) says:
    error --43: A file could not be found (Welcome.mov)

I would have attributed it to some problem from scping it to the zfs
server had it not been for finding that if I scp it to a linux server
the problem does not occur.

Both the zfs and linux (Gentoo) servers are on a home lan.. but using
the same router/switch[s] over gigabit network adaptors.

On both occasions the files were copied using cygwin/ssh on a Vista
laptop.

Anyone have an idea what might cause this.

Any more details I can add that would make diagnostics easier?

Checksum all of the files using something like md5sum and see if
they''re actually identical.  Then test each step of the copy and see
which one is corrupting your files.

On Fri, Aug 21, 2009 at 1:43 PM, Harry Putnam<reader at newsguy.com>
wrote:
> During the course of backup I had occassion to copy a number of
> quicktime video (*.mov) files to zfs server disk.
>
> Once there... navigating to them with quicktime player and opening
> results in a failure that (From windows Vista laptop) says:
> ? ?error --43: A file could not be found (Welcome.mov)
>
> I would have attributed it to some problem from scping it to the zfs
> server had it not been for finding that if I scp it to a linux server
> the problem does not occur.
>
> Both the zfs and linux (Gentoo) servers are on a home lan.. but using
> the same router/switch[s] over gigabit network adaptors.
>
> On both occasions the files were copied using cygwin/ssh on a Vista
> laptop.
>
> Anyone have an idea what might cause this.
>
> Any more details I can add that would make diagnostics easier?
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>

Check with Vista if you have permissions to read the file. I
experienced the same problem (that''s why I posted another questions to
the CIFS mailing list about mapping users with idmap). It always
happens when I copy these files from the iPhone. These files result
with such permissions:

$ ls -dV IMG_0004.MOV
-rw-------   1 enrico   staff    12949182 Jul 17 19:39 IMG_0004.MOV
                 owner@:--x-----------:-------:deny
                 owner@:rw-p---A-W-Co-:-------:allow
                 group@:rwxp----------:-------:deny
                 group@:--------------:-------:allow
              everyone@:rwxp---A-W-Co-:-------:deny
              everyone@:------a-R-c--s:-------:allow

On the Vista side, having mapped with idmap even the staff group to
the corresponding Windows group of the enrico user, the file results
with the following special permissions for the user enrico:
List folder/read data
Create files/write data
Create folders/append data
Write attributes
Write extended attributes

Here something''s missing to me and documentation hasn''t helped
me
(yet)... There''s no "read" here. Just set it (on the Vista
side it''s
just one click) and Quicktime will work. I''ve got a script which
resets my files'' permissions something like:

find /yourdir -type f -exec chmod A- "{}" +
find /yourdir -type f -exec chmod 644 "{}" +

Hope this helps,
Enrico


On Fri, Aug 21, 2009 at 11:35 PM, Scott Laird<scott at sigkill.org>
wrote:
> Checksum all of the files using something like md5sum and see if
> they''re actually identical. ?Then test each step of the copy and
see
> which one is corrupting your files.
>
> On Fri, Aug 21, 2009 at 1:43 PM, Harry Putnam<reader at newsguy.com>
wrote:
>> During the course of backup I had occassion to copy a number of
>> quicktime video (*.mov) files to zfs server disk.
>>
>> Once there... navigating to them with quicktime player and opening
>> results in a failure that (From windows Vista laptop) says:
>> ? ?error --43: A file could not be found (Welcome.mov)
>>
>> I would have attributed it to some problem from scping it to the zfs
>> server had it not been for finding that if I scp it to a linux server
>> the problem does not occur.
>>
>> Both the zfs and linux (Gentoo) servers are on a home lan.. but using
>> the same router/switch[s] over gigabit network adaptors.
>>
>> On both occasions the files were copied using cygwin/ssh on a Vista
>> laptop.
>>
>> Anyone have an idea what might cause this.
>>
>> Any more details I can add that would make diagnostics easier?
>>
>> _______________________________________________
>> zfs-discuss mailing list
>> zfs-discuss at opensolaris.org
>> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>


-- 
????????? ? ???????
"Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs, and the Universe trying
to produce bigger and better idiots. So far, the Universe is winning."
GPG key: 1024D/FD2229AF

On 21 Aug 2009, at 22:35, Scott Laird wrote:
> Checksum all of the files using something like md5sum and see if
> they''re actually identical.  Then test each step of the copy and
see
> which one is corrupting your files.
It might be worth checking if they''ve got funny Unicode chars in the  
names. What normalization''s happening on both servers, what version of
NFS is being used? How big are the files?

Cheers,

Chris

i had something similar happen to me when i switched to ZFS but it turned
out to be an error with cpio and the mkv format...i''m not sure exactly
why
but whenever i tried to backup mkv files with cpio onto ZFS it would give me
corrupted files.

On Fri, Aug 21, 2009 at 4:43 PM, Harry Putnam <reader at newsguy.com>
wrote:
> During the course of backup I had occassion to copy a number of
> quicktime video (*.mov) files to zfs server disk.
>
> Once there... navigating to them with quicktime player and opening
> results in a failure that (From windows Vista laptop) says:
>    error --43: A file could not be found (Welcome.mov)
>
> I would have attributed it to some problem from scping it to the zfs
> server had it not been for finding that if I scp it to a linux server
> the problem does not occur.
>
> Both the zfs and linux (Gentoo) servers are on a home lan.. but using
> the same router/switch[s] over gigabit network adaptors.
>
> On both occasions the files were copied using cygwin/ssh on a Vista
> laptop.
>
> Anyone have an idea what might cause this.
>
> Any more details I can add that would make diagnostics easier?
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090822/946ebd8f/attachment.html>

Scott Laird <scott at sigkill.org> writes:
> Checksum all of the files using something like md5sum and see if
> they''re actually identical.  Then test each step of the copy and
see
> which one is corrupting your files.
>
> On Fri, Aug 21, 2009 at 1:43 PM, Harry Putnam<reader at newsguy.com>
wrote:
[...]

I didn''t do that since I''ve found that opening the file from
vista
with a file browser started as `administator'' worked.. so apparently
the files are indenticle enough to play in quicktime player started as
"administrator" .

Enrico Maria Crisostomo <enrico.m.crisostomo at gmail.com> writes:

[...]

Thanks for the input... I''ve found now that the directory on zfs
server that I was scping the files to had not gotten included in a
previous chmod cmd run on the zfs server.

   chmod -R A=everyone@:full_set:fd:allow

That particular directory where the transferred files were landing,
was created after having run the chmod cmd above on the server.
> Here something''s missing to me and documentation hasn''t
helped me
> (yet)... There''s no "read" here. Just set it (on the
Vista side it''s
> just one click) and Quicktime will work. I''ve got a script which
> resets my files'' permissions something like:
>
> find /yourdir -type f -exec chmod A- "{}" +
> find /yourdir -type f -exec chmod 644 "{}" +
The chmod command I mentioned above appears many times on the
cifs-server list, as a way to avoid permissions problems and as it
turns out it works in my case too. ... although it appears to only be
of use when called after the files are transferred.

The commands you show also appear to make things work... however, at
first I thought the executable bits that are set when the files are
created on windows... doesn''t really seem to prevent running it from a
remote vista laptop.  It appears here that the permission bits as they
occur work, and so does chmoding to 644.
 
[...]

Chris Wrote:
> It might be worth checking if they''ve got funny Unicode chars in
the
> names. What normalization''s happening on both servers, what
version of
> NFS is being used? How big are the files?
Apparently not the problem in my case... thanks for the input.

Thomas Burgess <wonslung at gmail.com> writes:
> i had something similar happen to me when i switched to ZFS but it turned
> out to be an error with cpio and the mkv format...i''m not sure
exactly why
> but whenever i tried to backup mkv files with cpio onto ZFS it would give
me
> corrupted files.
This was also apparently not the problem in my case... thanks for the input.

Hi Harry.

Glad to hear you solved the problem. As soon as I saw that Quicktime''s
error I thought about permissions. Unfortunately I sort of got used to
it. To follow on this discussion, I think there''s something strange
here. It might simply be a Quicktime idiosyncrasy, which is chocking
onto those permissions because it''s expecting some more.

Although it''s a CIFS-related discussion rather than ZFS, I want to
make my point clear. My reasoning is: I start with what I think is a
reasonable (and intuitive) choice: mode 600 for my files (no other
ACL). If I examine permissions on the Windows side they are pretty
odd: as I said, there''s no "read" there. If you examine
special
permissions, my user is indeed allowed "List folders/Read data"
(amongst others).

Nevertheless, almost everything works as expected. I can indeed read
the files'' contents and copy them locally, although I cannot launch
files with Quicktime directly on the share. That''s why I think
Quicktime is chocking somewhere.

If I give my user the Windows'' read permission, on the Solaris side it
materializes like this:
$ ls -dV IMG_0003.JPG
----------+  1 enrico   staff    1082821 Jul 17 19:39 IMG_0003.JPG
              everyone@:rwxp---A-W-Co-:-------:deny
            user:enrico:--x-----------:-------:deny
            group:staff:rwxp----------:-------:deny
              everyone@:------a-R-c--s:-------:allow
            user:enrico:rw-p--aARWc--s:-------:allow
            user:enrico:rw-p---A-W-Cos:-------:allow
            group:staff:-------------s:-------:allow

I''d like to find some documentation about how we should map
Windows''
permissions to Solaris ACLs. It doesn''t seem so intuitive to me, so
far. By the way: when I set up CIFS shares and then manage them from
Windows, I don''t usually encounter any problem. The first time I saw
the strange Quicktime behaviour was due to a cron-scheduled script
which was resetting permissions on the shared files to Solaris
friendly values such as 600 or 644 for files and 700 and 755 for
directories. I thought it was sufficient, after mapping users and
groups with idmap, but it seems to fall short sometimes.

Thank you for any pointer,
Enrico

On Sat, Aug 22, 2009 at 9:00 PM, Harry Putnam<reader at newsguy.com>
wrote:
> Scott Laird <scott at sigkill.org> writes:
>
>> Checksum all of the files using something like md5sum and see if
>> they''re actually identical. ?Then test each step of the copy
and see
>> which one is corrupting your files.
>>
>> On Fri, Aug 21, 2009 at 1:43 PM, Harry Putnam<reader at
newsguy.com> wrote:
>
> [...]
>
> I didn''t do that since I''ve found that opening the file
from vista
> with a file browser started as `administator'' worked.. so
apparently
> the files are indenticle enough to play in quicktime player started as
> "administrator" .
>
> Enrico Maria Crisostomo <enrico.m.crisostomo at gmail.com> writes:
>
> [...]
>
> Thanks for the input... I''ve found now that the directory on zfs
> server that I was scping the files to had not gotten included in a
> previous chmod cmd run on the zfs server.
>
> ? chmod -R A=everyone@:full_set:fd:allow
>
> That particular directory where the transferred files were landing,
> was created after having run the chmod cmd above on the server.
>
>> Here something''s missing to me and documentation
hasn''t helped me
>> (yet)... There''s no "read" here. Just set it (on the
Vista side it''s
>> just one click) and Quicktime will work. I''ve got a script
which
>> resets my files'' permissions something like:
>>
>> find /yourdir -type f -exec chmod A- "{}" +
>> find /yourdir -type f -exec chmod 644 "{}" +
>
> The chmod command I mentioned above appears many times on the
> cifs-server list, as a way to avoid permissions problems and as it
> turns out it works in my case too. ... although it appears to only be
> of use when called after the files are transferred.
>
> The commands you show also appear to make things work... however, at
> first I thought the executable bits that are set when the files are
> created on windows... doesn''t really seem to prevent running it
from a
> remote vista laptop. ?It appears here that the permission bits as they
> occur work, and so does chmoding to 644.
>
> [...]
>
> Chris Wrote:
>
>> It might be worth checking if they''ve got funny Unicode chars
in the
>> names. What normalization''s happening on both servers, what
version of
>> NFS is being used? How big are the files?
>
> Apparently not the problem in my case... thanks for the input.
>
> Thomas Burgess <wonslung at gmail.com> writes:
>
>> i had something similar happen to me when i switched to ZFS but it
turned
>> out to be an error with cpio and the mkv format...i''m not sure
exactly why
>> but whenever i tried to backup mkv files with cpio onto ZFS it would
give me
>> corrupted files.
>
> This was also apparently not the problem in my case... thanks for the
input.
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>


-- 
????????? ? ???????
"Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs, and the Universe trying
to produce bigger and better idiots. So far, the Universe is winning."
GPG key: 1024D/FD2229AF