Lustre discuss - Nov 2006 - Lustre breaking sendfile functionality

Recently i hit a bug with lustre and apache/vsftpd (using sendfile)

symptoms are :

kernel 2.6.12.6 and lustre 1.5.95 and vserver patch for 2.6.12
- from inside a vserver i get only first ~114kb of any file when files are 
stored on lustre
- from outside of a vserver i get whole file when they are stored on local 
fs

kernel 2.6.18.1 and lustre 1.5.95 and vserver patch for 2.6.18
- from inside of a vserver i get first ~64kb of any file when files are 
stored on lustre
- from outside of a vserver i get whole file (same size) but transmission 
ends with an error (tested on vsftp server and windows client) when files 
are stored on lustre
- when files are stored on local fs it works fine

TIA
Konrad Gutkowski

how i remember vserver has many sendfile issues.
look into vserver mail archive to corresponding patches.

? ???, 01.11.2006, ? 18:59, konrad.gutkowski@ffs.pl
?????:
> Recently i hit a bug with lustre and apache/vsftpd (using sendfile)
> 
> symptoms are :
> 
> kernel 2.6.12.6 and lustre 1.5.95 and vserver patch for 2.6.12
> - from inside a vserver i get only first ~114kb of any file when files are 
> stored on lustre
> - from outside of a vserver i get whole file when they are stored on local 
> fs
> 
> kernel 2.6.18.1 and lustre 1.5.95 and vserver patch for 2.6.18
> - from inside of a vserver i get first ~64kb of any file when files are 
> stored on lustre
> - from outside of a vserver i get whole file (same size) but transmission 
> ends with an error (tested on vsftp server and windows client) when files 
> are stored on lustre
> - when files are stored on local fs it works fine
> 
> TIA
> Konrad Gutkowski 
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss@clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
-- 
Alexey Lyashkov <shadow@clusterfs.com>

so i did, the patches was half a year old atleast, mostly for development 
versions, ill make a non-VS kernel later to test it, also will prolly do non 
lustre one and use VS from local hd for another test,
but in the worst case it can be connected to both (it works on non-lustre 
patched 2.6.15 with older VS ptach)

Konrad Gutkowski

----- Original Message ----- 
From: "Alexey Lyashkov" <shadow@clusterfs.com>
To: <konrad.gutkowski@ffs.pl>
Cc: <lustre-discuss@clusterfs.com>
Sent: Wednesday, November 01, 2006 8:14 PM
Subject: Re: [Lustre-discuss] Lustre breaking sendfile functionality


how i remember vserver has many sendfile issues.
look into vserver mail archive to corresponding patches.

? ???, 01.11.2006, ? 18:59, konrad.gutkowski@ffs.pl
?????:
> Recently i hit a bug with lustre and apache/vsftpd (using sendfile)
>
> symptoms are :
>
> kernel 2.6.12.6 and lustre 1.5.95 and vserver patch for 2.6.12
> - from inside a vserver i get only first ~114kb of any file when files are
> stored on lustre
> - from outside of a vserver i get whole file when they are stored on local
> fs
>
> kernel 2.6.18.1 and lustre 1.5.95 and vserver patch for 2.6.18
> - from inside of a vserver i get first ~64kb of any file when files are
> stored on lustre
> - from outside of a vserver i get whole file (same size) but transmission
> ends with an error (tested on vsftp server and windows client) when files
> are stored on lustre
> - when files are stored on local fs it works fine
>
> TIA
> Konrad Gutkowski
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss@clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
-- 
Alexey Lyashkov <shadow@clusterfs.com>

I have my own, high-performance, network-data-copy utility:

http://www.sandia.gov/MPSCP/mpscp_design.htm

I introduced sendfile last spring, to transfer files from an LFS on a 
Cray XT3, and also a full, Linux cluster. Initially it worked exactly 
per the man page description. I don''t know our levels, but I can get 
them if you are interested. In the last few months, we went through some 
upgrades. When I went back to the binary that had worked before, it 
crashed. I jacked with the sendfile code, and found an irregular 
workaround, which appears to give better performance than standard posix 
read and write. I''m conducting further testing right now.

Marty Barnaby



konrad.gutkowski@ffs.pl wrote:
> so i did, the patches was half a year old atleast, mostly for 
> development versions, ill make a non-VS kernel later to test it, also 
> will prolly do non lustre one and use VS from local hd for another test,
> but in the worst case it can be connected to both (it works on 
> non-lustre patched 2.6.15 with older VS ptach)
>
> Konrad Gutkowski
>
> ----- Original Message ----- From: "Alexey Lyashkov" 
> <shadow@clusterfs.com>
> To: <konrad.gutkowski@ffs.pl>
> Cc: <lustre-discuss@clusterfs.com>
> Sent: Wednesday, November 01, 2006 8:14 PM
> Subject: Re: [Lustre-discuss] Lustre breaking sendfile functionality
>
>
> how i remember vserver has many sendfile issues.
> look into vserver mail archive to corresponding patches.
>
> ? ???, 01.11.2006, ? 18:59, konrad.gutkowski@ffs.pl ?????:
>> Recently i hit a bug with lustre and apache/vsftpd (using sendfile)
>>
>> symptoms are :
>>
>> kernel 2.6.12.6 and lustre 1.5.95 and vserver patch for 2.6.12
>> - from inside a vserver i get only first ~114kb of any file when 
>> files are
>> stored on lustre
>> - from outside of a vserver i get whole file when they are stored on 
>> local
>> fs
>>
>> kernel 2.6.18.1 and lustre 1.5.95 and vserver patch for 2.6.18
>> - from inside of a vserver i get first ~64kb of any file when files are
>> stored on lustre
>> - from outside of a vserver i get whole file (same size) but 
>> transmission
>> ends with an error (tested on vsftp server and windows client) when 
>> files
>> are stored on lustre
>> - when files are stored on local fs it works fine
>>
>> TIA
>> Konrad Gutkowski
>>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss@clusterfs.com
>> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss