Gluster users - Jun 2011 - Strange errors reading/writing/editing/deleting JPGs, PDFs and PNG from PHP Application

Hello,

I have a PHP web application that uses Gluster to store its files.
There are a few areas of the application that perform multiple
operations on small to medium size files (such as .jpg and .pdf files)
in quick succession.? My dev environment does not use Gluster and had
no problems - but in production, I am seeing some strange errors and
am wondering if perhaps Gluster could be the cause.

Example 1: I may perform the following sequence of operations on a
batch of 3,000 photos:

1. Copy the photo from remote URL to a temp folder
2. Create a unique directory based on that photo_id
3. Move the photo from the temp dir to the new dir
4. Create a medium (1000px), small (250px), and tiny (72px) versions
of the photo and save them to the temp dir.

Example 2: Turning a PDF to a PNG

1. Create a PDF document, ranging from a few MB to tens of MB or more.
2. Use Ghostscript to read a specific page and create a temp .png file from it.
3. Use Imagemagick to resize the .png and add a drop shadow.
4. Delete the temp .png file

In example 1, I am finding that some of the resulting JPG files wind
up being corrupted - when attempting to read the problematic JPGs, I
receive messages such as "bad Huffman code" or "premature end of
data
segment".

In example 2, I am getting strange errors on step 3 - it sometimes has
a problem finding the file created by step 2.? And then on step 4, I
sometimes get an error that really baffles me - a line of PHP code
such as:

? if (is_file($file)) unlink($file);

.. will sometimes produce an error "unlink: no such file or
directory".? I have no idea how it passes the is_file() check and then
unlink() says it's not there.

At this point I'm just wondering if Gluster could be the culprit for
any of this strange behavior, considering the types of operations and
file sizes I'm working with. ?I've included my Gluster volume info
below.

Thanks for the help.

- Alan

Gluster Volume Info
[alan at file1:10.X.X.X:d1]$?sudo cat info
type=2
count=2
status=1
sub_count=2
version=1
transport-type=0
volume-id=3fc69046-a324-42de-bf8a-e1bd2e6e45ab
brick-0=10.X.X.X:-data-d1
brick-1=10.Y.Y.Y.38:-data-d1
performance.cache-size=1024MB
performance.write-behind-window-size=4MB
performance.io-thread-count=8
[alan at file1:10.X.X.X:d1]$

Do you find anything in the client logs?

On Fri, Jun 10, 2011 at 3:20 AM, Alan Zapolsky <alan at
droptheworld.com>wrote:
> Hello,
>
> I have a PHP web application that uses Gluster to store its files.
> There are a few areas of the application that perform multiple
> operations on small to medium size files (such as .jpg and .pdf files)
> in quick succession.  My dev environment does not use Gluster and had
> no problems - but in production, I am seeing some strange errors and
> am wondering if perhaps Gluster could be the cause.
>
> Example 1: I may perform the following sequence of operations on a
> batch of 3,000 photos:
>
> 1. Copy the photo from remote URL to a temp folder
> 2. Create a unique directory based on that photo_id
> 3. Move the photo from the temp dir to the new dir
> 4. Create a medium (1000px), small (250px), and tiny (72px) versions
> of the photo and save them to the temp dir.
>
> Example 2: Turning a PDF to a PNG
>
> 1. Create a PDF document, ranging from a few MB to tens of MB or more.
> 2. Use Ghostscript to read a specific page and create a temp .png file from
> it.
> 3. Use Imagemagick to resize the .png and add a drop shadow.
> 4. Delete the temp .png file
>
> In example 1, I am finding that some of the resulting JPG files wind
> up being corrupted - when attempting to read the problematic JPGs, I
> receive messages such as "bad Huffman code" or "premature
end of data
> segment".
>
> In example 2, I am getting strange errors on step 3 - it sometimes has
> a problem finding the file created by step 2.  And then on step 4, I
> sometimes get an error that really baffles me - a line of PHP code
> such as:
>
>   if (is_file($file)) unlink($file);
>
> .. will sometimes produce an error "unlink: no such file or
> directory".  I have no idea how it passes the is_file() check and then
> unlink() says it's not there.
>
> At this point I'm just wondering if Gluster could be the culprit for
> any of this strange behavior, considering the types of operations and
> file sizes I'm working with.  I've included my Gluster volume info
> below.
>
> Thanks for the help.
>
> - Alan
>
> Gluster Volume Info
> [alan at file1:10.X.X.X:d1]$ sudo cat info
> type=2
> count=2
> status=1
> sub_count=2
> version=1
> transport-type=0
> volume-id=3fc69046-a324-42de-bf8a-e1bd2e6e45ab
> brick-0=10.X.X.X:-data-d1
> brick-1=10.Y.Y.Y.38:-data-d1
> performance.cache-size=1024MB
> performance.write-behind-window-size=4MB
> performance.io-thread-count=8
> [alan at file1:10.X.X.X:d1]$
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110610/053ed2ac/attachment.html>