rsync - Jan 2002 - meaning of "IO Error: skipping the delete...."

On 23 Jan 2002, Nitin Agarwal <nitin.agarwal@timesgroup.com>
wrote:
> Hi,
> I am Nitin from India working with a leading Indian newspaper "The
Times
> 
> of India".
> We recently downloaded "rsync" and found it to be very useful and
good
> utility.
> We tried to implement it between our main server and the DRS (Disaster
> Recovery System), located in different locations, connected by leased
> lines. The data that we want to synchronize using this utility is around
> 
> 80 GB.
> 
> We were successfull in synchronizing the data first time, but later on
> when we tried to use the "--delete" feature (to delete the files
in DRS
> which no longer exists in the main servers) by using following command:
> [rsync -avz --delete host::module /PPI/production]
> we encounterd the following error:
> [IO Error: skipping the delete.....]
> and the rest of the process continued.
> 
> Can you please let me know the reason & solution for the same.
Yes, this means that there was an error copying one of the files.  A
common possibility is that the process did not have permission to read
or write one of the files.

Because a problem occurred, rsync refrains from deleting any files.

This is because the error may reflect a configuration problem, and we
don't want to delete files that you may have wanted to preserve just
because you made a mistake on the command line.

If you look through the error output, or perhaps turn off -v, then you
should see messages explaining the errors.
> Thanks & Regards
> Nitin Agarwal
-- 
Martin

Thanks for replying.
As adviced by you, we again ran the command by removing the -v option. The
error it gives now is as follows:
readlink dispatcher/9/done/020119.185124.0000251489.0003: No such file or
directory

We went through our file system for finding the above file, and we found the
file name to be different. It was: "020119.185124.0000251489.3" And
this file
was NOT PRESENT in the main server (source)

The same error was encountered for many files, and all the error messages were
showing the last extension of the file to be added with "000".

OUR VIEWS OVER IT:
There could be something programed in rsync which while deleting the files
(which are no longer in the main server) is adding "000" to the last
extension. (.... i mean... could be something like integer extension or
something ?????)

Kindly help me over this....

Thanks & Regards
Nitin Agarwal
NOTE: 1) PLEASE NOTE THAT WE ARE HAVING VERY LARGE FILE SYSTEM. (THE NO. OF
FILES IS VERY LARGE)
2) THE EXACT ERROR WHICH I TOLD U IN MY LAST MAIL WAS "IO Error
encountered:
Skipping file deletion
Thanks
Nitin

Martin Pool wrote:
> On 23 Jan 2002, Nitin Agarwal <nitin.agarwal@timesgroup.com> wrote:
> > Hi,
> > I am Nitin from India working with a leading Indian newspaper
"The Times
> >
> > of India".
> > We recently downloaded "rsync" and found it to be very
useful and good
> > utility.
> > We tried to implement it between our main server and the DRS (Disaster
> > Recovery System), located in different locations, connected by leased
> > lines. The data that we want to synchronize using this utility is
around
> >
> > 80 GB.
> >
>
> > We were successfull in synchronizing the data first time, but later on
> > when we tried to use the "--delete" feature (to delete the
files in DRS
> > which no longer exists in the main servers) by using following
command:
> > [rsync -avz --delete host::module /PPI/production]
> > we encounterd the following error:
> > [IO Error: skipping the delete.....]
> > and the rest of the process continued.
> >
> > Can you please let me know the reason & solution for the same.
>
> Yes, this means that there was an error copying one of the files.  A
> common possibility is that the process did not have permission to read
> or write one of the files.
>
> Because a problem occurred, rsync refrains from deleting any files.
>
> This is because the error may reflect a configuration problem, and we
> don't want to delete files that you may have wanted to preserve just
> because you made a mistake on the command line.
>
> If you look through the error output, or perhaps turn off -v, then you
> should see messages explaining the errors.
>
> > Thanks & Regards
> > Nitin Agarwal
>
> --
> Martin

On 23 Jan 2002, Nitin Agarwal <nitin.agarwal@timesgroup.com>
wrote:
> Thanks for replying.
> As adviced by you, we again ran the command by removing the -v option. The
> error it gives now is as follows:
> readlink dispatcher/9/done/020119.185124.0000251489.0003: No such file or
> directory
> 
> We went through our file system for finding the above file, and we found
the
> file name to be different. It was: "020119.185124.0000251489.3"
And this file
> was NOT PRESENT in the main server (source)
Was some other program accessing this area of the filesystem while
rsync was running?  It seems likely that this file was created and
then deleted all while rsync was running.  As a result, rsync expected
to be able to transfer the file, and was disappointed not to do so.

If the same problem occurs and you are sure no other program is
manipulating the directory then there may be a problem in rsync.

--
Martin

On 24 Jan 2002, Nitin Agarwal <nitin.agarwal@timesgroup.com>
wrote:
> Dear Mr. Martin,
> I am dead sure that none of the process was running which was creating
files on
> that file system. Further, our process are not creating any files having
last
> extension as "0003". Kindly provide us with the solution of the
same.
Please use the --ignore-errors option, and rsync will delete files on
the destination in this situation.

--
Martin

On 25 Jan 2002, Nitin Agarwal <nitin.agarwal@timesgroup.com>
wrote:
> Dear  Mr. Martin,
> Thanks for the reply.
> We tried as per your advice. Now, its not showing us any errors.
> 
> But, we observed that while copying the files from the main server to the
backup
> machine, rsync is changing the file names of those files having last
extension as any
> single digit integer (e.g. ".3", ".2", ".1").
While copying it shows us copying the
> file with name "12345.1234554321.3" to the destination as
"12345.1234554321.0003".
> i.e. it is adding "000" to the extension.
This is very strange behaviour.  I cannot at the moment think of any
aspect of rsync that would cause it to manipulate the filenames in
this way.

What operating system and what version of rsync are you using?  

Can you please find a minimal set of instructions that recreate the
problem: in other words, the commands to run to create files with the
names that are causing problems, and then the rsync command that
copies them.

--
Martin

Hi,
I am doing the testing over this. As soon as the error will repeat, I will send
u a
snapshot of the same, and the command i used that time.

Right now, we are facing another problem, it is showing the following error:
"send_files failed to open dir1/dir2/dir3/abcde.eps : permission
denied"

Kindly let us know the reason.

Thanks & Regards
Nitin Agarwal


Martin Pool wrote:
> On 25 Jan 2002, Nitin Agarwal <nitin.agarwal@timesgroup.com> wrote:
> > Dear  Mr. Martin,
> > Thanks for the reply.
> > We tried as per your advice. Now, its not showing us any errors.
> >
> > But, we observed that while copying the files from the main server to
the backup
> > machine, rsync is changing the file names of those files having last
extension as any
> > single digit integer (e.g. ".3", ".2",
".1"). While copying it shows us copying the
> > file with name "12345.1234554321.3" to the destination as
"12345.1234554321.0003".
> > i.e. it is adding "000" to the extension.
>
> This is very strange behaviour.  I cannot at the moment think of any
> aspect of rsync that would cause it to manipulate the filenames in
> this way.
>
> What operating system and what version of rsync are you using?
>
> Can you please find a minimal set of instructions that recreate the
> problem: in other words, the commands to run to create files with the
> names that are causing problems, and then the rsync command that
> copies them.
>
> --
> Martin

Dear Mr. Martin,
Thanks for the replies. Our problems are sorted out upto an extend.
We are facing one more problem....
i.e. while transferring the files, sometimes the transfer breaks in between and
gives us
the error message:
readerror: connection reset by peer

We are having a large file system (approx. 100 GB).
Kindly let us know the reason behind this.
Regards
Nitin Agarwal

Martin Pool wrote:
> On 25 Jan 2002, Nitin Agarwal <nitin.agarwal@timesgroup.com> wrote:
> > Dear  Mr. Martin,
> > Thanks for the reply.
> > We tried as per your advice. Now, its not showing us any errors.
> >
> > But, we observed that while copying the files from the main server to
the backup
> > machine, rsync is changing the file names of those files having last
extension as any
> > single digit integer (e.g. ".3", ".2",
".1"). While copying it shows us copying the
> > file with name "12345.1234554321.3" to the destination as
"12345.1234554321.0003".
> > i.e. it is adding "000" to the extension.
>
> This is very strange behaviour.  I cannot at the moment think of any
> aspect of rsync that would cause it to manipulate the filenames in
> this way.
>
> What operating system and what version of rsync are you using?
>
> Can you please find a minimal set of instructions that recreate the
> problem: in other words, the commands to run to create files with the
> names that are causing problems, and then the rsync command that
> copies them.
>
> --
> Martin