On Fri, 2021-08-06 at 15:06 +0200, SZIGETV?RI J?nos via samba wrote:> Hi Rowland and Nico, > > I wasn't at all saying that the problem is rsync related. I just > included > its output and the error string for context. > The whole thing seems to us as if something would go wrong between > the > kernel CIFS code and the remote Windows server providing the share, > after a > few hours into the copy operation. > > Unfortunately this is an appliance we are talking about, and we can't > easily modify subcomponents to work entirely differently than before. > Regarding the directory structure of the data we copy: we usually aim > to > move few or several directories every day, with each directory > containing > about 150 GB of data, with one larger file (around 100 GB in size), > and > 1000+ smaller files, with a size of below 100 MB each. Either way, we > far > from tens of thousands of files per directory. > > The source files that are being moved are not in active use. We are > talking > about an archive job for files that are not being written to any > longer. > And even on the destination side, the only purpose of the storage > server is > to work as an off-appliance storage space, so neither the host, nor > are > other clients accessing the files on the destination side, while they > are > being copied or even afterwards. > The archive jobs use locks to ensure that only one is running at a > time, > and execute sequentially if multiple jobs queue up. > > So all in all, I would kindly ask for tips how to troubleshoot this > problem, and whether a bug report should be opened for Samba or the > kernel > CIFS code, or if I should ask for help at one of the other samba-* > mailing > lists. >I seem to remember saying that rsync has its own mailing list and so has the CIFS protocol: http://lists.samba.org/pipermail/cifs-protocol/ It looks to myself that Samba has nothing to do with your problem (I could be wrong). I think you need to ask on the cifs-protocol list, they should be able to point you in the right direction, even if it is back here. Rowland
ve2cii at ve2cii.com
2021-Aug-06 13:29 UTC
[Samba] rsync copy operation fails on a CIFS mount
??? Hi, ?? I am going to go out on a limb here. For windows there is a timer for sharing that will disconnect the share after some time. This was broken at one point and the share would disconnect even when data was flowing.? I solved my issue by turning off this timer. On 8/6/21 09:25, Rowland Penny via samba wrote:> On Fri, 2021-08-06 at 15:06 +0200, SZIGETV?RI J?nos via samba wrote: >> Hi Rowland and Nico, >> >> I wasn't at all saying that the problem is rsync related. I just >> included >> its output and the error string for context. >> The whole thing seems to us as if something would go wrong between >> the >> kernel CIFS code and the remote Windows server providing the share, >> after a >> few hours into the copy operation. >> >> Unfortunately this is an appliance we are talking about, and we can't >> easily modify subcomponents to work entirely differently than before. >> Regarding the directory structure of the data we copy: we usually aim >> to >> move few or several directories every day, with each directory >> containing >> about 150 GB of data, with one larger file (around 100 GB in size), >> and >> 1000+ smaller files, with a size of below 100 MB each. Either way, we >> far >> from tens of thousands of files per directory. >> >> The source files that are being moved are not in active use. We are >> talking >> about an archive job for files that are not being written to any >> longer. >> And even on the destination side, the only purpose of the storage >> server is >> to work as an off-appliance storage space, so neither the host, nor >> are >> other clients accessing the files on the destination side, while they >> are >> being copied or even afterwards. >> The archive jobs use locks to ensure that only one is running at a >> time, >> and execute sequentially if multiple jobs queue up. >> >> So all in all, I would kindly ask for tips how to troubleshoot this >> problem, and whether a bug report should be opened for Samba or the >> kernel >> CIFS code, or if I should ask for help at one of the other samba-* >> mailing >> lists. >> > I seem to remember saying that rsync has its own mailing list and so > has the CIFS protocol: http://lists.samba.org/pipermail/cifs-protocol/ > > It looks to myself that Samba has nothing to do with your problem (I > could be wrong). I think you need to ask on the cifs-protocol list, > they should be able to point you in the right direction, even if it is > back here. > > Rowland > > >