samba - May 2002 - copy a large volume of data from PC to samba server

Hi there,

Is there someone over there knows a method or script
that can copy a large volume of data (in folders) from PCs
to a samba server (suppose there is enough space there)?

I know using smbclient or smbtar can backup data on PCs
to samba server, but it is written in tar files.I wonder whether 
this method can handle a large folder, say, in 10GB size.

Also, using smbclient with mget and recurse on, one can
copy folders from PC to samba server. But in doing so,
it is in interactive mode and keeps asking yes/no to continue.
Is there a way to get rid of these prompts?

Thanks,
Tian.

-------------- next part --------------
HTML attachment scrubbed and removed

What kind of clients?

If it's NT/Win2k/XP, you could use the
"at" service to run a batch file...use
robocopy or scopy to do the transfer...

Jim

-----Original Message-----
From: Tian-xiong Lu [mailto:tianl@agrecon.canberra.edu.au]
Sent: Thursday, May 09, 2002 9:29 PM
To: samba@lists.samba.org
Subject: [Samba] copy a large volume of data from PC to samba server


Hi there,

Is there someone over there knows a method or script
that can copy a large volume of data (in folders) from PCs
to a samba server (suppose there is enough space there)?

I know using smbclient or smbtar can backup data on PCs
to samba server, but it is written in tar files.I wonder whether 
this method can handle a large folder, say, in 10GB size.

Also, using smbclient with mget and recurse on, one can
copy folders from PC to samba server. But in doing so,
it is in interactive mode and keeps asking yes/no to continue.
Is there a way to get rid of these prompts?

Thanks,
Tian.

Hi,
I guess I responded directly to you.  Sorry, I meant to respond to the list.

I was under the impression you had mounted the PC drive.  One way to mount it
is:
mount -t smbfs -o username=user,password=pass //computer/share /mnt/point

Then /dir1 = /mnt/point.  Of course, if /mnt/point is made available through
Samba,
then you can use Windoze Explorer.

I hope this helps.

Tian-xiong Lu (tianl@agrecon.canberra.edu.au) wrote*:
>
>Hi, thanks.
>
>but /dir1 is on the PC. How can I cd to there on a samba server (UNIX)?
>
>Tian.
>----- Original Message -----
>From: "Bob Crandell" <bob@assuredcomp.com>
>To: "Tian-xiong Lu" <tianl@agrecon.canberra.edu.au>
>Sent: Saturday, May 11, 2002 12:25 AM
>Subject: Re: [Samba] copy a large volume of data from PC to samba server
>
>
>> A friend of mine wrote this for me:
>> (cd /dir1; tar cf - .) | (cd /dir2; tar xfp -)
>> Where dir1 is the source and dir2 is your destination.  This was
written
>to protect
>> permissions on Linux drives but should do the job for you.
>>
>> Bob
>>
>> Tian-xiong Lu (tianl@agrecon.canberra.edu.au) wrote*:
>> >
>> >Hi there,
>> >
>> >Is there someone over there knows a method or script
>> >that can copy a large volume of data (in folders) from PCs
>> >to a samba server (suppose there is enough space there)?
>> >
>> >I know using smbclient or smbtar can backup data on PCs
>> >to samba server, but it is written in tar files.I wonder whether
>> >this method can handle a large folder, say, in 10GB size.
>> >
>> >Also, using smbclient with mget and recurse on, one can
>> >copy folders from PC to samba server. But in doing so,
>> >it is in interactive mode and keeps asking yes/no to continue.
>> >Is there a way to get rid of these prompts?
>> >
>> >Thanks,
>> >Tian.
>> >
>> >
>> --
>> Bob Crandell
>> Assured Computing
>> When you need to be sure.
>> Cell 541-914-3985
>> FAX  240-371-7237
>> bob@assuredcomp.com
>> www.assuredcomp.com
>> Eugene, Or. 97402
>>
>>
>
>
--
Bob Crandell
Assured Computing
When you need to be sure.
Cell 541-914-3985
FAX  240-371-7237
bob@assuredcomp.com
www.assuredcomp.com
Eugene, Or. 97402

Thanks, Bob.

Just checked documentation. mount -t smbfs works for Linux.
Unfortunately, I am using a Solaris box.

Anyone knows a way to see a windows share (physically on a PC) as a
(logical) directory on Solaris?

Thanks,
Tian.

----- Original Message -----
From: "Bob Crandell" <bob@assuredcomp.com>
To: "Tian-xiong Lu" <tianl@agrecon.canberra.edu.au>;
<samba@lists.samba.org>
Sent: Monday, May 13, 2002 11:42 AM
Subject: Re: [Samba] copy a large volume of data from PC to samba server

> Hi,
> I guess I responded directly to you.  Sorry, I meant to respond to the
list.
>
> I was under the impression you had mounted the PC drive.  One way to mount
it is:
> mount -t smbfs -o username=user,password=pass //computer/share /mnt/point
>
> Then /dir1 = /mnt/point.  Of course, if /mnt/point is made available
through Samba,
> then you can use Windoze Explorer.
>
> I hope this helps.
>
> Tian-xiong Lu (tianl@agrecon.canberra.edu.au) wrote*:
> >
> >Hi, thanks.
> >
> >but /dir1 is on the PC. How can I cd to there on a samba server (UNIX)?
> >
> >Tian.
> >----- Original Message -----
> >From: "Bob Crandell" <bob@assuredcomp.com>
> >To: "Tian-xiong Lu" <tianl@agrecon.canberra.edu.au>
> >Sent: Saturday, May 11, 2002 12:25 AM
> >Subject: Re: [Samba] copy a large volume of data from PC to samba
server
> >
> >
> >> A friend of mine wrote this for me:
> >> (cd /dir1; tar cf - .) | (cd /dir2; tar xfp -)
> >> Where dir1 is the source and dir2 is your destination.  This was
written
> >to protect
> >> permissions on Linux drives but should do the job for you.
> >>
> >> Bob
> >>
> >> Tian-xiong Lu (tianl@agrecon.canberra.edu.au) wrote*:
> >> >
> >> >Hi there,
> >> >
> >> >Is there someone over there knows a method or script
> >> >that can copy a large volume of data (in folders) from PCs
> >> >to a samba server (suppose there is enough space there)?
> >> >
> >> >I know using smbclient or smbtar can backup data on PCs
> >> >to samba server, but it is written in tar files.I wonder
whether
> >> >this method can handle a large folder, say, in 10GB size.
> >> >
> >> >Also, using smbclient with mget and recurse on, one can
> >> >copy folders from PC to samba server. But in doing so,
> >> >it is in interactive mode and keeps asking yes/no to continue.
> >> >Is there a way to get rid of these prompts?
> >> >
> >> >Thanks,
> >> >Tian.
> >> >
> >> >
> >> --
> >> Bob Crandell
> >> Assured Computing
> >> When you need to be sure.
> >> Cell 541-914-3985
> >> FAX  240-371-7237
> >> bob@assuredcomp.com
> >> www.assuredcomp.com
> >> Eugene, Or. 97402
> >>
> >>
> >
> >
> --
> Bob Crandell
> Assured Computing
> When you need to be sure.
> Cell 541-914-3985
> FAX  240-371-7237
> bob@assuredcomp.com
> www.assuredcomp.com
> Eugene, Or. 97402
>
>

Robocopy is in the NT & Win2k Resource Kit, and you can
download the most current copy from Microsoft separately.

I'd do the first copy manually, to make sure the
command line works.  After that, run the automatic
backup while you're logged in, and while you're
logged out to see if it authenticates ok.  Change
a file or two between each test, to give you a
quick file check for success/failure.

If you're using samba, then you can share the dir on
/path2, and copy to that via the UNC name.

Create the directory c:\batch, and put the files there.
--------------------
First copy;  all files in /dir1 to /path2:

dircopy.bat

attrib +A c:\data /s /sd
robocopy c:\data\ \\unixbox\path2
/r:2 /w:0 /e /v /LOG:c:\1stcopy.log /ETA /Z /SEC

(You may have to omit the /SEC if the unix/samba version
doesn't support NTFS security.)

attrib switches:
+A   Archive file attribute.  (Sets "on" archive bit)
/S  Processes matching files in the current folder
    and all subfolders.
/D  Processes folders as well.

robocopy switches:
/R:n : number of Retries on failed copies: default is 1 million.
/W:n : Wait time between retries: default is 30 seconds.
/E : copy subdirectories, including Empty ones.
/V : produce Verbose output, showing skipped files.
/LOG:file : output status to LOG file (overwrite existing log).
/ETA : show Estimated Time of Arrival of copied files.
/Z : copy files in restartable mode.
/SEC : copy SECurity info (both source and dest must be NTFS).

----------------------

Second copy;  all new/changed files in /dir1 to /path2:

dirupdat.bat
robocopy c:\data\ \\unixbox\path2
/r:2 /w:0 /e /v /LOG+:c:\dirupdat.log /ETA /SEC /Z /M

/M : copy only files with the Archive attribute set
     and remove Archive attribute from source files.
     (new files and changed files will have the
     archive bit set; only these will be copied)
/LOG+:file : output status to LOG file (append to existing log).

(You may have to omit the /SEC if the unix/samba version
doesn't support NTFS security.)

---------------------------------------

Copy a couple of new files into /dir1
after you run the test, and create a new
folder.  Then run dirupdat.bat with "at",
and look in /path2 to see if the new files
and folder show up after it's run.

  Look at the properties of the new files
and folder after the copy, and make sure the
Archive bit is not set (dirupdate.bat should
reset it when it copies it.)

second copy, "at" command line:

at 10:22 "c:\batch\dirupdat.bat"

10:22  : 3 minutes from now, your time.

-----------------------------

Third copy;  run with no users logged in:

at 10:42 "c:\batch\dirupdat.bat"


10:42  : 3 minutes from now, your time.  Log off
         after entering this command.

  Add files/a folder to make sure they're copied.
---------------------------------------------------

Once you know the copy process is working, you can
remove the /V from the batch files.  This will
reduce the size of the log files.

Then you can add the /every:date[...
to the "at" command and set the copying to run
every day/other day, etc.

You can run it in the morning or after hours;
more than once a day, if you need to.


---------------------------------------------------
The AT command schedules commands and programs to run on a computer at
a specified time and date. The Schedule service must be running to use
the AT command.

AT [\\computername] [ [id] [/DELETE] | /DELETE [/YES]]
AT [\\computername] time [/INTERACTIVE]
    [ /EVERY:date[,...] | /NEXT:date[,...]] "command"

\\computername     Specifies a remote computer. Commands are scheduled on
the
                   local computer if this parameter is omitted.
id                 Is an identification number assigned to a scheduled
                   command.
/delete            Cancels a scheduled command. If id is omitted, all the
                   scheduled commands on the computer are canceled.
/yes               Used with cancel all jobs command when no further
                   confirmation is desired.
time               Specifies the time when command is to run.
/interactive       Allows the job to interact with the desktop of the user
                   who is logged on at the time the job runs.
/every:date[,...]  Runs the command on each specified day(s) of the week or
                   month. If date is omitted, the current day of the month
                   is assumed.
/next:date[,...]   Runs the specified command on the next occurrence of the
                   day (for example, next Thursday).  If date is omitted,
the
                   current day of the month is assumed.
"command"          Is the Windows NT command, or batch program to be
run.

----------------------------------------------------------------------------
---------

Hope this helps,

Jim



> -----Original Message-----
> From: Tian-xiong Lu [mailto:tianl@agrecon.canberra.edu.au]
> Sent: Sunday, May 12, 2002 3:01 PM
> To: Van Sickler, Jim
> Subject: Re: [Samba] copy a large volume of data from PC to 
> samba server
> 
> 
> Thanks for the reply.
> We use NT and W2K.
> Both UNIX and windows have "at". But I can find command 
> robocopy and scopy
> either on PC or on UNIX box (solaris 7).
> Can you give an example, say I want to copy directory \dir1 on PC1 to
> directory /path2 on UNIX (Solaris) UNIX2.
> 
> Thanks,
> Tian.
> 
> ----- Original Message -----
> From: "Van Sickler, Jim" <vansickj-eodc@Kaman.com>
> To: "'Tian-xiong Lu'"
<tianl@agrecon.canberra.edu.au>;
> <samba@lists.samba.org>
> Sent: Saturday, May 11, 2002 3:18 AM
> Subject: RE: [Samba] copy a large volume of data from PC to 
> samba server
> 
> 
> > What kind of clients?
> >
> > If it's NT/Win2k/XP, you could use the
> > "at" service to run a batch file...use
> > robocopy or scopy to do the transfer...
> >
> > Jim
> >
> > -----Original Message-----
> > From: Tian-xiong Lu [mailto:tianl@agrecon.canberra.edu.au]
> > Sent: Thursday, May 09, 2002 9:29 PM
> > To: samba@lists.samba.org
> > Subject: [Samba] copy a large volume of data from PC to samba server
> >
> >
> > Hi there,
> >
> > Is there someone over there knows a method or script
> > that can copy a large volume of data (in folders) from PCs
> > to a samba server (suppose there is enough space there)?
> >
> > I know using smbclient or smbtar can backup data on PCs
> > to samba server, but it is written in tar files.I wonder whether
> > this method can handle a large folder, say, in 10GB size.
> >
> > Also, using smbclient with mget and recurse on, one can
> > copy folders from PC to samba server. But in doing so,
> > it is in interactive mode and keeps asking yes/no to continue.
> > Is there a way to get rid of these prompts?
> >
> > Thanks,
> > Tian.
>