rsync - Jul 2003 - Some C help patching sender.c (from:plain source -> encrypted destination: rsync + gpg)

Hi,

I am exploring extending Kyle Jones' patch (which implements 
post-transfer filter in receiver.c) to pre-filter or otherwise 
pre-process the file before it is sent.

Kyle Jones patch can be found here
<http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&selm=b6f55s%24256q%241%40FreeBSD.csie.NCTU.edu.tw&rnum=6>

and the rationale for what I am trying to do here
<http://marc.theaimsgroup.com/?l=rsync&m=105660948415347&w=2>

Down to what I am asking help with: I am _not_ a C hacker at all, my 
strenghts are Perl -- if anywhere at all. So I would appreciate your 
advise before I shoot myself in the foot.

Thanks for your patience.

I have studied the sources and  gotten down to sender.c (cvs 1.17), in 
particular the line where it calls do_open() and I am intending to 
create do_popen as an alternative to do_open() (in syscall.c).

I have seen Kyle's approach of handling the forking directly, and I know 
I cannot handle that level of complexity. Using something like popen (I 
am using 
<http://www.gnu.org/manual/glibc-2.2.3/html_chapter/libc_15.html> as 
reference) I can probably get away with.

One of my concerns is that even though we are setting --whole-file, when 
I follow the program logic and the output of test runs with -vvv it is 
still computing CRCs and (at least seems to be) preparing deltas.

Hmmm, re-reading it now it seems that the do_open() I am thinking of is 
in an if(!read_batch) block and the CRC/deltas are under if(read_batch). 
Is read_batch always true when using --whole-files? Seems to be.

Second concern: where is the reading and transmission of th file 
happening? If the do_open is in line 183, where's the read/transmit, 
map_file() line 202 puts the data uin buf, and then, what? You see, 
write_int doesn't use buf.

And where is the handle closed?

Thanks for any tips here...

Ahhh, life.






martin
http://nzl.com.ar/

On Mon, Jul 07, 2003 at 11:42:04PM +1200, Martin Langhoff
wrote:
> Hi,
> 
> I am exploring extending Kyle Jones' patch (which implements 
> post-transfer filter in receiver.c) to pre-filter or otherwise 
> pre-process the file before it is sent.
> 
> Kyle Jones patch can be found here
>
<http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&selm=b6f55s%24256q%241%40FreeBSD.csie.NCTU.edu.tw&rnum=6>
> 
> and the rationale for what I am trying to do here
> <http://marc.theaimsgroup.com/?l=rsync&m=105660948415347&w=2>
> 
> Down to what I am asking help with: I am _not_ a C hacker at all, my 
> strenghts are Perl -- if anywhere at all. So I would appreciate your 
> advise before I shoot myself in the foot.
My _advice_ is to miss.
> Thanks for your patience.
> 
> I have studied the sources and  gotten down to sender.c (cvs 1.17), in 
> particular the line where it calls do_open() and I am intending to 
> create do_popen as an alternative to do_open() (in syscall.c).
>
> I have seen Kyle's approach of handling the forking directly, and I
know
> I cannot handle that level of complexity. Using something like popen (I 
> am using 
> <http://www.gnu.org/manual/glibc-2.2.3/html_chapter/libc_15.html> as 
> reference) I can probably get away with.
Fine, as long as it is for your own use only.
> One of my concerns is that even though we are setting --whole-file, when 
> I follow the program logic and the output of test runs with -vvv it is 
> still computing CRCs and (at least seems to be) preparing deltas.
Correct.  As can be found by a grep for whole_file,
whole-file works by operating on an empty blocksum array.
Besides we want the CRCs calculated as an extra level of
integrity checks.
> Hmmm, re-reading it now it seems that the do_open() I am thinking of is 
> in an if(!read_batch) block and the CRC/deltas are under if(read_batch). 
> Is read_batch always true when using --whole-files? Seems to be.
No.  Read_batch is only true when set using the command-line
option.
> Second concern: where is the reading and transmission of th file 
> happening? If the do_open is in line 183, where's the read/transmit, 
> map_file() line 202 puts the data uin buf, and then, what? You see, 
> write_int doesn't use buf.
> 
> And where is the handle closed?
Look further down, buf is transmitted by match_sums() and fd is
closed shortly thereafter.




-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt