John E. Malmberg
2003-Oct-12 13:21 UTC
rsync.h always including syslog.h even when not used.
Hello, rsync.h is always including the syslog.h file, even though it is only used when the LOG_DAEMON macro is defined. Could someone change it to be as follows: #ifdef LOG_DAEMON #include <syslog.h> #endif OpenVMS currently does not have a syslog facility, so it does not have a syslog.h. Thank you, -John wb8tyw@qsl.net Personal Opinion Only
On Sat, Oct 11, 2003 at 11:21:19PM -0400, John E. Malmberg wrote:> Hello, > > rsync.h is always including the syslog.h file, even though it is only > used when the LOG_DAEMON macro is defined. > > Could someone change it to be as follows: > > #ifdef LOG_DAEMON > #include <syslog.h> > #endifWon't work. LOG_DAEMON is defined in syslog.h.> > OpenVMS currently does not have a syslog facility, so it does not have a > syslog.h.If you don't have a syslog facility i'd expect you to be getting link errors. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt
John E. Malmberg wrote:> > While my testing is far from complete, and I have a few OpenVMS specific > things to work out, it appears that the changes to make the client use > pthreads were minor, and as of yesterday a client on OpenVMS is able to > properly replicate a sample directory of plain text files from a local > server running rsync on Windows 2000.I just successfully used rsync running on OpenVMS to download the current rsync development tree. I deleted a few files, and reran rsync which repopulated just the missing files. It looks like I have a working client. :-) My next step is to make sure that I can build the current CVS build and post a "beta" kit for ftp download with build instructions for the other OpenVMS developers that have expressed an interest in working on this. -John wb8tyw@qsl.network Personal Opinion Only
On Mon, Oct 20, 2003 at 12:20:27AM -0400, John E. Malmberg wrote:> John E. Malmberg wrote: > > > >While my testing is far from complete, and I have a few OpenVMS specific > >things to work out, it appears that the changes to make the client use > >pthreads were minor, and as of yesterday a client on OpenVMS is able to > >properly replicate a sample directory of plain text files from a local > >server running rsync on Windows 2000. > > I just successfully used rsync running on OpenVMS to download the > current rsync development tree. I deleted a few files, and reran rsync > which repopulated just the missing files. > > It looks like I have a working client. :-) > > My next step is to make sure that I can build the current CVS build and > post a "beta" kit for ftp download with build instructions for the other > OpenVMS developers that have expressed an interest in working on this.Sounds promising. The pitfall you with rsync in threads is that rsync forks with a COW expectation using a great deal of data set prior to the fork. Some of that data is altered. In particular a slew of global variables that must become thread unique when modified or things will break in subtle ways. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt
jw schultz wrote:> > Sounds promising. > > The pitfall you with rsync in threads is that rsync forks > with a COW expectation using a great deal of data set prior > to the fork. Some of that data is altered. In particular a > slew of global variables that must become thread unique when > modified or things will break in subtle ways.Yes, I can see a few cases of this. There are approximately 235 global and static variables in rsync, and of those 105 are obviously never modified after the fork() takes place. That may be the case for several of the others, like the 10 in batch.c As the use of the remaining ones are identified, that issue is easily fixed. What I need to do is map the routines called by the receive process/thread to see what static/global variables that they modify. All the more reason to get more people working on this that want it to work on the OpenVMS operating system. Several people have contacted me that they are willing to assist. -John wb8tyw@qsl.net Personal Opinion Only
On Tue, Oct 21, 2003 at 11:28:11PM -0400, John E. Malmberg wrote:> jw schultz wrote: > > > >Sounds promising. > > > >The pitfall you with rsync in threads is that rsync forks > >with a COW expectation using a great deal of data set prior > >to the fork. Some of that data is altered. In particular a > >slew of global variables that must become thread unique when > >modified or things will break in subtle ways. > > Yes, I can see a few cases of this. > > There are approximately 235 global and static variables in rsync, and > of those 105 are obviously never modified after the fork() takes place. > That may be the case for several of the others, like the 10 in batch.c > > As the use of the remaining ones are identified, that issue is easily > fixed. What I need to do is map the routines called by the receive > process/thread to see what static/global variables that they modify. > > All the more reason to get more people working on this that want it to > work on the OpenVMS operating system. Several people have contacted me > that they are willing to assist.If you put the ones that get modified post-fork into a thread-local structure there is a reasonable chance of that getting into mainline. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt
jw schultz wrote:> On Tue, Oct 21, 2003 at 11:28:11PM -0400, John E. Malmberg wrote: > >> >>There are approximately 235 global and static variables in rsync, and >>of those 105 are obviously never modified after the fork() takes place. >> That may be the case for several of the others, like the 10 in batch.c >> >>As the use of the remaining ones are identified, that issue is easily >>fixed. What I need to do is map the routines called by the receive >>process/thread to see what static/global variables that they modify. >> > > If you put the ones that get modified post-fork into a > thread-local structure there is a reasonable chance of that > getting into mainline.Now what would be more acceptable: Adding a context parameter to all the routines that reference the global/static variables that are currently shared? or Having each routine that uses those type of variables to go through the work of determining which thread that they are in? It seems to me that adding the context parameter would make it easier to maintain, and have a lower overhead. It also saves having thread lookup calls added to the routines. In some other cases, it looks like the sharing should not hurt, but their should be some synchronization. Since there are only two processes or threads that need to be tracked, while the parameter could be a pointer to global structure, it can be disruptive to put all the local thread specific static values in that structure. Mainly because a lot of them have the same symbol names, just different scope. So there could be a global variable that is void * recv_context, and a void * send_context for the sender for completeness. A routine that needs separate local static variables would check the context parameter passed to it against one of those pointers, and then use the appropriate set of local variables. With this, the static variables would become an array, and the context parameter would determine if index 0 or 1 was used. -John wb8tyw@qsl.net Personal Opinion Only
On Wed, Oct 22, 2003 at 11:59:48PM -0400, John E. Malmberg wrote:> jw schultz wrote: > >On Tue, Oct 21, 2003 at 11:28:11PM -0400, John E. Malmberg wrote: > > > >> > >>There are approximately 235 global and static variables in rsync, and > >>of those 105 are obviously never modified after the fork() takes place. > >>That may be the case for several of the others, like the 10 in batch.c > >> > >>As the use of the remaining ones are identified, that issue is easily > >>fixed. What I need to do is map the routines called by the receive > >>process/thread to see what static/global variables that they modify. > >> > > > >If you put the ones that get modified post-fork into a > >thread-local structure there is a reasonable chance of that > >getting into mainline. > > Now what would be more acceptable: > > Adding a context parameter to all the routines that reference the > global/static variables that are currently shared? > > or > > Having each routine that uses those type of variables to go through the > work of determining which thread that they are in? > > > It seems to me that adding the context parameter would make it easier to > maintain, and have a lower overhead. It also saves having thread lookup > calls added to the routines. > > In some other cases, it looks like the sharing should not hurt, but > their should be some synchronization.Long term thread support would be to use a pointer in thread local storage or to pass it to any functions that use it. For now just define the structure and treat it as global in mainline.> Since there are only two processes or threads that need to be tracked, > while the parameter could be a pointer to global structure, it can be > disruptive to put all the local thread specific static values in that > structure. Mainly because a lot of them have the same symbol names, > just different scope. > > So there could be a global variable that is void * recv_context, and a > void * send_context for the sender for completeness.That sounds like it might be OK to me but will require much more thought and discussion.> A routine that needs separate local static variables would check the > context parameter passed to it against one of those pointers, and then > use the appropriate set of local variables.That sort of indirection just complicates things. If you are going to pass a context parameter just pass the pointer to the structure itself.> With this, the static variables would become an array, and the context > parameter would determine if index 0 or 1 was used.There are up to three processes. Local has sender, generator and receiver. Daemon has the daemon and either the sender, or the generator and receiver. Client has either sender or generator and receiver. Getting it threadable will have to be done by inches. The structure is a single step that should help to reduce the size of your patches while enhancing the documentation of some of the globals. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt