If any kernel module has the following, or a similar line in it:
-----
char x[100] = {0};
-----
building of the GENERIC kernel on FreeBSD 5 -STABLE for amd64
as of 03/19/05, fails with the following message at the time of linking:
"undefined reference to `memset'".
The same problem is not seen on i386.
The problem goes away if the above line is changed to:
-----
char x[100];
memset(x, 0, 100);
-----
Adding CFLAGS+=-fbuiltin, or CFLAGS+=-fno-builtin to /sys/conf/Makefile.amd64
does not help.
Anyone knows what's happening?
Thanks,
Vinod.
On Wednesday 23 March 2005 01:48 pm, Vinod Kashyap wrote:> If any kernel module has the following, or a similar line in it: > ----- > char x[100] = {0}; > ----- > building of the GENERIC kernel on FreeBSD 5 -STABLE for amd64 > as of 03/19/05, fails with the following message at the time of > linking: "undefined reference to `memset'". > > The same problem is not seen on i386. > > The problem goes away if the above line is changed to: > ----- > char x[100]; > memset(x, 0, 100); > ----- > > Adding CFLAGS+=-fbuiltin, or CFLAGS+=-fno-builtin to > /sys/conf/Makefile.amd64 does not help. > > Anyone knows what's happening?Something fishy is going on. I've tried this on both 5.x and 6.0 and both systems created a bss object called 'x', not something that called memset. For example: peter@fb5-amd64[9:17pm]/home/src/sys/modules/twe-21# nm obj/twe.ko | grep x U busdma_lock_mutex U sysctl_ctx_free U sysctl_ctx_init 0000000000000080 b x peter@fb5-amd64[9:17pm]/home/src/sys/modules/twe-22# nm obj/twe.ko | grep memset peter@fb5-amd64[9:17pm]/home/src/sys/modules/twe-23# I wondered if it might be because of something like -O2 (don't do that) or no -O at all, but I couldn't make it happen even then. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5
On Wed, 2005-Mar-23 13:48:04 -0800, Vinod Kashyap wrote:>If any kernel module has the following, or a similar line in it: >----- >char x[100] = {0}; >----- >building of the GENERIC kernel on FreeBSD 5 -STABLE for amd64 >as of 03/19/05, fails with the following message at the time of linking: >"undefined reference to `memset'". > >The same problem is not seen on i386. > >The problem goes away if the above line is changed to: >----- >char x[100]; >memset(x, 0, 100); >-----Can you post a complete (compilable) example please. Based on your second example, I suspect that you are putting the variable declaration inside a function definition - the second example doesn't make sense outside a function. If I add "char x[100] = {0};" into a function on i386 and compile it as a kernel module on 5.3, a static memset symbol is generated - it's possible that the amd64 compiler gets confused about the implicit reference to memset that this code needs. -- Peter Jeremy
On Wed, 23 Mar 2005, Vinod Kashyap wrote:> If any kernel module has the following, or a similar line in it: > ----- > char x[100] = {0}; > -----I think you mean: ----- auto char x[100] = {0}; ----- or after fixing some style bugs: ----- char x[100] = { 0 }; -----> building of the GENERIC kernel on FreeBSD 5 -STABLE for amd64 > as of 03/19/05, fails with the following message at the time of linking: > "undefined reference to `memset'". > > The same problem is not seen on i386. > > The problem goes away if the above line is changed to: > ----- > char x[100]; > memset(x, 0, 100); > -----This version makes the pessimizations and potential bugs clear: - clearing 100 bytes on every entry to the function is wasteful. C90's auto initializers hide pessimizations like this. They should be used very rarely, especially in kernels. But they are often misused, even in kernels, even for read-only data that should be static. gcc doesn't optimize even "auto const x[100] = { 0 };" to a static initialization -- the programmer must declare the object as static to prevent gcc laboriously clearing it on every entry to the function. - 100 bytes may be too much to put on the kernel stack. Objects just a little larger than this must be dynamically allocated unless they can be read-only.> Adding CFLAGS+=-fbuiltin, or CFLAGS+=-fno-builtin to /sys/conf/Makefile.amd64 > does not help.-fno-builtin is already in CFLAGS, and if it has any effect on this then it should be to cause gcc to generate a call to memset() instead of doing the memory clearing inline. I think gcc has a builtin memset() which is turned off by -fno-builtin, but -fno-builtin doesn't affect cases where memset() is not referenced in the source code. -ffreestanding should prevent gcc generating calls to library functions like memset(). However, -ffreestanding is already in CFLAGS too, and there is a problem: certain initializations like the one in your example need to use an interface like memset(), and struct copies need to use and interface like memcpy(), so what is gcc to do when -fno-builtin tells it to turn off its builtins and -ffreestanding tells it that the relevant interfaces might not exist in the library?> Anyone knows what's happening?gcc is expecting that memset() is in the library, but the FreeBSD kernel is freestanding and happens not to have memset() in its library. Related bugs: - the FreeBSD kernel shouldn't have memset() at all. The kernel interface for clearing memory is bzero(). A few files misspelled bzero() as memset() and provided a macro to convert from memset() to bzero(), and instead of fixing them a low-quality memset() was added to <sys/libkern.h>. This gives an inline memset() so it doesn't help here. memset() is of some use for setting to nonzero, but this is rarely needed and can easily be repeated as necessary. The support for the nonzero case in <sys/libkern.h> is of particularly low quality -- e.g., it crashes if asked to set a length of 0. - memset() to zero and/or gcc methods for initialization to 0 might be much slower than the library's methods for clearing memory. This is not a problem in practice, although bzero() is much faster than gcc's methods in some cases, because: (a) -fno-builtin turns off builtin memset(). (b) the inline memset() just uses bzero() in the fill_byte = 0 case, so using it instead of bzero() is only a tiny pessimization. (c) large copies that bzero() can handle better than gcc's inline method (which is stosl on i386's for your example) cannot because the data would be too large to fit on the kernel statck. - there are slightly different problems for memcpy(): (a) memcpy() is in the library and is not inline, so there is no linkage problem if gcc generates a call to memcpy() for a struct copy. (b) the library memcpy() never uses bcopy(), so it is much slower than bcopy() in much cases. (c) the reason that memcpy() is in the library is to let gcc inline memcpy() for efficiency, but this reason was turned into nonsense by adding -fno-builtin to CFLAGS, and all calls to memcpy() are style bugs and ask for inefficiency. (The inefficiency is small or negative in practice because bzero() has optimizations for large copies that are small pessimizations for non-large copies.) - the FreeBSD kernel shouldn't have memcmp(). It has an inline one that has even lower quality than the inline memset(). memcmp() cannot be implemented using bcmp() since memcmp() is tri-state but bcmp() is boolean, but the inline memcmp() just calls bcmp(). This works, if at all, because nothing actually needs memcmp() and memcmp() is just a misspelling of bcmp(). Bruce
At 2005-03-24 08:31:14+0000, Bruce Evans writes:> what is gcc to do when -fno-builtin tells it to turn off its > builtins and -ffreestanding tells it that the relevant interfaces > might not exist in the library?Plainly, GCC should generate code which fills the array with zeroes. It's not obliged to generate code which calls memset (either builtin or in a library). If it knows that it can do so, then fine. Otherwise it must do it the Old Fashioned Way. So this is surely a bug in GCC. Nick B, who used to write compilers for a living
> This version makes the pessimizations and potential bugs clear: > - clearing 100 bytes on every entry to the function is > wasteful. C90's > auto initializers hide pessimizations like this. They should be > used very rarely, especially in kernels. But they are > often misused, > even in kernels, even for read-only data that should be > static. gcc > doesn't optimize even "auto const x[100] = { 0 };" to a static > initialization -- the programmer must declare the object > as static to > prevent gcc laboriously clearing it on every entry to the function. > - 100 bytes may be too much to put on the kernel stack. Objects just > a little larger than this must be dynamically allocated unless they > can be read-only. >A statement like this (auto and not static) is necessary if you are dealing with re-entrancy. Whatever the issues with wastage or bad performance, a programmer should definitely be able to do it, if he so desires. Also, the line I mentioned earlier was only an example. Something like this also fails (your response already explains this): ----- struct x_type x = {0}; -----> > Adding CFLAGS+=-fbuiltin, or CFLAGS+=-fno-builtin to > /sys/conf/Makefile.amd64 > > does not help. > > -fno-builtin is already in CFLAGS, and if it has any effect > on this then > it should be to cause gcc to generate a call to memset() > instead of doing > the memory clearing inline. I think gcc has a builtin > memset() which is > turned off by -fno-builtin, but -fno-builtin doesn't affect > cases where > memset() is not referenced in the source code. > > -ffreestanding should prevent gcc generating calls to library > functions > like memset(). However, -ffreestanding is already in CFLAGS too, and > there is a problem: certain initializations like the one in > your example > need to use an interface like memset(), and struct copies > need to use and > interface like memcpy(), so what is gcc to do when > -fno-builtin tells it > to turn off its builtins and -ffreestanding tells it that the relevant > interfaces might not exist in the library? > > > Anyone knows what's happening? > > gcc is expecting that memset() is in the library, but the > FreeBSD kernel > is freestanding and happens not to have memset() in its library. >How is it then, that an explicit call to memset (like in my example) works? As about other questions in the thread: 1. Yes, the example code is within a function, 2. I should have mentioned that I don't see the problem if I am building only the kernel module. It happens only when I am building the kernel integrating the module containing the example code.
> -----Original Message----- > From: Peter Jeremy [mailto:PeterJeremy@optushome.com.au] > Sent: Thursday, March 24, 2005 2:46 PM > To: Vinod Kashyap > Cc: freebsd-stable@freebsd.org; freebsd-amd64@freebsd.org > Subject: Re: undefined reference to `memset' > > > On Thu, 2005-Mar-24 12:03:19 -0800, Vinod Kashyap wrote: > [ char x[100] = { 0 }; ] > >A statement like this (auto and not static) > > I'd point out that this is the first time that you've mentioned that > the variable is auto. Leaving out critical information will not > encourage people to help you. >It is "char x[100] = {0};" and that's it. Doesn't it make it auto? Isn't auto the default?> > is necessary if you > >are dealing with re-entrancy. > > This isn't completely true. The preferred approach is: > char *x; > x = malloc(100, MEM_POOL_xxx, M_ZERO | M_WAITOK); > (with a matching free() later). >Well, I am in a function that is OS-independent, and cannot assume malloc (or a wrapper to it) is available.> > Whatever the issues with wastage or > >bad performance, a programmer should definitely be able to do it, > >if he so desires. > > Again, untrue. The FreeBSD kernel is not a standard C environment. > Kernel stack is a relatively small, fixed size and using excessive > kernel stack will lead to panics. Increasing the kernel stack size is > undesirable because it's expensive in RAM consumption. >Whatever that may be, I don't think the compiler should be stopping me from doing standard C stuff. I could be having this statement in my module with full knowledge that it would not cause a kernel stack overflow.> >How is it then, that an explicit call to memset (like in my > example) works? > > The code > auto char x[100] = {0}; > is equivalent to > auto char x[100]; > memset(x, 0, sizeof(x)); > but memset only exists as a static inline function (defined > in libkern.h). > If an explicit call to memset works then the problem would > appear to be > that the compiler's implicit expansion is failing to detect the static > inline definition, and generating an external reference which can't be > satisfied. This would seem to be a gcc bug. > > >2. I should have mentioned that I don't see the problem if I am > > building only the kernel module. It happens only when I > am building > > the kernel integrating the module containing the example code. > > This is the opposite of what you implied previously. There are some > differences in how kernel modules are built so this >No, it's not. This is what I wrote in my first e-mail: "building of the GENERIC kernel on FreeBSD 5 -STABLE for amd64". I just forgot to mention that the problem did not occur when I build only the module. This is possibly due to different gcc flags being used in the 2 cases.> How about posting a (short) compilable piece of C code that shows the > problem. I would expect that an "nm" of the resultant object would > show "U memset" when the code was compiled for linking into the kernel > and "<some_address> t memset" or not reference memset at all when > compiled as a module. >Just like the problem is not seen when I build only the module, it's not seen if I simply write a foo.c (with the example code) and compile it. That's the reason I posted the patch to /sys/dev/twa/twa.c, which would cause the problem if applied, and then followed with a kernel build. I can send the result of running nm on twa.o tomorrow.
> -----Original Message----- > From: Sean McNeil [mailto:sean@mcneil.com] > Sent: Thursday, March 24, 2005 7:15 PM > To: Vinod Kashyap > Cc: Peter Jeremy; freebsd-stable@freebsd.org; > freebsd-amd64@freebsd.org > Subject: RE: undefined reference to `memset' > > > Vinod, > > On Thu, 2005-03-24 at 19:01 -0800, Vinod Kashyap wrote: > > Just like the problem is not seen when I build only the module, it's > > not seen if I simply write a foo.c (with the example code) > and compile it. > > That's the reason I posted the patch to /sys/dev/twa/twa.c, > which would > > cause the problem if applied, and then followed with a kernel build. > > I can send the result of running nm on twa.o tomorrow. > > Please take a look at other messages in this thread, like some of the > ones I have posted. They clearly show your problem in a small example > and how it is happening in the -O2 case as memset is being optimized > away. -O would appear to do the right thing and adding > -minline-all-stringops (at either optimization level) would > produce even > better code. >I did look at your posting Sean, thanks. But did you see the "undefined reference to `memset'" linker error when you built it? It's obvious that a reference to memset is being generated by the initialization of an array of 100 bytes to 0. The linker is getting the needed memset if you build a stand alone program, or even build a stand alone kernel module, but is not being able to find it when building the kernel itself. This implies to me that it is a difference in the use of flags, or linking/not linking with particular libraries that's causing the problem. I am also confused as to how an explicit call to memset works, when compiler generated call doesn't! Are we talking 2 different memset's here? Maybe a memset and an __memset?
> > Can you not submit it with the patch I sent? > > Submit the entire /usr/src/sys kernel source?? > > > A "stand alone" foo.c will have to be a new module under a new > > directory (/sys/dev/foo), with its own Makefile, with changes to > > /sys/conf/files, and an entry in GENERIC. > > A module would still depend on files in /usr/share/mk, > /usr/src/sys/conf, > /usr/src/sys/modules, /usr/src/sys/dev/foo. > > NO! nothing other than a single file -- foo.c in my home > directory that > shows the problem is a "stand alone" example.Here is memset_issue.c (also attached). But note that you cannot reproduce the "undefined reference to `memset'" issue with this (that's what I have been trying to explain). You could also use the earlier posting from Sean McNeil. #include <stdio.h> main() { char a[100] = {0}; char *ap; ap = &(a[0]); printf("a[0] = %d\n", a[0]); } With the patch that I sent earlier applied to /sys/dev/twa/twa.c, 'nm twa.o' shows this when twa is built as a stand alone module: 00000000000003f0 t memset 'nm twa.o' shows this when twa is built as part of the kernel: U memset And now, moving to the important thing... in kern.pre.mk, I changed COPTFLAGS from -O2 to -O for amd64 (just like i386), and the problem is gone!! -------------- next part -------------- #include <stdio.h> main() { char a[100] = {0}; char *ap; ap = &(a[0]); printf("a[0] = %d\n", a[0]); }