freebsd stable - May 2017 - GCC + FreeBSD 11.0 Stable - stat.h does not have vm_ooffset_t definition

On 29 Apr 2017, at 19:00, Gerald Pfeifer <gerald at pfeifer.com>
wrote:
> 
> On Thu, 27 Apr 2017, Jung-uk Kim wrote:
>>>>>>> I found the problem,  but I do not know how to
resolve this.  When you
>>>>>>> install the GCC compiler from the PKG repository it
appears to create a
>>>>>>> modified set of include files from the system
(default?) include files
>>>>>>> (/usr/include).  However, when the modified
/usr/include/sys/types.h
>>>>>>> file is created, the typedef for vm_ooffset_t is
modified,  and there is
>>>>>>> no reference to __vm_ooffset_t that the compiler
can resolve.
>>>>>>> 
>>>>>>> < typedef       __int64_t       vm_ooffset_t;
>>>>>>> ---
>>>>>>>> typedef       __vm_ooffset_t  vm_ooffset_t;
>>>>>> ...
>>>>>> You have to rebuild lang/gcc from the ports tree to fix
this problem.
>>>>>> 
>>>>>>
https://lists.freebsd.org/pipermail/freebsd-current/2017-February/064937.html
>>>>> Does this mean that the GCC port/package needs to be
updated?  If so,
>>>>> should I file a PR report on this issue?
>>>>> I (temporarily) fixed this problem by hand editting the
modified types.h
>>>>> file and things seem to work.
>>>> I already wrote a patch (attached). :-)
>> If the maintainer (gerald) approves.  CC'd.
> 
> Thanks for bringing this to my attention.
> 
> Can you please help me understand why this is necessary?
This is because gcc's fixincludes process makes copies of certain system
headers (in this case, /usr/include/sys/types.h) with slight
modifications.  Then, it places the directory containing the modified
headers at the front of the include search path.  So far so good.

Now, whenever sys/types.h is updated, as happened with the vm_ooffset_t
change, the header in gcc's own preferred directory might not match the
definitions which are expected, leading to compilation errors.

> If the
> port/package is builts from scratch, does this trigger the problem?
Yes, basically you need to rebuild all gcc ports from scratch, whenever
you update any system header that matches gcc's list of files it wants
to modify.

But getting those errors in the first place can be very confusing to an
end-user.  And having to rebuild all those ports might be a burden.

As some people pointed out, simply moving away or deleting the directory
with fixed includes appears to work around the problems.  So maybe the
question is if gcc really needs to modify those headers at all?

I have looked at gcc's build system a bit, but it does not seem very
easy to disable the fixincludes step.  I guess that is simply not
supported.

So in that case, if Jung-uk's solution works, it is probably the best
way forward, and it can even be upstreamed.  Jung-uk, how does your
patch handle an updated header under /usr/include which contains e.g.
new definitions, which are not in the fixed includes directory?

-Dimitry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: Message signed with OpenPGP
URL:
<http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20170429/25727835/attachment.sig>

On Sat, Apr 29, 2017 at 07:55:24PM +0200, Dimitry Andric
wrote:
> On 29 Apr 2017, at 19:00, Gerald Pfeifer <gerald at pfeifer.com>
wrote:
> > 
> > On Thu, 27 Apr 2017, Jung-uk Kim wrote:
> >>>>>>> I found the problem,  but I do not know how to
resolve this.  When you
> >>>>>>> install the GCC compiler from the PKG
repository it appears to create a
> >>>>>>> modified set of include files from the system
(default?) include files
> >>>>>>> (/usr/include).  However, when the modified
/usr/include/sys/types.h
> >>>>>>> file is created, the typedef for vm_ooffset_t
is modified,  and there is
> >>>>>>> no reference to __vm_ooffset_t that the
compiler can resolve.
> >>>>>>> 
> >>>>>>> < typedef       __int64_t      
vm_ooffset_t;
> >>>>>>> ---
> >>>>>>>> typedef       __vm_ooffset_t 
vm_ooffset_t;
> >>>>>> ...
> >>>>>> You have to rebuild lang/gcc from the ports tree
to fix this problem.
> >>>>>> 
> >>>>>>
https://lists.freebsd.org/pipermail/freebsd-current/2017-February/064937.html
> >>>>> Does this mean that the GCC port/package needs to be
updated?  If so,
> >>>>> should I file a PR report on this issue?
> >>>>> I (temporarily) fixed this problem by hand editting
the modified types.h
> >>>>> file and things seem to work.
> >>>> I already wrote a patch (attached). :-)
> >> If the maintainer (gerald) approves.  CC'd.
> > 
> > Thanks for bringing this to my attention.
> > 
> > Can you please help me understand why this is necessary?
> 
> This is because gcc's fixincludes process makes copies of certain
system
> headers (in this case, /usr/include/sys/types.h) with slight
> modifications.  Then, it places the directory containing the modified
> headers at the front of the include search path.  So far so good.
> 
> Now, whenever sys/types.h is updated, as happened with the vm_ooffset_t
> change, the header in gcc's own preferred directory might not match the
> definitions which are expected, leading to compilation errors.
> 
> 
> > If the
> > port/package is builts from scratch, does this trigger the problem?
> 
> Yes, basically you need to rebuild all gcc ports from scratch, whenever
> you update any system header that matches gcc's list of files it wants
> to modify.
> 
> But getting those errors in the first place can be very confusing to an
> end-user.  And having to rebuild all those ports might be a burden.
> 
> As some people pointed out, simply moving away or deleting the directory
> with fixed includes appears to work around the problems.  So maybe the
> question is if gcc really needs to modify those headers at all?
> 
> I have looked at gcc's build system a bit, but it does not seem very
> easy to disable the fixincludes step.  I guess that is simply not
> supported.
> 
> So in that case, if Jung-uk's solution works, it is probably the best
> way forward, and it can even be upstreamed.  Jung-uk, how does your
> patch handle an updated header under /usr/include which contains e.g.
> new definitions, which are not in the fixed includes directory?
Am I right that Jung-uk fix replaces vm_ooffset_t and vm_pindex_t with
explicit int64_t and uint64_t use, as the course of action for gcc
fixincludes step ?  If yes, I completely disagree.

The change blocks any future changes to the type that might occur in the
base system, for the code compiled by gcc.  End result might be as bad
as mismatched ABI, in the worst case.

I share the opinion that fixincludes is not only useless, but really
damaging.  Gcc ships workarounds for e.g. issues in X11 headers, which
application depends on the presence of the corresponding headers at the
gcc build time.  For clean (poudriere-like) builds these fixes are never
applied, so port build results are inconsistent, at least.

Nobody so far explained why fixincludes is needed for the modern base
headers. IMO if we have real problems in headers we ship, we must fix it
in the base.

With all of the above, IMO most sane way to fix problems is to
rename fixincludes directory to some name which is ignored by gcc,
e.g. include-fixed -> include-fixed.saved. This can be done as
post-installation step in the ports.

On 04/30/2017 08:06, Konstantin Belousov wrote:
> On Sat, Apr 29, 2017 at 07:55:24PM +0200, Dimitry Andric wrote:
>> On 29 Apr 2017, at 19:00, Gerald Pfeifer <gerald at pfeifer.com>
wrote:
>>>
>>> On Thu, 27 Apr 2017, Jung-uk Kim wrote:
>>>>>>>>> I found the problem,  but I do not know how
to resolve this.  When you
>>>>>>>>> install the GCC compiler from the PKG
repository it appears to create a
>>>>>>>>> modified set of include files from the
system (default?) include files
>>>>>>>>> (/usr/include).  However, when the modified
/usr/include/sys/types.h
>>>>>>>>> file is created, the typedef for
vm_ooffset_t is modified,  and there is
>>>>>>>>> no reference to __vm_ooffset_t that the
compiler can resolve.
>>>>>>>>>
>>>>>>>>> < typedef       __int64_t      
vm_ooffset_t;
>>>>>>>>> ---
>>>>>>>>>> typedef       __vm_ooffset_t 
vm_ooffset_t;
>>>>>>>> ...
>>>>>>>> You have to rebuild lang/gcc from the ports
tree to fix this problem.
>>>>>>>>
>>>>>>>>
https://lists.freebsd.org/pipermail/freebsd-current/2017-February/064937.html
>>>>>>> Does this mean that the GCC port/package needs to
be updated?  If so,
>>>>>>> should I file a PR report on this issue?
>>>>>>> I (temporarily) fixed this problem by hand editting
the modified types.h
>>>>>>> file and things seem to work.
>>>>>> I already wrote a patch (attached). :-)
>>>> If the maintainer (gerald) approves.  CC'd.
>>>
>>> Thanks for bringing this to my attention.
>>>
>>> Can you please help me understand why this is necessary?
>>
>> This is because gcc's fixincludes process makes copies of certain
system
>> headers (in this case, /usr/include/sys/types.h) with slight
>> modifications.  Then, it places the directory containing the modified
>> headers at the front of the include search path.  So far so good.
>>
>> Now, whenever sys/types.h is updated, as happened with the vm_ooffset_t
>> change, the header in gcc's own preferred directory might not match
the
>> definitions which are expected, leading to compilation errors.
>>
>>
>>> If the
>>> port/package is builts from scratch, does this trigger the problem?
>>
>> Yes, basically you need to rebuild all gcc ports from scratch, whenever
>> you update any system header that matches gcc's list of files it
wants
>> to modify.
>>
>> But getting those errors in the first place can be very confusing to an
>> end-user.  And having to rebuild all those ports might be a burden.
>>
>> As some people pointed out, simply moving away or deleting the
directory
>> with fixed includes appears to work around the problems.  So maybe the
>> question is if gcc really needs to modify those headers at all?
>>
>> I have looked at gcc's build system a bit, but it does not seem
very
>> easy to disable the fixincludes step.  I guess that is simply not
>> supported.
>>
>> So in that case, if Jung-uk's solution works, it is probably the
best
>> way forward, and it can even be upstreamed.  Jung-uk, how does your
>> patch handle an updated header under /usr/include which contains e.g.
>> new definitions, which are not in the fixed includes directory?
> 
> Am I right that Jung-uk fix replaces vm_ooffset_t and vm_pindex_t with
> explicit int64_t and uint64_t use, as the course of action for gcc
> fixincludes step ?  If yes, I completely disagree.
> 
> The change blocks any future changes to the type that might occur in the
> base system, for the code compiled by gcc.  End result might be as bad
> as mismatched ABI, in the worst case.
Good point.
> I share the opinion that fixincludes is not only useless, but really
> damaging.  Gcc ships workarounds for e.g. issues in X11 headers, which
> application depends on the presence of the corresponding headers at the
> gcc build time.  For clean (poudriere-like) builds these fixes are never
> applied, so port build results are inconsistent, at least.
> 
> Nobody so far explained why fixincludes is needed for the modern base
> headers. IMO if we have real problems in headers we ship, we must fix it
> in the base.
> 
> With all of the above, IMO most sane way to fix problems is to
> rename fixincludes directory to some name which is ignored by gcc,
> e.g. include-fixed -> include-fixed.saved. This can be done as
> post-installation step in the ports.
Agreed.

Jung-uk Kim

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20170501/36c67def/attachment.sig>