Xen devel - Apr 2013 - pv-grub will cause page fault if build with flag -fstack-protector

Hi, List,

In checking the page fault problem when using pv-grub to start domU in
opensuse12.3[1], found that the problem is caused by
''-fstack-protector''
flag which is introduced in rpm build:
export EXTRA_CFLAGS_XEN_TOOLS="$RPM_OPT_FLAGS"
export EXTRA_CFLAGS_QEMU_TRADITIONAL="$RPM_OPT_FLAGS"
export EXTRA_CFLAGS_QEMU_XEN="$RPM_OPT_FLAGS"
with ''-fstack-protector'', pv-grub will cause page fault when
start domU;
without the flag, pv-grub has no problem.

We asked for Jan''s opinion whether we can remove the flag or there
isn''t a
problem with stubdom overwriting something on the stack . Jan viewed it as
a mistake use of flag:
> I generally view it as a mistake enforce certain kinds of (intrusive)
flags onto builds of code that may assume to have full control of
everything (e.g. kernel like, which stubdom clearly is).

I wonder if there is other opinion? If it is a flag use problem, then we
can simply remove the flag from rpm build.

I checked the objdump of problem pv-grub, the page fault place is in
xc_interface_open_common, the detail line is a line inserted by
-fstack-protector (see 0x404bb in following):
00000000000404b0 <xc_interface_open_common>:
   404b0:       55                      push   %rbp
   404b1:       48 89 e5                mov    %rsp,%rbp
   404b4:       48 81 ec 00 05 00 00    sub    $0x500,%rsp
   404bb:       64 48 8b 04 25 28 00    mov    %fs:0x28,%rax
   404c2:       00 00
   404c4:       48 89 45 c8             mov    %rax,-0x38(%rbp)
   404c8:       31 c0                   xor    %eax,%eax
   404ca:       48 85 ff                test   %rdi,%rdi
   404cd:       4c 89 6d e8             mov    %r13,-0x18(%rbp)
   404d1:       4c 89 75 f0             mov    %r14,-0x10(%rbp)
   404d5:       48 89 5d d8             mov    %rbx,-0x28(%rbp)
   404d9:       4c 89 65 e0             mov    %r12,-0x20(%rbp)

[1]http://lists.xen.org/archives/html/xen-users/2013-03/msg00164.html

Thanks,
Chunyan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Tue, 2013-04-16 at 10:34 +0100, Chunyan Liu wrote:
> We asked for Jan''s opinion whether we can remove the flag or there
> isn''t a problem with stubdom overwriting something on the stack .
Jan
> viewed it as a mistake use of flag:
> > I generally view it as a mistake enforce certain kinds of
> (intrusive) flags onto builds of code that may assume to have full
> control of everything (e.g. kernel like, which stubdom clearly is).
> 
> I wonder if there is other opinion?
I agree with Jan.
>  If it is a flag use problem, then we can simply remove the flag from
> rpm build.
Please do so, you may want to only do it for the stubdom bits of the
build though.

If you can figure out a way to override this in stubdom/* regardless of
the flags set by the user (or RPM) at the toplevel (e.g. by inserting
-fno-stack-protector in the right place) then that might be a patch we
could consider for upstream.
> I checked the objdump of problem pv-grub, the page fault place is in
> xc_interface_open_common, the detail line is a line inserted by
> -fstack-protector (see 0x404bb in following):
If you wanted to get really advanced you could implement stack protector
support for mini-os. I''ve no idea how hard that would be though.

Ian.

Ian Campbell, le Tue 16 Apr 2013 10:47:18 +0100, a écrit
:
> On Tue, 2013-04-16 at 10:34 +0100, Chunyan Liu wrote:
> > We asked for Jan''s opinion whether we can remove the flag or
there
> > isn''t a problem with stubdom overwriting something on the
stack . Jan
> > viewed it as a mistake use of flag:
> > > I generally view it as a mistake enforce certain kinds of
> > (intrusive) flags onto builds of code that may assume to have full
> > control of everything (e.g. kernel like, which stubdom clearly is).
> > 
> > I wonder if there is other opinion?
> 
> I agree with Jan.
mini-os, which stubdoms are based on, explicitly tries to disable
stack-protector, see extras/mini-os/minios.mk
> > I checked the objdump of problem pv-grub, the page fault place is in
> > xc_interface_open_common, the detail line is a line inserted by
> > -fstack-protector (see 0x404bb in following):
> 
> If you wanted to get really advanced you could implement stack protector
> support for mini-os. I''ve no idea how hard that would be though.
Actually we can''t really support it, because the code introduced by
gcc, which typically uses fs: or gs: adressing, is host-dependent, i.e.
depending on being built on linux, or on freebsd, etc. mini-os would
have to behave differently because the code introduced by gcc behaves
differently... The solution would be to use a separate host triplet, but
we ended up avoiding doing it due to the introduced complexity.

Samuel

On Tue, 2013-04-16 at 12:10 +0100, Samuel Thibault
wrote:
> Ian Campbell, le Tue 16 Apr 2013 10:47:18 +0100, a écrit :
> > On Tue, 2013-04-16 at 10:34 +0100, Chunyan Liu wrote:
> > > We asked for Jan's opinion whether we can remove the flag or
there
> > > isn't a problem with stubdom overwriting something on the
stack . Jan
> > > viewed it as a mistake use of flag:
> > > > I generally view it as a mistake enforce certain kinds of
> > > (intrusive) flags onto builds of code that may assume to have
full
> > > control of everything (e.g. kernel like, which stubdom clearly
is).
> > > 
> > > I wonder if there is other opinion?
> > 
> > I agree with Jan.
> 
> mini-os, which stubdoms are based on, explicitly tries to disable
> stack-protector, see extras/mini-os/minios.mk
OK, so I guess that is just broken in the face of whatever the suse rpms
are doing.
> > > I checked the objdump of problem pv-grub, the page fault place is
in
> > > xc_interface_open_common, the detail line is a line inserted by
> > > -fstack-protector (see 0x404bb in following):
> > 
> > If you wanted to get really advanced you could implement stack
protector
> > support for mini-os. I've no idea how hard that would be though.
> 
> Actually we can't really support it, because the code introduced by
> gcc, which typically uses fs: or gs: adressing, is host-dependent, i.e.
> depending on being built on linux, or on freebsd, etc. mini-os would
> have to behave differently because the code introduced by gcc behaves
> differently... 
I didn't know about this. Does this not affect the hypervisor itself
too? I don't see any stack protector disabling going on there. Maybe I'm
looking in the wrong places.
> The solution would be to use a separate host triplet, but
> we ended up avoiding doing it due to the introduced complexity.
Right. I've occasionally flirted with implementing the separate triplet
but it's a lot of pieces to get right (binutils etc etc).

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Ian Campbell, le Tue 16 Apr 2013 12:18:59 +0100, a écrit
:
> > Actually we can''t really support it, because the code
introduced by
> > gcc, [...] is host-dependent,
> 
> I didn''t know about this. Does this not affect the hypervisor
itself
> too?
Yes.
> I don''t see any stack protector disabling going on there. Maybe
I''m
> looking in the wrong places.
Probably:

./Config.mk:EMBEDDED_EXTRA_CFLAGS := -nopie -fno-stack-protector
-fno-stack-protector-all
./xen/arch/x86/Rules.mk:$(call
cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
> > The solution would be to use a separate host triplet, but
> > we ended up avoiding doing it due to the introduced complexity.
> 
> Right. I''ve occasionally flirted with implementing the separate
triplet
> but it''s a lot of pieces to get right (binutils etc etc).
Yes.  That was mostly why we dropped the support I had initially
written.

Samuel