Antony Saba
2013-Jun-04 21:34 UTC
xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
Hello, Can anyone verify if reinjecting int3 mem_events work for them under Xen 4.2.2? I''m trying to move some systems which are making use of int3 mem_events from Xen 4.1.x to Xen 4.2.2, but seem to having an issue with xc_hvm_inject_trap(). I''ve got a simple program that doesn''t do anything but "_asm int 3" in it''s main. Using the example in "tools/text/xen-access/", this is what the output of "xen-access 14 int3" looks like. The domain is frozen after xen-access exits. xenaccess init max_pages = 40100 starting int3 14 Got event from Xen Got event from Xen INT3: rip=0000000000401000, gfn=1418f (vcpu 0) xc: error: Error -1 injecting int3: Internal error xenaccess shutting down on signal -1 xenaccess shut down on signal -1 xenaccess exit code -1 This shows up in xl dmesg (nothing else shows up with debug=y): (XEN) d14v0: bogus time -341046118 (offsets -3367129229295/0) The same behavior occurs with both 32-bit and 64-bit HVM guests. I''m using Xen 4.2.2 built from the dist tarball. dom0 is Ubuntu 12.04.2 using kernel 3.2.0-45-generic #70-Ubuntu SMP Wed May 29 20:12:06 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux My CPU is an Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz I''ve also tried on 2nd machine with the same Ubuntu/kernel versions, but with the following CPU: Intel(R) Core(TM) i7-2720QM CPU @ 2.20GHz On 4.1.2, the result of xc_hvm_inject_trap() is always non-zero and errno is set to ENOENT, but the int3 is reinjected anyway and works as expected. -Tony -- Antony Saba, antony.saba@mandiant.com
(Adding xen-devel) On Tue, Jun 4, 2013 at 2:34 PM, Antony Saba <Antony.Saba@mandiant.com> wrote:> > Hello, > > Can anyone verify if reinjecting int3 mem_events work for them under Xen > 4.2.2? > > I''m trying to move some systems which are making use of int3 mem_events > from Xen 4.1.x to Xen 4.2.2, but seem to having an issue with > xc_hvm_inject_trap().Can you try with the following patch? --- a/tools/tests/xen-access/xen-access.c +++ b/tools/tests/xen-access/xen-access.c @@ -664,7 +664,7 @@ int main(int argc, char *argv[]) /* Reinject */ rc = xc_hvm_inject_trap( xch, domain_id, req.vcpu_id, 3, - HVMOP_TRAP_sw_exc, -1, 0, 0); + HVMOP_TRAP_sw_exc, -1, 1, 0); if (rc < 0) { ERROR("Error %d injecting int3\n", rc); BTW, I don''t think you need to specify the instruction length for int3 as the IP should have been moved forward. But it might give us a clue as to what is going on.> I''ve got a simple program that doesn''t do anything but "_asm int 3" in > it''s main. Using the example in "tools/text/xen-access/", this is what > the output of "xen-access 14 int3" looks like. The domain is frozen > after xen-access exits. > xenaccess init > max_pages = 40100 > starting int3 14 > Got event from Xen > Got event from Xen > INT3: rip=0000000000401000, gfn=1418f (vcpu 0) > xc: error: Error -1 injecting int3: Internal error > xenaccess shutting down on signal -1 > xenaccess shut down on signal -1 > xenaccess exit code -1If you set access required, then this is the expected behavior otherwise the domain should continue running.> This shows up in xl dmesg (nothing else shows up with debug=y): > (XEN) d14v0: bogus time -341046118 (offsets -3367129229295/0) > > The same behavior occurs with both 32-bit and 64-bit HVM guests. > > I''m using Xen 4.2.2 built from the dist tarball. > > dom0 is Ubuntu 12.04.2 using kernel 3.2.0-45-generic #70-Ubuntu SMP Wed > May 29 20:12:06 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux > > My CPU is an Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz > > I''ve also tried on 2nd machine with the same Ubuntu/kernel versions, but > with the following CPU: Intel(R) Core(TM) i7-2720QM CPU @ 2.20GHz > > On 4.1.2, the result of xc_hvm_inject_trap() is always non-zero and > errno is set to ENOENT, but the int3 is reinjected anyway and works as > expected. > > -Tony > > -- > Antony Saba, antony.saba@mandiant.com > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users
AP
2013-Jun-07 00:16 UTC
Re: [Xen-users] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
(Adding xen-devel) On Tue, Jun 4, 2013 at 2:34 PM, Antony Saba <Antony.Saba@mandiant.com> wrote:> > Hello, > > Can anyone verify if reinjecting int3 mem_events work for them under Xen > 4.2.2? > > I''m trying to move some systems which are making use of int3 mem_events > from Xen 4.1.x to Xen 4.2.2, but seem to having an issue with > xc_hvm_inject_trap().Can you try with the following patch? --- a/tools/tests/xen-access/xen-access.c +++ b/tools/tests/xen-access/xen-access.c @@ -664,7 +664,7 @@ int main(int argc, char *argv[]) /* Reinject */ rc = xc_hvm_inject_trap( xch, domain_id, req.vcpu_id, 3, - HVMOP_TRAP_sw_exc, -1, 0, 0); + HVMOP_TRAP_sw_exc, -1, 1, 0); if (rc < 0) { ERROR("Error %d injecting int3\n", rc); BTW, I don''t think you need to specify the instruction length for int3 as the IP should have been moved forward. But it might give us a clue as to what is going on.> I''ve got a simple program that doesn''t do anything but "_asm int 3" in > it''s main. Using the example in "tools/text/xen-access/", this is what > the output of "xen-access 14 int3" looks like. The domain is frozen > after xen-access exits. > xenaccess init > max_pages = 40100 > starting int3 14 > Got event from Xen > Got event from Xen > INT3: rip=0000000000401000, gfn=1418f (vcpu 0) > xc: error: Error -1 injecting int3: Internal error > xenaccess shutting down on signal -1 > xenaccess shut down on signal -1 > xenaccess exit code -1If you set access required, then this is the expected behavior otherwise the domain should continue running.> This shows up in xl dmesg (nothing else shows up with debug=y): > (XEN) d14v0: bogus time -341046118 (offsets -3367129229295/0) > > The same behavior occurs with both 32-bit and 64-bit HVM guests. > > I''m using Xen 4.2.2 built from the dist tarball. > > dom0 is Ubuntu 12.04.2 using kernel 3.2.0-45-generic #70-Ubuntu SMP Wed > May 29 20:12:06 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux > > My CPU is an Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz > > I''ve also tried on 2nd machine with the same Ubuntu/kernel versions, but > with the following CPU: Intel(R) Core(TM) i7-2720QM CPU @ 2.20GHz > > On 4.1.2, the result of xc_hvm_inject_trap() is always non-zero and > errno is set to ENOENT, but the int3 is reinjected anyway and works as > expected. > > -Tony > > -- > Antony Saba, antony.saba@mandiant.com > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users
Steven Maresca
2013-Jun-07 15:51 UTC
Re: [Xen-devel] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
On Thu, Jun 6, 2013 at 8:16 PM, AP <apxeng@gmail.com> wrote:> (Adding xen-devel) > > On Tue, Jun 4, 2013 at 2:34 PM, Antony Saba <Antony.Saba@mandiant.com> wrote: >> >> Hello, >> >> Can anyone verify if reinjecting int3 mem_events work for them under Xen >> 4.2.2? >> >> I''m trying to move some systems which are making use of int3 mem_events >> from Xen 4.1.x to Xen 4.2.2, but seem to having an issue with >> xc_hvm_inject_trap(). > > Can you try with the following patch? > > --- a/tools/tests/xen-access/xen-access.c > +++ b/tools/tests/xen-access/xen-access.c > @@ -664,7 +664,7 @@ int main(int argc, char *argv[]) > /* Reinject */ > rc = xc_hvm_inject_trap( > xch, domain_id, req.vcpu_id, 3, > - HVMOP_TRAP_sw_exc, -1, 0, 0); > + HVMOP_TRAP_sw_exc, -1, 1, 0); > if (rc < 0) > { > ERROR("Error %d injecting int3\n", rc); > > BTW, I don''t think you need to specify the instruction length for int3 > as the IP should have been moved forward. But it might give us a clue > as to what is going on. > >> I''ve got a simple program that doesn''t do anything but "_asm int 3" in >> it''s main. Using the example in "tools/text/xen-access/", this is what >> the output of "xen-access 14 int3" looks like. The domain is frozen >> after xen-access exits. >> xenaccess init >> max_pages = 40100 >> starting int3 14 >> Got event from Xen >> Got event from Xen >> INT3: rip=0000000000401000, gfn=1418f (vcpu 0) >> xc: error: Error -1 injecting int3: Internal error >> xenaccess shutting down on signal -1 >> xenaccess shut down on signal -1 >> xenaccess exit code -1 > > If you set access required, then this is the expected behavior > otherwise the domain should continue running. > >> This shows up in xl dmesg (nothing else shows up with debug=y): >> (XEN) d14v0: bogus time -341046118 (offsets -3367129229295/0) >> >> The same behavior occurs with both 32-bit and 64-bit HVM guests. >> >> I''m using Xen 4.2.2 built from the dist tarball. >> >> dom0 is Ubuntu 12.04.2 using kernel 3.2.0-45-generic #70-Ubuntu SMP Wed >> May 29 20:12:06 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux >> >> My CPU is an Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz >> >> I''ve also tried on 2nd machine with the same Ubuntu/kernel versions, but >> with the following CPU: Intel(R) Core(TM) i7-2720QM CPU @ 2.20GHz >> >> On 4.1.2, the result of xc_hvm_inject_trap() is always non-zero and >> errno is set to ENOENT, but the int3 is reinjected anyway and works as >> expected. >> >> -Tony >> >> -- >> Antony Saba, antony.saba@mandiant.comTony, I can confirm INT3 re-injection does work on 4.2.x and 4.3, but the problem you observed is certainly present. As suggested, it was necessary when invoking xc_hvm_inject_trap to specify the 1-byte instruction length for 0xCC (without which the VM was intentionally crashed by Xen). In this case, there''s no need to inspect the actual instruction referenced by the IP because it seems the trap is only fired for the one-byte variant (0xCD03 of course works properly, but no event is emitted). Mirroring your experience with 4.1.2, for my testing on 4.2+ the return of xc_hvm_inject_trap is also always non-zero even for successful re-injection..whether that''s intended is another question. Steve NOTE: I would definitely consider it a bug that the xen-access.c example crashes guests when attempting to use the INT3 mode...non-critical for most users, but nevertheless.
Steven Maresca
2013-Jun-07 15:51 UTC
Re: [Xen-users] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
On Thu, Jun 6, 2013 at 8:16 PM, AP <apxeng@gmail.com> wrote:> (Adding xen-devel) > > On Tue, Jun 4, 2013 at 2:34 PM, Antony Saba <Antony.Saba@mandiant.com> wrote: >> >> Hello, >> >> Can anyone verify if reinjecting int3 mem_events work for them under Xen >> 4.2.2? >> >> I''m trying to move some systems which are making use of int3 mem_events >> from Xen 4.1.x to Xen 4.2.2, but seem to having an issue with >> xc_hvm_inject_trap(). > > Can you try with the following patch? > > --- a/tools/tests/xen-access/xen-access.c > +++ b/tools/tests/xen-access/xen-access.c > @@ -664,7 +664,7 @@ int main(int argc, char *argv[]) > /* Reinject */ > rc = xc_hvm_inject_trap( > xch, domain_id, req.vcpu_id, 3, > - HVMOP_TRAP_sw_exc, -1, 0, 0); > + HVMOP_TRAP_sw_exc, -1, 1, 0); > if (rc < 0) > { > ERROR("Error %d injecting int3\n", rc); > > BTW, I don''t think you need to specify the instruction length for int3 > as the IP should have been moved forward. But it might give us a clue > as to what is going on. > >> I''ve got a simple program that doesn''t do anything but "_asm int 3" in >> it''s main. Using the example in "tools/text/xen-access/", this is what >> the output of "xen-access 14 int3" looks like. The domain is frozen >> after xen-access exits. >> xenaccess init >> max_pages = 40100 >> starting int3 14 >> Got event from Xen >> Got event from Xen >> INT3: rip=0000000000401000, gfn=1418f (vcpu 0) >> xc: error: Error -1 injecting int3: Internal error >> xenaccess shutting down on signal -1 >> xenaccess shut down on signal -1 >> xenaccess exit code -1 > > If you set access required, then this is the expected behavior > otherwise the domain should continue running. > >> This shows up in xl dmesg (nothing else shows up with debug=y): >> (XEN) d14v0: bogus time -341046118 (offsets -3367129229295/0) >> >> The same behavior occurs with both 32-bit and 64-bit HVM guests. >> >> I''m using Xen 4.2.2 built from the dist tarball. >> >> dom0 is Ubuntu 12.04.2 using kernel 3.2.0-45-generic #70-Ubuntu SMP Wed >> May 29 20:12:06 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux >> >> My CPU is an Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz >> >> I''ve also tried on 2nd machine with the same Ubuntu/kernel versions, but >> with the following CPU: Intel(R) Core(TM) i7-2720QM CPU @ 2.20GHz >> >> On 4.1.2, the result of xc_hvm_inject_trap() is always non-zero and >> errno is set to ENOENT, but the int3 is reinjected anyway and works as >> expected. >> >> -Tony >> >> -- >> Antony Saba, antony.saba@mandiant.comTony, I can confirm INT3 re-injection does work on 4.2.x and 4.3, but the problem you observed is certainly present. As suggested, it was necessary when invoking xc_hvm_inject_trap to specify the 1-byte instruction length for 0xCC (without which the VM was intentionally crashed by Xen). In this case, there''s no need to inspect the actual instruction referenced by the IP because it seems the trap is only fired for the one-byte variant (0xCD03 of course works properly, but no event is emitted). Mirroring your experience with 4.1.2, for my testing on 4.2+ the return of xc_hvm_inject_trap is also always non-zero even for successful re-injection..whether that''s intended is another question. Steve NOTE: I would definitely consider it a bug that the xen-access.c example crashes guests when attempting to use the INT3 mode...non-critical for most users, but nevertheless.
Antony Saba
2013-Jun-07 17:43 UTC
Re: [Xen-users] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
On 06/07/2013 09:51 AM, Steven Maresca wrote:> On Thu, Jun 6, 2013 at 8:16 PM, AP <apxeng@gmail.com> wrote: >> (Adding xen-devel) >> >> On Tue, Jun 4, 2013 at 2:34 PM, Antony Saba <Antony.Saba@mandiant.com> wrote: >>> >>> Hello, >>> >>> Can anyone verify if reinjecting int3 mem_events work for them under Xen >>> 4.2.2? >>> >>> I''m trying to move some systems which are making use of int3 mem_events >>> from Xen 4.1.x to Xen 4.2.2, but seem to having an issue with >>> xc_hvm_inject_trap(). >> >> Can you try with the following patch? >> >> --- a/tools/tests/xen-access/xen-access.c >> +++ b/tools/tests/xen-access/xen-access.c >> @@ -664,7 +664,7 @@ int main(int argc, char *argv[]) >> /* Reinject */ >> rc = xc_hvm_inject_trap( >> xch, domain_id, req.vcpu_id, 3, >> - HVMOP_TRAP_sw_exc, -1, 0, 0); >> + HVMOP_TRAP_sw_exc, -1, 1, 0); >> if (rc < 0) >> { >> ERROR("Error %d injecting int3\n", rc); >> >> BTW, I don''t think you need to specify the instruction length for int3 >> as the IP should have been moved forward. But it might give us a clue >> as to what is going on.The same behavior occurs, rc < 0 and errno set.>> >>> I''ve got a simple program that doesn''t do anything but "_asm int 3" in >>> it''s main. Using the example in "tools/text/xen-access/", this is what >>> the output of "xen-access 14 int3" looks like. The domain is frozen >>> after xen-access exits. >>> xenaccess init >>> max_pages = 40100 >>> starting int3 14 >>> Got event from Xen >>> Got event from Xen >>> INT3: rip=0000000000401000, gfn=1418f (vcpu 0) >>> xc: error: Error -1 injecting int3: Internal error >>> xenaccess shutting down on signal -1 >>> xenaccess shut down on signal -1 >>> xenaccess exit code -1 >> >> If you set access required, then this is the expected behavior >> otherwise the domain should continue running.The domain is frozen whether or not I''ve passed -m to the xen-access binary.>> >>> This shows up in xl dmesg (nothing else shows up with debug=y): >>> (XEN) d14v0: bogus time -341046118 (offsets -3367129229295/0) >>> >>> The same behavior occurs with both 32-bit and 64-bit HVM guests. >>> >>> I''m using Xen 4.2.2 built from the dist tarball. >>> >>> dom0 is Ubuntu 12.04.2 using kernel 3.2.0-45-generic #70-Ubuntu SMP Wed >>> May 29 20:12:06 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux >>> >>> My CPU is an Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz >>> >>> I''ve also tried on 2nd machine with the same Ubuntu/kernel versions, but >>> with the following CPU: Intel(R) Core(TM) i7-2720QM CPU @ 2.20GHz >>> >>> On 4.1.2, the result of xc_hvm_inject_trap() is always non-zero and >>> errno is set to ENOENT, but the int3 is reinjected anyway and works as> > Tony, > > I can confirm INT3 re-injection does work on 4.2.x and 4.3, but the > problem you observed is certainly present. > > As suggested, it was necessary when invoking xc_hvm_inject_trap to > specify the 1-byte instruction length for 0xCC (without which the VM > was intentionally crashed by Xen).Based on this, I went a ahead and made the change in our own application, which now appears to reinject correctly, despite returning the error "No such file or directory". There is a whole lot more than int3 events being handled in that application, so it''s definitely doing something that the xen-access example is not.> > In this case, there''s no need to inspect the actual instruction > referenced by the IP because it seems the trap is only fired for the > one-byte variant (0xCD03 of course works properly, but no event is > emitted). > > Mirroring your experience with 4.1.2, for my testing on 4.2+ the > return of xc_hvm_inject_trap is also always non-zero even for > successful re-injection..whether that''s intended is another question. >Ignoring the error in xen-accesss.c, the domain still remains frozen after the call to resume_page(), etc.> Steve > > NOTE: I would definitely consider it a bug that the xen-access.c > example crashes guests when attempting to use the INT3 > mode...non-critical for most users, but nevertheless. >If this does end up getting filed as a bug, I saw the same issues under 4.3-RC3. Thanks. -Tony -- Antony Saba, antony.saba@mandiant.com
Antony Saba
2013-Jun-07 17:43 UTC
Re: [Xen-devel] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
On 06/07/2013 09:51 AM, Steven Maresca wrote:> On Thu, Jun 6, 2013 at 8:16 PM, AP <apxeng@gmail.com> wrote: >> (Adding xen-devel) >> >> On Tue, Jun 4, 2013 at 2:34 PM, Antony Saba <Antony.Saba@mandiant.com> wrote: >>> >>> Hello, >>> >>> Can anyone verify if reinjecting int3 mem_events work for them under Xen >>> 4.2.2? >>> >>> I''m trying to move some systems which are making use of int3 mem_events >>> from Xen 4.1.x to Xen 4.2.2, but seem to having an issue with >>> xc_hvm_inject_trap(). >> >> Can you try with the following patch? >> >> --- a/tools/tests/xen-access/xen-access.c >> +++ b/tools/tests/xen-access/xen-access.c >> @@ -664,7 +664,7 @@ int main(int argc, char *argv[]) >> /* Reinject */ >> rc = xc_hvm_inject_trap( >> xch, domain_id, req.vcpu_id, 3, >> - HVMOP_TRAP_sw_exc, -1, 0, 0); >> + HVMOP_TRAP_sw_exc, -1, 1, 0); >> if (rc < 0) >> { >> ERROR("Error %d injecting int3\n", rc); >> >> BTW, I don''t think you need to specify the instruction length for int3 >> as the IP should have been moved forward. But it might give us a clue >> as to what is going on.The same behavior occurs, rc < 0 and errno set.>> >>> I''ve got a simple program that doesn''t do anything but "_asm int 3" in >>> it''s main. Using the example in "tools/text/xen-access/", this is what >>> the output of "xen-access 14 int3" looks like. The domain is frozen >>> after xen-access exits. >>> xenaccess init >>> max_pages = 40100 >>> starting int3 14 >>> Got event from Xen >>> Got event from Xen >>> INT3: rip=0000000000401000, gfn=1418f (vcpu 0) >>> xc: error: Error -1 injecting int3: Internal error >>> xenaccess shutting down on signal -1 >>> xenaccess shut down on signal -1 >>> xenaccess exit code -1 >> >> If you set access required, then this is the expected behavior >> otherwise the domain should continue running.The domain is frozen whether or not I''ve passed -m to the xen-access binary.>> >>> This shows up in xl dmesg (nothing else shows up with debug=y): >>> (XEN) d14v0: bogus time -341046118 (offsets -3367129229295/0) >>> >>> The same behavior occurs with both 32-bit and 64-bit HVM guests. >>> >>> I''m using Xen 4.2.2 built from the dist tarball. >>> >>> dom0 is Ubuntu 12.04.2 using kernel 3.2.0-45-generic #70-Ubuntu SMP Wed >>> May 29 20:12:06 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux >>> >>> My CPU is an Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz >>> >>> I''ve also tried on 2nd machine with the same Ubuntu/kernel versions, but >>> with the following CPU: Intel(R) Core(TM) i7-2720QM CPU @ 2.20GHz >>> >>> On 4.1.2, the result of xc_hvm_inject_trap() is always non-zero and >>> errno is set to ENOENT, but the int3 is reinjected anyway and works as> > Tony, > > I can confirm INT3 re-injection does work on 4.2.x and 4.3, but the > problem you observed is certainly present. > > As suggested, it was necessary when invoking xc_hvm_inject_trap to > specify the 1-byte instruction length for 0xCC (without which the VM > was intentionally crashed by Xen).Based on this, I went a ahead and made the change in our own application, which now appears to reinject correctly, despite returning the error "No such file or directory". There is a whole lot more than int3 events being handled in that application, so it''s definitely doing something that the xen-access example is not.> > In this case, there''s no need to inspect the actual instruction > referenced by the IP because it seems the trap is only fired for the > one-byte variant (0xCD03 of course works properly, but no event is > emitted). > > Mirroring your experience with 4.1.2, for my testing on 4.2+ the > return of xc_hvm_inject_trap is also always non-zero even for > successful re-injection..whether that''s intended is another question. >Ignoring the error in xen-accesss.c, the domain still remains frozen after the call to resume_page(), etc.> Steve > > NOTE: I would definitely consider it a bug that the xen-access.c > example crashes guests when attempting to use the INT3 > mode...non-critical for most users, but nevertheless. >If this does end up getting filed as a bug, I saw the same issues under 4.3-RC3. Thanks. -Tony -- Antony Saba, antony.saba@mandiant.com
George Dunlap
2013-Jun-10 11:29 UTC
Re: [Xen-devel] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
On Fri, Jun 7, 2013 at 4:51 PM, Steven Maresca <steve@zentific.com> wrote:> Tony, > > I can confirm INT3 re-injection does work on 4.2.x and 4.3, but the > problem you observed is certainly present. > > As suggested, it was necessary when invoking xc_hvm_inject_trap to > specify the 1-byte instruction length for 0xCC (without which the VM > was intentionally crashed by Xen). > > In this case, there''s no need to inspect the actual instruction > referenced by the IP because it seems the trap is only fired for the > one-byte variant (0xCD03 of course works properly, but no event is > emitted). > > Mirroring your experience with 4.1.2, for my testing on 4.2+ the > return of xc_hvm_inject_trap is also always non-zero even for > successful re-injection..whether that''s intended is another question. > > Steve > > NOTE: I would definitely consider it a bug that the xen-access.c > example crashes guests when attempting to use the INT3 > mode...non-critical for most users, but nevertheless.I''m having a bit of trouble finding the conclusion here. So it seems the problem is that if a *guest* is doing int3 instructions, that will interfere with the ability of the debugger to use int3 to do introspection -- is that right? If anyone on this thread were able to make and test a proper fix, I''m sure we would all appreciate it. :-) At this point it would definitely not be a release blocker, but we would obviously like to have it fixed. -George
George Dunlap
2013-Jun-10 11:29 UTC
Re: [Xen-users] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
On Fri, Jun 7, 2013 at 4:51 PM, Steven Maresca <steve@zentific.com> wrote:> Tony, > > I can confirm INT3 re-injection does work on 4.2.x and 4.3, but the > problem you observed is certainly present. > > As suggested, it was necessary when invoking xc_hvm_inject_trap to > specify the 1-byte instruction length for 0xCC (without which the VM > was intentionally crashed by Xen). > > In this case, there''s no need to inspect the actual instruction > referenced by the IP because it seems the trap is only fired for the > one-byte variant (0xCD03 of course works properly, but no event is > emitted). > > Mirroring your experience with 4.1.2, for my testing on 4.2+ the > return of xc_hvm_inject_trap is also always non-zero even for > successful re-injection..whether that''s intended is another question. > > Steve > > NOTE: I would definitely consider it a bug that the xen-access.c > example crashes guests when attempting to use the INT3 > mode...non-critical for most users, but nevertheless.I''m having a bit of trouble finding the conclusion here. So it seems the problem is that if a *guest* is doing int3 instructions, that will interfere with the ability of the debugger to use int3 to do introspection -- is that right? If anyone on this thread were able to make and test a proper fix, I''m sure we would all appreciate it. :-) At this point it would definitely not be a release blocker, but we would obviously like to have it fixed. -George
Antony Saba
2013-Jun-10 16:57 UTC
Re: [Xen-devel] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
On 06/10/2013 05:29 AM, George Dunlap wrote:> On Fri, Jun 7, 2013 at 4:51 PM, Steven Maresca <steve@zentific.com> wrote: >> Tony, >> >> I can confirm INT3 re-injection does work on 4.2.x and 4.3, but the >> problem you observed is certainly present. >> >> As suggested, it was necessary when invoking xc_hvm_inject_trap to >> specify the 1-byte instruction length for 0xCC (without which the VM >> was intentionally crashed by Xen). >> >> In this case, there''s no need to inspect the actual instruction >> referenced by the IP because it seems the trap is only fired for the >> one-byte variant (0xCD03 of course works properly, but no event is >> emitted). >> >> Mirroring your experience with 4.1.2, for my testing on 4.2+ the >> return of xc_hvm_inject_trap is also always non-zero even for >> successful re-injection..whether that''s intended is another question. >> >> Steve >> >> NOTE: I would definitely consider it a bug that the xen-access.c >> example crashes guests when attempting to use the INT3 >> mode...non-critical for most users, but nevertheless. > > I''m having a bit of trouble finding the conclusion here. > > So it seems the problem is that if a *guest* is doing int3 > instructions, that will interfere with the ability of the debugger to > use int3 to do introspection -- is that right? >Yes, that is one scenario. The one I was experiencing was some (apparently legitimate) background process on a Windows 7 x64 guest that just always executes an int3 when it runs. I''ll try to summarize, someone please correct me if I''m wrong. There are 2 things going on here: 1) The patch previously posted by AP is the correct way to call xc_hvm_inject_trap() for int 3 (0xcc). That is, the instruction_length parameter must be set to 1. 2) xc_hvm_inject_trap() always returns a negative value, even when there is not a problem and the guest receives the trap as expected. There hasn''t been a clarification as to whether it''s supposed to return non-negative, but one would assume that it should because of the way the xen-access.c example checks for it. There was an error in my modifications to xen-access.c to ignore the error from xc_hvm_inject_trap(), which was causing resume_page() to not get called, resulting in a frozen guest on my machines. Thanks again for the previous responses; I did get it working without freezing the guest.> If anyone on this thread were able to make and test a proper fix, I''m > sure we would all appreciate it. :-) > > At this point it would definitely not be a release blocker, but we > would obviously like to have it fixed.-Tony -- Antony Saba, antony.saba@mandiant.com
Antony Saba
2013-Jun-10 16:57 UTC
Re: [Xen-users] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
On 06/10/2013 05:29 AM, George Dunlap wrote:> On Fri, Jun 7, 2013 at 4:51 PM, Steven Maresca <steve@zentific.com> wrote: >> Tony, >> >> I can confirm INT3 re-injection does work on 4.2.x and 4.3, but the >> problem you observed is certainly present. >> >> As suggested, it was necessary when invoking xc_hvm_inject_trap to >> specify the 1-byte instruction length for 0xCC (without which the VM >> was intentionally crashed by Xen). >> >> In this case, there''s no need to inspect the actual instruction >> referenced by the IP because it seems the trap is only fired for the >> one-byte variant (0xCD03 of course works properly, but no event is >> emitted). >> >> Mirroring your experience with 4.1.2, for my testing on 4.2+ the >> return of xc_hvm_inject_trap is also always non-zero even for >> successful re-injection..whether that''s intended is another question. >> >> Steve >> >> NOTE: I would definitely consider it a bug that the xen-access.c >> example crashes guests when attempting to use the INT3 >> mode...non-critical for most users, but nevertheless. > > I''m having a bit of trouble finding the conclusion here. > > So it seems the problem is that if a *guest* is doing int3 > instructions, that will interfere with the ability of the debugger to > use int3 to do introspection -- is that right? >Yes, that is one scenario. The one I was experiencing was some (apparently legitimate) background process on a Windows 7 x64 guest that just always executes an int3 when it runs. I''ll try to summarize, someone please correct me if I''m wrong. There are 2 things going on here: 1) The patch previously posted by AP is the correct way to call xc_hvm_inject_trap() for int 3 (0xcc). That is, the instruction_length parameter must be set to 1. 2) xc_hvm_inject_trap() always returns a negative value, even when there is not a problem and the guest receives the trap as expected. There hasn''t been a clarification as to whether it''s supposed to return non-negative, but one would assume that it should because of the way the xen-access.c example checks for it. There was an error in my modifications to xen-access.c to ignore the error from xc_hvm_inject_trap(), which was causing resume_page() to not get called, resulting in a frozen guest on my machines. Thanks again for the previous responses; I did get it working without freezing the guest.> If anyone on this thread were able to make and test a proper fix, I''m > sure we would all appreciate it. :-) > > At this point it would definitely not be a release blocker, but we > would obviously like to have it fixed.-Tony -- Antony Saba, antony.saba@mandiant.com
Tim Deegan
2013-Jun-10 18:36 UTC
Re: [Xen-users] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
At 16:57 +0000 on 10 Jun (1370883430), Antony Saba wrote:> On 06/10/2013 05:29 AM, George Dunlap wrote: > > On Fri, Jun 7, 2013 at 4:51 PM, Steven Maresca <steve@zentific.com> wrote: > >> Tony, > >> > >> I can confirm INT3 re-injection does work on 4.2.x and 4.3, but the > >> problem you observed is certainly present. > >> > >> As suggested, it was necessary when invoking xc_hvm_inject_trap to > >> specify the 1-byte instruction length for 0xCC (without which the VM > >> was intentionally crashed by Xen). > >> > >> In this case, there''s no need to inspect the actual instruction > >> referenced by the IP because it seems the trap is only fired for the > >> one-byte variant (0xCD03 of course works properly, but no event is > >> emitted). > >> > >> Mirroring your experience with 4.1.2, for my testing on 4.2+ the > >> return of xc_hvm_inject_trap is also always non-zero even for > >> successful re-injection..whether that''s intended is another question. > >> > >> Steve > >> > >> NOTE: I would definitely consider it a bug that the xen-access.c > >> example crashes guests when attempting to use the INT3 > >> mode...non-critical for most users, but nevertheless. > > > > I''m having a bit of trouble finding the conclusion here. > > > > So it seems the problem is that if a *guest* is doing int3 > > instructions, that will interfere with the ability of the debugger to > > use int3 to do introspection -- is that right? > > > Yes, that is one scenario. The one I was experiencing was some > (apparently legitimate) background process on a Windows 7 x64 guest that > just always executes an int3 when it runs. > > I''ll try to summarize, someone please correct me if I''m wrong. There > are 2 things going on here: > > 1) The patch previously posted by AP is the correct way to call > xc_hvm_inject_trap() for int 3 (0xcc). That is, the instruction_length > parameter must be set to 1.Not necessarily, AFAICT -- you''d need to fetch and decode the instruction in order to detect prefix bytes (other than LOCK, which is explicitly disallowed).> 2) xc_hvm_inject_trap() always returns a negative value, even when there > is not a problem and the guest receives the trap as expected. There > hasn''t been a clarification as to whether it''s supposed to return > non-negative, but one would assume that it should because of the way the > xen-access.c example checks for it.That looks like a hypervisor bug to me: does this (untested) patch fix it for you? commit 67b9272fcedcb5dc73cc77a2adf580f2572117d7 Author: Tim Deegan <tim@xen.org> Date: Mon Jun 10 19:35:34 2013 +0100 x86/hvm: Fix HVMOP_inject_trap return value on success. Reported-by: Antony Saba <Antony.Saba@mandiant.com> Signed-off-by: Tim Deegan <tim@xen.org> diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index ce44bff..6c86fc2 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -4430,6 +4430,7 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg) v->arch.hvm_vcpu.inject_trap.error_code = tr.error_code; v->arch.hvm_vcpu.inject_trap.insn_len = tr.insn_len; v->arch.hvm_vcpu.inject_trap.cr2 = tr.cr2; + rc = 0; } param_fail8:
Tim Deegan
2013-Jun-10 18:36 UTC
Re: [Xen-devel] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
At 16:57 +0000 on 10 Jun (1370883430), Antony Saba wrote:> On 06/10/2013 05:29 AM, George Dunlap wrote: > > On Fri, Jun 7, 2013 at 4:51 PM, Steven Maresca <steve@zentific.com> wrote: > >> Tony, > >> > >> I can confirm INT3 re-injection does work on 4.2.x and 4.3, but the > >> problem you observed is certainly present. > >> > >> As suggested, it was necessary when invoking xc_hvm_inject_trap to > >> specify the 1-byte instruction length for 0xCC (without which the VM > >> was intentionally crashed by Xen). > >> > >> In this case, there''s no need to inspect the actual instruction > >> referenced by the IP because it seems the trap is only fired for the > >> one-byte variant (0xCD03 of course works properly, but no event is > >> emitted). > >> > >> Mirroring your experience with 4.1.2, for my testing on 4.2+ the > >> return of xc_hvm_inject_trap is also always non-zero even for > >> successful re-injection..whether that''s intended is another question. > >> > >> Steve > >> > >> NOTE: I would definitely consider it a bug that the xen-access.c > >> example crashes guests when attempting to use the INT3 > >> mode...non-critical for most users, but nevertheless. > > > > I''m having a bit of trouble finding the conclusion here. > > > > So it seems the problem is that if a *guest* is doing int3 > > instructions, that will interfere with the ability of the debugger to > > use int3 to do introspection -- is that right? > > > Yes, that is one scenario. The one I was experiencing was some > (apparently legitimate) background process on a Windows 7 x64 guest that > just always executes an int3 when it runs. > > I''ll try to summarize, someone please correct me if I''m wrong. There > are 2 things going on here: > > 1) The patch previously posted by AP is the correct way to call > xc_hvm_inject_trap() for int 3 (0xcc). That is, the instruction_length > parameter must be set to 1.Not necessarily, AFAICT -- you''d need to fetch and decode the instruction in order to detect prefix bytes (other than LOCK, which is explicitly disallowed).> 2) xc_hvm_inject_trap() always returns a negative value, even when there > is not a problem and the guest receives the trap as expected. There > hasn''t been a clarification as to whether it''s supposed to return > non-negative, but one would assume that it should because of the way the > xen-access.c example checks for it.That looks like a hypervisor bug to me: does this (untested) patch fix it for you? commit 67b9272fcedcb5dc73cc77a2adf580f2572117d7 Author: Tim Deegan <tim@xen.org> Date: Mon Jun 10 19:35:34 2013 +0100 x86/hvm: Fix HVMOP_inject_trap return value on success. Reported-by: Antony Saba <Antony.Saba@mandiant.com> Signed-off-by: Tim Deegan <tim@xen.org> diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index ce44bff..6c86fc2 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -4430,6 +4430,7 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg) v->arch.hvm_vcpu.inject_trap.error_code = tr.error_code; v->arch.hvm_vcpu.inject_trap.insn_len = tr.insn_len; v->arch.hvm_vcpu.inject_trap.cr2 = tr.cr2; + rc = 0; } param_fail8:
Antony Saba
2013-Jun-15 14:51 UTC
Re: [Xen-devel] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
On 06/10/2013 12:36 PM, Tim Deegan wrote:> At 16:57 +0000 on 10 Jun (1370883430), Antony Saba wrote: >> On 06/10/2013 05:29 AM, George Dunlap wrote: >>> On Fri, Jun 7, 2013 at 4:51 PM, Steven Maresca <steve@zentific.com> wrote: >>>> Tony, >>>> >>>> I can confirm INT3 re-injection does work on 4.2.x and 4.3, but the >>>> problem you observed is certainly present. >>>> >>>> As suggested, it was necessary when invoking xc_hvm_inject_trap to >>>> specify the 1-byte instruction length for 0xCC (without which the VM >>>> was intentionally crashed by Xen). >>>> >>>> In this case, there''s no need to inspect the actual instruction >>>> referenced by the IP because it seems the trap is only fired for the >>>> one-byte variant (0xCD03 of course works properly, but no event is >>>> emitted). >>>> >>>> Mirroring your experience with 4.1.2, for my testing on 4.2+ the >>>> return of xc_hvm_inject_trap is also always non-zero even for >>>> successful re-injection..whether that''s intended is another question. >>>> >>>> Steve >>>> >>>> NOTE: I would definitely consider it a bug that the xen-access.c >>>> example crashes guests when attempting to use the INT3 >>>> mode...non-critical for most users, but nevertheless. >>> >>> I''m having a bit of trouble finding the conclusion here. >>> >>> So it seems the problem is that if a *guest* is doing int3 >>> instructions, that will interfere with the ability of the debugger to >>> use int3 to do introspection -- is that right? >>> >> Yes, that is one scenario. The one I was experiencing was some >> (apparently legitimate) background process on a Windows 7 x64 guest that >> just always executes an int3 when it runs. >> >> I''ll try to summarize, someone please correct me if I''m wrong. There >> are 2 things going on here: >> >> 1) The patch previously posted by AP is the correct way to call >> xc_hvm_inject_trap() for int 3 (0xcc). That is, the instruction_length >> parameter must be set to 1. > > Not necessarily, AFAICT -- you''d need to fetch and decode the > instruction in order to detect prefix bytes (other than LOCK, which is > explicitly disallowed).I just verified this again under 4.2.2, here is the crash dump from xl dmesg: (XEN) <vm_resume_fail> error code 7 (XEN) domain_crash_sync called from vmcs.c:1068 (XEN) Domain 2 (vcpu#0) crashed on cpu#6: (XEN) ----[ Xen-4.2.2 x86_64 debug=n Not tainted ]---- (XEN) CPU: 6 (XEN) RIP: 001b:[<0000000000401000>] (XEN) RFLAGS: 0000000000000246 CONTEXT: hvm guest (XEN) rax: ffff82c4801053e2 rbx: ffff83017838e000 rcx: 000000007ffd4000 (XEN) rdx: ffff82c4801cc600 rsi: 0000000000000000 rdi: ffff82c4801d5b2d (XEN) rbp: ffff82c480180a99 rsp: 000000000012ff7c r8: 000000000012ffc0 (XEN) r9: ffff8302334f7f18 r10: 0000000000000002 r11: ffff82c4801053bb (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: ffff8302334f7f18 cr0: 000000008001003b cr4: 00000000000006f9 (XEN) cr3: 000000000b600180 cr2: 0000000000153005 (XEN) ds: 0023 es: 0023 fs: 003b gs: 0000 ss: 0023 cs: 001b This is the change to xen_access to ignore the error and attempt to resume that causes it: diff --git a/tools/tests/xen-access/xen-access.c b/tools/tests/xen-access/xen-access.c index 9ec7332..77d7b12 100644 --- a/tools/tests/xen-access/xen-access.c +++ b/tools/tests/xen-access/xen-access.c @@ -668,8 +668,8 @@ int main(int argc, char *argv[]) if (rc < 0) { ERROR("Error %d injecting int3\n", rc); - interrupted = -1; - continue; + //interrupted = -1; + //continue; } break;> >> 2) xc_hvm_inject_trap() always returns a negative value, even when there >> is not a problem and the guest receives the trap as expected. There >> hasn''t been a clarification as to whether it''s supposed to return >> non-negative, but one would assume that it should because of the way the >> xen-access.c example checks for it. > > That looks like a hypervisor bug to me: does this (untested) patch fix > it for you? > > commit 67b9272fcedcb5dc73cc77a2adf580f2572117d7 > Author: Tim Deegan <tim@xen.org> > Date: Mon Jun 10 19:35:34 2013 +0100 > > x86/hvm: Fix HVMOP_inject_trap return value on success. > > Reported-by: Antony Saba <Antony.Saba@mandiant.com> > Signed-off-by: Tim Deegan <tim@xen.org> > > diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c > index ce44bff..6c86fc2 100644 > --- a/xen/arch/x86/hvm/hvm.c > +++ b/xen/arch/x86/hvm/hvm.c > @@ -4430,6 +4430,7 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg) > v->arch.hvm_vcpu.inject_trap.error_code = tr.error_code; > v->arch.hvm_vcpu.inject_trap.insn_len = tr.insn_len; > v->arch.hvm_vcpu.inject_trap.cr2 = tr.cr2; > + rc = 0; > } > > param_fail8: > > >This works, but the instruction size must be set to 1, at least on 4.2.2 to work for me. Here is the patch against RELEASE-4.2.2. diff --git a/tools/tests/xen-access/xen-access.c b/tools/tests/xen-access/xen-access.c index 9ec7332..8bcd88b 100644 --- a/tools/tests/xen-access/xen-access.c +++ b/tools/tests/xen-access/xen-access.c @@ -664,7 +664,7 @@ int main(int argc, char *argv[]) /* Reinject */ rc = xc_hvm_inject_trap( xch, domain_id, req.vcpu_id, 3, - HVMOP_TRAP_sw_exc, -1, 0, 0); + HVMOP_TRAP_sw_exc, -1, 1, 0); if (rc < 0) { ERROR("Error %d injecting int3\n", rc); diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 3d471a5..4c2320e 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -4372,6 +4372,7 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE(void) arg) v->arch.hvm_vcpu.inject_trap.error_code = tr.error_code; v->arch.hvm_vcpu.inject_trap.insn_len = tr.insn_len; v->arch.hvm_vcpu.inject_trap.cr2 = tr.cr2; + rc = 0; } param_fail8: -- Antony Saba, antony.saba@mandiant.com
Antony Saba
2013-Jun-15 14:51 UTC
Re: [Xen-users] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
On 06/10/2013 12:36 PM, Tim Deegan wrote:> At 16:57 +0000 on 10 Jun (1370883430), Antony Saba wrote: >> On 06/10/2013 05:29 AM, George Dunlap wrote: >>> On Fri, Jun 7, 2013 at 4:51 PM, Steven Maresca <steve@zentific.com> wrote: >>>> Tony, >>>> >>>> I can confirm INT3 re-injection does work on 4.2.x and 4.3, but the >>>> problem you observed is certainly present. >>>> >>>> As suggested, it was necessary when invoking xc_hvm_inject_trap to >>>> specify the 1-byte instruction length for 0xCC (without which the VM >>>> was intentionally crashed by Xen). >>>> >>>> In this case, there''s no need to inspect the actual instruction >>>> referenced by the IP because it seems the trap is only fired for the >>>> one-byte variant (0xCD03 of course works properly, but no event is >>>> emitted). >>>> >>>> Mirroring your experience with 4.1.2, for my testing on 4.2+ the >>>> return of xc_hvm_inject_trap is also always non-zero even for >>>> successful re-injection..whether that''s intended is another question. >>>> >>>> Steve >>>> >>>> NOTE: I would definitely consider it a bug that the xen-access.c >>>> example crashes guests when attempting to use the INT3 >>>> mode...non-critical for most users, but nevertheless. >>> >>> I''m having a bit of trouble finding the conclusion here. >>> >>> So it seems the problem is that if a *guest* is doing int3 >>> instructions, that will interfere with the ability of the debugger to >>> use int3 to do introspection -- is that right? >>> >> Yes, that is one scenario. The one I was experiencing was some >> (apparently legitimate) background process on a Windows 7 x64 guest that >> just always executes an int3 when it runs. >> >> I''ll try to summarize, someone please correct me if I''m wrong. There >> are 2 things going on here: >> >> 1) The patch previously posted by AP is the correct way to call >> xc_hvm_inject_trap() for int 3 (0xcc). That is, the instruction_length >> parameter must be set to 1. > > Not necessarily, AFAICT -- you''d need to fetch and decode the > instruction in order to detect prefix bytes (other than LOCK, which is > explicitly disallowed).I just verified this again under 4.2.2, here is the crash dump from xl dmesg: (XEN) <vm_resume_fail> error code 7 (XEN) domain_crash_sync called from vmcs.c:1068 (XEN) Domain 2 (vcpu#0) crashed on cpu#6: (XEN) ----[ Xen-4.2.2 x86_64 debug=n Not tainted ]---- (XEN) CPU: 6 (XEN) RIP: 001b:[<0000000000401000>] (XEN) RFLAGS: 0000000000000246 CONTEXT: hvm guest (XEN) rax: ffff82c4801053e2 rbx: ffff83017838e000 rcx: 000000007ffd4000 (XEN) rdx: ffff82c4801cc600 rsi: 0000000000000000 rdi: ffff82c4801d5b2d (XEN) rbp: ffff82c480180a99 rsp: 000000000012ff7c r8: 000000000012ffc0 (XEN) r9: ffff8302334f7f18 r10: 0000000000000002 r11: ffff82c4801053bb (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: ffff8302334f7f18 cr0: 000000008001003b cr4: 00000000000006f9 (XEN) cr3: 000000000b600180 cr2: 0000000000153005 (XEN) ds: 0023 es: 0023 fs: 003b gs: 0000 ss: 0023 cs: 001b This is the change to xen_access to ignore the error and attempt to resume that causes it: diff --git a/tools/tests/xen-access/xen-access.c b/tools/tests/xen-access/xen-access.c index 9ec7332..77d7b12 100644 --- a/tools/tests/xen-access/xen-access.c +++ b/tools/tests/xen-access/xen-access.c @@ -668,8 +668,8 @@ int main(int argc, char *argv[]) if (rc < 0) { ERROR("Error %d injecting int3\n", rc); - interrupted = -1; - continue; + //interrupted = -1; + //continue; } break;> >> 2) xc_hvm_inject_trap() always returns a negative value, even when there >> is not a problem and the guest receives the trap as expected. There >> hasn''t been a clarification as to whether it''s supposed to return >> non-negative, but one would assume that it should because of the way the >> xen-access.c example checks for it. > > That looks like a hypervisor bug to me: does this (untested) patch fix > it for you? > > commit 67b9272fcedcb5dc73cc77a2adf580f2572117d7 > Author: Tim Deegan <tim@xen.org> > Date: Mon Jun 10 19:35:34 2013 +0100 > > x86/hvm: Fix HVMOP_inject_trap return value on success. > > Reported-by: Antony Saba <Antony.Saba@mandiant.com> > Signed-off-by: Tim Deegan <tim@xen.org> > > diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c > index ce44bff..6c86fc2 100644 > --- a/xen/arch/x86/hvm/hvm.c > +++ b/xen/arch/x86/hvm/hvm.c > @@ -4430,6 +4430,7 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg) > v->arch.hvm_vcpu.inject_trap.error_code = tr.error_code; > v->arch.hvm_vcpu.inject_trap.insn_len = tr.insn_len; > v->arch.hvm_vcpu.inject_trap.cr2 = tr.cr2; > + rc = 0; > } > > param_fail8: > > >This works, but the instruction size must be set to 1, at least on 4.2.2 to work for me. Here is the patch against RELEASE-4.2.2. diff --git a/tools/tests/xen-access/xen-access.c b/tools/tests/xen-access/xen-access.c index 9ec7332..8bcd88b 100644 --- a/tools/tests/xen-access/xen-access.c +++ b/tools/tests/xen-access/xen-access.c @@ -664,7 +664,7 @@ int main(int argc, char *argv[]) /* Reinject */ rc = xc_hvm_inject_trap( xch, domain_id, req.vcpu_id, 3, - HVMOP_TRAP_sw_exc, -1, 0, 0); + HVMOP_TRAP_sw_exc, -1, 1, 0); if (rc < 0) { ERROR("Error %d injecting int3\n", rc); diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 3d471a5..4c2320e 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -4372,6 +4372,7 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE(void) arg) v->arch.hvm_vcpu.inject_trap.error_code = tr.error_code; v->arch.hvm_vcpu.inject_trap.insn_len = tr.insn_len; v->arch.hvm_vcpu.inject_trap.cr2 = tr.cr2; + rc = 0; } param_fail8: -- Antony Saba, antony.saba@mandiant.com
Tim Deegan
2013-Jun-20 10:33 UTC
Re: [Xen-devel] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
(Cc-ing a few more people, moving xen-users to Bcc) At 14:51 +0000 on 15 Jun (1371307878), Antony Saba wrote:> >> 2) xc_hvm_inject_trap() always returns a negative value, even when there > >> is not a problem and the guest receives the trap as expected. There > >> hasn''t been a clarification as to whether it''s supposed to return > >> non-negative, but one would assume that it should because of the way the > >> xen-access.c example checks for it. > > > > That looks like a hypervisor bug to me: does this (untested) patch fix > > it for you? > > > > commit 67b9272fcedcb5dc73cc77a2adf580f2572117d7 > > Author: Tim Deegan <tim@xen.org> > > Date: Mon Jun 10 19:35:34 2013 +0100 > > > > x86/hvm: Fix HVMOP_inject_trap return value on success. > > > > Reported-by: Antony Saba <Antony.Saba@mandiant.com> > > Signed-off-by: Tim Deegan <tim@xen.org> > > > > diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c > > index ce44bff..6c86fc2 100644 > > --- a/xen/arch/x86/hvm/hvm.c > > +++ b/xen/arch/x86/hvm/hvm.c > > @@ -4430,6 +4430,7 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg) > > v->arch.hvm_vcpu.inject_trap.error_code = tr.error_code; > > v->arch.hvm_vcpu.inject_trap.insn_len = tr.insn_len; > > v->arch.hvm_vcpu.inject_trap.cr2 = tr.cr2; > > + rc = 0; > > } > > > > param_fail8: > > > > > > > > This worksThanks. I''ll take that as a Tested-by:. Keir, Jan, can I get an Ack? George, this is a clean bug-fix for something seen in the field, in a path that doesn''t affect any other features. OK before 4.3? Jan, this is a candidate for backporting to 4.1.> but the instruction size must be set to 1, at least on 4.2.2 > to work for me. Here is the patch against RELEASE-4.2.2.Sorry, I wasn''t clear: setting it to 1 is certainly an improvement over zero (which is always wrong), but if you''re relying on this to be correct you should also handle cases where prefixes make the instruction longer than 1 byte. In any case, this tools-side change needs to be acked/nacked by a tools maintainer, and probably Aravindh too. Cheers, Tim.> diff --git a/tools/tests/xen-access/xen-access.c > b/tools/tests/xen-access/xen-access.c > index 9ec7332..8bcd88b 100644 > --- a/tools/tests/xen-access/xen-access.c > +++ b/tools/tests/xen-access/xen-access.c > @@ -664,7 +664,7 @@ int main(int argc, char *argv[]) > /* Reinject */ > rc = xc_hvm_inject_trap( > xch, domain_id, req.vcpu_id, 3, > - HVMOP_TRAP_sw_exc, -1, 0, 0); > + HVMOP_TRAP_sw_exc, -1, 1, 0); > if (rc < 0) > { > ERROR("Error %d injecting int3\n", rc);
Tim Deegan
2013-Jun-20 10:33 UTC
Re: [Xen-users] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
(Cc-ing a few more people, moving xen-users to Bcc) At 14:51 +0000 on 15 Jun (1371307878), Antony Saba wrote:> >> 2) xc_hvm_inject_trap() always returns a negative value, even when there > >> is not a problem and the guest receives the trap as expected. There > >> hasn''t been a clarification as to whether it''s supposed to return > >> non-negative, but one would assume that it should because of the way the > >> xen-access.c example checks for it. > > > > That looks like a hypervisor bug to me: does this (untested) patch fix > > it for you? > > > > commit 67b9272fcedcb5dc73cc77a2adf580f2572117d7 > > Author: Tim Deegan <tim@xen.org> > > Date: Mon Jun 10 19:35:34 2013 +0100 > > > > x86/hvm: Fix HVMOP_inject_trap return value on success. > > > > Reported-by: Antony Saba <Antony.Saba@mandiant.com> > > Signed-off-by: Tim Deegan <tim@xen.org> > > > > diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c > > index ce44bff..6c86fc2 100644 > > --- a/xen/arch/x86/hvm/hvm.c > > +++ b/xen/arch/x86/hvm/hvm.c > > @@ -4430,6 +4430,7 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg) > > v->arch.hvm_vcpu.inject_trap.error_code = tr.error_code; > > v->arch.hvm_vcpu.inject_trap.insn_len = tr.insn_len; > > v->arch.hvm_vcpu.inject_trap.cr2 = tr.cr2; > > + rc = 0; > > } > > > > param_fail8: > > > > > > > > This worksThanks. I''ll take that as a Tested-by:. Keir, Jan, can I get an Ack? George, this is a clean bug-fix for something seen in the field, in a path that doesn''t affect any other features. OK before 4.3? Jan, this is a candidate for backporting to 4.1.> but the instruction size must be set to 1, at least on 4.2.2 > to work for me. Here is the patch against RELEASE-4.2.2.Sorry, I wasn''t clear: setting it to 1 is certainly an improvement over zero (which is always wrong), but if you''re relying on this to be correct you should also handle cases where prefixes make the instruction longer than 1 byte. In any case, this tools-side change needs to be acked/nacked by a tools maintainer, and probably Aravindh too. Cheers, Tim.> diff --git a/tools/tests/xen-access/xen-access.c > b/tools/tests/xen-access/xen-access.c > index 9ec7332..8bcd88b 100644 > --- a/tools/tests/xen-access/xen-access.c > +++ b/tools/tests/xen-access/xen-access.c > @@ -664,7 +664,7 @@ int main(int argc, char *argv[]) > /* Reinject */ > rc = xc_hvm_inject_trap( > xch, domain_id, req.vcpu_id, 3, > - HVMOP_TRAP_sw_exc, -1, 0, 0); > + HVMOP_TRAP_sw_exc, -1, 1, 0); > if (rc < 0) > { > ERROR("Error %d injecting int3\n", rc);
Keir Fraser
2013-Jun-20 11:19 UTC
Re: [Xen-users] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
On 20/06/2013 11:33, "Tim Deegan" <tim@xen.org> wrote:> (Cc-ing a few more people, moving xen-users to Bcc) > > At 14:51 +0000 on 15 Jun (1371307878), Antony Saba wrote: >>>> 2) xc_hvm_inject_trap() always returns a negative value, even when there >>>> is not a problem and the guest receives the trap as expected. There >>>> hasn''t been a clarification as to whether it''s supposed to return >>>> non-negative, but one would assume that it should because of the way the >>>> xen-access.c example checks for it. >>> >>> That looks like a hypervisor bug to me: does this (untested) patch fix >>> it for you? >>> >>> commit 67b9272fcedcb5dc73cc77a2adf580f2572117d7 >>> Author: Tim Deegan <tim@xen.org> >>> Date: Mon Jun 10 19:35:34 2013 +0100 >>> >>> x86/hvm: Fix HVMOP_inject_trap return value on success. >>> >>> Reported-by: Antony Saba <Antony.Saba@mandiant.com> >>> Signed-off-by: Tim Deegan <tim@xen.org> >>> >>> diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c >>> index ce44bff..6c86fc2 100644 >>> --- a/xen/arch/x86/hvm/hvm.c >>> +++ b/xen/arch/x86/hvm/hvm.c >>> @@ -4430,6 +4430,7 @@ long do_hvm_op(unsigned long op, >>> XEN_GUEST_HANDLE_PARAM(void) arg) >>> v->arch.hvm_vcpu.inject_trap.error_code = tr.error_code; >>> v->arch.hvm_vcpu.inject_trap.insn_len = tr.insn_len; >>> v->arch.hvm_vcpu.inject_trap.cr2 = tr.cr2; >>> + rc = 0; >>> } >>> >>> param_fail8: >>> >>> >>> >> >> This works > > Thanks. I''ll take that as a Tested-by:. > > Keir, Jan, can I get an Ack?By preference, and even for 4.3 since it is still obviously correct, I would change the handling of inject_trap.vector != -1 to be the same as previous error checks, moving the actual work out of an else block. But maybe that should be a later cleanup. Either way: Acked-by: Keir Fraser <keir@xen.org>> George, this is a clean bug-fix for something seen in the field, in a > path that doesn''t affect any other features. OK before 4.3?Agreed, this is really a no-brainer for 4.3. It''s fixing an obvio bug. -- Keir> Jan, this is a candidate for backporting to 4.1. > >> but the instruction size must be set to 1, at least on 4.2.2 >> to work for me. Here is the patch against RELEASE-4.2.2. > > Sorry, I wasn''t clear: setting it to 1 is certainly an improvement over > zero (which is always wrong), but if you''re relying on this to be > correct you should also handle cases where prefixes make the instruction > longer than 1 byte. > > In any case, this tools-side change needs to be acked/nacked by a tools > maintainer, and probably Aravindh too. > > Cheers, > > Tim. > >> diff --git a/tools/tests/xen-access/xen-access.c >> b/tools/tests/xen-access/xen-access.c >> index 9ec7332..8bcd88b 100644 >> --- a/tools/tests/xen-access/xen-access.c >> +++ b/tools/tests/xen-access/xen-access.c >> @@ -664,7 +664,7 @@ int main(int argc, char *argv[]) >> /* Reinject */ >> rc = xc_hvm_inject_trap( >> xch, domain_id, req.vcpu_id, 3, >> - HVMOP_TRAP_sw_exc, -1, 0, 0); >> + HVMOP_TRAP_sw_exc, -1, 1, 0); >> if (rc < 0) >> { >> ERROR("Error %d injecting int3\n", rc);
Aravindh Puthiyaparambil (aravindp)
2013-Jun-20 21:44 UTC
Re: [Xen-users] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
> (Cc-ing a few more people, moving xen-users to Bcc) > > At 14:51 +0000 on 15 Jun (1371307878), Antony Saba wrote: > > >> 2) xc_hvm_inject_trap() always returns a negative value, even when > > >> there is not a problem and the guest receives the trap as expected. > > >> There hasn''t been a clarification as to whether it''s supposed to > > >> return non-negative, but one would assume that it should because of > > >> the way the xen-access.c example checks for it. > > > > > > That looks like a hypervisor bug to me: does this (untested) patch > > > fix it for you? > > > > > > commit 67b9272fcedcb5dc73cc77a2adf580f2572117d7 > > > Author: Tim Deegan <tim@xen.org> > > > Date: Mon Jun 10 19:35:34 2013 +0100 > > > > > > x86/hvm: Fix HVMOP_inject_trap return value on success. > > > > > > Reported-by: Antony Saba <Antony.Saba@mandiant.com> > > > Signed-off-by: Tim Deegan <tim@xen.org> > > > > > > diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index > > > ce44bff..6c86fc2 100644 > > > --- a/xen/arch/x86/hvm/hvm.c > > > +++ b/xen/arch/x86/hvm/hvm.c > > > @@ -4430,6 +4430,7 @@ long do_hvm_op(unsigned long op, > XEN_GUEST_HANDLE_PARAM(void) arg) > > > v->arch.hvm_vcpu.inject_trap.error_code = tr.error_code; > > > v->arch.hvm_vcpu.inject_trap.insn_len = tr.insn_len; > > > v->arch.hvm_vcpu.inject_trap.cr2 = tr.cr2; > > > + rc = 0; > > > } > > > > > > param_fail8: > > > > > > > > > > > > > This works > > Thanks. I''ll take that as a Tested-by:. > > Keir, Jan, can I get an Ack? > > George, this is a clean bug-fix for something seen in the field, in a path that > doesn''t affect any other features. OK before 4.3? > > Jan, this is a candidate for backporting to 4.1. > > > but the instruction size must be set to 1, at least on 4.2.2 to work > > for me. Here is the patch against RELEASE-4.2.2. > > Sorry, I wasn''t clear: setting it to 1 is certainly an improvement over zero > (which is always wrong), but if you''re relying on this to be correct you should > also handle cases where prefixes make the instruction longer than 1 byte.I agree this is definitely an improvement. I will try adding the prefix check when I get the chance.> In any case, this tools-side change needs to be acked/nacked by a tools > maintainer, and probably Aravindh too.Acked-by: Aravindh Puthiyaparambil <aravindp@cisco.com> Thanks, Aravindh
George Dunlap
2013-Jun-21 14:45 UTC
Re: [Xen-users] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
On Thu, Jun 20, 2013 at 12:19 PM, Keir Fraser <keir.xen@gmail.com> wrote:> On 20/06/2013 11:33, "Tim Deegan" <tim@xen.org> wrote: > >> (Cc-ing a few more people, moving xen-users to Bcc) >> >> At 14:51 +0000 on 15 Jun (1371307878), Antony Saba wrote: >>>>> 2) xc_hvm_inject_trap() always returns a negative value, even when there >>>>> is not a problem and the guest receives the trap as expected. There >>>>> hasn''t been a clarification as to whether it''s supposed to return >>>>> non-negative, but one would assume that it should because of the way the >>>>> xen-access.c example checks for it. >>>> >>>> That looks like a hypervisor bug to me: does this (untested) patch fix >>>> it for you? >>>> >>>> commit 67b9272fcedcb5dc73cc77a2adf580f2572117d7 >>>> Author: Tim Deegan <tim@xen.org> >>>> Date: Mon Jun 10 19:35:34 2013 +0100 >>>> >>>> x86/hvm: Fix HVMOP_inject_trap return value on success. >>>> >>>> Reported-by: Antony Saba <Antony.Saba@mandiant.com> >>>> Signed-off-by: Tim Deegan <tim@xen.org> >>>> >>>> diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c >>>> index ce44bff..6c86fc2 100644 >>>> --- a/xen/arch/x86/hvm/hvm.c >>>> +++ b/xen/arch/x86/hvm/hvm.c >>>> @@ -4430,6 +4430,7 @@ long do_hvm_op(unsigned long op, >>>> XEN_GUEST_HANDLE_PARAM(void) arg) >>>> v->arch.hvm_vcpu.inject_trap.error_code = tr.error_code; >>>> v->arch.hvm_vcpu.inject_trap.insn_len = tr.insn_len; >>>> v->arch.hvm_vcpu.inject_trap.cr2 = tr.cr2; >>>> + rc = 0; >>>> } >>>> >>>> param_fail8: >>>> >>>> >>>> >>> >>> This works >> >> Thanks. I''ll take that as a Tested-by:. >> >> Keir, Jan, can I get an Ack? > > By preference, and even for 4.3 since it is still obviously correct, I would > change the handling of inject_trap.vector != -1 to be the same as previous > error checks, moving the actual work out of an else block. But maybe that > should be a later cleanup. Either way: > Acked-by: Keir Fraser <keir@xen.org> > >> George, this is a clean bug-fix for something seen in the field, in a >> path that doesn''t affect any other features. OK before 4.3? > > Agreed, this is really a no-brainer for 4.3. It''s fixing an obvio bug.Looks good -- sorry for the delay in responding. Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
Tim Deegan
2013-Jun-27 09:49 UTC
Re: [Xen-users] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2
Hi, At 21:44 +0000 on 20 Jun (1371764675), Aravindh Puthiyaparambil (aravindp) wrote:> > > but the instruction size must be set to 1, at least on 4.2.2 to work > > > for me. Here is the patch against RELEASE-4.2.2. > > > > Sorry, I wasn''t clear: setting it to 1 is certainly an improvement over zero > > (which is always wrong), but if you''re relying on this to be correct you should > > also handle cases where prefixes make the instruction longer than 1 byte. > > I agree this is definitely an improvement. I will try adding the prefix check when I get the chance. > > > In any case, this tools-side change needs to be acked/nacked by a tools > > maintainer, and probably Aravindh too. > > Acked-by: Aravindh Puthiyaparambil <aravindp@cisco.com>Antony, if you want this tools-side patch merged, please send it as a separate patch, with a description and appropriate Signed-off-by: lines as described here: http://wiki.xen.org/wiki/Submitting_Xen_Patches You should add Aravindh''s ack, but not mine, and Cc: the tools maintainers. If you want this to go in before 4.3, you should also Cc: George and explain why it fits his release criteria (though I suspect it''s too late now). Cheers, Tim.