Hi all, I originally posted this on the xenproject Q&A forum, and it was suggested that I repost it here. If it would be better posted elsewhere then please let me know. Please skip the next paragraph if you''re not interested in the why. Over the last 2 decades I have been developing a real-time OS with one of my customers (I was an employee when we started, a consultant for the last 15 years). This runs on our own hardware (originally TI TMS320C3x, then MIPS64, then ARM9). The system started off 80% C 20% Assembly, and is now about 99% C and 1% Assembly (just the bootstrapping). We now have a requirement to run this OS in parallel with Windows. I have looked at various real-time hypervisors and Windows extensions, however I think we can get more mileage out of Xen due to the open source nature of the project. I bought the book "The Definitive Guide to the Xen Hypervisor" as it was recommended by the xenproject Q&A forum and it is very interesting, however the examples are 32 bit and I am working 64 bit, and let''s say that my x86 assembler is a bit rusty to say the least! My questions (I am running Xen 4.1.4 on Debian Wheezy): 1.- I have looked at MirageOS but I think it is overkill for what I want to do. I just want a basic C wrapper to allow Xen to load my software, I would prefer not to have to learn OCaml, but if this is what it takes, then I''ll have to do it. 2.- Is there any sample simple x86_64 code that I can work from to get out of the starting blocks? 3.- xl complains about my image not being a bzImage. Is there anyway I can avoid this error/warning? If not how do I convert my image to a bzImage? I''d rather not have to port in half the Linux bootstrap. 4.- Despite xl complaining about the kernel not being in bzImage format it still seems to run (I get the same error message from the MirageOS but the image loads and runs OK). However it crashes and I have no way of seeing what''s going on. Is there any way to debug my guest startup inside Xen? TIA and Regards. -- _ _ Debian GNU User Simon Martin | | (_)_ __ _ ___ __ Project Manager | | | | ''_ \| | | \ \/ / Milliways | |___| | | | | |_| |> < mailto: smartin@milliways.cl |_____|_|_| |_|\__,_/_/\_\ Si Hoc Legere Scis Nimium Eruditionis Habes _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On 13/10/2013 21:28, Simon Martin wrote:> Hi all, > > I originally posted this on the xenproject Q&A forum, and it was > suggested that I repost it here. If it would be better posted > elsewhere then please let me know. Please skip the next paragraph if > you''re not interested in the why.I would say that you have found the correct place to be asking questions like this.> > Over the last 2 decades I have been developing a real-time OS with one > of my customers (I was an employee when we started, a consultant for > the last 15 years). This runs on our own hardware (originally TI > TMS320C3x, then MIPS64, then ARM9). The system started off 80% C 20% > Assembly, and is now about 99% C and 1% Assembly (just the > bootstrapping). We now have a requirement to run this OS in parallel > with Windows. I have looked at various real-time hypervisors and > Windows extensions, however I think we can get more mileage out of Xen > due to the open source nature of the project. > > I bought the book "The Definitive Guide to the Xen Hypervisor" as it > was recommended by the xenproject Q&A forum and it is very > interesting, however the examples are 32 bit and I am working 64 bit, > and let''s say that my x86 assembler is a bit rusty to say the least! > > My questions (I am running Xen 4.1.4 on Debian Wheezy): > > 1.- I have looked at MirageOS but I think it is overkill for what I > want to do. I just want a basic C wrapper to allow Xen to load my > software, I would prefer not to have to learn OCaml, but if this is > what it takes, then I''ll have to do it. > > 2.- Is there any sample simple x86_64 code that I can work from to get > out of the starting blocks?Yes. I am not familiar with what is distributed with Wheezy; you are probably better checking out from the git tree ( http://xenbits.xen.org/gitweb/?p=xen.git;a=summary ). In extras/MiniOS, you will find MiniOS which is a minimal example of getting code working as a paravirtualised guest. It should certainly be a good starting point and reference.> > 3.- xl complains about my image not being a bzImage. Is there anyway I > can avoid this error/warning? If not how do I convert my image to a > bzImage? I''d rather not have to port in half the Linux bootstrap.What format are you trying to boot? It should be happy booting any format that a Linux kernel might be, which would typically be an Elf multiboot image with some form of compression.> > 4.- Despite xl complaining about the kernel not being in bzImage > format it still seems to run (I get the same error message from the > MirageOS but the image loads and runs OK). However it crashes and I > have no way of seeing what''s going on. Is there any way to debug my > guest startup inside Xen?There are several options depending on your preference. on_crash="pause" in your domain configuration file will leave the VM state to be inspected. "coredump-destroy" and "coredump-restart" are also alternatives. xl dump-core <domid> <file> should create an Elf CORE file representing the domain. The gdbsx set of tools (tools/debugger/gdbsx) allow you to attach gdb to a running Xen domain. Getting a PV console working (see the early code from MiniOS) would allow you to print inforation. Failing a PV console, if you build yourself a debug version of Xen, you can use the console_io hypercall from your guest and get debugging messages into the Xen console. Hopefully that should be enough to get started with debugging. One final thing which come to mind given your request which you might or might not be aware of: Xen has "arinc653", a maintained realtime scheduler which will likely be more appropriate to your requirements than the default "credit" scheduler. Hope this is somewhat helpful ~Andrew _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Hi Andrew Thank you very much for your response. Comments in line. Regards. ------ Original Message ------ From: "Andrew Cooper" <andrew.cooper3@citrix.com> To: "Simon Martin" <smartin@milliways.cl>; xen-devel@lists.xen.org Sent: 13/10/2013 20:07:25 Subject: Re: [Xen-devel] Porting an OS to Xen>On 13/10/2013 21:28, Simon Martin wrote: >>2.- Is there any sample simple x86_64 code that I can work from to get >>out of the starting blocks? > >Yes. I am not familiar with what is distributed with Wheezy; you are >probably better checking out from the git tree ( >http://xenbits.xen.org/gitweb/?p=xen.git;a=summary ). In >extras/MiniOS, you will find MiniOS which is a minimal example of >getting code working as a paravirtualised guest. It should certainly >be a good starting point and reference.Got it. Merging it with the 32 bit example code I have. Slowly getting there!> >> >>3.- xl complains about my image not being a bzImage. Is there anyway I >>can avoid this error/warning? If not how do I convert my image to a >>bzImage? I''d rather not have to port in half the Linux bootstrap. > >What format are you trying to boot? It should be happy booting any >format that a Linux kernel might be, which would typically be an Elf >multiboot image with some form of compression.I am generating an ELF-64 binary. I have a feeling that this is a "normal error" when starting a non-bzImage kernel. The mini-os example causes the same error, so I will ignore it.> >> >>4.- Despite xl complaining about the kernel not being in bzImage >>format it still seems to run (I get the same error message from the >>MirageOS but the image loads and runs OK). However it crashes and I >>have no way of seeing what''s going on. Is there any way to debug my >>guest startup inside Xen? > >There are several options depending on your preference. > >on_crash="pause" in your domain configuration file will leave the VM >state to be inspected. "coredump-destroy" and "coredump-restart" are >also alternatives. > >xl dump-core <domid> <file> should create an Elf CORE file representing >the domain. > >The gdbsx set of tools (tools/debugger/gdbsx) allow you to attach gdb >to a running Xen domain. > >Getting a PV console working (see the early code from MiniOS) would >allow you to print inforation. > >Failing a PV console, if you build yourself a debug version of Xen, you >can use the console_io hypercall from your guest and get debugging >messages into the Xen console. > >Hopefully that should be enough to get started with debugging.Perfect.> >One final thing which come to mind given your request which you might >or might not be aware of: Xen has "arinc653", a maintained realtime >scheduler which will likely be more appropriate to your requirements >than the default "credit" scheduler.For simplicity''s sake at the moment I propose to run my VM as the only VM on a vcpu. My assumption is that this way there will be no scheduling. Is this assumption correct?> > >Hope this is somewhat helpful > >~Andrew_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Mon, 2013-10-14 at 16:22 +0000, Simon Martin wrote:> > > 3.- xl complains about my image not being a bzImage. Is there > > > anyway I can avoid this error/warning? If not how do I convert my > > > image to a bzImage? I''d rather not have to port in half the Linux > > > bootstrap. > > > > What format are you trying to boot? It should be happy booting any > > format that a Linux kernel might be, which would typically be an Elf > > multiboot image with some form of compression. > I am generating an ELF-64 binary. I have a feeling that this is a > "normal error" when starting a non-bzImage kernel. The mini-os example > causes the same error, so I will ignore it.I think the error is just during probing where it checks to see if it is a bzImage and falls through to trying ELF if not. The message could probably be made less error-ry. Ian.
On 14/10/13 17:22, Simon Martin wrote:>> >> One final thing which come to mind given your request which you might >> or might not be aware of: Xen has "arinc653", a maintained realtime >> scheduler which will likely be more appropriate to your requirements >> than the default "credit" scheduler. > For simplicity''s sake at the moment I propose to run my VM as the > only VM on a vcpu. My assumption is that this way there will be no > scheduling. Is this assumption correct?Your best option would be to create a cpupool for the individual cpu you want your VM to be running on. This way you can schedule that cpu using arinc653 independently of dom0 and domU being scheduled using regular credit. ~Andrew --------------020304080801020706090708 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: 8bit <html> <head> <meta content="text/html; charset=UTF-8" http-equiv="Content-Type"> </head> <body text="#000000" bgcolor="#FFFFFF"> <div class="moz-cite-prefix">On 14/10/13 17:22, Simon Martin wrote:<br> </div> <blockquote cite="mid:em8e0ab2b8-7b9f-489a-8c94-42105ad47619@smartin-alien" type="cite"> <style id="eMClientCss">blockquote.cite { margin-left: 5px; margin-right: 0px; padding-left: 10px; padding-right:0px; border-left: 1px solid #000000 } .plain pre, .plain tt { font-family: monospace; font-size: 100%; font-weight: normal; font-style: normal; } body {font-family: Courier New;font-size: 10pt;} .plain pre, .plain tt {font-family: Courier New;font-size: 10pt;} </style> <style>#5682aaa335db44bc88fba4f9c2b767cb .plain PRE, #5682aaa335db44bc88fba4f9c2b767cb .plain TT {FONT-SIZE: 100%; FONT-FAMILY: monospace; FONT-WEIGHT: normal; FONT-STYLE: normal} #5682aaa335db44bc88fba4f9c2b767cb, #5682aaa335db44bc88fba4f9c2b767cb .plain PRE, #5682aaa335db44bc88fba4f9c2b767cb #72df6a49821d4aec8bef7a4ece771698, #5682aaa335db44bc88fba4f9c2b767cb .plain TT {FONT-SIZE: 10pt; FONT-FAMILY: Courier New}</style> <blockquote class="cite" cite="525B27AD.9050103@citrix.com" type="cite"> <div id="5682aaa335db44bc88fba4f9c2b767cb" style="COLOR: #000000"><br> One final thing which come to mind given your request which you might or might not be aware of: Xen has "arinc653", a maintained realtime scheduler which will likely be more appropriate to your requirements than the default "credit" scheduler.</div> </blockquote> <div>For simplicity''s sake at the moment I propose to run my VM as the only VM on a vcpu. My assumption is that this way there will be no scheduling. Is this assumption correct?</div> </blockquote> <br> Your best option would be to create a cpupool for the individual cpu you want your VM to be running on. This way you can schedule that cpu using arinc653 independently of dom0 and domU being scheduled using regular credit.<br> <br> ~Andrew<br> </body> </html> --------------020304080801020706090708-- --===============4481462987722499127=Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============4481462987722499127==--