Hi all, I am running on Xen 4.4 with all Nate''s ARINC653 patches, and am trying to run a PV guest in a cpu pool with a single CPU and the ARINC653 scheduler. I set up the pools as follows: xl cpupool-cpu-remove Pool-0 0 xl cpupool-create pv400_cpupool where pv400_cpupool has the following name = "pv400" sched = "arinc653" cpus = ["0"] When I start my guest it is loaded, but nothing happens after that. If I debug the domain, the I find that PC is stuck at 0, i.e. it has never been assigned CPU. Any ideas on what I''m doing wrong? -- _ _ Debian GNU User Simon Martin | | (_)_ __ _ ___ __ Project Manager | | | | ''_ \| | | \ \/ / Milliways | |___| | | | | |_| |> < mailto: smartin@milliways.cl |_____|_|_| |_|\__,_/_/\_\ Si Hoc Legere Scis Nimium Eruditionis Habes _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Hi All, Think I''ve answered my own question. Have to run a program on dom0 to configure scheduler. This looks like the same problem: http://www.gossamer-threads.com/lists/xen/users/306701 with Nate''s answer. Regards. ------ Original Message ------ From: "Simon Martin" <smartin@milliways.cl> To: "xen-devel" <xen-devel@lists.xen.org> Sent: 28/11/2013 10:07:36 Subject: ARINC653>Hi all, > >I am running on Xen 4.4 with all Nate''s ARINC653 patches, and am trying >to run a PV guest in a cpu pool with a single CPU and the ARINC653 >scheduler. I set up the pools as follows: > >xl cpupool-cpu-remove Pool-0 0 >xl cpupool-create pv400_cpupool > >where pv400_cpupool has the following > >name = "pv400" >sched = "arinc653" >cpus = ["0"] >When I start my guest it is loaded, but nothing happens after that. If >I debug the domain, the I find that PC is stuck at 0, i.e. it has never >been assigned CPU. > >Any ideas on what I''m doing wrong? > >-- > _ _ Debian GNU User Simon Martin >| | (_)_ __ _ ___ __ Project Manager >| | | | ''_ \| | | \ \/ / Milliways >| |___| | | | | |_| |> < mailto: smartin@milliways.cl >|_____|_|_| |_|\__,_/_/\_\ >Si Hoc Legere Scis Nimium Eruditionis Habes >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Hi Robbie, Nate said that I should contact you if I was having problems with the scheduler, and unfortunately I have. As far as I can see I have created a cpupool with 1 CPU running the arinc653 scheduler. Here the output from xl. root@smartin-xen:~# xl cpupool-list Name CPUs Sched Active Domain count Pool-0 3 credit y 1 pv400 1 arinc653 y 1 However the domU I load into the pv400 cpupool is never scheduled. I found a few mails talking about this, and it looks like I need to run a program on dom0 to configure the scheduler. I wrote the program and as far as I can see it should work, however whatever I do it fails. I tried to put a few printk statements in the corresponding arinc653_sched_set but I don''t see any output in the xl dmesg, or domU log either. I attach the code I wrote to configure the scheduler. It accepts one parameter, the UUID of the dom_handle I need to specify. The mail I found wasn''t too clear so I test this using the dom0 UUID (00000000-0000-0000-0000-000000000000) and with the domU UUID as well. Both fail the do_sysctl call in xc_sched_arinc653_schedule_set (xc_arinc653.c line 51). Can you give me some idea of what I''m doing wrong here? Regards. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On 12/3/2013 7:07 AM, Simon Martin wrote:> Hi Robbie, > > Nate said that I should contact you if I was having problems with the scheduler, > and unfortunately I have.I am actually back, so I can field this one.> > As far as I can see I have created a cpupool with 1 CPU running the arinc653 > scheduler. Here the output from xl. > > root@smartin-xen <mailto:root@smartin-xen>:~# xl cpupool-list > Name CPUs Sched Active Domain count > Pool-0 3 credit y 1 > pv400 1 arinc653 y 1 > > However the domU I load into the pv400 cpupool is never scheduled. I found a few > mails talking about this, and it looks like I need to run a program on dom0 to > configure the scheduler. I wrote the program and as far as I can see it should > work, however whatever I do it fails. I tried to put a few printk statements in > the corresponding arinc653_sched_set but I don''t see any output in the xl dmesg, > or domU log either. > > I attach the code I wrote to configure the scheduler. It accepts one parameter, > the UUID of the dom_handle I need to specify. The mail I found wasn''t too clear > so I test this using the dom0 UUID (00000000-0000-0000-0000-000000000000) and > with the domU UUID as well. Both fail the do_sysctl call in > xc_sched_arinc653_schedule_set (xc_arinc653.c line 51). > > Can you give me some idea of what I''m doing wrong here?Did you apply the patches I posted to the list a couple weeks ago? I noticed that you are calling the xc_sched_arinc653_schedule_set function without a pool id, so I think the answer is no. http://lists.xenproject.org/archives/html/xen-devel/2013-11/msg02549.html Without these patches, you can only load a schedule into Pool-0. If you just want to wait a little bit, I should be addressing Andrew''s comments soon, in order to get them into unstable. Nate> > Regards. >
Thanks Nate. I hope everything went well!>Did you apply the patches I posted to the list a couple weeks ago? I >noticed >that you are calling the xc_sched_arinc653_schedule_set function >without a pool >id, so I think the answer is no. > >http://lists.xenproject.org/archives/html/xen-devel/2013-11/msg02549.html > >Without these patches, you can only load a schedule into Pool-0. > >If you just want to wait a little bit, I should be addressing Andrew''s >comments >soon, in order to get them into unstable.I didn''t apply those patches. I just brought in the patches that were committed to git. Sounds like it''s better if I wait for you to commit your stuff before continuing with this so I''ll to onto other bits of the system for the moment. Regards. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Hi Nate, I finally got round to pulling in the latest and greatest patches and after a few silly mistakes (RTFM) I now have a working system. As mentioned before I have 2 pools: smartin@smartin-xen:~/projects/trio/xen/pv499$ sudo xl cpupool-list Name CPUs Sched Active Domain count Pool-0 3 credit y 1 pv499 1 arinc653 y 1 After loading the domU into the pv499 and running the program that configures the arinc scheduler I get this: smartin@smartin-xen:~/projects/trio/xen/pv499$ sudo xl list Name ID Mem VCPUs State Time(s) Domain-0 0 2967 4 r----- 117.2 pv499 24 32 1 r----- 373.0 So far so good. Now looking at the timings things are not too different from when I was using the credit scheduler. Running a single shot timer at a period of 250 µs gives me a jitter in the timer handler latency in the range [3.5 µs, 45 µs]. This makes it look like the problem in the jitter is not the scheduling but the underlying hypervisor. It is going off to do things on this CPU which is causing the deadlines not to be met. So unless I am doing something silly it looks like I will have to go to RT-Xen to get better timings. For the moment I''m going to leave this and go on to getting more of my operating system working. Regards. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On lun, 2013-12-09 at 20:23 +0000, Simon Martin wrote:> smartin@smartin-xen:~/projects/trio/xen/pv499$ sudo xl cpupool-list > Name CPUs Sched Active Domain count > Pool-0 3 credit y 1 > pv499 1 arinc653 y 1 >Ok, which pcpus are in each pool? In theory, it shouldn''t matter, but, you know... :-P> After loading the domU into the pv499 and running the program that > configures the arinc scheduler I get this: > > smartin@smartin-xen:~/projects/trio/xen/pv499$ sudo xl list > Name ID Mem VCPUs State > Time(s) > Domain-0 0 2967 4 r----- > 117.2 > pv499 24 32 1 r----- > 373.0 > > So far so good. Now looking at the timings things are not too > different from when I was using the credit scheduler. Running a single > shot timer at a period of 250 µs gives me a jitter in the timer > handler latency in the range [3.5 µs, 45 µs]. This makes it look like > the problem in the jitter is not the scheduling but the underlying > hypervisor. It is going off to do things on this CPU which is causing > the deadlines not to be met. > > So unless I am doing something silly it looks like I will have to go > to RT-Xen to get better timings. For the moment I''m going to leave > this and go on to getting more of my operating system working. >Well, RT-Xen is a great project, but it''s mostly, if not all, about scheduling. I mean, all they do is introducing two new schedulers (with a couple of operational mode in each), and that''s it... There is nothing there being done for reducing latency or jitter. I think, we''d better off (as soon as, as you said, you have more pieces in place), trying to figure out what is causing the behavior you''re seeing and workaround/fix/amend it! It''s a busy period, with the 4.4 release, etc., but, as repeatedly said, I''m up for helping with this, if help is needed! :-P Regards, Dario -- <<This happens because I choose it to happen!>> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Thanks Dario,>> smartin@smartin-xen:~/projects/trio/xen/pv499$ sudo xl cpupool-list >> Name CPUs Sched Active Domain count >> Pool-0 3 credit y 1 >> pv499 1 arinc653 y 1 >> >Ok, which pcpus are in each pool? In theory, it shouldn''t matter, but, >you know... >Pool-0 has pcpu 1, 2 and 3 pv499 has pcpu 0.>> So unless I am doing something silly it looks like I will have to go >> to RT-Xen to get better timings. For the moment I''m going to leave >> this and go on to getting more of my operating system working. >> >Well, RT-Xen is a great project, but it''s mostly, if not all, about >scheduling. I mean, all they do is introducing two new schedulers (with >a couple of operational mode in each), and that''s it... There is >nothing >there being done for reducing latency or jitter.Thanks for the pointer. As my tests have shown, changing the scheduler does not really alter the behaviour.> >I think, we''d better off (as soon as, as you said, you have more pieces >in place), trying to figure out what is causing the behavior you''re >seeing and workaround/fix/amend it!I agree.> >It''s a busy period, with the 4.4 release, etc., but, as repeatedly >said, >I''m up for helping with this, if help is needed!Thanks. This is something that is going to take a while to get sorted so I''ll definitely be plugging until way after 4.4 is done and dusted. Regards.>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On lun, 2013-12-09 at 23:08 +0000, Simon Martin wrote:> Thanks Dario, > > > > smartin@smartin-xen:~/projects/trio/xen/pv499$ sudo xl > > > cpupool-list > > > Name CPUs Sched Active Domain count > > > Pool-0 3 credit y 1 > > > pv499 1 arinc653 y 1 > > > > > Ok, which pcpus are in each pool? In theory, it shouldn''t matter, > > but, > > you know... :-P > > > Pool-0 has pcpu 1, 2 and 3 > pv499 has pcpu 0. >Can you try the opposite? I mean, something like having the 1 pCPU only pool when you run your domU *not* including pCPU #0? I can''t point the finger to anything specific right now, but, you now, pCPU #0 is the first being brought up, etc. Probably, as scheduling does not matter much to you, you can even just use vCPU affinities/pinning, like having Dom0 pinned to pCPUs 0-2, and your DomU to pCPU 3.> > It''s a busy period, with the 4.4 release, etc., but, as repeatedly > > said, > > I''m up for helping with this, if help is needed! :-P > Thanks. This is something that is going to take a while to get sorted > so I''ll definitely be plugging until way after 4.4 is done > and dusted. >Well, in that case I''m sure I''ll be able to help a bit. :-) Regards, Dario -- <<This happens because I choose it to happen!>> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Hi Dario,>> Pool-0 has pcpu 1, 2 and 3 >> pv499 has pcpu 0. >> >Can you try the opposite? I mean, something like having the 1 pCPU only >pool when you run your domU *not* including pCPU #0? >I changed it so that Pool-0 has pcpu 0, 1 and 2, and pv499 has pcpu 3. No difference. Which is good in a way :-). Regards. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel