Hi! I am interested in writing a scheduler for Xen for academic purposes. I need to generate random numbers. Is it possible to generate random numbers in xen hypervisor developement?If this is possible, how can I do it? Michele _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > Hi! > I am interested in writing a scheduler for Xen for academic purposes.I need> to generate random numbers. > Is it possible to generate random numbers in xen hypervisordevelopement?If> this is possible, how can I do it? >How many bits do you random numbers need to be? At what rate do you need them? (10/second?, 1000000/second?) Would pseudo-random numbers do? If so, what repeat interval is sufficient? James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I need less than 10 bits at rate of 10 milliseconds. With a random number I will select the next VCPU to schedule. Michele On Fri, Feb 12, 2010 at 2:51 AM, James Harper <james.harper@bendigoit.com.au> wrote:> > > > Hi! > > I am interested in writing a scheduler for Xen for academic purposes. > I need > > to generate random numbers. > > Is it possible to generate random numbers in xen hypervisor > developement?If > > this is possible, how can I do it? > > > > How many bits do you random numbers need to be? > > At what rate do you need them? (10/second?, 1000000/second?) > > Would pseudo-random numbers do? If so, what repeat interval is > sufficient? > > James >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I think the short answer is, Xen does not have a mechanism to collect true randomness at the moment. I''m not an expert in random numbers, so the bitrate doesn''t mean anything to me. A couple of possible solutions come to mind: * Use pseudo-random numbers to start out with and test your theories, while working on getting truly random numbers in. * Punt the problem to dom0: Have a daemon in dom0 to read /dev/urandom and "upload" values into a ring read by Xen. If the ring is empty, use pseudo-random numbers seeded by old values in the ring (?). * Add entropy-collection to Xen. * If interrupted by a timer that''s longer than 1ms, just take a TSC and lop off the lower 10 bits. If you haven''t been interrupted by a timer, use pseudorandom numbers seeded by the lower 10 bits of the last TSC. As I said, I''m not an expert in collecting entropy, so some of these may be obviously brain-dead ideas. But it might give you enough to get started. -George On Fri, Feb 12, 2010 at 8:32 AM, michele.paolino <michele.paolino@studio.unibo.it> wrote:> I need less than 10 bits at rate of 10 milliseconds. With a random number I > will select the next VCPU to schedule. > > Michele > > On Fri, Feb 12, 2010 at 2:51 AM, James Harper > <james.harper@bendigoit.com.au> wrote: >> >> > >> > Hi! >> > I am interested in writing a scheduler for Xen for academic purposes. >> I need >> > to generate random numbers. >> > Is it possible to generate random numbers in xen hypervisor >> developement?If >> > this is possible, how can I do it? >> > >> >> How many bits do you random numbers need to be? >> >> At what rate do you need them? (10/second?, 1000000/second?) >> >> Would pseudo-random numbers do? If so, what repeat interval is >> sufficient? >> >> James > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ok, thanks! this is enought to get started. but how can I read a random number from /dev/urandom? is Xend the deamon that you talk me about? Thanks Michele On Fri, Feb 12, 2010 at 1:15 PM, George Dunlap <George.Dunlap@eu.citrix.com>wrote:> I think the short answer is, Xen does not have a mechanism to collect > true randomness at the moment. I''m not an expert in random numbers, > so the bitrate doesn''t mean anything to me. A couple of possible > solutions come to mind: > * Use pseudo-random numbers to start out with and test your theories, > while working on getting truly random numbers in. > * Punt the problem to dom0: Have a daemon in dom0 to read /dev/urandom > and "upload" values into a ring read by Xen. If the ring is empty, > use pseudo-random numbers seeded by old values in the ring (?). > * Add entropy-collection to Xen. > * If interrupted by a timer that''s longer than 1ms, just take a TSC > and lop off the lower 10 bits. If you haven''t been interrupted by a > timer, use pseudorandom numbers seeded by the lower 10 bits of the > last TSC. > > As I said, I''m not an expert in collecting entropy, so some of these > may be obviously brain-dead ideas. But it might give you enough to > get started. > > -George > > On Fri, Feb 12, 2010 at 8:32 AM, michele.paolino > <michele.paolino@studio.unibo.it> wrote: > > I need less than 10 bits at rate of 10 milliseconds. With a random number > I > > will select the next VCPU to schedule. > > > > Michele > > > > On Fri, Feb 12, 2010 at 2:51 AM, James Harper > > <james.harper@bendigoit.com.au> wrote: > >> > >> > > >> > Hi! > >> > I am interested in writing a scheduler for Xen for academic purposes. > >> I need > >> > to generate random numbers. > >> > Is it possible to generate random numbers in xen hypervisor > >> developement?If > >> > this is possible, how can I do it? > >> > > >> > >> How many bits do you random numbers need to be? > >> > >> At what rate do you need them? (10/second?, 1000000/second?) > >> > >> Would pseudo-random numbers do? If so, what repeat interval is > >> sufficient? > >> > >> James > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, 2010-02-12 at 17:42 -0500, michele.paolino wrote:> Ok, thanks! this is enought to get started. > but how can I read a random number from /dev/urandom?open(), read(). Just try a cat | hexdump on the cli, it''s really that simple.> is Xend the deamon that you talk me about?I think George was rather suggesting a standalone daemon. It would be better done just writing a background task in dom0. Establish a ring buffer on a page shared with xen. Xen consumes bits from the buffer and advances a consumer pointer. Your daemon reads bits from /dev/urandom and forwards them to the ring, advancing a producer pointer. A lot of communications in Xen is shared memory in the exact same fashion. Copy xen/include/public/io/console.h and study some of the pointer offset and barrier stuff in ring.h. If you''re stuck, see related code for samples, about any PV driver deals with this kind of interface, the console code probably being the most simple one. Advanced flow control: You will come across event channels anyway. But for the bandwith you''re indicating, you will most likely find polling works okay. It''s just about the ideal project to get started with basic Xen hacking. Daniel> Thanks > Michele > > > On Fri, Feb 12, 2010 at 1:15 PM, George Dunlap > <George.Dunlap@eu.citrix.com> wrote: > I think the short answer is, Xen does not have a mechanism to > collect > true randomness at the moment. I''m not an expert in random > numbers, > so the bitrate doesn''t mean anything to me. A couple of > possible > solutions come to mind: > * Use pseudo-random numbers to start out with and test your > theories, > while working on getting truly random numbers in. > * Punt the problem to dom0: Have a daemon in dom0 to > read /dev/urandom > and "upload" values into a ring read by Xen. If the ring is > empty, > use pseudo-random numbers seeded by old values in the ring > (?). > * Add entropy-collection to Xen. > * If interrupted by a timer that''s longer than 1ms, just take > a TSC > and lop off the lower 10 bits. If you haven''t been > interrupted by a > timer, use pseudorandom numbers seeded by the lower 10 bits of > the > last TSC. > > As I said, I''m not an expert in collecting entropy, so some of > these > may be obviously brain-dead ideas. But it might give you > enough to > get started. > > -George > > > On Fri, Feb 12, 2010 at 8:32 AM, michele.paolino > <michele.paolino@studio.unibo.it> wrote: > > I need less than 10 bits at rate of 10 milliseconds. With a > random number I > > will select the next VCPU to schedule. > > > > Michele > > > > On Fri, Feb 12, 2010 at 2:51 AM, James Harper > > <james.harper@bendigoit.com.au> wrote: > >> > >> > > >> > Hi! > >> > I am interested in writing a scheduler for Xen for > academic purposes. > >> I need > >> > to generate random numbers. > >> > Is it possible to generate random numbers in xen > hypervisor > >> developement?If > >> > this is possible, how can I do it? > >> > > >> > >> How many bits do you random numbers need to be? > >> > >> At what rate do you need them? (10/second?, > 1000000/second?) > >> > >> Would pseudo-random numbers do? If so, what repeat interval > is > >> sufficient? > >> > >> James > > > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Here''s my random scheduler. It works until I start a virtual machine (error file attached is the serial console''s log). In xen call trace there isn''t any of my functions. To generate random numbers I''m using the hash (MD5) of NOW() function. I would also Know why at boot time there are two calls at vcpu_init function for vcpu with id = 0 ?? Thanks Michele _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
The problem is that you''re choosing a random vcpu to run, without considering whether it *wants* to run or not. In this case, you''re running a vcpu before it''s even been completely set up yet (write_cr3 is failing because the guest *has* no cr3 ready yet). The normal way schedulers deal with this is to keep one list of all vcpus (or all domains), and another list with "runnable" vcpus. You can keep track of which vcpus are runnable with the vcpu_wake() callback and by using vcpu_runnable() in schedule(). At very least, your loop in schedule should check vcpu_runnable() before selecting it. -George michele.paolino wrote:> Here''s my random scheduler. It works until I start a virtual machine > (error file attached is the serial console''s log). > In xen call trace there isn''t any of my functions. > To generate random numbers I''m using the hash (MD5) of NOW() function. > I would also Know why at boot time there are two calls at vcpu_init > function for vcpu with id = 0 ?? > > Thanks > Michele_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Following George''s advices I have rewrited my scheduler. But now serial console''s log report me a bug in a function of timer.c (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) Xen BUG at timer.c:184 (XEN) **************************************** This is caused by cpumask_raise_softirq() in my wake() function. What can I do? Michele On Thu, Feb 25, 2010 at 10:25 PM, George Dunlap <george.dunlap@eu.citrix.com> wrote:> The problem is that you''re choosing a random vcpu to run, without > considering whether it *wants* to run or not. In this case, you''re running > a vcpu before it''s even been completely set up yet (write_cr3 is failing > because the guest *has* no cr3 ready yet). > > The normal way schedulers deal with this is to keep one list of all vcpus > (or all domains), and another list with "runnable" vcpus. You can keep > track of which vcpus are runnable with the vcpu_wake() callback and by using > vcpu_runnable() in schedule(). > > At very least, your loop in schedule should check vcpu_runnable() before > selecting it. > > -George > > > michele.paolino wrote: > >> Here''s my random scheduler. It works until I start a virtual machine >> (error file attached is the serial console''s log). >> In xen call trace there isn''t any of my functions. >> To generate random numbers I''m using the hash (MD5) of NOW() function. >> I would also Know why at boot time there are two calls at vcpu_init >> function for vcpu with id = 0 ?? >> >> Thanks >> Michele >> > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Did you look at timer.c, line 184, to find out what the BUG is? -George michele.paolino wrote:> Following George''s advices I have rewrited my scheduler. But now > serial console''s log report me a bug in a function of timer.c > > (XEN) **************************************** > (XEN) Panic on CPU 0: > (XEN) Xen BUG at timer.c:184 > (XEN) **************************************** > > This is caused by cpumask_raise_softirq() in my wake() function. What > can I do? > > Michele > > On Thu, Feb 25, 2010 at 10:25 PM, George Dunlap > <george.dunlap@eu.citrix.com <mailto:george.dunlap@eu.citrix.com>> wrote: > > The problem is that you''re choosing a random vcpu to run, without > considering whether it *wants* to run or not. In this case, > you''re running a vcpu before it''s even been completely set up yet > (write_cr3 is failing because the guest *has* no cr3 ready yet). > > The normal way schedulers deal with this is to keep one list of > all vcpus (or all domains), and another list with "runnable" > vcpus. You can keep track of which vcpus are runnable with the > vcpu_wake() callback and by using vcpu_runnable() in schedule(). > > At very least, your loop in schedule should check vcpu_runnable() > before selecting it. > > -George > > > michele.paolino wrote: > > Here''s my random scheduler. It works until I start a virtual > machine (error file attached is the serial console''s log). > In xen call trace there isn''t any of my functions. > To generate random numbers I''m using the hash (MD5) of NOW() > function. > I would also Know why at boot time there are two calls at > vcpu_init function for vcpu with id = 0 ?? > > Thanks > Michele > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Line 184 correspond at rc = 0 (an int variable) of remove_entry() function. I don''t have modified that file. Michele On Fri, Feb 26, 2010 at 6:44 PM, George Dunlap <george.dunlap@eu.citrix.com>wrote:> Did you look at timer.c, line 184, to find out what the BUG is? > -George > > michele.paolino wrote: > >> Following George''s advices I have rewrited my scheduler. But now serial >> console''s log report me a bug in a function of timer.c >> >> (XEN) **************************************** >> (XEN) Panic on CPU 0: >> (XEN) Xen BUG at timer.c:184 >> (XEN) **************************************** >> >> This is caused by cpumask_raise_softirq() in my wake() function. What can >> I do? >> >> Michele >> >> On Thu, Feb 25, 2010 at 10:25 PM, George Dunlap < >> george.dunlap@eu.citrix.com <mailto:george.dunlap@eu.citrix.com>> wrote: >> >> The problem is that you''re choosing a random vcpu to run, without >> considering whether it *wants* to run or not. In this case, >> you''re running a vcpu before it''s even been completely set up yet >> (write_cr3 is failing because the guest *has* no cr3 ready yet). >> >> The normal way schedulers deal with this is to keep one list of >> all vcpus (or all domains), and another list with "runnable" >> vcpus. You can keep track of which vcpus are runnable with the >> vcpu_wake() callback and by using vcpu_runnable() in schedule(). >> >> At very least, your loop in schedule should check vcpu_runnable() >> before selecting it. >> >> -George >> >> >> michele.paolino wrote: >> >> Here''s my random scheduler. It works until I start a virtual >> machine (error file attached is the serial console''s log). >> In xen call trace there isn''t any of my functions. >> To generate random numbers I''m using the hash (MD5) of NOW() >> function. >> I would also Know why at boot time there are two calls at >> vcpu_init function for vcpu with id = 0 ?? >> >> Thanks >> Michele >> >> >> >> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
If you want us to spend some of our time helping you, you need to demonstrate that you''ve tried to spend some time helping yourself. How might rc come to be 0 at that line? What is remove_entry() doing? How could this have come about because of something your code did? An example mail that would get more help would be something like this: "I''ve looked at the remove_entry() function, and it seems that the bug can be triggered if the input is {some condition}. I looked at where remove_entry() is being called, but I can''t figure out how it can get that condition because of my code." -George michele.paolino wrote:> Line 184 correspond at rc = 0 (an int variable) of remove_entry() > function. I don''t have modified that file. > > Michele > > > On Fri, Feb 26, 2010 at 6:44 PM, George Dunlap > <george.dunlap@eu.citrix.com <mailto:george.dunlap@eu.citrix.com>> wrote: > > Did you look at timer.c, line 184, to find out what the BUG is? > -George > > michele.paolino wrote: > > Following George''s advices I have rewrited my scheduler. But > now serial console''s log report me a bug in a function of timer.c > > (XEN) **************************************** > (XEN) Panic on CPU 0: > (XEN) Xen BUG at timer.c:184 > (XEN) **************************************** > > This is caused by cpumask_raise_softirq() in my wake() > function. What can I do? > > Michele > > On Thu, Feb 25, 2010 at 10:25 PM, George Dunlap > <george.dunlap@eu.citrix.com > <mailto:george.dunlap@eu.citrix.com> > <mailto:george.dunlap@eu.citrix.com > <mailto:george.dunlap@eu.citrix.com>>> wrote: > > The problem is that you''re choosing a random vcpu to run, > without > considering whether it *wants* to run or not. In this case, > you''re running a vcpu before it''s even been completely set > up yet > (write_cr3 is failing because the guest *has* no cr3 ready > yet). > > The normal way schedulers deal with this is to keep one list of > all vcpus (or all domains), and another list with "runnable" > vcpus. You can keep track of which vcpus are runnable with the > vcpu_wake() callback and by using vcpu_runnable() in > schedule(). > > At very least, your loop in schedule should check > vcpu_runnable() > before selecting it. > > -George > > > michele.paolino wrote: > > Here''s my random scheduler. It works until I start a > virtual > machine (error file attached is the serial console''s log). > In xen call trace there isn''t any of my functions. > To generate random numbers I''m using the hash (MD5) of > NOW() > function. > I would also Know why at boot time there are two calls at > vcpu_init function for vcpu with id = 0 ?? > > Thanks > Michele > > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I''m really sorry, maybe I misunderstood your mail. I don''t want anyone to work for me.:-) I looked at the functions listed in Xen call trace...I think my problem is the timer variable called periodic_timer of struct vcpu. In function remove_entry() there is a switch/case construct that check if vcpu->periodic_timer->status is initialized at TIMER_STATUS_in_heap or TIMER_STATUS_in_list. Which one should I use? Should Initialize it at one of timer status defined in timer.h file? Thanks, Michele On Mon, Mar 1, 2010 at 12:41 PM, George Dunlap <george.dunlap@eu.citrix.com>wrote:> If you want us to spend some of our time helping you, you need to > demonstrate that you''ve tried to spend some time helping yourself. How > might rc come to be 0 at that line? What is remove_entry() doing? How > could this have come about because of something your code did? > > An example mail that would get more help would be something like this: > > "I''ve looked at the remove_entry() function, and it seems that the bug can > be triggered if the input is {some condition}. I looked at where > remove_entry() is being called, but I can''t figure out how it can get that > condition because of my code." > > -George > > michele.paolino wrote: > >> Line 184 correspond at rc = 0 (an int variable) of remove_entry() >> function. I don''t have modified that file. >> >> Michele >> >> >> On Fri, Feb 26, 2010 at 6:44 PM, George Dunlap < >> george.dunlap@eu.citrix.com <mailto:george.dunlap@eu.citrix.com>> wrote: >> >> Did you look at timer.c, line 184, to find out what the BUG is? >> -George >> >> michele.paolino wrote: >> >> Following George''s advices I have rewrited my scheduler. But >> now serial console''s log report me a bug in a function of timer.c >> >> (XEN) **************************************** >> (XEN) Panic on CPU 0: >> (XEN) Xen BUG at timer.c:184 >> (XEN) **************************************** >> >> This is caused by cpumask_raise_softirq() in my wake() >> function. What can I do? >> >> Michele >> >> On Thu, Feb 25, 2010 at 10:25 PM, George Dunlap >> <george.dunlap@eu.citrix.com >> <mailto:george.dunlap@eu.citrix.com> >> <mailto:george.dunlap@eu.citrix.com >> <mailto:george.dunlap@eu.citrix.com>>> wrote: >> >> The problem is that you''re choosing a random vcpu to run, >> without >> considering whether it *wants* to run or not. In this case, >> you''re running a vcpu before it''s even been completely set >> up yet >> (write_cr3 is failing because the guest *has* no cr3 ready >> yet). >> >> The normal way schedulers deal with this is to keep one list of >> all vcpus (or all domains), and another list with "runnable" >> vcpus. You can keep track of which vcpus are runnable with the >> vcpu_wake() callback and by using vcpu_runnable() in >> schedule(). >> >> At very least, your loop in schedule should check >> vcpu_runnable() >> before selecting it. >> >> -George >> >> >> michele.paolino wrote: >> >> Here''s my random scheduler. It works until I start a >> virtual >> machine (error file attached is the serial console''s log). >> In xen call trace there isn''t any of my functions. >> To generate random numbers I''m using the hash (MD5) of >> NOW() >> function. >> I would also Know why at boot time there are two calls at >> vcpu_init function for vcpu with id = 0 ?? >> >> Thanks >> Michele >> >> >> >> >> >> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
michele.paolino wrote:> I''m really sorry, maybe I misunderstood your mail. I don''t want > anyone to work for me.:-)Whenever you ask a question, you''re asking someone to work for you, because it takes time and effort for someone to answer it. Even a short e-mail response may take 5 minutes of reading, understanding, thinking, and typing. To answer your question about the BUG() requires me to download the repository you''re using, look at and trace around code I''ve never looked at before, and possibly to read and understand your code. That''s probably at least a half-hour of my time. And that''s OK. I like helping people, I like solving problems. It''s also an investment: if you become a good OS coder, the computer industry as a whole (and myself included in that) benefit; and if you become better at Xen, I benefit more directly. However, I don''t want to waste my time. How you can demonstrate to me that you are committed to learning this, and that you are capable of becoming an OS coder, is by demonstrating that you''ve spent effort and thought on something before answering a question. I''ll take a closer look and try to answer your question either later today or tomorrow. -George _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
thanks, I have solved the timer.c problem right now. I am able to start dom0. Michele On 3/2/10, George Dunlap <george.dunlap@eu.citrix.com> wrote:> michele.paolino wrote: >> I''m really sorry, maybe I misunderstood your mail. I don''t want >> anyone to work for me.:-) > Whenever you ask a question, you''re asking someone to work for you, > because it takes time and effort for someone to answer it. Even a short > e-mail response may take 5 minutes of reading, understanding, thinking, > and typing. To answer your question about the BUG() requires me to > download the repository you''re using, look at and trace around code I''ve > never looked at before, and possibly to read and understand your code. > That''s probably at least a half-hour of my time. > > And that''s OK. I like helping people, I like solving problems. It''s > also an investment: if you become a good OS coder, the computer industry > as a whole (and myself included in that) benefit; and if you become > better at Xen, I benefit more directly. > > However, I don''t want to waste my time. How you can demonstrate to me > that you are committed to learning this, and that you are capable of > becoming an OS coder, is by demonstrating that you''ve spent effort and > thought on something before answering a question. > > I''ll take a closer look and try to answer your question either later > today or tomorrow. > > -George >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel