Hi all, I am having confusion regarding the way a newly created domain is allocated vcpu. Initially during dom0 creation alloc_vcpu is called to create vcpu structs for all the available cpu''s and assigned to dom0. But its not the case for domU creation. 1. So how will dom0 relinquish/share vcpu to/with a newly created domU. Does this happen as part of the shared_info page mapping?? 2. Till the end of DOMCTL_createdomain hypercall d->vcpu is null but by the start of xc_linux_bulid_internal function d->vcpu is pointing to some vcpu. How is this?? 3. As part of the domU config file we can constraint the domU to a specific CPU. How is this achieved unless there is an explict way of assigning vcpu associated with that cpu number. I apologise if the questions seem vague and silly. Any kind of hints or pointers would be very helpful. Thanks in advance Regards, Sandesh Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Grzegorz Miłoś
2008-Jun-18 11:13 UTC
Re: [Xen-devel] Vcpu allocation for a newly created domU
> Hi all, > > I am having confusion regarding the way a newly created domain is > allocated vcpu. > Initially during dom0 creation alloc_vcpu is called to create vcpu > structs for all the available cpu''s and assigned to dom0. But its not > the case for domU creation. > 1. So how will dom0 relinquish/share vcpu to/with a newly created domU. > Does this happen as part of the shared_info page mapping??vcpus are created for each new domain, they are not shared with Dom0.> 2. Till the end of DOMCTL_createdomain hypercall d->vcpu is null but by > the start of xc_linux_bulid_internal function d->vcpu is pointing to > some vcpu. How is this??vcpus are created as a part of DOMCTL_max_vcpus. This hypercall is issued by the tools in Dom0, specifically by: _constructDomain() in python/xen/xend/XendDomainInfo.py. Look for the following line: xc.domain_max_vcpus(self.domid, int(self.info[''VCPUs_max'']))> 3. As part of the domU config file we can constraint the domU to a > specific CPU. How is this achieved unless there is an explict way of > assigning vcpu associated with that cpu number.This is done by the following call chain: DOMCTL_setvcpuaffinity -> vcpu_set_affinity -> __vcpu_set_affinity Hope this helps. cheers Gr(z)egor(z) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, 2008-06-18 at 12:13 +0100, Grzegorz Miłoś wrote:> > Hi all, > > > > I am having confusion regarding the way a newly created domain is > > allocated vcpu. > > Initially during dom0 creation alloc_vcpu is called to create vcpu > > structs for all the available cpu''s and assigned to dom0. But its not > > the case for domU creation. > > 1. So how will dom0 relinquish/share vcpu to/with a newly created domU. > > Does this happen as part of the shared_info page mapping?? > > vcpus are created for each new domain, they are not shared with Dom0.in xen/arch/x86/setup.c right after domain_create(0,0, DOM0_SSIDREF) which creates dom0, alloc_vcpu(dom0..) is called which will allocate vcpu for dom0. Then in construct_dom0 alloc_vcpu is called for dom0_max_cpus. So aren''t all vcpu''s getting allocated to dom0?? Or is it that domU will get allocated separate vcpu''s, irrespective of these above vcpu''s??> > > 2. Till the end of DOMCTL_createdomain hypercall d->vcpu is null but by > > the start of xc_linux_bulid_internal function d->vcpu is pointing to > > some vcpu. How is this?? > > vcpus are created as a part of DOMCTL_max_vcpus. This hypercall is > issued by the tools in Dom0, specifically by: _constructDomain() in > python/xen/xend/XendDomainInfo.py. Look for the following line: > xc.domain_max_vcpus(self.domid, int(self.info[''VCPUs_max''])) >> > 3. As part of the domU config file we can constraint the domU to a > > specific CPU. How is this achieved unless there is an explict way of > > assigning vcpu associated with that cpu number. > > This is done by the following call chain: > DOMCTL_setvcpuaffinity -> > vcpu_set_affinity -> > __vcpu_set_affinity >Ohh ya!! i had completely missed the above two part...now things are much clear.> Hope this helps.Thanks for the info, it certainly cleared some amount of confusion.> cheers > Gr(z)egor(z)Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Grzegorz Miłoś
2008-Jun-18 16:09 UTC
Re: [Xen-devel] Vcpu allocation for a newly created domU
>> > 1. So how will dom0 relinquish/share vcpu to/with a newly created domU. >> > Does this happen as part of the shared_info page mapping?? >> >> vcpus are created for each new domain, they are not shared with Dom0. > in xen/arch/x86/setup.c right after domain_create(0,0, DOM0_SSIDREF) > which creates dom0, alloc_vcpu(dom0..) is called which will allocate > vcpu for dom0. Then in construct_dom0 alloc_vcpu is called for > dom0_max_cpus. So aren''t all vcpu''s getting allocated to dom0?? > > Or is it that domU will get allocated separate vcpu''s, irrespective of > these above vcpu''s??Each domain, Dom0 and any DomU, will get separate VCPU(s). These are dynamically bound/run on physical CPUs. Cheers Gr(z)egor(z) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, 2008-06-18 at 17:09 +0100, Grzegorz Miłoś wrote:> >> > 1. So how will dom0 relinquish/share vcpu to/with a newly created domU. > >> > Does this happen as part of the shared_info page mapping?? > >> > >> vcpus are created for each new domain, they are not shared with Dom0. > > in xen/arch/x86/setup.c right after domain_create(0,0, DOM0_SSIDREF) > > which creates dom0, alloc_vcpu(dom0..) is called which will allocate > > vcpu for dom0. Then in construct_dom0 alloc_vcpu is called for > > dom0_max_cpus. So aren''t all vcpu''s getting allocated to dom0?? > > > > Or is it that domU will get allocated separate vcpu''s, irrespective of > > these above vcpu''s?? > > Each domain, Dom0 and any DomU, will get separate VCPU(s). These are > dynamically bound/run on physical CPUs.Ok, thanks for the clarifications.> > Cheers > Gr(z)egor(z)Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, 2008-06-18 at 17:09 +0100, Grzegorz Miłoś wrote:> > Or is it that domU will get allocated separate vcpu''s, irrespective of > > these above vcpu''s?? > > Each domain, Dom0 and any DomU, will get separate VCPU(s). These are > dynamically bound/run on physical CPUs.This last statement has aroused another doubt...so iam deviating a bit from the main subject. As far as binding VCPUs to CPUs is concerned, when i looked at the scheduler code, after chosing the next VCPU to be scheduled, during context_switch only the curret_cpu on which the scheuler is running is selected. If we look at xen/arch/x86/domain.c: __context_switch() chooses cpu using smp_processor_id(). Also during switch_kernel_stack() the tss struct is assigned to init_tss indexed by smp_processor_id() which represents the current cpu. I was assuming the ''processor'' field of the vcpu defines the binding to a particular CPU. But it seems its not the case. Then whats the use of processor field?? Please correct me if my understanding is wrong. Thanks, Sandesh> > Cheers > Gr(z)egor(z)Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Grzegorz Miłoś
2008-Jun-19 14:30 UTC
Re: [Xen-devel] Vcpu allocation for a newly created domU
> > > Or is it that domU will get allocated separate vcpu''s, irrespective of > > > these above vcpu''s?? > > > > Each domain, Dom0 and any DomU, will get separate VCPU(s). These are > > dynamically bound/run on physical CPUs. > > This last statement has aroused another doubt...so iam deviating a bit > from the main subject. > > As far as binding VCPUs to CPUs is concerned, when i looked at the > scheduler code, after chosing the next VCPU to be scheduled, during > context_switch only the curret_cpu on which the scheuler is running is > selected. > If we look at xen/arch/x86/domain.c: __context_switch() chooses cpu > using smp_processor_id(). Also during switch_kernel_stack() the tss > struct is assigned to init_tss indexed by smp_processor_id() which > represents the current cpu. > I was assuming the ''processor'' field of the vcpu defines the binding to > a particular CPU. But it seems its not the case. Then whats the use of > processor field??processor field defines on which physical CPU was the vcpu run the last time. A vcpu can be migrated to a different CPU (look: vcpu_migrate() in common/schedule.c). smp_processor_id() and current->processor should match in __context_switch(). Indeed both smp_processor_id() and current rely on the same cpu_info structure. Cheers Gr(z)egor(z)>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> >> smp_processor_id() and current->processor should match in > >> __context_switch(). > > need not be..cant the scheduler schedule the vcpu on a cpu other than > > the cpu its currently running on...?? > > When you enter the scheduler (and also __context_switch) you are > running VCPU=current on CPU=smp_processor_id().this is true only if current is not set to the next vcpu. but if you look at context_switch() just before calling __context_switch , set_curent(next) is done. At this point current is actually the next vcpu to be scheduled.> At this point in time > current->processor and smp_processor_id() will match.So vcpu->processor need not match smp_processor_id().... Thanks, Sandesh Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Grzegorz Miłoś
2008-Jun-20 09:41 UTC
Re: [Xen-devel] Vcpu allocation for a newly created domU
>> When you enter the scheduler (and also __context_switch) you are >> running VCPU=current on CPU=smp_processor_id(). > > this is true only if current is not set to the next vcpu. but if you > look at context_switch() just before calling __context_switch , > set_curent(next) is done. At this point current is actually the next > vcpu to be scheduled. >First of all: I only said that current->proccessor =smp_processor_id() will hold on scheduler entry. But this equality is actually stronger, I believe it will hold always.> >> At this point in time >> current->processor and smp_processor_id() will match. > > So vcpu->processor need not match smp_processor_id()....Except that the next vcpu (or current, after set_current has been executed) will also have ->processor filed set to smp_processor_id(). You have to explicitly migrate VCPUs between CPUs (and set their ->processor field in the process) to make them runnable on a different CPU. For example the credit scheduler does that in csched_runq_steal(). If it deems a VCPU to be migratable (__csched_vcpu_is_migrateable) it will remove it from it''s current runqueue (__runq_remove) and update the processor field (vc->processor = cpu) Is that clear now? Gr(z)egor(z)> > Thanks, > Sandesh > > > Please do not print this email unless it is absolutely necessary. > > The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. > > WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. > > www.wipro.com > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel