Linux Virtualization - Apr 2007 - pv_ops smp support

Jeremy Fitzhardinge wrote:
> I'm looking at adding Xen SMP support, so I'm trying to work out
what
> pv_ops we need, and how to cut into the existing smp stuff.
>
> So at first pass, it looks like we need 6-8 new pv_ops for SMP, which 
> isn't very appealing.
>
> Any thoughts about how to come up with a more elegant interface?  I'm 
> digging through smpboot.c and friends, but there's a lot of goo in 
> there...
Most of the goo is apic related goo, which can be conveniently be tucked 
out of the way by just ignoring or putting in appropriate hooks.  Before 
we try to dissect this into operations, I think there are two bigger 
fundamental questions you need to answer -

1) What do you plan to do to address per-cpu data structures?
2) What is your remote TLB shootdown model?

Both of these could have major impacts on how you want to carve up smp.c 
and smpboot.c.

Zach

I'm looking at adding Xen SMP support, so I'm trying to work out what 
pv_ops we need, and how to cut into the existing smp stuff.

smpboot.c has a mixture of stuff which is generally useful for SMP stuff 
(the various CPU sets, and presumably the sibling relationships are 
useful in principle), but also a whole pile of APIC stuff which is 
irrelevent to Xen.  It has these exported symbols, with my first pass 
comments:

    00000644 T __cpu_die			-- need pv_op
    000008a5 T __cpu_disable		-- need pv_op
    000006aa T __cpu_up			-- need pv_op
    00000000 T cpu_coregroup_map		-- ? ignore
    00000868 T cpu_exit_clear		-- ?
    00000d9f T initialize_secondary		-- Xen no-op
    000008ee T smp_alloc_memory		-- Xen no-op (doesn't matter if it gets
called)
    00000057 T smp_cpus_done		-- unwanted for Xen
    00000015 T smp_intr_init		-- need something to set up IPIs, but APIC
independent
    0000082a T smp_prepare_boot_cpu		-- looks OK for Xen, I think
    00000066 T smp_prepare_cpus		-- need pv_op
      


So at first pass, it looks like we need 6-8 new pv_ops for SMP, which 
isn't very appealing.

Any thoughts about how to come up with a more elegant interface?  I'm 
digging through smpboot.c and friends, but there's a lot of goo in there...

It's not clear to me what we should do with all the topology stuff.  
Clearly a VCPU won't have any fixed physical relationship with other 
VCPUs unless they're pinned; I'm wondering if the existing code will 
confuse itself if it gets basically random topology info from cpuid.

    J