Hello, I am working on a project using the VCPU hotplug feature in Xen, and I think that there is a problem with how Linux is processing CPU hotplug events. The problem I have seen is that the /proc/stat file does not change the number of CPU entries in response to the: xm vcpu-set <domain> <num> command. However, the /proc/cpuinfo file is updated correctly. The reason this is important is that some applications use the sysconf(_SC_NPROCESSORS_ONLN) to determine how many threads to run. In Linux this function parses the /proc/stat file to determine how many processors are available. However, the /proc/stat file is not being updated correctly in response to the xm vcpu-set command. It would be a big help for me if anyone else could test to see if they experience the same problem. I''m not sure if this would qualify as a problem with Linux or with Xen; however, any ideas for fixing it are appreciated as well. Thanks, -Gabriel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Gabriel Southern wrote:> Hello, > > I am working on a project using the VCPU hotplug feature in Xen, and I > think that there is a problem with how Linux is processing CPU hotplug > events. The problem I have seen is that the /proc/stat file does not > change the number of CPU entries in response to the: xm vcpu-set > <domain> <num> command. However, the /proc/cpuinfo file is updated > correctly. > > The reason this is important is that some applications use the > sysconf(_SC_NPROCESSORS_ONLN) to determine how many threads to run. > In Linux this function parses the /proc/stat file to determine how > many processors are available. However, the /proc/stat file is not > being updated correctly in response to the xm vcpu-set command. > > It would be a big help for me if anyone else could test to see if they > experience the same problem. I''m not sure if this would qualify as a > problem with Linux or with Xen; however, any ideas for fixing it are > appreciated as well.Hm, it would be helpful if you told us what version of Xen you are running and which kernel you are running. For what it is worth, running a RHEL-5.1 PV guest on a RHEL-5.1 dom0 (both x86_64) seems to do the right thing for me; xm vcpu-set <dom> 1 updates both /proc/cpuinfo and /proc/stat, and xm vcpu-set <dom> 2 updates both again. Chris Lalancette _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Thanks for the reply. I had Ubuntu 7.10 x86_64 installed, and I compiled the Xen 3.2 source that I downloaded from xen.org. So my kernel is 2.6.18.8-xen x86_64 and I am running Xen 3.2. I had the same problem with Xen 3.1. Which version of the Linux kernel does RHEL-5.1 use? I''m curious to know if the problem I am seeing is a result of something I did wrong in compiling Xen, or if it is a kernel bug that has been fixed in a later release of Linux. -Gabriel On Feb 11, 2008 10:42 PM, Chris Lalancette <clalance@redhat.com> wrote:> > Hm, it would be helpful if you told us what version of Xen you are > running and which kernel you are running. For what it is worth, running > a RHEL-5.1 PV guest on a RHEL-5.1 dom0 (both x86_64) seems to do the > right thing for me; xm vcpu-set <dom> 1 updates both /proc/cpuinfo and > /proc/stat, and xm vcpu-set <dom> 2 updates both again. > > Chris Lalancette >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>> It would be a big help for me if anyone else could test to see if they >> experience the same problem. I''m not sure if this would qualify as a >> problem with Linux or with Xen; however, any ideas for fixing it are >> appreciated as well. > > Hm, it would be helpful if you told us what version of Xen you are > running and which kernel you are running. For what it is worth, running > a RHEL-5.1 PV guest on a RHEL-5.1 dom0 (both x86_64) seems to do the > right thing for me; xm vcpu-set <dom> 1 updates both /proc/cpuinfo and > /proc/stat, and xm vcpu-set <dom> 2 updates both again.The issue is the result of a deliberate change in some kernels (including our 2.6.18-xen tree) in which fs/proc/proc_misc.c:show_stat() iterates over each_possible_cpu() rather than each_online_cpu(). This of course means it prints info for all offline cpus as well as online cpus. The original reason for this is that some user-space tools, such as older versions of top, would get quite upset when CPUs are hot-unplugged while they run. This would cause top to crash! I don''t know whether this was fixed by now in the user-space packages. The issue was originally seen years ago so I suppose it''s fairly likely it has been, and the hack could be removed. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Thanks for the explanation. When I started looking at the problem, I had not previously looked at the Xen source code, and I made the mistake of looking at the Linux 2.6.18 kernel from kernel.org, without the xen patches applied. So when I was tracing through the code everything looked fine because it had the for_each_online_cpu() macro in place. After reading your message, I checked the code that I was actually compiling and it had the change you described. Changing from for_each_possible_cpu() back to for_each_online_cpu() fixed the problem for me. Incidentially in a Ubuntu 7.10 VM, after this change is made, top will still crash if a CPU is hot-unplugged while top is running. However, in centos-5.1 the hot-unplug events are handled differently and top does not crash. Thanks again for the help. -Gabriel> > The issue is the result of a deliberate change in some kernels (including > our 2.6.18-xen tree) in which fs/proc/proc_misc.c:show_stat() iterates over > each_possible_cpu() rather than each_online_cpu(). This of course means it > prints info for all offline cpus as well as online cpus. > > The original reason for this is that some user-space tools, such as older > versions of top, would get quite upset when CPUs are hot-unplugged while > they run. This would cause top to crash! I don''t know whether this was fixed > by now in the user-space packages. The issue was originally seen years ago > so I suppose it''s fairly likely it has been, and the hack could be removed. > > -- Keir >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel