[kernel-xen] Maximum number of CPUs set into 16 (CONFIG_NR_CPUS=16) ?

[Chris Wik]

We have some machines now with 32 cores running Steven's kernel-xen. We allocate 4 cores to dom0, eg. in /etc/grub.conf:

kernel /xen.gz dom0_mem=1G dom0_max_vcpus=4 dom0_vcpus_pin cpufreq=xen

We have VMs with 16 cores running fine, have not tested any higher numbers...

Chris

On 24 Jan 13, at 08:59, Yuriy Kohut <ykohut at onapp.com> wrote:

> Hi, Steven
> 
> I faced with the issue yesterday. HV has 2 following processors:
> 
> processor       : 0
> vendor_id       : GenuineIntel
> cpu family      : 6
> model           : 45
> model name      : Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
> stepping        : 7
> microcode       : 0x70b
> cpu MHz         : 2000.052
> cache size      : 15360 KB
> physical id     : 0
> siblings        : 12
> core id         : 0
> cpu cores       : 6
> apicid          : 0
> initial apicid  : 0
> fpu             : yes
> fpu_exception   : yes
> cpuid level     : 13
> wp              : yes
> flags           : fpu de tsc msr pae mce cx8 apic sep mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl nonstop_tsc eagerfpu pni pclmulqdq est ssse3 cx16 sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm ida arat epb xsaveopt pln pts dtherm
> bogomips        : 4000.10
> clflush size    : 64
> cache_alignment : 64
> address sizes   : 46 bits physical, 48 bits virtual
> power management:
> 
> This is 6 core (12 threads) CPUs.
> 
> So we should get 12 cores without threading and 24 with threading.
> 
> Here is what we get:
> [root at CC-HVXS-2 ~]# grep processor /proc/cpuinfo 
> processor    : 0
> processor    : 1
> processor    : 2
> processor    : 3
> processor    : 4
> processor    : 5
> processor    : 6
> processor    : 7
> processor    : 8
> processor    : 9
> processor    : 10
> processor    : 11
> processor    : 12
> processor    : 13
> processor    : 14
> processor    : 15
> 
> This result is greater then 12 - so it is with threading. As the result Dom0 kernel will use only 16 threaded kernels out of 24.
> The other thing is, I'm NOT 100% sure that all vCPUs (not only 16) will be available for DomUs.
> 
> And the last thing is, that I don't understand why 16 ? 
> While kernel.org sets as default 64, and RedHat 5.x default is 255...
> 
> So, is there any chance to get the number updated in upcoming kernel-xen RPM(s) ?
> 
> ---
> Yura
> 
> On Jan 24, 2013, at 5:19 AM, Steven Haigh <netwiz at crc.id.au> wrote:
> 
>> On 24/01/2013 2:16 AM, Yuriy Kohut wrote:
>>> Hi,
>>> 
>>> What is the reason to set the maximum number of CPUs into 16 ?
>>> 
>>> The latest kernel-xen-3.7.4-1.el6xen.src.rpm package includes the config
>>> with:
>>> # grep -r CONFIG_NR_CPUS rpmbuild
>>> rpmbuild/SOURCES/config-3.7.x86_64:CONFIG_NR_CPUS=16
>>> 
>>> While there is 64 CPU allowed in the default x86_64 config on kernel.org
>>> <http://kernel.org>):
>>> 
>>> http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=blob_plain;f=arch/x86/configs/x86_64_defconfig;hb=8a69ca20f06d1878aa0ee8000a60d8226ab6b6a1
>> 
>> I don't think anyone has hit the 16 CPU limit - nor would anyone be likely to hit it...
>> 
>> Keep in mind that the number here is ONLY for the number of VCPUs that Dom0 has. The hardware may have 64 CPU cores - but only 1-2 allocated for Dom0. I really doubt any sane configuration would have more than 16 VCPUs set for Dom0.
>> 
>> -- 
>> Steven Haigh
>> 
>> Email: netwiz at crc.id.au
>> Web: http://www.crc.id.au
>> Phone: (03) 9001 6090 - 0412 935 897
>> Fax: (03) 8338 0299
>> 
>> _______________________________________________
>> kernel-xen mailing list
>> kernel-xen at lists.wireless.org.au
>> https://lists.wireless.org.au/mailman/listinfo/kernel-xen
> 
> _______________________________________________
> kernel-xen mailing list
> kernel-xen at lists.wireless.org.au
> https://lists.wireless.org.au/mailman/listinfo/kernel-xen

--
Chris Wik
Anu Internet Services Ltd
www.cwik.ch | www.anu.net