[kernel-xen] Maximum number of CPUs set into 16 (CONFIG_NR_CPUS=16) ?

[Yuriy Kohut]

Hi, Chris

Thank for your reply.

So, I'm OK with 16 cores for Dom0. But could you please confirm more the 16 could be assigned to VMs ?

Thank you
---
Yura

On Jan 24, 2013, at 10:48 AM, Chris Wik <chris at anu.net> wrote:

> We have some machines now with 32 cores running Steven's kernel-xen. We allocate 4 cores to dom0, eg. in /etc/grub.conf:
> 
> kernel /xen.gz dom0_mem=1G dom0_max_vcpus=4 dom0_vcpus_pin cpufreq=xen
> 
> We have VMs with 16 cores running fine, have not tested any higher numbers...
> 
> Chris
> 
> On 24 Jan 13, at 08:59, Yuriy Kohut <ykohut at onapp.com> wrote:
> 
>> Hi, Steven
>> 
>> I faced with the issue yesterday. HV has 2 following processors:
>> 
>> processor       : 0
>> vendor_id       : GenuineIntel
>> cpu family      : 6
>> model           : 45
>> model name      : Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
>> stepping        : 7
>> microcode       : 0x70b
>> cpu MHz         : 2000.052
>> cache size      : 15360 KB
>> physical id     : 0
>> siblings        : 12
>> core id         : 0
>> cpu cores       : 6
>> apicid          : 0
>> initial apicid  : 0
>> fpu             : yes
>> fpu_exception   : yes
>> cpuid level     : 13
>> wp              : yes
>> flags           : fpu de tsc msr pae mce cx8 apic sep mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl nonstop_tsc eagerfpu pni pclmulqdq est ssse3 cx16 sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm ida arat epb xsaveopt pln pts dtherm
>> bogomips        : 4000.10
>> clflush size    : 64
>> cache_alignment : 64
>> address sizes   : 46 bits physical, 48 bits virtual
>> power management:
>> 
>> This is 6 core (12 threads) CPUs.
>> 
>> So we should get 12 cores without threading and 24 with threading.
>> 
>> Here is what we get:
>> [root at CC-HVXS-2 ~]# grep processor /proc/cpuinfo 
>> processor    : 0
>> processor    : 1
>> processor    : 2
>> processor    : 3
>> processor    : 4
>> processor    : 5
>> processor    : 6
>> processor    : 7
>> processor    : 8
>> processor    : 9
>> processor    : 10
>> processor    : 11
>> processor    : 12
>> processor    : 13
>> processor    : 14
>> processor    : 15
>> 
>> This result is greater then 12 - so it is with threading. As the result Dom0 kernel will use only 16 threaded kernels out of 24.
>> The other thing is, I'm NOT 100% sure that all vCPUs (not only 16) will be available for DomUs.
>> 
>> And the last thing is, that I don't understand why 16 ? 
>> While kernel.org sets as default 64, and RedHat 5.x default is 255...
>> 
>> So, is there any chance to get the number updated in upcoming kernel-xen RPM(s) ?
>> 
>> ---
>> Yura
>> 
>> On Jan 24, 2013, at 5:19 AM, Steven Haigh <netwiz at crc.id.au> wrote:
>> 
>>> On 24/01/2013 2:16 AM, Yuriy Kohut wrote:
>>>> Hi,
>>>> 
>>>> What is the reason to set the maximum number of CPUs into 16 ?
>>>> 
>>>> The latest kernel-xen-3.7.4-1.el6xen.src.rpm package includes the config
>>>> with:
>>>> # grep -r CONFIG_NR_CPUS rpmbuild
>>>> rpmbuild/SOURCES/config-3.7.x86_64:CONFIG_NR_CPUS=16
>>>> 
>>>> While there is 64 CPU allowed in the default x86_64 config on kernel.org
>>>> <http://kernel.org>):
>>>> 
>>>> http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=blob_plain;f=arch/x86/configs/x86_64_defconfig;hb=8a69ca20f06d1878aa0ee8000a60d8226ab6b6a1
>>> 
>>> I don't think anyone has hit the 16 CPU limit - nor would anyone be likely to hit it...
>>> 
>>> Keep in mind that the number here is ONLY for the number of VCPUs that Dom0 has. The hardware may have 64 CPU cores - but only 1-2 allocated for Dom0. I really doubt any sane configuration would have more than 16 VCPUs set for Dom0.
>>> 
>>> -- 
>>> Steven Haigh
>>> 
>>> Email: netwiz at crc.id.au
>>> Web: http://www.crc.id.au
>>> Phone: (03) 9001 6090 - 0412 935 897
>>> Fax: (03) 8338 0299
>>> 
>>> _______________________________________________
>>> kernel-xen mailing list
>>> kernel-xen at lists.wireless.org.au
>>> https://lists.wireless.org.au/mailman/listinfo/kernel-xen
>> 
>> _______________________________________________
>> kernel-xen mailing list
>> kernel-xen at lists.wireless.org.au
>> https://lists.wireless.org.au/mailman/listinfo/kernel-xen
> 
> --
> Chris Wik
> Anu Internet Services Ltd
> www.cwik.ch | www.anu.net
> 
> _______________________________________________
> kernel-xen mailing list
> kernel-xen at lists.wireless.org.au
> https://lists.wireless.org.au/mailman/listinfo/kernel-xen