Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

Merge tag 'x86-apic-2024-03-10' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull x86 APIC updates from Thomas Gleixner:
"Rework of APIC enumeration and topology evaluation.

The current implementation has a couple of shortcomings:

- It fails to handle hybrid systems correctly.

- The APIC registration code which handles CPU number assignents is
in the middle of the APIC code and detached from the topology
evaluation.

- The various mechanisms which enumerate APICs, ACPI, MPPARSE and
guest specific ones, tweak global variables as they see fit or in
case of XENPV just hack around the generic mechanisms completely.

- The CPUID topology evaluation code is sprinkled all over the vendor
code and reevaluates global variables on every hotplug operation.

- There is no way to analyze topology on the boot CPU before bringing
up the APs. This causes problems for infrastructure like PERF which
needs to size certain aspects upfront or could be simplified if
that would be possible.

- The APIC admission and CPU number association logic is
incomprehensible and overly complex and needs to be kept around
after boot instead of completing this right after the APIC
enumeration.

This update addresses these shortcomings with the following changes:

- Rework the CPUID evaluation code so it is common for all vendors
and provides information about the APIC ID segments in a uniform
way independent of the number of segments (Thread, Core, Module,
..., Die, Package) so that this information can be computed instead
of rewriting global variables of dubious value over and over.

- A few cleanups and simplifcations of the APIC, IO/APIC and related
interfaces to prepare for the topology evaluation changes.

- Seperation of the parser stages so the early evaluation which tries
to find the APIC address can be seperately overridden from the late
evaluation which enumerates and registers the local APIC as further
preparation for sanitizing the topology evaluation.

- A new registration and admission logic which

- encapsulates the inner workings so that parsers and guest logic
cannot longer fiddle in it

- uses the APIC ID segments to build topology bitmaps at
registration time

- provides a sane admission logic

- allows to detect the crash kernel case, where CPU0 does not run
on the real BSP, automatically. This is required to prevent
sending INIT/SIPI sequences to the real BSP which would reset
the whole machine. This was so far handled by a tedious command
line parameter, which does not even work in nested crash
scenarios.

- Associates CPU number after the enumeration completed and
prevents the late registration of APICs, which was somehow
tolerated before.

- Converting all parsers and guest enumeration mechanisms over to the
new interfaces.

This allows to get rid of all global variable tweaking from the
parsers and enumeration mechanisms and sanitizes the XEN[PV]
handling so it can use CPUID evaluation for the first time.

- Mopping up existing sins by taking the information from the APIC ID
segment bitmaps.

This evaluates hybrid systems correctly on the boot CPU and allows
for cleanups and fixes in the related drivers, e.g. PERF.

The series has been extensively tested and the minimal late fallout
due to a broken ACPI/MADT table has been addressed by tightening the
admission logic further"

* tag 'x86-apic-2024-03-10' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (76 commits)
x86/topology: Ignore non-present APIC IDs in a present package
x86/apic: Build the x86 topology enumeration functions on UP APIC builds too
smp: Provide 'setup_max_cpus' definition on UP too
smp: Avoid 'setup_max_cpus' namespace collision/shadowing
x86/bugs: Use fixed addressing for VERW operand
x86/cpu/topology: Get rid of cpuinfo::x86_max_cores
x86/cpu/topology: Provide __num_[cores|threads]_per_package
x86/cpu/topology: Rename topology_max_die_per_package()
x86/cpu/topology: Rename smp_num_siblings
x86/cpu/topology: Retrieve cores per package from topology bitmaps
x86/cpu/topology: Use topology logical mapping mechanism
x86/cpu/topology: Provide logical pkg/die mapping
x86/cpu/topology: Simplify cpu_mark_primary_thread()
x86/cpu/topology: Mop up primary thread mask handling
x86/cpu/topology: Use topology bitmaps for sizing
x86/cpu/topology: Let XEN/PV use topology from CPUID/MADT
x86/xen/smp_pv: Count number of vCPUs early
x86/cpu/topology: Assign hotpluggable CPUIDs during init
x86/cpu/topology: Reject unknown APIC IDs on ACPI hotplug
x86/topology: Add a mechanism to track topology via APIC IDs
...

+1566 -1580
+2 -5
Documentation/admin-guide/kdump/kdump.rst
··· 191 191 CPU is enough for kdump kernel to dump vmcore on most of systems. 192 192 193 193 However, you can also specify nr_cpus=X to enable multiple processors 194 - in kdump kernel. In this case, "disable_cpu_apicid=" is needed to 195 - tell kdump kernel which cpu is 1st kernel's BSP. Please refer to 196 - admin-guide/kernel-parameters.txt for more details. 194 + in kdump kernel. 197 195 198 196 With CONFIG_SMP=n, the above things are not related. 199 197 ··· 452 454 to use multi-thread programs with it, such as parallel dump feature of 453 455 makedumpfile. Otherwise, the multi-thread program may have a great 454 456 performance degradation. To enable multi-cpu support, you should bring up an 455 - SMP dump-capture kernel and specify maxcpus/nr_cpus, disable_cpu_apicid=[X] 456 - options while loading it. 457 + SMP dump-capture kernel and specify maxcpus/nr_cpus options while loading it. 457 458 458 459 * For s390x there are two kdump modes: If a ELF header is specified with 459 460 the elfcorehdr= kernel parameter, it is used by the kdump kernel as it
-9
Documentation/admin-guide/kernel-parameters.txt
··· 1095 1095 Disable TLBIE instruction. Currently does not work 1096 1096 with KVM, with HASH MMU, or with coherent accelerators. 1097 1097 1098 - disable_cpu_apicid= [X86,APIC,SMP] 1099 - Format: <int> 1100 - The number of initial APIC ID for the 1101 - corresponding CPU to be disabled at boot, 1102 - mostly used for the kdump 2nd kernel to 1103 - disable BSP to wake up multiple CPUs without 1104 - causing system reset or hang due to sending 1105 - INIT from AP to BSP. 1106 - 1107 1098 disable_ddw [PPC/PSERIES,EARLY] 1108 1099 Disable Dynamic DMA Window support. Use this 1109 1100 to workaround buggy firmware.
+9 -15
Documentation/arch/x86/topology.rst
··· 47 47 48 48 Package-related topology information in the kernel: 49 49 50 - - cpuinfo_x86.x86_max_cores: 50 + - topology_num_threads_per_package() 51 51 52 - The number of cores in a package. This information is retrieved via CPUID. 52 + The number of threads in a package. 53 53 54 - - cpuinfo_x86.x86_max_dies: 54 + - topology_num_cores_per_package() 55 55 56 - The number of dies in a package. This information is retrieved via CPUID. 56 + The number of cores in a package. 57 + 58 + - topology_max_dies_per_package() 59 + 60 + The maximum number of dies in a package. 57 61 58 62 - cpuinfo_x86.topo.die_id: 59 63 60 - The physical ID of the die. This information is retrieved via CPUID. 64 + The physical ID of the die. 61 65 62 66 - cpuinfo_x86.topo.pkg_id: 63 67 ··· 99 95 100 96 AMDs nomenclature for a CMT core is "Compute Unit". The kernel always uses 101 97 "core". 102 - 103 - Core-related topology information in the kernel: 104 - 105 - - smp_num_siblings: 106 - 107 - The number of threads in a core. The number of threads in a package can be 108 - calculated by:: 109 - 110 - threads_per_package = cpuinfo_x86.x86_max_cores * smp_num_siblings 111 - 112 98 113 99 Threads 114 100 =======
+1 -1
arch/x86/events/amd/core.c
··· 579 579 if (!x86_pmu.amd_nb_constraints) 580 580 return; 581 581 582 - nb_id = topology_die_id(cpu); 582 + nb_id = topology_amd_node_id(cpu); 583 583 WARN_ON_ONCE(nb_id == BAD_APICID); 584 584 585 585 for_each_online_cpu(i) {
+1 -1
arch/x86/events/intel/cstate.c
··· 834 834 } 835 835 836 836 if (has_cstate_pkg) { 837 - if (topology_max_die_per_package() > 1) { 837 + if (topology_max_dies_per_package() > 1) { 838 838 err = perf_pmu_register(&cstate_pkg_pmu, 839 839 "cstate_die", -1); 840 840 } else {
+1 -1
arch/x86/events/intel/uncore.c
··· 1893 1893 return -ENODEV; 1894 1894 1895 1895 __uncore_max_dies = 1896 - topology_max_packages() * topology_max_die_per_package(); 1896 + topology_max_packages() * topology_max_dies_per_package(); 1897 1897 1898 1898 id = x86_match_cpu(intel_uncore_match); 1899 1899 if (!id) {
+2 -2
arch/x86/events/intel/uncore_nhmex.c
··· 1221 1221 uncore_nhmex = true; 1222 1222 else 1223 1223 nhmex_uncore_mbox.event_descs = wsmex_uncore_mbox_events; 1224 - if (nhmex_uncore_cbox.num_boxes > boot_cpu_data.x86_max_cores) 1225 - nhmex_uncore_cbox.num_boxes = boot_cpu_data.x86_max_cores; 1224 + if (nhmex_uncore_cbox.num_boxes > topology_num_cores_per_package()) 1225 + nhmex_uncore_cbox.num_boxes = topology_num_cores_per_package(); 1226 1226 uncore_msr_uncores = nhmex_msr_uncores; 1227 1227 } 1228 1228 /* end of Nehalem-EX uncore support */
+4 -4
arch/x86/events/intel/uncore_snb.c
··· 364 364 void snb_uncore_cpu_init(void) 365 365 { 366 366 uncore_msr_uncores = snb_msr_uncores; 367 - if (snb_uncore_cbox.num_boxes > boot_cpu_data.x86_max_cores) 368 - snb_uncore_cbox.num_boxes = boot_cpu_data.x86_max_cores; 367 + if (snb_uncore_cbox.num_boxes > topology_num_cores_per_package()) 368 + snb_uncore_cbox.num_boxes = topology_num_cores_per_package(); 369 369 } 370 370 371 371 static void skl_uncore_msr_init_box(struct intel_uncore_box *box) ··· 428 428 void skl_uncore_cpu_init(void) 429 429 { 430 430 uncore_msr_uncores = skl_msr_uncores; 431 - if (skl_uncore_cbox.num_boxes > boot_cpu_data.x86_max_cores) 432 - skl_uncore_cbox.num_boxes = boot_cpu_data.x86_max_cores; 431 + if (skl_uncore_cbox.num_boxes > topology_num_cores_per_package()) 432 + skl_uncore_cbox.num_boxes = topology_num_cores_per_package(); 433 433 snb_uncore_arb.ops = &skl_uncore_msr_ops; 434 434 } 435 435
+9 -9
arch/x86/events/intel/uncore_snbep.c
··· 1172 1172 1173 1173 void snbep_uncore_cpu_init(void) 1174 1174 { 1175 - if (snbep_uncore_cbox.num_boxes > boot_cpu_data.x86_max_cores) 1176 - snbep_uncore_cbox.num_boxes = boot_cpu_data.x86_max_cores; 1175 + if (snbep_uncore_cbox.num_boxes > topology_num_cores_per_package()) 1176 + snbep_uncore_cbox.num_boxes = topology_num_cores_per_package(); 1177 1177 uncore_msr_uncores = snbep_msr_uncores; 1178 1178 } 1179 1179 ··· 1406 1406 */ 1407 1407 for (i = 0; i < 8; i++) { 1408 1408 if (nodeid == GIDNIDMAP(gidnid, i)) { 1409 - if (topology_max_die_per_package() > 1) 1409 + if (topology_max_dies_per_package() > 1) 1410 1410 die_id = i; 1411 1411 else 1412 1412 die_id = topology_phys_to_logical_pkg(i); ··· 1845 1845 1846 1846 void ivbep_uncore_cpu_init(void) 1847 1847 { 1848 - if (ivbep_uncore_cbox.num_boxes > boot_cpu_data.x86_max_cores) 1849 - ivbep_uncore_cbox.num_boxes = boot_cpu_data.x86_max_cores; 1848 + if (ivbep_uncore_cbox.num_boxes > topology_num_cores_per_package()) 1849 + ivbep_uncore_cbox.num_boxes = topology_num_cores_per_package(); 1850 1850 uncore_msr_uncores = ivbep_msr_uncores; 1851 1851 } 1852 1852 ··· 2917 2917 2918 2918 void hswep_uncore_cpu_init(void) 2919 2919 { 2920 - if (hswep_uncore_cbox.num_boxes > boot_cpu_data.x86_max_cores) 2921 - hswep_uncore_cbox.num_boxes = boot_cpu_data.x86_max_cores; 2920 + if (hswep_uncore_cbox.num_boxes > topology_num_cores_per_package()) 2921 + hswep_uncore_cbox.num_boxes = topology_num_cores_per_package(); 2922 2922 2923 2923 /* Detect 6-8 core systems with only two SBOXes */ 2924 2924 if (hswep_has_limit_sbox(HSWEP_PCU_DID)) ··· 3280 3280 3281 3281 void bdx_uncore_cpu_init(void) 3282 3282 { 3283 - if (bdx_uncore_cbox.num_boxes > boot_cpu_data.x86_max_cores) 3284 - bdx_uncore_cbox.num_boxes = boot_cpu_data.x86_max_cores; 3283 + if (bdx_uncore_cbox.num_boxes > topology_num_cores_per_package()) 3284 + bdx_uncore_cbox.num_boxes = topology_num_cores_per_package(); 3285 3285 uncore_msr_uncores = bdx_msr_uncores; 3286 3286 3287 3287 /* Detect systems with no SBOXes */
+1 -1
arch/x86/events/rapl.c
··· 674 674 675 675 static int __init init_rapl_pmus(void) 676 676 { 677 - int maxdie = topology_max_packages() * topology_max_die_per_package(); 677 + int maxdie = topology_max_packages() * topology_max_dies_per_package(); 678 678 size_t size; 679 679 680 680 size = sizeof(*rapl_pmus) + maxdie * sizeof(struct rapl_pmu *);
+3 -2
arch/x86/hyperv/hv_vtl.c
··· 31 31 x86_init.timers.timer_init = x86_init_noop; 32 32 33 33 /* Avoid searching for BIOS MP tables */ 34 - x86_init.mpparse.find_smp_config = x86_init_noop; 35 - x86_init.mpparse.get_smp_config = x86_init_uint_noop; 34 + x86_init.mpparse.find_mptable = x86_init_noop; 35 + x86_init.mpparse.early_parse_smp_cfg = x86_init_noop; 36 + x86_init.mpparse.parse_smp_cfg = x86_init_noop; 36 37 37 38 x86_platform.get_wallclock = get_rtc_noop; 38 39 x86_platform.set_wallclock = set_rtc_noop;
+14 -8
arch/x86/include/asm/apic.h
··· 46 46 static inline void x86_32_probe_apic(void) { } 47 47 #endif 48 48 49 + extern u32 cpuid_to_apicid[]; 50 + 51 + #define CPU_ACPIID_INVALID U32_MAX 52 + 49 53 #ifdef CONFIG_X86_LOCAL_APIC 50 54 51 55 extern int apic_verbosity; ··· 57 53 58 54 extern bool apic_is_disabled; 59 55 extern unsigned int lapic_timer_period; 60 - 61 - extern u32 cpuid_to_apicid[]; 62 56 63 57 extern enum apic_intr_mode_id apic_intr_mode; 64 58 enum apic_intr_mode_id { ··· 171 169 172 170 extern void apic_send_IPI_allbutself(unsigned int vector); 173 171 172 + extern void topology_register_apic(u32 apic_id, u32 acpi_id, bool present); 173 + extern void topology_register_boot_apic(u32 apic_id); 174 + extern int topology_hotplug_apic(u32 apic_id, u32 acpi_id); 175 + extern void topology_hotunplug_apic(unsigned int cpu); 176 + extern void topology_apply_cmdline_limits_early(void); 177 + extern void topology_init_possible_cpus(void); 178 + extern void topology_reset_possible_cpus_up(void); 179 + 174 180 #else /* !CONFIG_X86_LOCAL_APIC */ 175 181 static inline void lapic_shutdown(void) { } 176 182 #define local_apic_timer_c2_ok 1 ··· 193 183 static inline void lapic_assign_system_vectors(void) { } 194 184 static inline void lapic_assign_legacy_vector(unsigned int i, bool r) { } 195 185 static inline bool apic_needs_pit(void) { return true; } 186 + static inline void topology_apply_cmdline_limits_early(void) { } 187 + static inline void topology_init_possible_cpus(void) { } 196 188 #endif /* !CONFIG_X86_LOCAL_APIC */ 197 189 198 190 #ifdef CONFIG_X86_X2APIC ··· 301 289 /* Probe, setup and smpboot functions */ 302 290 int (*probe)(void); 303 291 int (*acpi_madt_oem_check)(char *oem_id, char *oem_table_id); 304 - bool (*apic_id_registered)(void); 305 292 306 - bool (*check_apicid_used)(physid_mask_t *map, u32 apicid); 307 293 void (*init_apic_ldr)(void); 308 - void (*ioapic_phys_id_map)(physid_mask_t *phys_map, physid_mask_t *retmap); 309 294 u32 (*cpu_present_to_apicid)(int mps_cpu); 310 - u32 (*phys_pkg_id)(u32 cpuid_apic, int index_msb); 311 295 312 296 u32 (*get_apic_id)(u32 id); 313 - u32 (*set_apic_id)(u32 apicid); 314 297 315 298 /* wakeup_secondary_cpu */ 316 299 int (*wakeup_secondary_cpu)(u32 apicid, unsigned long start_eip); ··· 534 527 extern u32 apic_default_calc_apicid(unsigned int cpu); 535 528 extern u32 apic_flat_calc_apicid(unsigned int cpu); 536 529 537 - extern void default_ioapic_phys_id_map(physid_mask_t *phys_map, physid_mask_t *retmap); 538 530 extern u32 default_cpu_present_to_apicid(int mps_cpu); 539 531 540 532 void apic_send_nmi_to_offline_cpu(unsigned int cpu);
+1 -9
arch/x86/include/asm/cpu.h
··· 9 9 #include <linux/percpu.h> 10 10 #include <asm/ibt.h> 11 11 12 - #ifdef CONFIG_SMP 13 - 14 - extern void prefill_possible_map(void); 15 - 16 - #else /* CONFIG_SMP */ 17 - 18 - static inline void prefill_possible_map(void) {} 19 - 12 + #ifndef CONFIG_SMP 20 13 #define cpu_physical_id(cpu) boot_cpu_physical_apicid 21 14 #define cpu_acpi_id(cpu) 0 22 15 #define safe_smp_processor_id() 0 23 - 24 16 #endif /* CONFIG_SMP */ 25 17 26 18 #ifdef CONFIG_HOTPLUG_CPU
+36
arch/x86/include/asm/cpuid.h
··· 127 127 return edx; 128 128 } 129 129 130 + static inline void __cpuid_read(unsigned int leaf, unsigned int subleaf, u32 *regs) 131 + { 132 + regs[CPUID_EAX] = leaf; 133 + regs[CPUID_ECX] = subleaf; 134 + __cpuid(regs + CPUID_EAX, regs + CPUID_EBX, regs + CPUID_ECX, regs + CPUID_EDX); 135 + } 136 + 137 + #define cpuid_subleaf(leaf, subleaf, regs) { \ 138 + static_assert(sizeof(*(regs)) == 16); \ 139 + __cpuid_read(leaf, subleaf, (u32 *)(regs)); \ 140 + } 141 + 142 + #define cpuid_leaf(leaf, regs) { \ 143 + static_assert(sizeof(*(regs)) == 16); \ 144 + __cpuid_read(leaf, 0, (u32 *)(regs)); \ 145 + } 146 + 147 + static inline void __cpuid_read_reg(unsigned int leaf, unsigned int subleaf, 148 + enum cpuid_regs_idx regidx, u32 *reg) 149 + { 150 + u32 regs[4]; 151 + 152 + __cpuid_read(leaf, subleaf, regs); 153 + *reg = regs[regidx]; 154 + } 155 + 156 + #define cpuid_subleaf_reg(leaf, subleaf, regidx, reg) { \ 157 + static_assert(sizeof(*(reg)) == 4); \ 158 + __cpuid_read_reg(leaf, subleaf, regidx, (u32 *)(reg)); \ 159 + } 160 + 161 + #define cpuid_leaf_reg(leaf, regidx, reg) { \ 162 + static_assert(sizeof(*(reg)) == 4); \ 163 + __cpuid_read_reg(leaf, 0, regidx, (u32 *)(reg)); \ 164 + } 165 + 130 166 static __always_inline bool cpuid_function_is_indexed(u32 function) 131 167 { 132 168 switch (function) {
-1
arch/x86/include/asm/io_apic.h
··· 140 140 extern int restore_ioapic_entries(void); 141 141 142 142 extern void setup_ioapic_ids_from_mpc(void); 143 - extern void setup_ioapic_ids_from_mpc_nocheck(void); 144 143 145 144 extern int mp_find_ioapic(u32 gsi); 146 145 extern int mp_find_ioapic_pin(int ioapic, u32 gsi);
+14 -52
arch/x86/include/asm/mpspec.h
··· 2 2 #ifndef _ASM_X86_MPSPEC_H 3 3 #define _ASM_X86_MPSPEC_H 4 4 5 + #include <linux/types.h> 5 6 6 7 #include <asm/mpspec_def.h> 7 8 #include <asm/x86_init.h> ··· 47 46 # define smp_found_config 0 48 47 #endif 49 48 50 - static inline void get_smp_config(void) 51 - { 52 - x86_init.mpparse.get_smp_config(0); 53 - } 54 - 55 - static inline void early_get_smp_config(void) 56 - { 57 - x86_init.mpparse.get_smp_config(1); 58 - } 59 - 60 - static inline void find_smp_config(void) 61 - { 62 - x86_init.mpparse.find_smp_config(); 63 - } 64 - 65 49 #ifdef CONFIG_X86_MPPARSE 66 50 extern void e820__memblock_alloc_reserved_mpc_new(void); 67 51 extern int enable_update_mptable; 68 - extern void default_find_smp_config(void); 69 - extern void default_get_smp_config(unsigned int early); 52 + extern void mpparse_find_mptable(void); 53 + extern void mpparse_parse_early_smp_config(void); 54 + extern void mpparse_parse_smp_config(void); 70 55 #else 71 56 static inline void e820__memblock_alloc_reserved_mpc_new(void) { } 72 - #define enable_update_mptable 0 73 - #define default_find_smp_config x86_init_noop 74 - #define default_get_smp_config x86_init_uint_noop 57 + #define enable_update_mptable 0 58 + #define mpparse_find_mptable x86_init_noop 59 + #define mpparse_parse_early_smp_config x86_init_noop 60 + #define mpparse_parse_smp_config x86_init_noop 75 61 #endif 76 62 77 - int generic_processor_info(int apicid); 63 + extern DECLARE_BITMAP(phys_cpu_present_map, MAX_LOCAL_APIC); 78 64 79 - #define PHYSID_ARRAY_SIZE BITS_TO_LONGS(MAX_LOCAL_APIC) 80 - 81 - struct physid_mask { 82 - unsigned long mask[PHYSID_ARRAY_SIZE]; 83 - }; 84 - 85 - typedef struct physid_mask physid_mask_t; 86 - 87 - #define physid_set(physid, map) set_bit(physid, (map).mask) 88 - #define physid_isset(physid, map) test_bit(physid, (map).mask) 89 - 90 - #define physids_or(dst, src1, src2) \ 91 - bitmap_or((dst).mask, (src1).mask, (src2).mask, MAX_LOCAL_APIC) 92 - 93 - #define physids_clear(map) \ 94 - bitmap_zero((map).mask, MAX_LOCAL_APIC) 95 - 96 - #define physids_empty(map) \ 97 - bitmap_empty((map).mask, MAX_LOCAL_APIC) 98 - 99 - static inline void physids_promote(unsigned long physids, physid_mask_t *map) 65 + static inline void reset_phys_cpu_present_map(u32 apicid) 100 66 { 101 - physids_clear(*map); 102 - map->mask[0] = physids; 67 + bitmap_zero(phys_cpu_present_map, MAX_LOCAL_APIC); 68 + set_bit(apicid, phys_cpu_present_map); 103 69 } 104 70 105 - static inline void physid_set_mask_of_physid(int physid, physid_mask_t *map) 71 + static inline void copy_phys_cpu_present_map(unsigned long *dst) 106 72 { 107 - physids_clear(*map); 108 - physid_set(physid, *map); 73 + bitmap_copy(dst, phys_cpu_present_map, MAX_LOCAL_APIC); 109 74 } 110 - 111 - #define PHYSID_MASK_ALL { {[0 ... PHYSID_ARRAY_SIZE-1] = ~0UL} } 112 - #define PHYSID_MASK_NONE { {[0 ... PHYSID_ARRAY_SIZE-1] = 0UL} } 113 - 114 - extern physid_mask_t phys_cpu_present_map; 115 75 116 76 #endif /* _ASM_X86_MPSPEC_H */
+1 -1
arch/x86/include/asm/nospec-branch.h
··· 323 323 * Note: Only the memory operand variant of VERW clears the CPU buffers. 324 324 */ 325 325 .macro CLEAR_CPU_BUFFERS 326 - ALTERNATIVE "", __stringify(verw _ASM_RIP(mds_verw_sel)), X86_FEATURE_CLEAR_CPU_BUF 326 + ALTERNATIVE "", __stringify(verw mds_verw_sel), X86_FEATURE_CLEAR_CPU_BUF 327 327 .endm 328 328 329 329 #else /* __ASSEMBLY__ */
+2 -2
arch/x86/include/asm/perf_event_p4.h
··· 181 181 static inline int p4_ht_active(void) 182 182 { 183 183 #ifdef CONFIG_SMP 184 - return smp_num_siblings > 1; 184 + return __max_threads_per_core > 1; 185 185 #endif 186 186 return 0; 187 187 } ··· 189 189 static inline int p4_ht_thread(int cpu) 190 190 { 191 191 #ifdef CONFIG_SMP 192 - if (smp_num_siblings == 2) 192 + if (__max_threads_per_core == 2) 193 193 return cpu != cpumask_first(this_cpu_cpumask_var_ptr(cpu_sibling_map)); 194 194 #endif 195 195 return 0;
+3 -6
arch/x86/include/asm/processor.h
··· 100 100 u32 logical_pkg_id; 101 101 u32 logical_die_id; 102 102 103 + // AMD Node ID and Nodes per Package info 104 + u32 amd_node_id; 105 + 103 106 // Cache level topology IDs 104 107 u32 llc_id; 105 108 u32 l2c_id; ··· 122 119 #endif 123 120 __u8 x86_virt_bits; 124 121 __u8 x86_phys_bits; 125 - /* CPUID returned core id bits: */ 126 - __u8 x86_coreid_bits; 127 122 /* Max extended CPUID function supported: */ 128 123 __u32 extended_cpuid_level; 129 124 /* Maximum supported CPUID level, -1=no CPUID: */ ··· 149 148 unsigned long loops_per_jiffy; 150 149 /* protected processor identification number */ 151 150 u64 ppin; 152 - /* cpuid returned max cores value: */ 153 - u16 x86_max_cores; 154 151 u16 x86_clflush_size; 155 152 /* number of cores as seen by the OS: */ 156 153 u16 booted_cores; ··· 703 704 } 704 705 705 706 #ifdef CONFIG_CPU_SUP_AMD 706 - extern u32 amd_get_nodes_per_socket(void); 707 707 extern u32 amd_get_highest_perf(void); 708 708 extern void amd_clear_divider(void); 709 709 extern void amd_check_microcode(void); 710 710 #else 711 - static inline u32 amd_get_nodes_per_socket(void) { return 0; } 712 711 static inline u32 amd_get_highest_perf(void) { return 0; } 713 712 static inline void amd_clear_divider(void) { } 714 713 static inline void amd_check_microcode(void) { }
+2 -2
arch/x86/include/asm/prom.h
··· 23 23 extern u64 initial_dtb; 24 24 extern void add_dtb(u64 data); 25 25 void x86_of_pci_init(void); 26 - void x86_dtb_init(void); 26 + void x86_dtb_parse_smp_config(void); 27 27 #else 28 28 static inline void add_dtb(u64 data) { } 29 29 static inline void x86_of_pci_init(void) { } 30 - static inline void x86_dtb_init(void) { } 30 + static inline void x86_dtb_parse_smp_config(void) { } 31 31 #define of_ioapic 0 32 32 #endif 33 33
-6
arch/x86/include/asm/smp.h
··· 8 8 #include <asm/current.h> 9 9 #include <asm/thread_info.h> 10 10 11 - extern int smp_num_siblings; 12 - extern unsigned int num_processors; 13 - 14 11 DECLARE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_map); 15 12 DECLARE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_map); 16 13 DECLARE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_die_map); ··· 107 110 void native_smp_prepare_boot_cpu(void); 108 111 void smp_prepare_cpus_common(void); 109 112 void native_smp_prepare_cpus(unsigned int max_cpus); 110 - void calculate_max_logical_packages(void); 111 113 void native_smp_cpus_done(unsigned int max_cpus); 112 114 int common_cpu_up(unsigned int cpunum, struct task_struct *tidle); 113 115 int native_kick_ap(unsigned int cpu, struct task_struct *tidle); ··· 169 173 return (struct cpumask *)cpumask_of(0); 170 174 } 171 175 #endif /* CONFIG_SMP */ 172 - 173 - extern unsigned disabled_cpus; 174 176 175 177 #ifdef CONFIG_DEBUG_NMI_SELFTEST 176 178 extern void nmi_selftest(void);
+76 -14
arch/x86/include/asm/topology.h
··· 102 102 103 103 #include <asm-generic/topology.h> 104 104 105 + /* Topology information */ 106 + enum x86_topology_domains { 107 + TOPO_SMT_DOMAIN, 108 + TOPO_CORE_DOMAIN, 109 + TOPO_MODULE_DOMAIN, 110 + TOPO_TILE_DOMAIN, 111 + TOPO_DIE_DOMAIN, 112 + TOPO_DIEGRP_DOMAIN, 113 + TOPO_PKG_DOMAIN, 114 + TOPO_MAX_DOMAIN, 115 + }; 116 + 117 + struct x86_topology_system { 118 + unsigned int dom_shifts[TOPO_MAX_DOMAIN]; 119 + unsigned int dom_size[TOPO_MAX_DOMAIN]; 120 + }; 121 + 122 + extern struct x86_topology_system x86_topo_system; 123 + 124 + static inline unsigned int topology_get_domain_size(enum x86_topology_domains dom) 125 + { 126 + return x86_topo_system.dom_size[dom]; 127 + } 128 + 129 + static inline unsigned int topology_get_domain_shift(enum x86_topology_domains dom) 130 + { 131 + return dom == TOPO_SMT_DOMAIN ? 0 : x86_topo_system.dom_shifts[dom - 1]; 132 + } 133 + 105 134 extern const struct cpumask *cpu_coregroup_mask(int cpu); 106 135 extern const struct cpumask *cpu_clustergroup_mask(int cpu); 107 136 ··· 141 112 #define topology_core_id(cpu) (cpu_data(cpu).topo.core_id) 142 113 #define topology_ppin(cpu) (cpu_data(cpu).ppin) 143 114 144 - extern unsigned int __max_die_per_package; 115 + #define topology_amd_node_id(cpu) (cpu_data(cpu).topo.amd_node_id) 116 + 117 + extern unsigned int __max_dies_per_package; 118 + extern unsigned int __max_logical_packages; 119 + extern unsigned int __max_threads_per_core; 120 + extern unsigned int __num_threads_per_package; 121 + extern unsigned int __num_cores_per_package; 122 + 123 + static inline unsigned int topology_max_packages(void) 124 + { 125 + return __max_logical_packages; 126 + } 127 + 128 + static inline unsigned int topology_max_dies_per_package(void) 129 + { 130 + return __max_dies_per_package; 131 + } 132 + 133 + static inline unsigned int topology_num_cores_per_package(void) 134 + { 135 + return __num_cores_per_package; 136 + } 137 + 138 + static inline unsigned int topology_num_threads_per_package(void) 139 + { 140 + return __num_threads_per_package; 141 + } 142 + 143 + #ifdef CONFIG_X86_LOCAL_APIC 144 + int topology_get_logical_id(u32 apicid, enum x86_topology_domains at_level); 145 + #else 146 + static inline int topology_get_logical_id(u32 apicid, enum x86_topology_domains at_level) 147 + { 148 + return 0; 149 + } 150 + #endif 145 151 146 152 #ifdef CONFIG_SMP 147 153 #define topology_cluster_id(cpu) (cpu_data(cpu).topo.l2c_id) ··· 185 121 #define topology_core_cpumask(cpu) (per_cpu(cpu_core_map, cpu)) 186 122 #define topology_sibling_cpumask(cpu) (per_cpu(cpu_sibling_map, cpu)) 187 123 188 - extern unsigned int __max_logical_packages; 189 - #define topology_max_packages() (__max_logical_packages) 190 124 191 - static inline int topology_max_die_per_package(void) 125 + static inline int topology_phys_to_logical_pkg(unsigned int pkg) 192 126 { 193 - return __max_die_per_package; 127 + return topology_get_logical_id(pkg << x86_topo_system.dom_shifts[TOPO_PKG_DOMAIN], 128 + TOPO_PKG_DOMAIN); 194 129 } 195 130 196 131 extern int __max_smt_threads; ··· 201 138 202 139 #include <linux/cpu_smt.h> 203 140 204 - int topology_update_package_map(unsigned int apicid, unsigned int cpu); 205 - int topology_update_die_map(unsigned int dieid, unsigned int cpu); 206 - int topology_phys_to_logical_pkg(unsigned int pkg); 141 + extern unsigned int __amd_nodes_per_pkg; 142 + 143 + static inline unsigned int topology_amd_nodes_per_pkg(void) 144 + { 145 + return __amd_nodes_per_pkg; 146 + } 207 147 208 148 extern struct cpumask __cpu_primary_thread_mask; 209 149 #define cpu_primary_thread_mask ((const struct cpumask *)&__cpu_primary_thread_mask) ··· 219 153 { 220 154 return cpumask_test_cpu(cpu, cpu_primary_thread_mask); 221 155 } 156 + 222 157 #else /* CONFIG_SMP */ 223 - #define topology_max_packages() (1) 224 - static inline int 225 - topology_update_package_map(unsigned int apicid, unsigned int cpu) { return 0; } 226 - static inline int 227 - topology_update_die_map(unsigned int dieid, unsigned int cpu) { return 0; } 228 158 static inline int topology_phys_to_logical_pkg(unsigned int pkg) { return 0; } 229 - static inline int topology_max_die_per_package(void) { return 1; } 230 159 static inline int topology_max_smt_threads(void) { return 1; } 231 160 static inline bool topology_is_primary_thread(unsigned int cpu) { return true; } 161 + static inline unsigned int topology_amd_nodes_per_pkg(void) { return 0; }; 232 162 #endif /* !CONFIG_SMP */ 233 163 234 164 static inline void arch_fix_phys_package_id(int num, u32 slot)
+6 -4
arch/x86/include/asm/x86_init.h
··· 15 15 /** 16 16 * struct x86_init_mpparse - platform specific mpparse ops 17 17 * @setup_ioapic_ids: platform specific ioapic id override 18 - * @find_smp_config: find the smp configuration 19 - * @get_smp_config: get the smp configuration 18 + * @find_mptable: Find MPTABLE early to reserve the memory region 19 + * @early_parse_smp_cfg: Parse the SMP configuration data early before initmem_init() 20 + * @parse_smp_cfg: Parse the SMP configuration data 20 21 */ 21 22 struct x86_init_mpparse { 22 23 void (*setup_ioapic_ids)(void); 23 - void (*find_smp_config)(void); 24 - void (*get_smp_config)(unsigned int early); 24 + void (*find_mptable)(void); 25 + void (*early_parse_smp_cfg)(void); 26 + void (*parse_smp_cfg)(void); 25 27 }; 26 28 27 29 /**
+12 -47
arch/x86/kernel/acpi/boot.c
··· 164 164 return 0; 165 165 } 166 166 167 - /** 168 - * acpi_register_lapic - register a local apic and generates a logic cpu number 169 - * @id: local apic id to register 170 - * @acpiid: ACPI id to register 171 - * @enabled: this cpu is enabled or not 172 - * 173 - * Returns the logic cpu number which maps to the local apic 174 - */ 175 - static int acpi_register_lapic(int id, u32 acpiid, u8 enabled) 176 - { 177 - int cpu; 178 - 179 - if (id >= MAX_LOCAL_APIC) { 180 - pr_info("skipped apicid that is too big\n"); 181 - return -EINVAL; 182 - } 183 - 184 - if (!enabled) { 185 - ++disabled_cpus; 186 - return -EINVAL; 187 - } 188 - 189 - cpu = generic_processor_info(id); 190 - if (cpu >= 0) 191 - early_per_cpu(x86_cpu_to_acpiid, cpu) = acpiid; 192 - 193 - return cpu; 194 - } 195 - 196 167 static bool __init acpi_is_processor_usable(u32 lapic_flags) 197 168 { 198 169 if (lapic_flags & ACPI_MADT_ENABLED) ··· 225 254 return 0; 226 255 } 227 256 228 - acpi_register_lapic(apic_id, processor->uid, enabled); 257 + topology_register_apic(apic_id, processor->uid, enabled); 229 258 #else 230 259 pr_warn("x2apic entry ignored\n"); 231 260 #endif ··· 260 289 * to not preallocating memory for all NR_CPUS 261 290 * when we use CPU hotplug. 262 291 */ 263 - acpi_register_lapic(processor->id, /* APIC ID */ 264 - processor->processor_id, /* ACPI ID */ 265 - processor->lapic_flags & ACPI_MADT_ENABLED); 292 + topology_register_apic(processor->id, /* APIC ID */ 293 + processor->processor_id, /* ACPI ID */ 294 + processor->lapic_flags & ACPI_MADT_ENABLED); 266 295 267 296 has_lapic_cpus = true; 268 297 return 0; ··· 280 309 281 310 acpi_table_print_madt_entry(&header->common); 282 311 283 - acpi_register_lapic((processor->id << 8) | processor->eid,/* APIC ID */ 284 - processor->processor_id, /* ACPI ID */ 285 - processor->lapic_flags & ACPI_MADT_ENABLED); 312 + topology_register_apic((processor->id << 8) | processor->eid,/* APIC ID */ 313 + processor->processor_id, /* ACPI ID */ 314 + processor->lapic_flags & ACPI_MADT_ENABLED); 286 315 287 316 return 0; 288 317 } ··· 815 844 return 0; 816 845 } 817 846 818 - int acpi_map_cpu(acpi_handle handle, phys_cpuid_t physid, u32 acpi_id, 819 - int *pcpu) 847 + int acpi_map_cpu(acpi_handle handle, phys_cpuid_t physid, u32 acpi_id, int *pcpu) 820 848 { 821 - int cpu; 849 + int cpu = topology_hotplug_apic(physid, acpi_id); 822 850 823 - cpu = acpi_register_lapic(physid, acpi_id, ACPI_MADT_ENABLED); 824 851 if (cpu < 0) { 825 852 pr_info("Unable to map lapic to logical cpu number\n"); 826 853 return cpu; ··· 837 868 #ifdef CONFIG_ACPI_NUMA 838 869 set_apicid_to_node(per_cpu(x86_cpu_to_apicid, cpu), NUMA_NO_NODE); 839 870 #endif 840 - 841 - per_cpu(x86_cpu_to_apicid, cpu) = BAD_APICID; 842 - set_cpu_present(cpu, false); 843 - num_processors--; 844 - 845 - return (0); 871 + topology_hotunplug_apic(cpu); 872 + return 0; 846 873 } 847 874 EXPORT_SYMBOL(acpi_unmap_cpu); 848 - #endif /* CONFIG_ACPI_HOTPLUG_CPU */ 875 + #endif /* CONFIG_ACPI_HOTPLUG_CPU */ 849 876 850 877 int acpi_register_ioapic(acpi_handle handle, u64 phys_addr, u32 gsi_base) 851 878 {
+2 -2
arch/x86/kernel/amd_nb.c
··· 386 386 387 387 int amd_get_subcaches(int cpu) 388 388 { 389 - struct pci_dev *link = node_to_amd_nb(topology_die_id(cpu))->link; 389 + struct pci_dev *link = node_to_amd_nb(topology_amd_node_id(cpu))->link; 390 390 unsigned int mask; 391 391 392 392 if (!amd_nb_has_feature(AMD_NB_L3_PARTITIONING)) ··· 400 400 int amd_set_subcaches(int cpu, unsigned long mask) 401 401 { 402 402 static unsigned int reset, ban; 403 - struct amd_northbridge *nb = node_to_amd_nb(topology_die_id(cpu)); 403 + struct amd_northbridge *nb = node_to_amd_nb(topology_amd_node_id(cpu)); 404 404 unsigned int reg; 405 405 int cuid; 406 406
+4 -203
arch/x86/kernel/apic/apic.c
··· 19 19 #include <linux/kernel_stat.h> 20 20 #include <linux/mc146818rtc.h> 21 21 #include <linux/acpi_pmtmr.h> 22 + #include <linux/bitmap.h> 22 23 #include <linux/clockchips.h> 23 24 #include <linux/interrupt.h> 24 25 #include <linux/memblock.h> ··· 68 67 69 68 #include "local.h" 70 69 71 - unsigned int num_processors; 72 - 73 - unsigned disabled_cpus; 74 - 75 70 /* Processor that is doing the boot up */ 76 71 u32 boot_cpu_physical_apicid __ro_after_init = BAD_APICID; 77 72 EXPORT_SYMBOL_GPL(boot_cpu_physical_apicid); 78 73 79 74 u8 boot_cpu_apic_version __ro_after_init; 80 - 81 - /* 82 - * Bitmask of physically existing CPUs: 83 - */ 84 - physid_mask_t phys_cpu_present_map; 85 - 86 - /* 87 - * Processor to be disabled specified by kernel parameter 88 - * disable_cpu_apicid=<int>, mostly used for the kdump 2nd kernel to 89 - * avoid undefined behaviour caused by sending INIT from AP to BSP. 90 - */ 91 - static u32 disabled_cpu_apicid __ro_after_init = BAD_APICID; 92 75 93 76 /* 94 77 * This variable controls which CPUs receive external NMIs. By default, ··· 92 107 { 93 108 return x2apic_mode || apic_mmio_base; 94 109 } 95 - 96 - /* 97 - * Map cpu index to physical APIC ID 98 - */ 99 - DEFINE_EARLY_PER_CPU_READ_MOSTLY(u32, x86_cpu_to_apicid, BAD_APICID); 100 - DEFINE_EARLY_PER_CPU_READ_MOSTLY(u32, x86_cpu_to_acpiid, U32_MAX); 101 - EXPORT_EARLY_PER_CPU_SYMBOL(x86_cpu_to_apicid); 102 - EXPORT_EARLY_PER_CPU_SYMBOL(x86_cpu_to_acpiid); 103 110 104 111 #ifdef CONFIG_X86_32 105 112 /* Local APIC was disabled by the BIOS and enabled by the kernel */ ··· 237 260 238 261 return icr1 | ((u64)icr2 << 32); 239 262 } 240 - 241 - #ifdef CONFIG_X86_32 242 - /** 243 - * get_physical_broadcast - Get number of physical broadcast IDs 244 - */ 245 - int get_physical_broadcast(void) 246 - { 247 - return modern_apic() ? 0xff : 0xf; 248 - } 249 - #endif 250 263 251 264 /** 252 265 * lapic_get_maxlvt - get the maximum number of local vector table entries ··· 1516 1549 apic_write(APIC_ESR, 0); 1517 1550 } 1518 1551 #endif 1519 - /* Validate that the APIC is registered if required */ 1520 - BUG_ON(apic->apic_id_registered && !apic->apic_id_registered()); 1521 - 1522 1552 /* 1523 1553 * Intel recommends to set DFR, LDR and TPR before enabling 1524 1554 * an APIC. See e.g. "AP-388 82489DX User's Manual" (Intel ··· 1654 1690 end_local_APIC_setup(); 1655 1691 } 1656 1692 1657 - static __init void cpu_set_boot_apic(void); 1658 - 1659 1693 static __init void apic_read_boot_cpu_id(bool x2apic) 1660 1694 { 1661 1695 /* ··· 1668 1706 boot_cpu_physical_apicid = read_apic_id(); 1669 1707 boot_cpu_apic_version = GET_APIC_VERSION(apic_read(APIC_LVR)); 1670 1708 } 1671 - cpu_set_boot_apic(); 1709 + topology_register_boot_apic(boot_cpu_physical_apicid); 1710 + x86_32_probe_bigsmp_early(); 1672 1711 } 1673 1712 1674 1713 #ifdef CONFIG_X86_X2APIC ··· 2054 2091 pr_info("APIC: disable apic facility\n"); 2055 2092 apic_disable(); 2056 2093 } 2057 - num_processors = 1; 2058 2094 } 2059 2095 } 2060 2096 ··· 2267 2305 apic_write(APIC_LVT1, value); 2268 2306 } 2269 2307 2270 - /* 2271 - * The number of allocated logical CPU IDs. Since logical CPU IDs are allocated 2272 - * contiguously, it equals to current allocated max logical CPU ID plus 1. 2273 - * All allocated CPU IDs should be in the [0, nr_logical_cpuids) range, 2274 - * so the maximum of nr_logical_cpuids is nr_cpu_ids. 2275 - * 2276 - * NOTE: Reserve 0 for BSP. 2277 - */ 2278 - static int nr_logical_cpuids = 1; 2279 - 2280 - /* 2281 - * Used to store mapping between logical CPU IDs and APIC IDs. 2282 - */ 2283 - u32 cpuid_to_apicid[] = { [0 ... NR_CPUS - 1] = BAD_APICID, }; 2284 - 2285 - bool arch_match_cpu_phys_id(int cpu, u64 phys_id) 2286 - { 2287 - return phys_id == (u64)cpuid_to_apicid[cpu]; 2288 - } 2289 - 2290 - #ifdef CONFIG_SMP 2291 - static void cpu_mark_primary_thread(unsigned int cpu, unsigned int apicid) 2292 - { 2293 - /* Isolate the SMT bit(s) in the APICID and check for 0 */ 2294 - u32 mask = (1U << (fls(smp_num_siblings) - 1)) - 1; 2295 - 2296 - if (smp_num_siblings == 1 || !(apicid & mask)) 2297 - cpumask_set_cpu(cpu, &__cpu_primary_thread_mask); 2298 - } 2299 - 2300 - /* 2301 - * Due to the utter mess of CPUID evaluation smp_num_siblings is not valid 2302 - * during early boot. Initialize the primary thread mask before SMP 2303 - * bringup. 2304 - */ 2305 - static int __init smp_init_primary_thread_mask(void) 2306 - { 2307 - unsigned int cpu; 2308 - 2309 - /* 2310 - * XEN/PV provides either none or useless topology information. 2311 - * Pretend that all vCPUs are primary threads. 2312 - */ 2313 - if (xen_pv_domain()) { 2314 - cpumask_copy(&__cpu_primary_thread_mask, cpu_possible_mask); 2315 - return 0; 2316 - } 2317 - 2318 - for (cpu = 0; cpu < nr_logical_cpuids; cpu++) 2319 - cpu_mark_primary_thread(cpu, cpuid_to_apicid[cpu]); 2320 - return 0; 2321 - } 2322 - early_initcall(smp_init_primary_thread_mask); 2323 - #else 2324 - static inline void cpu_mark_primary_thread(unsigned int cpu, unsigned int apicid) { } 2325 - #endif 2326 - 2327 - /* 2328 - * Should use this API to allocate logical CPU IDs to keep nr_logical_cpuids 2329 - * and cpuid_to_apicid[] synchronized. 2330 - */ 2331 - static int allocate_logical_cpuid(int apicid) 2332 - { 2333 - int i; 2334 - 2335 - /* 2336 - * cpuid <-> apicid mapping is persistent, so when a cpu is up, 2337 - * check if the kernel has allocated a cpuid for it. 2338 - */ 2339 - for (i = 0; i < nr_logical_cpuids; i++) { 2340 - if (cpuid_to_apicid[i] == apicid) 2341 - return i; 2342 - } 2343 - 2344 - /* Allocate a new cpuid. */ 2345 - if (nr_logical_cpuids >= nr_cpu_ids) { 2346 - WARN_ONCE(1, "APIC: NR_CPUS/possible_cpus limit of %u reached. " 2347 - "Processor %d/0x%x and the rest are ignored.\n", 2348 - nr_cpu_ids, nr_logical_cpuids, apicid); 2349 - return -EINVAL; 2350 - } 2351 - 2352 - cpuid_to_apicid[nr_logical_cpuids] = apicid; 2353 - return nr_logical_cpuids++; 2354 - } 2355 - 2356 - static void cpu_update_apic(int cpu, u32 apicid) 2357 - { 2358 - #if defined(CONFIG_SMP) || defined(CONFIG_X86_64) 2359 - early_per_cpu(x86_cpu_to_apicid, cpu) = apicid; 2360 - #endif 2361 - set_cpu_possible(cpu, true); 2362 - physid_set(apicid, phys_cpu_present_map); 2363 - set_cpu_present(cpu, true); 2364 - num_processors++; 2365 - 2366 - if (system_state != SYSTEM_BOOTING) 2367 - cpu_mark_primary_thread(cpu, apicid); 2368 - } 2369 - 2370 - static __init void cpu_set_boot_apic(void) 2371 - { 2372 - cpuid_to_apicid[0] = boot_cpu_physical_apicid; 2373 - cpu_update_apic(0, boot_cpu_physical_apicid); 2374 - x86_32_probe_bigsmp_early(); 2375 - } 2376 - 2377 - int generic_processor_info(int apicid) 2378 - { 2379 - int cpu, max = nr_cpu_ids; 2380 - 2381 - /* The boot CPU must be set before MADT/MPTABLE parsing happens */ 2382 - if (cpuid_to_apicid[0] == BAD_APICID) 2383 - panic("Boot CPU APIC not registered yet\n"); 2384 - 2385 - if (apicid == boot_cpu_physical_apicid) 2386 - return 0; 2387 - 2388 - if (disabled_cpu_apicid == apicid) { 2389 - int thiscpu = num_processors + disabled_cpus; 2390 - 2391 - pr_warn("APIC: Disabling requested cpu. Processor %d/0x%x ignored.\n", 2392 - thiscpu, apicid); 2393 - 2394 - disabled_cpus++; 2395 - return -ENODEV; 2396 - } 2397 - 2398 - if (num_processors >= nr_cpu_ids) { 2399 - int thiscpu = max + disabled_cpus; 2400 - 2401 - pr_warn("APIC: NR_CPUS/possible_cpus limit of %i reached. " 2402 - "Processor %d/0x%x ignored.\n", max, thiscpu, apicid); 2403 - 2404 - disabled_cpus++; 2405 - return -EINVAL; 2406 - } 2407 - 2408 - cpu = allocate_logical_cpuid(apicid); 2409 - if (cpu < 0) { 2410 - disabled_cpus++; 2411 - return -EINVAL; 2412 - } 2413 - 2414 - cpu_update_apic(cpu, apicid); 2415 - return cpu; 2416 - } 2417 - 2418 - 2419 2308 void __irq_msi_compose_msg(struct irq_cfg *cfg, struct msi_msg *msg, 2420 2309 bool dmar) 2421 2310 { ··· 2309 2496 2310 2497 static void __init apic_bsp_up_setup(void) 2311 2498 { 2312 - #ifdef CONFIG_X86_64 2313 - apic_write(APIC_ID, apic->set_apic_id(boot_cpu_physical_apicid)); 2314 - #endif 2315 - physid_set_mask_of_physid(boot_cpu_physical_apicid, &phys_cpu_present_map); 2499 + reset_phys_cpu_present_map(boot_cpu_physical_apicid); 2316 2500 } 2317 2501 2318 2502 /** ··· 2654 2844 * that is using request_resource 2655 2845 */ 2656 2846 late_initcall(lapic_insert_resource); 2657 - 2658 - static int __init apic_set_disabled_cpu_apicid(char *arg) 2659 - { 2660 - if (!arg || !get_option(&arg, &disabled_cpu_apicid)) 2661 - return -EINVAL; 2662 - 2663 - return 0; 2664 - } 2665 - early_param("disable_cpu_apicid", apic_set_disabled_cpu_apicid); 2666 2847 2667 2848 static int __init apic_set_extnmi(char *arg) 2668 2849 {
-15
arch/x86/kernel/apic/apic_common.c
··· 18 18 return 1U << cpu; 19 19 } 20 20 21 - bool default_check_apicid_used(physid_mask_t *map, u32 apicid) 22 - { 23 - return physid_isset(apicid, *map); 24 - } 25 - 26 - void default_ioapic_phys_id_map(physid_mask_t *phys_map, physid_mask_t *retmap) 27 - { 28 - *retmap = *phys_map; 29 - } 30 - 31 21 u32 default_cpu_present_to_apicid(int mps_cpu) 32 22 { 33 23 if (mps_cpu < nr_cpu_ids && cpu_present(mps_cpu)) ··· 26 36 return BAD_APICID; 27 37 } 28 38 EXPORT_SYMBOL_GPL(default_cpu_present_to_apicid); 29 - 30 - bool default_apic_id_registered(void) 31 - { 32 - return physid_isset(read_apic_id(), phys_cpu_present_map); 33 - } 34 39 35 40 /* 36 41 * Set up the logical destination ID when the APIC operates in logical
-16
arch/x86/kernel/apic/apic_flat_64.c
··· 61 61 return (x >> 24) & 0xFF; 62 62 } 63 63 64 - static u32 set_apic_id(u32 id) 65 - { 66 - return (id & 0xFF) << 24; 67 - } 68 - 69 - static u32 flat_phys_pkg_id(u32 initial_apic_id, int index_msb) 70 - { 71 - return initial_apic_id >> index_msb; 72 - } 73 - 74 64 static int flat_probe(void) 75 65 { 76 66 return 1; ··· 70 80 .name = "flat", 71 81 .probe = flat_probe, 72 82 .acpi_madt_oem_check = flat_acpi_madt_oem_check, 73 - .apic_id_registered = default_apic_id_registered, 74 83 75 84 .dest_mode_logical = true, 76 85 ··· 77 88 78 89 .init_apic_ldr = default_init_apic_ldr, 79 90 .cpu_present_to_apicid = default_cpu_present_to_apicid, 80 - .phys_pkg_id = flat_phys_pkg_id, 81 91 82 92 .max_apic_id = 0xFE, 83 93 .get_apic_id = flat_get_apic_id, 84 - .set_apic_id = set_apic_id, 85 94 86 95 .calc_dest_apicid = apic_flat_calc_apicid, 87 96 ··· 138 151 .name = "physical flat", 139 152 .probe = physflat_probe, 140 153 .acpi_madt_oem_check = physflat_acpi_madt_oem_check, 141 - .apic_id_registered = default_apic_id_registered, 142 154 143 155 .dest_mode_logical = false, 144 156 145 157 .disable_esr = 0, 146 158 147 159 .cpu_present_to_apicid = default_cpu_present_to_apicid, 148 - .phys_pkg_id = flat_phys_pkg_id, 149 160 150 161 .max_apic_id = 0xFE, 151 162 .get_apic_id = flat_get_apic_id, 152 - .set_apic_id = set_apic_id, 153 163 154 164 .calc_dest_apicid = apic_default_calc_apicid, 155 165
-5
arch/x86/kernel/apic/apic_noop.c
··· 29 29 static void noop_apic_icr_write(u32 low, u32 id) { } 30 30 static int noop_wakeup_secondary_cpu(u32 apicid, unsigned long start_eip) { return -1; } 31 31 static u64 noop_apic_icr_read(void) { return 0; } 32 - static u32 noop_phys_pkg_id(u32 cpuid_apic, int index_msb) { return 0; } 33 32 static u32 noop_get_apic_id(u32 apicid) { return 0; } 34 33 static void noop_apic_eoi(void) { } 35 34 ··· 50 51 51 52 .disable_esr = 0, 52 53 53 - .check_apicid_used = default_check_apicid_used, 54 - .ioapic_phys_id_map = default_ioapic_phys_id_map, 55 54 .cpu_present_to_apicid = default_cpu_present_to_apicid, 56 - 57 - .phys_pkg_id = noop_phys_pkg_id, 58 55 59 56 .max_apic_id = 0xFE, 60 57 .get_apic_id = noop_get_apic_id,
-19
arch/x86/kernel/apic/apic_numachip.c
··· 38 38 return id; 39 39 } 40 40 41 - static u32 numachip1_set_apic_id(u32 id) 42 - { 43 - return (id & 0xff) << 24; 44 - } 45 - 46 41 static u32 numachip2_get_apic_id(u32 x) 47 42 { 48 43 u64 mcfg; 49 44 50 45 rdmsrl(MSR_FAM10H_MMIO_CONF_BASE, mcfg); 51 46 return ((mcfg >> (28 - 8)) & 0xfff00) | (x >> 24); 52 - } 53 - 54 - static u32 numachip2_set_apic_id(u32 id) 55 - { 56 - return id << 24; 57 - } 58 - 59 - static u32 numachip_phys_pkg_id(u32 initial_apic_id, int index_msb) 60 - { 61 - return initial_apic_id >> index_msb; 62 47 } 63 48 64 49 static void numachip1_apic_icr_write(int apicid, unsigned int val) ··· 212 227 .disable_esr = 0, 213 228 214 229 .cpu_present_to_apicid = default_cpu_present_to_apicid, 215 - .phys_pkg_id = numachip_phys_pkg_id, 216 230 217 231 .max_apic_id = UINT_MAX, 218 232 .get_apic_id = numachip1_get_apic_id, 219 - .set_apic_id = numachip1_set_apic_id, 220 233 221 234 .calc_dest_apicid = apic_default_calc_apicid, 222 235 ··· 246 263 .disable_esr = 0, 247 264 248 265 .cpu_present_to_apicid = default_cpu_present_to_apicid, 249 - .phys_pkg_id = numachip_phys_pkg_id, 250 266 251 267 .max_apic_id = UINT_MAX, 252 268 .get_apic_id = numachip2_get_apic_id, 253 - .set_apic_id = numachip2_set_apic_id, 254 269 255 270 .calc_dest_apicid = apic_default_calc_apicid, 256 271
-20
arch/x86/kernel/apic/bigsmp_32.c
··· 18 18 return (x >> 24) & 0xFF; 19 19 } 20 20 21 - static bool bigsmp_check_apicid_used(physid_mask_t *map, u32 apicid) 22 - { 23 - return false; 24 - } 25 - 26 - static void bigsmp_ioapic_phys_id_map(physid_mask_t *phys_map, physid_mask_t *retmap) 27 - { 28 - /* For clustered we don't have a good way to do this yet - hack */ 29 - physids_promote(0xFFL, retmap); 30 - } 31 - 32 - static u32 bigsmp_phys_pkg_id(u32 cpuid_apic, int index_msb) 33 - { 34 - return cpuid_apic >> index_msb; 35 - } 36 - 37 21 static void bigsmp_send_IPI_allbutself(int vector) 38 22 { 39 23 default_send_IPI_mask_allbutself_phys(cpu_online_mask, vector); ··· 68 84 69 85 .disable_esr = 1, 70 86 71 - .check_apicid_used = bigsmp_check_apicid_used, 72 - .ioapic_phys_id_map = bigsmp_ioapic_phys_id_map, 73 87 .cpu_present_to_apicid = default_cpu_present_to_apicid, 74 - .phys_pkg_id = bigsmp_phys_pkg_id, 75 88 76 89 .max_apic_id = 0xFE, 77 90 .get_apic_id = bigsmp_get_apic_id, 78 - .set_apic_id = NULL, 79 91 80 92 .calc_dest_apicid = apic_default_calc_apicid, 81 93
+36 -56
arch/x86/kernel/apic/io_apic.c
··· 1458 1458 * 1459 1459 * by Matt Domsch <Matt_Domsch@dell.com> Tue Dec 21 12:25:05 CST 1999 1460 1460 */ 1461 - void __init setup_ioapic_ids_from_mpc_nocheck(void) 1461 + static void __init setup_ioapic_ids_from_mpc_nocheck(void) 1462 1462 { 1463 + DECLARE_BITMAP(phys_id_present_map, MAX_LOCAL_APIC); 1464 + const u32 broadcast_id = 0xF; 1463 1465 union IO_APIC_reg_00 reg_00; 1464 - physid_mask_t phys_id_present_map; 1465 - int ioapic_idx; 1466 - int i; 1467 1466 unsigned char old_id; 1468 1467 unsigned long flags; 1468 + int ioapic_idx, i; 1469 1469 1470 1470 /* 1471 1471 * This is broken; anything with a real cpu count has to 1472 1472 * circumvent this idiocy regardless. 1473 1473 */ 1474 - apic->ioapic_phys_id_map(&phys_cpu_present_map, &phys_id_present_map); 1474 + copy_phys_cpu_present_map(phys_id_present_map); 1475 1475 1476 1476 /* 1477 1477 * Set the IOAPIC ID to the value stored in the MPC table. ··· 1484 1484 1485 1485 old_id = mpc_ioapic_id(ioapic_idx); 1486 1486 1487 - if (mpc_ioapic_id(ioapic_idx) >= get_physical_broadcast()) { 1488 - printk(KERN_ERR "BIOS bug, IO-APIC#%d ID is %d in the MPC table!...\n", 1489 - ioapic_idx, mpc_ioapic_id(ioapic_idx)); 1490 - printk(KERN_ERR "... fixing up to %d. (tell your hw vendor)\n", 1491 - reg_00.bits.ID); 1487 + if (mpc_ioapic_id(ioapic_idx) >= broadcast_id) { 1488 + pr_err(FW_BUG "IO-APIC#%d ID is %d in the MPC table!...\n", 1489 + ioapic_idx, mpc_ioapic_id(ioapic_idx)); 1490 + pr_err("... fixing up to %d. (tell your hw vendor)\n", reg_00.bits.ID); 1492 1491 ioapics[ioapic_idx].mp_config.apicid = reg_00.bits.ID; 1493 1492 } 1494 1493 ··· 1496 1497 * system must have a unique ID or we get lots of nice 1497 1498 * 'stuck on smp_invalidate_needed IPI wait' messages. 1498 1499 */ 1499 - if (apic->check_apicid_used(&phys_id_present_map, 1500 - mpc_ioapic_id(ioapic_idx))) { 1501 - printk(KERN_ERR "BIOS bug, IO-APIC#%d ID %d is already used!...\n", 1502 - ioapic_idx, mpc_ioapic_id(ioapic_idx)); 1503 - for (i = 0; i < get_physical_broadcast(); i++) 1504 - if (!physid_isset(i, phys_id_present_map)) 1500 + if (test_bit(mpc_ioapic_id(ioapic_idx), phys_id_present_map)) { 1501 + pr_err(FW_BUG "IO-APIC#%d ID %d is already used!...\n", 1502 + ioapic_idx, mpc_ioapic_id(ioapic_idx)); 1503 + for (i = 0; i < broadcast_id; i++) 1504 + if (!test_bit(i, phys_id_present_map)) 1505 1505 break; 1506 - if (i >= get_physical_broadcast()) 1506 + if (i >= broadcast_id) 1507 1507 panic("Max APIC ID exceeded!\n"); 1508 - printk(KERN_ERR "... fixing up to %d. (tell your hw vendor)\n", 1509 - i); 1510 - physid_set(i, phys_id_present_map); 1508 + pr_err("... fixing up to %d. (tell your hw vendor)\n", i); 1509 + set_bit(i, phys_id_present_map); 1511 1510 ioapics[ioapic_idx].mp_config.apicid = i; 1512 1511 } else { 1513 1512 apic_printk(APIC_VERBOSE, "Setting %d in the phys_id_present_map\n", 1514 1513 mpc_ioapic_id(ioapic_idx)); 1515 - physid_set(mpc_ioapic_id(ioapic_idx), phys_id_present_map); 1514 + set_bit(mpc_ioapic_id(ioapic_idx), phys_id_present_map); 1516 1515 } 1517 1516 1518 1517 /* ··· 2206 2209 * 8259A. 2207 2210 */ 2208 2211 if (pin1 == -1) { 2209 - panic_if_irq_remap("BIOS bug: timer not connected to IO-APIC"); 2212 + panic_if_irq_remap(FW_BUG "Timer not connected to IO-APIC"); 2210 2213 pin1 = pin2; 2211 2214 apic1 = apic2; 2212 2215 no_pin1 = 1; ··· 2491 2494 #ifdef CONFIG_X86_32 2492 2495 static int io_apic_get_unique_id(int ioapic, int apic_id) 2493 2496 { 2497 + static DECLARE_BITMAP(apic_id_map, MAX_LOCAL_APIC); 2498 + const u32 broadcast_id = 0xF; 2494 2499 union IO_APIC_reg_00 reg_00; 2495 - static physid_mask_t apic_id_map = PHYSID_MASK_NONE; 2496 - physid_mask_t tmp; 2497 2500 unsigned long flags; 2498 2501 int i = 0; 2499 2502 2500 - /* 2501 - * The P4 platform supports up to 256 APIC IDs on two separate APIC 2502 - * buses (one for LAPICs, one for IOAPICs), where predecessors only 2503 - * supports up to 16 on one shared APIC bus. 2504 - * 2505 - * TBD: Expand LAPIC/IOAPIC support on P4-class systems to take full 2506 - * advantage of new APIC bus architecture. 2507 - */ 2508 - 2509 - if (physids_empty(apic_id_map)) 2510 - apic->ioapic_phys_id_map(&phys_cpu_present_map, &apic_id_map); 2503 + /* Initialize the ID map */ 2504 + if (bitmap_empty(apic_id_map, MAX_LOCAL_APIC)) 2505 + copy_phys_cpu_present_map(apic_id_map); 2511 2506 2512 2507 raw_spin_lock_irqsave(&ioapic_lock, flags); 2513 2508 reg_00.raw = io_apic_read(ioapic, 0); 2514 2509 raw_spin_unlock_irqrestore(&ioapic_lock, flags); 2515 2510 2516 - if (apic_id >= get_physical_broadcast()) { 2517 - printk(KERN_WARNING "IOAPIC[%d]: Invalid apic_id %d, trying " 2518 - "%d\n", ioapic, apic_id, reg_00.bits.ID); 2511 + if (apic_id >= broadcast_id) { 2512 + pr_warn("IOAPIC[%d]: Invalid apic_id %d, trying %d\n", 2513 + ioapic, apic_id, reg_00.bits.ID); 2519 2514 apic_id = reg_00.bits.ID; 2520 2515 } 2521 2516 2522 - /* 2523 - * Every APIC in a system must have a unique ID or we get lots of nice 2524 - * 'stuck on smp_invalidate_needed IPI wait' messages. 2525 - */ 2526 - if (apic->check_apicid_used(&apic_id_map, apic_id)) { 2527 - 2528 - for (i = 0; i < get_physical_broadcast(); i++) { 2529 - if (!apic->check_apicid_used(&apic_id_map, i)) 2517 + /* Every APIC in a system must have a unique ID */ 2518 + if (test_bit(apic_id, apic_id_map)) { 2519 + for (i = 0; i < broadcast_id; i++) { 2520 + if (!test_bit(i, apic_id_map)) 2530 2521 break; 2531 2522 } 2532 2523 2533 - if (i == get_physical_broadcast()) 2524 + if (i == broadcast_id) 2534 2525 panic("Max apic_id exceeded!\n"); 2535 2526 2536 - printk(KERN_WARNING "IOAPIC[%d]: apic_id %d already used, " 2537 - "trying %d\n", ioapic, apic_id, i); 2538 - 2527 + pr_warn("IOAPIC[%d]: apic_id %d already used, trying %d\n", ioapic, apic_id, i); 2539 2528 apic_id = i; 2540 2529 } 2541 2530 2542 - physid_set_mask_of_physid(apic_id, &tmp); 2543 - physids_or(apic_id_map, apic_id_map, tmp); 2531 + set_bit(apic_id, apic_id_map); 2544 2532 2545 2533 if (reg_00.bits.ID != apic_id) { 2546 2534 reg_00.bits.ID = apic_id; ··· 2551 2569 2552 2570 static u8 io_apic_unique_id(int idx, u8 id) 2553 2571 { 2554 - if ((boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) && 2555 - !APIC_XAPIC(boot_cpu_apic_version)) 2572 + if ((boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) && !APIC_XAPIC(boot_cpu_apic_version)) 2556 2573 return io_apic_get_unique_id(idx, id); 2557 - else 2558 - return id; 2574 + return id; 2559 2575 } 2560 2576 #else 2561 2577 static u8 io_apic_unique_id(int idx, u8 id)
-5
arch/x86/kernel/apic/local.h
··· 16 16 /* X2APIC */ 17 17 void __x2apic_send_IPI_dest(unsigned int apicid, int vector, unsigned int dest); 18 18 u32 x2apic_get_apic_id(u32 id); 19 - u32 x2apic_set_apic_id(u32 id); 20 - u32 x2apic_phys_pkg_id(u32 initial_apicid, int index_msb); 21 19 22 20 void x2apic_send_IPI_all(int vector); 23 21 void x2apic_send_IPI_allbutself(int vector); ··· 60 62 void default_send_IPI_allbutself(int vector); 61 63 void default_send_IPI_all(int vector); 62 64 void default_send_IPI_self(int vector); 63 - 64 - bool default_apic_id_registered(void); 65 - bool default_check_apicid_used(physid_mask_t *map, u32 apicid); 66 65 67 66 #ifdef CONFIG_X86_32 68 67 void default_send_IPI_mask_sequence_logical(const struct cpumask *mask, int vector);
-9
arch/x86/kernel/apic/probe_32.c
··· 18 18 19 19 #include "local.h" 20 20 21 - static u32 default_phys_pkg_id(u32 cpuid_apic, int index_msb) 22 - { 23 - return cpuid_apic >> index_msb; 24 - } 25 - 26 21 static u32 default_get_apic_id(u32 x) 27 22 { 28 23 unsigned int ver = GET_APIC_VERSION(apic_read(APIC_LVR)); ··· 38 43 39 44 .name = "default", 40 45 .probe = probe_default, 41 - .apic_id_registered = default_apic_id_registered, 42 46 43 47 .dest_mode_logical = true, 44 48 45 49 .disable_esr = 0, 46 50 47 - .check_apicid_used = default_check_apicid_used, 48 51 .init_apic_ldr = default_init_apic_ldr, 49 - .ioapic_phys_id_map = default_ioapic_phys_id_map, 50 52 .cpu_present_to_apicid = default_cpu_present_to_apicid, 51 - .phys_pkg_id = default_phys_pkg_id, 52 53 53 54 .max_apic_id = 0xFE, 54 55 .get_apic_id = default_get_apic_id,
-4
arch/x86/kernel/apic/x2apic_cluster.c
··· 231 231 232 232 .disable_esr = 0, 233 233 234 - .check_apicid_used = NULL, 235 234 .init_apic_ldr = init_x2apic_ldr, 236 - .ioapic_phys_id_map = NULL, 237 235 .cpu_present_to_apicid = default_cpu_present_to_apicid, 238 - .phys_pkg_id = x2apic_phys_pkg_id, 239 236 240 237 .max_apic_id = UINT_MAX, 241 238 .x2apic_set_max_apicid = true, 242 239 .get_apic_id = x2apic_get_apic_id, 243 - .set_apic_id = x2apic_set_apic_id, 244 240 245 241 .calc_dest_apicid = x2apic_calc_apicid, 246 242
-12
arch/x86/kernel/apic/x2apic_phys.c
··· 129 129 return id; 130 130 } 131 131 132 - u32 x2apic_set_apic_id(u32 id) 133 - { 134 - return id; 135 - } 136 - 137 - u32 x2apic_phys_pkg_id(u32 initial_apicid, int index_msb) 138 - { 139 - return initial_apicid >> index_msb; 140 - } 141 - 142 132 static struct apic apic_x2apic_phys __ro_after_init = { 143 133 144 134 .name = "physical x2apic", ··· 140 150 .disable_esr = 0, 141 151 142 152 .cpu_present_to_apicid = default_cpu_present_to_apicid, 143 - .phys_pkg_id = x2apic_phys_pkg_id, 144 153 145 154 .max_apic_id = UINT_MAX, 146 155 .x2apic_set_max_apicid = true, 147 156 .get_apic_id = x2apic_get_apic_id, 148 - .set_apic_id = x2apic_set_apic_id, 149 157 150 158 .calc_dest_apicid = apic_default_calc_apicid, 151 159
+9 -60
arch/x86/kernel/apic/x2apic_uv_x.c
··· 241 241 is_uv(UV3) ? sname.s3.field : \ 242 242 undef) 243 243 244 - /* [Copied from arch/x86/kernel/cpu/topology.c:detect_extended_topology()] */ 245 - 246 - #define SMT_LEVEL 0 /* Leaf 0xb SMT level */ 247 - #define INVALID_TYPE 0 /* Leaf 0xb sub-leaf types */ 248 - #define SMT_TYPE 1 249 - #define CORE_TYPE 2 250 - #define LEAFB_SUBTYPE(ecx) (((ecx) >> 8) & 0xff) 251 - #define BITS_SHIFT_NEXT_LEVEL(eax) ((eax) & 0x1f) 252 - 253 - static void set_x2apic_bits(void) 254 - { 255 - unsigned int eax, ebx, ecx, edx, sub_index; 256 - unsigned int sid_shift; 257 - 258 - cpuid(0, &eax, &ebx, &ecx, &edx); 259 - if (eax < 0xb) { 260 - pr_info("UV: CPU does not have CPUID.11\n"); 261 - return; 262 - } 263 - 264 - cpuid_count(0xb, SMT_LEVEL, &eax, &ebx, &ecx, &edx); 265 - if (ebx == 0 || (LEAFB_SUBTYPE(ecx) != SMT_TYPE)) { 266 - pr_info("UV: CPUID.11 not implemented\n"); 267 - return; 268 - } 269 - 270 - sid_shift = BITS_SHIFT_NEXT_LEVEL(eax); 271 - sub_index = 1; 272 - do { 273 - cpuid_count(0xb, sub_index, &eax, &ebx, &ecx, &edx); 274 - if (LEAFB_SUBTYPE(ecx) == CORE_TYPE) { 275 - sid_shift = BITS_SHIFT_NEXT_LEVEL(eax); 276 - break; 277 - } 278 - sub_index++; 279 - } while (LEAFB_SUBTYPE(ecx) != INVALID_TYPE); 280 - 281 - uv_cpuid.apicid_shift = 0; 282 - uv_cpuid.apicid_mask = (~(-1 << sid_shift)); 283 - uv_cpuid.socketid_shift = sid_shift; 284 - } 285 - 286 244 static void __init early_get_apic_socketid_shift(void) 287 245 { 246 + unsigned int sid_shift = topology_get_domain_shift(TOPO_PKG_DOMAIN); 247 + 288 248 if (is_uv2_hub() || is_uv3_hub()) 289 249 uvh_apicid.v = uv_early_read_mmr(UVH_APICID); 290 250 291 - set_x2apic_bits(); 251 + if (sid_shift) { 252 + uv_cpuid.apicid_shift = 0; 253 + uv_cpuid.apicid_mask = (~(-1 << sid_shift)); 254 + uv_cpuid.socketid_shift = sid_shift; 255 + } else { 256 + pr_info("UV: CPU does not have valid CPUID.11\n"); 257 + } 292 258 293 259 pr_info("UV: apicid_shift:%d apicid_mask:0x%x\n", uv_cpuid.apicid_shift, uv_cpuid.apicid_mask); 294 260 pr_info("UV: socketid_shift:%d pnode_mask:0x%x\n", uv_cpuid.socketid_shift, uv_cpuid.pnode_mask); ··· 745 779 uv_send_IPI_mask(cpu_online_mask, vector); 746 780 } 747 781 748 - static u32 set_apic_id(u32 id) 749 - { 750 - return id; 751 - } 752 - 753 - static unsigned int uv_read_apic_id(void) 754 - { 755 - return x2apic_get_apic_id(apic_read(APIC_ID)); 756 - } 757 - 758 - static u32 uv_phys_pkg_id(u32 initial_apicid, int index_msb) 759 - { 760 - return uv_read_apic_id() >> index_msb; 761 - } 762 - 763 782 static int uv_probe(void) 764 783 { 765 784 return apic == &apic_x2apic_uv_x; ··· 761 810 .disable_esr = 0, 762 811 763 812 .cpu_present_to_apicid = default_cpu_present_to_apicid, 764 - .phys_pkg_id = uv_phys_pkg_id, 765 813 766 814 .max_apic_id = UINT_MAX, 767 815 .get_apic_id = x2apic_get_apic_id, 768 - .set_apic_id = set_apic_id, 769 816 770 817 .calc_dest_apicid = apic_default_calc_apicid, 771 818
+9 -6
arch/x86/kernel/cpu/Makefile
··· 17 17 # As above, instrumenting secondary CPU boot code causes boot hangs. 18 18 KCSAN_SANITIZE_common.o := n 19 19 20 - obj-y := cacheinfo.o scattered.o topology.o 20 + obj-y := cacheinfo.o scattered.o 21 + obj-y += topology_common.o topology_ext.o topology_amd.o 21 22 obj-y += common.o 22 23 obj-y += rdrand.o 23 24 obj-y += match.o ··· 26 25 obj-y += aperfmperf.o 27 26 obj-y += cpuid-deps.o 28 27 obj-y += umwait.o 28 + obj-y += capflags.o powerflags.o 29 29 30 - obj-$(CONFIG_PROC_FS) += proc.o 31 - obj-y += capflags.o powerflags.o 30 + obj-$(CONFIG_X86_LOCAL_APIC) += topology.o 32 31 33 - obj-$(CONFIG_IA32_FEAT_CTL) += feat_ctl.o 32 + obj-$(CONFIG_PROC_FS) += proc.o 33 + 34 + obj-$(CONFIG_IA32_FEAT_CTL) += feat_ctl.o 34 35 ifdef CONFIG_CPU_SUP_INTEL 35 - obj-y += intel.o intel_pconfig.o tsx.o 36 - obj-$(CONFIG_PM) += intel_epb.o 36 + obj-y += intel.o intel_pconfig.o tsx.o 37 + obj-$(CONFIG_PM) += intel_epb.o 37 38 endif 38 39 obj-$(CONFIG_CPU_SUP_AMD) += amd.o 39 40 obj-$(CONFIG_CPU_SUP_HYGON) += hygon.o
-146
arch/x86/kernel/cpu/amd.c
··· 27 27 28 28 #include "cpu.h" 29 29 30 - /* 31 - * nodes_per_socket: Stores the number of nodes per socket. 32 - * Refer to Fam15h Models 00-0fh BKDG - CPUID Fn8000_001E_ECX 33 - * Node Identifiers[10:8] 34 - */ 35 - static u32 nodes_per_socket = 1; 36 - 37 30 static inline int rdmsrl_amd_safe(unsigned msr, unsigned long long *p) 38 31 { 39 32 u32 gprs[8] = { 0 }; ··· 293 300 } 294 301 #endif 295 302 296 - /* 297 - * Fix up topo::core_id for pre-F17h systems to be in the 298 - * [0 .. cores_per_node - 1] range. Not really needed but 299 - * kept so as not to break existing setups. 300 - */ 301 - static void legacy_fixup_core_id(struct cpuinfo_x86 *c) 302 - { 303 - u32 cus_per_node; 304 - 305 - if (c->x86 >= 0x17) 306 - return; 307 - 308 - cus_per_node = c->x86_max_cores / nodes_per_socket; 309 - c->topo.core_id %= cus_per_node; 310 - } 311 - 312 - /* 313 - * Fixup core topology information for 314 - * (1) AMD multi-node processors 315 - * Assumption: Number of cores in each internal node is the same. 316 - * (2) AMD processors supporting compute units 317 - */ 318 - static void amd_get_topology(struct cpuinfo_x86 *c) 319 - { 320 - /* get information required for multi-node processors */ 321 - if (boot_cpu_has(X86_FEATURE_TOPOEXT)) { 322 - int err; 323 - u32 eax, ebx, ecx, edx; 324 - 325 - cpuid(0x8000001e, &eax, &ebx, &ecx, &edx); 326 - 327 - c->topo.die_id = ecx & 0xff; 328 - 329 - if (c->x86 == 0x15) 330 - c->topo.cu_id = ebx & 0xff; 331 - 332 - if (c->x86 >= 0x17) { 333 - c->topo.core_id = ebx & 0xff; 334 - 335 - if (smp_num_siblings > 1) 336 - c->x86_max_cores /= smp_num_siblings; 337 - } 338 - 339 - /* 340 - * In case leaf B is available, use it to derive 341 - * topology information. 342 - */ 343 - err = detect_extended_topology(c); 344 - if (!err) 345 - c->x86_coreid_bits = get_count_order(c->x86_max_cores); 346 - 347 - cacheinfo_amd_init_llc_id(c); 348 - 349 - } else if (cpu_has(c, X86_FEATURE_NODEID_MSR)) { 350 - u64 value; 351 - 352 - rdmsrl(MSR_FAM10H_NODE_ID, value); 353 - c->topo.die_id = value & 7; 354 - c->topo.llc_id = c->topo.die_id; 355 - } else 356 - return; 357 - 358 - if (nodes_per_socket > 1) { 359 - set_cpu_cap(c, X86_FEATURE_AMD_DCM); 360 - legacy_fixup_core_id(c); 361 - } 362 - } 363 - 364 - /* 365 - * On a AMD dual core setup the lower bits of the APIC id distinguish the cores. 366 - * Assumes number of cores is a power of two. 367 - */ 368 - static void amd_detect_cmp(struct cpuinfo_x86 *c) 369 - { 370 - unsigned bits; 371 - 372 - bits = c->x86_coreid_bits; 373 - /* Low order bits define the core id (index of core in socket) */ 374 - c->topo.core_id = c->topo.initial_apicid & ((1 << bits)-1); 375 - /* Convert the initial APIC ID into the socket ID */ 376 - c->topo.pkg_id = c->topo.initial_apicid >> bits; 377 - /* use socket ID also for last level cache */ 378 - c->topo.llc_id = c->topo.die_id = c->topo.pkg_id; 379 - } 380 - 381 - u32 amd_get_nodes_per_socket(void) 382 - { 383 - return nodes_per_socket; 384 - } 385 - EXPORT_SYMBOL_GPL(amd_get_nodes_per_socket); 386 - 387 303 static void srat_detect_node(struct cpuinfo_x86 *c) 388 304 { 389 305 #ifdef CONFIG_NUMA ··· 344 442 #endif 345 443 } 346 444 347 - static void early_init_amd_mc(struct cpuinfo_x86 *c) 348 - { 349 - #ifdef CONFIG_SMP 350 - unsigned bits, ecx; 351 - 352 - /* Multi core CPU? */ 353 - if (c->extended_cpuid_level < 0x80000008) 354 - return; 355 - 356 - ecx = cpuid_ecx(0x80000008); 357 - 358 - c->x86_max_cores = (ecx & 0xff) + 1; 359 - 360 - /* CPU telling us the core id bits shift? */ 361 - bits = (ecx >> 12) & 0xF; 362 - 363 - /* Otherwise recompute */ 364 - if (bits == 0) { 365 - while ((1 << bits) < c->x86_max_cores) 366 - bits++; 367 - } 368 - 369 - c->x86_coreid_bits = bits; 370 - #endif 371 - } 372 - 373 445 static void bsp_init_amd(struct cpuinfo_x86 *c) 374 446 { 375 447 if (cpu_has(c, X86_FEATURE_CONSTANT_TSC)) { ··· 375 499 376 500 if (cpu_has(c, X86_FEATURE_MWAITX)) 377 501 use_mwaitx_delay(); 378 - 379 - if (boot_cpu_has(X86_FEATURE_TOPOEXT)) { 380 - u32 ecx; 381 - 382 - ecx = cpuid_ecx(0x8000001e); 383 - __max_die_per_package = nodes_per_socket = ((ecx >> 8) & 7) + 1; 384 - } else if (boot_cpu_has(X86_FEATURE_NODEID_MSR)) { 385 - u64 value; 386 - 387 - rdmsrl(MSR_FAM10H_NODE_ID, value); 388 - __max_die_per_package = nodes_per_socket = ((value >> 3) & 7) + 1; 389 - } 390 502 391 503 if (!boot_cpu_has(X86_FEATURE_AMD_SSBD) && 392 504 !boot_cpu_has(X86_FEATURE_VIRT_SSBD) && ··· 513 649 u64 value; 514 650 u32 dummy; 515 651 516 - early_init_amd_mc(c); 517 - 518 652 if (c->x86 >= 0xf) 519 653 set_cpu_cap(c, X86_FEATURE_K8); 520 654 ··· 591 729 } 592 730 } 593 731 } 594 - 595 - if (cpu_has(c, X86_FEATURE_TOPOEXT)) 596 - smp_num_siblings = ((cpuid_ebx(0x8000001e) >> 8) & 0xff) + 1; 597 732 598 733 if (!cpu_has(c, X86_FEATURE_HYPERVISOR) && !cpu_has(c, X86_FEATURE_IBPB_BRTYPE)) { 599 734 if (c->x86 == 0x17 && boot_cpu_has(X86_FEATURE_AMD_IBPB)) ··· 935 1076 if (cpu_has(c, X86_FEATURE_FSRM)) 936 1077 set_cpu_cap(c, X86_FEATURE_FSRS); 937 1078 938 - /* get apicid instead of initial apic id from cpuid */ 939 - c->topo.apicid = read_apic_id(); 940 - 941 1079 /* K6s reports MCEs but don't actually have all the MSRs */ 942 1080 if (c->x86 < 6) 943 1081 clear_cpu_cap(c, X86_FEATURE_MCE); ··· 970 1114 971 1115 cpu_detect_cache_sizes(c); 972 1116 973 - amd_detect_cmp(c); 974 - amd_get_topology(c); 975 1117 srat_detect_node(c); 976 1118 977 1119 init_amd_cacheinfo(c);
+4 -4
arch/x86/kernel/cpu/cacheinfo.c
··· 301 301 eax->split.type = types[leaf]; 302 302 eax->split.level = levels[leaf]; 303 303 eax->split.num_threads_sharing = 0; 304 - eax->split.num_cores_on_die = __this_cpu_read(cpu_info.x86_max_cores) - 1; 304 + eax->split.num_cores_on_die = topology_num_cores_per_package(); 305 305 306 306 307 307 if (assoc == 0xffff) ··· 595 595 if (index < 3) 596 596 return; 597 597 598 - node = topology_die_id(smp_processor_id()); 598 + node = topology_amd_node_id(smp_processor_id()); 599 599 this_leaf->nb = node_to_amd_nb(node); 600 600 if (this_leaf->nb && !this_leaf->nb->l3_cache.indices) 601 601 amd_calc_l3_indices(this_leaf->nb); ··· 661 661 return i; 662 662 } 663 663 664 - void cacheinfo_amd_init_llc_id(struct cpuinfo_x86 *c) 664 + void cacheinfo_amd_init_llc_id(struct cpuinfo_x86 *c, u16 die_id) 665 665 { 666 666 /* 667 667 * We may have multiple LLCs if L3 caches exist, so check if we ··· 672 672 673 673 if (c->x86 < 0x17) { 674 674 /* LLC is at the node level. */ 675 - c->topo.llc_id = c->topo.die_id; 675 + c->topo.llc_id = die_id; 676 676 } else if (c->x86 == 0x17 && c->x86_model <= 0x1F) { 677 677 /* 678 678 * LLC is at the core complex level.
-4
arch/x86/kernel/cpu/centaur.c
··· 128 128 #endif 129 129 early_init_centaur(c); 130 130 init_intel_cacheinfo(c); 131 - detect_num_cpu_cores(c); 132 - #ifdef CONFIG_X86_32 133 - detect_ht(c); 134 - #endif 135 131 136 132 if (c->cpuid_level > 9) { 137 133 unsigned int eax = cpuid_eax(10);
+20 -111
arch/x86/kernel/cpu/common.c
··· 73 73 u32 elf_hwcap2 __read_mostly; 74 74 75 75 /* Number of siblings per CPU package */ 76 - int smp_num_siblings = 1; 77 - EXPORT_SYMBOL(smp_num_siblings); 76 + unsigned int __max_threads_per_core __ro_after_init = 1; 77 + EXPORT_SYMBOL(__max_threads_per_core); 78 + 79 + unsigned int __max_dies_per_package __ro_after_init = 1; 80 + EXPORT_SYMBOL(__max_dies_per_package); 81 + 82 + unsigned int __max_logical_packages __ro_after_init = 1; 83 + EXPORT_SYMBOL(__max_logical_packages); 84 + 85 + unsigned int __num_cores_per_package __ro_after_init = 1; 86 + EXPORT_SYMBOL(__num_cores_per_package); 87 + 88 + unsigned int __num_threads_per_package __ro_after_init = 1; 89 + EXPORT_SYMBOL(__num_threads_per_package); 78 90 79 91 static struct ppin_info { 80 92 int feature; ··· 802 790 *(s + 1) = '\0'; 803 791 } 804 792 805 - void detect_num_cpu_cores(struct cpuinfo_x86 *c) 806 - { 807 - unsigned int eax, ebx, ecx, edx; 808 - 809 - c->x86_max_cores = 1; 810 - if (!IS_ENABLED(CONFIG_SMP) || c->cpuid_level < 4) 811 - return; 812 - 813 - cpuid_count(4, 0, &eax, &ebx, &ecx, &edx); 814 - if (eax & 0x1f) 815 - c->x86_max_cores = (eax >> 26) + 1; 816 - } 817 - 818 793 void cpu_detect_cache_sizes(struct cpuinfo_x86 *c) 819 794 { 820 795 unsigned int n, dummy, ebx, ecx, edx, l2size; ··· 861 862 pr_info("Last level dTLB entries: 4KB %d, 2MB %d, 4MB %d, 1GB %d\n", 862 863 tlb_lld_4k[ENTRIES], tlb_lld_2m[ENTRIES], 863 864 tlb_lld_4m[ENTRIES], tlb_lld_1g[ENTRIES]); 864 - } 865 - 866 - int detect_ht_early(struct cpuinfo_x86 *c) 867 - { 868 - #ifdef CONFIG_SMP 869 - u32 eax, ebx, ecx, edx; 870 - 871 - if (!cpu_has(c, X86_FEATURE_HT)) 872 - return -1; 873 - 874 - if (cpu_has(c, X86_FEATURE_CMP_LEGACY)) 875 - return -1; 876 - 877 - if (cpu_has(c, X86_FEATURE_XTOPOLOGY)) 878 - return -1; 879 - 880 - cpuid(1, &eax, &ebx, &ecx, &edx); 881 - 882 - smp_num_siblings = (ebx & 0xff0000) >> 16; 883 - if (smp_num_siblings == 1) 884 - pr_info_once("CPU0: Hyper-Threading is disabled\n"); 885 - #endif 886 - return 0; 887 - } 888 - 889 - void detect_ht(struct cpuinfo_x86 *c) 890 - { 891 - #ifdef CONFIG_SMP 892 - int index_msb, core_bits; 893 - 894 - if (detect_ht_early(c) < 0) 895 - return; 896 - 897 - index_msb = get_count_order(smp_num_siblings); 898 - c->topo.pkg_id = apic->phys_pkg_id(c->topo.initial_apicid, index_msb); 899 - 900 - smp_num_siblings = smp_num_siblings / c->x86_max_cores; 901 - 902 - index_msb = get_count_order(smp_num_siblings); 903 - 904 - core_bits = get_count_order(c->x86_max_cores); 905 - 906 - c->topo.core_id = apic->phys_pkg_id(c->topo.initial_apicid, index_msb) & 907 - ((1 << core_bits) - 1); 908 - #endif 909 865 } 910 866 911 867 static void get_cpu_vendor(struct cpuinfo_x86 *c) ··· 1546 1592 get_cpu_address_sizes(c); 1547 1593 cpu_parse_early_param(); 1548 1594 1595 + cpu_init_topology(c); 1596 + 1549 1597 if (this_cpu->c_early_init) 1550 1598 this_cpu->c_early_init(c); 1551 1599 ··· 1559 1603 } else { 1560 1604 setup_clear_cpu_cap(X86_FEATURE_CPUID); 1561 1605 get_cpu_address_sizes(c); 1606 + cpu_init_topology(c); 1562 1607 } 1563 1608 1564 1609 setup_force_cpu_cap(X86_FEATURE_ALWAYS); ··· 1705 1748 1706 1749 get_cpu_address_sizes(c); 1707 1750 1708 - if (c->cpuid_level >= 0x00000001) { 1709 - c->topo.initial_apicid = (cpuid_ebx(1) >> 24) & 0xFF; 1710 - #ifdef CONFIG_X86_32 1711 - # ifdef CONFIG_SMP 1712 - c->topo.apicid = apic->phys_pkg_id(c->topo.initial_apicid, 0); 1713 - # else 1714 - c->topo.apicid = c->topo.initial_apicid; 1715 - # endif 1716 - #endif 1717 - c->topo.pkg_id = c->topo.initial_apicid; 1718 - } 1719 - 1720 1751 get_model_name(c); /* Default name */ 1721 1752 1722 1753 /* ··· 1726 1781 } 1727 1782 1728 1783 /* 1729 - * Validate that ACPI/mptables have the same information about the 1730 - * effective APIC id and update the package map. 1731 - */ 1732 - static void validate_apic_and_package_id(struct cpuinfo_x86 *c) 1733 - { 1734 - #ifdef CONFIG_SMP 1735 - unsigned int cpu = smp_processor_id(); 1736 - u32 apicid; 1737 - 1738 - apicid = apic->cpu_present_to_apicid(cpu); 1739 - 1740 - if (apicid != c->topo.apicid) { 1741 - pr_err(FW_BUG "CPU%u: APIC id mismatch. Firmware: %x APIC: %x\n", 1742 - cpu, apicid, c->topo.initial_apicid); 1743 - } 1744 - BUG_ON(topology_update_package_map(c->topo.pkg_id, cpu)); 1745 - BUG_ON(topology_update_die_map(c->topo.die_id, cpu)); 1746 - #else 1747 - c->topo.logical_pkg_id = 0; 1748 - #endif 1749 - } 1750 - 1751 - /* 1752 1784 * This does the hard work of actually picking apart the CPU stuff... 1753 1785 */ 1754 1786 static void identify_cpu(struct cpuinfo_x86 *c) ··· 1738 1816 c->x86_model = c->x86_stepping = 0; /* So far unknown... */ 1739 1817 c->x86_vendor_id[0] = '\0'; /* Unset */ 1740 1818 c->x86_model_id[0] = '\0'; /* Unset */ 1741 - c->x86_max_cores = 1; 1742 - c->x86_coreid_bits = 0; 1743 - c->topo.cu_id = 0xff; 1744 - c->topo.llc_id = BAD_APICID; 1745 - c->topo.l2c_id = BAD_APICID; 1746 1819 #ifdef CONFIG_X86_64 1747 1820 c->x86_clflush_size = 64; 1748 1821 c->x86_phys_bits = 36; ··· 1756 1839 1757 1840 generic_identify(c); 1758 1841 1842 + cpu_parse_topology(c); 1843 + 1759 1844 if (this_cpu->c_identify) 1760 1845 this_cpu->c_identify(c); 1761 1846 1762 1847 /* Clear/Set all flags overridden by options, after probe */ 1763 1848 apply_forced_caps(c); 1764 - 1765 - #ifdef CONFIG_X86_64 1766 - c->topo.apicid = apic->phys_pkg_id(c->topo.initial_apicid, 0); 1767 - #endif 1768 - 1769 1849 1770 1850 /* 1771 1851 * Set default APIC and TSC_DEADLINE MSR fencing flag. AMD and ··· 1816 1902 sprintf(c->x86_model_id, "%02x/%02x", 1817 1903 c->x86, c->x86_model); 1818 1904 } 1819 - 1820 - #ifdef CONFIG_X86_64 1821 - detect_ht(c); 1822 - #endif 1823 1905 1824 1906 x86_init_rdrand(c); 1825 1907 setup_pku(c); ··· 1908 1998 #ifdef CONFIG_X86_32 1909 1999 enable_sep_cpu(); 1910 2000 #endif 1911 - validate_apic_and_package_id(c); 1912 2001 x86_spec_ctrl_setup_ap(); 1913 2002 update_srbds_msr(); 1914 2003 if (boot_cpu_has_bug(X86_BUG_GDS)) ··· 2256 2347 * identify_boot_cpu() initialized SMT support information, let the 2257 2348 * core code know. 2258 2349 */ 2259 - cpu_smt_set_num_threads(smp_num_siblings, smp_num_siblings); 2350 + cpu_smt_set_num_threads(__max_threads_per_core, __max_threads_per_core); 2260 2351 2261 2352 if (!IS_ENABLED(CONFIG_SMP)) { 2262 2353 pr_info("CPU: ");
+7 -6
arch/x86/kernel/cpu/cpu.h
··· 2 2 #ifndef ARCH_X86_CPU_H 3 3 #define ARCH_X86_CPU_H 4 4 5 + #include <asm/cpu.h> 6 + #include <asm/topology.h> 7 + 8 + #include "topology.h" 9 + 5 10 /* attempt to consolidate cpu attributes */ 6 11 struct cpu_dev { 7 12 const char *c_vendor; ··· 76 71 extern void init_amd_cacheinfo(struct cpuinfo_x86 *c); 77 72 extern void init_hygon_cacheinfo(struct cpuinfo_x86 *c); 78 73 79 - extern void detect_num_cpu_cores(struct cpuinfo_x86 *c); 80 - extern int detect_extended_topology_early(struct cpuinfo_x86 *c); 81 - extern int detect_extended_topology(struct cpuinfo_x86 *c); 82 - extern int detect_ht_early(struct cpuinfo_x86 *c); 83 - extern void detect_ht(struct cpuinfo_x86 *c); 84 74 extern void check_null_seg_clears_base(struct cpuinfo_x86 *c); 85 75 86 - void cacheinfo_amd_init_llc_id(struct cpuinfo_x86 *c); 76 + void cacheinfo_amd_init_llc_id(struct cpuinfo_x86 *c, u16 die_id); 87 77 void cacheinfo_hygon_init_llc_id(struct cpuinfo_x86 *c); 88 78 89 79 unsigned int aperfmperf_get_khz(int cpu); ··· 96 96 mode == SPECTRE_V2_EIBRS_RETPOLINE || 97 97 mode == SPECTRE_V2_EIBRS_LFENCE; 98 98 } 99 + 99 100 #endif /* ARCH_X86_CPU_H */
+44 -3
arch/x86/kernel/cpu/debugfs.c
··· 5 5 #include <asm/apic.h> 6 6 #include <asm/processor.h> 7 7 8 + #include "cpu.h" 9 + 8 10 static int cpu_debug_show(struct seq_file *m, void *p) 9 11 { 10 12 unsigned long cpu = (unsigned long)m->private; ··· 26 24 seq_printf(m, "logical_die_id: %u\n", c->topo.logical_die_id); 27 25 seq_printf(m, "llc_id: %u\n", c->topo.llc_id); 28 26 seq_printf(m, "l2c_id: %u\n", c->topo.l2c_id); 29 - seq_printf(m, "max_cores: %u\n", c->x86_max_cores); 30 - seq_printf(m, "max_die_per_pkg: %u\n", __max_die_per_package); 31 - seq_printf(m, "smp_num_siblings: %u\n", smp_num_siblings); 27 + seq_printf(m, "amd_node_id: %u\n", c->topo.amd_node_id); 28 + seq_printf(m, "amd_nodes_per_pkg: %u\n", topology_amd_nodes_per_pkg()); 29 + seq_printf(m, "num_threads: %u\n", __num_threads_per_package); 30 + seq_printf(m, "num_cores: %u\n", __num_cores_per_package); 31 + seq_printf(m, "max_dies_per_pkg: %u\n", __max_dies_per_package); 32 + seq_printf(m, "max_threads_per_core:%u\n", __max_threads_per_core); 32 33 return 0; 33 34 } 34 35 ··· 47 42 .release = single_release, 48 43 }; 49 44 45 + static int dom_debug_show(struct seq_file *m, void *p) 46 + { 47 + static const char *domain_names[TOPO_MAX_DOMAIN] = { 48 + [TOPO_SMT_DOMAIN] = "Thread", 49 + [TOPO_CORE_DOMAIN] = "Core", 50 + [TOPO_MODULE_DOMAIN] = "Module", 51 + [TOPO_TILE_DOMAIN] = "Tile", 52 + [TOPO_DIE_DOMAIN] = "Die", 53 + [TOPO_DIEGRP_DOMAIN] = "DieGrp", 54 + [TOPO_PKG_DOMAIN] = "Package", 55 + }; 56 + unsigned int dom, nthreads = 1; 57 + 58 + for (dom = 0; dom < TOPO_MAX_DOMAIN; dom++) { 59 + nthreads *= x86_topo_system.dom_size[dom]; 60 + seq_printf(m, "domain: %-10s shift: %u dom_size: %5u max_threads: %5u\n", 61 + domain_names[dom], x86_topo_system.dom_shifts[dom], 62 + x86_topo_system.dom_size[dom], nthreads); 63 + } 64 + return 0; 65 + } 66 + 67 + static int dom_debug_open(struct inode *inode, struct file *file) 68 + { 69 + return single_open(file, dom_debug_show, inode->i_private); 70 + } 71 + 72 + static const struct file_operations dfs_dom_ops = { 73 + .open = dom_debug_open, 74 + .read = seq_read, 75 + .llseek = seq_lseek, 76 + .release = single_release, 77 + }; 78 + 50 79 static __init int cpu_init_debugfs(void) 51 80 { 52 81 struct dentry *dir, *base = debugfs_create_dir("topo", arch_debugfs_dir); 53 82 unsigned long id; 54 83 char name[24]; 84 + 85 + debugfs_create_file("domains", 0444, base, NULL, &dfs_dom_ops); 55 86 56 87 dir = debugfs_create_dir("cpus", base); 57 88 for_each_possible_cpu(id) {
-129
arch/x86/kernel/cpu/hygon.c
··· 18 18 19 19 #include "cpu.h" 20 20 21 - #define APICID_SOCKET_ID_BIT 6 22 - 23 - /* 24 - * nodes_per_socket: Stores the number of nodes per socket. 25 - * Refer to CPUID Fn8000_001E_ECX Node Identifiers[10:8] 26 - */ 27 - static u32 nodes_per_socket = 1; 28 - 29 21 #ifdef CONFIG_NUMA 30 22 /* 31 23 * To workaround broken NUMA config. Read the comment in ··· 40 48 return first_node(node_online_map); /* Shouldn't happen */ 41 49 } 42 50 #endif 43 - 44 - static void hygon_get_topology_early(struct cpuinfo_x86 *c) 45 - { 46 - if (cpu_has(c, X86_FEATURE_TOPOEXT)) 47 - smp_num_siblings = ((cpuid_ebx(0x8000001e) >> 8) & 0xff) + 1; 48 - } 49 - 50 - /* 51 - * Fixup core topology information for 52 - * (1) Hygon multi-node processors 53 - * Assumption: Number of cores in each internal node is the same. 54 - * (2) Hygon processors supporting compute units 55 - */ 56 - static void hygon_get_topology(struct cpuinfo_x86 *c) 57 - { 58 - /* get information required for multi-node processors */ 59 - if (boot_cpu_has(X86_FEATURE_TOPOEXT)) { 60 - int err; 61 - u32 eax, ebx, ecx, edx; 62 - 63 - cpuid(0x8000001e, &eax, &ebx, &ecx, &edx); 64 - 65 - c->topo.die_id = ecx & 0xff; 66 - 67 - c->topo.core_id = ebx & 0xff; 68 - 69 - if (smp_num_siblings > 1) 70 - c->x86_max_cores /= smp_num_siblings; 71 - 72 - /* 73 - * In case leaf B is available, use it to derive 74 - * topology information. 75 - */ 76 - err = detect_extended_topology(c); 77 - if (!err) 78 - c->x86_coreid_bits = get_count_order(c->x86_max_cores); 79 - 80 - /* 81 - * Socket ID is ApicId[6] for the processors with model <= 0x3 82 - * when running on host. 83 - */ 84 - if (!boot_cpu_has(X86_FEATURE_HYPERVISOR) && c->x86_model <= 0x3) 85 - c->topo.pkg_id = c->topo.apicid >> APICID_SOCKET_ID_BIT; 86 - 87 - cacheinfo_hygon_init_llc_id(c); 88 - } else if (cpu_has(c, X86_FEATURE_NODEID_MSR)) { 89 - u64 value; 90 - 91 - rdmsrl(MSR_FAM10H_NODE_ID, value); 92 - c->topo.die_id = value & 7; 93 - c->topo.llc_id = c->topo.die_id; 94 - } else 95 - return; 96 - 97 - if (nodes_per_socket > 1) 98 - set_cpu_cap(c, X86_FEATURE_AMD_DCM); 99 - } 100 - 101 - /* 102 - * On Hygon setup the lower bits of the APIC id distinguish the cores. 103 - * Assumes number of cores is a power of two. 104 - */ 105 - static void hygon_detect_cmp(struct cpuinfo_x86 *c) 106 - { 107 - unsigned int bits; 108 - 109 - bits = c->x86_coreid_bits; 110 - /* Low order bits define the core id (index of core in socket) */ 111 - c->topo.core_id = c->topo.initial_apicid & ((1 << bits)-1); 112 - /* Convert the initial APIC ID into the socket ID */ 113 - c->topo.pkg_id = c->topo.initial_apicid >> bits; 114 - /* Use package ID also for last level cache */ 115 - c->topo.llc_id = c->topo.die_id = c->topo.pkg_id; 116 - } 117 51 118 52 static void srat_detect_node(struct cpuinfo_x86 *c) 119 53 { ··· 91 173 #endif 92 174 } 93 175 94 - static void early_init_hygon_mc(struct cpuinfo_x86 *c) 95 - { 96 - #ifdef CONFIG_SMP 97 - unsigned int bits, ecx; 98 - 99 - /* Multi core CPU? */ 100 - if (c->extended_cpuid_level < 0x80000008) 101 - return; 102 - 103 - ecx = cpuid_ecx(0x80000008); 104 - 105 - c->x86_max_cores = (ecx & 0xff) + 1; 106 - 107 - /* CPU telling us the core id bits shift? */ 108 - bits = (ecx >> 12) & 0xF; 109 - 110 - /* Otherwise recompute */ 111 - if (bits == 0) { 112 - while ((1 << bits) < c->x86_max_cores) 113 - bits++; 114 - } 115 - 116 - c->x86_coreid_bits = bits; 117 - #endif 118 - } 119 - 120 176 static void bsp_init_hygon(struct cpuinfo_x86 *c) 121 177 { 122 178 if (cpu_has(c, X86_FEATURE_CONSTANT_TSC)) { ··· 103 211 104 212 if (cpu_has(c, X86_FEATURE_MWAITX)) 105 213 use_mwaitx_delay(); 106 - 107 - if (boot_cpu_has(X86_FEATURE_TOPOEXT)) { 108 - u32 ecx; 109 - 110 - ecx = cpuid_ecx(0x8000001e); 111 - __max_die_per_package = nodes_per_socket = ((ecx >> 8) & 7) + 1; 112 - } else if (boot_cpu_has(X86_FEATURE_NODEID_MSR)) { 113 - u64 value; 114 - 115 - rdmsrl(MSR_FAM10H_NODE_ID, value); 116 - __max_die_per_package = nodes_per_socket = ((value >> 3) & 7) + 1; 117 - } 118 214 119 215 if (!boot_cpu_has(X86_FEATURE_AMD_SSBD) && 120 216 !boot_cpu_has(X86_FEATURE_VIRT_SSBD)) { ··· 121 241 static void early_init_hygon(struct cpuinfo_x86 *c) 122 242 { 123 243 u32 dummy; 124 - 125 - early_init_hygon_mc(c); 126 244 127 245 set_cpu_cap(c, X86_FEATURE_K8); 128 246 ··· 162 284 * we can set it unconditionally. 163 285 */ 164 286 set_cpu_cap(c, X86_FEATURE_VMMCALL); 165 - 166 - hygon_get_topology_early(c); 167 287 } 168 288 169 289 static void init_hygon(struct cpuinfo_x86 *c) ··· 178 302 179 303 set_cpu_cap(c, X86_FEATURE_REP_GOOD); 180 304 181 - /* get apicid instead of initial apic id from cpuid */ 182 - c->topo.apicid = read_apic_id(); 183 - 184 305 /* 185 306 * XXX someone from Hygon needs to confirm this DTRT 186 307 * ··· 189 316 190 317 cpu_detect_cache_sizes(c); 191 318 192 - hygon_detect_cmp(c); 193 - hygon_get_topology(c); 194 319 srat_detect_node(c); 195 320 196 321 init_hygon_cacheinfo(c);
-25
arch/x86/kernel/cpu/intel.c
··· 401 401 check_memory_type_self_snoop_errata(c); 402 402 403 403 /* 404 - * Get the number of SMT siblings early from the extended topology 405 - * leaf, if available. Otherwise try the legacy SMT detection. 406 - */ 407 - if (detect_extended_topology_early(c) < 0) 408 - detect_ht_early(c); 409 - 410 - /* 411 404 * Adjust the number of physical bits early because it affects the 412 405 * valid bits of the MTRR mask registers. 413 406 */ ··· 602 609 early_init_intel(c); 603 610 604 611 intel_workarounds(c); 605 - 606 - /* 607 - * Detect the extended topology information if available. This 608 - * will reinitialise the initial_apicid which will be used 609 - * in init_intel_cacheinfo() 610 - */ 611 - detect_extended_topology(c); 612 - 613 - if (!cpu_has(c, X86_FEATURE_XTOPOLOGY)) { 614 - /* 615 - * let's use the legacy cpuid vector 0x1 and 0x4 for topology 616 - * detection. 617 - */ 618 - detect_num_cpu_cores(c); 619 - #ifdef CONFIG_X86_32 620 - detect_ht(c); 621 - #endif 622 - } 623 612 624 613 init_intel_cacheinfo(c); 625 614
+2 -2
arch/x86/kernel/cpu/mce/amd.c
··· 1231 1231 return -ENODEV; 1232 1232 1233 1233 if (is_shared_bank(bank)) { 1234 - nb = node_to_amd_nb(topology_die_id(cpu)); 1234 + nb = node_to_amd_nb(topology_amd_node_id(cpu)); 1235 1235 1236 1236 /* threshold descriptor already initialized on this node? */ 1237 1237 if (nb && nb->bank4) { ··· 1335 1335 * The last CPU on this node using the shared bank is going 1336 1336 * away, remove that bank now. 1337 1337 */ 1338 - nb = node_to_amd_nb(topology_die_id(smp_processor_id())); 1338 + nb = node_to_amd_nb(topology_amd_node_id(smp_processor_id())); 1339 1339 nb->bank4 = NULL; 1340 1340 } 1341 1341
+3 -5
arch/x86/kernel/cpu/mce/inject.c
··· 430 430 431 431 static u32 get_nbc_for_node(int node_id) 432 432 { 433 - struct cpuinfo_x86 *c = &boot_cpu_data; 434 433 u32 cores_per_node; 435 434 436 - cores_per_node = (c->x86_max_cores * smp_num_siblings) / amd_get_nodes_per_socket(); 437 - 435 + cores_per_node = topology_num_threads_per_package() / topology_amd_nodes_per_pkg(); 438 436 return cores_per_node * node_id; 439 437 } 440 438 ··· 541 543 if (boot_cpu_has(X86_FEATURE_AMD_DCM) && 542 544 b == 4 && 543 545 boot_cpu_data.x86 < 0x17) { 544 - toggle_nb_mca_mst_cpu(topology_die_id(cpu)); 545 - cpu = get_nbc_for_node(topology_die_id(cpu)); 546 + toggle_nb_mca_mst_cpu(topology_amd_node_id(cpu)); 547 + cpu = get_nbc_for_node(topology_amd_node_id(cpu)); 546 548 } 547 549 548 550 cpus_read_lock();
+1 -1
arch/x86/kernel/cpu/microcode/intel.c
··· 641 641 { 642 642 u64 llc_size = c->x86_cache_size * 1024ULL; 643 643 644 - do_div(llc_size, c->x86_max_cores); 644 + do_div(llc_size, topology_num_cores_per_package()); 645 645 llc_size_per_core = (unsigned int)llc_size; 646 646 } 647 647
+471 -128
arch/x86/kernel/cpu/topology.c
··· 1 - // SPDX-License-Identifier: GPL-2.0 1 + // SPDX-License-Identifier: GPL-2.0-only 2 2 /* 3 - * Check for extended topology enumeration cpuid leaf 0xb and if it 4 - * exists, use it for populating initial_apicid and cpu topology 5 - * detection. 3 + * CPU/APIC topology 4 + * 5 + * The APIC IDs describe the system topology in multiple domain levels. 6 + * The CPUID topology parser provides the information which part of the 7 + * APIC ID is associated to the individual levels: 8 + * 9 + * [PACKAGE][DIEGRP][DIE][TILE][MODULE][CORE][THREAD] 10 + * 11 + * The root space contains the package (socket) IDs. 12 + * 13 + * Not enumerated levels consume 0 bits space, but conceptually they are 14 + * always represented. If e.g. only CORE and THREAD levels are enumerated 15 + * then the DIE, MODULE and TILE have the same physical ID as the PACKAGE. 16 + * 17 + * If SMT is not supported, then the THREAD domain is still used. It then 18 + * has the same physical ID as the CORE domain and is the only child of 19 + * the core domain. 20 + * 21 + * This allows a unified view on the system independent of the enumerated 22 + * domain levels without requiring any conditionals in the code. 6 23 */ 7 - 24 + #define pr_fmt(fmt) "CPU topo: " fmt 8 25 #include <linux/cpu.h> 26 + 27 + #include <xen/xen.h> 28 + 9 29 #include <asm/apic.h> 10 - #include <asm/memtype.h> 11 - #include <asm/processor.h> 30 + #include <asm/hypervisor.h> 31 + #include <asm/io_apic.h> 32 + #include <asm/mpspec.h> 33 + #include <asm/smp.h> 12 34 13 35 #include "cpu.h" 14 36 15 - /* leaf 0xb SMT level */ 16 - #define SMT_LEVEL 0 37 + /* 38 + * Map cpu index to physical APIC ID 39 + */ 40 + DEFINE_EARLY_PER_CPU_READ_MOSTLY(u32, x86_cpu_to_apicid, BAD_APICID); 41 + DEFINE_EARLY_PER_CPU_READ_MOSTLY(u32, x86_cpu_to_acpiid, CPU_ACPIID_INVALID); 42 + EXPORT_EARLY_PER_CPU_SYMBOL(x86_cpu_to_apicid); 43 + EXPORT_EARLY_PER_CPU_SYMBOL(x86_cpu_to_acpiid); 17 44 18 - /* extended topology sub-leaf types */ 19 - #define INVALID_TYPE 0 20 - #define SMT_TYPE 1 21 - #define CORE_TYPE 2 22 - #define DIE_TYPE 5 45 + /* Bitmap of physically present CPUs. */ 46 + DECLARE_BITMAP(phys_cpu_present_map, MAX_LOCAL_APIC) __read_mostly; 23 47 24 - #define LEAFB_SUBTYPE(ecx) (((ecx) >> 8) & 0xff) 25 - #define BITS_SHIFT_NEXT_LEVEL(eax) ((eax) & 0x1f) 26 - #define LEVEL_MAX_SIBLINGS(ebx) ((ebx) & 0xffff) 48 + /* Used for CPU number allocation and parallel CPU bringup */ 49 + u32 cpuid_to_apicid[] __ro_after_init = { [0 ... NR_CPUS - 1] = BAD_APICID, }; 27 50 28 - unsigned int __max_die_per_package __read_mostly = 1; 29 - EXPORT_SYMBOL(__max_die_per_package); 51 + /* Bitmaps to mark registered APICs at each topology domain */ 52 + static struct { DECLARE_BITMAP(map, MAX_LOCAL_APIC); } apic_maps[TOPO_MAX_DOMAIN] __ro_after_init; 53 + 54 + /* 55 + * Keep track of assigned, disabled and rejected CPUs. Present assigned 56 + * with 1 as CPU #0 is reserved for the boot CPU. 57 + */ 58 + static struct { 59 + unsigned int nr_assigned_cpus; 60 + unsigned int nr_disabled_cpus; 61 + unsigned int nr_rejected_cpus; 62 + u32 boot_cpu_apic_id; 63 + u32 real_bsp_apic_id; 64 + } topo_info __ro_after_init = { 65 + .nr_assigned_cpus = 1, 66 + .boot_cpu_apic_id = BAD_APICID, 67 + .real_bsp_apic_id = BAD_APICID, 68 + }; 69 + 70 + #define domain_weight(_dom) bitmap_weight(apic_maps[_dom].map, MAX_LOCAL_APIC) 71 + 72 + bool arch_match_cpu_phys_id(int cpu, u64 phys_id) 73 + { 74 + return phys_id == (u64)cpuid_to_apicid[cpu]; 75 + } 30 76 31 77 #ifdef CONFIG_SMP 32 - /* 33 - * Check if given CPUID extended topology "leaf" is implemented 34 - */ 35 - static int check_extended_topology_leaf(int leaf) 78 + static void cpu_mark_primary_thread(unsigned int cpu, unsigned int apicid) 36 79 { 37 - unsigned int eax, ebx, ecx, edx; 38 - 39 - cpuid_count(leaf, SMT_LEVEL, &eax, &ebx, &ecx, &edx); 40 - 41 - if (ebx == 0 || (LEAFB_SUBTYPE(ecx) != SMT_TYPE)) 42 - return -1; 43 - 44 - return 0; 80 + if (!(apicid & (__max_threads_per_core - 1))) 81 + cpumask_set_cpu(cpu, &__cpu_primary_thread_mask); 45 82 } 46 - /* 47 - * Return best CPUID Extended Topology Leaf supported 48 - */ 49 - static int detect_extended_topology_leaf(struct cpuinfo_x86 *c) 50 - { 51 - if (c->cpuid_level >= 0x1f) { 52 - if (check_extended_topology_leaf(0x1f) == 0) 53 - return 0x1f; 54 - } 55 - 56 - if (c->cpuid_level >= 0xb) { 57 - if (check_extended_topology_leaf(0xb) == 0) 58 - return 0xb; 59 - } 60 - 61 - return -1; 62 - } 83 + #else 84 + static inline void cpu_mark_primary_thread(unsigned int cpu, unsigned int apicid) { } 63 85 #endif 64 86 65 - int detect_extended_topology_early(struct cpuinfo_x86 *c) 87 + /* 88 + * Convert the APIC ID to a domain level ID by masking out the low bits 89 + * below the domain level @dom. 90 + */ 91 + static inline u32 topo_apicid(u32 apicid, enum x86_topology_domains dom) 66 92 { 67 - #ifdef CONFIG_SMP 68 - unsigned int eax, ebx, ecx, edx; 69 - int leaf; 70 - 71 - leaf = detect_extended_topology_leaf(c); 72 - if (leaf < 0) 73 - return -1; 74 - 75 - set_cpu_cap(c, X86_FEATURE_XTOPOLOGY); 76 - 77 - cpuid_count(leaf, SMT_LEVEL, &eax, &ebx, &ecx, &edx); 78 - /* 79 - * initial apic id, which also represents 32-bit extended x2apic id. 80 - */ 81 - c->topo.initial_apicid = edx; 82 - smp_num_siblings = max_t(int, smp_num_siblings, LEVEL_MAX_SIBLINGS(ebx)); 83 - #endif 84 - return 0; 93 + if (dom == TOPO_SMT_DOMAIN) 94 + return apicid; 95 + return apicid & (UINT_MAX << x86_topo_system.dom_shifts[dom - 1]); 85 96 } 86 97 87 - /* 88 - * Check for extended topology enumeration cpuid leaf, and if it 89 - * exists, use it for populating initial_apicid and cpu topology 90 - * detection. 91 - */ 92 - int detect_extended_topology(struct cpuinfo_x86 *c) 98 + static int topo_lookup_cpuid(u32 apic_id) 93 99 { 94 - #ifdef CONFIG_SMP 95 - unsigned int eax, ebx, ecx, edx, sub_index; 96 - unsigned int ht_mask_width, core_plus_mask_width, die_plus_mask_width; 97 - unsigned int core_select_mask, core_level_siblings; 98 - unsigned int die_select_mask, die_level_siblings; 99 - unsigned int pkg_mask_width; 100 - bool die_level_present = false; 101 - int leaf; 100 + int i; 102 101 103 - leaf = detect_extended_topology_leaf(c); 104 - if (leaf < 0) 105 - return -1; 102 + /* CPU# to APICID mapping is persistent once it is established */ 103 + for (i = 0; i < topo_info.nr_assigned_cpus; i++) { 104 + if (cpuid_to_apicid[i] == apic_id) 105 + return i; 106 + } 107 + return -ENODEV; 108 + } 106 109 110 + static __init int topo_get_cpunr(u32 apic_id) 111 + { 112 + int cpu = topo_lookup_cpuid(apic_id); 113 + 114 + if (cpu >= 0) 115 + return cpu; 116 + 117 + return topo_info.nr_assigned_cpus++; 118 + } 119 + 120 + static void topo_set_cpuids(unsigned int cpu, u32 apic_id, u32 acpi_id) 121 + { 122 + #if defined(CONFIG_SMP) || defined(CONFIG_X86_64) 123 + early_per_cpu(x86_cpu_to_apicid, cpu) = apic_id; 124 + early_per_cpu(x86_cpu_to_acpiid, cpu) = acpi_id; 125 + #endif 126 + set_cpu_possible(cpu, true); 127 + set_cpu_present(cpu, true); 128 + } 129 + 130 + static __init bool check_for_real_bsp(u32 apic_id) 131 + { 107 132 /* 108 - * Populate HT related information from sub-leaf level 0. 133 + * There is no real good way to detect whether this a kdump() 134 + * kernel, but except on the Voyager SMP monstrosity which is not 135 + * longer supported, the real BSP APIC ID is the first one which is 136 + * enumerated by firmware. That allows to detect whether the boot 137 + * CPU is the real BSP. If it is not, then do not register the APIC 138 + * because sending INIT to the real BSP would reset the whole 139 + * system. 140 + * 141 + * The first APIC ID which is enumerated by firmware is detectable 142 + * because the boot CPU APIC ID is registered before that without 143 + * invoking this code. 109 144 */ 110 - cpuid_count(leaf, SMT_LEVEL, &eax, &ebx, &ecx, &edx); 111 - c->topo.initial_apicid = edx; 112 - core_level_siblings = LEVEL_MAX_SIBLINGS(ebx); 113 - smp_num_siblings = max_t(int, smp_num_siblings, LEVEL_MAX_SIBLINGS(ebx)); 114 - core_plus_mask_width = ht_mask_width = BITS_SHIFT_NEXT_LEVEL(eax); 115 - die_level_siblings = LEVEL_MAX_SIBLINGS(ebx); 116 - pkg_mask_width = die_plus_mask_width = BITS_SHIFT_NEXT_LEVEL(eax); 145 + if (topo_info.real_bsp_apic_id != BAD_APICID) 146 + return false; 117 147 118 - sub_index = 1; 119 - while (true) { 120 - cpuid_count(leaf, sub_index, &eax, &ebx, &ecx, &edx); 148 + if (apic_id == topo_info.boot_cpu_apic_id) { 149 + topo_info.real_bsp_apic_id = apic_id; 150 + return false; 151 + } 152 + 153 + pr_warn("Boot CPU APIC ID not the first enumerated APIC ID: %x > %x\n", 154 + topo_info.boot_cpu_apic_id, apic_id); 155 + pr_warn("Crash kernel detected. Disabling real BSP to prevent machine INIT\n"); 156 + 157 + topo_info.real_bsp_apic_id = apic_id; 158 + return true; 159 + } 160 + 161 + static unsigned int topo_unit_count(u32 lvlid, enum x86_topology_domains at_level, 162 + unsigned long *map) 163 + { 164 + unsigned int id, end, cnt = 0; 165 + 166 + /* Calculate the exclusive end */ 167 + end = lvlid + (1U << x86_topo_system.dom_shifts[at_level]); 168 + 169 + /* Unfortunately there is no bitmap_weight_range() */ 170 + for (id = find_next_bit(map, end, lvlid); id < end; id = find_next_bit(map, end, ++id)) 171 + cnt++; 172 + return cnt; 173 + } 174 + 175 + static __init void topo_register_apic(u32 apic_id, u32 acpi_id, bool present) 176 + { 177 + int cpu, dom; 178 + 179 + if (present) { 180 + set_bit(apic_id, phys_cpu_present_map); 121 181 122 182 /* 123 - * Check for the Core type in the implemented sub leaves. 183 + * Double registration is valid in case of the boot CPU 184 + * APIC because that is registered before the enumeration 185 + * of the APICs via firmware parsers or VM guest 186 + * mechanisms. 124 187 */ 125 - if (LEAFB_SUBTYPE(ecx) == CORE_TYPE) { 126 - core_level_siblings = LEVEL_MAX_SIBLINGS(ebx); 127 - core_plus_mask_width = BITS_SHIFT_NEXT_LEVEL(eax); 128 - die_level_siblings = core_level_siblings; 129 - die_plus_mask_width = BITS_SHIFT_NEXT_LEVEL(eax); 130 - } 131 - if (LEAFB_SUBTYPE(ecx) == DIE_TYPE) { 132 - die_level_present = true; 133 - die_level_siblings = LEVEL_MAX_SIBLINGS(ebx); 134 - die_plus_mask_width = BITS_SHIFT_NEXT_LEVEL(eax); 135 - } 136 - 137 - if (LEAFB_SUBTYPE(ecx) != INVALID_TYPE) 138 - pkg_mask_width = BITS_SHIFT_NEXT_LEVEL(eax); 188 + if (apic_id == topo_info.boot_cpu_apic_id) 189 + cpu = 0; 139 190 else 140 - break; 191 + cpu = topo_get_cpunr(apic_id); 141 192 142 - sub_index++; 193 + cpuid_to_apicid[cpu] = apic_id; 194 + topo_set_cpuids(cpu, apic_id, acpi_id); 195 + } else { 196 + u32 pkgid = topo_apicid(apic_id, TOPO_PKG_DOMAIN); 197 + 198 + /* 199 + * Check for present APICs in the same package when running 200 + * on bare metal. Allow the bogosity in a guest. 201 + */ 202 + if (hypervisor_is_type(X86_HYPER_NATIVE) && 203 + topo_unit_count(pkgid, TOPO_PKG_DOMAIN, phys_cpu_present_map)) { 204 + pr_info_once("Ignoring hot-pluggable APIC ID %x in present package.\n", 205 + apic_id); 206 + topo_info.nr_rejected_cpus++; 207 + return; 208 + } 209 + 210 + topo_info.nr_disabled_cpus++; 143 211 } 144 212 145 - core_select_mask = (~(-1 << pkg_mask_width)) >> ht_mask_width; 146 - die_select_mask = (~(-1 << die_plus_mask_width)) >> 147 - core_plus_mask_width; 213 + /* Register present and possible CPUs in the domain maps */ 214 + for (dom = TOPO_SMT_DOMAIN; dom < TOPO_MAX_DOMAIN; dom++) 215 + set_bit(topo_apicid(apic_id, dom), apic_maps[dom].map); 216 + } 148 217 149 - c->topo.core_id = apic->phys_pkg_id(c->topo.initial_apicid, 150 - ht_mask_width) & core_select_mask; 151 - 152 - if (die_level_present) { 153 - c->topo.die_id = apic->phys_pkg_id(c->topo.initial_apicid, 154 - core_plus_mask_width) & die_select_mask; 218 + /** 219 + * topology_register_apic - Register an APIC in early topology maps 220 + * @apic_id: The APIC ID to set up 221 + * @acpi_id: The ACPI ID associated to the APIC 222 + * @present: True if the corresponding CPU is present 223 + */ 224 + void __init topology_register_apic(u32 apic_id, u32 acpi_id, bool present) 225 + { 226 + if (apic_id >= MAX_LOCAL_APIC) { 227 + pr_err_once("APIC ID %x exceeds kernel limit of: %x\n", apic_id, MAX_LOCAL_APIC - 1); 228 + topo_info.nr_rejected_cpus++; 229 + return; 155 230 } 156 231 157 - c->topo.pkg_id = apic->phys_pkg_id(c->topo.initial_apicid, pkg_mask_width); 158 - /* 159 - * Reinit the apicid, now that we have extended initial_apicid. 160 - */ 161 - c->topo.apicid = apic->phys_pkg_id(c->topo.initial_apicid, 0); 232 + if (check_for_real_bsp(apic_id)) { 233 + topo_info.nr_rejected_cpus++; 234 + return; 235 + } 162 236 163 - c->x86_max_cores = (core_level_siblings / smp_num_siblings); 164 - __max_die_per_package = (die_level_siblings / core_level_siblings); 237 + /* CPU numbers exhausted? */ 238 + if (apic_id != topo_info.boot_cpu_apic_id && topo_info.nr_assigned_cpus >= nr_cpu_ids) { 239 + pr_warn_once("CPU limit of %d reached. Ignoring further CPUs\n", nr_cpu_ids); 240 + topo_info.nr_rejected_cpus++; 241 + return; 242 + } 243 + 244 + topo_register_apic(apic_id, acpi_id, present); 245 + } 246 + 247 + /** 248 + * topology_register_boot_apic - Register the boot CPU APIC 249 + * @apic_id: The APIC ID to set up 250 + * 251 + * Separate so CPU #0 can be assigned 252 + */ 253 + void __init topology_register_boot_apic(u32 apic_id) 254 + { 255 + WARN_ON_ONCE(topo_info.boot_cpu_apic_id != BAD_APICID); 256 + 257 + topo_info.boot_cpu_apic_id = apic_id; 258 + topo_register_apic(apic_id, CPU_ACPIID_INVALID, true); 259 + } 260 + 261 + /** 262 + * topology_get_logical_id - Retrieve the logical ID at a given topology domain level 263 + * @apicid: The APIC ID for which to lookup the logical ID 264 + * @at_level: The topology domain level to use 265 + * 266 + * @apicid must be a full APIC ID, not the normalized variant. It's valid to have 267 + * all bits below the domain level specified by @at_level to be clear. So both 268 + * real APIC IDs and backshifted normalized APIC IDs work correctly. 269 + * 270 + * Returns: 271 + * - >= 0: The requested logical ID 272 + * - -ERANGE: @apicid is out of range 273 + * - -ENODEV: @apicid is not registered 274 + */ 275 + int topology_get_logical_id(u32 apicid, enum x86_topology_domains at_level) 276 + { 277 + /* Remove the bits below @at_level to get the proper level ID of @apicid */ 278 + unsigned int lvlid = topo_apicid(apicid, at_level); 279 + 280 + if (lvlid >= MAX_LOCAL_APIC) 281 + return -ERANGE; 282 + if (!test_bit(lvlid, apic_maps[at_level].map)) 283 + return -ENODEV; 284 + /* Get the number of set bits before @lvlid. */ 285 + return bitmap_weight(apic_maps[at_level].map, lvlid); 286 + } 287 + EXPORT_SYMBOL_GPL(topology_get_logical_id); 288 + 289 + /** 290 + * topology_unit_count - Retrieve the count of specified units at a given topology domain level 291 + * @apicid: The APIC ID which specifies the search range 292 + * @which_units: The domain level specifying the units to count 293 + * @at_level: The domain level at which @which_units have to be counted 294 + * 295 + * This returns the number of possible units according to the enumerated 296 + * information. 297 + * 298 + * E.g. topology_count_units(apicid, TOPO_CORE_DOMAIN, TOPO_PKG_DOMAIN) 299 + * counts the number of possible cores in the package to which @apicid 300 + * belongs. 301 + * 302 + * @at_level must obviously be greater than @which_level to produce useful 303 + * results. If @at_level is equal to @which_units the result is 304 + * unsurprisingly 1. If @at_level is less than @which_units the results 305 + * is by definition undefined and the function returns 0. 306 + */ 307 + unsigned int topology_unit_count(u32 apicid, enum x86_topology_domains which_units, 308 + enum x86_topology_domains at_level) 309 + { 310 + /* Remove the bits below @at_level to get the proper level ID of @apicid */ 311 + unsigned int lvlid = topo_apicid(apicid, at_level); 312 + 313 + if (lvlid >= MAX_LOCAL_APIC) 314 + return 0; 315 + if (!test_bit(lvlid, apic_maps[at_level].map)) 316 + return 0; 317 + if (which_units > at_level) 318 + return 0; 319 + if (which_units == at_level) 320 + return 1; 321 + return topo_unit_count(lvlid, at_level, apic_maps[which_units].map); 322 + } 323 + 324 + #ifdef CONFIG_ACPI_HOTPLUG_CPU 325 + /** 326 + * topology_hotplug_apic - Handle a physical hotplugged APIC after boot 327 + * @apic_id: The APIC ID to set up 328 + * @acpi_id: The ACPI ID associated to the APIC 329 + */ 330 + int topology_hotplug_apic(u32 apic_id, u32 acpi_id) 331 + { 332 + int cpu; 333 + 334 + if (apic_id >= MAX_LOCAL_APIC) 335 + return -EINVAL; 336 + 337 + /* Reject if the APIC ID was not registered during enumeration. */ 338 + if (!test_bit(apic_id, apic_maps[TOPO_SMT_DOMAIN].map)) 339 + return -ENODEV; 340 + 341 + cpu = topo_lookup_cpuid(apic_id); 342 + if (cpu < 0) 343 + return -ENOSPC; 344 + 345 + set_bit(apic_id, phys_cpu_present_map); 346 + topo_set_cpuids(cpu, apic_id, acpi_id); 347 + cpu_mark_primary_thread(cpu, apic_id); 348 + return cpu; 349 + } 350 + 351 + /** 352 + * topology_hotunplug_apic - Remove a physical hotplugged APIC after boot 353 + * @cpu: The CPU number for which the APIC ID is removed 354 + */ 355 + void topology_hotunplug_apic(unsigned int cpu) 356 + { 357 + u32 apic_id = cpuid_to_apicid[cpu]; 358 + 359 + if (apic_id == BAD_APICID) 360 + return; 361 + 362 + per_cpu(x86_cpu_to_apicid, cpu) = BAD_APICID; 363 + clear_bit(apic_id, phys_cpu_present_map); 364 + set_cpu_present(cpu, false); 365 + } 165 366 #endif 367 + 368 + #ifdef CONFIG_X86_LOCAL_APIC 369 + static unsigned int max_possible_cpus __initdata = NR_CPUS; 370 + 371 + /** 372 + * topology_apply_cmdline_limits_early - Apply topology command line limits early 373 + * 374 + * Ensure that command line limits are in effect before firmware parsing 375 + * takes place. 376 + */ 377 + void __init topology_apply_cmdline_limits_early(void) 378 + { 379 + unsigned int possible = nr_cpu_ids; 380 + 381 + /* 'maxcpus=0' 'nosmp' 'nolapic' 'disableapic' 'noapic' */ 382 + if (!setup_max_cpus || ioapic_is_disabled || apic_is_disabled) 383 + possible = 1; 384 + 385 + /* 'possible_cpus=N' */ 386 + possible = min_t(unsigned int, max_possible_cpus, possible); 387 + 388 + if (possible < nr_cpu_ids) { 389 + pr_info("Limiting to %u possible CPUs\n", possible); 390 + set_nr_cpu_ids(possible); 391 + } 392 + } 393 + 394 + static __init bool restrict_to_up(void) 395 + { 396 + if (!smp_found_config || ioapic_is_disabled) 397 + return true; 398 + /* 399 + * XEN PV is special as it does not advertise the local APIC 400 + * properly, but provides a fake topology for it so that the 401 + * infrastructure works. So don't apply the restrictions vs. APIC 402 + * here. 403 + */ 404 + if (xen_pv_domain()) 405 + return false; 406 + 407 + return apic_is_disabled; 408 + } 409 + 410 + void __init topology_init_possible_cpus(void) 411 + { 412 + unsigned int assigned = topo_info.nr_assigned_cpus; 413 + unsigned int disabled = topo_info.nr_disabled_cpus; 414 + unsigned int cnta, cntb, cpu, allowed = 1; 415 + unsigned int total = assigned + disabled; 416 + u32 apicid, firstid; 417 + 418 + if (!restrict_to_up()) { 419 + if (WARN_ON_ONCE(assigned > nr_cpu_ids)) { 420 + disabled += assigned - nr_cpu_ids; 421 + assigned = nr_cpu_ids; 422 + } 423 + allowed = min_t(unsigned int, total, nr_cpu_ids); 424 + } 425 + 426 + if (total > allowed) 427 + pr_warn("%u possible CPUs exceed the limit of %u\n", total, allowed); 428 + 429 + assigned = min_t(unsigned int, allowed, assigned); 430 + disabled = allowed - assigned; 431 + 432 + topo_info.nr_assigned_cpus = assigned; 433 + topo_info.nr_disabled_cpus = disabled; 434 + 435 + total_cpus = allowed; 436 + set_nr_cpu_ids(allowed); 437 + 438 + cnta = domain_weight(TOPO_PKG_DOMAIN); 439 + cntb = domain_weight(TOPO_DIE_DOMAIN); 440 + __max_logical_packages = cnta; 441 + __max_dies_per_package = 1U << (get_count_order(cntb) - get_count_order(cnta)); 442 + 443 + pr_info("Max. logical packages: %3u\n", cnta); 444 + pr_info("Max. logical dies: %3u\n", cntb); 445 + pr_info("Max. dies per package: %3u\n", __max_dies_per_package); 446 + 447 + cnta = domain_weight(TOPO_CORE_DOMAIN); 448 + cntb = domain_weight(TOPO_SMT_DOMAIN); 449 + /* 450 + * Can't use order delta here as order(cnta) can be equal 451 + * order(cntb) even if cnta != cntb. 452 + */ 453 + __max_threads_per_core = DIV_ROUND_UP(cntb, cnta); 454 + pr_info("Max. threads per core: %3u\n", __max_threads_per_core); 455 + 456 + firstid = find_first_bit(apic_maps[TOPO_SMT_DOMAIN].map, MAX_LOCAL_APIC); 457 + __num_cores_per_package = topology_unit_count(firstid, TOPO_CORE_DOMAIN, TOPO_PKG_DOMAIN); 458 + pr_info("Num. cores per package: %3u\n", __num_cores_per_package); 459 + __num_threads_per_package = topology_unit_count(firstid, TOPO_SMT_DOMAIN, TOPO_PKG_DOMAIN); 460 + pr_info("Num. threads per package: %3u\n", __num_threads_per_package); 461 + 462 + pr_info("Allowing %u present CPUs plus %u hotplug CPUs\n", assigned, disabled); 463 + if (topo_info.nr_rejected_cpus) 464 + pr_info("Rejected CPUs %u\n", topo_info.nr_rejected_cpus); 465 + 466 + init_cpu_present(cpumask_of(0)); 467 + init_cpu_possible(cpumask_of(0)); 468 + 469 + /* Assign CPU numbers to non-present CPUs */ 470 + for (apicid = 0; disabled; disabled--, apicid++) { 471 + apicid = find_next_andnot_bit(apic_maps[TOPO_SMT_DOMAIN].map, phys_cpu_present_map, 472 + MAX_LOCAL_APIC, apicid); 473 + if (apicid >= MAX_LOCAL_APIC) 474 + break; 475 + cpuid_to_apicid[topo_info.nr_assigned_cpus++] = apicid; 476 + } 477 + 478 + for (cpu = 0; cpu < allowed; cpu++) { 479 + apicid = cpuid_to_apicid[cpu]; 480 + 481 + set_cpu_possible(cpu, true); 482 + 483 + if (apicid == BAD_APICID) 484 + continue; 485 + 486 + cpu_mark_primary_thread(cpu, apicid); 487 + set_cpu_present(cpu, test_bit(apicid, phys_cpu_present_map)); 488 + } 489 + } 490 + 491 + /* 492 + * Late SMP disable after sizing CPU masks when APIC/IOAPIC setup failed. 493 + */ 494 + void __init topology_reset_possible_cpus_up(void) 495 + { 496 + init_cpu_present(cpumask_of(0)); 497 + init_cpu_possible(cpumask_of(0)); 498 + 499 + bitmap_zero(phys_cpu_present_map, MAX_LOCAL_APIC); 500 + if (topo_info.boot_cpu_apic_id != BAD_APICID) 501 + set_bit(topo_info.boot_cpu_apic_id, phys_cpu_present_map); 502 + } 503 + 504 + static int __init setup_possible_cpus(char *str) 505 + { 506 + get_option(&str, &max_possible_cpus); 166 507 return 0; 167 508 } 509 + early_param("possible_cpus", setup_possible_cpus); 510 + #endif
+67
arch/x86/kernel/cpu/topology.h
··· 1 + /* SPDX-License-Identifier: GPL-2.0 */ 2 + #ifndef ARCH_X86_TOPOLOGY_H 3 + #define ARCH_X86_TOPOLOGY_H 4 + 5 + struct topo_scan { 6 + struct cpuinfo_x86 *c; 7 + unsigned int dom_shifts[TOPO_MAX_DOMAIN]; 8 + unsigned int dom_ncpus[TOPO_MAX_DOMAIN]; 9 + 10 + /* Legacy CPUID[1]:EBX[23:16] number of logical processors */ 11 + unsigned int ebx1_nproc_shift; 12 + 13 + /* AMD specific node ID which cannot be mapped into APIC space. */ 14 + u16 amd_nodes_per_pkg; 15 + u16 amd_node_id; 16 + }; 17 + 18 + void cpu_init_topology(struct cpuinfo_x86 *c); 19 + void cpu_parse_topology(struct cpuinfo_x86 *c); 20 + void topology_set_dom(struct topo_scan *tscan, enum x86_topology_domains dom, 21 + unsigned int shift, unsigned int ncpus); 22 + bool cpu_parse_topology_ext(struct topo_scan *tscan); 23 + void cpu_parse_topology_amd(struct topo_scan *tscan); 24 + void cpu_topology_fixup_amd(struct topo_scan *tscan); 25 + 26 + static inline u32 topo_shift_apicid(u32 apicid, enum x86_topology_domains dom) 27 + { 28 + if (dom == TOPO_SMT_DOMAIN) 29 + return apicid; 30 + return apicid >> x86_topo_system.dom_shifts[dom - 1]; 31 + } 32 + 33 + static inline u32 topo_relative_domain_id(u32 apicid, enum x86_topology_domains dom) 34 + { 35 + if (dom != TOPO_SMT_DOMAIN) 36 + apicid >>= x86_topo_system.dom_shifts[dom - 1]; 37 + return apicid & (x86_topo_system.dom_size[dom] - 1); 38 + } 39 + 40 + static inline u32 topo_domain_mask(enum x86_topology_domains dom) 41 + { 42 + return (1U << x86_topo_system.dom_shifts[dom]) - 1; 43 + } 44 + 45 + /* 46 + * Update a domain level after the fact without propagating. Used to fixup 47 + * broken CPUID enumerations. 48 + */ 49 + static inline void topology_update_dom(struct topo_scan *tscan, enum x86_topology_domains dom, 50 + unsigned int shift, unsigned int ncpus) 51 + { 52 + tscan->dom_shifts[dom] = shift; 53 + tscan->dom_ncpus[dom] = ncpus; 54 + } 55 + 56 + #ifdef CONFIG_X86_LOCAL_APIC 57 + unsigned int topology_unit_count(u32 apicid, enum x86_topology_domains which_units, 58 + enum x86_topology_domains at_level); 59 + #else 60 + static inline unsigned int topology_unit_count(u32 apicid, enum x86_topology_domains which_units, 61 + enum x86_topology_domains at_level) 62 + { 63 + return 1; 64 + } 65 + #endif 66 + 67 + #endif /* ARCH_X86_TOPOLOGY_H */
+183
arch/x86/kernel/cpu/topology_amd.c
··· 1 + // SPDX-License-Identifier: GPL-2.0 2 + #include <linux/cpu.h> 3 + 4 + #include <asm/apic.h> 5 + #include <asm/memtype.h> 6 + #include <asm/processor.h> 7 + 8 + #include "cpu.h" 9 + 10 + static bool parse_8000_0008(struct topo_scan *tscan) 11 + { 12 + struct { 13 + // ecx 14 + u32 cpu_nthreads : 8, // Number of physical threads - 1 15 + : 4, // Reserved 16 + apicid_coreid_len : 4, // Number of thread core ID bits (shift) in APIC ID 17 + perf_tsc_len : 2, // Performance time-stamp counter size 18 + : 14; // Reserved 19 + } ecx; 20 + unsigned int sft; 21 + 22 + if (tscan->c->extended_cpuid_level < 0x80000008) 23 + return false; 24 + 25 + cpuid_leaf_reg(0x80000008, CPUID_ECX, &ecx); 26 + 27 + /* If the thread bits are 0, then get the shift value from ecx.cpu_nthreads */ 28 + sft = ecx.apicid_coreid_len; 29 + if (!sft) 30 + sft = get_count_order(ecx.cpu_nthreads + 1); 31 + 32 + topology_set_dom(tscan, TOPO_SMT_DOMAIN, sft, ecx.cpu_nthreads + 1); 33 + return true; 34 + } 35 + 36 + static void store_node(struct topo_scan *tscan, unsigned int nr_nodes, u16 node_id) 37 + { 38 + /* 39 + * Starting with Fam 17h the DIE domain could probably be used to 40 + * retrieve the node info on AMD/HYGON. Analysis of CPUID dumps 41 + * suggests it's the topmost bit(s) of the CPU cores area, but 42 + * that's guess work and neither enumerated nor documented. 43 + * 44 + * Up to Fam 16h this does not work at all and the legacy node ID 45 + * has to be used. 46 + */ 47 + tscan->amd_nodes_per_pkg = nr_nodes; 48 + tscan->amd_node_id = node_id; 49 + } 50 + 51 + static bool parse_8000_001e(struct topo_scan *tscan, bool has_0xb) 52 + { 53 + struct { 54 + // eax 55 + u32 ext_apic_id : 32; // Extended APIC ID 56 + // ebx 57 + u32 core_id : 8, // Unique per-socket logical core unit ID 58 + core_nthreads : 8, // #Threads per core (zero-based) 59 + : 16; // Reserved 60 + // ecx 61 + u32 node_id : 8, // Node (die) ID of invoking logical CPU 62 + nnodes_per_socket : 3, // #nodes in invoking logical CPU's package/socket 63 + : 21; // Reserved 64 + // edx 65 + u32 : 32; // Reserved 66 + } leaf; 67 + 68 + if (!boot_cpu_has(X86_FEATURE_TOPOEXT)) 69 + return false; 70 + 71 + cpuid_leaf(0x8000001e, &leaf); 72 + 73 + tscan->c->topo.initial_apicid = leaf.ext_apic_id; 74 + 75 + /* 76 + * If leaf 0xb is available, then SMT shift is set already. If not 77 + * take it from ecx.threads_per_core and use topo_update_dom() - 78 + * topology_set_dom() would propagate and overwrite the already 79 + * propagated CORE level. 80 + */ 81 + if (!has_0xb) { 82 + unsigned int nthreads = leaf.core_nthreads + 1; 83 + 84 + topology_update_dom(tscan, TOPO_SMT_DOMAIN, get_count_order(nthreads), nthreads); 85 + } 86 + 87 + store_node(tscan, leaf.nnodes_per_socket + 1, leaf.node_id); 88 + 89 + if (tscan->c->x86_vendor == X86_VENDOR_AMD) { 90 + if (tscan->c->x86 == 0x15) 91 + tscan->c->topo.cu_id = leaf.core_id; 92 + 93 + cacheinfo_amd_init_llc_id(tscan->c, leaf.node_id); 94 + } else { 95 + /* 96 + * Package ID is ApicId[6..] on certain Hygon CPUs. See 97 + * commit e0ceeae708ce for explanation. The topology info 98 + * is screwed up: The package shift is always 6 and the 99 + * node ID is bit [4:5]. 100 + */ 101 + if (!boot_cpu_has(X86_FEATURE_HYPERVISOR) && tscan->c->x86_model <= 0x3) { 102 + topology_set_dom(tscan, TOPO_CORE_DOMAIN, 6, 103 + tscan->dom_ncpus[TOPO_CORE_DOMAIN]); 104 + } 105 + cacheinfo_hygon_init_llc_id(tscan->c); 106 + } 107 + return true; 108 + } 109 + 110 + static bool parse_fam10h_node_id(struct topo_scan *tscan) 111 + { 112 + struct { 113 + union { 114 + u64 node_id : 3, 115 + nodes_per_pkg : 3, 116 + unused : 58; 117 + u64 msr; 118 + }; 119 + } nid; 120 + 121 + if (!boot_cpu_has(X86_FEATURE_NODEID_MSR)) 122 + return false; 123 + 124 + rdmsrl(MSR_FAM10H_NODE_ID, nid.msr); 125 + store_node(tscan, nid.nodes_per_pkg + 1, nid.node_id); 126 + tscan->c->topo.llc_id = nid.node_id; 127 + return true; 128 + } 129 + 130 + static void legacy_set_llc(struct topo_scan *tscan) 131 + { 132 + unsigned int apicid = tscan->c->topo.initial_apicid; 133 + 134 + /* parse_8000_0008() set everything up except llc_id */ 135 + tscan->c->topo.llc_id = apicid >> tscan->dom_shifts[TOPO_CORE_DOMAIN]; 136 + } 137 + 138 + static void parse_topology_amd(struct topo_scan *tscan) 139 + { 140 + bool has_0xb = false; 141 + 142 + /* 143 + * If the extended topology leaf 0x8000_001e is available 144 + * try to get SMT and CORE shift from leaf 0xb first, then 145 + * try to get the CORE shift from leaf 0x8000_0008. 146 + */ 147 + if (cpu_feature_enabled(X86_FEATURE_TOPOEXT)) 148 + has_0xb = cpu_parse_topology_ext(tscan); 149 + 150 + if (!has_0xb && !parse_8000_0008(tscan)) 151 + return; 152 + 153 + /* Prefer leaf 0x8000001e if available */ 154 + if (parse_8000_001e(tscan, has_0xb)) 155 + return; 156 + 157 + /* Try the NODEID MSR */ 158 + if (parse_fam10h_node_id(tscan)) 159 + return; 160 + 161 + legacy_set_llc(tscan); 162 + } 163 + 164 + void cpu_parse_topology_amd(struct topo_scan *tscan) 165 + { 166 + tscan->amd_nodes_per_pkg = 1; 167 + parse_topology_amd(tscan); 168 + 169 + if (tscan->amd_nodes_per_pkg > 1) 170 + set_cpu_cap(tscan->c, X86_FEATURE_AMD_DCM); 171 + } 172 + 173 + void cpu_topology_fixup_amd(struct topo_scan *tscan) 174 + { 175 + struct cpuinfo_x86 *c = tscan->c; 176 + 177 + /* 178 + * Adjust the core_id relative to the node when there is more than 179 + * one node. 180 + */ 181 + if (tscan->c->x86 < 0x17 && tscan->amd_nodes_per_pkg > 1) 182 + c->topo.core_id %= tscan->dom_ncpus[TOPO_CORE_DOMAIN] / tscan->amd_nodes_per_pkg; 183 + }
+218
arch/x86/kernel/cpu/topology_common.c
··· 1 + // SPDX-License-Identifier: GPL-2.0 2 + #include <linux/cpu.h> 3 + 4 + #include <xen/xen.h> 5 + 6 + #include <asm/apic.h> 7 + #include <asm/processor.h> 8 + #include <asm/smp.h> 9 + 10 + #include "cpu.h" 11 + 12 + struct x86_topology_system x86_topo_system __ro_after_init; 13 + EXPORT_SYMBOL_GPL(x86_topo_system); 14 + 15 + unsigned int __amd_nodes_per_pkg __ro_after_init; 16 + EXPORT_SYMBOL_GPL(__amd_nodes_per_pkg); 17 + 18 + void topology_set_dom(struct topo_scan *tscan, enum x86_topology_domains dom, 19 + unsigned int shift, unsigned int ncpus) 20 + { 21 + topology_update_dom(tscan, dom, shift, ncpus); 22 + 23 + /* Propagate to the upper levels */ 24 + for (dom++; dom < TOPO_MAX_DOMAIN; dom++) { 25 + tscan->dom_shifts[dom] = tscan->dom_shifts[dom - 1]; 26 + tscan->dom_ncpus[dom] = tscan->dom_ncpus[dom - 1]; 27 + } 28 + } 29 + 30 + static unsigned int __maybe_unused parse_num_cores_legacy(struct cpuinfo_x86 *c) 31 + { 32 + struct { 33 + u32 cache_type : 5, 34 + unused : 21, 35 + ncores : 6; 36 + } eax; 37 + 38 + if (c->cpuid_level < 4) 39 + return 1; 40 + 41 + cpuid_subleaf_reg(4, 0, CPUID_EAX, &eax); 42 + if (!eax.cache_type) 43 + return 1; 44 + 45 + return eax.ncores + 1; 46 + } 47 + 48 + static void parse_legacy(struct topo_scan *tscan) 49 + { 50 + unsigned int cores, core_shift, smt_shift = 0; 51 + struct cpuinfo_x86 *c = tscan->c; 52 + 53 + cores = parse_num_cores_legacy(c); 54 + core_shift = get_count_order(cores); 55 + 56 + if (cpu_has(c, X86_FEATURE_HT)) { 57 + if (!WARN_ON_ONCE(tscan->ebx1_nproc_shift < core_shift)) 58 + smt_shift = tscan->ebx1_nproc_shift - core_shift; 59 + /* 60 + * The parser expects leaf 0xb/0x1f format, which means 61 + * the number of logical processors at core level is 62 + * counting threads. 63 + */ 64 + core_shift += smt_shift; 65 + cores <<= smt_shift; 66 + } 67 + 68 + topology_set_dom(tscan, TOPO_SMT_DOMAIN, smt_shift, 1U << smt_shift); 69 + topology_set_dom(tscan, TOPO_CORE_DOMAIN, core_shift, cores); 70 + } 71 + 72 + static bool fake_topology(struct topo_scan *tscan) 73 + { 74 + /* 75 + * Preset the CORE level shift for CPUID less systems and XEN_PV, 76 + * which has useless CPUID information. 77 + */ 78 + topology_set_dom(tscan, TOPO_SMT_DOMAIN, 0, 1); 79 + topology_set_dom(tscan, TOPO_CORE_DOMAIN, 0, 1); 80 + 81 + return tscan->c->cpuid_level < 1; 82 + } 83 + 84 + static void parse_topology(struct topo_scan *tscan, bool early) 85 + { 86 + const struct cpuinfo_topology topo_defaults = { 87 + .cu_id = 0xff, 88 + .llc_id = BAD_APICID, 89 + .l2c_id = BAD_APICID, 90 + }; 91 + struct cpuinfo_x86 *c = tscan->c; 92 + struct { 93 + u32 unused0 : 16, 94 + nproc : 8, 95 + apicid : 8; 96 + } ebx; 97 + 98 + c->topo = topo_defaults; 99 + 100 + if (fake_topology(tscan)) 101 + return; 102 + 103 + /* Preset Initial APIC ID from CPUID leaf 1 */ 104 + cpuid_leaf_reg(1, CPUID_EBX, &ebx); 105 + c->topo.initial_apicid = ebx.apicid; 106 + 107 + /* 108 + * The initial invocation from early_identify_cpu() happens before 109 + * the APIC is mapped or X2APIC enabled. For establishing the 110 + * topology, that's not required. Use the initial APIC ID. 111 + */ 112 + if (early) 113 + c->topo.apicid = c->topo.initial_apicid; 114 + else 115 + c->topo.apicid = read_apic_id(); 116 + 117 + /* The above is sufficient for UP */ 118 + if (!IS_ENABLED(CONFIG_SMP)) 119 + return; 120 + 121 + tscan->ebx1_nproc_shift = get_count_order(ebx.nproc); 122 + 123 + switch (c->x86_vendor) { 124 + case X86_VENDOR_AMD: 125 + if (IS_ENABLED(CONFIG_CPU_SUP_AMD)) 126 + cpu_parse_topology_amd(tscan); 127 + break; 128 + case X86_VENDOR_CENTAUR: 129 + case X86_VENDOR_ZHAOXIN: 130 + parse_legacy(tscan); 131 + break; 132 + case X86_VENDOR_INTEL: 133 + if (!IS_ENABLED(CONFIG_CPU_SUP_INTEL) || !cpu_parse_topology_ext(tscan)) 134 + parse_legacy(tscan); 135 + break; 136 + case X86_VENDOR_HYGON: 137 + if (IS_ENABLED(CONFIG_CPU_SUP_HYGON)) 138 + cpu_parse_topology_amd(tscan); 139 + break; 140 + } 141 + } 142 + 143 + static void topo_set_ids(struct topo_scan *tscan) 144 + { 145 + struct cpuinfo_x86 *c = tscan->c; 146 + u32 apicid = c->topo.apicid; 147 + 148 + c->topo.pkg_id = topo_shift_apicid(apicid, TOPO_PKG_DOMAIN); 149 + c->topo.die_id = topo_shift_apicid(apicid, TOPO_DIE_DOMAIN); 150 + 151 + c->topo.logical_pkg_id = topology_get_logical_id(apicid, TOPO_PKG_DOMAIN); 152 + c->topo.logical_die_id = topology_get_logical_id(apicid, TOPO_DIE_DOMAIN); 153 + 154 + /* Package relative core ID */ 155 + c->topo.core_id = (apicid & topo_domain_mask(TOPO_PKG_DOMAIN)) >> 156 + x86_topo_system.dom_shifts[TOPO_SMT_DOMAIN]; 157 + 158 + c->topo.amd_node_id = tscan->amd_node_id; 159 + 160 + if (c->x86_vendor == X86_VENDOR_AMD) 161 + cpu_topology_fixup_amd(tscan); 162 + } 163 + 164 + void cpu_parse_topology(struct cpuinfo_x86 *c) 165 + { 166 + unsigned int dom, cpu = smp_processor_id(); 167 + struct topo_scan tscan = { .c = c, }; 168 + 169 + parse_topology(&tscan, false); 170 + 171 + if (IS_ENABLED(CONFIG_X86_LOCAL_APIC)) { 172 + if (c->topo.initial_apicid != c->topo.apicid) { 173 + pr_err(FW_BUG "CPU%4u: APIC ID mismatch. CPUID: 0x%04x APIC: 0x%04x\n", 174 + cpu, c->topo.initial_apicid, c->topo.apicid); 175 + } 176 + 177 + if (c->topo.apicid != cpuid_to_apicid[cpu]) { 178 + pr_err(FW_BUG "CPU%4u: APIC ID mismatch. Firmware: 0x%04x APIC: 0x%04x\n", 179 + cpu, cpuid_to_apicid[cpu], c->topo.apicid); 180 + } 181 + } 182 + 183 + for (dom = TOPO_SMT_DOMAIN; dom < TOPO_MAX_DOMAIN; dom++) { 184 + if (tscan.dom_shifts[dom] == x86_topo_system.dom_shifts[dom]) 185 + continue; 186 + pr_err(FW_BUG "CPU%d: Topology domain %u shift %u != %u\n", cpu, dom, 187 + tscan.dom_shifts[dom], x86_topo_system.dom_shifts[dom]); 188 + } 189 + 190 + topo_set_ids(&tscan); 191 + } 192 + 193 + void __init cpu_init_topology(struct cpuinfo_x86 *c) 194 + { 195 + struct topo_scan tscan = { .c = c, }; 196 + unsigned int dom, sft; 197 + 198 + parse_topology(&tscan, true); 199 + 200 + /* Copy the shift values and calculate the unit sizes. */ 201 + memcpy(x86_topo_system.dom_shifts, tscan.dom_shifts, sizeof(x86_topo_system.dom_shifts)); 202 + 203 + dom = TOPO_SMT_DOMAIN; 204 + x86_topo_system.dom_size[dom] = 1U << x86_topo_system.dom_shifts[dom]; 205 + 206 + for (dom++; dom < TOPO_MAX_DOMAIN; dom++) { 207 + sft = x86_topo_system.dom_shifts[dom] - x86_topo_system.dom_shifts[dom - 1]; 208 + x86_topo_system.dom_size[dom] = 1U << sft; 209 + } 210 + 211 + topo_set_ids(&tscan); 212 + 213 + /* 214 + * AMD systems have Nodes per package which cannot be mapped to 215 + * APIC ID. 216 + */ 217 + __amd_nodes_per_pkg = tscan.amd_nodes_per_pkg; 218 + }
+130
arch/x86/kernel/cpu/topology_ext.c
··· 1 + // SPDX-License-Identifier: GPL-2.0 2 + #include <linux/cpu.h> 3 + 4 + #include <asm/apic.h> 5 + #include <asm/memtype.h> 6 + #include <asm/processor.h> 7 + 8 + #include "cpu.h" 9 + 10 + enum topo_types { 11 + INVALID_TYPE = 0, 12 + SMT_TYPE = 1, 13 + CORE_TYPE = 2, 14 + MAX_TYPE_0B = 3, 15 + MODULE_TYPE = 3, 16 + TILE_TYPE = 4, 17 + DIE_TYPE = 5, 18 + DIEGRP_TYPE = 6, 19 + MAX_TYPE_1F = 7, 20 + }; 21 + 22 + /* 23 + * Use a lookup table for the case that there are future types > 6 which 24 + * describe an intermediate domain level which does not exist today. 25 + */ 26 + static const unsigned int topo_domain_map_0b_1f[MAX_TYPE_1F] = { 27 + [SMT_TYPE] = TOPO_SMT_DOMAIN, 28 + [CORE_TYPE] = TOPO_CORE_DOMAIN, 29 + [MODULE_TYPE] = TOPO_MODULE_DOMAIN, 30 + [TILE_TYPE] = TOPO_TILE_DOMAIN, 31 + [DIE_TYPE] = TOPO_DIE_DOMAIN, 32 + [DIEGRP_TYPE] = TOPO_DIEGRP_DOMAIN, 33 + }; 34 + 35 + static inline bool topo_subleaf(struct topo_scan *tscan, u32 leaf, u32 subleaf, 36 + unsigned int *last_dom) 37 + { 38 + unsigned int dom, maxtype; 39 + const unsigned int *map; 40 + struct { 41 + // eax 42 + u32 x2apic_shift : 5, // Number of bits to shift APIC ID right 43 + // for the topology ID at the next level 44 + : 27; // Reserved 45 + // ebx 46 + u32 num_processors : 16, // Number of processors at current level 47 + : 16; // Reserved 48 + // ecx 49 + u32 level : 8, // Current topology level. Same as sub leaf number 50 + type : 8, // Level type. If 0, invalid 51 + : 16; // Reserved 52 + // edx 53 + u32 x2apic_id : 32; // X2APIC ID of the current logical processor 54 + } sl; 55 + 56 + switch (leaf) { 57 + case 0x0b: maxtype = MAX_TYPE_0B; map = topo_domain_map_0b_1f; break; 58 + case 0x1f: maxtype = MAX_TYPE_1F; map = topo_domain_map_0b_1f; break; 59 + default: return false; 60 + } 61 + 62 + cpuid_subleaf(leaf, subleaf, &sl); 63 + 64 + if (!sl.num_processors || sl.type == INVALID_TYPE) 65 + return false; 66 + 67 + if (sl.type >= maxtype) { 68 + pr_err_once("Topology: leaf 0x%x:%d Unknown domain type %u\n", 69 + leaf, subleaf, sl.type); 70 + /* 71 + * It really would have been too obvious to make the domain 72 + * type space sparse and leave a few reserved types between 73 + * the points which might change instead of following the 74 + * usual "this can be fixed in software" principle. 75 + */ 76 + dom = *last_dom + 1; 77 + } else { 78 + dom = map[sl.type]; 79 + *last_dom = dom; 80 + } 81 + 82 + if (!dom) { 83 + tscan->c->topo.initial_apicid = sl.x2apic_id; 84 + } else if (tscan->c->topo.initial_apicid != sl.x2apic_id) { 85 + pr_warn_once(FW_BUG "CPUID leaf 0x%x subleaf %d APIC ID mismatch %x != %x\n", 86 + leaf, subleaf, tscan->c->topo.initial_apicid, sl.x2apic_id); 87 + } 88 + 89 + topology_set_dom(tscan, dom, sl.x2apic_shift, sl.num_processors); 90 + return true; 91 + } 92 + 93 + static bool parse_topology_leaf(struct topo_scan *tscan, u32 leaf) 94 + { 95 + unsigned int last_dom; 96 + u32 subleaf; 97 + 98 + /* Read all available subleafs and populate the levels */ 99 + for (subleaf = 0, last_dom = 0; topo_subleaf(tscan, leaf, subleaf, &last_dom); subleaf++); 100 + 101 + /* If subleaf 0 failed to parse, give up */ 102 + if (!subleaf) 103 + return false; 104 + 105 + /* 106 + * There are machines in the wild which have shift 0 in the subleaf 107 + * 0, but advertise 2 logical processors at that level. They are 108 + * truly SMT. 109 + */ 110 + if (!tscan->dom_shifts[TOPO_SMT_DOMAIN] && tscan->dom_ncpus[TOPO_SMT_DOMAIN] > 1) { 111 + unsigned int sft = get_count_order(tscan->dom_ncpus[TOPO_SMT_DOMAIN]); 112 + 113 + pr_warn_once(FW_BUG "CPUID leaf 0x%x subleaf 0 has shift level 0 but %u CPUs. Fixing it up.\n", 114 + leaf, tscan->dom_ncpus[TOPO_SMT_DOMAIN]); 115 + topology_update_dom(tscan, TOPO_SMT_DOMAIN, sft, tscan->dom_ncpus[TOPO_SMT_DOMAIN]); 116 + } 117 + 118 + set_cpu_cap(tscan->c, X86_FEATURE_XTOPOLOGY); 119 + return true; 120 + } 121 + 122 + bool cpu_parse_topology_ext(struct topo_scan *tscan) 123 + { 124 + /* Intel: Try leaf 0x1F first. */ 125 + if (tscan->c->cpuid_level >= 0x1f && parse_topology_leaf(tscan, 0x1f)) 126 + return true; 127 + 128 + /* Intel/AMD: Fall back to leaf 0xB if available */ 129 + return tscan->c->cpuid_level >= 0x0b && parse_topology_leaf(tscan, 0x0b); 130 + }
-4
arch/x86/kernel/cpu/zhaoxin.c
··· 71 71 { 72 72 early_init_zhaoxin(c); 73 73 init_intel_cacheinfo(c); 74 - detect_num_cpu_cores(c); 75 - #ifdef CONFIG_X86_32 76 - detect_ht(c); 77 - #endif 78 74 79 75 if (c->cpuid_level > 9) { 80 76 unsigned int eax = cpuid_eax(10);
+2 -2
arch/x86/kernel/devicetree.c
··· 136 136 pr_warn("%pOF: missing local APIC ID\n", dn); 137 137 continue; 138 138 } 139 - generic_processor_info(apic_id); 139 + topology_register_apic(apic_id, CPU_ACPIID_INVALID, true); 140 140 } 141 141 } 142 142 ··· 302 302 } 303 303 #endif 304 304 305 - void __init x86_dtb_init(void) 305 + void __init x86_dtb_parse_smp_config(void) 306 306 { 307 307 if (!of_have_populated_dt()) 308 308 return;
+16 -14
arch/x86/kernel/jailhouse.c
··· 89 89 #endif 90 90 } 91 91 92 - static void __init jailhouse_get_smp_config(unsigned int early) 92 + static void __init jailhouse_parse_smp_config(void) 93 93 { 94 94 struct ioapic_domain_cfg ioapic_cfg = { 95 95 .type = IOAPIC_DOMAIN_STRICT, ··· 102 102 register_lapic_address(0xfee00000); 103 103 104 104 for (cpu = 0; cpu < setup_data.v1.num_cpus; cpu++) 105 - generic_processor_info(setup_data.v1.cpu_ids[cpu]); 105 + topology_register_apic(setup_data.v1.cpu_ids[cpu], CPU_ACPIID_INVALID, true); 106 106 107 107 smp_found_config = 1; 108 108 ··· 201 201 struct setup_data header; 202 202 void *mapping; 203 203 204 - x86_init.irqs.pre_vector_init = x86_init_noop; 205 - x86_init.timers.timer_init = jailhouse_timer_init; 206 - x86_init.mpparse.get_smp_config = jailhouse_get_smp_config; 207 - x86_init.pci.arch_init = jailhouse_pci_arch_init; 204 + x86_init.irqs.pre_vector_init = x86_init_noop; 205 + x86_init.timers.timer_init = jailhouse_timer_init; 206 + x86_init.mpparse.find_mptable = x86_init_noop; 207 + x86_init.mpparse.early_parse_smp_cfg = x86_init_noop; 208 + x86_init.mpparse.parse_smp_cfg = jailhouse_parse_smp_config; 209 + x86_init.pci.arch_init = jailhouse_pci_arch_init; 208 210 209 - x86_platform.calibrate_cpu = jailhouse_get_tsc; 210 - x86_platform.calibrate_tsc = jailhouse_get_tsc; 211 - x86_platform.get_wallclock = jailhouse_get_wallclock; 212 - x86_platform.legacy.rtc = 0; 213 - x86_platform.legacy.warm_reset = 0; 214 - x86_platform.legacy.i8042 = X86_LEGACY_I8042_PLATFORM_ABSENT; 211 + x86_platform.calibrate_cpu = jailhouse_get_tsc; 212 + x86_platform.calibrate_tsc = jailhouse_get_tsc; 213 + x86_platform.get_wallclock = jailhouse_get_wallclock; 214 + x86_platform.legacy.rtc = 0; 215 + x86_platform.legacy.warm_reset = 0; 216 + x86_platform.legacy.i8042 = X86_LEGACY_I8042_PLATFORM_ABSENT; 215 217 216 - legacy_pic = &null_legacy_pic; 218 + legacy_pic = &null_legacy_pic; 217 219 218 - machine_ops.emergency_restart = jailhouse_no_restart; 220 + machine_ops.emergency_restart = jailhouse_no_restart; 219 221 220 222 while (pa_data) { 221 223 mapping = early_memremap(pa_data, sizeof(header));
+21 -10
arch/x86/kernel/mpparse.c
··· 36 36 * Checksum an MP configuration block. 37 37 */ 38 38 39 + static unsigned int num_procs __initdata; 40 + 39 41 static int __init mpf_checksum(unsigned char *mp, int len) 40 42 { 41 43 int sum = 0; ··· 52 50 { 53 51 char *bootup_cpu = ""; 54 52 55 - if (!(m->cpuflag & CPU_ENABLED)) { 56 - disabled_cpus++; 53 + topology_register_apic(m->apicid, CPU_ACPIID_INVALID, m->cpuflag & CPU_ENABLED); 54 + if (!(m->cpuflag & CPU_ENABLED)) 57 55 return; 58 - } 59 56 60 57 if (m->cpuflag & CPU_BOOTPROCESSOR) 61 58 bootup_cpu = " (Bootup-CPU)"; 62 59 63 60 pr_info("Processor #%d%s\n", m->apicid, bootup_cpu); 64 - generic_processor_info(m->apicid); 61 + num_procs++; 65 62 } 66 63 67 64 #ifdef CONFIG_X86_IO_APIC ··· 237 236 } 238 237 } 239 238 240 - if (!num_processors) 239 + if (!num_procs && !acpi_lapic) 241 240 pr_err("MPTABLE: no processors registered!\n"); 242 - return num_processors; 241 + return num_procs || acpi_lapic; 243 242 } 244 243 245 244 #ifdef CONFIG_X86_IO_APIC ··· 474 473 /* 475 474 * Scan the memory blocks for an SMP configuration block. 476 475 */ 477 - void __init default_get_smp_config(unsigned int early) 476 + static __init void mpparse_get_smp_config(unsigned int early) 478 477 { 479 478 struct mpf_intel *mpf; 480 479 ··· 530 529 } else 531 530 BUG(); 532 531 533 - if (!early) 534 - pr_info("Processors: %d\n", num_processors); 532 + if (!early && !acpi_lapic) 533 + pr_info("Processors: %d\n", num_procs); 535 534 /* 536 535 * Only use the first configuration found. 537 536 */ 538 537 out: 539 538 early_memunmap(mpf, sizeof(*mpf)); 539 + } 540 + 541 + void __init mpparse_parse_early_smp_config(void) 542 + { 543 + mpparse_get_smp_config(true); 544 + } 545 + 546 + void __init mpparse_parse_smp_config(void) 547 + { 548 + mpparse_get_smp_config(false); 540 549 } 541 550 542 551 static void __init smp_reserve_memory(struct mpf_intel *mpf) ··· 598 587 return ret; 599 588 } 600 589 601 - void __init default_find_smp_config(void) 590 + void __init mpparse_find_mptable(void) 602 591 { 603 592 unsigned int address; 604 593
+1 -1
arch/x86/kernel/process.c
··· 936 936 void select_idle_routine(const struct cpuinfo_x86 *c) 937 937 { 938 938 #ifdef CONFIG_SMP 939 - if (boot_option_idle_override == IDLE_POLL && smp_num_siblings > 1) 939 + if (boot_option_idle_override == IDLE_POLL && __max_threads_per_core > 1) 940 940 pr_warn_once("WARNING: polling idle and HT enabled, performance may degrade\n"); 941 941 #endif 942 942 if (x86_idle_set() || boot_option_idle_override == IDLE_POLL)
+11 -16
arch/x86/kernel/setup.c
··· 970 970 high_memory = (void *)__va(max_pfn * PAGE_SIZE - 1) + 1; 971 971 #endif 972 972 973 - /* 974 - * Find and reserve possible boot-time SMP configuration: 975 - */ 976 - find_smp_config(); 973 + /* Find and reserve MPTABLE area */ 974 + x86_init.mpparse.find_mptable(); 977 975 978 976 early_alloc_pgt_buf(); 979 977 ··· 1088 1090 1089 1091 early_platform_quirks(); 1090 1092 1093 + /* Some platforms need the APIC registered for NUMA configuration */ 1091 1094 early_acpi_boot_init(); 1095 + x86_init.mpparse.early_parse_smp_cfg(); 1092 1096 1093 1097 x86_flattree_get_config(); 1094 1098 ··· 1131 1131 1132 1132 early_quirks(); 1133 1133 1134 + topology_apply_cmdline_limits_early(); 1135 + 1134 1136 /* 1135 - * Read APIC and some other early information from ACPI tables. 1137 + * Parse SMP configuration. Try ACPI first and then the platform 1138 + * specific parser. 1136 1139 */ 1137 1140 acpi_boot_init(); 1138 - x86_dtb_init(); 1141 + x86_init.mpparse.parse_smp_cfg(); 1139 1142 1140 - /* 1141 - * get boot-time SMP configuration: 1142 - */ 1143 - get_smp_config(); 1144 - 1145 - /* 1146 - * Systems w/o ACPI and mptables might not have it mapped the local 1147 - * APIC yet, but prefill_possible_map() might need to access it. 1148 - */ 1143 + /* Last opportunity to detect and map the local APIC */ 1149 1144 init_apic_mappings(); 1150 1145 1151 - prefill_possible_map(); 1146 + topology_init_possible_cpus(); 1152 1147 1153 1148 init_cpu_to_node(); 1154 1149 init_gi_nodes();
+19 -214
arch/x86/kernel/smpboot.c
··· 125 125 */ 126 126 static DEFINE_PER_CPU_ALIGNED(struct mwait_cpu_dead, mwait_cpu_dead); 127 127 128 - /* Logical package management. */ 129 - struct logical_maps { 130 - u32 phys_pkg_id; 131 - u32 phys_die_id; 132 - u32 logical_pkg_id; 133 - u32 logical_die_id; 134 - }; 135 - 136 - /* Temporary workaround until the full topology mechanics is in place */ 137 - static DEFINE_PER_CPU_READ_MOSTLY(struct logical_maps, logical_maps) = { 138 - .phys_pkg_id = U32_MAX, 139 - .phys_die_id = U32_MAX, 140 - }; 141 - 142 - unsigned int __max_logical_packages __read_mostly; 143 - EXPORT_SYMBOL(__max_logical_packages); 144 - static unsigned int logical_packages __read_mostly; 145 - static unsigned int logical_die __read_mostly; 146 - 147 128 /* Maximum number of SMT threads on any online core */ 148 129 int __read_mostly __max_smt_threads = 1; 149 130 ··· 317 336 cpu_startup_entry(CPUHP_AP_ONLINE_IDLE); 318 337 } 319 338 320 - /** 321 - * topology_phys_to_logical_pkg - Map a physical package id to a logical 322 - * @phys_pkg: The physical package id to map 323 - * 324 - * Returns logical package id or -1 if not found 325 - */ 326 - int topology_phys_to_logical_pkg(unsigned int phys_pkg) 327 - { 328 - int cpu; 329 - 330 - for_each_possible_cpu(cpu) { 331 - if (per_cpu(logical_maps.phys_pkg_id, cpu) == phys_pkg) 332 - return per_cpu(logical_maps.logical_pkg_id, cpu); 333 - } 334 - return -1; 335 - } 336 - EXPORT_SYMBOL(topology_phys_to_logical_pkg); 337 - 338 - /** 339 - * topology_phys_to_logical_die - Map a physical die id to logical 340 - * @die_id: The physical die id to map 341 - * @cur_cpu: The CPU for which the mapping is done 342 - * 343 - * Returns logical die id or -1 if not found 344 - */ 345 - static int topology_phys_to_logical_die(unsigned int die_id, unsigned int cur_cpu) 346 - { 347 - int cpu, proc_id = cpu_data(cur_cpu).topo.pkg_id; 348 - 349 - for_each_possible_cpu(cpu) { 350 - if (per_cpu(logical_maps.phys_pkg_id, cpu) == proc_id && 351 - per_cpu(logical_maps.phys_die_id, cpu) == die_id) 352 - return per_cpu(logical_maps.logical_die_id, cpu); 353 - } 354 - return -1; 355 - } 356 - 357 - /** 358 - * topology_update_package_map - Update the physical to logical package map 359 - * @pkg: The physical package id as retrieved via CPUID 360 - * @cpu: The cpu for which this is updated 361 - */ 362 - int topology_update_package_map(unsigned int pkg, unsigned int cpu) 363 - { 364 - int new; 365 - 366 - /* Already available somewhere? */ 367 - new = topology_phys_to_logical_pkg(pkg); 368 - if (new >= 0) 369 - goto found; 370 - 371 - new = logical_packages++; 372 - if (new != pkg) { 373 - pr_info("CPU %u Converting physical %u to logical package %u\n", 374 - cpu, pkg, new); 375 - } 376 - found: 377 - per_cpu(logical_maps.phys_pkg_id, cpu) = pkg; 378 - per_cpu(logical_maps.logical_pkg_id, cpu) = new; 379 - cpu_data(cpu).topo.logical_pkg_id = new; 380 - return 0; 381 - } 382 - /** 383 - * topology_update_die_map - Update the physical to logical die map 384 - * @die: The die id as retrieved via CPUID 385 - * @cpu: The cpu for which this is updated 386 - */ 387 - int topology_update_die_map(unsigned int die, unsigned int cpu) 388 - { 389 - int new; 390 - 391 - /* Already available somewhere? */ 392 - new = topology_phys_to_logical_die(die, cpu); 393 - if (new >= 0) 394 - goto found; 395 - 396 - new = logical_die++; 397 - if (new != die) { 398 - pr_info("CPU %u Converting physical %u to logical die %u\n", 399 - cpu, die, new); 400 - } 401 - found: 402 - per_cpu(logical_maps.phys_die_id, cpu) = die; 403 - per_cpu(logical_maps.logical_die_id, cpu) = new; 404 - cpu_data(cpu).topo.logical_die_id = new; 405 - return 0; 406 - } 407 - 408 339 static void __init smp_store_boot_cpu_info(void) 409 340 { 410 - int id = 0; /* CPU 0 */ 411 - struct cpuinfo_x86 *c = &cpu_data(id); 341 + struct cpuinfo_x86 *c = &cpu_data(0); 412 342 413 343 *c = boot_cpu_data; 414 - c->cpu_index = id; 415 - topology_update_package_map(c->topo.pkg_id, id); 416 - topology_update_die_map(c->topo.die_id, id); 417 344 c->initialized = true; 418 345 } 419 346 ··· 377 488 378 489 if (c->topo.pkg_id == o->topo.pkg_id && 379 490 c->topo.die_id == o->topo.die_id && 491 + c->topo.amd_node_id == o->topo.amd_node_id && 380 492 per_cpu_llc_id(cpu1) == per_cpu_llc_id(cpu2)) { 381 493 if (c->topo.core_id == o->topo.core_id) 382 494 return topology_sane(c, o, "smt"); ··· 399 509 400 510 static bool match_die(struct cpuinfo_x86 *c, struct cpuinfo_x86 *o) 401 511 { 402 - if (c->topo.pkg_id == o->topo.pkg_id && 403 - c->topo.die_id == o->topo.die_id) 404 - return true; 405 - return false; 512 + if (c->topo.pkg_id != o->topo.pkg_id || c->topo.die_id != o->topo.die_id) 513 + return false; 514 + 515 + if (cpu_feature_enabled(X86_FEATURE_TOPOEXT) && topology_amd_nodes_per_pkg() > 1) 516 + return c->topo.amd_node_id == o->topo.amd_node_id; 517 + 518 + return true; 406 519 } 407 520 408 521 static bool match_l2c(struct cpuinfo_x86 *c, struct cpuinfo_x86 *o) ··· 563 670 564 671 void set_cpu_sibling_map(int cpu) 565 672 { 566 - bool has_smt = smp_num_siblings > 1; 567 - bool has_mp = has_smt || boot_cpu_data.x86_max_cores > 1; 673 + bool has_smt = __max_threads_per_core > 1; 674 + bool has_mp = has_smt || topology_num_cores_per_package() > 1; 568 675 struct cpuinfo_x86 *c = &cpu_data(cpu); 569 676 struct cpuinfo_x86 *o; 570 677 int i, threads; ··· 961 1068 962 1069 pr_debug("++++++++++++++++++++=_---CPU UP %u\n", cpu); 963 1070 964 - if (apicid == BAD_APICID || !physid_isset(apicid, phys_cpu_present_map) || 965 - !apic_id_valid(apicid)) { 966 - pr_err("%s: bad cpu %d\n", __func__, cpu); 1071 + if (apicid == BAD_APICID || !apic_id_valid(apicid)) { 1072 + pr_err("CPU %u has invalid APIC ID %x. Aborting bringup\n", cpu, apicid); 1073 + return -EINVAL; 1074 + } 1075 + 1076 + if (!test_bit(apicid, phys_cpu_present_map)) { 1077 + pr_err("CPU %u APIC ID %x is not present. Aborting bringup\n", cpu, apicid); 967 1078 return -EINVAL; 968 1079 } 969 1080 ··· 1036 1139 pr_info("SMP disabled\n"); 1037 1140 1038 1141 disable_ioapic_support(); 1142 + topology_reset_possible_cpus_up(); 1039 1143 1040 - init_cpu_present(cpumask_of(0)); 1041 - init_cpu_possible(cpumask_of(0)); 1042 - 1043 - if (smp_found_config) 1044 - physid_set_mask_of_physid(boot_cpu_physical_apicid, &phys_cpu_present_map); 1045 - else 1046 - physid_set_mask_of_physid(0, &phys_cpu_present_map); 1047 1144 cpumask_set_cpu(0, topology_sibling_cpumask(0)); 1048 1145 cpumask_set_cpu(0, topology_core_cpumask(0)); 1049 1146 cpumask_set_cpu(0, topology_die_cpumask(0)); ··· 1156 1265 native_pv_lock_init(); 1157 1266 } 1158 1267 1159 - void __init calculate_max_logical_packages(void) 1160 - { 1161 - int ncpus; 1162 - 1163 - /* 1164 - * Today neither Intel nor AMD support heterogeneous systems so 1165 - * extrapolate the boot cpu's data to all packages. 1166 - */ 1167 - ncpus = cpu_data(0).booted_cores * topology_max_smt_threads(); 1168 - __max_logical_packages = DIV_ROUND_UP(total_cpus, ncpus); 1169 - pr_info("Max logical packages: %u\n", __max_logical_packages); 1170 - } 1171 - 1172 1268 void __init native_smp_cpus_done(unsigned int max_cpus) 1173 1269 { 1174 1270 pr_debug("Boot done\n"); 1175 1271 1176 - calculate_max_logical_packages(); 1177 1272 build_sched_topology(); 1178 1273 nmi_selftest(); 1179 1274 impress_friends(); 1180 1275 cache_aps_init(); 1181 - } 1182 - 1183 - static int __initdata setup_possible_cpus = -1; 1184 - static int __init _setup_possible_cpus(char *str) 1185 - { 1186 - get_option(&str, &setup_possible_cpus); 1187 - return 0; 1188 - } 1189 - early_param("possible_cpus", _setup_possible_cpus); 1190 - 1191 - 1192 - /* 1193 - * cpu_possible_mask should be static, it cannot change as cpu's 1194 - * are onlined, or offlined. The reason is per-cpu data-structures 1195 - * are allocated by some modules at init time, and don't expect to 1196 - * do this dynamically on cpu arrival/departure. 1197 - * cpu_present_mask on the other hand can change dynamically. 1198 - * In case when cpu_hotplug is not compiled, then we resort to current 1199 - * behaviour, which is cpu_possible == cpu_present. 1200 - * - Ashok Raj 1201 - * 1202 - * Three ways to find out the number of additional hotplug CPUs: 1203 - * - If the BIOS specified disabled CPUs in ACPI/mptables use that. 1204 - * - The user can overwrite it with possible_cpus=NUM 1205 - * - Otherwise don't reserve additional CPUs. 1206 - * We do this because additional CPUs waste a lot of memory. 1207 - * -AK 1208 - */ 1209 - __init void prefill_possible_map(void) 1210 - { 1211 - int i, possible; 1212 - 1213 - i = setup_max_cpus ?: 1; 1214 - if (setup_possible_cpus == -1) { 1215 - possible = num_processors; 1216 - #ifdef CONFIG_HOTPLUG_CPU 1217 - if (setup_max_cpus) 1218 - possible += disabled_cpus; 1219 - #else 1220 - if (possible > i) 1221 - possible = i; 1222 - #endif 1223 - } else 1224 - possible = setup_possible_cpus; 1225 - 1226 - total_cpus = max_t(int, possible, num_processors + disabled_cpus); 1227 - 1228 - /* nr_cpu_ids could be reduced via nr_cpus= */ 1229 - if (possible > nr_cpu_ids) { 1230 - pr_warn("%d Processors exceeds NR_CPUS limit of %u\n", 1231 - possible, nr_cpu_ids); 1232 - possible = nr_cpu_ids; 1233 - } 1234 - 1235 - #ifdef CONFIG_HOTPLUG_CPU 1236 - if (!setup_max_cpus) 1237 - #endif 1238 - if (possible > i) { 1239 - pr_warn("%d Processors exceeds max_cpus limit of %u\n", 1240 - possible, setup_max_cpus); 1241 - possible = i; 1242 - } 1243 - 1244 - set_nr_cpu_ids(possible); 1245 - 1246 - pr_info("Allowing %d CPUs, %d hotplug CPUs\n", 1247 - possible, max_t(int, possible - num_processors, 0)); 1248 - 1249 - reset_cpu_possible_mask(); 1250 - 1251 - for (i = 0; i < possible; i++) 1252 - set_cpu_possible(i, true); 1253 1276 } 1254 1277 1255 1278 /* correctly size the local cpu masks */
-13
arch/x86/kernel/vsmp_64.c
··· 127 127 #endif 128 128 } 129 129 130 - static u32 apicid_phys_pkg_id(u32 initial_apic_id, int index_msb) 131 - { 132 - return read_apic_id() >> index_msb; 133 - } 134 - 135 - static void vsmp_apic_post_init(void) 136 - { 137 - /* need to update phys_pkg_id */ 138 - apic->phys_pkg_id = apicid_phys_pkg_id; 139 - } 140 - 141 130 void __init vsmp_init(void) 142 131 { 143 132 detect_vsmp_box(); 144 133 if (!is_vsmp_box()) 145 134 return; 146 - 147 - x86_platform.apic_post_init = vsmp_apic_post_init; 148 135 149 136 vsmp_cap_cpus(); 150 137
+3 -2
arch/x86/kernel/x86_init.c
··· 70 70 71 71 .mpparse = { 72 72 .setup_ioapic_ids = x86_init_noop, 73 - .find_smp_config = default_find_smp_config, 74 - .get_smp_config = default_get_smp_config, 73 + .find_mptable = mpparse_find_mptable, 74 + .early_parse_smp_cfg = mpparse_parse_early_smp_config, 75 + .parse_smp_cfg = mpparse_parse_smp_config, 75 76 }, 76 77 77 78 .irqs = {
+12 -22
arch/x86/mm/amdtopology.c
··· 54 54 55 55 int __init amd_numa_init(void) 56 56 { 57 - u64 start = PFN_PHYS(0); 57 + unsigned int numnodes, cores, apicid; 58 + u64 prevbase, start = PFN_PHYS(0); 58 59 u64 end = PFN_PHYS(max_pfn); 59 - unsigned numnodes; 60 - u64 prevbase; 61 - int i, j, nb; 62 60 u32 nodeid, reg; 63 - unsigned int bits, cores, apicid_base; 61 + int i, j, nb; 64 62 65 63 if (!early_pci_allowed()) 66 64 return -EINVAL; ··· 156 158 return -ENOENT; 157 159 158 160 /* 159 - * We seem to have valid NUMA configuration. Map apicids to nodes 160 - * using the coreid bits from early_identify_cpu. 161 + * We seem to have valid NUMA configuration. Map apicids to nodes 162 + * using the size of the core domain in the APIC space. 161 163 */ 162 - bits = boot_cpu_data.x86_coreid_bits; 163 - cores = 1 << bits; 164 - apicid_base = 0; 164 + cores = topology_get_domain_size(TOPO_CORE_DOMAIN); 165 165 166 - /* 167 - * get boot-time SMP configuration: 168 - */ 169 - early_get_smp_config(); 166 + apicid = boot_cpu_physical_apicid; 167 + if (apicid > 0) 168 + pr_info("BSP APIC ID: %02x\n", apicid); 170 169 171 - if (boot_cpu_physical_apicid > 0) { 172 - pr_info("BSP APIC ID: %02x\n", boot_cpu_physical_apicid); 173 - apicid_base = boot_cpu_physical_apicid; 170 + for_each_node_mask(i, numa_nodes_parsed) { 171 + for (j = 0; j < cores; j++, apicid++) 172 + set_apicid_to_node(apicid, i); 174 173 } 175 - 176 - for_each_node_mask(i, numa_nodes_parsed) 177 - for (j = apicid_base; j < cores + apicid_base; j++) 178 - set_apicid_to_node((i << bits) + j, i); 179 - 180 174 return 0; 181 175 }
+7 -7
arch/x86/platform/ce4100/ce4100.c
··· 135 135 */ 136 136 void __init x86_ce4100_early_setup(void) 137 137 { 138 - x86_init.oem.arch_setup = sdv_arch_setup; 139 - x86_init.resources.probe_roms = x86_init_noop; 140 - x86_init.mpparse.get_smp_config = x86_init_uint_noop; 141 - x86_init.mpparse.find_smp_config = x86_init_noop; 142 - x86_init.mpparse.setup_ioapic_ids = setup_ioapic_ids_from_mpc_nocheck; 143 - x86_init.pci.init = ce4100_pci_init; 144 - x86_init.pci.init_irq = sdv_pci_init; 138 + x86_init.oem.arch_setup = sdv_arch_setup; 139 + x86_init.resources.probe_roms = x86_init_noop; 140 + x86_init.mpparse.find_mptable = x86_init_noop; 141 + x86_init.mpparse.early_parse_smp_cfg = x86_init_noop; 142 + x86_init.mpparse.parse_smp_cfg = x86_dtb_parse_smp_config; 143 + x86_init.pci.init = ce4100_pci_init; 144 + x86_init.pci.init_irq = sdv_pci_init; 145 145 146 146 /* 147 147 * By default, the reboot method is ACPI which is supported by the
+3 -2
arch/x86/platform/intel-mid/intel-mid.c
··· 118 118 machine_ops.emergency_restart = intel_mid_reboot; 119 119 120 120 /* Avoid searching for BIOS MP tables */ 121 - x86_init.mpparse.find_smp_config = x86_init_noop; 122 - x86_init.mpparse.get_smp_config = x86_init_uint_noop; 121 + x86_init.mpparse.find_mptable = x86_init_noop; 122 + x86_init.mpparse.early_parse_smp_cfg = x86_init_noop; 123 + x86_init.mpparse.parse_smp_cfg = x86_init_noop; 123 124 set_bit(MP_BUS_ISA, mp_bus_not_pci); 124 125 }
+7 -20
arch/x86/xen/apic.c
··· 33 33 return 0xfd; 34 34 } 35 35 36 - static u32 xen_set_apic_id(u32 x) 37 - { 38 - WARN_ON(1); 39 - return x; 40 - } 41 - 42 36 static u32 xen_get_apic_id(u32 x) 43 37 { 44 38 return ((x)>>24) & 0xFFu; ··· 43 49 struct xen_platform_op op = { 44 50 .cmd = XENPF_get_cpuinfo, 45 51 .interface_version = XENPF_INTERFACE_VERSION, 46 - .u.pcpu_info.xen_cpuid = 0, 47 52 }; 48 - int ret; 49 - 50 - /* Shouldn't need this as APIC is turned off for PV, and we only 51 - * get called on the bootup processor. But just in case. */ 52 - if (!xen_initial_domain() || smp_processor_id()) 53 - return 0; 53 + int ret, cpu; 54 54 55 55 if (reg == APIC_LVR) 56 56 return 0x14; 57 57 if (reg != APIC_ID) 58 58 return 0; 59 + 60 + cpu = smp_processor_id(); 61 + if (!xen_initial_domain()) 62 + return cpu ? cpuid_to_apicid[cpu] << 24 : 0; 63 + 64 + op.u.pcpu_info.xen_cpuid = cpu; 59 65 60 66 ret = HYPERVISOR_platform_op(&op); 61 67 if (ret) ··· 104 110 return xen_pv_domain(); 105 111 } 106 112 107 - static u32 xen_phys_pkg_id(u32 initial_apic_id, int index_msb) 108 - { 109 - return initial_apic_id >> index_msb; 110 - } 111 - 112 113 static u32 xen_cpu_present_to_apicid(int cpu) 113 114 { 114 115 if (cpu_present(cpu)) ··· 122 133 .disable_esr = 0, 123 134 124 135 .cpu_present_to_apicid = xen_cpu_present_to_apicid, 125 - .phys_pkg_id = xen_phys_pkg_id, /* detect_ht */ 126 136 127 137 .max_apic_id = UINT_MAX, 128 138 .get_apic_id = xen_get_apic_id, 129 - .set_apic_id = xen_set_apic_id, 130 139 131 140 .calc_dest_apicid = apic_flat_calc_apicid, 132 141
+1 -1
arch/x86/xen/enlighten_hvm.c
··· 168 168 */ 169 169 xen_uninit_lock_cpu(cpu); 170 170 171 - if (cpu_acpi_id(cpu) != U32_MAX) 171 + if (cpu_acpi_id(cpu) != CPU_ACPIID_INVALID) 172 172 per_cpu(xen_vcpu_id, cpu) = cpu_acpi_id(cpu); 173 173 else 174 174 per_cpu(xen_vcpu_id, cpu) = cpu;
+3
arch/x86/xen/enlighten_pv.c
··· 200 200 xen_set_mtrr_data(); 201 201 else 202 202 mtrr_overwrite_state(NULL, 0, MTRR_TYPE_WRBACK); 203 + 204 + /* Adjust nr_cpu_ids before "enumeration" happens */ 205 + xen_smp_count_cpus(); 203 206 } 204 207 205 208 static void __init xen_pv_guest_late_init(void)
-2
arch/x86/xen/smp.c
··· 135 135 { 136 136 if (xen_hvm_domain()) 137 137 native_smp_cpus_done(max_cpus); 138 - else 139 - calculate_max_logical_packages(); 140 138 } 141 139 142 140 void xen_smp_send_reschedule(int cpu)
+2
arch/x86/xen/smp.h
··· 19 19 int xen_smp_intr_init_pv(unsigned int cpu); 20 20 void xen_smp_intr_free_pv(unsigned int cpu); 21 21 22 + void xen_smp_count_cpus(void); 22 23 void xen_smp_cpus_done(unsigned int max_cpus); 23 24 24 25 void xen_smp_send_reschedule(int cpu); ··· 45 44 return 0; 46 45 } 47 46 static inline void xen_smp_intr_free_pv(unsigned int cpu) {} 47 + static inline void xen_smp_count_cpus(void) { } 48 48 #endif /* CONFIG_SMP */ 49 49 50 50 #endif
+30 -35
arch/x86/xen/smp_pv.c
··· 29 29 #include <asm/idtentry.h> 30 30 #include <asm/desc.h> 31 31 #include <asm/cpu.h> 32 + #include <asm/apic.h> 32 33 #include <asm/io_apic.h> 33 34 34 35 #include <xen/interface/xen.h> ··· 74 73 } 75 74 cpu = smp_processor_id(); 76 75 smp_store_cpu_info(cpu); 77 - cpu_data(cpu).x86_max_cores = 1; 78 76 set_cpu_sibling_map(cpu); 79 77 80 78 speculative_store_bypass_ht_init(); ··· 149 149 return rc; 150 150 } 151 151 152 - static void __init _get_smp_config(unsigned int early) 152 + static void __init xen_pv_smp_config(void) 153 153 { 154 - int i, rc; 155 - unsigned int subtract = 0; 154 + u32 apicid = 0; 155 + int i; 156 156 157 - if (early) 158 - return; 157 + topology_register_boot_apic(apicid++); 159 158 160 - num_processors = 0; 161 - disabled_cpus = 0; 162 - for (i = 0; i < nr_cpu_ids; i++) { 163 - rc = HYPERVISOR_vcpu_op(VCPUOP_is_up, i, NULL); 164 - if (rc >= 0) { 165 - num_processors++; 166 - set_cpu_possible(i, true); 167 - } else { 168 - set_cpu_possible(i, false); 169 - set_cpu_present(i, false); 170 - subtract++; 171 - } 172 - } 173 - #ifdef CONFIG_HOTPLUG_CPU 174 - /* This is akin to using 'nr_cpus' on the Linux command line. 175 - * Which is OK as when we use 'dom0_max_vcpus=X' we can only 176 - * have up to X, while nr_cpu_ids is greater than X. This 177 - * normally is not a problem, except when CPU hotplugging 178 - * is involved and then there might be more than X CPUs 179 - * in the guest - which will not work as there is no 180 - * hypercall to expand the max number of VCPUs an already 181 - * running guest has. So cap it up to X. */ 182 - if (subtract) 183 - set_nr_cpu_ids(nr_cpu_ids - subtract); 184 - #endif 159 + for (i = 1; i < nr_cpu_ids; i++) 160 + topology_register_apic(apicid++, CPU_ACPIID_INVALID, true); 161 + 185 162 /* Pretend to be a proper enumerated system */ 186 163 smp_found_config = 1; 187 164 } ··· 200 223 xen_init_lock_cpu(0); 201 224 202 225 smp_prepare_cpus_common(); 203 - 204 - cpu_data(0).x86_max_cores = 1; 205 226 206 227 speculative_store_bypass_ht_init(); 207 228 ··· 409 434 return IRQ_HANDLED; 410 435 } 411 436 437 + void __init xen_smp_count_cpus(void) 438 + { 439 + unsigned int cpus; 440 + 441 + for (cpus = 0; cpus < nr_cpu_ids; cpus++) { 442 + if (HYPERVISOR_vcpu_op(VCPUOP_is_up, cpus, NULL) < 0) 443 + break; 444 + } 445 + 446 + pr_info("Xen PV: Detected %u vCPUS\n", cpus); 447 + if (cpus < nr_cpu_ids) 448 + set_nr_cpu_ids(cpus); 449 + } 450 + 412 451 static const struct smp_ops xen_smp_ops __initconst = { 413 452 .smp_prepare_boot_cpu = xen_pv_smp_prepare_boot_cpu, 414 453 .smp_prepare_cpus = xen_pv_smp_prepare_cpus, ··· 447 458 smp_ops = xen_smp_ops; 448 459 449 460 /* Avoid searching for BIOS MP tables */ 450 - x86_init.mpparse.find_smp_config = x86_init_noop; 451 - x86_init.mpparse.get_smp_config = _get_smp_config; 461 + x86_init.mpparse.find_mptable = x86_init_noop; 462 + x86_init.mpparse.early_parse_smp_cfg = x86_init_noop; 463 + 464 + /* XEN/PV Dom0 has halfways sane topology information via CPUID/MADT */ 465 + if (xen_initial_domain()) 466 + x86_init.mpparse.parse_smp_cfg = x86_init_noop; 467 + else 468 + x86_init.mpparse.parse_smp_cfg = xen_pv_smp_config; 452 469 }
+2 -2
drivers/edac/amd64_edac.c
··· 1915 1915 /* On F10h and later ErrAddr is MC4_ADDR[47:1] */ 1916 1916 static u64 get_error_address(struct amd64_pvt *pvt, struct mce *m) 1917 1917 { 1918 - u16 mce_nid = topology_die_id(m->extcpu); 1918 + u16 mce_nid = topology_amd_node_id(m->extcpu); 1919 1919 struct mem_ctl_info *mci; 1920 1920 u8 start_bit = 1; 1921 1921 u8 end_bit = 47; ··· 3446 3446 int cpu; 3447 3447 3448 3448 for_each_online_cpu(cpu) 3449 - if (topology_die_id(cpu) == nid) 3449 + if (topology_amd_node_id(cpu) == nid) 3450 3450 cpumask_set_cpu(cpu, mask); 3451 3451 } 3452 3452
+2 -2
drivers/edac/mce_amd.c
··· 584 584 static void decode_mc4_mce(struct mce *m) 585 585 { 586 586 unsigned int fam = x86_family(m->cpuid); 587 - int node_id = topology_die_id(m->extcpu); 587 + int node_id = topology_amd_node_id(m->extcpu); 588 588 u16 ec = EC(m->status); 589 589 u8 xec = XEC(m->status, 0x1f); 590 590 u8 offset = 0; ··· 746 746 747 747 if ((bank_type == SMCA_UMC || bank_type == SMCA_UMC_V2) && 748 748 xec == 0 && decode_dram_ecc) 749 - decode_dram_ecc(topology_die_id(m->extcpu), m); 749 + decode_dram_ecc(topology_amd_node_id(m->extcpu), m); 750 750 } 751 751 752 752 static inline void amd_decode_err_code(u16 ec)
+1 -1
drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c
··· 451 451 452 452 #ifdef CONFIG_X86 453 453 /* AMD x86 APU only */ 454 - smu->cpu_core_num = boot_cpu_data.x86_max_cores; 454 + smu->cpu_core_num = topology_num_cores_per_package(); 455 455 #else 456 456 smu->cpu_core_num = 4; 457 457 #endif
+1 -1
drivers/hwmon/coretemp.c
··· 782 782 if (!x86_match_cpu(coretemp_ids)) 783 783 return -ENODEV; 784 784 785 - max_zones = topology_max_packages() * topology_max_die_per_package(); 785 + max_zones = topology_max_packages() * topology_max_dies_per_package(); 786 786 zone_devices = kcalloc(max_zones, sizeof(struct platform_device *), 787 787 GFP_KERNEL); 788 788 if (!zone_devices)
+1 -1
drivers/hwmon/fam15h_power.c
··· 209 209 * With the new x86 topology modelling, x86_max_cores is the 210 210 * compute unit number. 211 211 */ 212 - cu_num = boot_cpu_data.x86_max_cores; 212 + cu_num = topology_num_cores_per_package(); 213 213 214 214 ret = read_registers(data); 215 215 if (ret)
+1 -1
drivers/platform/x86/intel/uncore-frequency/uncore-frequency.c
··· 242 242 return -ENODEV; 243 243 244 244 uncore_max_entries = topology_max_packages() * 245 - topology_max_die_per_package(); 245 + topology_max_dies_per_package(); 246 246 uncore_instances = kcalloc(uncore_max_entries, 247 247 sizeof(*uncore_instances), GFP_KERNEL); 248 248 if (!uncore_instances)
+1 -1
drivers/powercap/intel_rapl_common.c
··· 1564 1564 if (id_is_cpu) { 1565 1565 rp->id = topology_logical_die_id(id); 1566 1566 rp->lead_cpu = id; 1567 - if (topology_max_die_per_package() > 1) 1567 + if (topology_max_dies_per_package() > 1) 1568 1568 snprintf(rp->name, PACKAGE_DOMAIN_NAME_LENGTH, "package-%d-die-%d", 1569 1569 topology_physical_package_id(id), topology_die_id(id)); 1570 1570 else
+1 -1
drivers/thermal/intel/intel_hfi.c
··· 607 607 608 608 /* There is one HFI instance per die/package. */ 609 609 max_hfi_instances = topology_max_packages() * 610 - topology_max_die_per_package(); 610 + topology_max_dies_per_package(); 611 611 612 612 /* 613 613 * This allocation may fail. CPU hotplug callbacks must check
+1 -1
drivers/thermal/intel/intel_powerclamp.c
··· 587 587 poll_pkg_cstate_enable = false; 588 588 if (cpumask_equal(cpu_present_mask, idle_injection_cpu_mask)) { 589 589 ii_dev = idle_inject_register_full(idle_injection_cpu_mask, idle_inject_update); 590 - if (topology_max_packages() == 1 && topology_max_die_per_package() == 1) 590 + if (topology_max_packages() == 1 && topology_max_dies_per_package() == 1) 591 591 poll_pkg_cstate_enable = true; 592 592 } else { 593 593 ii_dev = idle_inject_register(idle_injection_cpu_mask);
+1 -1
drivers/thermal/intel/x86_pkg_temp_thermal.c
··· 494 494 if (!x86_match_cpu(pkg_temp_thermal_ids)) 495 495 return -ENODEV; 496 496 497 - max_id = topology_max_packages() * topology_max_die_per_package(); 497 + max_id = topology_max_packages() * topology_max_dies_per_package(); 498 498 zones = kcalloc(max_id, sizeof(struct zone_device *), 499 499 GFP_KERNEL); 500 500 if (!zones)
+1 -1
include/linux/cpu.h
··· 112 112 extern void cpu_maps_update_begin(void); 113 113 extern void cpu_maps_update_done(void); 114 114 int bringup_hibernate_cpu(unsigned int sleep_cpu); 115 - void bringup_nonboot_cpus(unsigned int setup_max_cpus); 115 + void bringup_nonboot_cpus(unsigned int max_cpus); 116 116 117 117 #else /* CONFIG_SMP */ 118 118 #define cpuhp_tasks_frozen 0
+2
include/linux/smp.h
··· 218 218 static inline void kick_all_cpus_sync(void) { } 219 219 static inline void wake_up_all_idle_cpus(void) { } 220 220 221 + #define setup_max_cpus 0 222 + 221 223 #ifdef CONFIG_UP_LATE_INIT 222 224 extern void __init up_late_init(void); 223 225 static inline void smp_init(void) { up_late_init(); }
-1
init/main.c
··· 604 604 __setup("rdinit=", rdinit_setup); 605 605 606 606 #ifndef CONFIG_SMP 607 - static const unsigned int setup_max_cpus = NR_CPUS; 608 607 static inline void setup_nr_cpu_ids(void) { } 609 608 static inline void smp_prepare_cpus(unsigned int maxcpus) { } 610 609 #endif
+3 -3
kernel/cpu.c
··· 1905 1905 static inline bool cpuhp_bringup_cpus_parallel(unsigned int ncpus) { return false; } 1906 1906 #endif /* CONFIG_HOTPLUG_PARALLEL */ 1907 1907 1908 - void __init bringup_nonboot_cpus(unsigned int setup_max_cpus) 1908 + void __init bringup_nonboot_cpus(unsigned int max_cpus) 1909 1909 { 1910 1910 /* Try parallel bringup optimization if enabled */ 1911 - if (cpuhp_bringup_cpus_parallel(setup_max_cpus)) 1911 + if (cpuhp_bringup_cpus_parallel(max_cpus)) 1912 1912 return; 1913 1913 1914 1914 /* Full per CPU serialized bringup */ 1915 - cpuhp_bringup_mask(cpu_present_mask, setup_max_cpus, CPUHP_ONLINE); 1915 + cpuhp_bringup_mask(cpu_present_mask, max_cpus, CPUHP_ONLINE); 1916 1916 } 1917 1917 1918 1918 #ifdef CONFIG_PM_SLEEP_SMP