x86/smp: Use dedicated cache-line for mwait_play_dead()

Monitoring idletask::thread_info::flags in mwait_play_dead() has been an
obvious choice as all what is needed is a cache line which is not written
by other CPUs.

But there is a use case where a "dead" CPU needs to be brought out of
MWAIT: kexec().

This is required as kexec() can overwrite text, pagetables, stacks and the
monitored cacheline of the original kernel. The latter causes MWAIT to
resume execution which obviously causes havoc on the kexec kernel which
results usually in triple faults.

Use a dedicated per CPU storage to prepare for that.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Ashok Raj <ashok.raj@intel.com>
Reviewed-by: Borislav Petkov (AMD) <bp@alien8.de>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/20230615193330.434553750@linutronix.de

+14 -10
+14 -10
arch/x86/kernel/smpboot.c
··· 101 DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info); 102 EXPORT_PER_CPU_SYMBOL(cpu_info); 103 104 /* Logical package management. We might want to allocate that dynamically */ 105 unsigned int __max_logical_packages __read_mostly; 106 EXPORT_SYMBOL(__max_logical_packages); ··· 1769 */ 1770 static inline void mwait_play_dead(void) 1771 { 1772 unsigned int eax, ebx, ecx, edx; 1773 unsigned int highest_cstate = 0; 1774 unsigned int highest_subcstate = 0; 1775 - void *mwait_ptr; 1776 int i; 1777 1778 if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD || ··· 1807 (highest_subcstate - 1); 1808 } 1809 1810 - /* 1811 - * This should be a memory location in a cache line which is 1812 - * unlikely to be touched by other processors. The actual 1813 - * content is immaterial as it is not actually modified in any way. 1814 - */ 1815 - mwait_ptr = &current_thread_info()->flags; 1816 - 1817 wbinvd(); 1818 1819 while (1) { ··· 1818 * case where we return around the loop. 1819 */ 1820 mb(); 1821 - clflush(mwait_ptr); 1822 mb(); 1823 - __monitor(mwait_ptr, 0, 0); 1824 mb(); 1825 __mwait(eax, 0); 1826
··· 101 DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info); 102 EXPORT_PER_CPU_SYMBOL(cpu_info); 103 104 + struct mwait_cpu_dead { 105 + unsigned int control; 106 + unsigned int status; 107 + }; 108 + 109 + /* 110 + * Cache line aligned data for mwait_play_dead(). Separate on purpose so 111 + * that it's unlikely to be touched by other CPUs. 112 + */ 113 + static DEFINE_PER_CPU_ALIGNED(struct mwait_cpu_dead, mwait_cpu_dead); 114 + 115 /* Logical package management. We might want to allocate that dynamically */ 116 unsigned int __max_logical_packages __read_mostly; 117 EXPORT_SYMBOL(__max_logical_packages); ··· 1758 */ 1759 static inline void mwait_play_dead(void) 1760 { 1761 + struct mwait_cpu_dead *md = this_cpu_ptr(&mwait_cpu_dead); 1762 unsigned int eax, ebx, ecx, edx; 1763 unsigned int highest_cstate = 0; 1764 unsigned int highest_subcstate = 0; 1765 int i; 1766 1767 if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD || ··· 1796 (highest_subcstate - 1); 1797 } 1798 1799 wbinvd(); 1800 1801 while (1) { ··· 1814 * case where we return around the loop. 1815 */ 1816 mb(); 1817 + clflush(md); 1818 mb(); 1819 + __monitor(md, 0, 0); 1820 mb(); 1821 __mwait(eax, 0); 1822