Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

x86/apic: Use non-atomic operations when possible

Using __clear_bit() and __cpumask_clear_cpu() is more efficient than using
their atomic counterparts.

Use them when atomicity is not needed, such as when manipulating bitmasks
that are on the stack.

Signed-off-by: Nadav Amit <namit@vmware.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://lkml.kernel.org/r/20190613064813.8102-10-namit@vmware.com

authored by

Nadav Amit and committed by
Thomas Gleixner
dde3626f 748b170c

+4 -4
+2 -2
arch/x86/kernel/apic/apic_flat_64.c
··· 78 78 int cpu = smp_processor_id(); 79 79 80 80 if (cpu < BITS_PER_LONG) 81 - clear_bit(cpu, &mask); 81 + __clear_bit(cpu, &mask); 82 82 83 83 _flat_send_IPI_mask(mask, vector); 84 84 } ··· 92 92 unsigned long mask = cpumask_bits(cpu_online_mask)[0]; 93 93 94 94 if (cpu < BITS_PER_LONG) 95 - clear_bit(cpu, &mask); 95 + __clear_bit(cpu, &mask); 96 96 97 97 _flat_send_IPI_mask(mask, vector); 98 98 }
+1 -1
arch/x86/kernel/apic/x2apic_cluster.c
··· 50 50 cpumask_copy(tmpmsk, mask); 51 51 /* If IPI should not be sent to self, clear current CPU */ 52 52 if (apic_dest != APIC_DEST_ALLINC) 53 - cpumask_clear_cpu(smp_processor_id(), tmpmsk); 53 + __cpumask_clear_cpu(smp_processor_id(), tmpmsk); 54 54 55 55 /* Collapse cpus in a cluster so a single IPI per cluster is sent */ 56 56 for_each_cpu(cpu, tmpmsk) {
+1 -1
arch/x86/kernel/smp.c
··· 146 146 } 147 147 148 148 cpumask_copy(allbutself, cpu_online_mask); 149 - cpumask_clear_cpu(smp_processor_id(), allbutself); 149 + __cpumask_clear_cpu(smp_processor_id(), allbutself); 150 150 151 151 if (cpumask_equal(mask, allbutself) && 152 152 cpumask_equal(cpu_online_mask, cpu_callout_mask))