Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

ACPI: created a dedicated workqueue for notify() execution

HP nx6125/nx6325/... machines have a _GPE handler with an infinite
loop sending Notify() events to different ACPI subsystems.

Notify handler in ACPI driver is a C-routine, which may call ACPI
interpreter again to get access to some ACPI variables
(acpi_evaluate_xxx).
On these HP machines such an evaluation changes state of some variable
and lets the loop above break.

In the current ACPI implementation Notify requests are being deferred
to the same kacpid workqueue on which the above GPE handler with
infinite loop is executing. Thus we have a deadlock -- loop will
continue to spin, sending notify events, and at the same time
preventing these notify events from being run on a workqueue. All
notify events are deferred, thus we see increase in memory consumption
noticed by author of the thread. Also as GPE handling is bloked,
machines overheat. Eventually by external poll of the same
acpi_evaluate, kacpid is released and all the queued notify events are
free to run, thus 100% cpu utilization by kacpid for several seconds
or more.

To prevent all these horrors it's needed to not put notify events to
kacpid workqueue by either executing them immediately or putting them
on some other thread. It's dangerous to execute notify events in
place, as it will put several ACPI interpreter stacks on top of each
other (at least 4 in case of nx6125), thus causing kernel stack
overflow.

First attempt to create a new thread was done by Peter Wainwright
He created a bunch of threads, which were stealing work from a kacpid
workqueue.
This patch appeared in 2.6.15 kernel shipped with Ubuntu 6.06 LTS.

Second attempt was done by me, I created a new thread for each Notify
event. This worked OK on HP nx machines, but broke Linus' Compaq
n620c, by producing threads with a speed what they stopped the machine
completely. Thus this patch was reverted from 18-rc2 as I remember.
I re-made the patch to create second workqueue just for notify events,
thus hopping it will not break Linus' machine. Patch was tested on the
same HP nx machines in #5534 and #7122, but I did not received reply
from Linus on a test patch sent to him.
Patch went to 19-rc and was rejected with much fanfare again.
There was 4th patch, which inserted schedule_timeout(1) into deferred
execution of kacpid, if we had any notify requests pending, but Linus
decided that it was too complex (involved either changes to workqueue
to see if it's empty or atomic inc/dec).
Now you see last variant which adds yield() to every GPE execution.

http://bugzilla.kernel.org/show_bug.cgi?id=5534
http://bugzilla.kernel.org/show_bug.cgi?id=8385

Signed-off-by: Alexey Starikovskiy <alexey.y.starikovskiy@intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>

authored by

Alexey Starikovskiy and committed by
Len Brown
88db5e14 262a7a28

+35 -10
+35 -10
drivers/acpi/osl.c
··· 71 71 static acpi_osd_handler acpi_irq_handler; 72 72 static void *acpi_irq_context; 73 73 static struct workqueue_struct *kacpid_wq; 74 + static struct workqueue_struct *kacpi_notify_wq; 74 75 75 76 static void __init acpi_request_region (struct acpi_generic_address *addr, 76 77 unsigned int length, char *desc) ··· 138 137 return AE_NULL_ENTRY; 139 138 } 140 139 kacpid_wq = create_singlethread_workqueue("kacpid"); 140 + kacpi_notify_wq = create_singlethread_workqueue("kacpi_notify"); 141 141 BUG_ON(!kacpid_wq); 142 - 142 + BUG_ON(!kacpi_notify_wq); 143 143 return AE_OK; 144 144 } 145 145 ··· 152 150 } 153 151 154 152 destroy_workqueue(kacpid_wq); 153 + destroy_workqueue(kacpi_notify_wq); 155 154 156 155 return AE_OK; 157 156 } ··· 606 603 static void acpi_os_execute_deferred(struct work_struct *work) 607 604 { 608 605 struct acpi_os_dpc *dpc = container_of(work, struct acpi_os_dpc, work); 606 + if (!dpc) { 607 + printk(KERN_ERR PREFIX "Invalid (NULL) context\n"); 608 + return; 609 + } 610 + 611 + dpc->function(dpc->context); 612 + kfree(dpc); 613 + 614 + /* Yield cpu to notify thread */ 615 + cond_resched(); 616 + 617 + return; 618 + } 619 + 620 + static void acpi_os_execute_notify(struct work_struct *work) 621 + { 622 + struct acpi_os_dpc *dpc = container_of(work, struct acpi_os_dpc, work); 609 623 610 624 if (!dpc) { 611 625 printk(KERN_ERR PREFIX "Invalid (NULL) context\n"); ··· 657 637 acpi_status status = AE_OK; 658 638 struct acpi_os_dpc *dpc; 659 639 660 - ACPI_FUNCTION_TRACE("os_queue_for_execution"); 661 - 662 640 ACPI_DEBUG_PRINT((ACPI_DB_EXEC, 663 641 "Scheduling function [%p(%p)] for deferred execution.\n", 664 642 function, context)); 665 643 666 644 if (!function) 667 - return_ACPI_STATUS(AE_BAD_PARAMETER); 645 + return AE_BAD_PARAMETER; 668 646 669 647 /* 670 648 * Allocate/initialize DPC structure. Note that this memory will be ··· 680 662 dpc->function = function; 681 663 dpc->context = context; 682 664 683 - INIT_WORK(&dpc->work, acpi_os_execute_deferred); 684 - if (!queue_work(kacpid_wq, &dpc->work)) { 685 - ACPI_DEBUG_PRINT((ACPI_DB_ERROR, 665 + if (type == OSL_NOTIFY_HANDLER) { 666 + INIT_WORK(&dpc->work, acpi_os_execute_notify); 667 + if (!queue_work(kacpi_notify_wq, &dpc->work)) { 668 + status = AE_ERROR; 669 + kfree(dpc); 670 + } 671 + } else { 672 + INIT_WORK(&dpc->work, acpi_os_execute_deferred); 673 + if (!queue_work(kacpid_wq, &dpc->work)) { 674 + ACPI_DEBUG_PRINT((ACPI_DB_ERROR, 686 675 "Call to queue_work() failed.\n")); 687 - kfree(dpc); 688 - status = AE_ERROR; 676 + status = AE_ERROR; 677 + kfree(dpc); 678 + } 689 679 } 690 - 691 680 return_ACPI_STATUS(status); 692 681 } 693 682