Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

eventpoll: Control irq suspension for prefer_busy_poll

When events are reported to userland and prefer_busy_poll is set, irqs
are temporarily suspended using napi_suspend_irqs.

If no events are found and ep_poll would go to sleep, irq suspension is
cancelled using napi_resume_irqs.

Signed-off-by: Martin Karsten <mkarsten@uwaterloo.ca>
Co-developed-by: Joe Damato <jdamato@fastly.com>
Signed-off-by: Joe Damato <jdamato@fastly.com>
Tested-by: Joe Damato <jdamato@fastly.com>
Tested-by: Martin Karsten <mkarsten@uwaterloo.ca>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Sridhar Samudrala <sridhar.samudrala@intel.com>
Link: https://patch.msgid.link/20241109050245.191288-5-jdamato@fastly.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

authored by

Martin Karsten and committed by
Jakub Kicinski
8a6de262 ab5b28b0

+31 -1
+31 -1
fs/eventpoll.c
··· 457 457 * it back in when we have moved a socket with a valid NAPI 458 458 * ID onto the ready list. 459 459 */ 460 + if (prefer_busy_poll) 461 + napi_resume_irqs(napi_id); 460 462 ep->napi_id = 0; 461 463 return false; 462 464 } ··· 542 540 } 543 541 } 544 542 543 + static void ep_suspend_napi_irqs(struct eventpoll *ep) 544 + { 545 + unsigned int napi_id = READ_ONCE(ep->napi_id); 546 + 547 + if (napi_id >= MIN_NAPI_ID && READ_ONCE(ep->prefer_busy_poll)) 548 + napi_suspend_irqs(napi_id); 549 + } 550 + 551 + static void ep_resume_napi_irqs(struct eventpoll *ep) 552 + { 553 + unsigned int napi_id = READ_ONCE(ep->napi_id); 554 + 555 + if (napi_id >= MIN_NAPI_ID && READ_ONCE(ep->prefer_busy_poll)) 556 + napi_resume_irqs(napi_id); 557 + } 558 + 545 559 #else 546 560 547 561 static inline bool ep_busy_loop(struct eventpoll *ep, int nonblock) ··· 573 555 unsigned long arg) 574 556 { 575 557 return -EOPNOTSUPP; 558 + } 559 + 560 + static void ep_suspend_napi_irqs(struct eventpoll *ep) 561 + { 562 + } 563 + 564 + static void ep_resume_napi_irqs(struct eventpoll *ep) 565 + { 576 566 } 577 567 578 568 #endif /* CONFIG_NET_RX_BUSY_POLL */ ··· 814 788 815 789 static void ep_free(struct eventpoll *ep) 816 790 { 791 + ep_resume_napi_irqs(ep); 817 792 mutex_destroy(&ep->mtx); 818 793 free_uid(ep->user); 819 794 wakeup_source_unregister(ep->ws); ··· 2032 2005 * trying again in search of more luck. 2033 2006 */ 2034 2007 res = ep_send_events(ep, events, maxevents); 2035 - if (res) 2008 + if (res) { 2009 + if (res > 0) 2010 + ep_suspend_napi_irqs(ep); 2036 2011 return res; 2012 + } 2037 2013 } 2038 2014 2039 2015 if (timed_out)