Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

io_uring: move iopoll ctx fields around

Move poll_multi_queue and iopoll_list to the submission cache line, it
doesn't make much sense to keep them separately, and is better place
for it in general.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/5b03cf7e6652e350e6e70a917eec72ba9f33b97b.1692916914.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

authored by

Pavel Begunkov and committed by
Jens Axboe
644c4a7a 0aa7aa5f

+11 -14
+11 -14
include/linux/io_uring_types.h
··· 256 256 struct io_hash_table cancel_table_locked; 257 257 struct io_alloc_cache apoll_cache; 258 258 struct io_alloc_cache netmsg_cache; 259 + 260 + /* 261 + * ->iopoll_list is protected by the ctx->uring_lock for 262 + * io_uring instances that don't use IORING_SETUP_SQPOLL. 263 + * For SQPOLL, only the single threaded io_sq_thread() will 264 + * manipulate the list, hence no extra locking is needed there. 265 + */ 266 + struct io_wq_work_list iopoll_list; 267 + bool poll_multi_queue; 259 268 } ____cacheline_aligned_in_smp; 260 269 261 270 struct { ··· 293 284 struct wait_queue_head cq_wait; 294 285 } ____cacheline_aligned_in_smp; 295 286 296 - struct { 297 - spinlock_t completion_lock; 298 - 299 - bool poll_multi_queue; 300 - 301 - /* 302 - * ->iopoll_list is protected by the ctx->uring_lock for 303 - * io_uring instances that don't use IORING_SETUP_SQPOLL. 304 - * For SQPOLL, only the single threaded io_sq_thread() will 305 - * manipulate the list, hence no extra locking is needed there. 306 - */ 307 - struct io_wq_work_list iopoll_list; 308 - } ____cacheline_aligned_in_smp; 309 - 310 287 /* timeouts */ 311 288 struct { 312 289 spinlock_t timeout_lock; ··· 302 307 } ____cacheline_aligned_in_smp; 303 308 304 309 struct io_uring_cqe completion_cqes[16]; 310 + 311 + spinlock_t completion_lock; 305 312 306 313 /* IRQ completion list, under ->completion_lock */ 307 314 struct io_wq_work_list locked_free_list;