Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

vfio/iova_bitmap: refactor iova_bitmap_set() to better handle page boundaries

Commit f38044e5ef58 ("vfio/iova_bitmap: Fix PAGE_SIZE unaligned bitmaps")
had fixed the unaligned bitmaps by capping the remaining iterable set at
the start of the bitmap. Although, that mistakenly worked around
iova_bitmap_set() incorrectly setting bits across page boundary.

Fix this by reworking the loop inside iova_bitmap_set() to iterate over a
range of bits to set (cur_bit .. last_bit) which may span different pinned
pages, thus updating @page_idx and @offset as it sets the bits. The
previous cap to the first page is now adjusted to be always accounted
rather than when there's only a non-zero pgoff.

While at it, make @page_idx , @offset and @nbits to be unsigned int given
that it won't be more than 512 and 4096 respectively (even a bigger
PAGE_SIZE or a smaller struct page size won't make this bigger than the
above 32-bit max). Also, delete the stale kdoc on Return type.

Cc: Avihai Horon <avihaih@nvidia.com>
Fixes: f38044e5ef58 ("vfio/iova_bitmap: Fix PAGE_SIZE unaligned bitmaps")
Co-developed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joao Martins <joao.m.martins@oracle.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Tested-by: Avihai Horon <avihaih@nvidia.com>
Link: https://lore.kernel.org/r/20221129131235.38880-1-joao.m.martins@oracle.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

authored by

Joao Martins and committed by
Alex Williamson
b058ea3a 2f5d8cef

+13 -17
+13 -17
drivers/vfio/iova_bitmap.c
··· 298 298 { 299 299 unsigned long remaining, bytes; 300 300 301 - /* Cap to one page in the first iteration, if PAGE_SIZE unaligned. */ 302 - bytes = !bitmap->mapped.pgoff ? bitmap->mapped.npages << PAGE_SHIFT : 303 - PAGE_SIZE - bitmap->mapped.pgoff; 301 + bytes = (bitmap->mapped.npages << PAGE_SHIFT) - bitmap->mapped.pgoff; 304 302 305 303 remaining = bitmap->mapped_total_index - bitmap->mapped_base_index; 306 304 remaining = min_t(unsigned long, remaining, ··· 397 399 * Set the bits corresponding to the range [iova .. iova+length-1] in 398 400 * the user bitmap. 399 401 * 400 - * Return: The number of bits set. 401 402 */ 402 403 void iova_bitmap_set(struct iova_bitmap *bitmap, 403 404 unsigned long iova, size_t length) 404 405 { 405 406 struct iova_bitmap_map *mapped = &bitmap->mapped; 406 - unsigned long offset = (iova - mapped->iova) >> mapped->pgshift; 407 - unsigned long nbits = max_t(unsigned long, 1, length >> mapped->pgshift); 408 - unsigned long page_idx = offset / BITS_PER_PAGE; 409 - unsigned long page_offset = mapped->pgoff; 410 - void *kaddr; 411 - 412 - offset = offset % BITS_PER_PAGE; 407 + unsigned long cur_bit = ((iova - mapped->iova) >> 408 + mapped->pgshift) + mapped->pgoff * BITS_PER_BYTE; 409 + unsigned long last_bit = (((iova + length - 1) - mapped->iova) >> 410 + mapped->pgshift) + mapped->pgoff * BITS_PER_BYTE; 413 411 414 412 do { 415 - unsigned long size = min(BITS_PER_PAGE - offset, nbits); 413 + unsigned int page_idx = cur_bit / BITS_PER_PAGE; 414 + unsigned int offset = cur_bit % BITS_PER_PAGE; 415 + unsigned int nbits = min(BITS_PER_PAGE - offset, 416 + last_bit - cur_bit + 1); 417 + void *kaddr; 416 418 417 419 kaddr = kmap_local_page(mapped->pages[page_idx]); 418 - bitmap_set(kaddr + page_offset, offset, size); 420 + bitmap_set(kaddr, offset, nbits); 419 421 kunmap_local(kaddr); 420 - page_offset = offset = 0; 421 - nbits -= size; 422 - page_idx++; 423 - } while (nbits > 0); 422 + cur_bit += nbits; 423 + } while (cur_bit <= last_bit); 424 424 } 425 425 EXPORT_SYMBOL_GPL(iova_bitmap_set);