Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

docs: Add Documentation for Mediated devices

Add file Documentation/vfio-mediated-device.txt that include details of
mediated device framework.

Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
Signed-off-by: Neo Jia <cjia@nvidia.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

authored by

Kirti Wankhede and committed by
Alex Williamson
8e1c5a40 2818c6e9

+298
+297
Documentation/vfio-mediated-device.txt
··· 1 + /* 2 + * VFIO Mediated devices 3 + * 4 + * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. 5 + * Author: Neo Jia <cjia@nvidia.com> 6 + * Kirti Wankhede <kwankhede@nvidia.com> 7 + * 8 + * This program is free software; you can redistribute it and/or modify 9 + * it under the terms of the GNU General Public License version 2 as 10 + * published by the Free Software Foundation. 11 + */ 12 + 13 + Virtual Function I/O (VFIO) Mediated devices[1] 14 + =============================================== 15 + 16 + The number of use cases for virtualizing DMA devices that do not have built-in 17 + SR_IOV capability is increasing. Previously, to virtualize such devices, 18 + developers had to create their own management interfaces and APIs, and then 19 + integrate them with user space software. To simplify integration with user space 20 + software, we have identified common requirements and a unified management 21 + interface for such devices. 22 + 23 + The VFIO driver framework provides unified APIs for direct device access. It is 24 + an IOMMU/device-agnostic framework for exposing direct device access to user 25 + space in a secure, IOMMU-protected environment. This framework is used for 26 + multiple devices, such as GPUs, network adapters, and compute accelerators. With 27 + direct device access, virtual machines or user space applications have direct 28 + access to the physical device. This framework is reused for mediated devices. 29 + 30 + The mediated core driver provides a common interface for mediated device 31 + management that can be used by drivers of different devices. This module 32 + provides a generic interface to perform these operations: 33 + 34 + * Create and destroy a mediated device 35 + * Add a mediated device to and remove it from a mediated bus driver 36 + * Add a mediated device to and remove it from an IOMMU group 37 + 38 + The mediated core driver also provides an interface to register a bus driver. 39 + For example, the mediated VFIO mdev driver is designed for mediated devices and 40 + supports VFIO APIs. The mediated bus driver adds a mediated device to and 41 + removes it from a VFIO group. 42 + 43 + The following high-level block diagram shows the main components and interfaces 44 + in the VFIO mediated driver framework. The diagram shows NVIDIA, Intel, and IBM 45 + devices as examples, as these devices are the first devices to use this module. 46 + 47 + +---------------+ 48 + | | 49 + | +-----------+ | mdev_register_driver() +--------------+ 50 + | | | +<------------------------+ | 51 + | | mdev | | | | 52 + | | bus | +------------------------>+ vfio_mdev.ko |<-> VFIO user 53 + | | driver | | probe()/remove() | | APIs 54 + | | | | +--------------+ 55 + | +-----------+ | 56 + | | 57 + | MDEV CORE | 58 + | MODULE | 59 + | mdev.ko | 60 + | +-----------+ | mdev_register_device() +--------------+ 61 + | | | +<------------------------+ | 62 + | | | | | nvidia.ko |<-> physical 63 + | | | +------------------------>+ | device 64 + | | | | callbacks +--------------+ 65 + | | Physical | | 66 + | | device | | mdev_register_device() +--------------+ 67 + | | interface | |<------------------------+ | 68 + | | | | | i915.ko |<-> physical 69 + | | | +------------------------>+ | device 70 + | | | | callbacks +--------------+ 71 + | | | | 72 + | | | | mdev_register_device() +--------------+ 73 + | | | +<------------------------+ | 74 + | | | | | ccw_device.ko|<-> physical 75 + | | | +------------------------>+ | device 76 + | | | | callbacks +--------------+ 77 + | +-----------+ | 78 + +---------------+ 79 + 80 + 81 + Registration Interfaces 82 + ======================= 83 + 84 + The mediated core driver provides the following types of registration 85 + interfaces: 86 + 87 + * Registration interface for a mediated bus driver 88 + * Physical device driver interface 89 + 90 + Registration Interface for a Mediated Bus Driver 91 + ------------------------------------------------ 92 + 93 + The registration interface for a mediated bus driver provides the following 94 + structure to represent a mediated device's driver: 95 + 96 + /* 97 + * struct mdev_driver [2] - Mediated device's driver 98 + * @name: driver name 99 + * @probe: called when new device created 100 + * @remove: called when device removed 101 + * @driver: device driver structure 102 + */ 103 + struct mdev_driver { 104 + const char *name; 105 + int (*probe) (struct device *dev); 106 + void (*remove) (struct device *dev); 107 + struct device_driver driver; 108 + }; 109 + 110 + A mediated bus driver for mdev should use this structure in the function calls 111 + to register and unregister itself with the core driver: 112 + 113 + * Register: 114 + 115 + extern int mdev_register_driver(struct mdev_driver *drv, 116 + struct module *owner); 117 + 118 + * Unregister: 119 + 120 + extern void mdev_unregister_driver(struct mdev_driver *drv); 121 + 122 + The mediated bus driver is responsible for adding mediated devices to the VFIO 123 + group when devices are bound to the driver and removing mediated devices from 124 + the VFIO when devices are unbound from the driver. 125 + 126 + 127 + Physical Device Driver Interface 128 + -------------------------------- 129 + 130 + The physical device driver interface provides the parent_ops[3] structure to 131 + define the APIs to manage work in the mediated core driver that is related to 132 + the physical device. 133 + 134 + The structures in the parent_ops structure are as follows: 135 + 136 + * dev_attr_groups: attributes of the parent device 137 + * mdev_attr_groups: attributes of the mediated device 138 + * supported_config: attributes to define supported configurations 139 + 140 + The functions in the parent_ops structure are as follows: 141 + 142 + * create: allocate basic resources in a driver for a mediated device 143 + * remove: free resources in a driver when a mediated device is destroyed 144 + 145 + The callbacks in the parent_ops structure are as follows: 146 + 147 + * open: open callback of mediated device 148 + * close: close callback of mediated device 149 + * ioctl: ioctl callback of mediated device 150 + * read : read emulation callback 151 + * write: write emulation callback 152 + * mmap: mmap emulation callback 153 + 154 + A driver should use the parent_ops structure in the function call to register 155 + itself with the mdev core driver: 156 + 157 + extern int mdev_register_device(struct device *dev, 158 + const struct parent_ops *ops); 159 + 160 + However, the parent_ops structure is not required in the function call that a 161 + driver should use to unregister itself with the mdev core driver: 162 + 163 + extern void mdev_unregister_device(struct device *dev); 164 + 165 + 166 + Mediated Device Management Interface Through sysfs 167 + ================================================== 168 + 169 + The management interface through sysfs enables user space software, such as 170 + libvirt, to query and configure mediated devices in a hardware-agnostic fashion. 171 + This management interface provides flexibility to the underlying physical 172 + device's driver to support features such as: 173 + 174 + * Mediated device hot plug 175 + * Multiple mediated devices in a single virtual machine 176 + * Multiple mediated devices from different physical devices 177 + 178 + Links in the mdev_bus Class Directory 179 + ------------------------------------- 180 + The /sys/class/mdev_bus/ directory contains links to devices that are registered 181 + with the mdev core driver. 182 + 183 + Directories and files under the sysfs for Each Physical Device 184 + -------------------------------------------------------------- 185 + 186 + |- [parent physical device] 187 + |--- Vendor-specific-attributes [optional] 188 + |--- [mdev_supported_types] 189 + | |--- [<type-id>] 190 + | | |--- create 191 + | | |--- name 192 + | | |--- available_instances 193 + | | |--- device_api 194 + | | |--- description 195 + | | |--- [devices] 196 + | |--- [<type-id>] 197 + | | |--- create 198 + | | |--- name 199 + | | |--- available_instances 200 + | | |--- device_api 201 + | | |--- description 202 + | | |--- [devices] 203 + | |--- [<type-id>] 204 + | |--- create 205 + | |--- name 206 + | |--- available_instances 207 + | |--- device_api 208 + | |--- description 209 + | |--- [devices] 210 + 211 + * [mdev_supported_types] 212 + 213 + The list of currently supported mediated device types and their details. 214 + 215 + [<type-id>], device_api, and available_instances are mandatory attributes 216 + that should be provided by vendor driver. 217 + 218 + * [<type-id>] 219 + 220 + The [<type-id>] name is created by adding the the device driver string as a 221 + prefix to the string provided by the vendor driver. This format of this name 222 + is as follows: 223 + 224 + sprintf(buf, "%s-%s", dev_driver_string(parent->dev), group->name); 225 + 226 + * device_api 227 + 228 + This attribute should show which device API is being created, for example, 229 + "vfio-pci" for a PCI device. 230 + 231 + * available_instances 232 + 233 + This attribute should show the number of devices of type <type-id> that can be 234 + created. 235 + 236 + * [device] 237 + 238 + This directory contains links to the devices of type <type-id> that have been 239 + created. 240 + 241 + * name 242 + 243 + This attribute should show human readable name. This is optional attribute. 244 + 245 + * description 246 + 247 + This attribute should show brief features/description of the type. This is 248 + optional attribute. 249 + 250 + Directories and Files Under the sysfs for Each mdev Device 251 + ---------------------------------------------------------- 252 + 253 + |- [parent phy device] 254 + |--- [$MDEV_UUID] 255 + |--- remove 256 + |--- mdev_type {link to its type} 257 + |--- vendor-specific-attributes [optional] 258 + 259 + * remove (write only) 260 + Writing '1' to the 'remove' file destroys the mdev device. The vendor driver can 261 + fail the remove() callback if that device is active and the vendor driver 262 + doesn't support hot unplug. 263 + 264 + Example: 265 + # echo 1 > /sys/bus/mdev/devices/$mdev_UUID/remove 266 + 267 + Mediated device Hot plug: 268 + ------------------------ 269 + 270 + Mediated devices can be created and assigned at runtime. The procedure to hot 271 + plug a mediated device is the same as the procedure to hot plug a PCI device. 272 + 273 + Translation APIs for Mediated Devices 274 + ===================================== 275 + 276 + The following APIs are provided for translating user pfn to host pfn in a VFIO 277 + driver: 278 + 279 + extern int vfio_pin_pages(struct device *dev, unsigned long *user_pfn, 280 + int npage, int prot, unsigned long *phys_pfn); 281 + 282 + extern int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, 283 + int npage); 284 + 285 + These functions call back into the back-end IOMMU module by using the pin_pages 286 + and unpin_pages callbacks of the struct vfio_iommu_driver_ops[4]. Currently 287 + these callbacks are supported in the TYPE1 IOMMU module. To enable them for 288 + other IOMMU backend modules, such as PPC64 sPAPR module, they need to provide 289 + these two callback functions. 290 + 291 + References 292 + ---------- 293 + 294 + [1] See Documentation/vfio.txt for more information on VFIO. 295 + [2] struct mdev_driver in include/linux/mdev.h 296 + [3] struct parent_ops in include/linux/mdev.h 297 + [4] struct vfio_iommu_driver_ops in include/linux/vfio.h
+1
drivers/vfio/mdev/Kconfig
··· 5 5 default n 6 6 help 7 7 Provides a framework to virtualize devices. 8 + See Documentation/vfio-mediated-device.txt for more details. 8 9 9 10 If you don't know what do here, say N. 10 11