tjh.dev
Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
io-mapping: Provide iomap_local variant
Similar to kmap local provide a iomap local variant which only disables
migration, but neither disables pagefaults nor preemption.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/r/20201118204007.561220818@linutronix.de
+73
-31
+45
-29
Documentation/driver-api/io-mapping.rst
+45
-29
Documentation/driver-api/io-mapping.rst
···
20
20
mappable, while 'size' indicates how large a mapping region to
21
21
enable. Both are in bytes.
22
22
23
-
This _wc variant provides a mapping which may only be used
24
-
with the io_mapping_map_atomic_wc or io_mapping_map_wc.
23
+
This _wc variant provides a mapping which may only be used with
24
+
io_mapping_map_atomic_wc(), io_mapping_map_local_wc() or
25
+
io_mapping_map_wc().
25
26
26
-
With this mapping object, individual pages can be mapped either atomically
27
-
or not, depending on the necessary scheduling environment. Of course, atomic
28
-
maps are more efficient::
27
+
With this mapping object, individual pages can be mapped either temporarily
28
+
or long term, depending on the requirements. Of course, temporary maps are
29
+
more efficient. They come in two flavours::
30
+
31
+
void *io_mapping_map_local_wc(struct io_mapping *mapping,
32
+
unsigned long offset)
29
33
30
34
void *io_mapping_map_atomic_wc(struct io_mapping *mapping,
31
35
unsigned long offset)
32
36
33
-
'offset' is the offset within the defined mapping region.
34
-
Accessing addresses beyond the region specified in the
35
-
creation function yields undefined results. Using an offset
36
-
which is not page aligned yields an undefined result. The
37
-
return value points to a single page in CPU address space.
37
+
'offset' is the offset within the defined mapping region. Accessing
38
+
addresses beyond the region specified in the creation function yields
39
+
undefined results. Using an offset which is not page aligned yields an
40
+
undefined result. The return value points to a single page in CPU address
41
+
space.
38
42
39
-
This _wc variant returns a write-combining map to the
40
-
page and may only be used with mappings created by
41
-
io_mapping_create_wc
43
+
This _wc variant returns a write-combining map to the page and may only be
44
+
used with mappings created by io_mapping_create_wc()
42
45
43
-
Note that the task may not sleep while holding this page
44
-
mapped.
46
+
Temporary mappings are only valid in the context of the caller. The mapping
47
+
is not guaranteed to be globaly visible.
45
48
46
-
::
49
+
io_mapping_map_local_wc() has a side effect on X86 32bit as it disables
50
+
migration to make the mapping code work. No caller can rely on this side
51
+
effect.
47
52
53
+
io_mapping_map_atomic_wc() has the side effect of disabling preemption and
54
+
pagefaults. Don't use in new code. Use io_mapping_map_local_wc() instead.
55
+
56
+
Nested mappings need to be undone in reverse order because the mapping
57
+
code uses a stack for keeping track of them::
58
+
59
+
addr1 = io_mapping_map_local_wc(map1, offset1);
60
+
addr2 = io_mapping_map_local_wc(map2, offset2);
61
+
...
62
+
io_mapping_unmap_local(addr2);
63
+
io_mapping_unmap_local(addr1);
64
+
65
+
The mappings are released with::
66
+
67
+
void io_mapping_unmap_local(void *vaddr)
48
68
void io_mapping_unmap_atomic(void *vaddr)
49
69
50
-
'vaddr' must be the value returned by the last
51
-
io_mapping_map_atomic_wc call. This unmaps the specified
52
-
page and allows the task to sleep once again.
70
+
'vaddr' must be the value returned by the last io_mapping_map_local_wc() or
71
+
io_mapping_map_atomic_wc() call. This unmaps the specified mapping and
72
+
undoes the side effects of the mapping functions.
53
73
54
-
If you need to sleep while holding the lock, you can use the non-atomic
55
-
variant, although they may be significantly slower.
56
-
57
-
::
74
+
If you need to sleep while holding a mapping, you can use the regular
75
+
variant, although this may be significantly slower::
58
76
59
77
void *io_mapping_map_wc(struct io_mapping *mapping,
60
78
unsigned long offset)
61
79
62
-
This works like io_mapping_map_atomic_wc except it allows
63
-
the task to sleep while holding the page mapped.
80
+
This works like io_mapping_map_atomic/local_wc() except it has no side
81
+
effects and the pointer is globaly visible.
64
82
65
-
66
-
::
83
+
The mappings are released with::
67
84
68
85
void io_mapping_unmap(void *vaddr)
69
86
70
-
This works like io_mapping_unmap_atomic, except it is used
71
-
for pages mapped with io_mapping_map_wc.
87
+
Use for pages mapped with io_mapping_map_wc().
72
88
73
89
At driver close time, the io_mapping object must be freed::
74
90
+28
-2
include/linux/io-mapping.h
+28
-2
include/linux/io-mapping.h
···
83
83
}
84
84
85
85
static inline void __iomem *
86
+
io_mapping_map_local_wc(struct io_mapping *mapping, unsigned long offset)
87
+
{
88
+
resource_size_t phys_addr;
89
+
90
+
BUG_ON(offset >= mapping->size);
91
+
phys_addr = mapping->base + offset;
92
+
return __iomap_local_pfn_prot(PHYS_PFN(phys_addr), mapping->prot);
93
+
}
94
+
95
+
static inline void io_mapping_unmap_local(void __iomem *vaddr)
96
+
{
97
+
kunmap_local_indexed((void __force *)vaddr);
98
+
}
99
+
100
+
static inline void __iomem *
86
101
io_mapping_map_wc(struct io_mapping *mapping,
87
102
unsigned long offset,
88
103
unsigned long size)
···
116
101
iounmap(vaddr);
117
102
}
118
103
119
-
#else
104
+
#else /* HAVE_ATOMIC_IOMAP */
120
105
121
106
#include <linux/uaccess.h>
122
107
···
181
166
preempt_enable();
182
167
}
183
168
184
-
#endif /* HAVE_ATOMIC_IOMAP */
169
+
static inline void __iomem *
170
+
io_mapping_map_local_wc(struct io_mapping *mapping, unsigned long offset)
171
+
{
172
+
return io_mapping_map_wc(mapping, offset, PAGE_SIZE);
173
+
}
174
+
175
+
static inline void io_mapping_unmap_local(void __iomem *vaddr)
176
+
{
177
+
io_mapping_unmap(vaddr);
178
+
}
179
+
180
+
#endif /* !HAVE_ATOMIC_IOMAP */
185
181
186
182
static inline struct io_mapping *
187
183
io_mapping_create_wc(resource_size_t base,