Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
tjh.dev
kernel
os
linux
1================================================
2Generic bitfield packing and unpacking functions
3================================================
4
5Problem statement
6-----------------
7
8When working with hardware, one has to choose between several approaches of
9interfacing with it.
10One can memory-map a pointer to a carefully crafted struct over the hardware
11device's memory region, and access its fields as struct members (potentially
12declared as bitfields). But writing code this way would make it less portable,
13due to potential endianness mismatches between the CPU and the hardware device.
14Additionally, one has to pay close attention when translating register
15definitions from the hardware documentation into bit field indices for the
16structs. Also, some hardware (typically networking equipment) tends to group
17its register fields in ways that violate any reasonable word boundaries
18(sometimes even 64 bit ones). This creates the inconvenience of having to
19define "high" and "low" portions of register fields within the struct.
20A more robust alternative to struct field definitions would be to extract the
21required fields by shifting the appropriate number of bits. But this would
22still not protect from endianness mismatches, except if all memory accesses
23were performed byte-by-byte. Also the code can easily get cluttered, and the
24high-level idea might get lost among the many bit shifts required.
25Many drivers take the bit-shifting approach and then attempt to reduce the
26clutter with tailored macros, but more often than not these macros take
27shortcuts that still prevent the code from being truly portable.
28
29The solution
30------------
31
32This API deals with 2 basic operations:
33
34 - Packing a CPU-usable number into a memory buffer (with hardware
35 constraints/quirks)
36 - Unpacking a memory buffer (which has hardware constraints/quirks)
37 into a CPU-usable number.
38
39The API offers an abstraction over said hardware constraints and quirks,
40over CPU endianness and therefore between possible mismatches between
41the two.
42
43The basic unit of these API functions is the u64. From the CPU's
44perspective, bit 63 always means bit offset 7 of byte 7, albeit only
45logically. The question is: where do we lay this bit out in memory?
46
47The following examples cover the memory layout of a packed u64 field.
48The byte offsets in the packed buffer are always implicitly 0, 1, ... 7.
49What the examples show is where the logical bytes and bits sit.
50
511. Normally (no quirks), we would do it like this:
52
53::
54
55 63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32
56 7 6 5 4
57 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
58 3 2 1 0
59
60That is, the MSByte (7) of the CPU-usable u64 sits at memory offset 0, and the
61LSByte (0) of the u64 sits at memory offset 7.
62This corresponds to what most folks would regard to as "big endian", where
63bit i corresponds to the number 2^i. This is also referred to in the code
64comments as "logical" notation.
65
66
672. If QUIRK_MSB_ON_THE_RIGHT is set, we do it like this:
68
69::
70
71 56 57 58 59 60 61 62 63 48 49 50 51 52 53 54 55 40 41 42 43 44 45 46 47 32 33 34 35 36 37 38 39
72 7 6 5 4
73 24 25 26 27 28 29 30 31 16 17 18 19 20 21 22 23 8 9 10 11 12 13 14 15 0 1 2 3 4 5 6 7
74 3 2 1 0
75
76That is, QUIRK_MSB_ON_THE_RIGHT does not affect byte positioning, but
77inverts bit offsets inside a byte.
78
79
803. If QUIRK_LITTLE_ENDIAN is set, we do it like this:
81
82::
83
84 39 38 37 36 35 34 33 32 47 46 45 44 43 42 41 40 55 54 53 52 51 50 49 48 63 62 61 60 59 58 57 56
85 4 5 6 7
86 7 6 5 4 3 2 1 0 15 14 13 12 11 10 9 8 23 22 21 20 19 18 17 16 31 30 29 28 27 26 25 24
87 0 1 2 3
88
89Therefore, QUIRK_LITTLE_ENDIAN means that inside the memory region, every
90byte from each 4-byte word is placed at its mirrored position compared to
91the boundary of that word.
92
934. If QUIRK_MSB_ON_THE_RIGHT and QUIRK_LITTLE_ENDIAN are both set, we do it
94 like this:
95
96::
97
98 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
99 4 5 6 7
100 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
101 0 1 2 3
102
103
1045. If just QUIRK_LSW32_IS_FIRST is set, we do it like this:
105
106::
107
108 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
109 3 2 1 0
110 63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32
111 7 6 5 4
112
113In this case the 8 byte memory region is interpreted as follows: first
1144 bytes correspond to the least significant 4-byte word, next 4 bytes to
115the more significant 4-byte word.
116
117
1186. If QUIRK_LSW32_IS_FIRST and QUIRK_MSB_ON_THE_RIGHT are set, we do it like
119 this:
120
121::
122
123 24 25 26 27 28 29 30 31 16 17 18 19 20 21 22 23 8 9 10 11 12 13 14 15 0 1 2 3 4 5 6 7
124 3 2 1 0
125 56 57 58 59 60 61 62 63 48 49 50 51 52 53 54 55 40 41 42 43 44 45 46 47 32 33 34 35 36 37 38 39
126 7 6 5 4
127
128
1297. If QUIRK_LSW32_IS_FIRST and QUIRK_LITTLE_ENDIAN are set, it looks like
130 this:
131
132::
133
134 7 6 5 4 3 2 1 0 15 14 13 12 11 10 9 8 23 22 21 20 19 18 17 16 31 30 29 28 27 26 25 24
135 0 1 2 3
136 39 38 37 36 35 34 33 32 47 46 45 44 43 42 41 40 55 54 53 52 51 50 49 48 63 62 61 60 59 58 57 56
137 4 5 6 7
138
139
1408. If QUIRK_LSW32_IS_FIRST, QUIRK_LITTLE_ENDIAN and QUIRK_MSB_ON_THE_RIGHT
141 are set, it looks like this:
142
143::
144
145 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
146 0 1 2 3
147 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
148 4 5 6 7
149
150
151We always think of our offsets as if there were no quirk, and we translate
152them afterwards, before accessing the memory region.
153
154Note on buffer lengths not multiple of 4
155----------------------------------------
156
157To deal with memory layout quirks where groups of 4 bytes are laid out "little
158endian" relative to each other, but "big endian" within the group itself, the
159concept of groups of 4 bytes is intrinsic to the packing API (not to be
160confused with the memory access, which is performed byte by byte, though).
161
162With buffer lengths not multiple of 4, this means one group will be incomplete.
163Depending on the quirks, this may lead to discontinuities in the bit fields
164accessible through the buffer. The packing API assumes discontinuities were not
165the intention of the memory layout, so it avoids them by effectively logically
166shortening the most significant group of 4 octets to the number of octets
167actually available.
168
169Example with a 31 byte sized buffer given below. Physical buffer offsets are
170implicit, and increase from left to right within a group, and from top to
171bottom within a column.
172
173No quirks:
174
175::
176
177 31 29 28 | Group 7 (most significant)
178 27 26 25 24 | Group 6
179 23 22 21 20 | Group 5
180 19 18 17 16 | Group 4
181 15 14 13 12 | Group 3
182 11 10 9 8 | Group 2
183 7 6 5 4 | Group 1
184 3 2 1 0 | Group 0 (least significant)
185
186QUIRK_LSW32_IS_FIRST:
187
188::
189
190 3 2 1 0 | Group 0 (least significant)
191 7 6 5 4 | Group 1
192 11 10 9 8 | Group 2
193 15 14 13 12 | Group 3
194 19 18 17 16 | Group 4
195 23 22 21 20 | Group 5
196 27 26 25 24 | Group 6
197 30 29 28 | Group 7 (most significant)
198
199QUIRK_LITTLE_ENDIAN:
200
201::
202
203 30 28 29 | Group 7 (most significant)
204 24 25 26 27 | Group 6
205 20 21 22 23 | Group 5
206 16 17 18 19 | Group 4
207 12 13 14 15 | Group 3
208 8 9 10 11 | Group 2
209 4 5 6 7 | Group 1
210 0 1 2 3 | Group 0 (least significant)
211
212QUIRK_LITTLE_ENDIAN | QUIRK_LSW32_IS_FIRST:
213
214::
215
216 0 1 2 3 | Group 0 (least significant)
217 4 5 6 7 | Group 1
218 8 9 10 11 | Group 2
219 12 13 14 15 | Group 3
220 16 17 18 19 | Group 4
221 20 21 22 23 | Group 5
222 24 25 26 27 | Group 6
223 28 29 30 | Group 7 (most significant)
224
225Intended use
226------------
227
228Drivers that opt to use this API first need to identify which of the above 3
229quirk combinations (for a total of 8) match what the hardware documentation
230describes.
231
232There are 3 supported usage patterns, detailed below.
233
234packing()
235^^^^^^^^^
236
237This API function is deprecated.
238
239The packing() function returns an int-encoded error code, which protects the
240programmer against incorrect API use. The errors are not expected to occur
241during runtime, therefore it is reasonable to wrap packing() into a custom
242function which returns void and swallows those errors. Optionally it can
243dump stack or print the error description.
244
245.. code-block:: c
246
247 void my_packing(void *buf, u64 *val, int startbit, int endbit,
248 size_t len, enum packing_op op)
249 {
250 int err;
251
252 /* Adjust quirks accordingly */
253 err = packing(buf, val, startbit, endbit, len, op, QUIRK_LSW32_IS_FIRST);
254 if (likely(!err))
255 return;
256
257 if (err == -EINVAL) {
258 pr_err("Start bit (%d) expected to be larger than end (%d)\n",
259 startbit, endbit);
260 } else if (err == -ERANGE) {
261 if ((startbit - endbit + 1) > 64)
262 pr_err("Field %d-%d too large for 64 bits!\n",
263 startbit, endbit);
264 else
265 pr_err("Cannot store %llx inside bits %d-%d (would truncate)\n",
266 *val, startbit, endbit);
267 }
268 dump_stack();
269 }
270
271pack() and unpack()
272^^^^^^^^^^^^^^^^^^^
273
274These are const-correct variants of packing(), and eliminate the last "enum
275packing_op op" argument.
276
277Calling pack(...) is equivalent, and preferred, to calling packing(..., PACK).
278
279Calling unpack(...) is equivalent, and preferred, to calling packing(..., UNPACK).
280
281pack_fields() and unpack_fields()
282^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
283
284The library exposes optimized functions for the scenario where there are many
285fields represented in a buffer, and it encourages consumer drivers to avoid
286repetitive calls to pack() and unpack() for each field, but instead use
287pack_fields() and unpack_fields(), which reduces the code footprint.
288
289These APIs use field definitions in arrays of ``struct packed_field_u8`` or
290``struct packed_field_u16``, allowing consumer drivers to minimize the size
291of these arrays according to their custom requirements.
292
293The pack_fields() and unpack_fields() API functions are actually macros which
294automatically select the appropriate function at compile time, based on the
295type of the fields array passed in.
296
297An additional benefit over pack() and unpack() is that sanity checks on the
298field definitions are handled at compile time with ``BUILD_BUG_ON`` rather
299than only when the offending code is executed. These functions return void and
300wrapping them to handle unexpected errors is not necessary.
301
302It is recommended, but not required, that you wrap your packed buffer into a
303structured type with a fixed size. This generally makes it easier for the
304compiler to enforce that the correct size buffer is used.
305
306Here is an example of how to use the fields APIs:
307
308.. code-block:: c
309
310 /* Ordering inside the unpacked structure is flexible and can be different
311 * from the packed buffer. Here, it is optimized to reduce padding.
312 */
313 struct data {
314 u64 field3;
315 u32 field4;
316 u16 field1;
317 u8 field2;
318 };
319
320 #define SIZE 13
321
322 typedef struct __packed { u8 buf[SIZE]; } packed_buf_t;
323
324 static const struct packed_field_u8 fields[] = {
325 PACKED_FIELD(100, 90, struct data, field1),
326 PACKED_FIELD(90, 87, struct data, field2),
327 PACKED_FIELD(86, 30, struct data, field3),
328 PACKED_FIELD(29, 0, struct data, field4),
329 };
330
331 void unpack_your_data(const packed_buf_t *buf, struct data *unpacked)
332 {
333 BUILD_BUG_ON(sizeof(*buf) != SIZE;
334
335 unpack_fields(buf, sizeof(*buf), unpacked, fields,
336 QUIRK_LITTLE_ENDIAN);
337 }
338
339 void pack_your_data(const struct data *unpacked, packed_buf_t *buf)
340 {
341 BUILD_BUG_ON(sizeof(*buf) != SIZE;
342
343 pack_fields(buf, sizeof(*buf), unpacked, fields,
344 QUIRK_LITTLE_ENDIAN);
345 }