Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

net: Implement fault injection forcing skb reallocation

Introduce a fault injection mechanism to force skb reallocation. The
primary goal is to catch bugs related to pointer invalidation after
potential skb reallocation.

The fault injection mechanism aims to identify scenarios where callers
retain pointers to various headers in the skb but fail to reload these
pointers after calling a function that may reallocate the data. This
type of bug can lead to memory corruption or crashes if the old,
now-invalid pointers are used.

By forcing reallocation through fault injection, we can stress-test code
paths and ensure proper pointer management after potential skb
reallocations.

Add a hook for fault injection in the following functions:

* pskb_trim_rcsum()
* pskb_may_pull_reason()
* pskb_trim()

As the other fault injection mechanism, protect it under a debug Kconfig
called CONFIG_FAIL_SKB_REALLOC.

This patch was *heavily* inspired by Jakub's proposal from:
https://lore.kernel.org/all/20240719174140.47a868e6@kernel.org/

CC: Akinobu Mita <akinobu.mita@gmail.com>
Suggested-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Akinobu Mita <akinobu.mita@gmail.com>
Acked-by: Paolo Abeni <pabeni@redhat.com>
Acked-by: Guillaume Nault <gnault@redhat.com>
Link: https://patch.msgid.link/20241107-fault_v6-v6-1-1b82cb6ecacd@debian.org
Signed-off-by: Paolo Abeni <pabeni@redhat.com>

authored by

Breno Leitao and committed by
Paolo Abeni
12079a59 12f077a7

+167
+1
Documentation/admin-guide/kernel-parameters.txt
··· 1546 1546 failslab= 1547 1547 fail_usercopy= 1548 1548 fail_page_alloc= 1549 + fail_skb_realloc= 1549 1550 fail_make_request=[KNL] 1550 1551 General fault injection mechanism. 1551 1552 Format: <interval>,<probability>,<space>,<times>
+40
Documentation/fault-injection/fault-injection.rst
··· 45 45 ALLOW_ERROR_INJECTION() macro, by setting debugfs entries 46 46 under /sys/kernel/debug/fail_function. No boot option supported. 47 47 48 + - fail_skb_realloc 49 + 50 + inject skb (socket buffer) reallocation events into the network path. The 51 + primary goal is to identify and prevent issues related to pointer 52 + mismanagement in the network subsystem. By forcing skb reallocation at 53 + strategic points, this feature creates scenarios where existing pointers to 54 + skb headers become invalid. 55 + 56 + When the fault is injected and the reallocation is triggered, cached pointers 57 + to skb headers and data no longer reference valid memory locations. This 58 + deliberate invalidation helps expose code paths where proper pointer updating 59 + is neglected after a reallocation event. 60 + 61 + By creating these controlled fault scenarios, the system can catch instances 62 + where stale pointers are used, potentially leading to memory corruption or 63 + system instability. 64 + 65 + To select the interface to act on, write the network name to 66 + /sys/kernel/debug/fail_skb_realloc/devname. 67 + If this field is left empty (which is the default value), skb reallocation 68 + will be forced on all network interfaces. 69 + 70 + The effectiveness of this fault detection is enhanced when KASAN is 71 + enabled, as it helps identify invalid memory references and use-after-free 72 + (UAF) issues. 73 + 48 74 - NVMe fault injection 49 75 50 76 inject NVMe status code and retry flag on devices permitted by setting ··· 242 216 use a negative errno, you better use 'printf' instead of 'echo', e.g.: 243 217 $ printf %#x -12 > retval 244 218 219 + - /sys/kernel/debug/fail_skb_realloc/devname: 220 + 221 + Specifies the network interface on which to force SKB reallocation. If 222 + left empty, SKB reallocation will be applied to all network interfaces. 223 + 224 + Example usage:: 225 + 226 + # Force skb reallocation on eth0 227 + echo "eth0" > /sys/kernel/debug/fail_skb_realloc/devname 228 + 229 + # Clear the selection and force skb reallocation on all interfaces 230 + echo "" > /sys/kernel/debug/fail_skb_realloc/devname 231 + 245 232 Boot option 246 233 ^^^^^^^^^^^ 247 234 ··· 266 227 fail_usercopy= 267 228 fail_make_request= 268 229 fail_futex= 230 + fail_skb_realloc= 269 231 mmc_core.fail_request=<interval>,<probability>,<space>,<times> 270 232 271 233 proc entries
+9
include/linux/skbuff.h
··· 2682 2682 #endif /* CONFIG_DEBUG_NET */ 2683 2683 } 2684 2684 2685 + #if defined(CONFIG_FAIL_SKB_REALLOC) 2686 + void skb_might_realloc(struct sk_buff *skb); 2687 + #else 2688 + static inline void skb_might_realloc(struct sk_buff *skb) {} 2689 + #endif 2690 + 2685 2691 /* 2686 2692 * Add data to an sk_buff 2687 2693 */ ··· 2788 2782 pskb_may_pull_reason(struct sk_buff *skb, unsigned int len) 2789 2783 { 2790 2784 DEBUG_NET_WARN_ON_ONCE(len > INT_MAX); 2785 + skb_might_realloc(skb); 2791 2786 2792 2787 if (likely(len <= skb_headlen(skb))) 2793 2788 return SKB_NOT_DROPPED_YET; ··· 3247 3240 3248 3241 static inline int pskb_trim(struct sk_buff *skb, unsigned int len) 3249 3242 { 3243 + skb_might_realloc(skb); 3250 3244 return (len < skb->len) ? __pskb_trim(skb, len) : 0; 3251 3245 } 3252 3246 ··· 4002 3994 4003 3995 static inline int pskb_trim_rcsum(struct sk_buff *skb, unsigned int len) 4004 3996 { 3997 + skb_might_realloc(skb); 4005 3998 if (likely(len >= skb->len)) 4006 3999 return 0; 4007 4000 return pskb_trim_rcsum_slow(skb, len);
+10
lib/Kconfig.debug
··· 2115 2115 Provide fault-injection capability for SunRPC and 2116 2116 its consumers. 2117 2117 2118 + config FAIL_SKB_REALLOC 2119 + bool "Fault-injection capability forcing skb to reallocate" 2120 + depends on FAULT_INJECTION_DEBUG_FS 2121 + help 2122 + Provide fault-injection capability that forces the skb to be 2123 + reallocated, catching possible invalid pointers to the skb. 2124 + 2125 + For more information, check 2126 + Documentation/dev-tools/fault-injection/fault-injection.rst 2127 + 2118 2128 config FAULT_INJECTION_CONFIGFS 2119 2129 bool "Configfs interface for fault-injection capabilities" 2120 2130 depends on FAULT_INJECTION
+1
net/core/Makefile
··· 46 46 obj-$(CONFIG_NET_TEST) += net_test.o 47 47 obj-$(CONFIG_NET_DEVMEM) += devmem.o 48 48 obj-$(CONFIG_DEBUG_NET_SMALL_RTNL) += rtnl_net_debug.o 49 + obj-$(CONFIG_FAIL_SKB_REALLOC) += skb_fault_injection.o
+106
net/core/skb_fault_injection.c
··· 1 + // SPDX-License-Identifier: GPL-2.0-only 2 + 3 + #include <linux/debugfs.h> 4 + #include <linux/fault-inject.h> 5 + #include <linux/netdevice.h> 6 + #include <linux/skbuff.h> 7 + 8 + static struct { 9 + struct fault_attr attr; 10 + char devname[IFNAMSIZ]; 11 + bool filtered; 12 + } skb_realloc = { 13 + .attr = FAULT_ATTR_INITIALIZER, 14 + .filtered = false, 15 + }; 16 + 17 + static bool should_fail_net_realloc_skb(struct sk_buff *skb) 18 + { 19 + struct net_device *net = skb->dev; 20 + 21 + if (skb_realloc.filtered && 22 + strncmp(net->name, skb_realloc.devname, IFNAMSIZ)) 23 + /* device name filter set, but names do not match */ 24 + return false; 25 + 26 + if (!should_fail(&skb_realloc.attr, 1)) 27 + return false; 28 + 29 + return true; 30 + } 31 + ALLOW_ERROR_INJECTION(should_fail_net_realloc_skb, TRUE); 32 + 33 + void skb_might_realloc(struct sk_buff *skb) 34 + { 35 + if (!should_fail_net_realloc_skb(skb)) 36 + return; 37 + 38 + pskb_expand_head(skb, 0, 0, GFP_ATOMIC); 39 + } 40 + EXPORT_SYMBOL(skb_might_realloc); 41 + 42 + static int __init fail_skb_realloc_setup(char *str) 43 + { 44 + return setup_fault_attr(&skb_realloc.attr, str); 45 + } 46 + __setup("fail_skb_realloc=", fail_skb_realloc_setup); 47 + 48 + static void reset_settings(void) 49 + { 50 + skb_realloc.filtered = false; 51 + memset(&skb_realloc.devname, 0, IFNAMSIZ); 52 + } 53 + 54 + static ssize_t devname_write(struct file *file, const char __user *buffer, 55 + size_t count, loff_t *ppos) 56 + { 57 + ssize_t ret; 58 + 59 + reset_settings(); 60 + ret = simple_write_to_buffer(&skb_realloc.devname, IFNAMSIZ, 61 + ppos, buffer, count); 62 + if (ret < 0) 63 + return ret; 64 + 65 + skb_realloc.devname[IFNAMSIZ - 1] = '\0'; 66 + /* Remove a possible \n at the end of devname */ 67 + strim(skb_realloc.devname); 68 + 69 + if (strnlen(skb_realloc.devname, IFNAMSIZ)) 70 + skb_realloc.filtered = true; 71 + 72 + return count; 73 + } 74 + 75 + static ssize_t devname_read(struct file *file, 76 + char __user *buffer, 77 + size_t size, loff_t *ppos) 78 + { 79 + if (!skb_realloc.filtered) 80 + return 0; 81 + 82 + return simple_read_from_buffer(buffer, size, ppos, &skb_realloc.devname, 83 + strlen(skb_realloc.devname)); 84 + } 85 + 86 + static const struct file_operations devname_ops = { 87 + .write = devname_write, 88 + .read = devname_read, 89 + }; 90 + 91 + static int __init fail_skb_realloc_debugfs(void) 92 + { 93 + umode_t mode = S_IFREG | 0600; 94 + struct dentry *dir; 95 + 96 + dir = fault_create_debugfs_attr("fail_skb_realloc", NULL, 97 + &skb_realloc.attr); 98 + if (IS_ERR(dir)) 99 + return PTR_ERR(dir); 100 + 101 + debugfs_create_file("devname", mode, dir, NULL, &devname_ops); 102 + 103 + return 0; 104 + } 105 + 106 + late_initcall(fail_skb_realloc_debugfs);