Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

Merge tag 'printk-for-5.13' of git://git.kernel.org/pub/scm/linux/kernel/git/printk/linux

Pull printk updates from Petr Mladek:

- Stop synchronizing kernel log buffer readers by logbuf_lock. As a
result, the access to the buffer is fully lockless now.

Note that printk() itself still uses locks because it tries to flush
the messages to the console immediately. Also the per-CPU temporary
buffers are still there because they prevent infinite recursion and
serialize backtraces from NMI. All this is going to change in the
future.

- kmsg_dump API rework and cleanup as a side effect of the logbuf_lock
removal.

- Make bstr_printf() aware that %pf and %pF formats could deference the
given pointer.

- Show also page flags by %pGp format.

- Clarify the documentation for plain pointer printing.

- Do not show no_hash_pointers warning multiple times.

- Update Senozhatsky email address.

- Some clean up.

* tag 'printk-for-5.13' of git://git.kernel.org/pub/scm/linux/kernel/git/printk/linux: (24 commits)
lib/vsprintf.c: remove leftover 'f' and 'F' cases from bstr_printf()
printk: clarify the documentation for plain pointer printing
kernel/printk.c: Fixed mundane typos
printk: rename vprintk_func to vprintk
vsprintf: dump full information of page flags in pGp
mm, slub: don't combine pr_err with INFO
mm, slub: use pGp to print page flags
MAINTAINERS: update Senozhatsky email address
lib/vsprintf: do not show no_hash_pointers message multiple times
printk: console: remove unnecessary safe buffer usage
printk: kmsg_dump: remove _nolock() variants
printk: remove logbuf_lock
printk: introduce a kmsg_dump iterator
printk: kmsg_dumper: remove @active field
printk: add syslog_lock
printk: use atomic64_t for devkmsg_user.seq
printk: use seqcount_latch for clear_seq
printk: introduce CONSOLE_LOG_MAX
printk: consolidate kmsg_dump_get_buffer/syslog_print_all code
printk: refactor kmsg_dump_get_buffer()
...

+502 -340
+26 -2
Documentation/core-api/printk-formats.rst
··· 79 79 hashed to prevent leaking information about the kernel memory layout. This 80 80 has the added benefit of providing a unique identifier. On 64-bit machines 81 81 the first 32 bits are zeroed. The kernel will print ``(ptrval)`` until it 82 - gathers enough entropy. If you *really* want the address see %px below. 82 + gathers enough entropy. 83 + 84 + When possible, use specialised modifiers such as %pS or %pB (described below) 85 + to avoid the need of providing an unhashed address that has to be interpreted 86 + post-hoc. If not possible, and the aim of printing the address is to provide 87 + more information for debugging, use %p and boot the kernel with the 88 + ``no_hash_pointers`` parameter during debugging, which will print all %p 89 + addresses unmodified. If you *really* always want the unmodified address, see 90 + %px below. 91 + 92 + If (and only if) you are printing addresses as a content of a virtual file in 93 + e.g. procfs or sysfs (using e.g. seq_printf(), not printk()) read by a 94 + userspace process, use the %pK modifier described below instead of %p or %px. 83 95 84 96 Error Pointers 85 97 -------------- ··· 151 139 users. The behaviour of %pK depends on the kptr_restrict sysctl - see 152 140 Documentation/admin-guide/sysctl/kernel.rst for more details. 153 141 142 + This modifier is *only* intended when producing content of a file read by 143 + userspace from e.g. procfs or sysfs, not for dmesg. Please refer to the 144 + section about %p above for discussion about how to manage hashing pointers 145 + in printk(). 146 + 154 147 Unmodified Addresses 155 148 -------------------- 156 149 ··· 169 152 equivalent to %lx (or %lu). %px is preferred because it is more uniquely 170 153 grep'able. If in the future we need to modify the way the kernel handles 171 154 printing pointers we will be better equipped to find the call sites. 155 + 156 + Before using %px, consider if using %p is sufficient together with enabling the 157 + ``no_hash_pointers`` kernel parameter during debugging sessions (see the %p 158 + description above). One valid scenario for %px might be printing information 159 + immediately before a panic, which prevents any sensitive information to be 160 + exploited anyway, and with %px there would be no need to reproduce the panic 161 + with no_hash_pointers. 172 162 173 163 Pointer Differences 174 164 ------------------- ··· 564 540 565 541 :: 566 542 567 - %pGp referenced|uptodate|lru|active|private 543 + %pGp referenced|uptodate|lru|active|private|node=0|zone=2|lastcpupid=0x1fffff 568 544 %pGg GFP_USER|GFP_DMA32|GFP_NOWARN 569 545 %pGv read|exec|mayread|maywrite|mayexec|denywrite 570 546
+4 -4
MAINTAINERS
··· 14535 14535 14536 14536 PRINTK 14537 14537 M: Petr Mladek <pmladek@suse.com> 14538 - M: Sergey Senozhatsky <sergey.senozhatsky@gmail.com> 14538 + M: Sergey Senozhatsky <senozhatsky@chromium.org> 14539 14539 R: Steven Rostedt <rostedt@goodmis.org> 14540 14540 R: John Ogness <john.ogness@linutronix.de> 14541 14541 S: Maintained ··· 19388 19388 VSPRINTF 19389 19389 M: Petr Mladek <pmladek@suse.com> 19390 19390 M: Steven Rostedt <rostedt@goodmis.org> 19391 - M: Sergey Senozhatsky <sergey.senozhatsky@gmail.com> 19391 + M: Sergey Senozhatsky <senozhatsky@chromium.org> 19392 19392 R: Andy Shevchenko <andriy.shevchenko@linux.intel.com> 19393 19393 R: Rasmus Villemoes <linux@rasmusvillemoes.dk> 19394 19394 S: Maintained ··· 20039 20039 ZRAM COMPRESSED RAM BLOCK DEVICE DRVIER 20040 20040 M: Minchan Kim <minchan@kernel.org> 20041 20041 M: Nitin Gupta <ngupta@vflare.org> 20042 - R: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> 20042 + R: Sergey Senozhatsky <senozhatsky@chromium.org> 20043 20043 L: linux-kernel@vger.kernel.org 20044 20044 S: Maintained 20045 20045 F: Documentation/admin-guide/blockdev/zram.rst ··· 20053 20053 ZSMALLOC COMPRESSED SLAB MEMORY ALLOCATOR 20054 20054 M: Minchan Kim <minchan@kernel.org> 20055 20055 M: Nitin Gupta <ngupta@vflare.org> 20056 - R: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> 20056 + R: Sergey Senozhatsky <senozhatsky@chromium.org> 20057 20057 L: linux-mm@kvack.org 20058 20058 S: Maintained 20059 20059 F: Documentation/vm/zsmalloc.rst
+5 -3
arch/powerpc/kernel/nvram_64.c
··· 647 647 { 648 648 struct oops_log_info *oops_hdr = (struct oops_log_info *)oops_buf; 649 649 static unsigned int oops_count = 0; 650 + static struct kmsg_dump_iter iter; 650 651 static bool panicking = false; 651 652 static DEFINE_SPINLOCK(lock); 652 653 unsigned long flags; ··· 682 681 return; 683 682 684 683 if (big_oops_buf) { 685 - kmsg_dump_get_buffer(dumper, false, 684 + kmsg_dump_rewind(&iter); 685 + kmsg_dump_get_buffer(&iter, false, 686 686 big_oops_buf, big_oops_buf_sz, &text_len); 687 687 rc = zip_oops(text_len); 688 688 } 689 689 if (rc != 0) { 690 - kmsg_dump_rewind(dumper); 691 - kmsg_dump_get_buffer(dumper, false, 690 + kmsg_dump_rewind(&iter); 691 + kmsg_dump_get_buffer(&iter, false, 692 692 oops_data, oops_data_sz, &text_len); 693 693 err_type = ERR_TYPE_KERNEL_PANIC; 694 694 oops_hdr->version = cpu_to_be16(OOPS_HDR_VERSION);
+3 -3
arch/powerpc/xmon/xmon.c
··· 3001 3001 static void 3002 3002 dump_log_buf(void) 3003 3003 { 3004 - struct kmsg_dumper dumper = { .active = 1 }; 3004 + struct kmsg_dump_iter iter; 3005 3005 unsigned char buf[128]; 3006 3006 size_t len; 3007 3007 ··· 3013 3013 catch_memory_errors = 1; 3014 3014 sync(); 3015 3015 3016 - kmsg_dump_rewind_nolock(&dumper); 3016 + kmsg_dump_rewind(&iter); 3017 3017 xmon_start_pagination(); 3018 - while (kmsg_dump_get_line_nolock(&dumper, false, buf, sizeof(buf), &len)) { 3018 + while (kmsg_dump_get_line(&iter, false, buf, sizeof(buf), &len)) { 3019 3019 buf[len] = '\0'; 3020 3020 printf("%s", buf); 3021 3021 }
+12 -1
arch/um/kernel/kmsg_dump.c
··· 1 1 // SPDX-License-Identifier: GPL-2.0 2 2 #include <linux/kmsg_dump.h> 3 + #include <linux/spinlock.h> 3 4 #include <linux/console.h> 4 5 #include <linux/string.h> 5 6 #include <shared/init.h> ··· 10 9 static void kmsg_dumper_stdout(struct kmsg_dumper *dumper, 11 10 enum kmsg_dump_reason reason) 12 11 { 12 + static struct kmsg_dump_iter iter; 13 + static DEFINE_SPINLOCK(lock); 13 14 static char line[1024]; 14 15 struct console *con; 16 + unsigned long flags; 15 17 size_t len = 0; 16 18 17 19 /* only dump kmsg when no console is available */ ··· 33 29 if (con) 34 30 return; 35 31 32 + if (!spin_trylock_irqsave(&lock, flags)) 33 + return; 34 + 35 + kmsg_dump_rewind(&iter); 36 + 36 37 printf("kmsg_dump:\n"); 37 - while (kmsg_dump_get_line(dumper, true, line, sizeof(line), &len)) { 38 + while (kmsg_dump_get_line(&iter, true, line, sizeof(line), &len)) { 38 39 line[len] = '\0'; 39 40 printf("%s", line); 40 41 } 42 + 43 + spin_unlock_irqrestore(&lock, flags); 41 44 } 42 45 43 46 static struct kmsg_dumper kmsg_dumper = {
+3 -1
drivers/hv/vmbus_drv.c
··· 1399 1399 static void hv_kmsg_dump(struct kmsg_dumper *dumper, 1400 1400 enum kmsg_dump_reason reason) 1401 1401 { 1402 + struct kmsg_dump_iter iter; 1402 1403 size_t bytes_written; 1403 1404 1404 1405 /* We are only interested in panics. */ ··· 1410 1409 * Write dump contents to the page. No need to synchronize; panic should 1411 1410 * be single-threaded. 1412 1411 */ 1413 - kmsg_dump_get_buffer(dumper, false, hv_panic_page, HV_HYP_PAGE_SIZE, 1412 + kmsg_dump_rewind(&iter); 1413 + kmsg_dump_get_buffer(&iter, false, hv_panic_page, HV_HYP_PAGE_SIZE, 1414 1414 &bytes_written); 1415 1415 if (!bytes_written) 1416 1416 return;
+15 -2
drivers/mtd/mtdoops.c
··· 52 52 int nextcount; 53 53 unsigned long *oops_page_used; 54 54 55 + unsigned long oops_buf_busy; 55 56 void *oops_buf; 56 57 } oops_cxt; 57 58 ··· 181 180 u32 *hdr; 182 181 int ret; 183 182 183 + if (test_and_set_bit(0, &cxt->oops_buf_busy)) 184 + return; 185 + 184 186 /* Add mtdoops header to the buffer */ 185 187 hdr = cxt->oops_buf; 186 188 hdr[0] = cxt->nextcount; ··· 194 190 record_size, &retlen, cxt->oops_buf); 195 191 if (ret == -EOPNOTSUPP) { 196 192 printk(KERN_ERR "mtdoops: Cannot write from panic without panic_write\n"); 197 - return; 193 + goto out; 198 194 } 199 195 } else 200 196 ret = mtd_write(mtd, cxt->nextpage * record_size, ··· 207 203 memset(cxt->oops_buf, 0xff, record_size); 208 204 209 205 mtdoops_inc_counter(cxt); 206 + out: 207 + clear_bit(0, &cxt->oops_buf_busy); 210 208 } 211 209 212 210 static void mtdoops_workfunc_write(struct work_struct *work) ··· 277 271 { 278 272 struct mtdoops_context *cxt = container_of(dumper, 279 273 struct mtdoops_context, dump); 274 + struct kmsg_dump_iter iter; 280 275 281 276 /* Only dump oopses if dump_oops is set */ 282 277 if (reason == KMSG_DUMP_OOPS && !dump_oops) 283 278 return; 284 279 285 - kmsg_dump_get_buffer(dumper, true, cxt->oops_buf + MTDOOPS_HEADER_SIZE, 280 + kmsg_dump_rewind(&iter); 281 + 282 + if (test_and_set_bit(0, &cxt->oops_buf_busy)) 283 + return; 284 + kmsg_dump_get_buffer(&iter, true, cxt->oops_buf + MTDOOPS_HEADER_SIZE, 286 285 record_size - MTDOOPS_HEADER_SIZE, NULL); 286 + clear_bit(0, &cxt->oops_buf_busy); 287 287 288 288 if (reason != KMSG_DUMP_OOPS) { 289 289 /* Panics must be written immediately */ ··· 406 394 return -ENOMEM; 407 395 } 408 396 memset(cxt->oops_buf, 0xff, record_size); 397 + cxt->oops_buf_busy = 0; 409 398 410 399 INIT_WORK(&cxt->work_erase, mtdoops_workfunc_erase); 411 400 INIT_WORK(&cxt->work_write, mtdoops_workfunc_write);
+4 -1
fs/pstore/platform.c
··· 385 385 static void pstore_dump(struct kmsg_dumper *dumper, 386 386 enum kmsg_dump_reason reason) 387 387 { 388 + struct kmsg_dump_iter iter; 388 389 unsigned long total = 0; 389 390 const char *why; 390 391 unsigned int part = 1; ··· 405 404 return; 406 405 } 407 406 } 407 + 408 + kmsg_dump_rewind(&iter); 408 409 409 410 oopscount++; 410 411 while (total < kmsg_bytes) { ··· 438 435 dst_size -= header_size; 439 436 440 437 /* Write dump contents. */ 441 - if (!kmsg_dump_get_buffer(dumper, true, dst + header_size, 438 + if (!kmsg_dump_get_buffer(&iter, true, dst + header_size, 442 439 dst_size, &dump_size)) 443 440 break; 444 441
+17 -30
include/linux/kmsg_dump.h
··· 30 30 }; 31 31 32 32 /** 33 + * struct kmsg_dump_iter - iterator for retrieving kernel messages 34 + * @cur_seq: Points to the oldest message to dump 35 + * @next_seq: Points after the newest message to dump 36 + */ 37 + struct kmsg_dump_iter { 38 + u64 cur_seq; 39 + u64 next_seq; 40 + }; 41 + 42 + /** 33 43 * struct kmsg_dumper - kernel crash message dumper structure 34 44 * @list: Entry in the dumper list (private) 35 45 * @dump: Call into dumping code which will retrieve the data with ··· 51 41 struct list_head list; 52 42 void (*dump)(struct kmsg_dumper *dumper, enum kmsg_dump_reason reason); 53 43 enum kmsg_dump_reason max_reason; 54 - bool active; 55 44 bool registered; 56 - 57 - /* private state of the kmsg iterator */ 58 - u32 cur_idx; 59 - u32 next_idx; 60 - u64 cur_seq; 61 - u64 next_seq; 62 45 }; 63 46 64 47 #ifdef CONFIG_PRINTK 65 48 void kmsg_dump(enum kmsg_dump_reason reason); 66 49 67 - bool kmsg_dump_get_line_nolock(struct kmsg_dumper *dumper, bool syslog, 68 - char *line, size_t size, size_t *len); 69 - 70 - bool kmsg_dump_get_line(struct kmsg_dumper *dumper, bool syslog, 50 + bool kmsg_dump_get_line(struct kmsg_dump_iter *iter, bool syslog, 71 51 char *line, size_t size, size_t *len); 72 52 73 - bool kmsg_dump_get_buffer(struct kmsg_dumper *dumper, bool syslog, 74 - char *buf, size_t size, size_t *len); 53 + bool kmsg_dump_get_buffer(struct kmsg_dump_iter *iter, bool syslog, 54 + char *buf, size_t size, size_t *len_out); 75 55 76 - void kmsg_dump_rewind_nolock(struct kmsg_dumper *dumper); 77 - 78 - void kmsg_dump_rewind(struct kmsg_dumper *dumper); 56 + void kmsg_dump_rewind(struct kmsg_dump_iter *iter); 79 57 80 58 int kmsg_dump_register(struct kmsg_dumper *dumper); 81 59 ··· 75 77 { 76 78 } 77 79 78 - static inline bool kmsg_dump_get_line_nolock(struct kmsg_dumper *dumper, 79 - bool syslog, const char *line, 80 - size_t size, size_t *len) 81 - { 82 - return false; 83 - } 84 - 85 - static inline bool kmsg_dump_get_line(struct kmsg_dumper *dumper, bool syslog, 80 + static inline bool kmsg_dump_get_line(struct kmsg_dump_iter *iter, bool syslog, 86 81 const char *line, size_t size, size_t *len) 87 82 { 88 83 return false; 89 84 } 90 85 91 - static inline bool kmsg_dump_get_buffer(struct kmsg_dumper *dumper, bool syslog, 86 + static inline bool kmsg_dump_get_buffer(struct kmsg_dump_iter *iter, bool syslog, 92 87 char *buf, size_t size, size_t *len) 93 88 { 94 89 return false; 95 90 } 96 91 97 - static inline void kmsg_dump_rewind_nolock(struct kmsg_dumper *dumper) 98 - { 99 - } 100 - 101 - static inline void kmsg_dump_rewind(struct kmsg_dumper *dumper) 92 + static inline void kmsg_dump_rewind(struct kmsg_dump_iter *iter) 102 93 { 103 94 } 104 95
+5 -5
kernel/debug/kdb/kdb_main.c
··· 2077 2077 int adjust = 0; 2078 2078 int n = 0; 2079 2079 int skip = 0; 2080 - struct kmsg_dumper dumper = { .active = 1 }; 2080 + struct kmsg_dump_iter iter; 2081 2081 size_t len; 2082 2082 char buf[201]; 2083 2083 ··· 2102 2102 kdb_set(2, setargs); 2103 2103 } 2104 2104 2105 - kmsg_dump_rewind_nolock(&dumper); 2106 - while (kmsg_dump_get_line_nolock(&dumper, 1, NULL, 0, NULL)) 2105 + kmsg_dump_rewind(&iter); 2106 + while (kmsg_dump_get_line(&iter, 1, NULL, 0, NULL)) 2107 2107 n++; 2108 2108 2109 2109 if (lines < 0) { ··· 2135 2135 if (skip >= n || skip < 0) 2136 2136 return 0; 2137 2137 2138 - kmsg_dump_rewind_nolock(&dumper); 2139 - while (kmsg_dump_get_line_nolock(&dumper, 1, buf, sizeof(buf), &len)) { 2138 + kmsg_dump_rewind(&iter); 2139 + while (kmsg_dump_get_line(&iter, 1, buf, sizeof(buf), &len)) { 2140 2140 if (skip) { 2141 2141 skip--; 2142 2142 continue;
+1 -6
kernel/printk/internal.h
··· 12 12 13 13 #define PRINTK_NMI_CONTEXT_OFFSET 0x010000000 14 14 15 - extern raw_spinlock_t logbuf_lock; 16 - 17 15 __printf(4, 0) 18 16 int vprintk_store(int facility, int level, 19 17 const struct dev_printk_info *dev_info, ··· 19 21 20 22 __printf(1, 0) int vprintk_default(const char *fmt, va_list args); 21 23 __printf(1, 0) int vprintk_deferred(const char *fmt, va_list args); 22 - __printf(1, 0) int vprintk_func(const char *fmt, va_list args); 23 24 void __printk_safe_enter(void); 24 25 void __printk_safe_exit(void); 25 26 ··· 53 56 54 57 #else 55 58 56 - __printf(1, 0) int vprintk_func(const char *fmt, va_list args) { return 0; } 57 - 58 59 /* 59 - * In !PRINTK builds we still export logbuf_lock spin_lock, console_sem 60 + * In !PRINTK builds we still export console_sem 60 61 * semaphore and some of console functions (console_unlock()/etc.), so 61 62 * printk-safe must preserve the existing local IRQ guarantees. 62 63 */
+240 -238
kernel/printk/printk.c
··· 262 262 * definitely not the perfect debug tool (we don't know if _WE_ 263 263 * hold it and are racing, but it helps tracking those weird code 264 264 * paths in the console code where we end up in places I want 265 - * locked without the console sempahore held). 265 + * locked without the console semaphore held). 266 266 */ 267 267 static int console_locked, console_suspended; 268 268 ··· 355 355 LOG_CONT = 8, /* text is a fragment of a continuation line */ 356 356 }; 357 357 358 - /* 359 - * The logbuf_lock protects kmsg buffer, indices, counters. This can be taken 360 - * within the scheduler's rq lock. It must be released before calling 361 - * console_unlock() or anything else that might wake up a process. 362 - */ 363 - DEFINE_RAW_SPINLOCK(logbuf_lock); 364 - 365 - /* 366 - * Helper macros to lock/unlock logbuf_lock and switch between 367 - * printk-safe/unsafe modes. 368 - */ 369 - #define logbuf_lock_irq() \ 370 - do { \ 371 - printk_safe_enter_irq(); \ 372 - raw_spin_lock(&logbuf_lock); \ 373 - } while (0) 374 - 375 - #define logbuf_unlock_irq() \ 376 - do { \ 377 - raw_spin_unlock(&logbuf_lock); \ 378 - printk_safe_exit_irq(); \ 379 - } while (0) 380 - 381 - #define logbuf_lock_irqsave(flags) \ 382 - do { \ 383 - printk_safe_enter_irqsave(flags); \ 384 - raw_spin_lock(&logbuf_lock); \ 385 - } while (0) 386 - 387 - #define logbuf_unlock_irqrestore(flags) \ 388 - do { \ 389 - raw_spin_unlock(&logbuf_lock); \ 390 - printk_safe_exit_irqrestore(flags); \ 391 - } while (0) 358 + /* syslog_lock protects syslog_* variables and write access to clear_seq. */ 359 + static DEFINE_RAW_SPINLOCK(syslog_lock); 392 360 393 361 #ifdef CONFIG_PRINTK 394 362 DECLARE_WAIT_QUEUE_HEAD(log_wait); 363 + /* All 3 protected by @syslog_lock. */ 395 364 /* the next printk record to read by syslog(READ) or /proc/kmsg */ 396 365 static u64 syslog_seq; 397 366 static size_t syslog_partial; 398 367 static bool syslog_time; 399 368 369 + /* All 3 protected by @console_sem. */ 400 370 /* the next printk record to write to the console */ 401 371 static u64 console_seq; 402 372 static u64 exclusive_console_stop_seq; 403 373 static unsigned long console_dropped; 404 374 405 - /* the next printk record to read after the last 'clear' command */ 406 - static u64 clear_seq; 375 + struct latched_seq { 376 + seqcount_latch_t latch; 377 + u64 val[2]; 378 + }; 379 + 380 + /* 381 + * The next printk record to read after the last 'clear' command. There are 382 + * two copies (updated with seqcount_latch) so that reads can locklessly 383 + * access a valid value. Writers are synchronized by @syslog_lock. 384 + */ 385 + static struct latched_seq clear_seq = { 386 + .latch = SEQCNT_LATCH_ZERO(clear_seq.latch), 387 + .val[0] = 0, 388 + .val[1] = 0, 389 + }; 407 390 408 391 #ifdef CONFIG_PRINTK_CALLER 409 392 #define PREFIX_MAX 48 410 393 #else 411 394 #define PREFIX_MAX 32 412 395 #endif 413 - #define LOG_LINE_MAX (1024 - PREFIX_MAX) 396 + 397 + /* the maximum size of a formatted record (i.e. with prefix added per line) */ 398 + #define CONSOLE_LOG_MAX 1024 399 + 400 + /* the maximum size allowed to be reserved for a record */ 401 + #define LOG_LINE_MAX (CONSOLE_LOG_MAX - PREFIX_MAX) 414 402 415 403 #define LOG_LEVEL(v) ((v) & 0x07) 416 404 #define LOG_FACILITY(v) ((v) >> 3 & 0xff) ··· 438 450 bool printk_percpu_data_ready(void) 439 451 { 440 452 return __printk_percpu_data_ready; 453 + } 454 + 455 + /* Must be called under syslog_lock. */ 456 + static void latched_seq_write(struct latched_seq *ls, u64 val) 457 + { 458 + raw_write_seqcount_latch(&ls->latch); 459 + ls->val[0] = val; 460 + raw_write_seqcount_latch(&ls->latch); 461 + ls->val[1] = val; 462 + } 463 + 464 + /* Can be called from any context. */ 465 + static u64 latched_seq_read_nolock(struct latched_seq *ls) 466 + { 467 + unsigned int seq; 468 + unsigned int idx; 469 + u64 val; 470 + 471 + do { 472 + seq = raw_read_seqcount_latch(&ls->latch); 473 + idx = seq & 0x1; 474 + val = ls->val[idx]; 475 + } while (read_seqcount_latch_retry(&ls->latch, seq)); 476 + 477 + return val; 441 478 } 442 479 443 480 /* Return log buffer address */ ··· 632 619 633 620 /* /dev/kmsg - userspace message inject/listen interface */ 634 621 struct devkmsg_user { 635 - u64 seq; 622 + atomic64_t seq; 636 623 struct ratelimit_state rs; 637 624 struct mutex lock; 638 625 char buf[CONSOLE_EXT_LOG_MAX]; ··· 732 719 if (ret) 733 720 return ret; 734 721 735 - logbuf_lock_irq(); 736 - if (!prb_read_valid(prb, user->seq, r)) { 722 + printk_safe_enter_irq(); 723 + if (!prb_read_valid(prb, atomic64_read(&user->seq), r)) { 737 724 if (file->f_flags & O_NONBLOCK) { 738 725 ret = -EAGAIN; 739 - logbuf_unlock_irq(); 726 + printk_safe_exit_irq(); 740 727 goto out; 741 728 } 742 729 743 - logbuf_unlock_irq(); 730 + printk_safe_exit_irq(); 744 731 ret = wait_event_interruptible(log_wait, 745 - prb_read_valid(prb, user->seq, r)); 732 + prb_read_valid(prb, atomic64_read(&user->seq), r)); 746 733 if (ret) 747 734 goto out; 748 - logbuf_lock_irq(); 735 + printk_safe_enter_irq(); 749 736 } 750 737 751 - if (r->info->seq != user->seq) { 738 + if (r->info->seq != atomic64_read(&user->seq)) { 752 739 /* our last seen message is gone, return error and reset */ 753 - user->seq = r->info->seq; 740 + atomic64_set(&user->seq, r->info->seq); 754 741 ret = -EPIPE; 755 - logbuf_unlock_irq(); 742 + printk_safe_exit_irq(); 756 743 goto out; 757 744 } 758 745 ··· 761 748 &r->text_buf[0], r->info->text_len, 762 749 &r->info->dev_info); 763 750 764 - user->seq = r->info->seq + 1; 765 - logbuf_unlock_irq(); 751 + atomic64_set(&user->seq, r->info->seq + 1); 752 + printk_safe_exit_irq(); 766 753 767 754 if (len > count) { 768 755 ret = -EINVAL; ··· 797 784 if (offset) 798 785 return -ESPIPE; 799 786 800 - logbuf_lock_irq(); 787 + printk_safe_enter_irq(); 801 788 switch (whence) { 802 789 case SEEK_SET: 803 790 /* the first record */ 804 - user->seq = prb_first_valid_seq(prb); 791 + atomic64_set(&user->seq, prb_first_valid_seq(prb)); 805 792 break; 806 793 case SEEK_DATA: 807 794 /* ··· 809 796 * like issued by 'dmesg -c'. Reading /dev/kmsg itself 810 797 * changes no global state, and does not clear anything. 811 798 */ 812 - user->seq = clear_seq; 799 + atomic64_set(&user->seq, latched_seq_read_nolock(&clear_seq)); 813 800 break; 814 801 case SEEK_END: 815 802 /* after the last record */ 816 - user->seq = prb_next_seq(prb); 803 + atomic64_set(&user->seq, prb_next_seq(prb)); 817 804 break; 818 805 default: 819 806 ret = -EINVAL; 820 807 } 821 - logbuf_unlock_irq(); 808 + printk_safe_exit_irq(); 822 809 return ret; 823 810 } 824 811 ··· 833 820 834 821 poll_wait(file, &log_wait, wait); 835 822 836 - logbuf_lock_irq(); 837 - if (prb_read_valid_info(prb, user->seq, &info, NULL)) { 823 + printk_safe_enter_irq(); 824 + if (prb_read_valid_info(prb, atomic64_read(&user->seq), &info, NULL)) { 838 825 /* return error when data has vanished underneath us */ 839 - if (info.seq != user->seq) 826 + if (info.seq != atomic64_read(&user->seq)) 840 827 ret = EPOLLIN|EPOLLRDNORM|EPOLLERR|EPOLLPRI; 841 828 else 842 829 ret = EPOLLIN|EPOLLRDNORM; 843 830 } 844 - logbuf_unlock_irq(); 831 + printk_safe_exit_irq(); 845 832 846 833 return ret; 847 834 } ··· 874 861 prb_rec_init_rd(&user->record, &user->info, 875 862 &user->text_buf[0], sizeof(user->text_buf)); 876 863 877 - logbuf_lock_irq(); 878 - user->seq = prb_first_valid_seq(prb); 879 - logbuf_unlock_irq(); 864 + printk_safe_enter_irq(); 865 + atomic64_set(&user->seq, prb_first_valid_seq(prb)); 866 + printk_safe_exit_irq(); 880 867 881 868 file->private_data = user; 882 869 return 0; ··· 968 955 969 956 VMCOREINFO_SIZE(atomic_long_t); 970 957 VMCOREINFO_TYPE_OFFSET(atomic_long_t, counter); 958 + 959 + VMCOREINFO_STRUCT_SIZE(latched_seq); 960 + VMCOREINFO_OFFSET(latched_seq, val); 971 961 } 972 962 #endif 973 963 ··· 1437 1421 return ((prefix_len * line_count) + info->text_len + 1); 1438 1422 } 1439 1423 1424 + /* 1425 + * Beginning with @start_seq, find the first record where it and all following 1426 + * records up to (but not including) @max_seq fit into @size. 1427 + * 1428 + * @max_seq is simply an upper bound and does not need to exist. If the caller 1429 + * does not require an upper bound, -1 can be used for @max_seq. 1430 + */ 1431 + static u64 find_first_fitting_seq(u64 start_seq, u64 max_seq, size_t size, 1432 + bool syslog, bool time) 1433 + { 1434 + struct printk_info info; 1435 + unsigned int line_count; 1436 + size_t len = 0; 1437 + u64 seq; 1438 + 1439 + /* Determine the size of the records up to @max_seq. */ 1440 + prb_for_each_info(start_seq, prb, seq, &info, &line_count) { 1441 + if (info.seq >= max_seq) 1442 + break; 1443 + len += get_record_print_text_size(&info, line_count, syslog, time); 1444 + } 1445 + 1446 + /* 1447 + * Adjust the upper bound for the next loop to avoid subtracting 1448 + * lengths that were never added. 1449 + */ 1450 + if (seq < max_seq) 1451 + max_seq = seq; 1452 + 1453 + /* 1454 + * Move first record forward until length fits into the buffer. Ignore 1455 + * newest messages that were not counted in the above cycle. Messages 1456 + * might appear and get lost in the meantime. This is a best effort 1457 + * that prevents an infinite loop that could occur with a retry. 1458 + */ 1459 + prb_for_each_info(start_seq, prb, seq, &info, &line_count) { 1460 + if (len <= size || info.seq >= max_seq) 1461 + break; 1462 + len -= get_record_print_text_size(&info, line_count, syslog, time); 1463 + } 1464 + 1465 + return seq; 1466 + } 1467 + 1440 1468 static int syslog_print(char __user *buf, int size) 1441 1469 { 1442 1470 struct printk_info info; ··· 1488 1428 char *text; 1489 1429 int len = 0; 1490 1430 1491 - text = kmalloc(LOG_LINE_MAX + PREFIX_MAX, GFP_KERNEL); 1431 + text = kmalloc(CONSOLE_LOG_MAX, GFP_KERNEL); 1492 1432 if (!text) 1493 1433 return -ENOMEM; 1494 1434 1495 - prb_rec_init_rd(&r, &info, text, LOG_LINE_MAX + PREFIX_MAX); 1435 + prb_rec_init_rd(&r, &info, text, CONSOLE_LOG_MAX); 1496 1436 1497 1437 while (size > 0) { 1498 1438 size_t n; 1499 1439 size_t skip; 1500 1440 1501 - logbuf_lock_irq(); 1441 + printk_safe_enter_irq(); 1442 + raw_spin_lock(&syslog_lock); 1502 1443 if (!prb_read_valid(prb, syslog_seq, &r)) { 1503 - logbuf_unlock_irq(); 1444 + raw_spin_unlock(&syslog_lock); 1445 + printk_safe_exit_irq(); 1504 1446 break; 1505 1447 } 1506 1448 if (r.info->seq != syslog_seq) { ··· 1531 1469 syslog_partial += n; 1532 1470 } else 1533 1471 n = 0; 1534 - logbuf_unlock_irq(); 1472 + raw_spin_unlock(&syslog_lock); 1473 + printk_safe_exit_irq(); 1535 1474 1536 1475 if (!n) 1537 1476 break; ··· 1555 1492 static int syslog_print_all(char __user *buf, int size, bool clear) 1556 1493 { 1557 1494 struct printk_info info; 1558 - unsigned int line_count; 1559 1495 struct printk_record r; 1560 1496 char *text; 1561 1497 int len = 0; 1562 1498 u64 seq; 1563 1499 bool time; 1564 1500 1565 - text = kmalloc(LOG_LINE_MAX + PREFIX_MAX, GFP_KERNEL); 1501 + text = kmalloc(CONSOLE_LOG_MAX, GFP_KERNEL); 1566 1502 if (!text) 1567 1503 return -ENOMEM; 1568 1504 1569 1505 time = printk_time; 1570 - logbuf_lock_irq(); 1506 + printk_safe_enter_irq(); 1571 1507 /* 1572 1508 * Find first record that fits, including all following records, 1573 1509 * into the user-provided buffer for this dump. 1574 1510 */ 1575 - prb_for_each_info(clear_seq, prb, seq, &info, &line_count) 1576 - len += get_record_print_text_size(&info, line_count, true, time); 1511 + seq = find_first_fitting_seq(latched_seq_read_nolock(&clear_seq), -1, 1512 + size, true, time); 1577 1513 1578 - /* move first record forward until length fits into the buffer */ 1579 - prb_for_each_info(clear_seq, prb, seq, &info, &line_count) { 1580 - if (len <= size) 1581 - break; 1582 - len -= get_record_print_text_size(&info, line_count, true, time); 1583 - } 1584 - 1585 - prb_rec_init_rd(&r, &info, text, LOG_LINE_MAX + PREFIX_MAX); 1514 + prb_rec_init_rd(&r, &info, text, CONSOLE_LOG_MAX); 1586 1515 1587 1516 len = 0; 1588 1517 prb_for_each_record(seq, prb, seq, &r) { ··· 1587 1532 break; 1588 1533 } 1589 1534 1590 - logbuf_unlock_irq(); 1535 + printk_safe_exit_irq(); 1591 1536 if (copy_to_user(buf + len, text, textlen)) 1592 1537 len = -EFAULT; 1593 1538 else 1594 1539 len += textlen; 1595 - logbuf_lock_irq(); 1540 + printk_safe_enter_irq(); 1596 1541 1597 1542 if (len < 0) 1598 1543 break; 1599 1544 } 1600 1545 1601 - if (clear) 1602 - clear_seq = seq; 1603 - logbuf_unlock_irq(); 1546 + if (clear) { 1547 + raw_spin_lock(&syslog_lock); 1548 + latched_seq_write(&clear_seq, seq); 1549 + raw_spin_unlock(&syslog_lock); 1550 + } 1551 + printk_safe_exit_irq(); 1604 1552 1605 1553 kfree(text); 1606 1554 return len; ··· 1611 1553 1612 1554 static void syslog_clear(void) 1613 1555 { 1614 - logbuf_lock_irq(); 1615 - clear_seq = prb_next_seq(prb); 1616 - logbuf_unlock_irq(); 1556 + printk_safe_enter_irq(); 1557 + raw_spin_lock(&syslog_lock); 1558 + latched_seq_write(&clear_seq, prb_next_seq(prb)); 1559 + raw_spin_unlock(&syslog_lock); 1560 + printk_safe_exit_irq(); 1561 + } 1562 + 1563 + /* Return a consistent copy of @syslog_seq. */ 1564 + static u64 read_syslog_seq_irq(void) 1565 + { 1566 + u64 seq; 1567 + 1568 + raw_spin_lock_irq(&syslog_lock); 1569 + seq = syslog_seq; 1570 + raw_spin_unlock_irq(&syslog_lock); 1571 + 1572 + return seq; 1617 1573 } 1618 1574 1619 1575 int do_syslog(int type, char __user *buf, int len, int source) ··· 1653 1581 return 0; 1654 1582 if (!access_ok(buf, len)) 1655 1583 return -EFAULT; 1584 + 1656 1585 error = wait_event_interruptible(log_wait, 1657 - prb_read_valid(prb, syslog_seq, NULL)); 1586 + prb_read_valid(prb, read_syslog_seq_irq(), NULL)); 1658 1587 if (error) 1659 1588 return error; 1660 1589 error = syslog_print(buf, len); ··· 1703 1630 break; 1704 1631 /* Number of chars in the log buffer */ 1705 1632 case SYSLOG_ACTION_SIZE_UNREAD: 1706 - logbuf_lock_irq(); 1633 + printk_safe_enter_irq(); 1634 + raw_spin_lock(&syslog_lock); 1707 1635 if (!prb_read_valid_info(prb, syslog_seq, &info, NULL)) { 1708 1636 /* No unread messages. */ 1709 - logbuf_unlock_irq(); 1637 + raw_spin_unlock(&syslog_lock); 1638 + printk_safe_exit_irq(); 1710 1639 return 0; 1711 1640 } 1712 1641 if (info.seq != syslog_seq) { ··· 1736 1661 } 1737 1662 error -= syslog_partial; 1738 1663 } 1739 - logbuf_unlock_irq(); 1664 + raw_spin_unlock(&syslog_lock); 1665 + printk_safe_exit_irq(); 1740 1666 break; 1741 1667 /* Size of the log buffer */ 1742 1668 case SYSLOG_ACTION_SIZE_BUFFER: ··· 2180 2104 } 2181 2105 EXPORT_SYMBOL(vprintk_emit); 2182 2106 2183 - asmlinkage int vprintk(const char *fmt, va_list args) 2184 - { 2185 - return vprintk_func(fmt, args); 2186 - } 2187 - EXPORT_SYMBOL(vprintk); 2188 - 2189 2107 int vprintk_default(const char *fmt, va_list args) 2190 2108 { 2191 2109 return vprintk_emit(0, LOGLEVEL_DEFAULT, NULL, fmt, args); ··· 2213 2143 int r; 2214 2144 2215 2145 va_start(args, fmt); 2216 - r = vprintk_func(fmt, args); 2146 + r = vprintk(fmt, args); 2217 2147 va_end(args); 2218 2148 2219 2149 return r; ··· 2222 2152 2223 2153 #else /* CONFIG_PRINTK */ 2224 2154 2225 - #define LOG_LINE_MAX 0 2226 - #define PREFIX_MAX 0 2155 + #define CONSOLE_LOG_MAX 0 2227 2156 #define printk_time false 2228 2157 2229 2158 #define prb_read_valid(rb, seq, r) false ··· 2331 2262 /* 2332 2263 * console="" or console=null have been suggested as a way to 2333 2264 * disable console output. Use ttynull that has been created 2334 - * for exacly this purpose. 2265 + * for exactly this purpose. 2335 2266 */ 2336 2267 if (str[0] == 0 || strcmp(str, "null") == 0) { 2337 2268 __add_preferred_console("ttynull", 0, NULL, NULL, true); ··· 2540 2471 void console_unlock(void) 2541 2472 { 2542 2473 static char ext_text[CONSOLE_EXT_LOG_MAX]; 2543 - static char text[LOG_LINE_MAX + PREFIX_MAX]; 2474 + static char text[CONSOLE_LOG_MAX]; 2544 2475 unsigned long flags; 2545 2476 bool do_cond_resched, retry; 2546 2477 struct printk_info info; ··· 2587 2518 size_t len; 2588 2519 2589 2520 printk_safe_enter_irqsave(flags); 2590 - raw_spin_lock(&logbuf_lock); 2591 2521 skip: 2592 2522 if (!prb_read_valid(prb, console_seq, &r)) 2593 2523 break; ··· 2630 2562 console_msg_format & MSG_FORMAT_SYSLOG, 2631 2563 printk_time); 2632 2564 console_seq++; 2633 - raw_spin_unlock(&logbuf_lock); 2634 2565 2635 2566 /* 2636 2567 * While actively printing out messages, if another printk() ··· 2656 2589 2657 2590 console_locked = 0; 2658 2591 2659 - raw_spin_unlock(&logbuf_lock); 2660 - 2661 2592 up_console_sem(); 2662 2593 2663 2594 /* ··· 2664 2599 * there's a new owner and the console_unlock() from them will do the 2665 2600 * flush, no worries. 2666 2601 */ 2667 - raw_spin_lock(&logbuf_lock); 2668 2602 retry = prb_read_valid(prb, console_seq, NULL); 2669 - raw_spin_unlock(&logbuf_lock); 2670 2603 printk_safe_exit_irqrestore(flags); 2671 2604 2672 2605 if (retry && console_trylock()) ··· 2731 2668 if (mode == CONSOLE_REPLAY_ALL) { 2732 2669 unsigned long flags; 2733 2670 2734 - logbuf_lock_irqsave(flags); 2671 + printk_safe_enter_irqsave(flags); 2735 2672 console_seq = prb_first_valid_seq(prb); 2736 - logbuf_unlock_irqrestore(flags); 2673 + printk_safe_exit_irqrestore(flags); 2737 2674 } 2738 2675 console_unlock(); 2739 2676 } ··· 2961 2898 /* 2962 2899 * console_unlock(); will print out the buffered messages 2963 2900 * for us. 2964 - */ 2965 - logbuf_lock_irqsave(flags); 2966 - /* 2901 + * 2967 2902 * We're about to replay the log buffer. Only do this to the 2968 2903 * just-registered console to avoid excessive message spam to 2969 2904 * the already-registered consoles. ··· 2972 2911 */ 2973 2912 exclusive_console = newcon; 2974 2913 exclusive_console_stop_seq = console_seq; 2914 + 2915 + /* Get a consistent copy of @syslog_seq. */ 2916 + raw_spin_lock_irqsave(&syslog_lock, flags); 2975 2917 console_seq = syslog_seq; 2976 - logbuf_unlock_irqrestore(flags); 2918 + raw_spin_unlock_irqrestore(&syslog_lock, flags); 2977 2919 } 2978 2920 console_unlock(); 2979 2921 console_sysfs_notify(); ··· 3106 3042 * 3107 3043 * To mitigate this problem somewhat, only unregister consoles whose memory 3108 3044 * intersects with the init section. Note that all other boot consoles will 3109 - * get unregistred when the real preferred console is registered. 3045 + * get unregistered when the real preferred console is registered. 3110 3046 */ 3111 3047 static int __init printk_late_init(void) 3112 3048 { ··· 3340 3276 void kmsg_dump(enum kmsg_dump_reason reason) 3341 3277 { 3342 3278 struct kmsg_dumper *dumper; 3343 - unsigned long flags; 3344 3279 3345 3280 rcu_read_lock(); 3346 3281 list_for_each_entry_rcu(dumper, &dump_list, list) { ··· 3356 3293 if (reason > max_reason) 3357 3294 continue; 3358 3295 3359 - /* initialize iterator with data about the stored records */ 3360 - dumper->active = true; 3361 - 3362 - logbuf_lock_irqsave(flags); 3363 - dumper->cur_seq = clear_seq; 3364 - dumper->next_seq = prb_next_seq(prb); 3365 - logbuf_unlock_irqrestore(flags); 3366 - 3367 3296 /* invoke dumper which will iterate over records */ 3368 3297 dumper->dump(dumper, reason); 3369 - 3370 - /* reset iterator */ 3371 - dumper->active = false; 3372 3298 } 3373 3299 rcu_read_unlock(); 3374 3300 } 3375 3301 3376 3302 /** 3377 - * kmsg_dump_get_line_nolock - retrieve one kmsg log line (unlocked version) 3378 - * @dumper: registered kmsg dumper 3303 + * kmsg_dump_get_line - retrieve one kmsg log line 3304 + * @iter: kmsg dump iterator 3379 3305 * @syslog: include the "<4>" prefixes 3380 3306 * @line: buffer to copy the line to 3381 3307 * @size: maximum size of the buffer ··· 3378 3326 * 3379 3327 * A return value of FALSE indicates that there are no more records to 3380 3328 * read. 3381 - * 3382 - * The function is similar to kmsg_dump_get_line(), but grabs no locks. 3383 3329 */ 3384 - bool kmsg_dump_get_line_nolock(struct kmsg_dumper *dumper, bool syslog, 3385 - char *line, size_t size, size_t *len) 3330 + bool kmsg_dump_get_line(struct kmsg_dump_iter *iter, bool syslog, 3331 + char *line, size_t size, size_t *len) 3386 3332 { 3333 + u64 min_seq = latched_seq_read_nolock(&clear_seq); 3387 3334 struct printk_info info; 3388 3335 unsigned int line_count; 3389 3336 struct printk_record r; 3337 + unsigned long flags; 3390 3338 size_t l = 0; 3391 3339 bool ret = false; 3392 3340 3393 - prb_rec_init_rd(&r, &info, line, size); 3341 + if (iter->cur_seq < min_seq) 3342 + iter->cur_seq = min_seq; 3394 3343 3395 - if (!dumper->active) 3396 - goto out; 3344 + printk_safe_enter_irqsave(flags); 3345 + prb_rec_init_rd(&r, &info, line, size); 3397 3346 3398 3347 /* Read text or count text lines? */ 3399 3348 if (line) { 3400 - if (!prb_read_valid(prb, dumper->cur_seq, &r)) 3349 + if (!prb_read_valid(prb, iter->cur_seq, &r)) 3401 3350 goto out; 3402 3351 l = record_print_text(&r, syslog, printk_time); 3403 3352 } else { 3404 - if (!prb_read_valid_info(prb, dumper->cur_seq, 3353 + if (!prb_read_valid_info(prb, iter->cur_seq, 3405 3354 &info, &line_count)) { 3406 3355 goto out; 3407 3356 } ··· 3411 3358 3412 3359 } 3413 3360 3414 - dumper->cur_seq = r.info->seq + 1; 3361 + iter->cur_seq = r.info->seq + 1; 3415 3362 ret = true; 3416 3363 out: 3364 + printk_safe_exit_irqrestore(flags); 3417 3365 if (len) 3418 3366 *len = l; 3419 - return ret; 3420 - } 3421 - 3422 - /** 3423 - * kmsg_dump_get_line - retrieve one kmsg log line 3424 - * @dumper: registered kmsg dumper 3425 - * @syslog: include the "<4>" prefixes 3426 - * @line: buffer to copy the line to 3427 - * @size: maximum size of the buffer 3428 - * @len: length of line placed into buffer 3429 - * 3430 - * Start at the beginning of the kmsg buffer, with the oldest kmsg 3431 - * record, and copy one record into the provided buffer. 3432 - * 3433 - * Consecutive calls will return the next available record moving 3434 - * towards the end of the buffer with the youngest messages. 3435 - * 3436 - * A return value of FALSE indicates that there are no more records to 3437 - * read. 3438 - */ 3439 - bool kmsg_dump_get_line(struct kmsg_dumper *dumper, bool syslog, 3440 - char *line, size_t size, size_t *len) 3441 - { 3442 - unsigned long flags; 3443 - bool ret; 3444 - 3445 - logbuf_lock_irqsave(flags); 3446 - ret = kmsg_dump_get_line_nolock(dumper, syslog, line, size, len); 3447 - logbuf_unlock_irqrestore(flags); 3448 - 3449 3367 return ret; 3450 3368 } 3451 3369 EXPORT_SYMBOL_GPL(kmsg_dump_get_line); 3452 3370 3453 3371 /** 3454 3372 * kmsg_dump_get_buffer - copy kmsg log lines 3455 - * @dumper: registered kmsg dumper 3373 + * @iter: kmsg dump iterator 3456 3374 * @syslog: include the "<4>" prefixes 3457 3375 * @buf: buffer to copy the line to 3458 3376 * @size: maximum size of the buffer 3459 - * @len: length of line placed into buffer 3377 + * @len_out: length of line placed into buffer 3460 3378 * 3461 3379 * Start at the end of the kmsg buffer and fill the provided buffer 3462 3380 * with as many of the *youngest* kmsg records that fit into it. ··· 3440 3416 * A return value of FALSE indicates that there are no more records to 3441 3417 * read. 3442 3418 */ 3443 - bool kmsg_dump_get_buffer(struct kmsg_dumper *dumper, bool syslog, 3444 - char *buf, size_t size, size_t *len) 3419 + bool kmsg_dump_get_buffer(struct kmsg_dump_iter *iter, bool syslog, 3420 + char *buf, size_t size, size_t *len_out) 3445 3421 { 3422 + u64 min_seq = latched_seq_read_nolock(&clear_seq); 3446 3423 struct printk_info info; 3447 - unsigned int line_count; 3448 3424 struct printk_record r; 3449 3425 unsigned long flags; 3450 3426 u64 seq; 3451 3427 u64 next_seq; 3452 - size_t l = 0; 3428 + size_t len = 0; 3453 3429 bool ret = false; 3454 3430 bool time = printk_time; 3455 3431 3456 - prb_rec_init_rd(&r, &info, buf, size); 3457 - 3458 - if (!dumper->active || !buf || !size) 3432 + if (!buf || !size) 3459 3433 goto out; 3460 3434 3461 - logbuf_lock_irqsave(flags); 3462 - if (prb_read_valid_info(prb, dumper->cur_seq, &info, NULL)) { 3463 - if (info.seq != dumper->cur_seq) { 3435 + if (iter->cur_seq < min_seq) 3436 + iter->cur_seq = min_seq; 3437 + 3438 + printk_safe_enter_irqsave(flags); 3439 + if (prb_read_valid_info(prb, iter->cur_seq, &info, NULL)) { 3440 + if (info.seq != iter->cur_seq) { 3464 3441 /* messages are gone, move to first available one */ 3465 - dumper->cur_seq = info.seq; 3442 + iter->cur_seq = info.seq; 3466 3443 } 3467 3444 } 3468 3445 3469 3446 /* last entry */ 3470 - if (dumper->cur_seq >= dumper->next_seq) { 3471 - logbuf_unlock_irqrestore(flags); 3447 + if (iter->cur_seq >= iter->next_seq) { 3448 + printk_safe_exit_irqrestore(flags); 3472 3449 goto out; 3473 3450 } 3474 3451 3475 - /* calculate length of entire buffer */ 3476 - seq = dumper->cur_seq; 3477 - while (prb_read_valid_info(prb, seq, &info, &line_count)) { 3478 - if (r.info->seq >= dumper->next_seq) 3479 - break; 3480 - l += get_record_print_text_size(&info, line_count, syslog, time); 3481 - seq = r.info->seq + 1; 3482 - } 3452 + /* 3453 + * Find first record that fits, including all following records, 3454 + * into the user-provided buffer for this dump. Pass in size-1 3455 + * because this function (by way of record_print_text()) will 3456 + * not write more than size-1 bytes of text into @buf. 3457 + */ 3458 + seq = find_first_fitting_seq(iter->cur_seq, iter->next_seq, 3459 + size - 1, syslog, time); 3483 3460 3484 - /* move first record forward until length fits into the buffer */ 3485 - seq = dumper->cur_seq; 3486 - while (l >= size && prb_read_valid_info(prb, seq, 3487 - &info, &line_count)) { 3488 - if (r.info->seq >= dumper->next_seq) 3489 - break; 3490 - l -= get_record_print_text_size(&info, line_count, syslog, time); 3491 - seq = r.info->seq + 1; 3492 - } 3493 - 3494 - /* last message in next interation */ 3461 + /* 3462 + * Next kmsg_dump_get_buffer() invocation will dump block of 3463 + * older records stored right before this one. 3464 + */ 3495 3465 next_seq = seq; 3496 3466 3497 - /* actually read text into the buffer now */ 3498 - l = 0; 3499 - while (prb_read_valid(prb, seq, &r)) { 3500 - if (r.info->seq >= dumper->next_seq) 3467 + prb_rec_init_rd(&r, &info, buf, size); 3468 + 3469 + len = 0; 3470 + prb_for_each_record(seq, prb, seq, &r) { 3471 + if (r.info->seq >= iter->next_seq) 3501 3472 break; 3502 3473 3503 - l += record_print_text(&r, syslog, time); 3474 + len += record_print_text(&r, syslog, time); 3504 3475 3505 - /* adjust record to store to remaining buffer space */ 3506 - prb_rec_init_rd(&r, &info, buf + l, size - l); 3507 - 3508 - seq = r.info->seq + 1; 3476 + /* Adjust record to store to remaining buffer space. */ 3477 + prb_rec_init_rd(&r, &info, buf + len, size - len); 3509 3478 } 3510 3479 3511 - dumper->next_seq = next_seq; 3480 + iter->next_seq = next_seq; 3512 3481 ret = true; 3513 - logbuf_unlock_irqrestore(flags); 3482 + printk_safe_exit_irqrestore(flags); 3514 3483 out: 3515 - if (len) 3516 - *len = l; 3484 + if (len_out) 3485 + *len_out = len; 3517 3486 return ret; 3518 3487 } 3519 3488 EXPORT_SYMBOL_GPL(kmsg_dump_get_buffer); 3520 3489 3521 3490 /** 3522 - * kmsg_dump_rewind_nolock - reset the iterator (unlocked version) 3523 - * @dumper: registered kmsg dumper 3524 - * 3525 - * Reset the dumper's iterator so that kmsg_dump_get_line() and 3526 - * kmsg_dump_get_buffer() can be called again and used multiple 3527 - * times within the same dumper.dump() callback. 3528 - * 3529 - * The function is similar to kmsg_dump_rewind(), but grabs no locks. 3530 - */ 3531 - void kmsg_dump_rewind_nolock(struct kmsg_dumper *dumper) 3532 - { 3533 - dumper->cur_seq = clear_seq; 3534 - dumper->next_seq = prb_next_seq(prb); 3535 - } 3536 - 3537 - /** 3538 3491 * kmsg_dump_rewind - reset the iterator 3539 - * @dumper: registered kmsg dumper 3492 + * @iter: kmsg dump iterator 3540 3493 * 3541 3494 * Reset the dumper's iterator so that kmsg_dump_get_line() and 3542 3495 * kmsg_dump_get_buffer() can be called again and used multiple 3543 3496 * times within the same dumper.dump() callback. 3544 3497 */ 3545 - void kmsg_dump_rewind(struct kmsg_dumper *dumper) 3498 + void kmsg_dump_rewind(struct kmsg_dump_iter *iter) 3546 3499 { 3547 3500 unsigned long flags; 3548 3501 3549 - logbuf_lock_irqsave(flags); 3550 - kmsg_dump_rewind_nolock(dumper); 3551 - logbuf_unlock_irqrestore(flags); 3502 + printk_safe_enter_irqsave(flags); 3503 + iter->cur_seq = latched_seq_read_nolock(&clear_seq); 3504 + iter->next_seq = prb_next_seq(prb); 3505 + printk_safe_exit_irqrestore(flags); 3552 3506 } 3553 3507 EXPORT_SYMBOL_GPL(kmsg_dump_rewind); 3554 3508
+11 -19
kernel/printk/printk_safe.c
··· 16 16 #include "internal.h" 17 17 18 18 /* 19 - * printk() could not take logbuf_lock in NMI context. Instead, 19 + * In NMI and safe mode, printk() avoids taking locks. Instead, 20 20 * it uses an alternative implementation that temporary stores 21 21 * the strings into a per-CPU buffer. The content of the buffer 22 22 * is later flushed into the main ring buffer via IRQ work. ··· 267 267 void printk_safe_flush_on_panic(void) 268 268 { 269 269 /* 270 - * Make sure that we could access the main ring buffer. 270 + * Make sure that we could access the safe buffers. 271 271 * Do not risk a double release when more CPUs are up. 272 272 */ 273 - if (raw_spin_is_locked(&logbuf_lock)) { 274 - if (num_online_cpus() > 1) 275 - return; 276 - 277 - debug_locks_off(); 278 - raw_spin_lock_init(&logbuf_lock); 279 - } 280 - 281 273 if (raw_spin_is_locked(&safe_read_lock)) { 282 274 if (num_online_cpus() > 1) 283 275 return; ··· 311 319 * reordering. 312 320 * 313 321 * It has effect only when called in NMI context. Then printk() 314 - * will try to store the messages into the main logbuf directly 315 - * and use the per-CPU buffers only as a fallback when the lock 316 - * is not available. 322 + * will store the messages into the main logbuf directly. 317 323 */ 318 324 void printk_nmi_direct_enter(void) 319 325 { ··· 357 367 this_cpu_dec(printk_context); 358 368 } 359 369 360 - __printf(1, 0) int vprintk_func(const char *fmt, va_list args) 370 + asmlinkage int vprintk(const char *fmt, va_list args) 361 371 { 362 372 #ifdef CONFIG_KGDB_KDB 363 373 /* Allow to pass printk() to kdb but avoid a recursion. */ ··· 366 376 #endif 367 377 368 378 /* 369 - * Try to use the main logbuf even in NMI. But avoid calling console 379 + * Use the main logbuf even in NMI. But avoid calling console 370 380 * drivers that might have their own locks. 371 381 */ 372 - if ((this_cpu_read(printk_context) & PRINTK_NMI_DIRECT_CONTEXT_MASK) && 373 - raw_spin_trylock(&logbuf_lock)) { 382 + if ((this_cpu_read(printk_context) & PRINTK_NMI_DIRECT_CONTEXT_MASK)) { 383 + unsigned long flags; 374 384 int len; 375 385 386 + printk_safe_enter_irqsave(flags); 376 387 len = vprintk_store(0, LOGLEVEL_DEFAULT, NULL, fmt, args); 377 - raw_spin_unlock(&logbuf_lock); 388 + printk_safe_exit_irqrestore(flags); 378 389 defer_console_output(); 379 390 return len; 380 391 } 381 392 382 - /* Use extra buffer in NMI when logbuf_lock is taken or in safe mode. */ 393 + /* Use extra buffer in NMI. */ 383 394 if (this_cpu_read(printk_context) & PRINTK_NMI_CONTEXT_MASK) 384 395 return vprintk_nmi(fmt, args); 385 396 ··· 411 420 /* Flush pending messages that did not have scheduled IRQ works. */ 412 421 printk_safe_flush(); 413 422 } 423 + EXPORT_SYMBOL(vprintk);
+80 -10
lib/test_printf.c
··· 577 577 { 578 578 } 579 579 580 + struct page_flags_test { 581 + int width; 582 + int shift; 583 + int mask; 584 + unsigned long value; 585 + const char *fmt; 586 + const char *name; 587 + }; 588 + 589 + static struct page_flags_test pft[] = { 590 + {SECTIONS_WIDTH, SECTIONS_PGSHIFT, SECTIONS_MASK, 591 + 0, "%d", "section"}, 592 + {NODES_WIDTH, NODES_PGSHIFT, NODES_MASK, 593 + 0, "%d", "node"}, 594 + {ZONES_WIDTH, ZONES_PGSHIFT, ZONES_MASK, 595 + 0, "%d", "zone"}, 596 + {LAST_CPUPID_WIDTH, LAST_CPUPID_PGSHIFT, LAST_CPUPID_MASK, 597 + 0, "%#x", "lastcpupid"}, 598 + {KASAN_TAG_WIDTH, KASAN_TAG_PGSHIFT, KASAN_TAG_MASK, 599 + 0, "%#x", "kasantag"}, 600 + }; 601 + 602 + static void __init 603 + page_flags_test(int section, int node, int zone, int last_cpupid, 604 + int kasan_tag, int flags, const char *name, char *cmp_buf) 605 + { 606 + unsigned long values[] = {section, node, zone, last_cpupid, kasan_tag}; 607 + unsigned long page_flags = 0; 608 + unsigned long size = 0; 609 + bool append = false; 610 + int i; 611 + 612 + flags &= BIT(NR_PAGEFLAGS) - 1; 613 + if (flags) { 614 + page_flags |= flags; 615 + snprintf(cmp_buf + size, BUF_SIZE - size, "%s", name); 616 + size = strlen(cmp_buf); 617 + #if SECTIONS_WIDTH || NODES_WIDTH || ZONES_WIDTH || \ 618 + LAST_CPUPID_WIDTH || KASAN_TAG_WIDTH 619 + /* Other information also included in page flags */ 620 + snprintf(cmp_buf + size, BUF_SIZE - size, "|"); 621 + size = strlen(cmp_buf); 622 + #endif 623 + } 624 + 625 + /* Set the test value */ 626 + for (i = 0; i < ARRAY_SIZE(pft); i++) 627 + pft[i].value = values[i]; 628 + 629 + for (i = 0; i < ARRAY_SIZE(pft); i++) { 630 + if (!pft[i].width) 631 + continue; 632 + 633 + if (append) { 634 + snprintf(cmp_buf + size, BUF_SIZE - size, "|"); 635 + size = strlen(cmp_buf); 636 + } 637 + 638 + page_flags |= (pft[i].value & pft[i].mask) << pft[i].shift; 639 + snprintf(cmp_buf + size, BUF_SIZE - size, "%s=", pft[i].name); 640 + size = strlen(cmp_buf); 641 + snprintf(cmp_buf + size, BUF_SIZE - size, pft[i].fmt, 642 + pft[i].value & pft[i].mask); 643 + size = strlen(cmp_buf); 644 + append = true; 645 + } 646 + 647 + test(cmp_buf, "%pGp", &page_flags); 648 + } 649 + 580 650 static void __init 581 651 flags(void) 582 652 { 583 653 unsigned long flags; 584 - gfp_t gfp; 585 654 char *cmp_buffer; 655 + gfp_t gfp; 656 + 657 + cmp_buffer = kmalloc(BUF_SIZE, GFP_KERNEL); 658 + if (!cmp_buffer) 659 + return; 586 660 587 661 flags = 0; 588 - test("", "%pGp", &flags); 662 + page_flags_test(0, 0, 0, 0, 0, flags, "", cmp_buffer); 589 663 590 - /* Page flags should filter the zone id */ 591 664 flags = 1UL << NR_PAGEFLAGS; 592 - test("", "%pGp", &flags); 665 + page_flags_test(0, 0, 0, 0, 0, flags, "", cmp_buffer); 593 666 594 667 flags |= 1UL << PG_uptodate | 1UL << PG_dirty | 1UL << PG_lru 595 668 | 1UL << PG_active | 1UL << PG_swapbacked; 596 - test("uptodate|dirty|lru|active|swapbacked", "%pGp", &flags); 597 - 669 + page_flags_test(1, 1, 1, 0x1fffff, 1, flags, 670 + "uptodate|dirty|lru|active|swapbacked", 671 + cmp_buffer); 598 672 599 673 flags = VM_READ | VM_EXEC | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC 600 674 | VM_DENYWRITE; ··· 682 608 683 609 gfp = __GFP_ATOMIC; 684 610 test("__GFP_ATOMIC", "%pGg", &gfp); 685 - 686 - cmp_buffer = kmalloc(BUF_SIZE, GFP_KERNEL); 687 - if (!cmp_buffer) 688 - return; 689 611 690 612 /* Any flags not translated by the table should remain numeric */ 691 613 gfp = ~__GFP_BITS_MASK;
+69 -9
lib/vsprintf.c
··· 1916 1916 return buf; 1917 1917 } 1918 1918 1919 + struct page_flags_fields { 1920 + int width; 1921 + int shift; 1922 + int mask; 1923 + const struct printf_spec *spec; 1924 + const char *name; 1925 + }; 1926 + 1927 + static const struct page_flags_fields pff[] = { 1928 + {SECTIONS_WIDTH, SECTIONS_PGSHIFT, SECTIONS_MASK, 1929 + &default_dec_spec, "section"}, 1930 + {NODES_WIDTH, NODES_PGSHIFT, NODES_MASK, 1931 + &default_dec_spec, "node"}, 1932 + {ZONES_WIDTH, ZONES_PGSHIFT, ZONES_MASK, 1933 + &default_dec_spec, "zone"}, 1934 + {LAST_CPUPID_WIDTH, LAST_CPUPID_PGSHIFT, LAST_CPUPID_MASK, 1935 + &default_flag_spec, "lastcpupid"}, 1936 + {KASAN_TAG_WIDTH, KASAN_TAG_PGSHIFT, KASAN_TAG_MASK, 1937 + &default_flag_spec, "kasantag"}, 1938 + }; 1939 + 1940 + static 1941 + char *format_page_flags(char *buf, char *end, unsigned long flags) 1942 + { 1943 + unsigned long main_flags = flags & (BIT(NR_PAGEFLAGS) - 1); 1944 + bool append = false; 1945 + int i; 1946 + 1947 + /* Page flags from the main area. */ 1948 + if (main_flags) { 1949 + buf = format_flags(buf, end, main_flags, pageflag_names); 1950 + append = true; 1951 + } 1952 + 1953 + /* Page flags from the fields area */ 1954 + for (i = 0; i < ARRAY_SIZE(pff); i++) { 1955 + /* Skip undefined fields. */ 1956 + if (!pff[i].width) 1957 + continue; 1958 + 1959 + /* Format: Flag Name + '=' (equals sign) + Number + '|' (separator) */ 1960 + if (append) { 1961 + if (buf < end) 1962 + *buf = '|'; 1963 + buf++; 1964 + } 1965 + 1966 + buf = string(buf, end, pff[i].name, default_str_spec); 1967 + if (buf < end) 1968 + *buf = '='; 1969 + buf++; 1970 + buf = number(buf, end, (flags >> pff[i].shift) & pff[i].mask, 1971 + *pff[i].spec); 1972 + 1973 + append = true; 1974 + } 1975 + 1976 + return buf; 1977 + } 1978 + 1919 1979 static noinline_for_stack 1920 1980 char *flags_string(char *buf, char *end, void *flags_ptr, 1921 1981 struct printf_spec spec, const char *fmt) ··· 1988 1928 1989 1929 switch (fmt[1]) { 1990 1930 case 'p': 1991 - flags = *(unsigned long *)flags_ptr; 1992 - /* Remove zone id */ 1993 - flags &= (1UL << NR_PAGEFLAGS) - 1; 1994 - names = pageflag_names; 1995 - break; 1931 + return format_page_flags(buf, end, *(unsigned long *)flags_ptr); 1996 1932 case 'v': 1997 1933 flags = *(unsigned long *)flags_ptr; 1998 1934 names = vmaflag_names; ··· 2152 2096 2153 2097 static int __init no_hash_pointers_enable(char *str) 2154 2098 { 2099 + if (no_hash_pointers) 2100 + return 0; 2101 + 2155 2102 no_hash_pointers = true; 2156 2103 2157 2104 pr_warn("**********************************************************\n"); ··· 2245 2186 * Implements a "recursive vsnprintf". 2246 2187 * Do not use this feature without some mechanism to verify the 2247 2188 * correctness of the format string and va_list arguments. 2248 - * - 'K' For a kernel pointer that should be hidden from unprivileged users 2189 + * - 'K' For a kernel pointer that should be hidden from unprivileged users. 2190 + * Use only for procfs, sysfs and similar files, not printk(); please 2191 + * read the documentation (path below) first. 2249 2192 * - 'NF' For a netdev_features_t 2250 2193 * - 'h[CDN]' For a variable-length buffer, it prints it as a hex string with 2251 2194 * a certain separator (' ' by default): ··· 2286 2225 * Without an option prints the full name of the node 2287 2226 * f full name 2288 2227 * P node name, including a possible unit address 2289 - * - 'x' For printing the address. Equivalent to "%lx". 2228 + * - 'x' For printing the address unmodified. Equivalent to "%lx". 2229 + * Please read the documentation (path below) before using! 2290 2230 * - '[ku]s' For a BPF/tracing related format specifier, e.g. used out of 2291 2231 * bpf_trace_printk() where [ku] prefix specifies either kernel (k) 2292 2232 * or user (u) memory to probe, and: ··· 3197 3135 switch (*fmt) { 3198 3136 case 'S': 3199 3137 case 's': 3200 - case 'F': 3201 - case 'f': 3202 3138 case 'x': 3203 3139 case 'K': 3204 3140 case 'e':
+7 -6
mm/slub.c
··· 624 624 if (!t->addr) 625 625 return; 626 626 627 - pr_err("INFO: %s in %pS age=%lu cpu=%u pid=%d\n", 627 + pr_err("%s in %pS age=%lu cpu=%u pid=%d\n", 628 628 s, (void *)t->addr, pr_time - t->when, t->cpu, t->pid); 629 629 #ifdef CONFIG_STACKTRACE 630 630 { ··· 650 650 651 651 static void print_page_info(struct page *page) 652 652 { 653 - pr_err("INFO: Slab 0x%p objects=%u used=%u fp=0x%p flags=0x%04lx\n", 654 - page, page->objects, page->inuse, page->freelist, page->flags); 653 + pr_err("Slab 0x%p objects=%u used=%u fp=0x%p flags=%#lx(%pGp)\n", 654 + page, page->objects, page->inuse, page->freelist, 655 + page->flags, &page->flags); 655 656 656 657 } 657 658 ··· 707 706 708 707 print_page_info(page); 709 708 710 - pr_err("INFO: Object 0x%p @offset=%tu fp=0x%p\n\n", 709 + pr_err("Object 0x%p @offset=%tu fp=0x%p\n\n", 711 710 p, p - addr, get_freepointer(s, p)); 712 711 713 712 if (s->flags & SLAB_RED_ZONE) ··· 800 799 end--; 801 800 802 801 slab_bug(s, "%s overwritten", what); 803 - pr_err("INFO: 0x%p-0x%p @offset=%tu. First byte 0x%x instead of 0x%x\n", 802 + pr_err("0x%p-0x%p @offset=%tu. First byte 0x%x instead of 0x%x\n", 804 803 fault, end - 1, fault - addr, 805 804 fault[0], value); 806 805 print_trailer(s, page, object); ··· 3899 3898 for_each_object(p, s, addr, page->objects) { 3900 3899 3901 3900 if (!test_bit(__obj_to_index(s, addr, p), map)) { 3902 - pr_err("INFO: Object 0x%p @offset=%tu\n", p, p - addr); 3901 + pr_err("Object 0x%p @offset=%tu\n", p, p - addr); 3903 3902 print_tracking(s, p); 3904 3903 } 3905 3904 }