Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

kexec: load and relocate purgatory at kernel load time

Load purgatory code in RAM and relocate it based on the location.
Relocation code has been inspired by module relocation code and purgatory
relocation code in kexec-tools.

Also compute the checksums of loaded kexec segments and store them in
purgatory.

Arch independent code provides this functionality so that arch dependent
bootloaders can make use of it.

Helper functions are provided to get/set symbol values in purgatory which
are used by bootloaders later to set things like stack and entry point of
second kernel etc.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: Michael Kerrisk <mtk.manpages@gmail.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Eric Biederman <ebiederm@xmission.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Matthew Garrett <mjg59@srcf.ucam.org>
Cc: Greg Kroah-Hartman <greg@kroah.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: WANG Chao <chaowang@redhat.com>
Cc: Baoquan He <bhe@redhat.com>
Cc: Andy Lutomirski <luto@amacapital.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

authored by

Vivek Goyal and committed by
Linus Torvalds
12db5562 8fc5b4d4

+736 -1
+2
arch/arm/Kconfig
··· 2065 2065 config KEXEC 2066 2066 bool "Kexec system call (EXPERIMENTAL)" 2067 2067 depends on (!SMP || PM_SLEEP_SMP) 2068 + select CRYPTO 2069 + select CRYPTO_SHA256 2068 2070 help 2069 2071 kexec is a system call that implements the ability to shutdown your 2070 2072 current kernel, and to start another kernel. It is like a reboot
+2
arch/ia64/Kconfig
··· 549 549 config KEXEC 550 550 bool "kexec system call" 551 551 depends on !IA64_HP_SIM && (!SMP || HOTPLUG_CPU) 552 + select CRYPTO 553 + select CRYPTO_SHA256 552 554 help 553 555 kexec is a system call that implements the ability to shutdown your 554 556 current kernel, and to start another kernel. It is like a reboot
+2
arch/m68k/Kconfig
··· 91 91 config KEXEC 92 92 bool "kexec system call" 93 93 depends on M68KCLASSIC 94 + select CRYPTO 95 + select CRYPTO_SHA256 94 96 help 95 97 kexec is a system call that implements the ability to shutdown your 96 98 current kernel, and to start another kernel. It is like a reboot
+2
arch/mips/Kconfig
··· 2396 2396 2397 2397 config KEXEC 2398 2398 bool "Kexec system call" 2399 + select CRYPTO 2400 + select CRYPTO_SHA256 2399 2401 help 2400 2402 kexec is a system call that implements the ability to shutdown your 2401 2403 current kernel, and to start another kernel. It is like a reboot
+2
arch/powerpc/Kconfig
··· 399 399 config KEXEC 400 400 bool "kexec system call" 401 401 depends on (PPC_BOOK3S || FSL_BOOKE || (44x && !SMP)) 402 + select CRYPTO 403 + select CRYPTO_SHA256 402 404 help 403 405 kexec is a system call that implements the ability to shutdown your 404 406 current kernel, and to start another kernel. It is like a reboot
+2
arch/s390/Kconfig
··· 48 48 49 49 config KEXEC 50 50 def_bool y 51 + select CRYPTO 52 + select CRYPTO_SHA256 51 53 52 54 config AUDIT_ARCH 53 55 def_bool y
+2
arch/sh/Kconfig
··· 595 595 config KEXEC 596 596 bool "kexec system call (EXPERIMENTAL)" 597 597 depends on SUPERH32 && MMU 598 + select CRYPTO 599 + select CRYPTO_SHA256 598 600 help 599 601 kexec is a system call that implements the ability to shutdown your 600 602 current kernel, and to start another kernel. It is like a reboot
+2
arch/tile/Kconfig
··· 191 191 192 192 config KEXEC 193 193 bool "kexec system call" 194 + select CRYPTO 195 + select CRYPTO_SHA256 194 196 ---help--- 195 197 kexec is a system call that implements the ability to shutdown your 196 198 current kernel, and to start another kernel. It is like a reboot
+2
arch/x86/Kconfig
··· 1583 1583 config KEXEC 1584 1584 bool "kexec system call" 1585 1585 select BUILD_BIN2C 1586 + select CRYPTO 1587 + select CRYPTO_SHA256 1586 1588 ---help--- 1587 1589 kexec is a system call that implements the ability to shutdown your 1588 1590 current kernel, and to start another kernel. It is like a reboot
+142
arch/x86/kernel/machine_kexec_64.c
··· 6 6 * Version 2. See the file COPYING for more details. 7 7 */ 8 8 9 + #define pr_fmt(fmt) "kexec: " fmt 10 + 9 11 #include <linux/mm.h> 10 12 #include <linux/kexec.h> 11 13 #include <linux/string.h> ··· 329 327 return 0; 330 328 331 329 return image->fops->cleanup(image); 330 + } 331 + 332 + /* 333 + * Apply purgatory relocations. 334 + * 335 + * ehdr: Pointer to elf headers 336 + * sechdrs: Pointer to section headers. 337 + * relsec: section index of SHT_RELA section. 338 + * 339 + * TODO: Some of the code belongs to generic code. Move that in kexec.c. 340 + */ 341 + int arch_kexec_apply_relocations_add(const Elf64_Ehdr *ehdr, 342 + Elf64_Shdr *sechdrs, unsigned int relsec) 343 + { 344 + unsigned int i; 345 + Elf64_Rela *rel; 346 + Elf64_Sym *sym; 347 + void *location; 348 + Elf64_Shdr *section, *symtabsec; 349 + unsigned long address, sec_base, value; 350 + const char *strtab, *name, *shstrtab; 351 + 352 + /* 353 + * ->sh_offset has been modified to keep the pointer to section 354 + * contents in memory 355 + */ 356 + rel = (void *)sechdrs[relsec].sh_offset; 357 + 358 + /* Section to which relocations apply */ 359 + section = &sechdrs[sechdrs[relsec].sh_info]; 360 + 361 + pr_debug("Applying relocate section %u to %u\n", relsec, 362 + sechdrs[relsec].sh_info); 363 + 364 + /* Associated symbol table */ 365 + symtabsec = &sechdrs[sechdrs[relsec].sh_link]; 366 + 367 + /* String table */ 368 + if (symtabsec->sh_link >= ehdr->e_shnum) { 369 + /* Invalid strtab section number */ 370 + pr_err("Invalid string table section index %d\n", 371 + symtabsec->sh_link); 372 + return -ENOEXEC; 373 + } 374 + 375 + strtab = (char *)sechdrs[symtabsec->sh_link].sh_offset; 376 + 377 + /* section header string table */ 378 + shstrtab = (char *)sechdrs[ehdr->e_shstrndx].sh_offset; 379 + 380 + for (i = 0; i < sechdrs[relsec].sh_size / sizeof(*rel); i++) { 381 + 382 + /* 383 + * rel[i].r_offset contains byte offset from beginning 384 + * of section to the storage unit affected. 385 + * 386 + * This is location to update (->sh_offset). This is temporary 387 + * buffer where section is currently loaded. This will finally 388 + * be loaded to a different address later, pointed to by 389 + * ->sh_addr. kexec takes care of moving it 390 + * (kexec_load_segment()). 391 + */ 392 + location = (void *)(section->sh_offset + rel[i].r_offset); 393 + 394 + /* Final address of the location */ 395 + address = section->sh_addr + rel[i].r_offset; 396 + 397 + /* 398 + * rel[i].r_info contains information about symbol table index 399 + * w.r.t which relocation must be made and type of relocation 400 + * to apply. ELF64_R_SYM() and ELF64_R_TYPE() macros get 401 + * these respectively. 402 + */ 403 + sym = (Elf64_Sym *)symtabsec->sh_offset + 404 + ELF64_R_SYM(rel[i].r_info); 405 + 406 + if (sym->st_name) 407 + name = strtab + sym->st_name; 408 + else 409 + name = shstrtab + sechdrs[sym->st_shndx].sh_name; 410 + 411 + pr_debug("Symbol: %s info: %02x shndx: %02x value=%llx size: %llx\n", 412 + name, sym->st_info, sym->st_shndx, sym->st_value, 413 + sym->st_size); 414 + 415 + if (sym->st_shndx == SHN_UNDEF) { 416 + pr_err("Undefined symbol: %s\n", name); 417 + return -ENOEXEC; 418 + } 419 + 420 + if (sym->st_shndx == SHN_COMMON) { 421 + pr_err("symbol '%s' in common section\n", name); 422 + return -ENOEXEC; 423 + } 424 + 425 + if (sym->st_shndx == SHN_ABS) 426 + sec_base = 0; 427 + else if (sym->st_shndx >= ehdr->e_shnum) { 428 + pr_err("Invalid section %d for symbol %s\n", 429 + sym->st_shndx, name); 430 + return -ENOEXEC; 431 + } else 432 + sec_base = sechdrs[sym->st_shndx].sh_addr; 433 + 434 + value = sym->st_value; 435 + value += sec_base; 436 + value += rel[i].r_addend; 437 + 438 + switch (ELF64_R_TYPE(rel[i].r_info)) { 439 + case R_X86_64_NONE: 440 + break; 441 + case R_X86_64_64: 442 + *(u64 *)location = value; 443 + break; 444 + case R_X86_64_32: 445 + *(u32 *)location = value; 446 + if (value != *(u32 *)location) 447 + goto overflow; 448 + break; 449 + case R_X86_64_32S: 450 + *(s32 *)location = value; 451 + if ((s64)value != *(s32 *)location) 452 + goto overflow; 453 + break; 454 + case R_X86_64_PC32: 455 + value -= (u64)address; 456 + *(u32 *)location = value; 457 + break; 458 + default: 459 + pr_err("Unknown rela relocation: %llu\n", 460 + ELF64_R_TYPE(rel[i].r_info)); 461 + return -ENOEXEC; 462 + } 463 + } 464 + return 0; 465 + 466 + overflow: 467 + pr_err("Overflow in relocation type %d value 0x%lx\n", 468 + (int)ELF64_R_TYPE(rel[i].r_info), value); 469 + return -ENOEXEC; 332 470 }
+33
include/linux/kexec.h
··· 10 10 #include <linux/ioport.h> 11 11 #include <linux/elfcore.h> 12 12 #include <linux/elf.h> 13 + #include <linux/module.h> 13 14 #include <asm/kexec.h> 14 15 15 16 /* Verify architecture specific macros are defined */ ··· 96 95 }; 97 96 #endif 98 97 98 + struct kexec_sha_region { 99 + unsigned long start; 100 + unsigned long len; 101 + }; 102 + 103 + struct purgatory_info { 104 + /* Pointer to elf header of read only purgatory */ 105 + Elf_Ehdr *ehdr; 106 + 107 + /* Pointer to purgatory sechdrs which are modifiable */ 108 + Elf_Shdr *sechdrs; 109 + /* 110 + * Temporary buffer location where purgatory is loaded and relocated 111 + * This memory can be freed post image load 112 + */ 113 + void *purgatory_buf; 114 + 115 + /* Address where purgatory is finally loaded and is executed from */ 116 + unsigned long purgatory_load_addr; 117 + }; 118 + 99 119 struct kimage { 100 120 kimage_entry_t head; 101 121 kimage_entry_t *entry; ··· 165 143 166 144 /* Image loader handling the kernel can store a pointer here */ 167 145 void *image_loader_data; 146 + 147 + /* Information for loading purgatory */ 148 + struct purgatory_info purgatory_info; 168 149 }; 169 150 170 151 /* ··· 214 189 unsigned long *load_addr); 215 190 extern struct page *kimage_alloc_control_pages(struct kimage *image, 216 191 unsigned int order); 192 + extern int kexec_load_purgatory(struct kimage *image, unsigned long min, 193 + unsigned long max, int top_down, 194 + unsigned long *load_addr); 195 + extern int kexec_purgatory_get_set_symbol(struct kimage *image, 196 + const char *name, void *buf, 197 + unsigned int size, bool get_value); 198 + extern void *kexec_purgatory_get_symbol_addr(struct kimage *image, 199 + const char *name); 217 200 extern void crash_kexec(struct pt_regs *); 218 201 int kexec_should_crash(struct task_struct *); 219 202 void crash_save_cpu(struct pt_regs *regs, int cpu);
+543 -1
kernel/kexec.c
··· 42 42 #include <asm/io.h> 43 43 #include <asm/sections.h> 44 44 45 + #include <crypto/hash.h> 46 + #include <crypto/sha.h> 47 + 45 48 /* Per cpu memory for storing cpu states in case of system crash. */ 46 49 note_buf_t __percpu *crash_notes; 47 50 ··· 56 53 57 54 /* Flag to indicate we are going to kexec a new kernel */ 58 55 bool kexec_in_progress = false; 56 + 57 + /* 58 + * Declare these symbols weak so that if architecture provides a purgatory, 59 + * these will be overridden. 60 + */ 61 + char __weak kexec_purgatory[0]; 62 + size_t __weak kexec_purgatory_size = 0; 63 + 64 + static int kexec_calculate_store_digests(struct kimage *image); 59 65 60 66 /* Location of the reserved area for the crash kernel */ 61 67 struct resource crashk_res = { ··· 416 404 { 417 405 } 418 406 407 + /* Apply relocations of type RELA */ 408 + int __weak 409 + arch_kexec_apply_relocations_add(const Elf_Ehdr *ehdr, Elf_Shdr *sechdrs, 410 + unsigned int relsec) 411 + { 412 + pr_err("RELA relocation unsupported.\n"); 413 + return -ENOEXEC; 414 + } 415 + 416 + /* Apply relocations of type REL */ 417 + int __weak 418 + arch_kexec_apply_relocations(const Elf_Ehdr *ehdr, Elf_Shdr *sechdrs, 419 + unsigned int relsec) 420 + { 421 + pr_err("REL relocation unsupported.\n"); 422 + return -ENOEXEC; 423 + } 424 + 419 425 /* 420 426 * Free up memory used by kernel, initrd, and comand line. This is temporary 421 427 * memory allocation which is not needed any more after these buffers have ··· 441 411 */ 442 412 static void kimage_file_post_load_cleanup(struct kimage *image) 443 413 { 414 + struct purgatory_info *pi = &image->purgatory_info; 415 + 444 416 vfree(image->kernel_buf); 445 417 image->kernel_buf = NULL; 446 418 ··· 451 419 452 420 kfree(image->cmdline_buf); 453 421 image->cmdline_buf = NULL; 422 + 423 + vfree(pi->purgatory_buf); 424 + pi->purgatory_buf = NULL; 425 + 426 + vfree(pi->sechdrs); 427 + pi->sechdrs = NULL; 454 428 455 429 /* See if architecture has anything to cleanup post load */ 456 430 arch_kimage_file_post_load_cleanup(image); ··· 1143 1105 } 1144 1106 ubytes -= uchunk; 1145 1107 maddr += mchunk; 1146 - buf += mchunk; 1108 + buf += mchunk; 1147 1109 mbytes -= mchunk; 1148 1110 } 1149 1111 out: ··· 1375 1337 goto out; 1376 1338 1377 1339 ret = machine_kexec_prepare(image); 1340 + if (ret) 1341 + goto out; 1342 + 1343 + ret = kexec_calculate_store_digests(image); 1378 1344 if (ret) 1379 1345 goto out; 1380 1346 ··· 2134 2092 return 0; 2135 2093 } 2136 2094 2095 + /* Calculate and store the digest of segments */ 2096 + static int kexec_calculate_store_digests(struct kimage *image) 2097 + { 2098 + struct crypto_shash *tfm; 2099 + struct shash_desc *desc; 2100 + int ret = 0, i, j, zero_buf_sz, sha_region_sz; 2101 + size_t desc_size, nullsz; 2102 + char *digest; 2103 + void *zero_buf; 2104 + struct kexec_sha_region *sha_regions; 2105 + struct purgatory_info *pi = &image->purgatory_info; 2106 + 2107 + zero_buf = __va(page_to_pfn(ZERO_PAGE(0)) << PAGE_SHIFT); 2108 + zero_buf_sz = PAGE_SIZE; 2109 + 2110 + tfm = crypto_alloc_shash("sha256", 0, 0); 2111 + if (IS_ERR(tfm)) { 2112 + ret = PTR_ERR(tfm); 2113 + goto out; 2114 + } 2115 + 2116 + desc_size = crypto_shash_descsize(tfm) + sizeof(*desc); 2117 + desc = kzalloc(desc_size, GFP_KERNEL); 2118 + if (!desc) { 2119 + ret = -ENOMEM; 2120 + goto out_free_tfm; 2121 + } 2122 + 2123 + sha_region_sz = KEXEC_SEGMENT_MAX * sizeof(struct kexec_sha_region); 2124 + sha_regions = vzalloc(sha_region_sz); 2125 + if (!sha_regions) 2126 + goto out_free_desc; 2127 + 2128 + desc->tfm = tfm; 2129 + desc->flags = 0; 2130 + 2131 + ret = crypto_shash_init(desc); 2132 + if (ret < 0) 2133 + goto out_free_sha_regions; 2134 + 2135 + digest = kzalloc(SHA256_DIGEST_SIZE, GFP_KERNEL); 2136 + if (!digest) { 2137 + ret = -ENOMEM; 2138 + goto out_free_sha_regions; 2139 + } 2140 + 2141 + for (j = i = 0; i < image->nr_segments; i++) { 2142 + struct kexec_segment *ksegment; 2143 + 2144 + ksegment = &image->segment[i]; 2145 + /* 2146 + * Skip purgatory as it will be modified once we put digest 2147 + * info in purgatory. 2148 + */ 2149 + if (ksegment->kbuf == pi->purgatory_buf) 2150 + continue; 2151 + 2152 + ret = crypto_shash_update(desc, ksegment->kbuf, 2153 + ksegment->bufsz); 2154 + if (ret) 2155 + break; 2156 + 2157 + /* 2158 + * Assume rest of the buffer is filled with zero and 2159 + * update digest accordingly. 2160 + */ 2161 + nullsz = ksegment->memsz - ksegment->bufsz; 2162 + while (nullsz) { 2163 + unsigned long bytes = nullsz; 2164 + 2165 + if (bytes > zero_buf_sz) 2166 + bytes = zero_buf_sz; 2167 + ret = crypto_shash_update(desc, zero_buf, bytes); 2168 + if (ret) 2169 + break; 2170 + nullsz -= bytes; 2171 + } 2172 + 2173 + if (ret) 2174 + break; 2175 + 2176 + sha_regions[j].start = ksegment->mem; 2177 + sha_regions[j].len = ksegment->memsz; 2178 + j++; 2179 + } 2180 + 2181 + if (!ret) { 2182 + ret = crypto_shash_final(desc, digest); 2183 + if (ret) 2184 + goto out_free_digest; 2185 + ret = kexec_purgatory_get_set_symbol(image, "sha_regions", 2186 + sha_regions, sha_region_sz, 0); 2187 + if (ret) 2188 + goto out_free_digest; 2189 + 2190 + ret = kexec_purgatory_get_set_symbol(image, "sha256_digest", 2191 + digest, SHA256_DIGEST_SIZE, 0); 2192 + if (ret) 2193 + goto out_free_digest; 2194 + } 2195 + 2196 + out_free_digest: 2197 + kfree(digest); 2198 + out_free_sha_regions: 2199 + vfree(sha_regions); 2200 + out_free_desc: 2201 + kfree(desc); 2202 + out_free_tfm: 2203 + kfree(tfm); 2204 + out: 2205 + return ret; 2206 + } 2207 + 2208 + /* Actually load purgatory. Lot of code taken from kexec-tools */ 2209 + static int __kexec_load_purgatory(struct kimage *image, unsigned long min, 2210 + unsigned long max, int top_down) 2211 + { 2212 + struct purgatory_info *pi = &image->purgatory_info; 2213 + unsigned long align, buf_align, bss_align, buf_sz, bss_sz, bss_pad; 2214 + unsigned long memsz, entry, load_addr, curr_load_addr, bss_addr, offset; 2215 + unsigned char *buf_addr, *src; 2216 + int i, ret = 0, entry_sidx = -1; 2217 + const Elf_Shdr *sechdrs_c; 2218 + Elf_Shdr *sechdrs = NULL; 2219 + void *purgatory_buf = NULL; 2220 + 2221 + /* 2222 + * sechdrs_c points to section headers in purgatory and are read 2223 + * only. No modifications allowed. 2224 + */ 2225 + sechdrs_c = (void *)pi->ehdr + pi->ehdr->e_shoff; 2226 + 2227 + /* 2228 + * We can not modify sechdrs_c[] and its fields. It is read only. 2229 + * Copy it over to a local copy where one can store some temporary 2230 + * data and free it at the end. We need to modify ->sh_addr and 2231 + * ->sh_offset fields to keep track of permanent and temporary 2232 + * locations of sections. 2233 + */ 2234 + sechdrs = vzalloc(pi->ehdr->e_shnum * sizeof(Elf_Shdr)); 2235 + if (!sechdrs) 2236 + return -ENOMEM; 2237 + 2238 + memcpy(sechdrs, sechdrs_c, pi->ehdr->e_shnum * sizeof(Elf_Shdr)); 2239 + 2240 + /* 2241 + * We seem to have multiple copies of sections. First copy is which 2242 + * is embedded in kernel in read only section. Some of these sections 2243 + * will be copied to a temporary buffer and relocated. And these 2244 + * sections will finally be copied to their final destination at 2245 + * segment load time. 2246 + * 2247 + * Use ->sh_offset to reflect section address in memory. It will 2248 + * point to original read only copy if section is not allocatable. 2249 + * Otherwise it will point to temporary copy which will be relocated. 2250 + * 2251 + * Use ->sh_addr to contain final address of the section where it 2252 + * will go during execution time. 2253 + */ 2254 + for (i = 0; i < pi->ehdr->e_shnum; i++) { 2255 + if (sechdrs[i].sh_type == SHT_NOBITS) 2256 + continue; 2257 + 2258 + sechdrs[i].sh_offset = (unsigned long)pi->ehdr + 2259 + sechdrs[i].sh_offset; 2260 + } 2261 + 2262 + /* 2263 + * Identify entry point section and make entry relative to section 2264 + * start. 2265 + */ 2266 + entry = pi->ehdr->e_entry; 2267 + for (i = 0; i < pi->ehdr->e_shnum; i++) { 2268 + if (!(sechdrs[i].sh_flags & SHF_ALLOC)) 2269 + continue; 2270 + 2271 + if (!(sechdrs[i].sh_flags & SHF_EXECINSTR)) 2272 + continue; 2273 + 2274 + /* Make entry section relative */ 2275 + if (sechdrs[i].sh_addr <= pi->ehdr->e_entry && 2276 + ((sechdrs[i].sh_addr + sechdrs[i].sh_size) > 2277 + pi->ehdr->e_entry)) { 2278 + entry_sidx = i; 2279 + entry -= sechdrs[i].sh_addr; 2280 + break; 2281 + } 2282 + } 2283 + 2284 + /* Determine how much memory is needed to load relocatable object. */ 2285 + buf_align = 1; 2286 + bss_align = 1; 2287 + buf_sz = 0; 2288 + bss_sz = 0; 2289 + 2290 + for (i = 0; i < pi->ehdr->e_shnum; i++) { 2291 + if (!(sechdrs[i].sh_flags & SHF_ALLOC)) 2292 + continue; 2293 + 2294 + align = sechdrs[i].sh_addralign; 2295 + if (sechdrs[i].sh_type != SHT_NOBITS) { 2296 + if (buf_align < align) 2297 + buf_align = align; 2298 + buf_sz = ALIGN(buf_sz, align); 2299 + buf_sz += sechdrs[i].sh_size; 2300 + } else { 2301 + /* bss section */ 2302 + if (bss_align < align) 2303 + bss_align = align; 2304 + bss_sz = ALIGN(bss_sz, align); 2305 + bss_sz += sechdrs[i].sh_size; 2306 + } 2307 + } 2308 + 2309 + /* Determine the bss padding required to align bss properly */ 2310 + bss_pad = 0; 2311 + if (buf_sz & (bss_align - 1)) 2312 + bss_pad = bss_align - (buf_sz & (bss_align - 1)); 2313 + 2314 + memsz = buf_sz + bss_pad + bss_sz; 2315 + 2316 + /* Allocate buffer for purgatory */ 2317 + purgatory_buf = vzalloc(buf_sz); 2318 + if (!purgatory_buf) { 2319 + ret = -ENOMEM; 2320 + goto out; 2321 + } 2322 + 2323 + if (buf_align < bss_align) 2324 + buf_align = bss_align; 2325 + 2326 + /* Add buffer to segment list */ 2327 + ret = kexec_add_buffer(image, purgatory_buf, buf_sz, memsz, 2328 + buf_align, min, max, top_down, 2329 + &pi->purgatory_load_addr); 2330 + if (ret) 2331 + goto out; 2332 + 2333 + /* Load SHF_ALLOC sections */ 2334 + buf_addr = purgatory_buf; 2335 + load_addr = curr_load_addr = pi->purgatory_load_addr; 2336 + bss_addr = load_addr + buf_sz + bss_pad; 2337 + 2338 + for (i = 0; i < pi->ehdr->e_shnum; i++) { 2339 + if (!(sechdrs[i].sh_flags & SHF_ALLOC)) 2340 + continue; 2341 + 2342 + align = sechdrs[i].sh_addralign; 2343 + if (sechdrs[i].sh_type != SHT_NOBITS) { 2344 + curr_load_addr = ALIGN(curr_load_addr, align); 2345 + offset = curr_load_addr - load_addr; 2346 + /* We already modifed ->sh_offset to keep src addr */ 2347 + src = (char *) sechdrs[i].sh_offset; 2348 + memcpy(buf_addr + offset, src, sechdrs[i].sh_size); 2349 + 2350 + /* Store load address and source address of section */ 2351 + sechdrs[i].sh_addr = curr_load_addr; 2352 + 2353 + /* 2354 + * This section got copied to temporary buffer. Update 2355 + * ->sh_offset accordingly. 2356 + */ 2357 + sechdrs[i].sh_offset = (unsigned long)(buf_addr + offset); 2358 + 2359 + /* Advance to the next address */ 2360 + curr_load_addr += sechdrs[i].sh_size; 2361 + } else { 2362 + bss_addr = ALIGN(bss_addr, align); 2363 + sechdrs[i].sh_addr = bss_addr; 2364 + bss_addr += sechdrs[i].sh_size; 2365 + } 2366 + } 2367 + 2368 + /* Update entry point based on load address of text section */ 2369 + if (entry_sidx >= 0) 2370 + entry += sechdrs[entry_sidx].sh_addr; 2371 + 2372 + /* Make kernel jump to purgatory after shutdown */ 2373 + image->start = entry; 2374 + 2375 + /* Used later to get/set symbol values */ 2376 + pi->sechdrs = sechdrs; 2377 + 2378 + /* 2379 + * Used later to identify which section is purgatory and skip it 2380 + * from checksumming. 2381 + */ 2382 + pi->purgatory_buf = purgatory_buf; 2383 + return ret; 2384 + out: 2385 + vfree(sechdrs); 2386 + vfree(purgatory_buf); 2387 + return ret; 2388 + } 2389 + 2390 + static int kexec_apply_relocations(struct kimage *image) 2391 + { 2392 + int i, ret; 2393 + struct purgatory_info *pi = &image->purgatory_info; 2394 + Elf_Shdr *sechdrs = pi->sechdrs; 2395 + 2396 + /* Apply relocations */ 2397 + for (i = 0; i < pi->ehdr->e_shnum; i++) { 2398 + Elf_Shdr *section, *symtab; 2399 + 2400 + if (sechdrs[i].sh_type != SHT_RELA && 2401 + sechdrs[i].sh_type != SHT_REL) 2402 + continue; 2403 + 2404 + /* 2405 + * For section of type SHT_RELA/SHT_REL, 2406 + * ->sh_link contains section header index of associated 2407 + * symbol table. And ->sh_info contains section header 2408 + * index of section to which relocations apply. 2409 + */ 2410 + if (sechdrs[i].sh_info >= pi->ehdr->e_shnum || 2411 + sechdrs[i].sh_link >= pi->ehdr->e_shnum) 2412 + return -ENOEXEC; 2413 + 2414 + section = &sechdrs[sechdrs[i].sh_info]; 2415 + symtab = &sechdrs[sechdrs[i].sh_link]; 2416 + 2417 + if (!(section->sh_flags & SHF_ALLOC)) 2418 + continue; 2419 + 2420 + /* 2421 + * symtab->sh_link contain section header index of associated 2422 + * string table. 2423 + */ 2424 + if (symtab->sh_link >= pi->ehdr->e_shnum) 2425 + /* Invalid section number? */ 2426 + continue; 2427 + 2428 + /* 2429 + * Respective archicture needs to provide support for applying 2430 + * relocations of type SHT_RELA/SHT_REL. 2431 + */ 2432 + if (sechdrs[i].sh_type == SHT_RELA) 2433 + ret = arch_kexec_apply_relocations_add(pi->ehdr, 2434 + sechdrs, i); 2435 + else if (sechdrs[i].sh_type == SHT_REL) 2436 + ret = arch_kexec_apply_relocations(pi->ehdr, 2437 + sechdrs, i); 2438 + if (ret) 2439 + return ret; 2440 + } 2441 + 2442 + return 0; 2443 + } 2444 + 2445 + /* Load relocatable purgatory object and relocate it appropriately */ 2446 + int kexec_load_purgatory(struct kimage *image, unsigned long min, 2447 + unsigned long max, int top_down, 2448 + unsigned long *load_addr) 2449 + { 2450 + struct purgatory_info *pi = &image->purgatory_info; 2451 + int ret; 2452 + 2453 + if (kexec_purgatory_size <= 0) 2454 + return -EINVAL; 2455 + 2456 + if (kexec_purgatory_size < sizeof(Elf_Ehdr)) 2457 + return -ENOEXEC; 2458 + 2459 + pi->ehdr = (Elf_Ehdr *)kexec_purgatory; 2460 + 2461 + if (memcmp(pi->ehdr->e_ident, ELFMAG, SELFMAG) != 0 2462 + || pi->ehdr->e_type != ET_REL 2463 + || !elf_check_arch(pi->ehdr) 2464 + || pi->ehdr->e_shentsize != sizeof(Elf_Shdr)) 2465 + return -ENOEXEC; 2466 + 2467 + if (pi->ehdr->e_shoff >= kexec_purgatory_size 2468 + || (pi->ehdr->e_shnum * sizeof(Elf_Shdr) > 2469 + kexec_purgatory_size - pi->ehdr->e_shoff)) 2470 + return -ENOEXEC; 2471 + 2472 + ret = __kexec_load_purgatory(image, min, max, top_down); 2473 + if (ret) 2474 + return ret; 2475 + 2476 + ret = kexec_apply_relocations(image); 2477 + if (ret) 2478 + goto out; 2479 + 2480 + *load_addr = pi->purgatory_load_addr; 2481 + return 0; 2482 + out: 2483 + vfree(pi->sechdrs); 2484 + vfree(pi->purgatory_buf); 2485 + return ret; 2486 + } 2487 + 2488 + static Elf_Sym *kexec_purgatory_find_symbol(struct purgatory_info *pi, 2489 + const char *name) 2490 + { 2491 + Elf_Sym *syms; 2492 + Elf_Shdr *sechdrs; 2493 + Elf_Ehdr *ehdr; 2494 + int i, k; 2495 + const char *strtab; 2496 + 2497 + if (!pi->sechdrs || !pi->ehdr) 2498 + return NULL; 2499 + 2500 + sechdrs = pi->sechdrs; 2501 + ehdr = pi->ehdr; 2502 + 2503 + for (i = 0; i < ehdr->e_shnum; i++) { 2504 + if (sechdrs[i].sh_type != SHT_SYMTAB) 2505 + continue; 2506 + 2507 + if (sechdrs[i].sh_link >= ehdr->e_shnum) 2508 + /* Invalid strtab section number */ 2509 + continue; 2510 + strtab = (char *)sechdrs[sechdrs[i].sh_link].sh_offset; 2511 + syms = (Elf_Sym *)sechdrs[i].sh_offset; 2512 + 2513 + /* Go through symbols for a match */ 2514 + for (k = 0; k < sechdrs[i].sh_size/sizeof(Elf_Sym); k++) { 2515 + if (ELF_ST_BIND(syms[k].st_info) != STB_GLOBAL) 2516 + continue; 2517 + 2518 + if (strcmp(strtab + syms[k].st_name, name) != 0) 2519 + continue; 2520 + 2521 + if (syms[k].st_shndx == SHN_UNDEF || 2522 + syms[k].st_shndx >= ehdr->e_shnum) { 2523 + pr_debug("Symbol: %s has bad section index %d.\n", 2524 + name, syms[k].st_shndx); 2525 + return NULL; 2526 + } 2527 + 2528 + /* Found the symbol we are looking for */ 2529 + return &syms[k]; 2530 + } 2531 + } 2532 + 2533 + return NULL; 2534 + } 2535 + 2536 + void *kexec_purgatory_get_symbol_addr(struct kimage *image, const char *name) 2537 + { 2538 + struct purgatory_info *pi = &image->purgatory_info; 2539 + Elf_Sym *sym; 2540 + Elf_Shdr *sechdr; 2541 + 2542 + sym = kexec_purgatory_find_symbol(pi, name); 2543 + if (!sym) 2544 + return ERR_PTR(-EINVAL); 2545 + 2546 + sechdr = &pi->sechdrs[sym->st_shndx]; 2547 + 2548 + /* 2549 + * Returns the address where symbol will finally be loaded after 2550 + * kexec_load_segment() 2551 + */ 2552 + return (void *)(sechdr->sh_addr + sym->st_value); 2553 + } 2554 + 2555 + /* 2556 + * Get or set value of a symbol. If "get_value" is true, symbol value is 2557 + * returned in buf otherwise symbol value is set based on value in buf. 2558 + */ 2559 + int kexec_purgatory_get_set_symbol(struct kimage *image, const char *name, 2560 + void *buf, unsigned int size, bool get_value) 2561 + { 2562 + Elf_Sym *sym; 2563 + Elf_Shdr *sechdrs; 2564 + struct purgatory_info *pi = &image->purgatory_info; 2565 + char *sym_buf; 2566 + 2567 + sym = kexec_purgatory_find_symbol(pi, name); 2568 + if (!sym) 2569 + return -EINVAL; 2570 + 2571 + if (sym->st_size != size) { 2572 + pr_err("symbol %s size mismatch: expected %lu actual %u\n", 2573 + name, (unsigned long)sym->st_size, size); 2574 + return -EINVAL; 2575 + } 2576 + 2577 + sechdrs = pi->sechdrs; 2578 + 2579 + if (sechdrs[sym->st_shndx].sh_type == SHT_NOBITS) { 2580 + pr_err("symbol %s is in a bss section. Cannot %s\n", name, 2581 + get_value ? "get" : "set"); 2582 + return -EINVAL; 2583 + } 2584 + 2585 + sym_buf = (unsigned char *)sechdrs[sym->st_shndx].sh_offset + 2586 + sym->st_value; 2587 + 2588 + if (get_value) 2589 + memcpy((void *)buf, sym_buf, size); 2590 + else 2591 + memcpy((void *)sym_buf, buf, size); 2592 + 2593 + return 0; 2594 + } 2137 2595 2138 2596 /* 2139 2597 * Move into place and start executing a preloaded standalone