Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux

Merge tag 'f2fs-for-6.15-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs

Pull f2fs updates from Jaegeuk Kim:
"In this round, there are three major updates: (1) folio conversion,
(2) refactoring for mount API conversion, (3) some performance
improvement such as direct IO, checkpoint speed, and IO priority
hints.

For stability, there are patches which add more sanity checks and
fixes some major issues like i_size in atomic write operations and
write pointer recovery in zoned devices.

Enhancements:
- huge folio converion work by Matthew Wilcox
- clean up for mount API conversion by Eric Sandeen
- improve direct IO speed in the overwrite case
- add some sanity check on node consistency
- set highest IO priority for checkpoint thread
- keep POSIX_FADV_NOREUSE ranges and add sysfs entry to reclaim pages
- add ioctl to get IO priority hint
- add carve_out sysfs node for fsstat

Bug fixes:
- disable nat_bits during umount to avoid potential nat entry corruption
- fix missing i_size update on atomic writes
- fix missing discard for active segments
- fix running out of free segments
- fix out-of-bounds access in f2fs_truncate_inode_blocks()
- call f2fs_recover_quota_end() correctly
- fix potential deadloop in prepare_compress_overwrite()
- fix the missing write pointer correction for zoned device
- fix to avoid panic once fallocation fails for pinfile
- don't retry IO for corrupted data scenario

There are many other clean up patches and minor bug fixes as usual"

* tag 'f2fs-for-6.15-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs: (68 commits)
f2fs: fix missing discard for active segments
f2fs: optimize f2fs DIO overwrites
f2fs: fix to avoid atomicity corruption of atomic file
f2fs: pass sbi rather than sb to parse_options()
f2fs: pass sbi rather than sb to quota qf_name helpers
f2fs: defer readonly check vs norecovery
f2fs: Pass sbi rather than sb to f2fs_set_test_dummy_encryption
f2fs: make LAZYTIME a mount option flag
f2fs: make INLINECRYPT a mount option flag
f2fs: factor out an f2fs_default_check function
f2fs: consolidate unsupported option handling errors
f2fs: use f2fs_sb_has_device_alias during option parsing
f2fs: add carve_out sysfs node
f2fs: fix to avoid running out of free segments
f2fs: Remove f2fs_write_node_page()
f2fs: Remove f2fs_write_meta_page()
f2fs: Remove f2fs_write_data_page()
f2fs: Remove check for ->writepage
Revert "f2fs: rebuild nat_bits during umount"
f2fs: fix to avoid accessing uninitialized curseg
...

+1060 -605
+18
Documentation/ABI/testing/sysfs-fs-f2fs
··· 734 734 FAULT_BLKADDR_VALIDITY 0x000040000 735 735 FAULT_BLKADDR_CONSISTENCE 0x000080000 736 736 FAULT_NO_SEGMENT 0x000100000 737 + FAULT_INCONSISTENT_FOOTER 0x000200000 737 738 =========================== =========== 738 739 739 740 What: /sys/fs/f2fs/<disk>/discard_io_aware_gran ··· 829 828 Contact: "Chao Yu" <chao@kernel.org> 830 829 Description: It controls max read extent count for per-inode, the value of threshold 831 830 is 10240 by default. 831 + 832 + What: /sys/fs/f2fs/tuning/reclaim_caches_kb 833 + Date: February 2025 834 + Contact: "Jaegeuk Kim" <jaegeuk@kernel.org> 835 + Description: It reclaims the given KBs of file-backed pages registered by 836 + ioctl(F2FS_IOC_DONATE_RANGE). 837 + For example, writing N tries to drop N KBs spaces in LRU. 838 + 839 + What: /sys/fs/f2fs/<disk>/carve_out 840 + Date: March 2025 841 + Contact: "Daeho Jeong" <daehojeong@google.com> 842 + Description: For several zoned storage devices, vendors will provide extra space which 843 + was used for device level GC than specs and F2FS can use this space for 844 + filesystem level GC. To do that, we can reserve the space using 845 + reserved_blocks. However, it is not enough, since this extra space should 846 + not be shown to users. So, with this new sysfs node, we can hide the space 847 + by substracting reserved_blocks from total bytes.
+3
Documentation/filesystems/f2fs.rst
··· 206 206 FAULT_BLKADDR_VALIDITY 0x000040000 207 207 FAULT_BLKADDR_CONSISTENCE 0x000080000 208 208 FAULT_NO_SEGMENT 0x000100000 209 + FAULT_INCONSISTENT_FOOTER 0x000200000 209 210 =========================== =========== 210 211 mode=%s Control block allocation mode which supports "adaptive" 211 212 and "lfs". In "lfs" mode, there should be no random ··· 366 365 pending node write drop keep N/A 367 366 pending meta write keep keep N/A 368 367 ====================== =============== =============== ======== 368 + nat_bits Enable nat_bits feature to enhance full/empty nat blocks access, 369 + by default it's disabled. 369 370 ======================== ============================================================ 370 371 371 372 Debugfs Entries
+29 -42
fs/f2fs/checkpoint.c
··· 21 21 #include "iostat.h" 22 22 #include <trace/events/f2fs.h> 23 23 24 - #define DEFAULT_CHECKPOINT_IOPRIO (IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, 3)) 24 + #define DEFAULT_CHECKPOINT_IOPRIO (IOPRIO_PRIO_VALUE(IOPRIO_CLASS_RT, 3)) 25 25 26 26 static struct kmem_cache *ino_entry_slab; 27 27 struct kmem_cache *f2fs_inode_entry_slab; ··· 58 58 bool is_meta) 59 59 { 60 60 struct address_space *mapping = META_MAPPING(sbi); 61 - struct page *page; 61 + struct folio *folio; 62 62 struct f2fs_io_info fio = { 63 63 .sbi = sbi, 64 64 .type = META, ··· 74 74 if (unlikely(!is_meta)) 75 75 fio.op_flags &= ~REQ_META; 76 76 repeat: 77 - page = f2fs_grab_cache_page(mapping, index, false); 78 - if (!page) { 77 + folio = f2fs_grab_cache_folio(mapping, index, false); 78 + if (IS_ERR(folio)) { 79 79 cond_resched(); 80 80 goto repeat; 81 81 } 82 - if (PageUptodate(page)) 82 + if (folio_test_uptodate(folio)) 83 83 goto out; 84 84 85 - fio.page = page; 85 + fio.page = &folio->page; 86 86 87 87 err = f2fs_submit_page_bio(&fio); 88 88 if (err) { 89 - f2fs_put_page(page, 1); 89 + f2fs_folio_put(folio, true); 90 90 return ERR_PTR(err); 91 91 } 92 92 93 93 f2fs_update_iostat(sbi, NULL, FS_META_READ_IO, F2FS_BLKSIZE); 94 94 95 - lock_page(page); 96 - if (unlikely(page->mapping != mapping)) { 97 - f2fs_put_page(page, 1); 95 + folio_lock(folio); 96 + if (unlikely(folio->mapping != mapping)) { 97 + f2fs_folio_put(folio, true); 98 98 goto repeat; 99 99 } 100 100 101 - if (unlikely(!PageUptodate(page))) { 102 - f2fs_handle_page_eio(sbi, page_folio(page), META); 103 - f2fs_put_page(page, 1); 101 + if (unlikely(!folio_test_uptodate(folio))) { 102 + f2fs_handle_page_eio(sbi, folio, META); 103 + f2fs_folio_put(folio, true); 104 104 return ERR_PTR(-EIO); 105 105 } 106 106 out: 107 - return page; 107 + return &folio->page; 108 108 } 109 109 110 110 struct page *f2fs_get_meta_page(struct f2fs_sb_info *sbi, pgoff_t index) ··· 381 381 return AOP_WRITEPAGE_ACTIVATE; 382 382 } 383 383 384 - static int f2fs_write_meta_page(struct page *page, 385 - struct writeback_control *wbc) 386 - { 387 - return __f2fs_write_meta_page(page, wbc, FS_META_IO); 388 - } 389 - 390 384 static int f2fs_write_meta_pages(struct address_space *mapping, 391 385 struct writeback_control *wbc) 392 386 { ··· 501 507 } 502 508 503 509 const struct address_space_operations f2fs_meta_aops = { 504 - .writepage = f2fs_write_meta_page, 505 510 .writepages = f2fs_write_meta_pages, 506 511 .dirty_folio = f2fs_dirty_meta_folio, 507 512 .invalidate_folio = f2fs_invalidate_folio, ··· 1230 1237 retry_flush_quotas: 1231 1238 f2fs_lock_all(sbi); 1232 1239 if (__need_flush_quota(sbi)) { 1233 - int locked; 1240 + bool need_lock = sbi->umount_lock_holder != current; 1234 1241 1235 1242 if (++cnt > DEFAULT_RETRY_QUOTA_FLUSH_COUNT) { 1236 1243 set_sbi_flag(sbi, SBI_QUOTA_SKIP_FLUSH); ··· 1239 1246 } 1240 1247 f2fs_unlock_all(sbi); 1241 1248 1242 - /* only failed during mount/umount/freeze/quotactl */ 1243 - locked = down_read_trylock(&sbi->sb->s_umount); 1244 - f2fs_quota_sync(sbi->sb, -1); 1245 - if (locked) 1249 + /* don't grab s_umount lock during mount/umount/remount/freeze/quotactl */ 1250 + if (!need_lock) { 1251 + f2fs_do_quota_sync(sbi->sb, -1); 1252 + } else if (down_read_trylock(&sbi->sb->s_umount)) { 1253 + f2fs_do_quota_sync(sbi->sb, -1); 1246 1254 up_read(&sbi->sb->s_umount); 1255 + } 1247 1256 cond_resched(); 1248 1257 goto retry_flush_quotas; 1249 1258 } ··· 1339 1344 struct f2fs_checkpoint *ckpt = F2FS_CKPT(sbi); 1340 1345 unsigned long flags; 1341 1346 1342 - if (cpc->reason & CP_UMOUNT) { 1343 - if (le32_to_cpu(ckpt->cp_pack_total_block_count) + 1344 - NM_I(sbi)->nat_bits_blocks > BLKS_PER_SEG(sbi)) { 1345 - clear_ckpt_flags(sbi, CP_NAT_BITS_FLAG); 1346 - f2fs_notice(sbi, "Disable nat_bits due to no space"); 1347 - } else if (!is_set_ckpt_flags(sbi, CP_NAT_BITS_FLAG) && 1348 - f2fs_nat_bitmap_enabled(sbi)) { 1349 - f2fs_enable_nat_bits(sbi); 1350 - set_ckpt_flags(sbi, CP_NAT_BITS_FLAG); 1351 - f2fs_notice(sbi, "Rebuild and enable nat_bits"); 1352 - } 1353 - } 1354 - 1355 1347 spin_lock_irqsave(&sbi->cp_lock, flags); 1348 + 1349 + if ((cpc->reason & CP_UMOUNT) && 1350 + le32_to_cpu(ckpt->cp_pack_total_block_count) > 1351 + sbi->blocks_per_seg - NM_I(sbi)->nat_bits_blocks) 1352 + disable_nat_bits(sbi, false); 1356 1353 1357 1354 if (cpc->reason & CP_TRIMMED) 1358 1355 __set_ckpt_flags(ckpt, CP_TRIMMED_FLAG); ··· 1528 1541 start_blk = __start_cp_next_addr(sbi); 1529 1542 1530 1543 /* write nat bits */ 1531 - if ((cpc->reason & CP_UMOUNT) && 1532 - is_set_ckpt_flags(sbi, CP_NAT_BITS_FLAG)) { 1544 + if (enabled_nat_bits(sbi, cpc)) { 1533 1545 __u64 cp_ver = cur_cp_version(ckpt); 1534 1546 block_t blk; 1535 1547 ··· 1853 1867 struct cp_control cpc; 1854 1868 1855 1869 cpc.reason = __get_cp_reason(sbi); 1856 - if (!test_opt(sbi, MERGE_CHECKPOINT) || cpc.reason != CP_SYNC) { 1870 + if (!test_opt(sbi, MERGE_CHECKPOINT) || cpc.reason != CP_SYNC || 1871 + sbi->umount_lock_holder == current) { 1857 1872 int ret; 1858 1873 1859 1874 f2fs_down_write(&sbi->gc_lock);
+1
fs/f2fs/compress.c
··· 1150 1150 f2fs_compress_ctx_add_page(cc, page_folio(page)); 1151 1151 1152 1152 if (!PageUptodate(page)) { 1153 + f2fs_handle_page_eio(sbi, page_folio(page), DATA); 1153 1154 release_and_retry: 1154 1155 f2fs_put_rpages(cc); 1155 1156 f2fs_unlock_rpages(cc, i + 1);
+91 -99
fs/f2fs/data.c
··· 319 319 static void f2fs_write_end_io(struct bio *bio) 320 320 { 321 321 struct f2fs_sb_info *sbi; 322 - struct bio_vec *bvec; 323 - struct bvec_iter_all iter_all; 322 + struct folio_iter fi; 324 323 325 324 iostat_update_and_unbind_ctx(bio); 326 325 sbi = bio->bi_private; ··· 327 328 if (time_to_inject(sbi, FAULT_WRITE_IO)) 328 329 bio->bi_status = BLK_STS_IOERR; 329 330 330 - bio_for_each_segment_all(bvec, bio, iter_all) { 331 - struct page *page = bvec->bv_page; 332 - enum count_type type = WB_DATA_TYPE(page, false); 331 + bio_for_each_folio_all(fi, bio) { 332 + struct folio *folio = fi.folio; 333 + enum count_type type; 333 334 334 - fscrypt_finalize_bounce_page(&page); 335 + if (fscrypt_is_bounce_folio(folio)) { 336 + struct folio *io_folio = folio; 337 + 338 + folio = fscrypt_pagecache_folio(io_folio); 339 + fscrypt_free_bounce_page(&io_folio->page); 340 + } 335 341 336 342 #ifdef CONFIG_F2FS_FS_COMPRESSION 337 - if (f2fs_is_compressed_page(page)) { 338 - f2fs_compress_write_end_io(bio, page); 343 + if (f2fs_is_compressed_page(&folio->page)) { 344 + f2fs_compress_write_end_io(bio, &folio->page); 339 345 continue; 340 346 } 341 347 #endif 342 348 349 + type = WB_DATA_TYPE(&folio->page, false); 350 + 343 351 if (unlikely(bio->bi_status)) { 344 - mapping_set_error(page->mapping, -EIO); 352 + mapping_set_error(folio->mapping, -EIO); 345 353 if (type == F2FS_WB_CP_DATA) 346 354 f2fs_stop_checkpoint(sbi, true, 347 355 STOP_CP_REASON_WRITE_FAIL); 348 356 } 349 357 350 - f2fs_bug_on(sbi, page->mapping == NODE_MAPPING(sbi) && 351 - page_folio(page)->index != nid_of_node(page)); 358 + f2fs_bug_on(sbi, folio->mapping == NODE_MAPPING(sbi) && 359 + folio->index != nid_of_node(&folio->page)); 352 360 353 361 dec_page_count(sbi, type); 354 - if (f2fs_in_warm_node_list(sbi, page)) 355 - f2fs_del_fsync_node_entry(sbi, page); 356 - clear_page_private_gcing(page); 357 - end_page_writeback(page); 362 + if (f2fs_in_warm_node_list(sbi, folio)) 363 + f2fs_del_fsync_node_entry(sbi, &folio->page); 364 + clear_page_private_gcing(&folio->page); 365 + folio_end_writeback(folio); 358 366 } 359 367 if (!get_pages(sbi, F2FS_WB_CP_DATA) && 360 368 wq_has_sleeper(&sbi->cp_wait)) ··· 419 413 static blk_opf_t f2fs_io_flags(struct f2fs_io_info *fio) 420 414 { 421 415 unsigned int temp_mask = GENMASK(NR_TEMP_TYPE - 1, 0); 416 + struct folio *fio_folio = page_folio(fio->page); 422 417 unsigned int fua_flag, meta_flag, io_flag; 423 418 blk_opf_t op_flags = 0; 424 419 ··· 445 438 op_flags |= REQ_META; 446 439 if (BIT(fio->temp) & fua_flag) 447 440 op_flags |= REQ_FUA; 441 + 442 + if (fio->type == DATA && 443 + F2FS_I(fio_folio->mapping->host)->ioprio_hint == F2FS_IOPRIO_WRITE) 444 + op_flags |= REQ_PRIO; 445 + 448 446 return op_flags; 449 447 } 450 448 ··· 888 876 struct bio *bio = *fio->bio; 889 877 struct page *page = fio->encrypted_page ? 890 878 fio->encrypted_page : fio->page; 879 + struct folio *folio = page_folio(fio->page); 891 880 892 881 if (!f2fs_is_valid_blkaddr(fio->sbi, fio->new_blkaddr, 893 882 __is_meta_io(fio) ? META_GENERIC : DATA_GENERIC)) ··· 902 889 alloc_new: 903 890 if (!bio) { 904 891 bio = __bio_alloc(fio, BIO_MAX_VECS); 905 - f2fs_set_bio_crypt_ctx(bio, fio->page->mapping->host, 906 - page_folio(fio->page)->index, fio, GFP_NOIO); 892 + f2fs_set_bio_crypt_ctx(bio, folio->mapping->host, 893 + folio->index, fio, GFP_NOIO); 907 894 908 895 add_bio_entry(fio->sbi, bio, page, fio->temp); 909 896 } else { ··· 912 899 } 913 900 914 901 if (fio->io_wbc) 915 - wbc_account_cgroup_owner(fio->io_wbc, page_folio(fio->page), 916 - PAGE_SIZE); 902 + wbc_account_cgroup_owner(fio->io_wbc, folio, folio_size(folio)); 917 903 918 904 inc_page_count(fio->sbi, WB_DATA_TYPE(page, false)); 919 905 ··· 1053 1041 bio = bio_alloc_bioset(bdev, bio_max_segs(nr_pages), 1054 1042 REQ_OP_READ | op_flag, 1055 1043 for_write ? GFP_NOIO : GFP_KERNEL, &f2fs_bioset); 1056 - if (!bio) 1057 - return ERR_PTR(-ENOMEM); 1058 1044 bio->bi_iter.bi_sector = sector; 1059 1045 f2fs_set_bio_crypt_ctx(bio, inode, first_idx, NULL, GFP_NOFS); 1060 1046 bio->bi_end_io = f2fs_read_end_io; ··· 1203 1193 return err; 1204 1194 } 1205 1195 1206 - struct page *f2fs_get_read_data_page(struct inode *inode, pgoff_t index, 1207 - blk_opf_t op_flags, bool for_write, 1208 - pgoff_t *next_pgofs) 1196 + struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t index, 1197 + blk_opf_t op_flags, bool for_write, pgoff_t *next_pgofs) 1209 1198 { 1210 1199 struct address_space *mapping = inode->i_mapping; 1211 1200 struct dnode_of_data dn; 1212 - struct page *page; 1201 + struct folio *folio; 1213 1202 int err; 1214 1203 1215 - page = f2fs_grab_cache_page(mapping, index, for_write); 1216 - if (!page) 1217 - return ERR_PTR(-ENOMEM); 1204 + folio = f2fs_grab_cache_folio(mapping, index, for_write); 1205 + if (IS_ERR(folio)) 1206 + return folio; 1218 1207 1219 1208 if (f2fs_lookup_read_extent_cache_block(inode, index, 1220 1209 &dn.data_blkaddr)) { ··· 1248 1239 goto put_err; 1249 1240 } 1250 1241 got_it: 1251 - if (PageUptodate(page)) { 1252 - unlock_page(page); 1253 - return page; 1242 + if (folio_test_uptodate(folio)) { 1243 + folio_unlock(folio); 1244 + return folio; 1254 1245 } 1255 1246 1256 1247 /* ··· 1261 1252 * f2fs_init_inode_metadata. 1262 1253 */ 1263 1254 if (dn.data_blkaddr == NEW_ADDR) { 1264 - zero_user_segment(page, 0, PAGE_SIZE); 1265 - if (!PageUptodate(page)) 1266 - SetPageUptodate(page); 1267 - unlock_page(page); 1268 - return page; 1255 + folio_zero_segment(folio, 0, folio_size(folio)); 1256 + if (!folio_test_uptodate(folio)) 1257 + folio_mark_uptodate(folio); 1258 + folio_unlock(folio); 1259 + return folio; 1269 1260 } 1270 1261 1271 - err = f2fs_submit_page_read(inode, page_folio(page), dn.data_blkaddr, 1262 + err = f2fs_submit_page_read(inode, folio, dn.data_blkaddr, 1272 1263 op_flags, for_write); 1273 1264 if (err) 1274 1265 goto put_err; 1275 - return page; 1266 + return folio; 1276 1267 1277 1268 put_err: 1278 - f2fs_put_page(page, 1); 1269 + f2fs_folio_put(folio, true); 1279 1270 return ERR_PTR(err); 1280 1271 } 1281 1272 1282 - struct page *f2fs_find_data_page(struct inode *inode, pgoff_t index, 1273 + struct folio *f2fs_find_data_folio(struct inode *inode, pgoff_t index, 1283 1274 pgoff_t *next_pgofs) 1284 1275 { 1285 1276 struct address_space *mapping = inode->i_mapping; 1286 - struct page *page; 1277 + struct folio *folio; 1287 1278 1288 - page = find_get_page_flags(mapping, index, FGP_ACCESSED); 1289 - if (page && PageUptodate(page)) 1290 - return page; 1291 - f2fs_put_page(page, 0); 1279 + folio = __filemap_get_folio(mapping, index, FGP_ACCESSED, 0); 1280 + if (IS_ERR(folio)) 1281 + goto read; 1282 + if (folio_test_uptodate(folio)) 1283 + return folio; 1284 + f2fs_folio_put(folio, false); 1292 1285 1293 - page = f2fs_get_read_data_page(inode, index, 0, false, next_pgofs); 1294 - if (IS_ERR(page)) 1295 - return page; 1286 + read: 1287 + folio = f2fs_get_read_data_folio(inode, index, 0, false, next_pgofs); 1288 + if (IS_ERR(folio)) 1289 + return folio; 1296 1290 1297 - if (PageUptodate(page)) 1298 - return page; 1291 + if (folio_test_uptodate(folio)) 1292 + return folio; 1299 1293 1300 - wait_on_page_locked(page); 1301 - if (unlikely(!PageUptodate(page))) { 1302 - f2fs_put_page(page, 0); 1294 + folio_wait_locked(folio); 1295 + if (unlikely(!folio_test_uptodate(folio))) { 1296 + f2fs_folio_put(folio, false); 1303 1297 return ERR_PTR(-EIO); 1304 1298 } 1305 - return page; 1299 + return folio; 1306 1300 } 1307 1301 1308 1302 /* ··· 1313 1301 * Because, the callers, functions in dir.c and GC, should be able to know 1314 1302 * whether this page exists or not. 1315 1303 */ 1316 - struct page *f2fs_get_lock_data_page(struct inode *inode, pgoff_t index, 1304 + struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t index, 1317 1305 bool for_write) 1318 1306 { 1319 1307 struct address_space *mapping = inode->i_mapping; 1320 - struct page *page; 1308 + struct folio *folio; 1321 1309 1322 - page = f2fs_get_read_data_page(inode, index, 0, for_write, NULL); 1323 - if (IS_ERR(page)) 1324 - return page; 1310 + folio = f2fs_get_read_data_folio(inode, index, 0, for_write, NULL); 1311 + if (IS_ERR(folio)) 1312 + return folio; 1325 1313 1326 1314 /* wait for read completion */ 1327 - lock_page(page); 1328 - if (unlikely(page->mapping != mapping || !PageUptodate(page))) { 1329 - f2fs_put_page(page, 1); 1315 + folio_lock(folio); 1316 + if (unlikely(folio->mapping != mapping || !folio_test_uptodate(folio))) { 1317 + f2fs_folio_put(folio, true); 1330 1318 return ERR_PTR(-EIO); 1331 1319 } 1332 - return page; 1320 + return folio; 1333 1321 } 1334 1322 1335 1323 /* ··· 2190 2178 int i; 2191 2179 int ret = 0; 2192 2180 2181 + if (unlikely(f2fs_cp_error(sbi))) { 2182 + ret = -EIO; 2183 + from_dnode = false; 2184 + goto out_put_dnode; 2185 + } 2186 + 2193 2187 f2fs_bug_on(sbi, f2fs_cluster_is_empty(cc)); 2194 2188 2195 2189 last_block_in_file = F2FS_BYTES_TO_BLK(f2fs_readpage_limit(inode) + ··· 2239 2221 if (ret) 2240 2222 goto out; 2241 2223 2242 - if (unlikely(f2fs_cp_error(sbi))) { 2243 - ret = -EIO; 2244 - goto out_put_dnode; 2245 - } 2246 2224 f2fs_bug_on(sbi, dn.data_blkaddr != COMPRESS_ADDR); 2247 2225 2248 2226 skip_reading_dnode: ··· 2935 2921 return err; 2936 2922 } 2937 2923 2938 - static int f2fs_write_data_page(struct page *page, 2939 - struct writeback_control *wbc) 2940 - { 2941 - struct folio *folio = page_folio(page); 2942 - #ifdef CONFIG_F2FS_FS_COMPRESSION 2943 - struct inode *inode = folio->mapping->host; 2944 - 2945 - if (unlikely(f2fs_cp_error(F2FS_I_SB(inode)))) 2946 - goto out; 2947 - 2948 - if (f2fs_compressed_file(inode)) { 2949 - if (f2fs_is_compressed_cluster(inode, folio->index)) { 2950 - folio_redirty_for_writepage(wbc, folio); 2951 - return AOP_WRITEPAGE_ACTIVATE; 2952 - } 2953 - } 2954 - out: 2955 - #endif 2956 - 2957 - return f2fs_write_single_data_page(folio, NULL, NULL, NULL, 2958 - wbc, FS_DATA_IO, 0, true); 2959 - } 2960 - 2961 2924 /* 2962 2925 * This function was copied from write_cache_pages from mm/page-writeback.c. 2963 2926 * The major change is making write step of cold data page separately from ··· 3257 3266 int ret; 3258 3267 bool locked = false; 3259 3268 3260 - /* deal with chardevs and other special file */ 3261 - if (!mapping->a_ops->writepage) 3262 - return 0; 3263 - 3264 3269 /* skip writing if there is no dirty page in this inode */ 3265 3270 if (!get_dirty_pages(inode) && wbc->sync_mode == WB_SYNC_NONE) 3266 3271 return 0; ··· 3377 3390 3378 3391 restart: 3379 3392 /* check inline_data */ 3380 - ipage = f2fs_get_node_page(sbi, inode->i_ino); 3393 + ipage = f2fs_get_inode_page(sbi, inode->i_ino); 3381 3394 if (IS_ERR(ipage)) { 3382 3395 err = PTR_ERR(ipage); 3383 3396 goto unlock_out; ··· 3440 3453 struct page *ipage; 3441 3454 int err = 0; 3442 3455 3443 - ipage = f2fs_get_node_page(F2FS_I_SB(inode), inode->i_ino); 3456 + ipage = f2fs_get_inode_page(F2FS_I_SB(inode), inode->i_ino); 3444 3457 if (IS_ERR(ipage)) 3445 3458 return PTR_ERR(ipage); 3446 3459 ··· 3470 3483 3471 3484 f2fs_map_lock(sbi, F2FS_GET_BLOCK_PRE_AIO); 3472 3485 3473 - ipage = f2fs_get_node_page(sbi, inode->i_ino); 3486 + ipage = f2fs_get_inode_page(sbi, inode->i_ino); 3474 3487 if (IS_ERR(ipage)) { 3475 3488 err = PTR_ERR(ipage); 3476 3489 goto unlock_out; ··· 4088 4101 const struct address_space_operations f2fs_dblock_aops = { 4089 4102 .read_folio = f2fs_read_data_folio, 4090 4103 .readahead = f2fs_readahead, 4091 - .writepage = f2fs_write_data_page, 4092 4104 .writepages = f2fs_write_data_pages, 4093 4105 .write_begin = f2fs_write_begin, 4094 4106 .write_end = f2fs_write_end, ··· 4181 4195 map.m_next_pgofs = &next_pgofs; 4182 4196 map.m_seg_type = f2fs_rw_hint_to_seg_type(F2FS_I_SB(inode), 4183 4197 inode->i_write_hint); 4184 - if (flags & IOMAP_WRITE) 4198 + 4199 + /* 4200 + * If the blocks being overwritten are already allocated, 4201 + * f2fs_map_lock and f2fs_balance_fs are not necessary. 4202 + */ 4203 + if ((flags & IOMAP_WRITE) && 4204 + !f2fs_overwrite_io(inode, offset, length)) 4185 4205 map.m_may_create = true; 4186 4206 4187 4207 err = f2fs_map_blocks(inode, &map, F2FS_GET_BLOCK_DIO);
+3
fs/f2fs/debug.c
··· 164 164 si->ndirty_imeta = get_pages(sbi, F2FS_DIRTY_IMETA); 165 165 si->ndirty_dirs = sbi->ndirty_inode[DIR_INODE]; 166 166 si->ndirty_files = sbi->ndirty_inode[FILE_INODE]; 167 + si->ndonate_files = sbi->donate_files; 167 168 si->nquota_files = sbi->nquota_files; 168 169 si->ndirty_all = sbi->ndirty_inode[DIRTY_META]; 169 170 si->aw_cnt = atomic_read(&sbi->atomic_files); ··· 502 501 si->compr_inode, si->compr_blocks); 503 502 seq_printf(s, " - Swapfile Inode: %u\n", 504 503 si->swapfile_inode); 504 + seq_printf(s, " - Donate Inode: %u\n", 505 + si->ndonate_files); 505 506 seq_printf(s, " - Orphan/Append/Update Inode: %u, %u, %u\n", 506 507 si->orphans, si->append, si->update); 507 508 seq_printf(s, "\nMain area: %d segs, %d secs %d zones\n",
+1 -1
fs/f2fs/dir.c
··· 551 551 goto put_error; 552 552 } 553 553 } else { 554 - page = f2fs_get_node_page(F2FS_I_SB(dir), inode->i_ino); 554 + page = f2fs_get_inode_page(F2FS_I_SB(dir), inode->i_ino); 555 555 if (IS_ERR(page)) 556 556 return page; 557 557 }
+126 -31
fs/f2fs/f2fs.h
··· 62 62 FAULT_BLKADDR_VALIDITY, 63 63 FAULT_BLKADDR_CONSISTENCE, 64 64 FAULT_NO_SEGMENT, 65 + FAULT_INCONSISTENT_FOOTER, 65 66 FAULT_MAX, 66 67 }; 67 68 ··· 115 114 #define F2FS_MOUNT_GC_MERGE 0x02000000 116 115 #define F2FS_MOUNT_COMPRESS_CACHE 0x04000000 117 116 #define F2FS_MOUNT_AGE_EXTENT_CACHE 0x08000000 117 + #define F2FS_MOUNT_NAT_BITS 0x10000000 118 + #define F2FS_MOUNT_INLINECRYPT 0x20000000 119 + /* 120 + * Some f2fs environments expect to be able to pass the "lazytime" option 121 + * string rather than using the MS_LAZYTIME flag, so this must remain. 122 + */ 123 + #define F2FS_MOUNT_LAZYTIME 0x40000000 118 124 119 125 #define F2FS_OPTION(sbi) ((sbi)->mount_opt) 120 126 #define clear_opt(sbi, option) (F2FS_OPTION(sbi).opt &= ~F2FS_MOUNT_##option) ··· 838 830 839 831 /* Use below internally in f2fs*/ 840 832 unsigned long flags[BITS_TO_LONGS(FI_MAX)]; /* use to pass per-file flags */ 833 + unsigned int ioprio_hint; /* hint for IO priority */ 841 834 struct f2fs_rwsem i_sem; /* protect fi info */ 842 835 atomic_t dirty_pages; /* # of dirty pages */ 843 836 f2fs_hash_t chash; /* hash value of given file name */ ··· 858 849 #endif 859 850 struct list_head dirty_list; /* dirty list for dirs and files */ 860 851 struct list_head gdirty_list; /* linked in global dirty list */ 852 + 853 + /* linked in global inode list for cache donation */ 854 + struct list_head gdonate_list; 855 + pgoff_t donate_start, donate_end; /* inclusive */ 856 + 861 857 struct task_struct *atomic_write_task; /* store atomic write task */ 862 858 struct extent_tree *extent_tree[NR_EXTENT_CACHES]; 863 859 /* cached extent_tree entry */ ··· 1287 1273 DIR_INODE, /* for dirty dir inode */ 1288 1274 FILE_INODE, /* for dirty regular/symlink inode */ 1289 1275 DIRTY_META, /* for all dirtied inode metadata */ 1276 + DONATE_INODE, /* for all inode to donate pages */ 1290 1277 NR_INODE_TYPE, 1291 1278 }; 1292 1279 ··· 1643 1628 unsigned int warm_data_age_threshold; 1644 1629 unsigned int last_age_weight; 1645 1630 1631 + /* control donate caches */ 1632 + unsigned int donate_files; 1633 + 1646 1634 /* basic filesystem units */ 1647 1635 unsigned int log_sectors_per_block; /* log2 sectors per block */ 1648 1636 unsigned int log_blocksize; /* log2 block size */ ··· 1677 1659 1678 1660 unsigned int nquota_files; /* # of quota sysfile */ 1679 1661 struct f2fs_rwsem quota_sem; /* blocking cp for flags */ 1662 + struct task_struct *umount_lock_holder; /* s_umount lock holder */ 1680 1663 1681 1664 /* # of pages, see count_type */ 1682 1665 atomic_t nr_pages[NR_COUNT_TYPE]; ··· 1818 1799 s64 peak_atomic_write; 1819 1800 u64 committed_atomic_block; 1820 1801 u64 revoked_atomic_block; 1802 + 1803 + /* carve out reserved_blocks from total blocks */ 1804 + bool carve_out; 1821 1805 1822 1806 #ifdef CONFIG_F2FS_FS_COMPRESSION 1823 1807 struct kmem_cache *page_array_slab; /* page array entry */ ··· 2037 2015 return (struct f2fs_checkpoint *)(sbi->ckpt); 2038 2016 } 2039 2017 2040 - static inline struct f2fs_node *F2FS_NODE(struct page *page) 2018 + static inline struct f2fs_node *F2FS_NODE(const struct page *page) 2041 2019 { 2042 2020 return (struct f2fs_node *)page_address(page); 2043 2021 } ··· 2239 2217 #ifdef CONFIG_F2FS_UNFAIR_RWSEM 2240 2218 wake_up_all(&sem->read_waiters); 2241 2219 #endif 2220 + } 2221 + 2222 + static inline void disable_nat_bits(struct f2fs_sb_info *sbi, bool lock) 2223 + { 2224 + unsigned long flags; 2225 + unsigned char *nat_bits; 2226 + 2227 + /* 2228 + * In order to re-enable nat_bits we need to call fsck.f2fs by 2229 + * set_sbi_flag(sbi, SBI_NEED_FSCK). But it may give huge cost, 2230 + * so let's rely on regular fsck or unclean shutdown. 2231 + */ 2232 + 2233 + if (lock) 2234 + spin_lock_irqsave(&sbi->cp_lock, flags); 2235 + __clear_ckpt_flags(F2FS_CKPT(sbi), CP_NAT_BITS_FLAG); 2236 + nat_bits = NM_I(sbi)->nat_bits; 2237 + NM_I(sbi)->nat_bits = NULL; 2238 + if (lock) 2239 + spin_unlock_irqrestore(&sbi->cp_lock, flags); 2240 + 2241 + kvfree(nat_bits); 2242 + } 2243 + 2244 + static inline bool enabled_nat_bits(struct f2fs_sb_info *sbi, 2245 + struct cp_control *cpc) 2246 + { 2247 + bool set = is_set_ckpt_flags(sbi, CP_NAT_BITS_FLAG); 2248 + 2249 + return (cpc) ? (cpc->reason & CP_UMOUNT) && set : set; 2242 2250 } 2243 2251 2244 2252 static inline void f2fs_lock_op(struct f2fs_sb_info *sbi) ··· 2817 2765 return percpu_counter_sum_positive(&sbi->total_valid_inode_count); 2818 2766 } 2819 2767 2820 - static inline struct page *f2fs_grab_cache_page(struct address_space *mapping, 2821 - pgoff_t index, bool for_write) 2768 + static inline struct folio *f2fs_grab_cache_folio(struct address_space *mapping, 2769 + pgoff_t index, bool for_write) 2822 2770 { 2823 - struct page *page; 2771 + struct folio *folio; 2824 2772 unsigned int flags; 2825 2773 2826 2774 if (IS_ENABLED(CONFIG_F2FS_FAULT_INJECTION)) { 2775 + fgf_t fgf_flags; 2776 + 2827 2777 if (!for_write) 2828 - page = find_get_page_flags(mapping, index, 2829 - FGP_LOCK | FGP_ACCESSED); 2778 + fgf_flags = FGP_LOCK | FGP_ACCESSED; 2830 2779 else 2831 - page = find_lock_page(mapping, index); 2832 - if (page) 2833 - return page; 2780 + fgf_flags = FGP_LOCK; 2781 + folio = __filemap_get_folio(mapping, index, fgf_flags, 0); 2782 + if (!IS_ERR(folio)) 2783 + return folio; 2834 2784 2835 2785 if (time_to_inject(F2FS_M_SB(mapping), FAULT_PAGE_ALLOC)) 2836 - return NULL; 2786 + return ERR_PTR(-ENOMEM); 2837 2787 } 2838 2788 2839 2789 if (!for_write) 2840 - return grab_cache_page(mapping, index); 2790 + return filemap_grab_folio(mapping, index); 2841 2791 2842 2792 flags = memalloc_nofs_save(); 2843 - page = grab_cache_page_write_begin(mapping, index); 2793 + folio = __filemap_get_folio(mapping, index, FGP_WRITEBEGIN, 2794 + mapping_gfp_mask(mapping)); 2844 2795 memalloc_nofs_restore(flags); 2845 2796 2846 - return page; 2797 + return folio; 2798 + } 2799 + 2800 + static inline struct page *f2fs_grab_cache_page(struct address_space *mapping, 2801 + pgoff_t index, bool for_write) 2802 + { 2803 + struct folio *folio = f2fs_grab_cache_folio(mapping, index, for_write); 2804 + 2805 + if (IS_ERR(folio)) 2806 + return NULL; 2807 + return &folio->page; 2847 2808 } 2848 2809 2849 2810 static inline struct page *f2fs_pagecache_get_page( ··· 2869 2804 return pagecache_get_page(mapping, index, fgp_flags, gfp_mask); 2870 2805 } 2871 2806 2807 + static inline void f2fs_folio_put(struct folio *folio, bool unlock) 2808 + { 2809 + if (!folio) 2810 + return; 2811 + 2812 + if (unlock) { 2813 + f2fs_bug_on(F2FS_F_SB(folio), !folio_test_locked(folio)); 2814 + folio_unlock(folio); 2815 + } 2816 + folio_put(folio); 2817 + } 2818 + 2872 2819 static inline void f2fs_put_page(struct page *page, int unlock) 2873 2820 { 2874 2821 if (!page) 2875 2822 return; 2876 - 2877 - if (unlock) { 2878 - f2fs_bug_on(F2FS_P_SB(page), !PageLocked(page)); 2879 - unlock_page(page); 2880 - } 2881 - put_page(page); 2823 + f2fs_folio_put(page_folio(page), unlock); 2882 2824 } 2883 2825 2884 2826 static inline void f2fs_put_dnode(struct dnode_of_data *dn) ··· 3696 3624 void f2fs_inode_synced(struct inode *inode); 3697 3625 int f2fs_dquot_initialize(struct inode *inode); 3698 3626 int f2fs_enable_quota_files(struct f2fs_sb_info *sbi, bool rdonly); 3699 - int f2fs_quota_sync(struct super_block *sb, int type); 3627 + int f2fs_do_quota_sync(struct super_block *sb, int type); 3700 3628 loff_t max_file_blocks(struct inode *inode); 3701 3629 void f2fs_quota_off_umount(struct super_block *sb); 3702 3630 void f2fs_save_errors(struct f2fs_sb_info *sbi, unsigned char flag); ··· 3719 3647 3720 3648 int f2fs_check_nid_range(struct f2fs_sb_info *sbi, nid_t nid); 3721 3649 bool f2fs_available_free_memory(struct f2fs_sb_info *sbi, int type); 3722 - bool f2fs_in_warm_node_list(struct f2fs_sb_info *sbi, struct page *page); 3650 + bool f2fs_in_warm_node_list(struct f2fs_sb_info *sbi, 3651 + const struct folio *folio); 3723 3652 void f2fs_init_fsync_node_info(struct f2fs_sb_info *sbi); 3724 3653 void f2fs_del_fsync_node_entry(struct f2fs_sb_info *sbi, struct page *page); 3725 3654 void f2fs_reset_fsync_node_info(struct f2fs_sb_info *sbi); ··· 3735 3662 int f2fs_truncate_xattr_node(struct inode *inode); 3736 3663 int f2fs_wait_on_node_pages_writeback(struct f2fs_sb_info *sbi, 3737 3664 unsigned int seq_id); 3738 - bool f2fs_nat_bitmap_enabled(struct f2fs_sb_info *sbi); 3739 3665 int f2fs_remove_inode_page(struct inode *inode); 3740 3666 struct page *f2fs_new_inode_page(struct inode *inode); 3741 3667 struct page *f2fs_new_node_page(struct dnode_of_data *dn, unsigned int ofs); 3742 3668 void f2fs_ra_node_page(struct f2fs_sb_info *sbi, nid_t nid); 3743 3669 struct page *f2fs_get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid); 3670 + struct folio *f2fs_get_inode_folio(struct f2fs_sb_info *sbi, pgoff_t ino); 3671 + struct page *f2fs_get_inode_page(struct f2fs_sb_info *sbi, pgoff_t ino); 3672 + struct page *f2fs_get_xnode_page(struct f2fs_sb_info *sbi, pgoff_t xnid); 3744 3673 struct page *f2fs_get_node_page_ra(struct page *parent, int start); 3745 3674 int f2fs_move_node_page(struct page *node_page, int gc_type); 3746 3675 void f2fs_flush_inline_data(struct f2fs_sb_info *sbi); ··· 3762 3687 int f2fs_recover_inode_page(struct f2fs_sb_info *sbi, struct page *page); 3763 3688 int f2fs_restore_node_summary(struct f2fs_sb_info *sbi, 3764 3689 unsigned int segno, struct f2fs_summary_block *sum); 3765 - void f2fs_enable_nat_bits(struct f2fs_sb_info *sbi); 3766 3690 int f2fs_flush_nat_entries(struct f2fs_sb_info *sbi, struct cp_control *cpc); 3767 3691 int f2fs_build_node_manager(struct f2fs_sb_info *sbi); 3768 3692 void f2fs_destroy_node_manager(struct f2fs_sb_info *sbi); ··· 3832 3758 struct f2fs_io_info *fio); 3833 3759 void f2fs_update_device_state(struct f2fs_sb_info *sbi, nid_t ino, 3834 3760 block_t blkaddr, unsigned int blkcnt); 3835 - void f2fs_wait_on_page_writeback(struct page *page, 3836 - enum page_type type, bool ordered, bool locked); 3761 + void f2fs_folio_wait_writeback(struct folio *folio, enum page_type type, 3762 + bool ordered, bool locked); 3763 + #define f2fs_wait_on_page_writeback(page, type, ordered, locked) \ 3764 + f2fs_folio_wait_writeback(page_folio(page), type, ordered, locked) 3837 3765 void f2fs_wait_on_block_writeback(struct inode *inode, block_t blkaddr); 3838 3766 void f2fs_wait_on_block_writeback_range(struct inode *inode, block_t blkaddr, 3839 3767 block_t len); ··· 3947 3871 int f2fs_reserve_new_block(struct dnode_of_data *dn); 3948 3872 int f2fs_get_block_locked(struct dnode_of_data *dn, pgoff_t index); 3949 3873 int f2fs_reserve_block(struct dnode_of_data *dn, pgoff_t index); 3950 - struct page *f2fs_get_read_data_page(struct inode *inode, pgoff_t index, 3951 - blk_opf_t op_flags, bool for_write, pgoff_t *next_pgofs); 3952 - struct page *f2fs_find_data_page(struct inode *inode, pgoff_t index, 3953 - pgoff_t *next_pgofs); 3954 - struct page *f2fs_get_lock_data_page(struct inode *inode, pgoff_t index, 3874 + struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t index, 3875 + blk_opf_t op_flags, bool for_write, pgoff_t *next_pgofs); 3876 + struct folio *f2fs_find_data_folio(struct inode *inode, pgoff_t index, 3877 + pgoff_t *next_pgofs); 3878 + struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t index, 3955 3879 bool for_write); 3956 3880 struct page *f2fs_get_new_data_page(struct inode *inode, 3957 3881 struct page *ipage, pgoff_t index, bool new_i_size); ··· 3977 3901 int f2fs_init_post_read_wq(struct f2fs_sb_info *sbi); 3978 3902 void f2fs_destroy_post_read_wq(struct f2fs_sb_info *sbi); 3979 3903 extern const struct iomap_ops f2fs_iomap_ops; 3904 + 3905 + static inline struct page *f2fs_find_data_page(struct inode *inode, 3906 + pgoff_t index, pgoff_t *next_pgofs) 3907 + { 3908 + struct folio *folio = f2fs_find_data_folio(inode, index, next_pgofs); 3909 + 3910 + return &folio->page; 3911 + } 3912 + 3913 + static inline struct page *f2fs_get_lock_data_page(struct inode *inode, 3914 + pgoff_t index, bool for_write) 3915 + { 3916 + struct folio *folio = f2fs_get_lock_data_folio(inode, index, for_write); 3917 + 3918 + return &folio->page; 3919 + } 3980 3920 3981 3921 /* 3982 3922 * gc.c ··· 4058 3966 unsigned long long allocated_data_blocks; 4059 3967 int ndirty_node, ndirty_dent, ndirty_meta, ndirty_imeta; 4060 3968 int ndirty_data, ndirty_qdata; 4061 - unsigned int ndirty_dirs, ndirty_files, nquota_files, ndirty_all; 3969 + unsigned int ndirty_dirs, ndirty_files, ndirty_all; 3970 + unsigned int nquota_files, ndonate_files; 4062 3971 int nats, dirty_nats, sits, dirty_sits; 4063 3972 int free_nids, avail_nids, alloc_nids; 4064 3973 int total_count, utilization; ··· 4324 4231 struct shrink_control *sc); 4325 4232 unsigned long f2fs_shrink_scan(struct shrinker *shrink, 4326 4233 struct shrink_control *sc); 4234 + unsigned int f2fs_donate_files(void); 4235 + void f2fs_reclaim_caches(unsigned int reclaim_caches_kb); 4327 4236 void f2fs_join_shrinker(struct f2fs_sb_info *sbi); 4328 4237 void f2fs_leave_shrinker(struct f2fs_sb_info *sbi); 4329 4238
+106 -20
fs/f2fs/file.c
··· 707 707 loff_t offset = from & (PAGE_SIZE - 1); 708 708 pgoff_t index = from >> PAGE_SHIFT; 709 709 struct address_space *mapping = inode->i_mapping; 710 - struct page *page; 710 + struct folio *folio; 711 711 712 712 if (!offset && !cache_only) 713 713 return 0; 714 714 715 715 if (cache_only) { 716 - page = find_lock_page(mapping, index); 717 - if (page && PageUptodate(page)) 716 + folio = filemap_lock_folio(mapping, index); 717 + if (IS_ERR(folio)) 718 + return 0; 719 + if (folio_test_uptodate(folio)) 718 720 goto truncate_out; 719 - f2fs_put_page(page, 1); 721 + f2fs_folio_put(folio, true); 720 722 return 0; 721 723 } 722 724 723 - page = f2fs_get_lock_data_page(inode, index, true); 724 - if (IS_ERR(page)) 725 - return PTR_ERR(page) == -ENOENT ? 0 : PTR_ERR(page); 725 + folio = f2fs_get_lock_data_folio(inode, index, true); 726 + if (IS_ERR(folio)) 727 + return PTR_ERR(folio) == -ENOENT ? 0 : PTR_ERR(folio); 726 728 truncate_out: 727 - f2fs_wait_on_page_writeback(page, DATA, true, true); 728 - zero_user(page, offset, PAGE_SIZE - offset); 729 + f2fs_folio_wait_writeback(folio, DATA, true, true); 730 + folio_zero_segment(folio, offset, folio_size(folio)); 729 731 730 732 /* An encrypted inode should have a key and truncate the last page. */ 731 733 f2fs_bug_on(F2FS_I_SB(inode), cache_only && IS_ENCRYPTED(inode)); 732 734 if (!cache_only) 733 - set_page_dirty(page); 734 - f2fs_put_page(page, 1); 735 + folio_mark_dirty(folio); 736 + f2fs_folio_put(folio, true); 735 737 return 0; 736 738 } 737 739 ··· 761 759 if (lock) 762 760 f2fs_lock_op(sbi); 763 761 764 - ipage = f2fs_get_node_page(sbi, inode->i_ino); 762 + ipage = f2fs_get_inode_page(sbi, inode->i_ino); 765 763 if (IS_ERR(ipage)) { 766 764 err = PTR_ERR(ipage); 767 765 goto out; ··· 1836 1834 1837 1835 map.m_len = sec_blks; 1838 1836 next_alloc: 1837 + f2fs_down_write(&sbi->pin_sem); 1838 + 1839 + if (unlikely(is_sbi_flag_set(sbi, SBI_CP_DISABLED))) { 1840 + if (has_not_enough_free_secs(sbi, 0, 0)) { 1841 + f2fs_up_write(&sbi->pin_sem); 1842 + err = -ENOSPC; 1843 + f2fs_warn_ratelimited(sbi, 1844 + "ino:%lu, start:%lu, end:%lu, need to trigger GC to " 1845 + "reclaim enough free segment when checkpoint is enabled", 1846 + inode->i_ino, pg_start, pg_end); 1847 + goto out_err; 1848 + } 1849 + } 1850 + 1839 1851 if (has_not_enough_free_secs(sbi, 0, f2fs_sb_has_blkzoned(sbi) ? 1840 1852 ZONED_PIN_SEC_REQUIRED_COUNT : 1841 1853 GET_SEC_FROM_SEG(sbi, overprovision_segments(sbi)))) { 1842 1854 f2fs_down_write(&sbi->gc_lock); 1843 1855 stat_inc_gc_call_count(sbi, FOREGROUND); 1844 1856 err = f2fs_gc(sbi, &gc_control); 1845 - if (err && err != -ENODATA) 1857 + if (err && err != -ENODATA) { 1858 + f2fs_up_write(&sbi->pin_sem); 1846 1859 goto out_err; 1860 + } 1847 1861 } 1848 - 1849 - f2fs_down_write(&sbi->pin_sem); 1850 1862 1851 1863 err = f2fs_allocate_pinning_section(sbi); 1852 1864 if (err) { ··· 2462 2446 mnt_drop_write_file(filp); 2463 2447 2464 2448 return ret; 2449 + } 2450 + 2451 + static void f2fs_keep_noreuse_range(struct inode *inode, 2452 + loff_t offset, loff_t len) 2453 + { 2454 + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); 2455 + u64 max_bytes = F2FS_BLK_TO_BYTES(max_file_blocks(inode)); 2456 + u64 start, end; 2457 + 2458 + if (!S_ISREG(inode->i_mode)) 2459 + return; 2460 + 2461 + if (offset >= max_bytes || len > max_bytes || 2462 + (offset + len) > max_bytes) 2463 + return; 2464 + 2465 + start = offset >> PAGE_SHIFT; 2466 + end = DIV_ROUND_UP(offset + len, PAGE_SIZE); 2467 + 2468 + inode_lock(inode); 2469 + if (f2fs_is_atomic_file(inode)) { 2470 + inode_unlock(inode); 2471 + return; 2472 + } 2473 + 2474 + spin_lock(&sbi->inode_lock[DONATE_INODE]); 2475 + /* let's remove the range, if len = 0 */ 2476 + if (!len) { 2477 + if (!list_empty(&F2FS_I(inode)->gdonate_list)) { 2478 + list_del_init(&F2FS_I(inode)->gdonate_list); 2479 + sbi->donate_files--; 2480 + } 2481 + } else { 2482 + if (list_empty(&F2FS_I(inode)->gdonate_list)) { 2483 + list_add_tail(&F2FS_I(inode)->gdonate_list, 2484 + &sbi->inode_list[DONATE_INODE]); 2485 + sbi->donate_files++; 2486 + } else { 2487 + list_move_tail(&F2FS_I(inode)->gdonate_list, 2488 + &sbi->inode_list[DONATE_INODE]); 2489 + } 2490 + F2FS_I(inode)->donate_start = start; 2491 + F2FS_I(inode)->donate_end = end - 1; 2492 + } 2493 + spin_unlock(&sbi->inode_lock[DONATE_INODE]); 2494 + inode_unlock(inode); 2465 2495 } 2466 2496 2467 2497 static int f2fs_ioc_fitrim(struct file *filp, unsigned long arg) ··· 3506 3444 { 3507 3445 return put_user(IS_DEVICE_ALIASING(file_inode(filp)) ? 1 : 0, 3508 3446 (u32 __user *)arg); 3447 + } 3448 + 3449 + static int f2fs_ioc_io_prio(struct file *filp, unsigned long arg) 3450 + { 3451 + struct inode *inode = file_inode(filp); 3452 + __u32 level; 3453 + 3454 + if (get_user(level, (__u32 __user *)arg)) 3455 + return -EFAULT; 3456 + 3457 + if (!S_ISREG(inode->i_mode) || level >= F2FS_IOPRIO_MAX) 3458 + return -EINVAL; 3459 + 3460 + inode_lock(inode); 3461 + F2FS_I(inode)->ioprio_hint = level; 3462 + inode_unlock(inode); 3463 + return 0; 3509 3464 } 3510 3465 3511 3466 int f2fs_precache_extents(struct inode *inode) ··· 4626 4547 return f2fs_ioc_compress_file(filp); 4627 4548 case F2FS_IOC_GET_DEV_ALIAS_FILE: 4628 4549 return f2fs_ioc_get_dev_alias_file(filp, arg); 4550 + case F2FS_IOC_IO_PRIO: 4551 + return f2fs_ioc_io_prio(filp, arg); 4629 4552 default: 4630 4553 return -ENOTTY; 4631 4554 } ··· 5228 5147 } 5229 5148 5230 5149 err = generic_fadvise(filp, offset, len, advice); 5231 - if (!err && advice == POSIX_FADV_DONTNEED && 5232 - test_opt(F2FS_I_SB(inode), COMPRESS_CACHE) && 5233 - f2fs_compressed_file(inode)) 5234 - f2fs_invalidate_compress_pages(F2FS_I_SB(inode), inode->i_ino); 5150 + if (err) 5151 + return err; 5235 5152 5236 - return err; 5153 + if (advice == POSIX_FADV_DONTNEED && 5154 + (test_opt(F2FS_I_SB(inode), COMPRESS_CACHE) && 5155 + f2fs_compressed_file(inode))) 5156 + f2fs_invalidate_compress_pages(F2FS_I_SB(inode), inode->i_ino); 5157 + else if (advice == POSIX_FADV_NOREUSE) 5158 + f2fs_keep_noreuse_range(inode, offset, len); 5159 + return 0; 5237 5160 } 5238 5161 5239 5162 #ifdef CONFIG_COMPAT ··· 5346 5261 case F2FS_IOC_DECOMPRESS_FILE: 5347 5262 case F2FS_IOC_COMPRESS_FILE: 5348 5263 case F2FS_IOC_GET_DEV_ALIAS_FILE: 5264 + case F2FS_IOC_IO_PRIO: 5349 5265 break; 5350 5266 default: 5351 5267 return -ENOIOCTLCMD;
+21 -21
fs/f2fs/gc.c
··· 1449 1449 } 1450 1450 1451 1451 static int move_data_page(struct inode *inode, block_t bidx, int gc_type, 1452 - unsigned int segno, int off) 1452 + unsigned int segno, int off) 1453 1453 { 1454 - struct page *page; 1454 + struct folio *folio; 1455 1455 int err = 0; 1456 1456 1457 - page = f2fs_get_lock_data_page(inode, bidx, true); 1458 - if (IS_ERR(page)) 1459 - return PTR_ERR(page); 1457 + folio = f2fs_get_lock_data_folio(inode, bidx, true); 1458 + if (IS_ERR(folio)) 1459 + return PTR_ERR(folio); 1460 1460 1461 1461 if (!check_valid_map(F2FS_I_SB(inode), segno, off)) { 1462 1462 err = -ENOENT; ··· 1468 1468 goto out; 1469 1469 1470 1470 if (gc_type == BG_GC) { 1471 - if (folio_test_writeback(page_folio(page))) { 1471 + if (folio_test_writeback(folio)) { 1472 1472 err = -EAGAIN; 1473 1473 goto out; 1474 1474 } 1475 - set_page_dirty(page); 1476 - set_page_private_gcing(page); 1475 + folio_mark_dirty(folio); 1476 + set_page_private_gcing(&folio->page); 1477 1477 } else { 1478 1478 struct f2fs_io_info fio = { 1479 1479 .sbi = F2FS_I_SB(inode), ··· 1483 1483 .op = REQ_OP_WRITE, 1484 1484 .op_flags = REQ_SYNC, 1485 1485 .old_blkaddr = NULL_ADDR, 1486 - .page = page, 1486 + .page = &folio->page, 1487 1487 .encrypted_page = NULL, 1488 1488 .need_lock = LOCK_REQ, 1489 1489 .io_type = FS_GC_DATA_IO, 1490 1490 }; 1491 - bool is_dirty = PageDirty(page); 1491 + bool is_dirty = folio_test_dirty(folio); 1492 1492 1493 1493 retry: 1494 - f2fs_wait_on_page_writeback(page, DATA, true, true); 1494 + f2fs_folio_wait_writeback(folio, DATA, true, true); 1495 1495 1496 - set_page_dirty(page); 1497 - if (clear_page_dirty_for_io(page)) { 1496 + folio_mark_dirty(folio); 1497 + if (folio_clear_dirty_for_io(folio)) { 1498 1498 inode_dec_dirty_pages(inode); 1499 1499 f2fs_remove_dirty_inode(inode); 1500 1500 } 1501 1501 1502 - set_page_private_gcing(page); 1502 + set_page_private_gcing(&folio->page); 1503 1503 1504 1504 err = f2fs_do_write_data_page(&fio); 1505 1505 if (err) { 1506 - clear_page_private_gcing(page); 1506 + clear_page_private_gcing(&folio->page); 1507 1507 if (err == -ENOMEM) { 1508 1508 memalloc_retry_wait(GFP_NOFS); 1509 1509 goto retry; 1510 1510 } 1511 1511 if (is_dirty) 1512 - set_page_dirty(page); 1512 + folio_mark_dirty(folio); 1513 1513 } 1514 1514 } 1515 1515 out: 1516 - f2fs_put_page(page, 1); 1516 + f2fs_folio_put(folio, true); 1517 1517 return err; 1518 1518 } 1519 1519 ··· 1542 1542 entry = sum; 1543 1543 1544 1544 for (off = 0; off < usable_blks_in_seg; off++, entry++) { 1545 - struct page *data_page; 1546 1545 struct inode *inode; 1547 1546 struct node_info dni; /* dnode info for the data */ 1548 1547 unsigned int ofs_in_node, nofs; ··· 1584 1585 ofs_in_node = le16_to_cpu(entry->ofs_in_node); 1585 1586 1586 1587 if (phase == 3) { 1588 + struct folio *data_folio; 1587 1589 int err; 1588 1590 1589 1591 inode = f2fs_iget(sb, dni.ino); ··· 1635 1635 continue; 1636 1636 } 1637 1637 1638 - data_page = f2fs_get_read_data_page(inode, start_bidx, 1638 + data_folio = f2fs_get_read_data_folio(inode, start_bidx, 1639 1639 REQ_RAHEAD, true, NULL); 1640 1640 f2fs_up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]); 1641 - if (IS_ERR(data_page)) { 1641 + if (IS_ERR(data_folio)) { 1642 1642 iput(inode); 1643 1643 continue; 1644 1644 } 1645 1645 1646 - f2fs_put_page(data_page, 0); 1646 + f2fs_folio_put(data_folio, false); 1647 1647 add_gc_inode(gc_list, inode); 1648 1648 continue; 1649 1649 }
+11 -11
fs/f2fs/inline.c
··· 119 119 { 120 120 struct page *ipage; 121 121 122 - ipage = f2fs_get_node_page(F2FS_I_SB(inode), inode->i_ino); 122 + ipage = f2fs_get_inode_page(F2FS_I_SB(inode), inode->i_ino); 123 123 if (IS_ERR(ipage)) { 124 124 folio_unlock(folio); 125 125 return PTR_ERR(ipage); ··· 237 237 238 238 f2fs_lock_op(sbi); 239 239 240 - ipage = f2fs_get_node_page(sbi, inode->i_ino); 240 + ipage = f2fs_get_inode_page(sbi, inode->i_ino); 241 241 if (IS_ERR(ipage)) { 242 242 err = PTR_ERR(ipage); 243 243 goto out; ··· 265 265 struct f2fs_sb_info *sbi = F2FS_I_SB(inode); 266 266 struct page *ipage; 267 267 268 - ipage = f2fs_get_node_page(sbi, inode->i_ino); 268 + ipage = f2fs_get_inode_page(sbi, inode->i_ino); 269 269 if (IS_ERR(ipage)) 270 270 return PTR_ERR(ipage); 271 271 ··· 312 312 if (f2fs_has_inline_data(inode) && 313 313 ri && (ri->i_inline & F2FS_INLINE_DATA)) { 314 314 process_inline: 315 - ipage = f2fs_get_node_page(sbi, inode->i_ino); 315 + ipage = f2fs_get_inode_page(sbi, inode->i_ino); 316 316 if (IS_ERR(ipage)) 317 317 return PTR_ERR(ipage); 318 318 ··· 331 331 } 332 332 333 333 if (f2fs_has_inline_data(inode)) { 334 - ipage = f2fs_get_node_page(sbi, inode->i_ino); 334 + ipage = f2fs_get_inode_page(sbi, inode->i_ino); 335 335 if (IS_ERR(ipage)) 336 336 return PTR_ERR(ipage); 337 337 f2fs_truncate_inline_inode(inode, ipage, 0); ··· 361 361 struct page *ipage; 362 362 void *inline_dentry; 363 363 364 - ipage = f2fs_get_node_page(sbi, dir->i_ino); 364 + ipage = f2fs_get_inode_page(sbi, dir->i_ino); 365 365 if (IS_ERR(ipage)) { 366 366 *res_page = ipage; 367 367 return NULL; ··· 609 609 if (err) 610 610 goto out; 611 611 612 - ipage = f2fs_get_node_page(sbi, dir->i_ino); 612 + ipage = f2fs_get_inode_page(sbi, dir->i_ino); 613 613 if (IS_ERR(ipage)) { 614 614 err = PTR_ERR(ipage); 615 615 goto out_fname; ··· 644 644 struct page *page = NULL; 645 645 int err = 0; 646 646 647 - ipage = f2fs_get_node_page(sbi, dir->i_ino); 647 + ipage = f2fs_get_inode_page(sbi, dir->i_ino); 648 648 if (IS_ERR(ipage)) 649 649 return PTR_ERR(ipage); 650 650 ··· 734 734 void *inline_dentry; 735 735 struct f2fs_dentry_ptr d; 736 736 737 - ipage = f2fs_get_node_page(sbi, dir->i_ino); 737 + ipage = f2fs_get_inode_page(sbi, dir->i_ino); 738 738 if (IS_ERR(ipage)) 739 739 return false; 740 740 ··· 765 765 if (ctx->pos == d.max) 766 766 return 0; 767 767 768 - ipage = f2fs_get_node_page(F2FS_I_SB(inode), inode->i_ino); 768 + ipage = f2fs_get_inode_page(F2FS_I_SB(inode), inode->i_ino); 769 769 if (IS_ERR(ipage)) 770 770 return PTR_ERR(ipage); 771 771 ··· 797 797 struct page *ipage; 798 798 int err = 0; 799 799 800 - ipage = f2fs_get_node_page(F2FS_I_SB(inode), inode->i_ino); 800 + ipage = f2fs_get_inode_page(F2FS_I_SB(inode), inode->i_ino); 801 801 if (IS_ERR(ipage)) 802 802 return PTR_ERR(ipage); 803 803
+28 -5
fs/f2fs/inode.c
··· 34 34 if (f2fs_inode_dirtied(inode, sync)) 35 35 return; 36 36 37 - if (f2fs_is_atomic_file(inode)) { 38 - set_inode_flag(inode, FI_ATOMIC_DIRTIED); 37 + if (f2fs_is_atomic_file(inode)) 39 38 return; 40 - } 41 39 42 40 mark_inode_dirty_sync(inode); 43 41 } ··· 408 410 if (f2fs_check_nid_range(sbi, inode->i_ino)) 409 411 return -EINVAL; 410 412 411 - node_page = f2fs_get_node_page(sbi, inode->i_ino); 413 + node_page = f2fs_get_inode_page(sbi, inode->i_ino); 412 414 if (IS_ERR(node_page)) 413 415 return PTR_ERR(node_page); 414 416 ··· 755 757 struct page *node_page; 756 758 int count = 0; 757 759 retry: 758 - node_page = f2fs_get_node_page(sbi, inode->i_ino); 760 + node_page = f2fs_get_inode_page(sbi, inode->i_ino); 759 761 if (IS_ERR(node_page)) { 760 762 int err = PTR_ERR(node_page); 761 763 ··· 763 765 if (err == -ENOENT) 764 766 return; 765 767 768 + if (err == -EFSCORRUPTED) 769 + goto stop_checkpoint; 770 + 766 771 if (err == -ENOMEM || ++count <= DEFAULT_RETRY_IO_COUNT) 767 772 goto retry; 773 + stop_checkpoint: 768 774 f2fs_stop_checkpoint(sbi, false, STOP_CP_REASON_UPDATE_INODE); 769 775 return; 770 776 } ··· 791 789 !is_inode_flag_set(inode, FI_DIRTY_INODE)) 792 790 return 0; 793 791 792 + /* 793 + * no need to update inode page, ultimately f2fs_evict_inode() will 794 + * clear dirty status of inode. 795 + */ 796 + if (f2fs_cp_error(sbi)) 797 + return -EIO; 798 + 794 799 if (!f2fs_is_checkpoint_ready(sbi)) { 795 800 f2fs_mark_inode_dirty_sync(inode, true); 796 801 return -ENOSPC; ··· 811 802 if (wbc && wbc->nr_to_write) 812 803 f2fs_balance_fs(sbi, true); 813 804 return 0; 805 + } 806 + 807 + static void f2fs_remove_donate_inode(struct inode *inode) 808 + { 809 + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); 810 + 811 + if (list_empty(&F2FS_I(inode)->gdonate_list)) 812 + return; 813 + 814 + spin_lock(&sbi->inode_lock[DONATE_INODE]); 815 + list_del_init(&F2FS_I(inode)->gdonate_list); 816 + sbi->donate_files--; 817 + spin_unlock(&sbi->inode_lock[DONATE_INODE]); 814 818 } 815 819 816 820 /* ··· 860 838 861 839 f2fs_bug_on(sbi, get_dirty_pages(inode)); 862 840 f2fs_remove_dirty_inode(inode); 841 + f2fs_remove_donate_inode(inode); 863 842 864 843 if (!IS_DEVICE_ALIASING(inode)) 865 844 f2fs_destroy_extent_tree(inode);
+8
fs/f2fs/namei.c
··· 502 502 goto out; 503 503 } 504 504 505 + if (inode->i_nlink == 0) { 506 + f2fs_warn(F2FS_I_SB(inode), "%s: inode (ino=%lx) has zero i_nlink", 507 + __func__, inode->i_ino); 508 + err = -EFSCORRUPTED; 509 + set_sbi_flag(F2FS_I_SB(inode), SBI_NEED_FSCK); 510 + goto out_iput; 511 + } 512 + 505 513 if (IS_ENCRYPTED(dir) && 506 514 (S_ISDIR(inode->i_mode) || S_ISLNK(inode->i_mode)) && 507 515 !fscrypt_has_permitted_context(dir, inode)) {
+216 -234
fs/f2fs/node.c
··· 310 310 start, nr); 311 311 } 312 312 313 - bool f2fs_in_warm_node_list(struct f2fs_sb_info *sbi, struct page *page) 313 + bool f2fs_in_warm_node_list(struct f2fs_sb_info *sbi, const struct folio *folio) 314 314 { 315 - return NODE_MAPPING(sbi) == page->mapping && 316 - IS_DNODE(page) && is_cold_node(page); 315 + return NODE_MAPPING(sbi) == folio->mapping && 316 + IS_DNODE(&folio->page) && is_cold_node(&folio->page); 317 317 } 318 318 319 319 void f2fs_init_fsync_node_info(struct f2fs_sb_info *sbi) ··· 778 778 npage[0] = dn->inode_page; 779 779 780 780 if (!npage[0]) { 781 - npage[0] = f2fs_get_node_page(sbi, nids[0]); 781 + npage[0] = f2fs_get_inode_page(sbi, nids[0]); 782 782 if (IS_ERR(npage[0])) 783 783 return PTR_ERR(npage[0]); 784 784 } ··· 1130 1130 unsigned int nofs = 0; 1131 1131 struct f2fs_inode *ri; 1132 1132 struct dnode_of_data dn; 1133 - struct page *page; 1133 + struct folio *folio; 1134 1134 1135 1135 trace_f2fs_truncate_inode_blocks_enter(inode, from); 1136 1136 1137 1137 level = get_node_path(inode, from, offset, noffset); 1138 - if (level < 0) { 1138 + if (level <= 0) { 1139 + if (!level) { 1140 + level = -EFSCORRUPTED; 1141 + f2fs_err(sbi, "%s: inode ino=%lx has corrupted node block, from:%lu addrs:%u", 1142 + __func__, inode->i_ino, 1143 + from, ADDRS_PER_INODE(inode)); 1144 + set_sbi_flag(sbi, SBI_NEED_FSCK); 1145 + } 1139 1146 trace_f2fs_truncate_inode_blocks_exit(inode, level); 1140 1147 return level; 1141 1148 } 1142 1149 1143 - page = f2fs_get_node_page(sbi, inode->i_ino); 1144 - if (IS_ERR(page)) { 1145 - trace_f2fs_truncate_inode_blocks_exit(inode, PTR_ERR(page)); 1146 - return PTR_ERR(page); 1150 + folio = f2fs_get_inode_folio(sbi, inode->i_ino); 1151 + if (IS_ERR(folio)) { 1152 + trace_f2fs_truncate_inode_blocks_exit(inode, PTR_ERR(folio)); 1153 + return PTR_ERR(folio); 1147 1154 } 1148 1155 1149 - set_new_dnode(&dn, inode, page, NULL, 0); 1150 - unlock_page(page); 1156 + set_new_dnode(&dn, inode, &folio->page, NULL, 0); 1157 + folio_unlock(folio); 1151 1158 1152 - ri = F2FS_INODE(page); 1159 + ri = F2FS_INODE(&folio->page); 1153 1160 switch (level) { 1154 1161 case 0: 1155 1162 case 1: ··· 1185 1178 1186 1179 skip_partial: 1187 1180 while (cont) { 1188 - dn.nid = get_nid(page, offset[0], true); 1181 + dn.nid = get_nid(&folio->page, offset[0], true); 1189 1182 switch (offset[0]) { 1190 1183 case NODE_DIR1_BLOCK: 1191 1184 case NODE_DIR2_BLOCK: ··· 1206 1199 BUG(); 1207 1200 } 1208 1201 if (err == -ENOENT) { 1209 - set_sbi_flag(F2FS_P_SB(page), SBI_NEED_FSCK); 1202 + set_sbi_flag(F2FS_F_SB(folio), SBI_NEED_FSCK); 1210 1203 f2fs_handle_error(sbi, ERROR_INVALID_BLKADDR); 1211 1204 f2fs_err_ratelimited(sbi, 1212 1205 "truncate node fail, ino:%lu, nid:%u, " ··· 1217 1210 } 1218 1211 if (err < 0) 1219 1212 goto fail; 1220 - if (offset[1] == 0 && get_nid(page, offset[0], true)) { 1221 - lock_page(page); 1222 - BUG_ON(page->mapping != NODE_MAPPING(sbi)); 1223 - set_nid(page, offset[0], 0, true); 1224 - unlock_page(page); 1213 + if (offset[1] == 0 && get_nid(&folio->page, offset[0], true)) { 1214 + folio_lock(folio); 1215 + BUG_ON(folio->mapping != NODE_MAPPING(sbi)); 1216 + set_nid(&folio->page, offset[0], 0, true); 1217 + folio_unlock(folio); 1225 1218 } 1226 1219 offset[1] = 0; 1227 1220 offset[0]++; 1228 1221 nofs += err; 1229 1222 } 1230 1223 fail: 1231 - f2fs_put_page(page, 0); 1224 + f2fs_folio_put(folio, false); 1232 1225 trace_f2fs_truncate_inode_blocks_exit(inode, err); 1233 1226 return err > 0 ? 0 : err; 1234 1227 } ··· 1245 1238 if (!nid) 1246 1239 return 0; 1247 1240 1248 - npage = f2fs_get_node_page(sbi, nid); 1241 + npage = f2fs_get_xnode_page(sbi, nid); 1249 1242 if (IS_ERR(npage)) 1250 1243 return PTR_ERR(npage); 1251 1244 ··· 1456 1449 f2fs_put_page(apage, err ? 1 : 0); 1457 1450 } 1458 1451 1459 - static struct page *__get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid, 1460 - struct page *parent, int start) 1452 + static int sanity_check_node_footer(struct f2fs_sb_info *sbi, 1453 + struct page *page, pgoff_t nid, 1454 + enum node_type ntype) 1461 1455 { 1462 - struct page *page; 1456 + if (unlikely(nid != nid_of_node(page) || 1457 + (ntype == NODE_TYPE_INODE && !IS_INODE(page)) || 1458 + (ntype == NODE_TYPE_XATTR && 1459 + !f2fs_has_xattr_block(ofs_of_node(page))) || 1460 + time_to_inject(sbi, FAULT_INCONSISTENT_FOOTER))) { 1461 + f2fs_warn(sbi, "inconsistent node block, node_type:%d, nid:%lu, " 1462 + "node_footer[nid:%u,ino:%u,ofs:%u,cpver:%llu,blkaddr:%u]", 1463 + ntype, nid, nid_of_node(page), ino_of_node(page), 1464 + ofs_of_node(page), cpver_of_node(page), 1465 + next_blkaddr_of_node(page)); 1466 + set_sbi_flag(sbi, SBI_NEED_FSCK); 1467 + f2fs_handle_error(sbi, ERROR_INCONSISTENT_FOOTER); 1468 + return -EFSCORRUPTED; 1469 + } 1470 + return 0; 1471 + } 1472 + 1473 + static struct folio *__get_node_folio(struct f2fs_sb_info *sbi, pgoff_t nid, 1474 + struct page *parent, int start, 1475 + enum node_type ntype) 1476 + { 1477 + struct folio *folio; 1463 1478 int err; 1464 1479 1465 1480 if (!nid) ··· 1489 1460 if (f2fs_check_nid_range(sbi, nid)) 1490 1461 return ERR_PTR(-EINVAL); 1491 1462 repeat: 1492 - page = f2fs_grab_cache_page(NODE_MAPPING(sbi), nid, false); 1493 - if (!page) 1494 - return ERR_PTR(-ENOMEM); 1463 + folio = f2fs_grab_cache_folio(NODE_MAPPING(sbi), nid, false); 1464 + if (IS_ERR(folio)) 1465 + return folio; 1495 1466 1496 - err = read_node_page(page, 0); 1467 + err = read_node_page(&folio->page, 0); 1497 1468 if (err < 0) { 1498 1469 goto out_put_err; 1499 1470 } else if (err == LOCKED_PAGE) { ··· 1504 1475 if (parent) 1505 1476 f2fs_ra_node_pages(parent, start + 1, MAX_RA_NODE); 1506 1477 1507 - lock_page(page); 1478 + folio_lock(folio); 1508 1479 1509 - if (unlikely(page->mapping != NODE_MAPPING(sbi))) { 1510 - f2fs_put_page(page, 1); 1480 + if (unlikely(folio->mapping != NODE_MAPPING(sbi))) { 1481 + f2fs_folio_put(folio, true); 1511 1482 goto repeat; 1512 1483 } 1513 1484 1514 - if (unlikely(!PageUptodate(page))) { 1485 + if (unlikely(!folio_test_uptodate(folio))) { 1515 1486 err = -EIO; 1516 1487 goto out_err; 1517 1488 } 1518 1489 1519 - if (!f2fs_inode_chksum_verify(sbi, page)) { 1490 + if (!f2fs_inode_chksum_verify(sbi, &folio->page)) { 1520 1491 err = -EFSBADCRC; 1521 1492 goto out_err; 1522 1493 } 1523 1494 page_hit: 1524 - if (likely(nid == nid_of_node(page))) 1525 - return page; 1526 - 1527 - f2fs_warn(sbi, "inconsistent node block, nid:%lu, node_footer[nid:%u,ino:%u,ofs:%u,cpver:%llu,blkaddr:%u]", 1528 - nid, nid_of_node(page), ino_of_node(page), 1529 - ofs_of_node(page), cpver_of_node(page), 1530 - next_blkaddr_of_node(page)); 1531 - set_sbi_flag(sbi, SBI_NEED_FSCK); 1532 - f2fs_handle_error(sbi, ERROR_INCONSISTENT_FOOTER); 1533 - err = -EFSCORRUPTED; 1495 + err = sanity_check_node_footer(sbi, &folio->page, nid, ntype); 1496 + if (!err) 1497 + return folio; 1534 1498 out_err: 1535 - ClearPageUptodate(page); 1499 + folio_clear_uptodate(folio); 1536 1500 out_put_err: 1537 1501 /* ENOENT comes from read_node_page which is not an error. */ 1538 1502 if (err != -ENOENT) 1539 - f2fs_handle_page_eio(sbi, page_folio(page), NODE); 1540 - f2fs_put_page(page, 1); 1503 + f2fs_handle_page_eio(sbi, folio, NODE); 1504 + f2fs_folio_put(folio, true); 1541 1505 return ERR_PTR(err); 1542 1506 } 1543 1507 1544 1508 struct page *f2fs_get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid) 1545 1509 { 1546 - return __get_node_page(sbi, nid, NULL, 0); 1510 + struct folio *folio = __get_node_folio(sbi, nid, NULL, 0, 1511 + NODE_TYPE_REGULAR); 1512 + 1513 + return &folio->page; 1514 + } 1515 + 1516 + struct folio *f2fs_get_inode_folio(struct f2fs_sb_info *sbi, pgoff_t ino) 1517 + { 1518 + return __get_node_folio(sbi, ino, NULL, 0, NODE_TYPE_INODE); 1519 + } 1520 + 1521 + struct page *f2fs_get_inode_page(struct f2fs_sb_info *sbi, pgoff_t ino) 1522 + { 1523 + struct folio *folio = f2fs_get_inode_folio(sbi, ino); 1524 + 1525 + return &folio->page; 1526 + } 1527 + 1528 + struct page *f2fs_get_xnode_page(struct f2fs_sb_info *sbi, pgoff_t xnid) 1529 + { 1530 + struct folio *folio = __get_node_folio(sbi, xnid, NULL, 0, 1531 + NODE_TYPE_XATTR); 1532 + 1533 + return &folio->page; 1547 1534 } 1548 1535 1549 1536 struct page *f2fs_get_node_page_ra(struct page *parent, int start) 1550 1537 { 1551 1538 struct f2fs_sb_info *sbi = F2FS_P_SB(parent); 1552 1539 nid_t nid = get_nid(parent, start, false); 1540 + struct folio *folio = __get_node_folio(sbi, nid, parent, start, 1541 + NODE_TYPE_REGULAR); 1553 1542 1554 - return __get_node_page(sbi, nid, parent, start); 1543 + return &folio->page; 1555 1544 } 1556 1545 1557 1546 static void flush_inline_data(struct f2fs_sb_info *sbi, nid_t ino) ··· 1608 1561 iput(inode); 1609 1562 } 1610 1563 1611 - static struct page *last_fsync_dnode(struct f2fs_sb_info *sbi, nid_t ino) 1564 + static struct folio *last_fsync_dnode(struct f2fs_sb_info *sbi, nid_t ino) 1612 1565 { 1613 1566 pgoff_t index; 1614 1567 struct folio_batch fbatch; 1615 - struct page *last_page = NULL; 1568 + struct folio *last_folio = NULL; 1616 1569 int nr_folios; 1617 1570 1618 1571 folio_batch_init(&fbatch); ··· 1624 1577 int i; 1625 1578 1626 1579 for (i = 0; i < nr_folios; i++) { 1627 - struct page *page = &fbatch.folios[i]->page; 1580 + struct folio *folio = fbatch.folios[i]; 1628 1581 1629 1582 if (unlikely(f2fs_cp_error(sbi))) { 1630 - f2fs_put_page(last_page, 0); 1583 + f2fs_folio_put(last_folio, false); 1631 1584 folio_batch_release(&fbatch); 1632 1585 return ERR_PTR(-EIO); 1633 1586 } 1634 1587 1635 - if (!IS_DNODE(page) || !is_cold_node(page)) 1588 + if (!IS_DNODE(&folio->page) || !is_cold_node(&folio->page)) 1636 1589 continue; 1637 - if (ino_of_node(page) != ino) 1590 + if (ino_of_node(&folio->page) != ino) 1638 1591 continue; 1639 1592 1640 - lock_page(page); 1593 + folio_lock(folio); 1641 1594 1642 - if (unlikely(page->mapping != NODE_MAPPING(sbi))) { 1595 + if (unlikely(folio->mapping != NODE_MAPPING(sbi))) { 1643 1596 continue_unlock: 1644 - unlock_page(page); 1597 + folio_unlock(folio); 1645 1598 continue; 1646 1599 } 1647 - if (ino_of_node(page) != ino) 1600 + if (ino_of_node(&folio->page) != ino) 1648 1601 goto continue_unlock; 1649 1602 1650 - if (!PageDirty(page)) { 1603 + if (!folio_test_dirty(folio)) { 1651 1604 /* someone wrote it for us */ 1652 1605 goto continue_unlock; 1653 1606 } 1654 1607 1655 - if (last_page) 1656 - f2fs_put_page(last_page, 0); 1608 + if (last_folio) 1609 + f2fs_folio_put(last_folio, false); 1657 1610 1658 - get_page(page); 1659 - last_page = page; 1660 - unlock_page(page); 1611 + folio_get(folio); 1612 + last_folio = folio; 1613 + folio_unlock(folio); 1661 1614 } 1662 1615 folio_batch_release(&fbatch); 1663 1616 cond_resched(); 1664 1617 } 1665 - return last_page; 1618 + return last_folio; 1666 1619 } 1667 1620 1668 1621 static int __write_node_page(struct page *page, bool atomic, bool *submitted, ··· 1741 1694 fio.op_flags |= REQ_PREFLUSH | REQ_FUA; 1742 1695 1743 1696 /* should add to global list before clearing PAGECACHE status */ 1744 - if (f2fs_in_warm_node_list(sbi, page)) { 1697 + if (f2fs_in_warm_node_list(sbi, folio)) { 1745 1698 seq = f2fs_add_fsync_node_entry(sbi, page); 1746 1699 if (seq_id) 1747 1700 *seq_id = seq; ··· 1816 1769 return err; 1817 1770 } 1818 1771 1819 - static int f2fs_write_node_page(struct page *page, 1820 - struct writeback_control *wbc) 1821 - { 1822 - return __write_node_page(page, false, NULL, wbc, false, 1823 - FS_NODE_IO, NULL); 1824 - } 1825 - 1826 1772 int f2fs_fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode, 1827 1773 struct writeback_control *wbc, bool atomic, 1828 1774 unsigned int *seq_id) ··· 1823 1783 pgoff_t index; 1824 1784 struct folio_batch fbatch; 1825 1785 int ret = 0; 1826 - struct page *last_page = NULL; 1786 + struct folio *last_folio = NULL; 1827 1787 bool marked = false; 1828 1788 nid_t ino = inode->i_ino; 1829 1789 int nr_folios; 1830 1790 int nwritten = 0; 1831 1791 1832 1792 if (atomic) { 1833 - last_page = last_fsync_dnode(sbi, ino); 1834 - if (IS_ERR_OR_NULL(last_page)) 1835 - return PTR_ERR_OR_ZERO(last_page); 1793 + last_folio = last_fsync_dnode(sbi, ino); 1794 + if (IS_ERR_OR_NULL(last_folio)) 1795 + return PTR_ERR_OR_ZERO(last_folio); 1836 1796 } 1837 1797 retry: 1838 1798 folio_batch_init(&fbatch); ··· 1844 1804 int i; 1845 1805 1846 1806 for (i = 0; i < nr_folios; i++) { 1847 - struct page *page = &fbatch.folios[i]->page; 1807 + struct folio *folio = fbatch.folios[i]; 1848 1808 bool submitted = false; 1849 1809 1850 1810 if (unlikely(f2fs_cp_error(sbi))) { 1851 - f2fs_put_page(last_page, 0); 1811 + f2fs_folio_put(last_folio, false); 1852 1812 folio_batch_release(&fbatch); 1853 1813 ret = -EIO; 1854 1814 goto out; 1855 1815 } 1856 1816 1857 - if (!IS_DNODE(page) || !is_cold_node(page)) 1817 + if (!IS_DNODE(&folio->page) || !is_cold_node(&folio->page)) 1858 1818 continue; 1859 - if (ino_of_node(page) != ino) 1819 + if (ino_of_node(&folio->page) != ino) 1860 1820 continue; 1861 1821 1862 - lock_page(page); 1822 + folio_lock(folio); 1863 1823 1864 - if (unlikely(page->mapping != NODE_MAPPING(sbi))) { 1824 + if (unlikely(folio->mapping != NODE_MAPPING(sbi))) { 1865 1825 continue_unlock: 1866 - unlock_page(page); 1826 + folio_unlock(folio); 1867 1827 continue; 1868 1828 } 1869 - if (ino_of_node(page) != ino) 1829 + if (ino_of_node(&folio->page) != ino) 1870 1830 goto continue_unlock; 1871 1831 1872 - if (!PageDirty(page) && page != last_page) { 1832 + if (!folio_test_dirty(folio) && folio != last_folio) { 1873 1833 /* someone wrote it for us */ 1874 1834 goto continue_unlock; 1875 1835 } 1876 1836 1877 - f2fs_wait_on_page_writeback(page, NODE, true, true); 1837 + f2fs_folio_wait_writeback(folio, NODE, true, true); 1878 1838 1879 - set_fsync_mark(page, 0); 1880 - set_dentry_mark(page, 0); 1839 + set_fsync_mark(&folio->page, 0); 1840 + set_dentry_mark(&folio->page, 0); 1881 1841 1882 - if (!atomic || page == last_page) { 1883 - set_fsync_mark(page, 1); 1842 + if (!atomic || folio == last_folio) { 1843 + set_fsync_mark(&folio->page, 1); 1884 1844 percpu_counter_inc(&sbi->rf_node_block_count); 1885 - if (IS_INODE(page)) { 1845 + if (IS_INODE(&folio->page)) { 1886 1846 if (is_inode_flag_set(inode, 1887 1847 FI_DIRTY_INODE)) 1888 - f2fs_update_inode(inode, page); 1889 - set_dentry_mark(page, 1848 + f2fs_update_inode(inode, &folio->page); 1849 + set_dentry_mark(&folio->page, 1890 1850 f2fs_need_dentry_mark(sbi, ino)); 1891 1851 } 1892 1852 /* may be written by other thread */ 1893 - if (!PageDirty(page)) 1894 - set_page_dirty(page); 1853 + if (!folio_test_dirty(folio)) 1854 + folio_mark_dirty(folio); 1895 1855 } 1896 1856 1897 - if (!clear_page_dirty_for_io(page)) 1857 + if (!folio_clear_dirty_for_io(folio)) 1898 1858 goto continue_unlock; 1899 1859 1900 - ret = __write_node_page(page, atomic && 1901 - page == last_page, 1860 + ret = __write_node_page(&folio->page, atomic && 1861 + folio == last_folio, 1902 1862 &submitted, wbc, true, 1903 1863 FS_NODE_IO, seq_id); 1904 1864 if (ret) { 1905 - unlock_page(page); 1906 - f2fs_put_page(last_page, 0); 1865 + folio_unlock(folio); 1866 + f2fs_folio_put(last_folio, false); 1907 1867 break; 1908 1868 } else if (submitted) { 1909 1869 nwritten++; 1910 1870 } 1911 1871 1912 - if (page == last_page) { 1913 - f2fs_put_page(page, 0); 1872 + if (folio == last_folio) { 1873 + f2fs_folio_put(folio, false); 1914 1874 marked = true; 1915 1875 break; 1916 1876 } ··· 1923 1883 } 1924 1884 if (!ret && atomic && !marked) { 1925 1885 f2fs_debug(sbi, "Retry to write fsync mark: ino=%u, idx=%lx", 1926 - ino, page_folio(last_page)->index); 1927 - lock_page(last_page); 1928 - f2fs_wait_on_page_writeback(last_page, NODE, true, true); 1929 - set_page_dirty(last_page); 1930 - unlock_page(last_page); 1886 + ino, last_folio->index); 1887 + folio_lock(last_folio); 1888 + f2fs_folio_wait_writeback(last_folio, NODE, true, true); 1889 + folio_mark_dirty(last_folio); 1890 + folio_unlock(last_folio); 1931 1891 goto retry; 1932 1892 } 1933 1893 out: ··· 1960 1920 return 1; 1961 1921 } 1962 1922 1963 - static bool flush_dirty_inode(struct page *page) 1923 + static bool flush_dirty_inode(struct folio *folio) 1964 1924 { 1965 - struct f2fs_sb_info *sbi = F2FS_P_SB(page); 1925 + struct f2fs_sb_info *sbi = F2FS_F_SB(folio); 1966 1926 struct inode *inode; 1967 - nid_t ino = ino_of_node(page); 1927 + nid_t ino = ino_of_node(&folio->page); 1968 1928 1969 1929 inode = find_inode_nowait(sbi->sb, ino, f2fs_match_ino, NULL); 1970 1930 if (!inode) 1971 1931 return false; 1972 1932 1973 - f2fs_update_inode(inode, page); 1974 - unlock_page(page); 1933 + f2fs_update_inode(inode, &folio->page); 1934 + folio_unlock(folio); 1975 1935 1976 1936 iput(inode); 1977 1937 return true; ··· 1991 1951 int i; 1992 1952 1993 1953 for (i = 0; i < nr_folios; i++) { 1994 - struct page *page = &fbatch.folios[i]->page; 1954 + struct folio *folio = fbatch.folios[i]; 1995 1955 1996 - if (!IS_INODE(page)) 1956 + if (!IS_INODE(&folio->page)) 1997 1957 continue; 1998 1958 1999 - lock_page(page); 1959 + folio_lock(folio); 2000 1960 2001 - if (unlikely(page->mapping != NODE_MAPPING(sbi))) { 2002 - continue_unlock: 2003 - unlock_page(page); 2004 - continue; 2005 - } 2006 - 2007 - if (!PageDirty(page)) { 2008 - /* someone wrote it for us */ 2009 - goto continue_unlock; 2010 - } 1961 + if (unlikely(folio->mapping != NODE_MAPPING(sbi))) 1962 + goto unlock; 1963 + if (!folio_test_dirty(folio)) 1964 + goto unlock; 2011 1965 2012 1966 /* flush inline_data, if it's async context. */ 2013 - if (page_private_inline(page)) { 2014 - clear_page_private_inline(page); 2015 - unlock_page(page); 2016 - flush_inline_data(sbi, ino_of_node(page)); 1967 + if (page_private_inline(&folio->page)) { 1968 + clear_page_private_inline(&folio->page); 1969 + folio_unlock(folio); 1970 + flush_inline_data(sbi, ino_of_node(&folio->page)); 2017 1971 continue; 2018 1972 } 2019 - unlock_page(page); 1973 + unlock: 1974 + folio_unlock(folio); 2020 1975 } 2021 1976 folio_batch_release(&fbatch); 2022 1977 cond_resched(); ··· 2040 2005 int i; 2041 2006 2042 2007 for (i = 0; i < nr_folios; i++) { 2043 - struct page *page = &fbatch.folios[i]->page; 2008 + struct folio *folio = fbatch.folios[i]; 2044 2009 bool submitted = false; 2045 2010 2046 2011 /* give a priority to WB_SYNC threads */ ··· 2056 2021 * 1. dentry dnodes 2057 2022 * 2. file dnodes 2058 2023 */ 2059 - if (step == 0 && IS_DNODE(page)) 2024 + if (step == 0 && IS_DNODE(&folio->page)) 2060 2025 continue; 2061 - if (step == 1 && (!IS_DNODE(page) || 2062 - is_cold_node(page))) 2026 + if (step == 1 && (!IS_DNODE(&folio->page) || 2027 + is_cold_node(&folio->page))) 2063 2028 continue; 2064 - if (step == 2 && (!IS_DNODE(page) || 2065 - !is_cold_node(page))) 2029 + if (step == 2 && (!IS_DNODE(&folio->page) || 2030 + !is_cold_node(&folio->page))) 2066 2031 continue; 2067 2032 lock_node: 2068 2033 if (wbc->sync_mode == WB_SYNC_ALL) 2069 - lock_page(page); 2070 - else if (!trylock_page(page)) 2034 + folio_lock(folio); 2035 + else if (!folio_trylock(folio)) 2071 2036 continue; 2072 2037 2073 - if (unlikely(page->mapping != NODE_MAPPING(sbi))) { 2038 + if (unlikely(folio->mapping != NODE_MAPPING(sbi))) { 2074 2039 continue_unlock: 2075 - unlock_page(page); 2040 + folio_unlock(folio); 2076 2041 continue; 2077 2042 } 2078 2043 2079 - if (!PageDirty(page)) { 2044 + if (!folio_test_dirty(folio)) { 2080 2045 /* someone wrote it for us */ 2081 2046 goto continue_unlock; 2082 2047 } ··· 2086 2051 goto write_node; 2087 2052 2088 2053 /* flush inline_data */ 2089 - if (page_private_inline(page)) { 2090 - clear_page_private_inline(page); 2091 - unlock_page(page); 2092 - flush_inline_data(sbi, ino_of_node(page)); 2054 + if (page_private_inline(&folio->page)) { 2055 + clear_page_private_inline(&folio->page); 2056 + folio_unlock(folio); 2057 + flush_inline_data(sbi, ino_of_node(&folio->page)); 2093 2058 goto lock_node; 2094 2059 } 2095 2060 2096 2061 /* flush dirty inode */ 2097 - if (IS_INODE(page) && flush_dirty_inode(page)) 2062 + if (IS_INODE(&folio->page) && flush_dirty_inode(folio)) 2098 2063 goto lock_node; 2099 2064 write_node: 2100 - f2fs_wait_on_page_writeback(page, NODE, true, true); 2065 + f2fs_folio_wait_writeback(folio, NODE, true, true); 2101 2066 2102 - if (!clear_page_dirty_for_io(page)) 2067 + if (!folio_clear_dirty_for_io(folio)) 2103 2068 goto continue_unlock; 2104 2069 2105 - set_fsync_mark(page, 0); 2106 - set_dentry_mark(page, 0); 2070 + set_fsync_mark(&folio->page, 0); 2071 + set_dentry_mark(&folio->page, 0); 2107 2072 2108 - ret = __write_node_page(page, false, &submitted, 2073 + ret = __write_node_page(&folio->page, false, &submitted, 2109 2074 wbc, do_balance, io_type, NULL); 2110 2075 if (ret) 2111 - unlock_page(page); 2076 + folio_unlock(folio); 2112 2077 else if (submitted) 2113 2078 nwritten++; 2114 2079 ··· 2242 2207 * Structure of the f2fs node operations 2243 2208 */ 2244 2209 const struct address_space_operations f2fs_node_aops = { 2245 - .writepage = f2fs_write_node_page, 2246 2210 .writepages = f2fs_write_node_pages, 2247 2211 .dirty_folio = f2fs_dirty_node_folio, 2248 2212 .invalidate_folio = f2fs_invalidate_folio, ··· 2301 2267 default: 2302 2268 BUG_ON(1); 2303 2269 } 2304 - } 2305 - 2306 - bool f2fs_nat_bitmap_enabled(struct f2fs_sb_info *sbi) 2307 - { 2308 - struct f2fs_nm_info *nm_i = NM_I(sbi); 2309 - unsigned int i; 2310 - bool ret = true; 2311 - 2312 - f2fs_down_read(&nm_i->nat_tree_lock); 2313 - for (i = 0; i < nm_i->nat_blocks; i++) { 2314 - if (!test_bit_le(i, nm_i->nat_block_bitmap)) { 2315 - ret = false; 2316 - break; 2317 - } 2318 - } 2319 - f2fs_up_read(&nm_i->nat_tree_lock); 2320 - 2321 - return ret; 2322 2270 } 2323 2271 2324 2272 static void update_free_nid_bitmap(struct f2fs_sb_info *sbi, nid_t nid, ··· 2733 2717 struct page *ipage; 2734 2718 struct f2fs_inode *ri; 2735 2719 2736 - ipage = f2fs_get_node_page(F2FS_I_SB(inode), inode->i_ino); 2720 + ipage = f2fs_get_inode_page(F2FS_I_SB(inode), inode->i_ino); 2737 2721 if (IS_ERR(ipage)) 2738 2722 return PTR_ERR(ipage); 2739 2723 ··· 2981 2965 list_add_tail(&nes->set_list, head); 2982 2966 } 2983 2967 2984 - static void __update_nat_bits(struct f2fs_nm_info *nm_i, unsigned int nat_ofs, 2985 - unsigned int valid) 2986 - { 2987 - if (valid == 0) { 2988 - __set_bit_le(nat_ofs, nm_i->empty_nat_bits); 2989 - __clear_bit_le(nat_ofs, nm_i->full_nat_bits); 2990 - return; 2991 - } 2992 - 2993 - __clear_bit_le(nat_ofs, nm_i->empty_nat_bits); 2994 - if (valid == NAT_ENTRY_PER_BLOCK) 2995 - __set_bit_le(nat_ofs, nm_i->full_nat_bits); 2996 - else 2997 - __clear_bit_le(nat_ofs, nm_i->full_nat_bits); 2998 - } 2999 - 3000 - static void update_nat_bits(struct f2fs_sb_info *sbi, nid_t start_nid, 2968 + static void __update_nat_bits(struct f2fs_sb_info *sbi, nid_t start_nid, 3001 2969 struct page *page) 3002 2970 { 3003 2971 struct f2fs_nm_info *nm_i = NM_I(sbi); ··· 2990 2990 int valid = 0; 2991 2991 int i = 0; 2992 2992 2993 - if (!is_set_ckpt_flags(sbi, CP_NAT_BITS_FLAG)) 2993 + if (!enabled_nat_bits(sbi, NULL)) 2994 2994 return; 2995 2995 2996 2996 if (nat_index == 0) { ··· 3001 3001 if (le32_to_cpu(nat_blk->entries[i].block_addr) != NULL_ADDR) 3002 3002 valid++; 3003 3003 } 3004 - 3005 - __update_nat_bits(nm_i, nat_index, valid); 3006 - } 3007 - 3008 - void f2fs_enable_nat_bits(struct f2fs_sb_info *sbi) 3009 - { 3010 - struct f2fs_nm_info *nm_i = NM_I(sbi); 3011 - unsigned int nat_ofs; 3012 - 3013 - f2fs_down_read(&nm_i->nat_tree_lock); 3014 - 3015 - for (nat_ofs = 0; nat_ofs < nm_i->nat_blocks; nat_ofs++) { 3016 - unsigned int valid = 0, nid_ofs = 0; 3017 - 3018 - /* handle nid zero due to it should never be used */ 3019 - if (unlikely(nat_ofs == 0)) { 3020 - valid = 1; 3021 - nid_ofs = 1; 3022 - } 3023 - 3024 - for (; nid_ofs < NAT_ENTRY_PER_BLOCK; nid_ofs++) { 3025 - if (!test_bit_le(nid_ofs, 3026 - nm_i->free_nid_bitmap[nat_ofs])) 3027 - valid++; 3028 - } 3029 - 3030 - __update_nat_bits(nm_i, nat_ofs, valid); 3004 + if (valid == 0) { 3005 + __set_bit_le(nat_index, nm_i->empty_nat_bits); 3006 + __clear_bit_le(nat_index, nm_i->full_nat_bits); 3007 + return; 3031 3008 } 3032 3009 3033 - f2fs_up_read(&nm_i->nat_tree_lock); 3010 + __clear_bit_le(nat_index, nm_i->empty_nat_bits); 3011 + if (valid == NAT_ENTRY_PER_BLOCK) 3012 + __set_bit_le(nat_index, nm_i->full_nat_bits); 3013 + else 3014 + __clear_bit_le(nat_index, nm_i->full_nat_bits); 3034 3015 } 3035 3016 3036 3017 static int __flush_nat_entry_set(struct f2fs_sb_info *sbi, ··· 3030 3049 * #1, flush nat entries to journal in current hot data summary block. 3031 3050 * #2, flush nat entries to nat page. 3032 3051 */ 3033 - if ((cpc->reason & CP_UMOUNT) || 3052 + if (enabled_nat_bits(sbi, cpc) || 3034 3053 !__has_cursum_space(journal, set->entry_cnt, NAT_JOURNAL)) 3035 3054 to_journal = false; 3036 3055 ··· 3077 3096 if (to_journal) { 3078 3097 up_write(&curseg->journal_rwsem); 3079 3098 } else { 3080 - update_nat_bits(sbi, start_nid, page); 3099 + __update_nat_bits(sbi, start_nid, page); 3081 3100 f2fs_put_page(page, 1); 3082 3101 } 3083 3102 ··· 3108 3127 * during unmount, let's flush nat_bits before checking 3109 3128 * nat_cnt[DIRTY_NAT]. 3110 3129 */ 3111 - if (cpc->reason & CP_UMOUNT) { 3130 + if (enabled_nat_bits(sbi, cpc)) { 3112 3131 f2fs_down_write(&nm_i->nat_tree_lock); 3113 3132 remove_nats_in_journal(sbi); 3114 3133 f2fs_up_write(&nm_i->nat_tree_lock); ··· 3124 3143 * entries, remove all entries from journal and merge them 3125 3144 * into nat entry set. 3126 3145 */ 3127 - if (cpc->reason & CP_UMOUNT || 3146 + if (enabled_nat_bits(sbi, cpc) || 3128 3147 !__has_cursum_space(journal, 3129 3148 nm_i->nat_cnt[DIRTY_NAT], NAT_JOURNAL)) 3130 3149 remove_nats_in_journal(sbi); ··· 3161 3180 __u64 cp_ver = cur_cp_version(ckpt); 3162 3181 block_t nat_bits_addr; 3163 3182 3183 + if (!enabled_nat_bits(sbi, NULL)) 3184 + return 0; 3185 + 3164 3186 nm_i->nat_bits_blocks = F2FS_BLK_ALIGN((nat_bits_bytes << 1) + 8); 3165 3187 nm_i->nat_bits = f2fs_kvzalloc(sbi, 3166 3188 F2FS_BLK_TO_BYTES(nm_i->nat_bits_blocks), GFP_KERNEL); 3167 3189 if (!nm_i->nat_bits) 3168 3190 return -ENOMEM; 3169 - 3170 - nm_i->full_nat_bits = nm_i->nat_bits + 8; 3171 - nm_i->empty_nat_bits = nm_i->full_nat_bits + nat_bits_bytes; 3172 - 3173 - if (!is_set_ckpt_flags(sbi, CP_NAT_BITS_FLAG)) 3174 - return 0; 3175 3191 3176 3192 nat_bits_addr = __start_cp_addr(sbi) + BLKS_PER_SEG(sbi) - 3177 3193 nm_i->nat_bits_blocks; ··· 3186 3208 3187 3209 cp_ver |= (cur_cp_crc(ckpt) << 32); 3188 3210 if (cpu_to_le64(cp_ver) != *(__le64 *)nm_i->nat_bits) { 3189 - clear_ckpt_flags(sbi, CP_NAT_BITS_FLAG); 3190 - f2fs_notice(sbi, "Disable nat_bits due to incorrect cp_ver (%llu, %llu)", 3191 - cp_ver, le64_to_cpu(*(__le64 *)nm_i->nat_bits)); 3211 + disable_nat_bits(sbi, true); 3192 3212 return 0; 3193 3213 } 3214 + 3215 + nm_i->full_nat_bits = nm_i->nat_bits + 8; 3216 + nm_i->empty_nat_bits = nm_i->full_nat_bits + nat_bits_bytes; 3194 3217 3195 3218 f2fs_notice(sbi, "Found nat_bits in checkpoint"); 3196 3219 return 0; ··· 3203 3224 unsigned int i = 0; 3204 3225 nid_t nid, last_nid; 3205 3226 3206 - if (!is_set_ckpt_flags(sbi, CP_NAT_BITS_FLAG)) 3227 + if (!enabled_nat_bits(sbi, NULL)) 3207 3228 return; 3208 3229 3209 3230 for (i = 0; i < nm_i->nat_blocks; i++) { ··· 3274 3295 GFP_KERNEL); 3275 3296 if (!nm_i->nat_bitmap) 3276 3297 return -ENOMEM; 3298 + 3299 + if (!test_opt(sbi, NAT_BITS)) 3300 + disable_nat_bits(sbi, true); 3277 3301 3278 3302 err = __get_nat_bitmaps(sbi); 3279 3303 if (err)
+10 -3
fs/f2fs/node.h
··· 52 52 IS_PREALLOC, /* nat entry is preallocated */ 53 53 }; 54 54 55 + /* For node type in __get_node_folio() */ 56 + enum node_type { 57 + NODE_TYPE_REGULAR, 58 + NODE_TYPE_INODE, 59 + NODE_TYPE_XATTR, 60 + }; 61 + 55 62 /* 56 63 * For node information 57 64 */ ··· 255 248 return le32_to_cpu(rn->footer.nid); 256 249 } 257 250 258 - static inline unsigned int ofs_of_node(struct page *node_page) 251 + static inline unsigned int ofs_of_node(const struct page *node_page) 259 252 { 260 253 struct f2fs_node *rn = F2FS_NODE(node_page); 261 254 unsigned flag = le32_to_cpu(rn->footer.flag); ··· 349 342 * `- indirect node ((6 + 2N) + (N - 1)(N + 1)) 350 343 * `- direct node 351 344 */ 352 - static inline bool IS_DNODE(struct page *node_page) 345 + static inline bool IS_DNODE(const struct page *node_page) 353 346 { 354 347 unsigned int ofs = ofs_of_node(node_page); 355 348 ··· 396 389 * - Mark cold data pages in page cache 397 390 */ 398 391 399 - static inline int is_node(struct page *page, int type) 392 + static inline int is_node(const struct page *page, int type) 400 393 { 401 394 struct f2fs_node *rn = F2FS_NODE(page); 402 395 return le32_to_cpu(rn->footer.flag) & BIT(type);
+28 -27
fs/f2fs/segment.c
··· 2096 2096 return false; 2097 2097 2098 2098 if (!force) { 2099 - if (!f2fs_realtime_discard_enable(sbi) || !se->valid_blocks || 2099 + if (!f2fs_realtime_discard_enable(sbi) || 2100 + (!se->valid_blocks && 2101 + !IS_CURSEG(sbi, cpc->trim_start)) || 2100 2102 SM_I(sbi)->dcc_info->nr_discards >= 2101 2103 SM_I(sbi)->dcc_info->max_discards) 2102 2104 return false; ··· 2322 2320 dcc->discard_granularity = DEFAULT_DISCARD_GRANULARITY; 2323 2321 dcc->max_ordered_discard = DEFAULT_MAX_ORDERED_DISCARD_GRANULARITY; 2324 2322 dcc->discard_io_aware = DPOLICY_IO_AWARE_ENABLE; 2325 - if (F2FS_OPTION(sbi).discard_unit == DISCARD_UNIT_SEGMENT) 2323 + if (F2FS_OPTION(sbi).discard_unit == DISCARD_UNIT_SEGMENT || 2324 + F2FS_OPTION(sbi).discard_unit == DISCARD_UNIT_SECTION) 2326 2325 dcc->discard_granularity = BLKS_PER_SEG(sbi); 2327 - else if (F2FS_OPTION(sbi).discard_unit == DISCARD_UNIT_SECTION) 2328 - dcc->discard_granularity = BLKS_PER_SEC(sbi); 2329 2326 2330 2327 INIT_LIST_HEAD(&dcc->entry_list); 2331 2328 for (i = 0; i < MAX_PLIST_NUM; i++) ··· 2807 2806 MAIN_SECS(sbi)); 2808 2807 if (secno >= MAIN_SECS(sbi)) { 2809 2808 ret = -ENOSPC; 2810 - f2fs_bug_on(sbi, 1); 2809 + f2fs_bug_on(sbi, !pinning); 2811 2810 goto out_unlock; 2812 2811 } 2813 2812 } ··· 2849 2848 out_unlock: 2850 2849 spin_unlock(&free_i->segmap_lock); 2851 2850 2852 - if (ret == -ENOSPC) 2851 + if (ret == -ENOSPC && !pinning) 2853 2852 f2fs_stop_checkpoint(sbi, false, STOP_CP_REASON_NO_SEGMENT); 2854 2853 return ret; 2855 2854 } ··· 2922 2921 return curseg->segno; 2923 2922 } 2924 2923 2924 + static void reset_curseg_fields(struct curseg_info *curseg) 2925 + { 2926 + curseg->inited = false; 2927 + curseg->segno = NULL_SEGNO; 2928 + curseg->next_segno = 0; 2929 + } 2930 + 2925 2931 /* 2926 2932 * Allocate a current working segment. 2927 2933 * This function always allocates a free segment in LFS manner. ··· 2947 2939 ret = get_new_segment(sbi, &segno, new_sec, pinning); 2948 2940 if (ret) { 2949 2941 if (ret == -ENOSPC) 2950 - curseg->segno = NULL_SEGNO; 2942 + reset_curseg_fields(curseg); 2951 2943 return ret; 2952 2944 } 2953 2945 ··· 3718 3710 get_random_u32_inclusive(1, sbi->max_fragment_hole); 3719 3711 } 3720 3712 3721 - static void reset_curseg_fields(struct curseg_info *curseg) 3722 - { 3723 - curseg->inited = false; 3724 - curseg->segno = NULL_SEGNO; 3725 - curseg->next_segno = 0; 3726 - } 3727 - 3728 3713 int f2fs_allocate_data_block(struct f2fs_sb_info *sbi, struct page *page, 3729 3714 block_t old_blkaddr, block_t *new_blkaddr, 3730 3715 struct f2fs_summary *sum, int type, ··· 3903 3902 3904 3903 static void do_write_page(struct f2fs_summary *sum, struct f2fs_io_info *fio) 3905 3904 { 3905 + struct folio *folio = page_folio(fio->page); 3906 3906 enum log_type type = __get_segment_type(fio); 3907 3907 int seg_type = log_type_to_seg_type(type); 3908 3908 bool keep_order = (f2fs_lfs_mode(fio->sbi) && ··· 3914 3912 3915 3913 if (f2fs_allocate_data_block(fio->sbi, fio->page, fio->old_blkaddr, 3916 3914 &fio->new_blkaddr, sum, type, fio)) { 3917 - if (fscrypt_inode_uses_fs_layer_crypto(fio->page->mapping->host)) 3915 + if (fscrypt_inode_uses_fs_layer_crypto(folio->mapping->host)) 3918 3916 fscrypt_finalize_bounce_page(&fio->encrypted_page); 3919 - end_page_writeback(fio->page); 3920 - if (f2fs_in_warm_node_list(fio->sbi, fio->page)) 3917 + folio_end_writeback(folio); 3918 + if (f2fs_in_warm_node_list(fio->sbi, folio)) 3921 3919 f2fs_del_fsync_node_entry(fio->sbi, fio->page); 3922 3920 goto out; 3923 3921 } ··· 4156 4154 f2fs_update_data_blkaddr(dn, new_addr); 4157 4155 } 4158 4156 4159 - void f2fs_wait_on_page_writeback(struct page *page, 4160 - enum page_type type, bool ordered, bool locked) 4157 + void f2fs_folio_wait_writeback(struct folio *folio, enum page_type type, 4158 + bool ordered, bool locked) 4161 4159 { 4162 - if (folio_test_writeback(page_folio(page))) { 4163 - struct f2fs_sb_info *sbi = F2FS_P_SB(page); 4160 + if (folio_test_writeback(folio)) { 4161 + struct f2fs_sb_info *sbi = F2FS_F_SB(folio); 4164 4162 4165 4163 /* submit cached LFS IO */ 4166 - f2fs_submit_merged_write_cond(sbi, NULL, page, 0, type); 4164 + f2fs_submit_merged_write_cond(sbi, NULL, &folio->page, 0, type); 4167 4165 /* submit cached IPU IO */ 4168 - f2fs_submit_merged_ipu_write(sbi, NULL, page); 4166 + f2fs_submit_merged_ipu_write(sbi, NULL, &folio->page); 4169 4167 if (ordered) { 4170 - wait_on_page_writeback(page); 4171 - f2fs_bug_on(sbi, locked && 4172 - folio_test_writeback(page_folio(page))); 4168 + folio_wait_writeback(folio); 4169 + f2fs_bug_on(sbi, locked && folio_test_writeback(folio)); 4173 4170 } else { 4174 - wait_for_stable_page(page); 4171 + folio_wait_stable(folio); 4175 4172 } 4176 4173 } 4177 4174 }
+8 -1
fs/f2fs/segment.h
··· 559 559 unsigned int node_blocks, unsigned int data_blocks, 560 560 unsigned int dent_blocks) 561 561 { 562 - 563 562 unsigned int segno, left_blocks, blocks; 564 563 int i; 565 564 566 565 /* check current data/node sections in the worst case. */ 567 566 for (i = CURSEG_HOT_DATA; i < NR_PERSISTENT_LOG; i++) { 568 567 segno = CURSEG_I(sbi, i)->segno; 568 + 569 + if (unlikely(segno == NULL_SEGNO)) 570 + return false; 571 + 569 572 left_blocks = CAP_BLKS_PER_SEC(sbi) - 570 573 get_ckpt_valid_blocks(sbi, segno, true); 571 574 ··· 579 576 580 577 /* check current data section for dentry blocks. */ 581 578 segno = CURSEG_I(sbi, CURSEG_HOT_DATA)->segno; 579 + 580 + if (unlikely(segno == NULL_SEGNO)) 581 + return false; 582 + 582 583 left_blocks = CAP_BLKS_PER_SEC(sbi) - 583 584 get_ckpt_valid_blocks(sbi, segno, true); 584 585 if (dent_blocks > left_blocks)
+91 -1
fs/f2fs/shrinker.c
··· 73 73 mutex_unlock(&sbi->umount_mutex); 74 74 } 75 75 spin_unlock(&f2fs_list_lock); 76 - return count; 76 + return count ?: SHRINK_EMPTY; 77 77 } 78 78 79 79 unsigned long f2fs_shrink_scan(struct shrinker *shrink, ··· 128 128 } 129 129 spin_unlock(&f2fs_list_lock); 130 130 return freed; 131 + } 132 + 133 + unsigned int f2fs_donate_files(void) 134 + { 135 + struct f2fs_sb_info *sbi; 136 + struct list_head *p; 137 + unsigned int donate_files = 0; 138 + 139 + spin_lock(&f2fs_list_lock); 140 + p = f2fs_list.next; 141 + while (p != &f2fs_list) { 142 + sbi = list_entry(p, struct f2fs_sb_info, s_list); 143 + 144 + /* stop f2fs_put_super */ 145 + if (!mutex_trylock(&sbi->umount_mutex)) { 146 + p = p->next; 147 + continue; 148 + } 149 + spin_unlock(&f2fs_list_lock); 150 + 151 + donate_files += sbi->donate_files; 152 + 153 + spin_lock(&f2fs_list_lock); 154 + p = p->next; 155 + mutex_unlock(&sbi->umount_mutex); 156 + } 157 + spin_unlock(&f2fs_list_lock); 158 + 159 + return donate_files; 160 + } 161 + 162 + static unsigned int do_reclaim_caches(struct f2fs_sb_info *sbi, 163 + unsigned int reclaim_caches_kb) 164 + { 165 + struct inode *inode; 166 + struct f2fs_inode_info *fi; 167 + unsigned int nfiles = sbi->donate_files; 168 + pgoff_t npages = reclaim_caches_kb >> (PAGE_SHIFT - 10); 169 + 170 + while (npages && nfiles--) { 171 + pgoff_t len; 172 + 173 + spin_lock(&sbi->inode_lock[DONATE_INODE]); 174 + if (list_empty(&sbi->inode_list[DONATE_INODE])) { 175 + spin_unlock(&sbi->inode_lock[DONATE_INODE]); 176 + break; 177 + } 178 + fi = list_first_entry(&sbi->inode_list[DONATE_INODE], 179 + struct f2fs_inode_info, gdonate_list); 180 + list_move_tail(&fi->gdonate_list, &sbi->inode_list[DONATE_INODE]); 181 + inode = igrab(&fi->vfs_inode); 182 + spin_unlock(&sbi->inode_lock[DONATE_INODE]); 183 + 184 + if (!inode) 185 + continue; 186 + 187 + len = fi->donate_end - fi->donate_start + 1; 188 + npages = npages < len ? 0 : npages - len; 189 + invalidate_inode_pages2_range(inode->i_mapping, 190 + fi->donate_start, fi->donate_end); 191 + iput(inode); 192 + cond_resched(); 193 + } 194 + return npages << (PAGE_SHIFT - 10); 195 + } 196 + 197 + void f2fs_reclaim_caches(unsigned int reclaim_caches_kb) 198 + { 199 + struct f2fs_sb_info *sbi; 200 + struct list_head *p; 201 + 202 + spin_lock(&f2fs_list_lock); 203 + p = f2fs_list.next; 204 + while (p != &f2fs_list && reclaim_caches_kb) { 205 + sbi = list_entry(p, struct f2fs_sb_info, s_list); 206 + 207 + /* stop f2fs_put_super */ 208 + if (!mutex_trylock(&sbi->umount_mutex)) { 209 + p = p->next; 210 + continue; 211 + } 212 + spin_unlock(&f2fs_list_lock); 213 + 214 + reclaim_caches_kb = do_reclaim_caches(sbi, reclaim_caches_kb); 215 + 216 + spin_lock(&f2fs_list_lock); 217 + p = p->next; 218 + mutex_unlock(&sbi->umount_mutex); 219 + } 220 + spin_unlock(&f2fs_list_lock); 131 221 } 132 222 133 223 void f2fs_join_shrinker(struct f2fs_sb_info *sbi)
+132 -59
fs/f2fs/super.c
··· 63 63 [FAULT_BLKADDR_VALIDITY] = "invalid blkaddr", 64 64 [FAULT_BLKADDR_CONSISTENCE] = "inconsistent blkaddr", 65 65 [FAULT_NO_SEGMENT] = "no free segment", 66 + [FAULT_INCONSISTENT_FOOTER] = "inconsistent footer", 66 67 }; 67 68 68 69 int f2fs_build_fault_attr(struct f2fs_sb_info *sbi, unsigned long rate, ··· 191 190 Opt_memory_mode, 192 191 Opt_age_extent_cache, 193 192 Opt_errors, 193 + Opt_nat_bits, 194 194 Opt_err, 195 195 }; 196 196 ··· 271 269 {Opt_memory_mode, "memory=%s"}, 272 270 {Opt_age_extent_cache, "age_extent_cache"}, 273 271 {Opt_errors, "errors=%s"}, 272 + {Opt_nat_bits, "nat_bits"}, 274 273 {Opt_err, NULL}, 275 274 }; 276 275 ··· 386 383 #ifdef CONFIG_QUOTA 387 384 static const char * const quotatypes[] = INITQFNAMES; 388 385 #define QTYPE2NAME(t) (quotatypes[t]) 389 - static int f2fs_set_qf_name(struct super_block *sb, int qtype, 386 + static int f2fs_set_qf_name(struct f2fs_sb_info *sbi, int qtype, 390 387 substring_t *args) 391 388 { 392 - struct f2fs_sb_info *sbi = F2FS_SB(sb); 389 + struct super_block *sb = sbi->sb; 393 390 char *qname; 394 391 int ret = -EINVAL; 395 392 ··· 427 424 return ret; 428 425 } 429 426 430 - static int f2fs_clear_qf_name(struct super_block *sb, int qtype) 427 + static int f2fs_clear_qf_name(struct f2fs_sb_info *sbi, int qtype) 431 428 { 432 - struct f2fs_sb_info *sbi = F2FS_SB(sb); 429 + struct super_block *sb = sbi->sb; 433 430 434 431 if (sb_any_quota_loaded(sb) && F2FS_OPTION(sbi).s_qf_names[qtype]) { 435 432 f2fs_err(sbi, "Cannot change journaled quota options when quota turned on"); ··· 486 483 } 487 484 #endif 488 485 489 - static int f2fs_set_test_dummy_encryption(struct super_block *sb, 486 + static int f2fs_set_test_dummy_encryption(struct f2fs_sb_info *sbi, 490 487 const char *opt, 491 488 const substring_t *arg, 492 489 bool is_remount) 493 490 { 494 - struct f2fs_sb_info *sbi = F2FS_SB(sb); 495 491 struct fs_parameter param = { 496 492 .type = fs_value_is_string, 497 493 .string = arg->from ? arg->from : "", ··· 673 671 #endif 674 672 #endif 675 673 676 - static int parse_options(struct super_block *sb, char *options, bool is_remount) 674 + static int parse_options(struct f2fs_sb_info *sbi, char *options, bool is_remount) 677 675 { 678 - struct f2fs_sb_info *sbi = F2FS_SB(sb); 679 676 substring_t args[MAX_OPT_ARGS]; 680 677 #ifdef CONFIG_F2FS_FS_COMPRESSION 681 678 unsigned char (*ext)[F2FS_EXTENSION_LEN]; ··· 688 687 int ret; 689 688 690 689 if (!options) 691 - goto default_check; 690 + return 0; 692 691 693 692 while ((p = strsep(&options, ",")) != NULL) { 694 693 int token; ··· 729 728 set_opt(sbi, DISABLE_ROLL_FORWARD); 730 729 break; 731 730 case Opt_norecovery: 732 - /* this option mounts f2fs with ro */ 731 + /* requires ro mount, checked in f2fs_default_check */ 733 732 set_opt(sbi, NORECOVERY); 734 - if (!f2fs_readonly(sb)) 735 - return -EINVAL; 736 733 break; 737 734 case Opt_discard: 738 735 if (!f2fs_hw_support_discard(sbi)) { ··· 771 772 break; 772 773 #else 773 774 case Opt_user_xattr: 774 - f2fs_info(sbi, "user_xattr options not supported"); 775 - break; 776 775 case Opt_nouser_xattr: 777 - f2fs_info(sbi, "nouser_xattr options not supported"); 778 - break; 779 776 case Opt_inline_xattr: 780 - f2fs_info(sbi, "inline_xattr options not supported"); 781 - break; 782 777 case Opt_noinline_xattr: 783 - f2fs_info(sbi, "noinline_xattr options not supported"); 778 + case Opt_inline_xattr_size: 779 + f2fs_info(sbi, "xattr options not supported"); 784 780 break; 785 781 #endif 786 782 #ifdef CONFIG_F2FS_FS_POSIX_ACL ··· 787 793 break; 788 794 #else 789 795 case Opt_acl: 790 - f2fs_info(sbi, "acl options not supported"); 791 - break; 792 796 case Opt_noacl: 793 - f2fs_info(sbi, "noacl options not supported"); 797 + f2fs_info(sbi, "acl options not supported"); 794 798 break; 795 799 #endif 796 800 case Opt_active_logs: ··· 830 838 set_opt(sbi, READ_EXTENT_CACHE); 831 839 break; 832 840 case Opt_noextent_cache: 833 - if (F2FS_HAS_FEATURE(sbi, F2FS_FEATURE_DEVICE_ALIAS)) { 841 + if (f2fs_sb_has_device_alias(sbi)) { 834 842 f2fs_err(sbi, "device aliasing requires extent cache"); 835 843 return -EINVAL; 836 844 } ··· 911 919 break; 912 920 #else 913 921 case Opt_fault_injection: 914 - f2fs_info(sbi, "fault_injection options not supported"); 915 - break; 916 - 917 922 case Opt_fault_type: 918 - f2fs_info(sbi, "fault_type options not supported"); 923 + f2fs_info(sbi, "fault injection options not supported"); 919 924 break; 920 925 #endif 921 926 case Opt_lazytime: 922 - sb->s_flags |= SB_LAZYTIME; 927 + set_opt(sbi, LAZYTIME); 923 928 break; 924 929 case Opt_nolazytime: 925 - sb->s_flags &= ~SB_LAZYTIME; 930 + clear_opt(sbi, LAZYTIME); 926 931 break; 927 932 #ifdef CONFIG_QUOTA 928 933 case Opt_quota: ··· 933 944 set_opt(sbi, PRJQUOTA); 934 945 break; 935 946 case Opt_usrjquota: 936 - ret = f2fs_set_qf_name(sb, USRQUOTA, &args[0]); 947 + ret = f2fs_set_qf_name(sbi, USRQUOTA, &args[0]); 937 948 if (ret) 938 949 return ret; 939 950 break; 940 951 case Opt_grpjquota: 941 - ret = f2fs_set_qf_name(sb, GRPQUOTA, &args[0]); 952 + ret = f2fs_set_qf_name(sbi, GRPQUOTA, &args[0]); 942 953 if (ret) 943 954 return ret; 944 955 break; 945 956 case Opt_prjjquota: 946 - ret = f2fs_set_qf_name(sb, PRJQUOTA, &args[0]); 957 + ret = f2fs_set_qf_name(sbi, PRJQUOTA, &args[0]); 947 958 if (ret) 948 959 return ret; 949 960 break; 950 961 case Opt_offusrjquota: 951 - ret = f2fs_clear_qf_name(sb, USRQUOTA); 962 + ret = f2fs_clear_qf_name(sbi, USRQUOTA); 952 963 if (ret) 953 964 return ret; 954 965 break; 955 966 case Opt_offgrpjquota: 956 - ret = f2fs_clear_qf_name(sb, GRPQUOTA); 967 + ret = f2fs_clear_qf_name(sbi, GRPQUOTA); 957 968 if (ret) 958 969 return ret; 959 970 break; 960 971 case Opt_offprjjquota: 961 - ret = f2fs_clear_qf_name(sb, PRJQUOTA); 972 + ret = f2fs_clear_qf_name(sbi, PRJQUOTA); 962 973 if (ret) 963 974 return ret; 964 975 break; ··· 1028 1039 kfree(name); 1029 1040 break; 1030 1041 case Opt_test_dummy_encryption: 1031 - ret = f2fs_set_test_dummy_encryption(sb, p, &args[0], 1042 + ret = f2fs_set_test_dummy_encryption(sbi, p, &args[0], 1032 1043 is_remount); 1033 1044 if (ret) 1034 1045 return ret; 1035 1046 break; 1036 1047 case Opt_inlinecrypt: 1037 1048 #ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT 1038 - sb->s_flags |= SB_INLINECRYPT; 1049 + set_opt(sbi, INLINECRYPT); 1039 1050 #else 1040 1051 f2fs_info(sbi, "inline encryption not supported"); 1041 1052 #endif ··· 1311 1322 } 1312 1323 kfree(name); 1313 1324 break; 1325 + case Opt_nat_bits: 1326 + set_opt(sbi, NAT_BITS); 1327 + break; 1314 1328 default: 1315 1329 f2fs_err(sbi, "Unrecognized mount option \"%s\" or missing value", 1316 1330 p); 1317 1331 return -EINVAL; 1318 1332 } 1319 1333 } 1320 - default_check: 1334 + return 0; 1335 + } 1336 + 1337 + static int f2fs_default_check(struct f2fs_sb_info *sbi) 1338 + { 1321 1339 #ifdef CONFIG_QUOTA 1322 1340 if (f2fs_check_quota_options(sbi)) 1323 1341 return -EINVAL; ··· 1414 1418 f2fs_err(sbi, "Allow to mount readonly mode only"); 1415 1419 return -EROFS; 1416 1420 } 1421 + 1422 + if (test_opt(sbi, NORECOVERY) && !f2fs_readonly(sbi->sb)) { 1423 + f2fs_err(sbi, "norecovery requires readonly mount"); 1424 + return -EINVAL; 1425 + } 1426 + 1417 1427 return 0; 1418 1428 } 1419 1429 ··· 1443 1441 spin_lock_init(&fi->i_size_lock); 1444 1442 INIT_LIST_HEAD(&fi->dirty_list); 1445 1443 INIT_LIST_HEAD(&fi->gdirty_list); 1444 + INIT_LIST_HEAD(&fi->gdonate_list); 1446 1445 init_f2fs_rwsem(&fi->i_gc_rwsem[READ]); 1447 1446 init_f2fs_rwsem(&fi->i_gc_rwsem[WRITE]); 1448 1447 init_f2fs_rwsem(&fi->i_xattr_sem); ··· 1530 1527 inc_page_count(sbi, F2FS_DIRTY_IMETA); 1531 1528 } 1532 1529 spin_unlock(&sbi->inode_lock[DIRTY_META]); 1530 + 1531 + if (!ret && f2fs_is_atomic_file(inode)) 1532 + set_inode_flag(inode, FI_ATOMIC_DIRTIED); 1533 + 1533 1534 return ret; 1534 1535 } 1535 1536 ··· 1744 1737 1745 1738 static int f2fs_freeze(struct super_block *sb) 1746 1739 { 1740 + struct f2fs_sb_info *sbi = F2FS_SB(sb); 1741 + 1747 1742 if (f2fs_readonly(sb)) 1748 1743 return 0; 1749 1744 1750 1745 /* IO error happened before */ 1751 - if (unlikely(f2fs_cp_error(F2FS_SB(sb)))) 1746 + if (unlikely(f2fs_cp_error(sbi))) 1752 1747 return -EIO; 1753 1748 1754 1749 /* must be clean, since sync_filesystem() was already called */ 1755 - if (is_sbi_flag_set(F2FS_SB(sb), SBI_IS_DIRTY)) 1750 + if (is_sbi_flag_set(sbi, SBI_IS_DIRTY)) 1756 1751 return -EINVAL; 1757 1752 1753 + sbi->umount_lock_holder = current; 1754 + 1758 1755 /* Let's flush checkpoints and stop the thread. */ 1759 - f2fs_flush_ckpt_thread(F2FS_SB(sb)); 1756 + f2fs_flush_ckpt_thread(sbi); 1757 + 1758 + sbi->umount_lock_holder = NULL; 1760 1759 1761 1760 /* to avoid deadlock on f2fs_evict_inode->SB_FREEZE_FS */ 1762 - set_sbi_flag(F2FS_SB(sb), SBI_IS_FREEZING); 1761 + set_sbi_flag(sbi, SBI_IS_FREEZING); 1763 1762 return 0; 1764 1763 } 1765 1764 ··· 1849 1836 buf->f_blocks = total_count - start_count; 1850 1837 1851 1838 spin_lock(&sbi->stat_lock); 1852 - 1839 + if (sbi->carve_out) 1840 + buf->f_blocks -= sbi->current_reserved_blocks; 1853 1841 user_block_count = sbi->user_block_count; 1854 1842 total_valid_node_count = valid_node_count(sbi); 1855 1843 avail_node_count = sbi->total_node_count - F2FS_RESERVED_NODE_NUM; ··· 2142 2128 else if (F2FS_OPTION(sbi).errors == MOUNT_ERRORS_PANIC) 2143 2129 seq_printf(seq, ",errors=%s", "panic"); 2144 2130 2131 + if (test_opt(sbi, NAT_BITS)) 2132 + seq_puts(seq, ",nat_bits"); 2133 + 2145 2134 return 0; 2146 2135 } 2147 2136 ··· 2192 2175 set_opt(sbi, INLINE_DATA); 2193 2176 set_opt(sbi, INLINE_DENTRY); 2194 2177 set_opt(sbi, MERGE_CHECKPOINT); 2178 + set_opt(sbi, LAZYTIME); 2195 2179 F2FS_OPTION(sbi).unusable_cap = 0; 2196 - sbi->sb->s_flags |= SB_LAZYTIME; 2197 2180 if (!f2fs_is_readonly(sbi)) 2198 2181 set_opt(sbi, FLUSH_MERGE); 2199 2182 if (f2fs_sb_has_blkzoned(sbi)) ··· 2335 2318 bool no_discard = !test_opt(sbi, DISCARD); 2336 2319 bool no_compress_cache = !test_opt(sbi, COMPRESS_CACHE); 2337 2320 bool block_unit_discard = f2fs_block_unit_discard(sbi); 2321 + bool no_nat_bits = !test_opt(sbi, NAT_BITS); 2338 2322 #ifdef CONFIG_QUOTA 2339 2323 int i, j; 2340 2324 #endif ··· 2346 2328 */ 2347 2329 org_mount_opt = sbi->mount_opt; 2348 2330 old_sb_flags = sb->s_flags; 2331 + 2332 + sbi->umount_lock_holder = current; 2349 2333 2350 2334 #ifdef CONFIG_QUOTA 2351 2335 org_mount_opt.s_jquota_fmt = F2FS_OPTION(sbi).s_jquota_fmt; ··· 2379 2359 default_options(sbi, true); 2380 2360 2381 2361 /* parse mount options */ 2382 - err = parse_options(sb, data, true); 2362 + err = parse_options(sbi, data, true); 2383 2363 if (err) 2384 2364 goto restore_opts; 2385 2365 ··· 2393 2373 goto restore_opts; 2394 2374 } 2395 2375 #endif 2376 + 2377 + err = f2fs_default_check(sbi); 2378 + if (err) 2379 + goto restore_opts; 2396 2380 2397 2381 /* flush outstanding errors before changing fs state */ 2398 2382 flush_work(&sbi->s_error_work); ··· 2465 2441 if (block_unit_discard != f2fs_block_unit_discard(sbi)) { 2466 2442 err = -EINVAL; 2467 2443 f2fs_warn(sbi, "switch discard_unit option is not allowed"); 2444 + goto restore_opts; 2445 + } 2446 + 2447 + if (no_nat_bits == !!test_opt(sbi, NAT_BITS)) { 2448 + err = -EINVAL; 2449 + f2fs_warn(sbi, "switch nat_bits option is not allowed"); 2468 2450 goto restore_opts; 2469 2451 } 2470 2452 ··· 2582 2552 2583 2553 limit_reserve_root(sbi); 2584 2554 *flags = (*flags & ~SB_LAZYTIME) | (sb->s_flags & SB_LAZYTIME); 2555 + 2556 + sbi->umount_lock_holder = NULL; 2585 2557 return 0; 2586 2558 restore_checkpoint: 2587 2559 if (need_enable_checkpoint) { ··· 2624 2592 #endif 2625 2593 sbi->mount_opt = org_mount_opt; 2626 2594 sb->s_flags = old_sb_flags; 2595 + 2596 + sbi->umount_lock_holder = NULL; 2627 2597 return err; 2628 2598 } 2629 2599 ··· 2942 2908 return ret; 2943 2909 } 2944 2910 2945 - int f2fs_quota_sync(struct super_block *sb, int type) 2911 + int f2fs_do_quota_sync(struct super_block *sb, int type) 2946 2912 { 2947 2913 struct f2fs_sb_info *sbi = F2FS_SB(sb); 2948 2914 struct quota_info *dqopt = sb_dqopt(sb); ··· 2990 2956 return ret; 2991 2957 } 2992 2958 2959 + static int f2fs_quota_sync(struct super_block *sb, int type) 2960 + { 2961 + int ret; 2962 + 2963 + F2FS_SB(sb)->umount_lock_holder = current; 2964 + ret = f2fs_do_quota_sync(sb, type); 2965 + F2FS_SB(sb)->umount_lock_holder = NULL; 2966 + return ret; 2967 + } 2968 + 2993 2969 static int f2fs_quota_on(struct super_block *sb, int type, int format_id, 2994 2970 const struct path *path) 2995 2971 { 2996 2972 struct inode *inode; 2997 - int err; 2973 + int err = 0; 2998 2974 2999 2975 /* if quota sysfile exists, deny enabling quota with specific file */ 3000 2976 if (f2fs_sb_has_quota_ino(F2FS_SB(sb))) { ··· 3015 2971 if (path->dentry->d_sb != sb) 3016 2972 return -EXDEV; 3017 2973 3018 - err = f2fs_quota_sync(sb, type); 2974 + F2FS_SB(sb)->umount_lock_holder = current; 2975 + 2976 + err = f2fs_do_quota_sync(sb, type); 3019 2977 if (err) 3020 - return err; 2978 + goto out; 3021 2979 3022 2980 inode = d_inode(path->dentry); 3023 2981 3024 2982 err = filemap_fdatawrite(inode->i_mapping); 3025 2983 if (err) 3026 - return err; 2984 + goto out; 3027 2985 3028 2986 err = filemap_fdatawait(inode->i_mapping); 3029 2987 if (err) 3030 - return err; 2988 + goto out; 3031 2989 3032 2990 err = dquot_quota_on(sb, type, format_id, path); 3033 2991 if (err) 3034 - return err; 2992 + goto out; 3035 2993 3036 2994 inode_lock(inode); 3037 2995 F2FS_I(inode)->i_flags |= F2FS_QUOTA_DEFAULT_FL; 3038 2996 f2fs_set_inode_flags(inode); 3039 2997 inode_unlock(inode); 3040 2998 f2fs_mark_inode_dirty_sync(inode, false); 3041 - 3042 - return 0; 2999 + out: 3000 + F2FS_SB(sb)->umount_lock_holder = NULL; 3001 + return err; 3043 3002 } 3044 3003 3045 3004 static int __f2fs_quota_off(struct super_block *sb, int type) ··· 3053 3006 if (!inode || !igrab(inode)) 3054 3007 return dquot_quota_off(sb, type); 3055 3008 3056 - err = f2fs_quota_sync(sb, type); 3009 + err = f2fs_do_quota_sync(sb, type); 3057 3010 if (err) 3058 3011 goto out_put; 3059 3012 ··· 3076 3029 struct f2fs_sb_info *sbi = F2FS_SB(sb); 3077 3030 int err; 3078 3031 3032 + F2FS_SB(sb)->umount_lock_holder = current; 3033 + 3079 3034 err = __f2fs_quota_off(sb, type); 3080 3035 3081 3036 /* ··· 3087 3038 */ 3088 3039 if (is_journalled_quota(sbi)) 3089 3040 set_sbi_flag(sbi, SBI_QUOTA_NEED_REPAIR); 3041 + 3042 + F2FS_SB(sb)->umount_lock_holder = NULL; 3043 + 3090 3044 return err; 3091 3045 } 3092 3046 ··· 3222 3170 return 0; 3223 3171 } 3224 3172 3225 - int f2fs_quota_sync(struct super_block *sb, int type) 3173 + int f2fs_do_quota_sync(struct super_block *sb, int type) 3226 3174 { 3227 3175 return 0; 3228 3176 } ··· 4272 4220 4273 4221 if (shutdown) 4274 4222 set_sbi_flag(sbi, SBI_IS_SHUTDOWN); 4223 + else 4224 + dump_stack(); 4275 4225 4276 4226 /* 4277 4227 * Continue filesystem operators if errors=continue. Should not set ··· 4549 4495 goto free_sb_buf; 4550 4496 } 4551 4497 4552 - err = parse_options(sb, options, false); 4498 + err = parse_options(sbi, options, false); 4499 + if (err) 4500 + goto free_options; 4501 + 4502 + err = f2fs_default_check(sbi); 4553 4503 if (err) 4554 4504 goto free_options; 4555 4505 ··· 4591 4533 sb->s_time_gran = 1; 4592 4534 sb->s_flags = (sb->s_flags & ~SB_POSIXACL) | 4593 4535 (test_opt(sbi, POSIX_ACL) ? SB_POSIXACL : 0); 4536 + if (test_opt(sbi, INLINECRYPT)) 4537 + sb->s_flags |= SB_INLINECRYPT; 4538 + 4539 + if (test_opt(sbi, LAZYTIME)) 4540 + sb->s_flags |= SB_LAZYTIME; 4541 + else 4542 + sb->s_flags &= ~SB_LAZYTIME; 4543 + 4594 4544 super_set_uuid(sb, (void *) raw_super->uuid, sizeof(raw_super->uuid)); 4595 4545 super_set_sysfs_name_bdev(sb); 4596 4546 sb->s_iflags |= SB_I_CGROUPWB; ··· 4769 4703 if (err) 4770 4704 goto free_compress_inode; 4771 4705 4706 + sbi->umount_lock_holder = current; 4772 4707 #ifdef CONFIG_QUOTA 4773 4708 /* Enable quota usage during mount */ 4774 4709 if (f2fs_sb_has_quota_ino(sbi) && !f2fs_readonly(sb)) { ··· 4785 4718 if (err) 4786 4719 goto free_meta; 4787 4720 4788 - if (unlikely(is_set_ckpt_flags(sbi, CP_DISABLED_FLAG))) 4721 + if (unlikely(is_set_ckpt_flags(sbi, CP_DISABLED_FLAG))) { 4722 + skip_recovery = true; 4789 4723 goto reset_checkpoint; 4724 + } 4790 4725 4791 4726 /* recover fsynced data */ 4792 4727 if (!test_opt(sbi, DISABLE_ROLL_FORWARD) && ··· 4838 4769 } 4839 4770 } 4840 4771 4772 + reset_checkpoint: 4841 4773 #ifdef CONFIG_QUOTA 4842 4774 f2fs_recover_quota_end(sbi, quota_enabled); 4843 4775 #endif 4844 - reset_checkpoint: 4845 4776 /* 4846 4777 * If the f2fs is not readonly and fsync data recovery succeeds, 4847 4778 * write pointer consistency of cursegs and other zones are already ··· 4898 4829 f2fs_update_time(sbi, CP_TIME); 4899 4830 f2fs_update_time(sbi, REQ_TIME); 4900 4831 clear_sbi_flag(sbi, SBI_CP_DISABLED_QUICK); 4832 + 4833 + sbi->umount_lock_holder = NULL; 4901 4834 return 0; 4902 4835 4903 4836 sync_free_meta: ··· 5002 4931 struct f2fs_sb_info *sbi = F2FS_SB(sb); 5003 4932 5004 4933 if (sb->s_root) { 4934 + sbi->umount_lock_holder = current; 4935 + 5005 4936 set_sbi_flag(sbi, SBI_IS_CLOSE); 5006 4937 f2fs_stop_gc_thread(sbi); 5007 4938 f2fs_stop_discard_thread(sbi);
+117 -22
fs/f2fs/sysfs.c
··· 61 61 int id; 62 62 }; 63 63 64 + struct f2fs_base_attr { 65 + struct attribute attr; 66 + ssize_t (*show)(struct f2fs_base_attr *a, char *buf); 67 + ssize_t (*store)(struct f2fs_base_attr *a, const char *buf, size_t len); 68 + }; 69 + 64 70 static ssize_t f2fs_sbi_show(struct f2fs_attr *a, 65 71 struct f2fs_sb_info *sbi, char *buf); 66 72 ··· 868 862 complete(&sbi->s_kobj_unregister); 869 863 } 870 864 865 + static ssize_t f2fs_base_attr_show(struct kobject *kobj, 866 + struct attribute *attr, char *buf) 867 + { 868 + struct f2fs_base_attr *a = container_of(attr, 869 + struct f2fs_base_attr, attr); 870 + 871 + return a->show ? a->show(a, buf) : 0; 872 + } 873 + 874 + static ssize_t f2fs_base_attr_store(struct kobject *kobj, 875 + struct attribute *attr, 876 + const char *buf, size_t len) 877 + { 878 + struct f2fs_base_attr *a = container_of(attr, 879 + struct f2fs_base_attr, attr); 880 + 881 + return a->store ? a->store(a, buf, len) : 0; 882 + } 883 + 871 884 /* 872 885 * Note that there are three feature list entries: 873 886 * 1) /sys/fs/f2fs/features ··· 905 880 * please add new on-disk feature in this list only. 906 881 * - ref. F2FS_SB_FEATURE_RO_ATTR() 907 882 */ 908 - static ssize_t f2fs_feature_show(struct f2fs_attr *a, 909 - struct f2fs_sb_info *sbi, char *buf) 883 + static ssize_t f2fs_feature_show(struct f2fs_base_attr *a, char *buf) 910 884 { 911 885 return sysfs_emit(buf, "supported\n"); 912 886 } 913 887 914 888 #define F2FS_FEATURE_RO_ATTR(_name) \ 915 - static struct f2fs_attr f2fs_attr_##_name = { \ 889 + static struct f2fs_base_attr f2fs_base_attr_##_name = { \ 916 890 .attr = {.name = __stringify(_name), .mode = 0444 }, \ 917 891 .show = f2fs_feature_show, \ 892 + } 893 + 894 + static ssize_t f2fs_tune_show(struct f2fs_base_attr *a, char *buf) 895 + { 896 + unsigned int res = 0; 897 + 898 + if (!strcmp(a->attr.name, "reclaim_caches_kb")) 899 + res = f2fs_donate_files(); 900 + 901 + return sysfs_emit(buf, "%u\n", res); 902 + } 903 + 904 + static ssize_t f2fs_tune_store(struct f2fs_base_attr *a, 905 + const char *buf, size_t count) 906 + { 907 + unsigned long t; 908 + int ret; 909 + 910 + ret = kstrtoul(skip_spaces(buf), 0, &t); 911 + if (ret) 912 + return ret; 913 + 914 + if (!strcmp(a->attr.name, "reclaim_caches_kb")) 915 + f2fs_reclaim_caches(t); 916 + 917 + return count; 918 + } 919 + 920 + #define F2FS_TUNE_RW_ATTR(_name) \ 921 + static struct f2fs_base_attr f2fs_base_attr_##_name = { \ 922 + .attr = {.name = __stringify(_name), .mode = 0644 }, \ 923 + .show = f2fs_tune_show, \ 924 + .store = f2fs_tune_store, \ 918 925 } 919 926 920 927 static ssize_t f2fs_sb_feature_show(struct f2fs_attr *a, ··· 1122 1065 F2FS_SBI_GENERAL_RO_ATTR(unusable_blocks_per_sec); 1123 1066 F2FS_SBI_GENERAL_RW_ATTR(blkzone_alloc_policy); 1124 1067 #endif 1068 + F2FS_SBI_GENERAL_RW_ATTR(carve_out); 1125 1069 1126 1070 /* STAT_INFO ATTR */ 1127 1071 #ifdef CONFIG_F2FS_STAT_FS ··· 1310 1252 ATTR_LIST(warm_data_age_threshold), 1311 1253 ATTR_LIST(last_age_weight), 1312 1254 ATTR_LIST(max_read_extent_count), 1255 + ATTR_LIST(carve_out), 1313 1256 NULL, 1314 1257 }; 1315 1258 ATTRIBUTE_GROUPS(f2fs); 1316 1259 1260 + #define BASE_ATTR_LIST(name) (&f2fs_base_attr_##name.attr) 1317 1261 static struct attribute *f2fs_feat_attrs[] = { 1318 1262 #ifdef CONFIG_FS_ENCRYPTION 1319 - ATTR_LIST(encryption), 1320 - ATTR_LIST(test_dummy_encryption_v2), 1263 + BASE_ATTR_LIST(encryption), 1264 + BASE_ATTR_LIST(test_dummy_encryption_v2), 1321 1265 #if IS_ENABLED(CONFIG_UNICODE) 1322 - ATTR_LIST(encrypted_casefold), 1266 + BASE_ATTR_LIST(encrypted_casefold), 1323 1267 #endif 1324 1268 #endif /* CONFIG_FS_ENCRYPTION */ 1325 1269 #ifdef CONFIG_BLK_DEV_ZONED 1326 - ATTR_LIST(block_zoned), 1270 + BASE_ATTR_LIST(block_zoned), 1327 1271 #endif 1328 - ATTR_LIST(atomic_write), 1329 - ATTR_LIST(extra_attr), 1330 - ATTR_LIST(project_quota), 1331 - ATTR_LIST(inode_checksum), 1332 - ATTR_LIST(flexible_inline_xattr), 1333 - ATTR_LIST(quota_ino), 1334 - ATTR_LIST(inode_crtime), 1335 - ATTR_LIST(lost_found), 1272 + BASE_ATTR_LIST(atomic_write), 1273 + BASE_ATTR_LIST(extra_attr), 1274 + BASE_ATTR_LIST(project_quota), 1275 + BASE_ATTR_LIST(inode_checksum), 1276 + BASE_ATTR_LIST(flexible_inline_xattr), 1277 + BASE_ATTR_LIST(quota_ino), 1278 + BASE_ATTR_LIST(inode_crtime), 1279 + BASE_ATTR_LIST(lost_found), 1336 1280 #ifdef CONFIG_FS_VERITY 1337 - ATTR_LIST(verity), 1281 + BASE_ATTR_LIST(verity), 1338 1282 #endif 1339 - ATTR_LIST(sb_checksum), 1283 + BASE_ATTR_LIST(sb_checksum), 1340 1284 #if IS_ENABLED(CONFIG_UNICODE) 1341 - ATTR_LIST(casefold), 1285 + BASE_ATTR_LIST(casefold), 1342 1286 #endif 1343 - ATTR_LIST(readonly), 1287 + BASE_ATTR_LIST(readonly), 1344 1288 #ifdef CONFIG_F2FS_FS_COMPRESSION 1345 - ATTR_LIST(compression), 1289 + BASE_ATTR_LIST(compression), 1346 1290 #endif 1347 - ATTR_LIST(pin_file), 1291 + BASE_ATTR_LIST(pin_file), 1348 1292 NULL, 1349 1293 }; 1350 1294 ATTRIBUTE_GROUPS(f2fs_feat); ··· 1403 1343 }; 1404 1344 ATTRIBUTE_GROUPS(f2fs_sb_feat); 1405 1345 1346 + F2FS_TUNE_RW_ATTR(reclaim_caches_kb); 1347 + 1348 + static struct attribute *f2fs_tune_attrs[] = { 1349 + BASE_ATTR_LIST(reclaim_caches_kb), 1350 + NULL, 1351 + }; 1352 + ATTRIBUTE_GROUPS(f2fs_tune); 1353 + 1406 1354 static const struct sysfs_ops f2fs_attr_ops = { 1407 1355 .show = f2fs_attr_show, 1408 1356 .store = f2fs_attr_store, ··· 1430 1362 .kobj = {.ktype = &f2fs_ktype}, 1431 1363 }; 1432 1364 1365 + static const struct sysfs_ops f2fs_feat_attr_ops = { 1366 + .show = f2fs_base_attr_show, 1367 + .store = f2fs_base_attr_store, 1368 + }; 1369 + 1433 1370 static const struct kobj_type f2fs_feat_ktype = { 1434 1371 .default_groups = f2fs_feat_groups, 1435 - .sysfs_ops = &f2fs_attr_ops, 1372 + .sysfs_ops = &f2fs_feat_attr_ops, 1436 1373 }; 1437 1374 1438 1375 static struct kobject f2fs_feat = { 1376 + .kset = &f2fs_kset, 1377 + }; 1378 + 1379 + static const struct sysfs_ops f2fs_tune_attr_ops = { 1380 + .show = f2fs_base_attr_show, 1381 + .store = f2fs_base_attr_store, 1382 + }; 1383 + 1384 + static const struct kobj_type f2fs_tune_ktype = { 1385 + .default_groups = f2fs_tune_groups, 1386 + .sysfs_ops = &f2fs_tune_attr_ops, 1387 + }; 1388 + 1389 + static struct kobject f2fs_tune = { 1439 1390 .kset = &f2fs_kset, 1440 1391 }; 1441 1392 ··· 1694 1607 if (ret) 1695 1608 goto put_kobject; 1696 1609 1610 + ret = kobject_init_and_add(&f2fs_tune, &f2fs_tune_ktype, 1611 + NULL, "tuning"); 1612 + if (ret) 1613 + goto put_kobject; 1614 + 1697 1615 f2fs_proc_root = proc_mkdir("fs/f2fs", NULL); 1698 1616 if (!f2fs_proc_root) { 1699 1617 ret = -ENOMEM; ··· 1706 1614 } 1707 1615 1708 1616 return 0; 1617 + 1709 1618 put_kobject: 1619 + kobject_put(&f2fs_tune); 1710 1620 kobject_put(&f2fs_feat); 1711 1621 kset_unregister(&f2fs_kset); 1712 1622 return ret; ··· 1716 1622 1717 1623 void f2fs_exit_sysfs(void) 1718 1624 { 1625 + kobject_put(&f2fs_tune); 1719 1626 kobject_put(&f2fs_feat); 1720 1627 kset_unregister(&f2fs_kset); 1721 1628 remove_proc_entry("fs/f2fs", NULL);
+4 -4
fs/f2fs/xattr.c
··· 282 282 if (ipage) { 283 283 inline_addr = inline_xattr_addr(inode, ipage); 284 284 } else { 285 - page = f2fs_get_node_page(sbi, inode->i_ino); 285 + page = f2fs_get_inode_page(sbi, inode->i_ino); 286 286 if (IS_ERR(page)) 287 287 return PTR_ERR(page); 288 288 ··· 303 303 void *xattr_addr; 304 304 305 305 /* The inode already has an extended attribute block. */ 306 - xpage = f2fs_get_node_page(sbi, xnid); 306 + xpage = f2fs_get_xnode_page(sbi, xnid); 307 307 if (IS_ERR(xpage)) 308 308 return PTR_ERR(xpage); 309 309 ··· 449 449 if (ipage) { 450 450 inline_addr = inline_xattr_addr(inode, ipage); 451 451 } else { 452 - in_page = f2fs_get_node_page(sbi, inode->i_ino); 452 + in_page = f2fs_get_inode_page(sbi, inode->i_ino); 453 453 if (IS_ERR(in_page)) { 454 454 f2fs_alloc_nid_failed(sbi, new_nid); 455 455 return PTR_ERR(in_page); ··· 475 475 476 476 /* write to xattr node block */ 477 477 if (F2FS_I(inode)->i_xattr_nid) { 478 - xpage = f2fs_get_node_page(sbi, F2FS_I(inode)->i_xattr_nid); 478 + xpage = f2fs_get_xnode_page(sbi, F2FS_I(inode)->i_xattr_nid); 479 479 if (IS_ERR(xpage)) { 480 480 err = PTR_ERR(xpage); 481 481 f2fs_alloc_nid_failed(sbi, new_nid);
-9
include/linux/pagemap.h
··· 990 990 unsigned filemap_get_folios_tag(struct address_space *mapping, pgoff_t *start, 991 991 pgoff_t end, xa_mark_t tag, struct folio_batch *fbatch); 992 992 993 - struct page *grab_cache_page_write_begin(struct address_space *mapping, 994 - pgoff_t index); 995 - 996 993 /* 997 994 * Returns locked page at given index in given cache, creating it if needed. 998 995 */ ··· 1244 1247 return folio_wait_bit_killable(folio, PG_locked); 1245 1248 } 1246 1249 1247 - static inline void wait_on_page_locked(struct page *page) 1248 - { 1249 - folio_wait_locked(page_folio(page)); 1250 - } 1251 - 1252 1250 void folio_end_read(struct folio *folio, bool success); 1253 1251 void wait_on_page_writeback(struct page *page); 1254 1252 void folio_wait_writeback(struct folio *folio); 1255 1253 int folio_wait_writeback_killable(struct folio *folio); 1256 1254 void end_page_writeback(struct page *page); 1257 1255 void folio_end_writeback(struct folio *folio); 1258 - void wait_for_stable_page(struct page *page); 1259 1256 void folio_wait_stable(struct folio *folio); 1260 1257 void __folio_mark_dirty(struct folio *folio, struct address_space *, int warn); 1261 1258 void folio_account_cleaned(struct folio *folio, struct bdi_writeback *wb);
+7
include/uapi/linux/f2fs.h
··· 44 44 #define F2FS_IOC_COMPRESS_FILE _IO(F2FS_IOCTL_MAGIC, 24) 45 45 #define F2FS_IOC_START_ATOMIC_REPLACE _IO(F2FS_IOCTL_MAGIC, 25) 46 46 #define F2FS_IOC_GET_DEV_ALIAS_FILE _IOR(F2FS_IOCTL_MAGIC, 26, __u32) 47 + #define F2FS_IOC_IO_PRIO _IOW(F2FS_IOCTL_MAGIC, 27, __u32) 47 48 48 49 /* 49 50 * should be same as XFS_IOC_GOINGDOWN. ··· 63 62 #define F2FS_TRIM_FILE_DISCARD 0x1 /* send discard command */ 64 63 #define F2FS_TRIM_FILE_ZEROOUT 0x2 /* zero out */ 65 64 #define F2FS_TRIM_FILE_MASK 0x3 65 + 66 + /* for F2FS_IOC_IO_PRIO */ 67 + enum { 68 + F2FS_IOPRIO_WRITE = 1, /* high write priority */ 69 + F2FS_IOPRIO_MAX, 70 + }; 66 71 67 72 struct f2fs_gc_range { 68 73 __u32 sync;
+1 -1
mm/filemap.c
··· 1390 1390 * @ptl: already locked ptl. This function will drop the lock. 1391 1391 * 1392 1392 * Wait for a migration entry referencing the given page to be removed. This is 1393 - * equivalent to put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE) except 1393 + * equivalent to folio_put_wait_locked(folio, TASK_UNINTERRUPTIBLE) except 1394 1394 * this can be called without taking a reference on the page. Instead this 1395 1395 * should be called while holding the ptl for the migration entry referencing 1396 1396 * the page.
-14
mm/folio-compat.c
··· 28 28 } 29 29 EXPORT_SYMBOL_GPL(wait_on_page_writeback); 30 30 31 - void wait_for_stable_page(struct page *page) 32 - { 33 - return folio_wait_stable(page_folio(page)); 34 - } 35 - EXPORT_SYMBOL_GPL(wait_for_stable_page); 36 - 37 31 void mark_page_accessed(struct page *page) 38 32 { 39 33 folio_mark_accessed(page_folio(page)); ··· 84 90 return folio_file_page(folio, index); 85 91 } 86 92 EXPORT_SYMBOL(pagecache_get_page); 87 - 88 - struct page *grab_cache_page_write_begin(struct address_space *mapping, 89 - pgoff_t index) 90 - { 91 - return pagecache_get_page(mapping, index, FGP_WRITEBEGIN, 92 - mapping_gfp_mask(mapping)); 93 - } 94 - EXPORT_SYMBOL(grab_cache_page_write_begin);