Merge tag 'f2fs-for-4.14' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs

Pull f2fs updates from Jaegeuk Kim:
"In this round, we've mostly tuned f2fs to provide better user
experience for Android. Especially, we've worked on atomic write
feature again with SQLite community in order to support it officially.
And we added or modified several facilities to analyze and enhance IO
behaviors.

Major changes include:
- add app/fs io stat
- add inode checksum feature
- support project/journalled quota
- enhance atomic write with new ioctl() which exposes feature set
- enhance background gc/discard/fstrim flows with new gc_urgent mode
- add F2FS_IOC_FS{GET,SET}XATTR
- fix some quota flows"

* tag 'f2fs-for-4.14' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs: (63 commits)
f2fs: hurry up to issue discard after io interruption
f2fs: fix to show correct discard_granularity in sysfs
f2fs: detect dirty inode in evict_inode
f2fs: clear radix tree dirty tag of pages whose dirty flag is cleared
f2fs: speed up gc_urgent mode with SSR
f2fs: better to wait for fstrim completion
f2fs: avoid race in between read xattr & write xattr
f2fs: make get_lock_data_page to handle encrypted inode
f2fs: use generic terms used for encrypted block management
f2fs: introduce f2fs_encrypted_file for clean-up
Revert "f2fs: add a new function get_ssr_cost"
f2fs: constify super_operations
f2fs: fix to wake up all sleeping flusher
f2fs: avoid race in between atomic_read & atomic_inc
f2fs: remove unneeded parameter of change_curseg
f2fs: update i_flags correctly
f2fs: don't check inode's checksum if it was dirtied or writebacked
f2fs: don't need to update inode checksum for recovery
f2fs: trigger fdatasync for non-atomic_write file
f2fs: fix to avoid race in between aio and gc
...

+2230 -513
+21
Documentation/ABI/testing/sysfs-fs-f2fs
··· 57 Description: 58 Controls the issue rate of small discard commands. 59 60 What: /sys/fs/f2fs/<disk>/max_victim_search 61 Date: January 2014 62 Contact: "Jaegeuk Kim" <jaegeuk.kim@samsung.com> ··· 139 Contact: "Chao Yu" <yuchao0@huawei.com> 140 Description: 141 Controls current reserved blocks in system.
··· 57 Description: 58 Controls the issue rate of small discard commands. 59 60 + What: /sys/fs/f2fs/<disk>/discard_granularity 61 + Date: July 2017 62 + Contact: "Chao Yu" <yuchao0@huawei.com> 63 + Description: 64 + Controls discard granularity of inner discard thread, inner thread 65 + will not issue discards with size that is smaller than granularity. 66 + The unit size is one block, now only support configuring in range 67 + of [1, 512]. 68 + 69 What: /sys/fs/f2fs/<disk>/max_victim_search 70 Date: January 2014 71 Contact: "Jaegeuk Kim" <jaegeuk.kim@samsung.com> ··· 130 Contact: "Chao Yu" <yuchao0@huawei.com> 131 Description: 132 Controls current reserved blocks in system. 133 + 134 + What: /sys/fs/f2fs/<disk>/gc_urgent 135 + Date: August 2017 136 + Contact: "Jaegeuk Kim" <jaegeuk@kernel.org> 137 + Description: 138 + Do background GC agressively 139 + 140 + What: /sys/fs/f2fs/<disk>/gc_urgent_sleep_time 141 + Date: August 2017 142 + Contact: "Jaegeuk Kim" <jaegeuk@kernel.org> 143 + Description: 144 + Controls sleep time of GC urgent mode
+19
Documentation/filesystems/f2fs.txt
··· 164 with "mode=lfs". 165 usrquota Enable plain user disk quota accounting. 166 grpquota Enable plain group disk quota accounting. 167 168 ================================================================================ 169 DEBUGFS ENTRIES ··· 218 (default) will disable this option. Setting 219 gc_idle = 1 will select the Cost Benefit approach 220 & setting gc_idle = 2 will select the greedy approach. 221 222 reclaim_segments This parameter controls the number of prefree 223 segments to be reclaimed. If the number of prefree
··· 164 with "mode=lfs". 165 usrquota Enable plain user disk quota accounting. 166 grpquota Enable plain group disk quota accounting. 167 + prjquota Enable plain project quota accounting. 168 + usrjquota=<file> Appoint specified file and type during mount, so that quota 169 + grpjquota=<file> information can be properly updated during recovery flow, 170 + prjjquota=<file> <quota file>: must be in root directory; 171 + jqfmt=<quota type> <quota type>: [vfsold,vfsv0,vfsv1]. 172 + offusrjquota Turn off user journelled quota. 173 + offgrpjquota Turn off group journelled quota. 174 + offprjjquota Turn off project journelled quota. 175 + quota Enable plain user disk quota accounting. 176 + noquota Disable all plain disk quota option. 177 178 ================================================================================ 179 DEBUGFS ENTRIES ··· 208 (default) will disable this option. Setting 209 gc_idle = 1 will select the Cost Benefit approach 210 & setting gc_idle = 2 will select the greedy approach. 211 + 212 + gc_urgent This parameter controls triggering background GCs 213 + urgently or not. Setting gc_urgent = 0 [default] 214 + makes back to default behavior, while if it is set 215 + to 1, background thread starts to do GC by given 216 + gc_urgent_sleep_time interval. 217 + 218 + gc_urgent_sleep_time This parameter controls sleep time for gc_urgent. 219 + 500 ms is set by default. See above gc_urgent. 220 221 reclaim_segments This parameter controls the number of prefree 222 segments to be reclaimed. If the number of prefree
+3 -2
fs/f2fs/acl.c
··· 207 void *value = NULL; 208 size_t size = 0; 209 int error; 210 211 switch (type) { 212 case ACL_TYPE_ACCESS: 213 name_index = F2FS_XATTR_INDEX_POSIX_ACL_ACCESS; 214 if (acl && !ipage) { 215 - error = posix_acl_update_mode(inode, &inode->i_mode, &acl); 216 if (error) 217 return error; 218 - set_acl_inode(inode, inode->i_mode); 219 } 220 break; 221
··· 207 void *value = NULL; 208 size_t size = 0; 209 int error; 210 + umode_t mode = inode->i_mode; 211 212 switch (type) { 213 case ACL_TYPE_ACCESS: 214 name_index = F2FS_XATTR_INDEX_POSIX_ACL_ACCESS; 215 if (acl && !ipage) { 216 + error = posix_acl_update_mode(inode, &mode, &acl); 217 if (error) 218 return error; 219 + set_acl_inode(inode, mode); 220 } 221 break; 222
+47 -13
fs/f2fs/checkpoint.c
··· 230 ra_meta_pages(sbi, index, BIO_MAX_PAGES, META_POR, true); 231 } 232 233 - static int f2fs_write_meta_page(struct page *page, 234 - struct writeback_control *wbc) 235 { 236 struct f2fs_sb_info *sbi = F2FS_P_SB(page); 237 ··· 245 if (unlikely(f2fs_cp_error(sbi))) 246 goto redirty_out; 247 248 - write_meta_page(sbi, page); 249 dec_page_count(sbi, F2FS_DIRTY_META); 250 251 if (wbc->for_reclaim) ··· 262 redirty_out: 263 redirty_page_for_writepage(wbc, page); 264 return AOP_WRITEPAGE_ACTIVATE; 265 } 266 267 static int f2fs_write_meta_pages(struct address_space *mapping, ··· 290 291 trace_f2fs_writepages(mapping->host, wbc, META); 292 diff = nr_pages_to_write(sbi, META, wbc); 293 - written = sync_meta_pages(sbi, META, wbc->nr_to_write); 294 mutex_unlock(&sbi->cp_mutex); 295 wbc->nr_to_write = max((long)0, wbc->nr_to_write - written - diff); 296 return 0; ··· 302 } 303 304 long sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type, 305 - long nr_to_write) 306 { 307 struct address_space *mapping = META_MAPPING(sbi); 308 pgoff_t index = 0, end = ULONG_MAX, prev = ULONG_MAX; ··· 353 if (!clear_page_dirty_for_io(page)) 354 goto continue_unlock; 355 356 - if (mapping->a_ops->writepage(page, &wbc)) { 357 unlock_page(page); 358 break; 359 } ··· 588 int recover_orphan_inodes(struct f2fs_sb_info *sbi) 589 { 590 block_t start_blk, orphan_blocks, i, j; 591 - int err; 592 593 if (!is_set_ckpt_flags(sbi, CP_ORPHAN_PRESENT_FLAG)) 594 return 0; 595 596 start_blk = __start_cp_addr(sbi) + 1 + __cp_payload(sbi); 597 orphan_blocks = __start_sum_addr(sbi) - 1 - __cp_payload(sbi); ··· 621 err = recover_orphan_inode(sbi, ino); 622 if (err) { 623 f2fs_put_page(page, 1); 624 - return err; 625 } 626 } 627 f2fs_put_page(page, 1); 628 } 629 /* clear Orphan Flag */ 630 clear_ckpt_flags(sbi, CP_ORPHAN_PRESENT_FLAG); 631 - return 0; 632 } 633 634 static void write_orphan_inodes(struct f2fs_sb_info *sbi, block_t start_blk) ··· 931 if (inode) { 932 unsigned long cur_ino = inode->i_ino; 933 934 filemap_fdatawrite(inode->i_mapping); 935 iput(inode); 936 /* We need to give cpu to another writers. */ 937 if (ino == cur_ino) { ··· 1051 1052 if (get_pages(sbi, F2FS_DIRTY_NODES)) { 1053 up_write(&sbi->node_write); 1054 - err = sync_node_pages(sbi, &wbc); 1055 if (err) { 1056 up_write(&sbi->node_change); 1057 f2fs_unlock_all(sbi); ··· 1149 1150 /* Flush all the NAT/SIT pages */ 1151 while (get_pages(sbi, F2FS_DIRTY_META)) { 1152 - sync_meta_pages(sbi, META, LONG_MAX); 1153 if (unlikely(f2fs_cp_error(sbi))) 1154 return -EIO; 1155 } ··· 1228 1229 /* Flush all the NAT BITS pages */ 1230 while (get_pages(sbi, F2FS_DIRTY_META)) { 1231 - sync_meta_pages(sbi, META, LONG_MAX); 1232 if (unlikely(f2fs_cp_error(sbi))) 1233 return -EIO; 1234 } ··· 1283 percpu_counter_set(&sbi->alloc_valid_block_count, 0); 1284 1285 /* Here, we only have one bio having CP pack */ 1286 - sync_meta_pages(sbi, META_FLUSH, LONG_MAX); 1287 1288 /* wait for previous submitted meta pages writeback */ 1289 wait_on_all_pages_writeback(sbi);
··· 230 ra_meta_pages(sbi, index, BIO_MAX_PAGES, META_POR, true); 231 } 232 233 + static int __f2fs_write_meta_page(struct page *page, 234 + struct writeback_control *wbc, 235 + enum iostat_type io_type) 236 { 237 struct f2fs_sb_info *sbi = F2FS_P_SB(page); 238 ··· 244 if (unlikely(f2fs_cp_error(sbi))) 245 goto redirty_out; 246 247 + write_meta_page(sbi, page, io_type); 248 dec_page_count(sbi, F2FS_DIRTY_META); 249 250 if (wbc->for_reclaim) ··· 261 redirty_out: 262 redirty_page_for_writepage(wbc, page); 263 return AOP_WRITEPAGE_ACTIVATE; 264 + } 265 + 266 + static int f2fs_write_meta_page(struct page *page, 267 + struct writeback_control *wbc) 268 + { 269 + return __f2fs_write_meta_page(page, wbc, FS_META_IO); 270 } 271 272 static int f2fs_write_meta_pages(struct address_space *mapping, ··· 283 284 trace_f2fs_writepages(mapping->host, wbc, META); 285 diff = nr_pages_to_write(sbi, META, wbc); 286 + written = sync_meta_pages(sbi, META, wbc->nr_to_write, FS_META_IO); 287 mutex_unlock(&sbi->cp_mutex); 288 wbc->nr_to_write = max((long)0, wbc->nr_to_write - written - diff); 289 return 0; ··· 295 } 296 297 long sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type, 298 + long nr_to_write, enum iostat_type io_type) 299 { 300 struct address_space *mapping = META_MAPPING(sbi); 301 pgoff_t index = 0, end = ULONG_MAX, prev = ULONG_MAX; ··· 346 if (!clear_page_dirty_for_io(page)) 347 goto continue_unlock; 348 349 + if (__f2fs_write_meta_page(page, &wbc, io_type)) { 350 unlock_page(page); 351 break; 352 } ··· 581 int recover_orphan_inodes(struct f2fs_sb_info *sbi) 582 { 583 block_t start_blk, orphan_blocks, i, j; 584 + unsigned int s_flags = sbi->sb->s_flags; 585 + int err = 0; 586 587 if (!is_set_ckpt_flags(sbi, CP_ORPHAN_PRESENT_FLAG)) 588 return 0; 589 + 590 + if (s_flags & MS_RDONLY) { 591 + f2fs_msg(sbi->sb, KERN_INFO, "orphan cleanup on readonly fs"); 592 + sbi->sb->s_flags &= ~MS_RDONLY; 593 + } 594 + 595 + #ifdef CONFIG_QUOTA 596 + /* Needed for iput() to work correctly and not trash data */ 597 + sbi->sb->s_flags |= MS_ACTIVE; 598 + /* Turn on quotas so that they are updated correctly */ 599 + f2fs_enable_quota_files(sbi); 600 + #endif 601 602 start_blk = __start_cp_addr(sbi) + 1 + __cp_payload(sbi); 603 orphan_blocks = __start_sum_addr(sbi) - 1 - __cp_payload(sbi); ··· 601 err = recover_orphan_inode(sbi, ino); 602 if (err) { 603 f2fs_put_page(page, 1); 604 + goto out; 605 } 606 } 607 f2fs_put_page(page, 1); 608 } 609 /* clear Orphan Flag */ 610 clear_ckpt_flags(sbi, CP_ORPHAN_PRESENT_FLAG); 611 + out: 612 + #ifdef CONFIG_QUOTA 613 + /* Turn quotas off */ 614 + f2fs_quota_off_umount(sbi->sb); 615 + #endif 616 + sbi->sb->s_flags = s_flags; /* Restore MS_RDONLY status */ 617 + 618 + return err; 619 } 620 621 static void write_orphan_inodes(struct f2fs_sb_info *sbi, block_t start_blk) ··· 904 if (inode) { 905 unsigned long cur_ino = inode->i_ino; 906 907 + if (is_dir) 908 + F2FS_I(inode)->cp_task = current; 909 + 910 filemap_fdatawrite(inode->i_mapping); 911 + 912 + if (is_dir) 913 + F2FS_I(inode)->cp_task = NULL; 914 + 915 iput(inode); 916 /* We need to give cpu to another writers. */ 917 if (ino == cur_ino) { ··· 1017 1018 if (get_pages(sbi, F2FS_DIRTY_NODES)) { 1019 up_write(&sbi->node_write); 1020 + err = sync_node_pages(sbi, &wbc, false, FS_CP_NODE_IO); 1021 if (err) { 1022 up_write(&sbi->node_change); 1023 f2fs_unlock_all(sbi); ··· 1115 1116 /* Flush all the NAT/SIT pages */ 1117 while (get_pages(sbi, F2FS_DIRTY_META)) { 1118 + sync_meta_pages(sbi, META, LONG_MAX, FS_CP_META_IO); 1119 if (unlikely(f2fs_cp_error(sbi))) 1120 return -EIO; 1121 } ··· 1194 1195 /* Flush all the NAT BITS pages */ 1196 while (get_pages(sbi, F2FS_DIRTY_META)) { 1197 + sync_meta_pages(sbi, META, LONG_MAX, FS_CP_META_IO); 1198 if (unlikely(f2fs_cp_error(sbi))) 1199 return -EIO; 1200 } ··· 1249 percpu_counter_set(&sbi->alloc_valid_block_count, 0); 1250 1251 /* Here, we only have one bio having CP pack */ 1252 + sync_meta_pages(sbi, META_FLUSH, LONG_MAX, FS_CP_META_IO); 1253 1254 /* wait for previous submitted meta pages writeback */ 1255 wait_on_all_pages_writeback(sbi);
+96 -81
fs/f2fs/data.c
··· 457 return err; 458 } 459 460 static void __set_data_blkaddr(struct dnode_of_data *dn) 461 { 462 struct f2fs_node *rn = F2FS_NODE(dn->node_page); 463 __le32 *addr_array; 464 465 /* Get physical address of data block */ 466 addr_array = blkaddr_in_node(rn); 467 - addr_array[dn->ofs_in_node] = cpu_to_le32(dn->data_blkaddr); 468 } 469 470 /* ··· 559 f2fs_wait_on_page_writeback(dn->node_page, NODE, true); 560 561 for (; count > 0; dn->ofs_in_node++) { 562 - block_t blkaddr = 563 - datablock_addr(dn->node_page, dn->ofs_in_node); 564 if (blkaddr == NULL_ADDR) { 565 dn->data_blkaddr = NEW_ADDR; 566 __set_data_blkaddr(dn); ··· 621 struct page *page; 622 struct extent_info ei = {0,0,0}; 623 int err; 624 - struct f2fs_io_info fio = { 625 - .sbi = F2FS_I_SB(inode), 626 - .type = DATA, 627 - .op = REQ_OP_READ, 628 - .op_flags = op_flags, 629 - .encrypted_page = NULL, 630 - }; 631 - 632 - if (f2fs_encrypted_inode(inode) && S_ISREG(inode->i_mode)) 633 - return read_mapping_page(mapping, index, NULL); 634 635 page = f2fs_grab_cache_page(mapping, index, for_write); 636 if (!page) ··· 661 return page; 662 } 663 664 - fio.new_blkaddr = fio.old_blkaddr = dn.data_blkaddr; 665 - fio.page = page; 666 - err = f2fs_submit_page_bio(&fio); 667 if (err) 668 goto put_err; 669 return page; ··· 795 if (unlikely(is_inode_flag_set(dn->inode, FI_NO_ALLOC))) 796 return -EPERM; 797 798 - dn->data_blkaddr = datablock_addr(dn->node_page, dn->ofs_in_node); 799 if (dn->data_blkaddr == NEW_ADDR) 800 goto alloc; 801 ··· 822 823 static inline bool __force_buffered_io(struct inode *inode, int rw) 824 { 825 - return ((f2fs_encrypted_inode(inode) && S_ISREG(inode->i_mode)) || 826 (rw == WRITE && test_opt(F2FS_I_SB(inode), LFS)) || 827 F2FS_I_SB(inode)->s_ndevs); 828 } ··· 854 F2FS_GET_BLOCK_PRE_AIO : 855 F2FS_GET_BLOCK_PRE_DIO); 856 } 857 - if (iocb->ki_pos + iov_iter_count(from) > MAX_INLINE_DATA) { 858 err = f2fs_convert_inline_inode(inode); 859 if (err) 860 return err; ··· 943 end_offset = ADDRS_PER_PAGE(dn.node_page, inode); 944 945 next_block: 946 - blkaddr = datablock_addr(dn.node_page, dn.ofs_in_node); 947 948 if (blkaddr == NEW_ADDR || blkaddr == NULL_ADDR) { 949 if (create) { ··· 1080 struct buffer_head *bh_result, int create) 1081 { 1082 return __get_data_block(inode, iblock, bh_result, create, 1083 - F2FS_GET_BLOCK_DIO, NULL); 1084 } 1085 1086 static int get_data_block_bmap(struct inode *inode, sector_t iblock, ··· 1186 return ret; 1187 } 1188 1189 - static struct bio *f2fs_grab_bio(struct inode *inode, block_t blkaddr, 1190 - unsigned nr_pages) 1191 - { 1192 - struct f2fs_sb_info *sbi = F2FS_I_SB(inode); 1193 - struct fscrypt_ctx *ctx = NULL; 1194 - struct bio *bio; 1195 - 1196 - if (f2fs_encrypted_inode(inode) && S_ISREG(inode->i_mode)) { 1197 - ctx = fscrypt_get_ctx(inode, GFP_NOFS); 1198 - if (IS_ERR(ctx)) 1199 - return ERR_CAST(ctx); 1200 - 1201 - /* wait the page to be moved by cleaning */ 1202 - f2fs_wait_on_encrypted_page_writeback(sbi, blkaddr); 1203 - } 1204 - 1205 - bio = bio_alloc(GFP_KERNEL, min_t(int, nr_pages, BIO_MAX_PAGES)); 1206 - if (!bio) { 1207 - if (ctx) 1208 - fscrypt_release_ctx(ctx); 1209 - return ERR_PTR(-ENOMEM); 1210 - } 1211 - f2fs_target_device(sbi, blkaddr, bio); 1212 - bio->bi_end_io = f2fs_read_end_io; 1213 - bio->bi_private = ctx; 1214 - 1215 - return bio; 1216 - } 1217 - 1218 /* 1219 * This function was originally taken from fs/mpage.c, and customized for f2fs. 1220 * Major change was from block_size == page_size in f2fs by default. ··· 1251 map.m_len = last_block - block_in_file; 1252 1253 if (f2fs_map_blocks(inode, &map, 0, 1254 - F2FS_GET_BLOCK_READ)) 1255 goto set_error_page; 1256 } 1257 got_it: ··· 1282 bio = NULL; 1283 } 1284 if (bio == NULL) { 1285 - bio = f2fs_grab_bio(inode, block_nr, nr_pages); 1286 if (IS_ERR(bio)) { 1287 bio = NULL; 1288 goto set_error_page; 1289 } 1290 - bio_set_op_attrs(bio, REQ_OP_READ, 0); 1291 } 1292 1293 if (bio_add_page(bio, page, blocksize, 0) < blocksize) ··· 1351 struct inode *inode = fio->page->mapping->host; 1352 gfp_t gfp_flags = GFP_NOFS; 1353 1354 - if (!f2fs_encrypted_inode(inode) || !S_ISREG(inode->i_mode)) 1355 return 0; 1356 1357 /* wait for GCed encrypted page writeback */ 1358 - f2fs_wait_on_encrypted_page_writeback(fio->sbi, fio->old_blkaddr); 1359 1360 retry_encrypt: 1361 fio->encrypted_page = fscrypt_encrypt_page(inode, fio->page, ··· 1481 } 1482 1483 static int __write_data_page(struct page *page, bool *submitted, 1484 - struct writeback_control *wbc) 1485 { 1486 struct inode *inode = page->mapping->host; 1487 struct f2fs_sb_info *sbi = F2FS_I_SB(inode); ··· 1503 .encrypted_page = NULL, 1504 .submitted = false, 1505 .need_lock = LOCK_RETRY, 1506 }; 1507 1508 trace_f2fs_writepage(page, DATA); ··· 1610 static int f2fs_write_data_page(struct page *page, 1611 struct writeback_control *wbc) 1612 { 1613 - return __write_data_page(page, NULL, wbc); 1614 } 1615 1616 /* ··· 1619 * warm/hot data page. 1620 */ 1621 static int f2fs_write_cache_pages(struct address_space *mapping, 1622 - struct writeback_control *wbc) 1623 { 1624 int ret = 0; 1625 int done = 0; ··· 1710 if (!clear_page_dirty_for_io(page)) 1711 goto continue_unlock; 1712 1713 - ret = __write_data_page(page, &submitted, wbc); 1714 if (unlikely(ret)) { 1715 /* 1716 * keep nr_to_write, since vfs uses this to ··· 1765 return ret; 1766 } 1767 1768 - static int f2fs_write_data_pages(struct address_space *mapping, 1769 - struct writeback_control *wbc) 1770 { 1771 struct inode *inode = mapping->host; 1772 struct f2fs_sb_info *sbi = F2FS_I_SB(inode); ··· 1804 goto skip_write; 1805 1806 blk_start_plug(&plug); 1807 - ret = f2fs_write_cache_pages(mapping, wbc); 1808 blk_finish_plug(&plug); 1809 1810 if (wbc->sync_mode == WB_SYNC_ALL) ··· 1821 wbc->pages_skipped += get_dirty_pages(inode); 1822 trace_f2fs_writepages(mapping->host, wbc, DATA); 1823 return 0; 1824 } 1825 1826 static void f2fs_write_failed(struct address_space *mapping, loff_t to) ··· 1882 set_new_dnode(&dn, inode, ipage, ipage, 0); 1883 1884 if (f2fs_has_inline_data(inode)) { 1885 - if (pos + len <= MAX_INLINE_DATA) { 1886 read_inline_data(page, ipage); 1887 set_inode_flag(inode, FI_DATA_EXIST); 1888 if (inode->i_nlink) ··· 1980 f2fs_wait_on_page_writeback(page, DATA, false); 1981 1982 /* wait for GCed encrypted page writeback */ 1983 - if (f2fs_encrypted_inode(inode) && S_ISREG(inode->i_mode)) 1984 - f2fs_wait_on_encrypted_page_writeback(sbi, blkaddr); 1985 1986 if (len == PAGE_SIZE || PageUptodate(page)) 1987 return 0; ··· 1995 zero_user_segment(page, 0, PAGE_SIZE); 1996 SetPageUptodate(page); 1997 } else { 1998 - struct bio *bio; 1999 - 2000 - bio = f2fs_grab_bio(inode, blkaddr, 1); 2001 - if (IS_ERR(bio)) { 2002 - err = PTR_ERR(bio); 2003 goto fail; 2004 - } 2005 - bio->bi_opf = REQ_OP_READ; 2006 - if (bio_add_page(bio, page, PAGE_SIZE, 0) < PAGE_SIZE) { 2007 - bio_put(bio); 2008 - err = -EFAULT; 2009 - goto fail; 2010 - } 2011 - 2012 - __submit_bio(sbi, bio, DATA); 2013 2014 lock_page(page); 2015 if (unlikely(page->mapping != mapping)) { ··· 2087 up_read(&F2FS_I(inode)->dio_rwsem[rw]); 2088 2089 if (rw == WRITE) { 2090 - if (err > 0) 2091 set_inode_flag(inode, FI_UPDATE_WRITE); 2092 - else if (err < 0) 2093 f2fs_write_failed(mapping, offset + count); 2094 } 2095 2096 trace_f2fs_direct_IO_exit(inode, offset, count, rw, err);
··· 457 return err; 458 } 459 460 + static struct bio *f2fs_grab_read_bio(struct inode *inode, block_t blkaddr, 461 + unsigned nr_pages) 462 + { 463 + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); 464 + struct fscrypt_ctx *ctx = NULL; 465 + struct bio *bio; 466 + 467 + if (f2fs_encrypted_file(inode)) { 468 + ctx = fscrypt_get_ctx(inode, GFP_NOFS); 469 + if (IS_ERR(ctx)) 470 + return ERR_CAST(ctx); 471 + 472 + /* wait the page to be moved by cleaning */ 473 + f2fs_wait_on_block_writeback(sbi, blkaddr); 474 + } 475 + 476 + bio = bio_alloc(GFP_KERNEL, min_t(int, nr_pages, BIO_MAX_PAGES)); 477 + if (!bio) { 478 + if (ctx) 479 + fscrypt_release_ctx(ctx); 480 + return ERR_PTR(-ENOMEM); 481 + } 482 + f2fs_target_device(sbi, blkaddr, bio); 483 + bio->bi_end_io = f2fs_read_end_io; 484 + bio->bi_private = ctx; 485 + bio_set_op_attrs(bio, REQ_OP_READ, 0); 486 + 487 + return bio; 488 + } 489 + 490 + /* This can handle encryption stuffs */ 491 + static int f2fs_submit_page_read(struct inode *inode, struct page *page, 492 + block_t blkaddr) 493 + { 494 + struct bio *bio = f2fs_grab_read_bio(inode, blkaddr, 1); 495 + 496 + if (IS_ERR(bio)) 497 + return PTR_ERR(bio); 498 + 499 + if (bio_add_page(bio, page, PAGE_SIZE, 0) < PAGE_SIZE) { 500 + bio_put(bio); 501 + return -EFAULT; 502 + } 503 + __submit_bio(F2FS_I_SB(inode), bio, DATA); 504 + return 0; 505 + } 506 + 507 static void __set_data_blkaddr(struct dnode_of_data *dn) 508 { 509 struct f2fs_node *rn = F2FS_NODE(dn->node_page); 510 __le32 *addr_array; 511 + int base = 0; 512 + 513 + if (IS_INODE(dn->node_page) && f2fs_has_extra_attr(dn->inode)) 514 + base = get_extra_isize(dn->inode); 515 516 /* Get physical address of data block */ 517 addr_array = blkaddr_in_node(rn); 518 + addr_array[base + dn->ofs_in_node] = cpu_to_le32(dn->data_blkaddr); 519 } 520 521 /* ··· 508 f2fs_wait_on_page_writeback(dn->node_page, NODE, true); 509 510 for (; count > 0; dn->ofs_in_node++) { 511 + block_t blkaddr = datablock_addr(dn->inode, 512 + dn->node_page, dn->ofs_in_node); 513 if (blkaddr == NULL_ADDR) { 514 dn->data_blkaddr = NEW_ADDR; 515 __set_data_blkaddr(dn); ··· 570 struct page *page; 571 struct extent_info ei = {0,0,0}; 572 int err; 573 574 page = f2fs_grab_cache_page(mapping, index, for_write); 575 if (!page) ··· 620 return page; 621 } 622 623 + err = f2fs_submit_page_read(inode, page, dn.data_blkaddr); 624 if (err) 625 goto put_err; 626 return page; ··· 756 if (unlikely(is_inode_flag_set(dn->inode, FI_NO_ALLOC))) 757 return -EPERM; 758 759 + dn->data_blkaddr = datablock_addr(dn->inode, 760 + dn->node_page, dn->ofs_in_node); 761 if (dn->data_blkaddr == NEW_ADDR) 762 goto alloc; 763 ··· 782 783 static inline bool __force_buffered_io(struct inode *inode, int rw) 784 { 785 + return (f2fs_encrypted_file(inode) || 786 (rw == WRITE && test_opt(F2FS_I_SB(inode), LFS)) || 787 F2FS_I_SB(inode)->s_ndevs); 788 } ··· 814 F2FS_GET_BLOCK_PRE_AIO : 815 F2FS_GET_BLOCK_PRE_DIO); 816 } 817 + if (iocb->ki_pos + iov_iter_count(from) > MAX_INLINE_DATA(inode)) { 818 err = f2fs_convert_inline_inode(inode); 819 if (err) 820 return err; ··· 903 end_offset = ADDRS_PER_PAGE(dn.node_page, inode); 904 905 next_block: 906 + blkaddr = datablock_addr(dn.inode, dn.node_page, dn.ofs_in_node); 907 908 if (blkaddr == NEW_ADDR || blkaddr == NULL_ADDR) { 909 if (create) { ··· 1040 struct buffer_head *bh_result, int create) 1041 { 1042 return __get_data_block(inode, iblock, bh_result, create, 1043 + F2FS_GET_BLOCK_DEFAULT, NULL); 1044 } 1045 1046 static int get_data_block_bmap(struct inode *inode, sector_t iblock, ··· 1146 return ret; 1147 } 1148 1149 /* 1150 * This function was originally taken from fs/mpage.c, and customized for f2fs. 1151 * Major change was from block_size == page_size in f2fs by default. ··· 1240 map.m_len = last_block - block_in_file; 1241 1242 if (f2fs_map_blocks(inode, &map, 0, 1243 + F2FS_GET_BLOCK_DEFAULT)) 1244 goto set_error_page; 1245 } 1246 got_it: ··· 1271 bio = NULL; 1272 } 1273 if (bio == NULL) { 1274 + bio = f2fs_grab_read_bio(inode, block_nr, nr_pages); 1275 if (IS_ERR(bio)) { 1276 bio = NULL; 1277 goto set_error_page; 1278 } 1279 } 1280 1281 if (bio_add_page(bio, page, blocksize, 0) < blocksize) ··· 1341 struct inode *inode = fio->page->mapping->host; 1342 gfp_t gfp_flags = GFP_NOFS; 1343 1344 + if (!f2fs_encrypted_file(inode)) 1345 return 0; 1346 1347 /* wait for GCed encrypted page writeback */ 1348 + f2fs_wait_on_block_writeback(fio->sbi, fio->old_blkaddr); 1349 1350 retry_encrypt: 1351 fio->encrypted_page = fscrypt_encrypt_page(inode, fio->page, ··· 1471 } 1472 1473 static int __write_data_page(struct page *page, bool *submitted, 1474 + struct writeback_control *wbc, 1475 + enum iostat_type io_type) 1476 { 1477 struct inode *inode = page->mapping->host; 1478 struct f2fs_sb_info *sbi = F2FS_I_SB(inode); ··· 1492 .encrypted_page = NULL, 1493 .submitted = false, 1494 .need_lock = LOCK_RETRY, 1495 + .io_type = io_type, 1496 }; 1497 1498 trace_f2fs_writepage(page, DATA); ··· 1598 static int f2fs_write_data_page(struct page *page, 1599 struct writeback_control *wbc) 1600 { 1601 + return __write_data_page(page, NULL, wbc, FS_DATA_IO); 1602 } 1603 1604 /* ··· 1607 * warm/hot data page. 1608 */ 1609 static int f2fs_write_cache_pages(struct address_space *mapping, 1610 + struct writeback_control *wbc, 1611 + enum iostat_type io_type) 1612 { 1613 int ret = 0; 1614 int done = 0; ··· 1697 if (!clear_page_dirty_for_io(page)) 1698 goto continue_unlock; 1699 1700 + ret = __write_data_page(page, &submitted, wbc, io_type); 1701 if (unlikely(ret)) { 1702 /* 1703 * keep nr_to_write, since vfs uses this to ··· 1752 return ret; 1753 } 1754 1755 + int __f2fs_write_data_pages(struct address_space *mapping, 1756 + struct writeback_control *wbc, 1757 + enum iostat_type io_type) 1758 { 1759 struct inode *inode = mapping->host; 1760 struct f2fs_sb_info *sbi = F2FS_I_SB(inode); ··· 1790 goto skip_write; 1791 1792 blk_start_plug(&plug); 1793 + ret = f2fs_write_cache_pages(mapping, wbc, io_type); 1794 blk_finish_plug(&plug); 1795 1796 if (wbc->sync_mode == WB_SYNC_ALL) ··· 1807 wbc->pages_skipped += get_dirty_pages(inode); 1808 trace_f2fs_writepages(mapping->host, wbc, DATA); 1809 return 0; 1810 + } 1811 + 1812 + static int f2fs_write_data_pages(struct address_space *mapping, 1813 + struct writeback_control *wbc) 1814 + { 1815 + struct inode *inode = mapping->host; 1816 + 1817 + return __f2fs_write_data_pages(mapping, wbc, 1818 + F2FS_I(inode)->cp_task == current ? 1819 + FS_CP_DATA_IO : FS_DATA_IO); 1820 } 1821 1822 static void f2fs_write_failed(struct address_space *mapping, loff_t to) ··· 1858 set_new_dnode(&dn, inode, ipage, ipage, 0); 1859 1860 if (f2fs_has_inline_data(inode)) { 1861 + if (pos + len <= MAX_INLINE_DATA(inode)) { 1862 read_inline_data(page, ipage); 1863 set_inode_flag(inode, FI_DATA_EXIST); 1864 if (inode->i_nlink) ··· 1956 f2fs_wait_on_page_writeback(page, DATA, false); 1957 1958 /* wait for GCed encrypted page writeback */ 1959 + if (f2fs_encrypted_file(inode)) 1960 + f2fs_wait_on_block_writeback(sbi, blkaddr); 1961 1962 if (len == PAGE_SIZE || PageUptodate(page)) 1963 return 0; ··· 1971 zero_user_segment(page, 0, PAGE_SIZE); 1972 SetPageUptodate(page); 1973 } else { 1974 + err = f2fs_submit_page_read(inode, page, blkaddr); 1975 + if (err) 1976 goto fail; 1977 1978 lock_page(page); 1979 if (unlikely(page->mapping != mapping)) { ··· 2075 up_read(&F2FS_I(inode)->dio_rwsem[rw]); 2076 2077 if (rw == WRITE) { 2078 + if (err > 0) { 2079 + f2fs_update_iostat(F2FS_I_SB(inode), APP_DIRECT_IO, 2080 + err); 2081 set_inode_flag(inode, FI_UPDATE_WRITE); 2082 + } else if (err < 0) { 2083 f2fs_write_failed(mapping, offset + count); 2084 + } 2085 } 2086 2087 trace_f2fs_direct_IO_exit(inode, offset, count, rw, err);
+7
fs/f2fs/dir.c
··· 705 struct f2fs_dentry_block *dentry_blk; 706 unsigned int bit_pos; 707 int slots = GET_DENTRY_SLOTS(le16_to_cpu(dentry->name_len)); 708 int i; 709 710 f2fs_update_time(F2FS_I_SB(dir), REQ_TIME); ··· 737 738 if (bit_pos == NR_DENTRY_IN_BLOCK && 739 !truncate_hole(dir, page->index, page->index + 1)) { 740 clear_page_dirty_for_io(page); 741 ClearPagePrivate(page); 742 ClearPageUptodate(page);
··· 705 struct f2fs_dentry_block *dentry_blk; 706 unsigned int bit_pos; 707 int slots = GET_DENTRY_SLOTS(le16_to_cpu(dentry->name_len)); 708 + struct address_space *mapping = page_mapping(page); 709 + unsigned long flags; 710 int i; 711 712 f2fs_update_time(F2FS_I_SB(dir), REQ_TIME); ··· 735 736 if (bit_pos == NR_DENTRY_IN_BLOCK && 737 !truncate_hole(dir, page->index, page->index + 1)) { 738 + spin_lock_irqsave(&mapping->tree_lock, flags); 739 + radix_tree_tag_clear(&mapping->page_tree, page_index(page), 740 + PAGECACHE_TAG_DIRTY); 741 + spin_unlock_irqrestore(&mapping->tree_lock, flags); 742 + 743 clear_page_dirty_for_io(page); 744 ClearPagePrivate(page); 745 ClearPageUptodate(page);
+253 -32
fs/f2fs/f2fs.h
··· 91 #define F2FS_MOUNT_LFS 0x00040000 92 #define F2FS_MOUNT_USRQUOTA 0x00080000 93 #define F2FS_MOUNT_GRPQUOTA 0x00100000 94 95 #define clear_opt(sbi, option) ((sbi)->mount_opt.opt &= ~F2FS_MOUNT_##option) 96 #define set_opt(sbi, option) ((sbi)->mount_opt.opt |= F2FS_MOUNT_##option) ··· 112 unsigned int opt; 113 }; 114 115 - #define F2FS_FEATURE_ENCRYPT 0x0001 116 - #define F2FS_FEATURE_BLKZONED 0x0002 117 118 #define F2FS_HAS_FEATURE(sb, mask) \ 119 ((F2FS_SB(sb)->raw_super->feature & cpu_to_le32(mask)) != 0) ··· 148 (BATCHED_TRIM_SEGMENTS(sbi) << (sbi)->log_blocks_per_seg) 149 #define MAX_DISCARD_BLOCKS(sbi) BLKS_PER_SEC(sbi) 150 #define DISCARD_ISSUE_RATE 8 151 #define DEF_CP_INTERVAL 60 /* 60 secs */ 152 #define DEF_IDLE_INTERVAL 5 /* 5 secs */ 153 ··· 198 unsigned char discard_map[SIT_VBLOCK_MAP_SIZE]; /* segment discard bitmap */ 199 }; 200 201 /* max discard pend list number */ 202 #define MAX_PLIST_NUM 512 203 #define plist_idx(blk_num) ((blk_num) >= MAX_PLIST_NUM ? \ 204 (MAX_PLIST_NUM - 1) : (blk_num - 1)) 205 206 enum { 207 D_PREP, ··· 245 struct task_struct *f2fs_issue_discard; /* discard thread */ 246 struct list_head entry_list; /* 4KB discard entry list */ 247 struct list_head pend_list[MAX_PLIST_NUM];/* store pending entries */ 248 struct list_head wait_list; /* store on-flushing entries */ 249 wait_queue_head_t discard_wait_queue; /* waiting queue for wake-up */ 250 struct mutex cmd_lock; 251 unsigned int nr_discards; /* # of discards in the list */ 252 unsigned int max_discards; /* max. discards to be issued */ 253 unsigned int undiscard_blks; /* # of undiscard blocks */ 254 atomic_t issued_discard; /* # of issued discard */ 255 atomic_t issing_discard; /* # of issing discard */ ··· 326 struct f2fs_flush_device) 327 #define F2FS_IOC_GARBAGE_COLLECT_RANGE _IOW(F2FS_IOCTL_MAGIC, 11, \ 328 struct f2fs_gc_range) 329 330 #define F2FS_IOC_SET_ENCRYPTION_POLICY FS_IOC_SET_ENCRYPTION_POLICY 331 #define F2FS_IOC_GET_ENCRYPTION_POLICY FS_IOC_GET_ENCRYPTION_POLICY ··· 351 #define F2FS_IOC32_GETVERSION FS_IOC32_GETVERSION 352 #endif 353 354 struct f2fs_gc_range { 355 u32 sync; 356 u64 start; ··· 377 u32 segments; /* # of segments to flush */ 378 }; 379 380 /* 381 * For INODE and NODE manager 382 */ 383 /* for directory operations */ 384 struct f2fs_dentry_ptr { 385 struct inode *inode; 386 - const void *bitmap; 387 struct f2fs_dir_entry *dentry; 388 __u8 (*filename)[F2FS_SLOT_LEN]; 389 int max; 390 }; 391 392 static inline void make_dentry_ptr_block(struct inode *inode, ··· 414 { 415 d->inode = inode; 416 d->max = NR_DENTRY_IN_BLOCK; 417 d->bitmap = &t->dentry_bitmap; 418 d->dentry = t->dentry; 419 d->filename = t->filename; 420 } 421 422 static inline void make_dentry_ptr_inline(struct inode *inode, 423 - struct f2fs_dentry_ptr *d, struct f2fs_inline_dentry *t) 424 { 425 d->inode = inode; 426 - d->max = NR_INLINE_DENTRY; 427 - d->bitmap = &t->dentry_bitmap; 428 - d->dentry = t->dentry; 429 - d->filename = t->filename; 430 } 431 432 /* ··· 522 }; 523 524 /* for flag in get_data_block */ 525 - #define F2FS_GET_BLOCK_READ 0 526 - #define F2FS_GET_BLOCK_DIO 1 527 - #define F2FS_GET_BLOCK_FIEMAP 2 528 - #define F2FS_GET_BLOCK_BMAP 3 529 - #define F2FS_GET_BLOCK_PRE_DIO 4 530 - #define F2FS_GET_BLOCK_PRE_AIO 5 531 532 /* 533 * i_advise uses FADVISE_XXX_BIT. We can add additional hints later. ··· 571 f2fs_hash_t chash; /* hash value of given file name */ 572 unsigned int clevel; /* maximum level of given file name */ 573 struct task_struct *task; /* lookup and create consistency */ 574 nid_t i_xattr_nid; /* node id that contains xattrs */ 575 loff_t last_disk_size; /* lastly written file size */ 576 ··· 584 struct list_head dirty_list; /* dirty list for dirs and files */ 585 struct list_head gdirty_list; /* linked in global dirty list */ 586 struct list_head inmem_pages; /* inmemory pages managed by f2fs */ 587 struct mutex inmem_lock; /* lock for inmemory pages */ 588 struct extent_tree *extent_tree; /* cached extent_tree entry */ 589 struct rw_semaphore dio_rwsem[2];/* avoid racing between dio and gc */ 590 struct rw_semaphore i_mmap_sem; 591 }; 592 593 static inline void get_extent_info(struct extent_info *ext, ··· 879 LOCK_RETRY, 880 }; 881 882 struct f2fs_io_info { 883 struct f2fs_sb_info *sbi; /* f2fs_sb_info pointer */ 884 enum page_type type; /* contains DATA/NODE/META/META_FLUSH */ ··· 910 bool submitted; /* indicate IO submission */ 911 int need_lock; /* indicate we need to lock cp_rwsem */ 912 bool in_list; /* indicate fio is in io_list */ 913 }; 914 915 #define is_read_io(rw) ((rw) == READ) ··· 1102 #endif 1103 spinlock_t stat_lock; /* lock for stat operations */ 1104 1105 /* For sysfs suppport */ 1106 struct kobject s_kobj; 1107 struct completion s_kobj_unregister; ··· 1125 /* Reference to checksum algorithm driver via cryptoapi */ 1126 struct crypto_shash *s_chksum_driver; 1127 1128 /* For fault injection */ 1129 #ifdef CONFIG_F2FS_FAULT_INJECTION 1130 struct f2fs_fault_info fault_info; 1131 #endif 1132 }; 1133 ··· 1223 void *buf, size_t buf_size) 1224 { 1225 return f2fs_crc32(sbi, buf, buf_size) == blk_crc; 1226 } 1227 1228 static inline struct f2fs_inode_info *F2FS_I(struct inode *inode) ··· 1869 return RAW_IS_INODE(p); 1870 } 1871 1872 static inline __le32 *blkaddr_in_node(struct f2fs_node *node) 1873 { 1874 return RAW_IS_INODE(node) ? node->i.i_addr : node->dn.addr; 1875 } 1876 1877 - static inline block_t datablock_addr(struct page *node_page, 1878 - unsigned int offset) 1879 { 1880 struct f2fs_node *raw_node; 1881 __le32 *addr_array; 1882 1883 raw_node = F2FS_NODE(node_page); 1884 addr_array = blkaddr_in_node(raw_node); 1885 - return le32_to_cpu(addr_array[offset]); 1886 } 1887 1888 static inline int f2fs_test_bit(unsigned int nr, char *addr) ··· 1963 *addr ^= mask; 1964 } 1965 1966 /* used for f2fs_inode_info->flags */ 1967 enum { 1968 FI_NEW_INODE, /* indicate newly allocated inode */ ··· 2005 FI_DIRTY_FILE, /* indicate regular/symlink has dirty pages */ 2006 FI_NO_PREALLOC, /* indicate skipped preallocated blocks */ 2007 FI_HOT_DATA, /* indicate file is hot */ 2008 }; 2009 2010 static inline void __mark_inode_dirty_flag(struct inode *inode, ··· 2126 set_bit(FI_DATA_EXIST, &fi->flags); 2127 if (ri->i_inline & F2FS_INLINE_DOTS) 2128 set_bit(FI_INLINE_DOTS, &fi->flags); 2129 } 2130 2131 static inline void set_raw_inline(struct inode *inode, struct f2fs_inode *ri) ··· 2144 ri->i_inline |= F2FS_DATA_EXIST; 2145 if (is_inode_flag_set(inode, FI_INLINE_DOTS)) 2146 ri->i_inline |= F2FS_INLINE_DOTS; 2147 } 2148 2149 static inline int f2fs_has_inline_xattr(struct inode *inode) ··· 2161 static inline unsigned int addrs_per_inode(struct inode *inode) 2162 { 2163 if (f2fs_has_inline_xattr(inode)) 2164 - return DEF_ADDRS_PER_INODE - F2FS_INLINE_XATTR_ADDRS; 2165 - return DEF_ADDRS_PER_INODE; 2166 } 2167 2168 static inline void *inline_xattr_addr(struct page *page) ··· 2221 return is_inode_flag_set(inode, FI_DROP_CACHE); 2222 } 2223 2224 - static inline void *inline_data_addr(struct page *page) 2225 { 2226 struct f2fs_inode *ri = F2FS_INODE(page); 2227 2228 - return (void *)&(ri->i_addr[1]); 2229 } 2230 2231 static inline int f2fs_has_inline_dentry(struct inode *inode) ··· 2317 return kmalloc(size, flags); 2318 } 2319 2320 #define get_inode_mode(i) \ 2321 ((is_inode_flag_set(i, FI_ACL_MODE)) ? \ 2322 (F2FS_I(i)->i_acl_mode) : ((i)->i_mode)) 2323 2324 /* 2325 * file.c ··· 2380 * inode.c 2381 */ 2382 void f2fs_set_inode_flags(struct inode *inode); 2383 struct inode *f2fs_iget(struct super_block *sb, unsigned long ino); 2384 struct inode *f2fs_iget_retry(struct super_block *sb, unsigned long ino); 2385 int try_to_free_nats(struct f2fs_sb_info *sbi, int nr_shrink); ··· 2450 */ 2451 int f2fs_inode_dirtied(struct inode *inode, bool sync); 2452 void f2fs_inode_synced(struct inode *inode); 2453 int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover); 2454 int f2fs_sync_fs(struct super_block *sb, int sync); 2455 extern __printf(3, 4) ··· 2482 int wait_on_node_pages_writeback(struct f2fs_sb_info *sbi, nid_t ino); 2483 int remove_inode_page(struct inode *inode); 2484 struct page *new_inode_page(struct inode *inode); 2485 - struct page *new_node_page(struct dnode_of_data *dn, 2486 - unsigned int ofs, struct page *ipage); 2487 void ra_node_page(struct f2fs_sb_info *sbi, nid_t nid); 2488 struct page *get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid); 2489 struct page *get_node_page_ra(struct page *parent, int start); 2490 void move_node_page(struct page *node_page, int gc_type); 2491 int fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode, 2492 struct writeback_control *wbc, bool atomic); 2493 - int sync_node_pages(struct f2fs_sb_info *sbi, struct writeback_control *wbc); 2494 void build_free_nids(struct f2fs_sb_info *sbi, bool sync, bool mount); 2495 bool alloc_nid(struct f2fs_sb_info *sbi, nid_t *nid); 2496 void alloc_nid_done(struct f2fs_sb_info *sbi, nid_t nid); ··· 2511 /* 2512 * segment.c 2513 */ 2514 void register_inmem_page(struct inode *inode, struct page *page); 2515 void drop_inmem_pages(struct inode *inode); 2516 void drop_inmem_page(struct inode *inode, struct page *page); ··· 2534 bool exist_trim_candidates(struct f2fs_sb_info *sbi, struct cp_control *cpc); 2535 struct page *get_sum_page(struct f2fs_sb_info *sbi, unsigned int segno); 2536 void update_meta_page(struct f2fs_sb_info *sbi, void *src, block_t blk_addr); 2537 - void write_meta_page(struct f2fs_sb_info *sbi, struct page *page); 2538 void write_node_page(unsigned int nid, struct f2fs_io_info *fio); 2539 void write_data_page(struct dnode_of_data *dn, struct f2fs_io_info *fio); 2540 int rewrite_data_page(struct f2fs_io_info *fio); ··· 2552 struct f2fs_io_info *fio, bool add_list); 2553 void f2fs_wait_on_page_writeback(struct page *page, 2554 enum page_type type, bool ordered); 2555 - void f2fs_wait_on_encrypted_page_writeback(struct f2fs_sb_info *sbi, 2556 - block_t blkaddr); 2557 void write_data_summaries(struct f2fs_sb_info *sbi, block_t start_blk); 2558 void write_node_summaries(struct f2fs_sb_info *sbi, block_t start_blk); 2559 int lookup_journal_in_cursum(struct f2fs_journal *journal, int type, ··· 2575 int type, bool sync); 2576 void ra_meta_pages_cond(struct f2fs_sb_info *sbi, pgoff_t index); 2577 long sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type, 2578 - long nr_to_write); 2579 void add_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type); 2580 void remove_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type); 2581 void release_ino_entry(struct f2fs_sb_info *sbi, bool all); ··· 2628 int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, 2629 u64 start, u64 len); 2630 void f2fs_set_page_dirty_nobuffers(struct page *page); 2631 void f2fs_invalidate_page(struct page *page, unsigned int offset, 2632 unsigned int length); 2633 int f2fs_release_page(struct page *page, gfp_t wait); ··· 2927 /* 2928 * sysfs.c 2929 */ 2930 - int __init f2fs_register_sysfs(void); 2931 - void f2fs_unregister_sysfs(void); 2932 - int f2fs_init_sysfs(struct f2fs_sb_info *sbi); 2933 - void f2fs_exit_sysfs(struct f2fs_sb_info *sbi); 2934 2935 /* 2936 * crypto support ··· 2938 static inline bool f2fs_encrypted_inode(struct inode *inode) 2939 { 2940 return file_is_encrypt(inode); 2941 } 2942 2943 static inline void f2fs_set_encrypted_inode(struct inode *inode) ··· 2965 static inline int f2fs_sb_mounted_blkzoned(struct super_block *sb) 2966 { 2967 return F2FS_HAS_FEATURE(sb, F2FS_FEATURE_BLKZONED); 2968 } 2969 2970 #ifdef CONFIG_BLK_DEV_ZONED
··· 91 #define F2FS_MOUNT_LFS 0x00040000 92 #define F2FS_MOUNT_USRQUOTA 0x00080000 93 #define F2FS_MOUNT_GRPQUOTA 0x00100000 94 + #define F2FS_MOUNT_PRJQUOTA 0x00200000 95 + #define F2FS_MOUNT_QUOTA 0x00400000 96 97 #define clear_opt(sbi, option) ((sbi)->mount_opt.opt &= ~F2FS_MOUNT_##option) 98 #define set_opt(sbi, option) ((sbi)->mount_opt.opt |= F2FS_MOUNT_##option) ··· 110 unsigned int opt; 111 }; 112 113 + #define F2FS_FEATURE_ENCRYPT 0x0001 114 + #define F2FS_FEATURE_BLKZONED 0x0002 115 + #define F2FS_FEATURE_ATOMIC_WRITE 0x0004 116 + #define F2FS_FEATURE_EXTRA_ATTR 0x0008 117 + #define F2FS_FEATURE_PRJQUOTA 0x0010 118 + #define F2FS_FEATURE_INODE_CHKSUM 0x0020 119 120 #define F2FS_HAS_FEATURE(sb, mask) \ 121 ((F2FS_SB(sb)->raw_super->feature & cpu_to_le32(mask)) != 0) ··· 142 (BATCHED_TRIM_SEGMENTS(sbi) << (sbi)->log_blocks_per_seg) 143 #define MAX_DISCARD_BLOCKS(sbi) BLKS_PER_SEC(sbi) 144 #define DISCARD_ISSUE_RATE 8 145 + #define DEF_MIN_DISCARD_ISSUE_TIME 50 /* 50 ms, if exists */ 146 + #define DEF_MAX_DISCARD_ISSUE_TIME 60000 /* 60 s, if no candidates */ 147 #define DEF_CP_INTERVAL 60 /* 60 secs */ 148 #define DEF_IDLE_INTERVAL 5 /* 5 secs */ 149 ··· 190 unsigned char discard_map[SIT_VBLOCK_MAP_SIZE]; /* segment discard bitmap */ 191 }; 192 193 + /* default discard granularity of inner discard thread, unit: block count */ 194 + #define DEFAULT_DISCARD_GRANULARITY 16 195 + 196 /* max discard pend list number */ 197 #define MAX_PLIST_NUM 512 198 #define plist_idx(blk_num) ((blk_num) >= MAX_PLIST_NUM ? \ 199 (MAX_PLIST_NUM - 1) : (blk_num - 1)) 200 + 201 + #define P_ACTIVE 0x01 202 + #define P_TRIM 0x02 203 + #define plist_issue(tag) (((tag) & P_ACTIVE) || ((tag) & P_TRIM)) 204 205 enum { 206 D_PREP, ··· 230 struct task_struct *f2fs_issue_discard; /* discard thread */ 231 struct list_head entry_list; /* 4KB discard entry list */ 232 struct list_head pend_list[MAX_PLIST_NUM];/* store pending entries */ 233 + unsigned char pend_list_tag[MAX_PLIST_NUM];/* tag for pending entries */ 234 struct list_head wait_list; /* store on-flushing entries */ 235 wait_queue_head_t discard_wait_queue; /* waiting queue for wake-up */ 236 + unsigned int discard_wake; /* to wake up discard thread */ 237 struct mutex cmd_lock; 238 unsigned int nr_discards; /* # of discards in the list */ 239 unsigned int max_discards; /* max. discards to be issued */ 240 + unsigned int discard_granularity; /* discard granularity */ 241 unsigned int undiscard_blks; /* # of undiscard blocks */ 242 atomic_t issued_discard; /* # of issued discard */ 243 atomic_t issing_discard; /* # of issing discard */ ··· 308 struct f2fs_flush_device) 309 #define F2FS_IOC_GARBAGE_COLLECT_RANGE _IOW(F2FS_IOCTL_MAGIC, 11, \ 310 struct f2fs_gc_range) 311 + #define F2FS_IOC_GET_FEATURES _IOR(F2FS_IOCTL_MAGIC, 12, __u32) 312 313 #define F2FS_IOC_SET_ENCRYPTION_POLICY FS_IOC_SET_ENCRYPTION_POLICY 314 #define F2FS_IOC_GET_ENCRYPTION_POLICY FS_IOC_GET_ENCRYPTION_POLICY ··· 332 #define F2FS_IOC32_GETVERSION FS_IOC32_GETVERSION 333 #endif 334 335 + #define F2FS_IOC_FSGETXATTR FS_IOC_FSGETXATTR 336 + #define F2FS_IOC_FSSETXATTR FS_IOC_FSSETXATTR 337 + 338 struct f2fs_gc_range { 339 u32 sync; 340 u64 start; ··· 355 u32 segments; /* # of segments to flush */ 356 }; 357 358 + /* for inline stuff */ 359 + #define DEF_INLINE_RESERVED_SIZE 1 360 + static inline int get_extra_isize(struct inode *inode); 361 + #define MAX_INLINE_DATA(inode) (sizeof(__le32) * \ 362 + (CUR_ADDRS_PER_INODE(inode) - \ 363 + DEF_INLINE_RESERVED_SIZE - \ 364 + F2FS_INLINE_XATTR_ADDRS)) 365 + 366 + /* for inline dir */ 367 + #define NR_INLINE_DENTRY(inode) (MAX_INLINE_DATA(inode) * BITS_PER_BYTE / \ 368 + ((SIZE_OF_DIR_ENTRY + F2FS_SLOT_LEN) * \ 369 + BITS_PER_BYTE + 1)) 370 + #define INLINE_DENTRY_BITMAP_SIZE(inode) ((NR_INLINE_DENTRY(inode) + \ 371 + BITS_PER_BYTE - 1) / BITS_PER_BYTE) 372 + #define INLINE_RESERVED_SIZE(inode) (MAX_INLINE_DATA(inode) - \ 373 + ((SIZE_OF_DIR_ENTRY + F2FS_SLOT_LEN) * \ 374 + NR_INLINE_DENTRY(inode) + \ 375 + INLINE_DENTRY_BITMAP_SIZE(inode))) 376 + 377 /* 378 * For INODE and NODE manager 379 */ 380 /* for directory operations */ 381 struct f2fs_dentry_ptr { 382 struct inode *inode; 383 + void *bitmap; 384 struct f2fs_dir_entry *dentry; 385 __u8 (*filename)[F2FS_SLOT_LEN]; 386 int max; 387 + int nr_bitmap; 388 }; 389 390 static inline void make_dentry_ptr_block(struct inode *inode, ··· 372 { 373 d->inode = inode; 374 d->max = NR_DENTRY_IN_BLOCK; 375 + d->nr_bitmap = SIZE_OF_DENTRY_BITMAP; 376 d->bitmap = &t->dentry_bitmap; 377 d->dentry = t->dentry; 378 d->filename = t->filename; 379 } 380 381 static inline void make_dentry_ptr_inline(struct inode *inode, 382 + struct f2fs_dentry_ptr *d, void *t) 383 { 384 + int entry_cnt = NR_INLINE_DENTRY(inode); 385 + int bitmap_size = INLINE_DENTRY_BITMAP_SIZE(inode); 386 + int reserved_size = INLINE_RESERVED_SIZE(inode); 387 + 388 d->inode = inode; 389 + d->max = entry_cnt; 390 + d->nr_bitmap = bitmap_size; 391 + d->bitmap = t; 392 + d->dentry = t + bitmap_size + reserved_size; 393 + d->filename = t + bitmap_size + reserved_size + 394 + SIZE_OF_DIR_ENTRY * entry_cnt; 395 } 396 397 /* ··· 473 }; 474 475 /* for flag in get_data_block */ 476 + enum { 477 + F2FS_GET_BLOCK_DEFAULT, 478 + F2FS_GET_BLOCK_FIEMAP, 479 + F2FS_GET_BLOCK_BMAP, 480 + F2FS_GET_BLOCK_PRE_DIO, 481 + F2FS_GET_BLOCK_PRE_AIO, 482 + }; 483 484 /* 485 * i_advise uses FADVISE_XXX_BIT. We can add additional hints later. ··· 521 f2fs_hash_t chash; /* hash value of given file name */ 522 unsigned int clevel; /* maximum level of given file name */ 523 struct task_struct *task; /* lookup and create consistency */ 524 + struct task_struct *cp_task; /* separate cp/wb IO stats*/ 525 nid_t i_xattr_nid; /* node id that contains xattrs */ 526 loff_t last_disk_size; /* lastly written file size */ 527 ··· 533 struct list_head dirty_list; /* dirty list for dirs and files */ 534 struct list_head gdirty_list; /* linked in global dirty list */ 535 struct list_head inmem_pages; /* inmemory pages managed by f2fs */ 536 + struct task_struct *inmem_task; /* store inmemory task */ 537 struct mutex inmem_lock; /* lock for inmemory pages */ 538 struct extent_tree *extent_tree; /* cached extent_tree entry */ 539 struct rw_semaphore dio_rwsem[2];/* avoid racing between dio and gc */ 540 struct rw_semaphore i_mmap_sem; 541 + struct rw_semaphore i_xattr_sem; /* avoid racing between reading and changing EAs */ 542 + 543 + int i_extra_isize; /* size of extra space located in i_addr */ 544 + kprojid_t i_projid; /* id for project quota */ 545 }; 546 547 static inline void get_extent_info(struct extent_info *ext, ··· 823 LOCK_RETRY, 824 }; 825 826 + enum iostat_type { 827 + APP_DIRECT_IO, /* app direct IOs */ 828 + APP_BUFFERED_IO, /* app buffered IOs */ 829 + APP_WRITE_IO, /* app write IOs */ 830 + APP_MAPPED_IO, /* app mapped IOs */ 831 + FS_DATA_IO, /* data IOs from kworker/fsync/reclaimer */ 832 + FS_NODE_IO, /* node IOs from kworker/fsync/reclaimer */ 833 + FS_META_IO, /* meta IOs from kworker/reclaimer */ 834 + FS_GC_DATA_IO, /* data IOs from forground gc */ 835 + FS_GC_NODE_IO, /* node IOs from forground gc */ 836 + FS_CP_DATA_IO, /* data IOs from checkpoint */ 837 + FS_CP_NODE_IO, /* node IOs from checkpoint */ 838 + FS_CP_META_IO, /* meta IOs from checkpoint */ 839 + FS_DISCARD, /* discard */ 840 + NR_IO_TYPE, 841 + }; 842 + 843 struct f2fs_io_info { 844 struct f2fs_sb_info *sbi; /* f2fs_sb_info pointer */ 845 enum page_type type; /* contains DATA/NODE/META/META_FLUSH */ ··· 837 bool submitted; /* indicate IO submission */ 838 int need_lock; /* indicate we need to lock cp_rwsem */ 839 bool in_list; /* indicate fio is in io_list */ 840 + enum iostat_type io_type; /* io type */ 841 }; 842 843 #define is_read_io(rw) ((rw) == READ) ··· 1028 #endif 1029 spinlock_t stat_lock; /* lock for stat operations */ 1030 1031 + /* For app/fs IO statistics */ 1032 + spinlock_t iostat_lock; 1033 + unsigned long long write_iostat[NR_IO_TYPE]; 1034 + bool iostat_enable; 1035 + 1036 /* For sysfs suppport */ 1037 struct kobject s_kobj; 1038 struct completion s_kobj_unregister; ··· 1046 /* Reference to checksum algorithm driver via cryptoapi */ 1047 struct crypto_shash *s_chksum_driver; 1048 1049 + /* Precomputed FS UUID checksum for seeding other checksums */ 1050 + __u32 s_chksum_seed; 1051 + 1052 /* For fault injection */ 1053 #ifdef CONFIG_F2FS_FAULT_INJECTION 1054 struct f2fs_fault_info fault_info; 1055 + #endif 1056 + 1057 + #ifdef CONFIG_QUOTA 1058 + /* Names of quota files with journalled quota */ 1059 + char *s_qf_names[MAXQUOTAS]; 1060 + int s_jquota_fmt; /* Format of quota to use */ 1061 #endif 1062 }; 1063 ··· 1135 void *buf, size_t buf_size) 1136 { 1137 return f2fs_crc32(sbi, buf, buf_size) == blk_crc; 1138 + } 1139 + 1140 + static inline u32 f2fs_chksum(struct f2fs_sb_info *sbi, u32 crc, 1141 + const void *address, unsigned int length) 1142 + { 1143 + struct { 1144 + struct shash_desc shash; 1145 + char ctx[4]; 1146 + } desc; 1147 + int err; 1148 + 1149 + BUG_ON(crypto_shash_descsize(sbi->s_chksum_driver) != sizeof(desc.ctx)); 1150 + 1151 + desc.shash.tfm = sbi->s_chksum_driver; 1152 + desc.shash.flags = 0; 1153 + *(u32 *)desc.ctx = crc; 1154 + 1155 + err = crypto_shash_update(&desc.shash, address, length); 1156 + BUG_ON(err); 1157 + 1158 + return *(u32 *)desc.ctx; 1159 } 1160 1161 static inline struct f2fs_inode_info *F2FS_I(struct inode *inode) ··· 1760 return RAW_IS_INODE(p); 1761 } 1762 1763 + static inline int offset_in_addr(struct f2fs_inode *i) 1764 + { 1765 + return (i->i_inline & F2FS_EXTRA_ATTR) ? 1766 + (le16_to_cpu(i->i_extra_isize) / sizeof(__le32)) : 0; 1767 + } 1768 + 1769 static inline __le32 *blkaddr_in_node(struct f2fs_node *node) 1770 { 1771 return RAW_IS_INODE(node) ? node->i.i_addr : node->dn.addr; 1772 } 1773 1774 + static inline int f2fs_has_extra_attr(struct inode *inode); 1775 + static inline block_t datablock_addr(struct inode *inode, 1776 + struct page *node_page, unsigned int offset) 1777 { 1778 struct f2fs_node *raw_node; 1779 __le32 *addr_array; 1780 + int base = 0; 1781 + bool is_inode = IS_INODE(node_page); 1782 1783 raw_node = F2FS_NODE(node_page); 1784 + 1785 + /* from GC path only */ 1786 + if (!inode) { 1787 + if (is_inode) 1788 + base = offset_in_addr(&raw_node->i); 1789 + } else if (f2fs_has_extra_attr(inode) && is_inode) { 1790 + base = get_extra_isize(inode); 1791 + } 1792 + 1793 addr_array = blkaddr_in_node(raw_node); 1794 + return le32_to_cpu(addr_array[base + offset]); 1795 } 1796 1797 static inline int f2fs_test_bit(unsigned int nr, char *addr) ··· 1836 *addr ^= mask; 1837 } 1838 1839 + #define F2FS_REG_FLMASK (~(FS_DIRSYNC_FL | FS_TOPDIR_FL)) 1840 + #define F2FS_OTHER_FLMASK (FS_NODUMP_FL | FS_NOATIME_FL) 1841 + #define F2FS_FL_INHERITED (FS_PROJINHERIT_FL) 1842 + 1843 + static inline __u32 f2fs_mask_flags(umode_t mode, __u32 flags) 1844 + { 1845 + if (S_ISDIR(mode)) 1846 + return flags; 1847 + else if (S_ISREG(mode)) 1848 + return flags & F2FS_REG_FLMASK; 1849 + else 1850 + return flags & F2FS_OTHER_FLMASK; 1851 + } 1852 + 1853 /* used for f2fs_inode_info->flags */ 1854 enum { 1855 FI_NEW_INODE, /* indicate newly allocated inode */ ··· 1864 FI_DIRTY_FILE, /* indicate regular/symlink has dirty pages */ 1865 FI_NO_PREALLOC, /* indicate skipped preallocated blocks */ 1866 FI_HOT_DATA, /* indicate file is hot */ 1867 + FI_EXTRA_ATTR, /* indicate file has extra attribute */ 1868 + FI_PROJ_INHERIT, /* indicate file inherits projectid */ 1869 }; 1870 1871 static inline void __mark_inode_dirty_flag(struct inode *inode, ··· 1983 set_bit(FI_DATA_EXIST, &fi->flags); 1984 if (ri->i_inline & F2FS_INLINE_DOTS) 1985 set_bit(FI_INLINE_DOTS, &fi->flags); 1986 + if (ri->i_inline & F2FS_EXTRA_ATTR) 1987 + set_bit(FI_EXTRA_ATTR, &fi->flags); 1988 } 1989 1990 static inline void set_raw_inline(struct inode *inode, struct f2fs_inode *ri) ··· 1999 ri->i_inline |= F2FS_DATA_EXIST; 2000 if (is_inode_flag_set(inode, FI_INLINE_DOTS)) 2001 ri->i_inline |= F2FS_INLINE_DOTS; 2002 + if (is_inode_flag_set(inode, FI_EXTRA_ATTR)) 2003 + ri->i_inline |= F2FS_EXTRA_ATTR; 2004 + } 2005 + 2006 + static inline int f2fs_has_extra_attr(struct inode *inode) 2007 + { 2008 + return is_inode_flag_set(inode, FI_EXTRA_ATTR); 2009 } 2010 2011 static inline int f2fs_has_inline_xattr(struct inode *inode) ··· 2009 static inline unsigned int addrs_per_inode(struct inode *inode) 2010 { 2011 if (f2fs_has_inline_xattr(inode)) 2012 + return CUR_ADDRS_PER_INODE(inode) - F2FS_INLINE_XATTR_ADDRS; 2013 + return CUR_ADDRS_PER_INODE(inode); 2014 } 2015 2016 static inline void *inline_xattr_addr(struct page *page) ··· 2069 return is_inode_flag_set(inode, FI_DROP_CACHE); 2070 } 2071 2072 + static inline void *inline_data_addr(struct inode *inode, struct page *page) 2073 { 2074 struct f2fs_inode *ri = F2FS_INODE(page); 2075 + int extra_size = get_extra_isize(inode); 2076 2077 + return (void *)&(ri->i_addr[extra_size + DEF_INLINE_RESERVED_SIZE]); 2078 } 2079 2080 static inline int f2fs_has_inline_dentry(struct inode *inode) ··· 2164 return kmalloc(size, flags); 2165 } 2166 2167 + static inline int get_extra_isize(struct inode *inode) 2168 + { 2169 + return F2FS_I(inode)->i_extra_isize / sizeof(__le32); 2170 + } 2171 + 2172 #define get_inode_mode(i) \ 2173 ((is_inode_flag_set(i, FI_ACL_MODE)) ? \ 2174 (F2FS_I(i)->i_acl_mode) : ((i)->i_mode)) 2175 + 2176 + #define F2FS_TOTAL_EXTRA_ATTR_SIZE \ 2177 + (offsetof(struct f2fs_inode, i_extra_end) - \ 2178 + offsetof(struct f2fs_inode, i_extra_isize)) \ 2179 + 2180 + #define F2FS_OLD_ATTRIBUTE_SIZE (offsetof(struct f2fs_inode, i_addr)) 2181 + #define F2FS_FITS_IN_INODE(f2fs_inode, extra_isize, field) \ 2182 + ((offsetof(typeof(*f2fs_inode), field) + \ 2183 + sizeof((f2fs_inode)->field)) \ 2184 + <= (F2FS_OLD_ATTRIBUTE_SIZE + extra_isize)) \ 2185 + 2186 + static inline void f2fs_reset_iostat(struct f2fs_sb_info *sbi) 2187 + { 2188 + int i; 2189 + 2190 + spin_lock(&sbi->iostat_lock); 2191 + for (i = 0; i < NR_IO_TYPE; i++) 2192 + sbi->write_iostat[i] = 0; 2193 + spin_unlock(&sbi->iostat_lock); 2194 + } 2195 + 2196 + static inline void f2fs_update_iostat(struct f2fs_sb_info *sbi, 2197 + enum iostat_type type, unsigned long long io_bytes) 2198 + { 2199 + if (!sbi->iostat_enable) 2200 + return; 2201 + spin_lock(&sbi->iostat_lock); 2202 + sbi->write_iostat[type] += io_bytes; 2203 + 2204 + if (type == APP_WRITE_IO || type == APP_DIRECT_IO) 2205 + sbi->write_iostat[APP_BUFFERED_IO] = 2206 + sbi->write_iostat[APP_WRITE_IO] - 2207 + sbi->write_iostat[APP_DIRECT_IO]; 2208 + spin_unlock(&sbi->iostat_lock); 2209 + } 2210 2211 /* 2212 * file.c ··· 2187 * inode.c 2188 */ 2189 void f2fs_set_inode_flags(struct inode *inode); 2190 + bool f2fs_inode_chksum_verify(struct f2fs_sb_info *sbi, struct page *page); 2191 + void f2fs_inode_chksum_set(struct f2fs_sb_info *sbi, struct page *page); 2192 struct inode *f2fs_iget(struct super_block *sb, unsigned long ino); 2193 struct inode *f2fs_iget_retry(struct super_block *sb, unsigned long ino); 2194 int try_to_free_nats(struct f2fs_sb_info *sbi, int nr_shrink); ··· 2255 */ 2256 int f2fs_inode_dirtied(struct inode *inode, bool sync); 2257 void f2fs_inode_synced(struct inode *inode); 2258 + void f2fs_enable_quota_files(struct f2fs_sb_info *sbi); 2259 + void f2fs_quota_off_umount(struct super_block *sb); 2260 int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover); 2261 int f2fs_sync_fs(struct super_block *sb, int sync); 2262 extern __printf(3, 4) ··· 2285 int wait_on_node_pages_writeback(struct f2fs_sb_info *sbi, nid_t ino); 2286 int remove_inode_page(struct inode *inode); 2287 struct page *new_inode_page(struct inode *inode); 2288 + struct page *new_node_page(struct dnode_of_data *dn, unsigned int ofs); 2289 void ra_node_page(struct f2fs_sb_info *sbi, nid_t nid); 2290 struct page *get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid); 2291 struct page *get_node_page_ra(struct page *parent, int start); 2292 void move_node_page(struct page *node_page, int gc_type); 2293 int fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode, 2294 struct writeback_control *wbc, bool atomic); 2295 + int sync_node_pages(struct f2fs_sb_info *sbi, struct writeback_control *wbc, 2296 + bool do_balance, enum iostat_type io_type); 2297 void build_free_nids(struct f2fs_sb_info *sbi, bool sync, bool mount); 2298 bool alloc_nid(struct f2fs_sb_info *sbi, nid_t *nid); 2299 void alloc_nid_done(struct f2fs_sb_info *sbi, nid_t nid); ··· 2314 /* 2315 * segment.c 2316 */ 2317 + bool need_SSR(struct f2fs_sb_info *sbi); 2318 void register_inmem_page(struct inode *inode, struct page *page); 2319 void drop_inmem_pages(struct inode *inode); 2320 void drop_inmem_page(struct inode *inode, struct page *page); ··· 2336 bool exist_trim_candidates(struct f2fs_sb_info *sbi, struct cp_control *cpc); 2337 struct page *get_sum_page(struct f2fs_sb_info *sbi, unsigned int segno); 2338 void update_meta_page(struct f2fs_sb_info *sbi, void *src, block_t blk_addr); 2339 + void write_meta_page(struct f2fs_sb_info *sbi, struct page *page, 2340 + enum iostat_type io_type); 2341 void write_node_page(unsigned int nid, struct f2fs_io_info *fio); 2342 void write_data_page(struct dnode_of_data *dn, struct f2fs_io_info *fio); 2343 int rewrite_data_page(struct f2fs_io_info *fio); ··· 2353 struct f2fs_io_info *fio, bool add_list); 2354 void f2fs_wait_on_page_writeback(struct page *page, 2355 enum page_type type, bool ordered); 2356 + void f2fs_wait_on_block_writeback(struct f2fs_sb_info *sbi, block_t blkaddr); 2357 void write_data_summaries(struct f2fs_sb_info *sbi, block_t start_blk); 2358 void write_node_summaries(struct f2fs_sb_info *sbi, block_t start_blk); 2359 int lookup_journal_in_cursum(struct f2fs_journal *journal, int type, ··· 2377 int type, bool sync); 2378 void ra_meta_pages_cond(struct f2fs_sb_info *sbi, pgoff_t index); 2379 long sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type, 2380 + long nr_to_write, enum iostat_type io_type); 2381 void add_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type); 2382 void remove_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type); 2383 void release_ino_entry(struct f2fs_sb_info *sbi, bool all); ··· 2430 int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, 2431 u64 start, u64 len); 2432 void f2fs_set_page_dirty_nobuffers(struct page *page); 2433 + int __f2fs_write_data_pages(struct address_space *mapping, 2434 + struct writeback_control *wbc, 2435 + enum iostat_type io_type); 2436 void f2fs_invalidate_page(struct page *page, unsigned int offset, 2437 unsigned int length); 2438 int f2fs_release_page(struct page *page, gfp_t wait); ··· 2726 /* 2727 * sysfs.c 2728 */ 2729 + int __init f2fs_init_sysfs(void); 2730 + void f2fs_exit_sysfs(void); 2731 + int f2fs_register_sysfs(struct f2fs_sb_info *sbi); 2732 + void f2fs_unregister_sysfs(struct f2fs_sb_info *sbi); 2733 2734 /* 2735 * crypto support ··· 2737 static inline bool f2fs_encrypted_inode(struct inode *inode) 2738 { 2739 return file_is_encrypt(inode); 2740 + } 2741 + 2742 + static inline bool f2fs_encrypted_file(struct inode *inode) 2743 + { 2744 + return f2fs_encrypted_inode(inode) && S_ISREG(inode->i_mode); 2745 } 2746 2747 static inline void f2fs_set_encrypted_inode(struct inode *inode) ··· 2759 static inline int f2fs_sb_mounted_blkzoned(struct super_block *sb) 2760 { 2761 return F2FS_HAS_FEATURE(sb, F2FS_FEATURE_BLKZONED); 2762 + } 2763 + 2764 + static inline int f2fs_sb_has_extra_attr(struct super_block *sb) 2765 + { 2766 + return F2FS_HAS_FEATURE(sb, F2FS_FEATURE_EXTRA_ATTR); 2767 + } 2768 + 2769 + static inline int f2fs_sb_has_project_quota(struct super_block *sb) 2770 + { 2771 + return F2FS_HAS_FEATURE(sb, F2FS_FEATURE_PRJQUOTA); 2772 + } 2773 + 2774 + static inline int f2fs_sb_has_inode_chksum(struct super_block *sb) 2775 + { 2776 + return F2FS_HAS_FEATURE(sb, F2FS_FEATURE_INODE_CHKSUM); 2777 } 2778 2779 #ifdef CONFIG_BLK_DEV_ZONED
+305 -55
fs/f2fs/file.c
··· 98 if (!PageUptodate(page)) 99 SetPageUptodate(page); 100 101 trace_f2fs_vm_page_mkwrite(page, DATA); 102 mapped: 103 /* fill the page */ 104 f2fs_wait_on_page_writeback(page, DATA, false); 105 106 /* wait for GCed encrypted page writeback */ 107 - if (f2fs_encrypted_inode(inode) && S_ISREG(inode->i_mode)) 108 - f2fs_wait_on_encrypted_page_writeback(sbi, dn.data_blkaddr); 109 110 out_sem: 111 up_read(&F2FS_I(inode)->i_mmap_sem); ··· 276 goto sync_nodes; 277 } 278 279 - ret = wait_on_node_pages_writeback(sbi, ino); 280 - if (ret) 281 - goto out; 282 283 /* once recovery info is written, don't need to tack this */ 284 remove_ino_entry(sbi, ino, APPEND_INO); ··· 394 dn.ofs_in_node++, pgofs++, 395 data_ofs = (loff_t)pgofs << PAGE_SHIFT) { 396 block_t blkaddr; 397 - blkaddr = datablock_addr(dn.node_page, dn.ofs_in_node); 398 399 if (__found_offset(blkaddr, dirty, pgofs, whence)) { 400 f2fs_put_dnode(&dn); ··· 480 struct f2fs_node *raw_node; 481 int nr_free = 0, ofs = dn->ofs_in_node, len = count; 482 __le32 *addr; 483 484 raw_node = F2FS_NODE(dn->node_page); 485 - addr = blkaddr_in_node(raw_node) + ofs; 486 487 for (; count > 0; count--, addr++, dn->ofs_in_node++) { 488 block_t blkaddr = le32_to_cpu(*addr); ··· 664 struct f2fs_inode_info *fi = F2FS_I(inode); 665 unsigned int flags; 666 667 - flags = fi->i_flags & FS_FL_USER_VISIBLE; 668 if (flags & FS_APPEND_FL) 669 stat->attributes |= STATX_ATTR_APPEND; 670 if (flags & FS_COMPR_FL) ··· 944 done = min((pgoff_t)ADDRS_PER_PAGE(dn.node_page, inode) - 945 dn.ofs_in_node, len); 946 for (i = 0; i < done; i++, blkaddr++, do_replace++, dn.ofs_in_node++) { 947 - *blkaddr = datablock_addr(dn.node_page, dn.ofs_in_node); 948 if (!is_checkpointed_data(sbi, *blkaddr)) { 949 950 if (test_opt(sbi, LFS)) { ··· 1021 ADDRS_PER_PAGE(dn.node_page, dst_inode) - 1022 dn.ofs_in_node, len - i); 1023 do { 1024 - dn.data_blkaddr = datablock_addr(dn.node_page, 1025 - dn.ofs_in_node); 1026 truncate_data_blocks_range(&dn, 1); 1027 1028 if (do_replace[i]) { ··· 1191 int ret; 1192 1193 for (; index < end; index++, dn->ofs_in_node++) { 1194 - if (datablock_addr(dn->node_page, dn->ofs_in_node) == NULL_ADDR) 1195 count++; 1196 } 1197 ··· 1203 1204 dn->ofs_in_node = ofs_in_node; 1205 for (index = start; index < end; index++, dn->ofs_in_node++) { 1206 - dn->data_blkaddr = 1207 - datablock_addr(dn->node_page, dn->ofs_in_node); 1208 /* 1209 * reserve_new_blocks will not guarantee entire block 1210 * allocation. ··· 1514 return 0; 1515 } 1516 1517 - #define F2FS_REG_FLMASK (~(FS_DIRSYNC_FL | FS_TOPDIR_FL)) 1518 - #define F2FS_OTHER_FLMASK (FS_NODUMP_FL | FS_NOATIME_FL) 1519 - 1520 - static inline __u32 f2fs_mask_flags(umode_t mode, __u32 flags) 1521 { 1522 - if (S_ISDIR(mode)) 1523 - return flags; 1524 - else if (S_ISREG(mode)) 1525 - return flags & F2FS_REG_FLMASK; 1526 - else 1527 - return flags & F2FS_OTHER_FLMASK; 1528 } 1529 1530 static int f2fs_ioc_getflags(struct file *filp, unsigned long arg) 1531 { 1532 struct inode *inode = file_inode(filp); 1533 struct f2fs_inode_info *fi = F2FS_I(inode); 1534 - unsigned int flags = fi->i_flags & FS_FL_USER_VISIBLE; 1535 return put_user(flags, (int __user *)arg); 1536 } 1537 1538 static int f2fs_ioc_setflags(struct file *filp, unsigned long arg) 1539 { 1540 struct inode *inode = file_inode(filp); 1541 - struct f2fs_inode_info *fi = F2FS_I(inode); 1542 unsigned int flags; 1543 - unsigned int oldflags; 1544 int ret; 1545 1546 if (!inode_owner_or_capable(inode)) ··· 1589 1590 inode_lock(inode); 1591 1592 - /* Is it quota file? Do not allow user to mess with it */ 1593 - if (IS_NOQUOTA(inode)) { 1594 - ret = -EPERM; 1595 - goto unlock_out; 1596 - } 1597 1598 - flags = f2fs_mask_flags(inode->i_mode, flags); 1599 - 1600 - oldflags = fi->i_flags; 1601 - 1602 - if ((flags ^ oldflags) & (FS_APPEND_FL | FS_IMMUTABLE_FL)) { 1603 - if (!capable(CAP_LINUX_IMMUTABLE)) { 1604 - ret = -EPERM; 1605 - goto unlock_out; 1606 - } 1607 - } 1608 - 1609 - flags = flags & FS_FL_USER_MODIFIABLE; 1610 - flags |= oldflags & ~FS_FL_USER_MODIFIABLE; 1611 - fi->i_flags = flags; 1612 - 1613 - inode->i_ctime = current_time(inode); 1614 - f2fs_set_inode_flags(inode); 1615 - f2fs_mark_inode_dirty_sync(inode, false); 1616 - unlock_out: 1617 inode_unlock(inode); 1618 mnt_drop_write_file(filp); 1619 return ret; ··· 1640 ret = filemap_write_and_wait_range(inode->i_mapping, 0, LLONG_MAX); 1641 if (ret) { 1642 clear_inode_flag(inode, FI_ATOMIC_FILE); 1643 goto out; 1644 } 1645 1646 inc_stat: 1647 stat_inc_atomic_write(inode); 1648 stat_update_max_atomic_write(inode); 1649 out: ··· 1679 ret = f2fs_do_sync_file(filp, 0, LLONG_MAX, 0, true); 1680 if (!ret) { 1681 clear_inode_flag(inode, FI_ATOMIC_FILE); 1682 stat_dec_atomic_write(inode); 1683 } 1684 } else { 1685 - ret = f2fs_do_sync_file(filp, 0, LLONG_MAX, 0, true); 1686 } 1687 err_out: 1688 inode_unlock(inode); ··· 1819 f2fs_stop_checkpoint(sbi, false); 1820 break; 1821 case F2FS_GOING_DOWN_METAFLUSH: 1822 - sync_meta_pages(sbi, META, LONG_MAX); 1823 f2fs_stop_checkpoint(sbi, false); 1824 break; 1825 default: ··· 2076 */ 2077 while (map.m_lblk < pg_end) { 2078 map.m_len = pg_end - map.m_lblk; 2079 - err = f2fs_map_blocks(inode, &map, 0, F2FS_GET_BLOCK_READ); 2080 if (err) 2081 goto out; 2082 ··· 2118 2119 do_map: 2120 map.m_len = pg_end - map.m_lblk; 2121 - err = f2fs_map_blocks(inode, &map, 0, F2FS_GET_BLOCK_READ); 2122 if (err) 2123 goto clear_out; 2124 ··· 2417 return ret; 2418 } 2419 2420 2421 long f2fs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) 2422 { ··· 2663 return f2fs_ioc_move_range(filp, arg); 2664 case F2FS_IOC_FLUSH_DEVICE: 2665 return f2fs_ioc_flush_device(filp, arg); 2666 default: 2667 return -ENOTTY; 2668 } ··· 2698 ret = __generic_file_write_iter(iocb, from); 2699 blk_finish_plug(&plug); 2700 clear_inode_flag(inode, FI_NO_PREALLOC); 2701 } 2702 inode_unlock(inode); 2703 ··· 2737 case F2FS_IOC_DEFRAGMENT: 2738 case F2FS_IOC_MOVE_RANGE: 2739 case F2FS_IOC_FLUSH_DEVICE: 2740 break; 2741 default: 2742 return -ENOIOCTLCMD; ··· 2755 .open = f2fs_file_open, 2756 .release = f2fs_release_file, 2757 .mmap = f2fs_file_mmap, 2758 .fsync = f2fs_sync_file, 2759 .fallocate = f2fs_fallocate, 2760 .unlocked_ioctl = f2fs_ioctl,
··· 98 if (!PageUptodate(page)) 99 SetPageUptodate(page); 100 101 + f2fs_update_iostat(sbi, APP_MAPPED_IO, F2FS_BLKSIZE); 102 + 103 trace_f2fs_vm_page_mkwrite(page, DATA); 104 mapped: 105 /* fill the page */ 106 f2fs_wait_on_page_writeback(page, DATA, false); 107 108 /* wait for GCed encrypted page writeback */ 109 + if (f2fs_encrypted_file(inode)) 110 + f2fs_wait_on_block_writeback(sbi, dn.data_blkaddr); 111 112 out_sem: 113 up_read(&F2FS_I(inode)->i_mmap_sem); ··· 274 goto sync_nodes; 275 } 276 277 + /* 278 + * If it's atomic_write, it's just fine to keep write ordering. So 279 + * here we don't need to wait for node write completion, since we use 280 + * node chain which serializes node blocks. If one of node writes are 281 + * reordered, we can see simply broken chain, resulting in stopping 282 + * roll-forward recovery. It means we'll recover all or none node blocks 283 + * given fsync mark. 284 + */ 285 + if (!atomic) { 286 + ret = wait_on_node_pages_writeback(sbi, ino); 287 + if (ret) 288 + goto out; 289 + } 290 291 /* once recovery info is written, don't need to tack this */ 292 remove_ino_entry(sbi, ino, APPEND_INO); ··· 382 dn.ofs_in_node++, pgofs++, 383 data_ofs = (loff_t)pgofs << PAGE_SHIFT) { 384 block_t blkaddr; 385 + blkaddr = datablock_addr(dn.inode, 386 + dn.node_page, dn.ofs_in_node); 387 388 if (__found_offset(blkaddr, dirty, pgofs, whence)) { 389 f2fs_put_dnode(&dn); ··· 467 struct f2fs_node *raw_node; 468 int nr_free = 0, ofs = dn->ofs_in_node, len = count; 469 __le32 *addr; 470 + int base = 0; 471 + 472 + if (IS_INODE(dn->node_page) && f2fs_has_extra_attr(dn->inode)) 473 + base = get_extra_isize(dn->inode); 474 475 raw_node = F2FS_NODE(dn->node_page); 476 + addr = blkaddr_in_node(raw_node) + base + ofs; 477 478 for (; count > 0; count--, addr++, dn->ofs_in_node++) { 479 block_t blkaddr = le32_to_cpu(*addr); ··· 647 struct f2fs_inode_info *fi = F2FS_I(inode); 648 unsigned int flags; 649 650 + flags = fi->i_flags & (FS_FL_USER_VISIBLE | FS_PROJINHERIT_FL); 651 if (flags & FS_APPEND_FL) 652 stat->attributes |= STATX_ATTR_APPEND; 653 if (flags & FS_COMPR_FL) ··· 927 done = min((pgoff_t)ADDRS_PER_PAGE(dn.node_page, inode) - 928 dn.ofs_in_node, len); 929 for (i = 0; i < done; i++, blkaddr++, do_replace++, dn.ofs_in_node++) { 930 + *blkaddr = datablock_addr(dn.inode, 931 + dn.node_page, dn.ofs_in_node); 932 if (!is_checkpointed_data(sbi, *blkaddr)) { 933 934 if (test_opt(sbi, LFS)) { ··· 1003 ADDRS_PER_PAGE(dn.node_page, dst_inode) - 1004 dn.ofs_in_node, len - i); 1005 do { 1006 + dn.data_blkaddr = datablock_addr(dn.inode, 1007 + dn.node_page, dn.ofs_in_node); 1008 truncate_data_blocks_range(&dn, 1); 1009 1010 if (do_replace[i]) { ··· 1173 int ret; 1174 1175 for (; index < end; index++, dn->ofs_in_node++) { 1176 + if (datablock_addr(dn->inode, dn->node_page, 1177 + dn->ofs_in_node) == NULL_ADDR) 1178 count++; 1179 } 1180 ··· 1184 1185 dn->ofs_in_node = ofs_in_node; 1186 for (index = start; index < end; index++, dn->ofs_in_node++) { 1187 + dn->data_blkaddr = datablock_addr(dn->inode, 1188 + dn->node_page, dn->ofs_in_node); 1189 /* 1190 * reserve_new_blocks will not guarantee entire block 1191 * allocation. ··· 1495 return 0; 1496 } 1497 1498 + static int f2fs_file_flush(struct file *file, fl_owner_t id) 1499 { 1500 + struct inode *inode = file_inode(file); 1501 + 1502 + /* 1503 + * If the process doing a transaction is crashed, we should do 1504 + * roll-back. Otherwise, other reader/write can see corrupted database 1505 + * until all the writers close its file. Since this should be done 1506 + * before dropping file lock, it needs to do in ->flush. 1507 + */ 1508 + if (f2fs_is_atomic_file(inode) && 1509 + F2FS_I(inode)->inmem_task == current) 1510 + drop_inmem_pages(inode); 1511 + return 0; 1512 } 1513 1514 static int f2fs_ioc_getflags(struct file *filp, unsigned long arg) 1515 { 1516 struct inode *inode = file_inode(filp); 1517 struct f2fs_inode_info *fi = F2FS_I(inode); 1518 + unsigned int flags = fi->i_flags & 1519 + (FS_FL_USER_VISIBLE | FS_PROJINHERIT_FL); 1520 return put_user(flags, (int __user *)arg); 1521 + } 1522 + 1523 + static int __f2fs_ioc_setflags(struct inode *inode, unsigned int flags) 1524 + { 1525 + struct f2fs_inode_info *fi = F2FS_I(inode); 1526 + unsigned int oldflags; 1527 + 1528 + /* Is it quota file? Do not allow user to mess with it */ 1529 + if (IS_NOQUOTA(inode)) 1530 + return -EPERM; 1531 + 1532 + flags = f2fs_mask_flags(inode->i_mode, flags); 1533 + 1534 + oldflags = fi->i_flags; 1535 + 1536 + if ((flags ^ oldflags) & (FS_APPEND_FL | FS_IMMUTABLE_FL)) 1537 + if (!capable(CAP_LINUX_IMMUTABLE)) 1538 + return -EPERM; 1539 + 1540 + flags = flags & (FS_FL_USER_MODIFIABLE | FS_PROJINHERIT_FL); 1541 + flags |= oldflags & ~(FS_FL_USER_MODIFIABLE | FS_PROJINHERIT_FL); 1542 + fi->i_flags = flags; 1543 + 1544 + if (fi->i_flags & FS_PROJINHERIT_FL) 1545 + set_inode_flag(inode, FI_PROJ_INHERIT); 1546 + else 1547 + clear_inode_flag(inode, FI_PROJ_INHERIT); 1548 + 1549 + inode->i_ctime = current_time(inode); 1550 + f2fs_set_inode_flags(inode); 1551 + f2fs_mark_inode_dirty_sync(inode, false); 1552 + return 0; 1553 } 1554 1555 static int f2fs_ioc_setflags(struct file *filp, unsigned long arg) 1556 { 1557 struct inode *inode = file_inode(filp); 1558 unsigned int flags; 1559 int ret; 1560 1561 if (!inode_owner_or_capable(inode)) ··· 1536 1537 inode_lock(inode); 1538 1539 + ret = __f2fs_ioc_setflags(inode, flags); 1540 1541 inode_unlock(inode); 1542 mnt_drop_write_file(filp); 1543 return ret; ··· 1610 ret = filemap_write_and_wait_range(inode->i_mapping, 0, LLONG_MAX); 1611 if (ret) { 1612 clear_inode_flag(inode, FI_ATOMIC_FILE); 1613 + clear_inode_flag(inode, FI_HOT_DATA); 1614 goto out; 1615 } 1616 1617 inc_stat: 1618 + F2FS_I(inode)->inmem_task = current; 1619 stat_inc_atomic_write(inode); 1620 stat_update_max_atomic_write(inode); 1621 out: ··· 1647 ret = f2fs_do_sync_file(filp, 0, LLONG_MAX, 0, true); 1648 if (!ret) { 1649 clear_inode_flag(inode, FI_ATOMIC_FILE); 1650 + clear_inode_flag(inode, FI_HOT_DATA); 1651 stat_dec_atomic_write(inode); 1652 } 1653 } else { 1654 + ret = f2fs_do_sync_file(filp, 0, LLONG_MAX, 1, false); 1655 } 1656 err_out: 1657 inode_unlock(inode); ··· 1786 f2fs_stop_checkpoint(sbi, false); 1787 break; 1788 case F2FS_GOING_DOWN_METAFLUSH: 1789 + sync_meta_pages(sbi, META, LONG_MAX, FS_META_IO); 1790 f2fs_stop_checkpoint(sbi, false); 1791 break; 1792 default: ··· 2043 */ 2044 while (map.m_lblk < pg_end) { 2045 map.m_len = pg_end - map.m_lblk; 2046 + err = f2fs_map_blocks(inode, &map, 0, F2FS_GET_BLOCK_DEFAULT); 2047 if (err) 2048 goto out; 2049 ··· 2085 2086 do_map: 2087 map.m_len = pg_end - map.m_lblk; 2088 + err = f2fs_map_blocks(inode, &map, 0, F2FS_GET_BLOCK_DEFAULT); 2089 if (err) 2090 goto clear_out; 2091 ··· 2384 return ret; 2385 } 2386 2387 + static int f2fs_ioc_get_features(struct file *filp, unsigned long arg) 2388 + { 2389 + struct inode *inode = file_inode(filp); 2390 + u32 sb_feature = le32_to_cpu(F2FS_I_SB(inode)->raw_super->feature); 2391 + 2392 + /* Must validate to set it with SQLite behavior in Android. */ 2393 + sb_feature |= F2FS_FEATURE_ATOMIC_WRITE; 2394 + 2395 + return put_user(sb_feature, (u32 __user *)arg); 2396 + } 2397 + 2398 + #ifdef CONFIG_QUOTA 2399 + static int f2fs_ioc_setproject(struct file *filp, __u32 projid) 2400 + { 2401 + struct inode *inode = file_inode(filp); 2402 + struct f2fs_inode_info *fi = F2FS_I(inode); 2403 + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); 2404 + struct super_block *sb = sbi->sb; 2405 + struct dquot *transfer_to[MAXQUOTAS] = {}; 2406 + struct page *ipage; 2407 + kprojid_t kprojid; 2408 + int err; 2409 + 2410 + if (!f2fs_sb_has_project_quota(sb)) { 2411 + if (projid != F2FS_DEF_PROJID) 2412 + return -EOPNOTSUPP; 2413 + else 2414 + return 0; 2415 + } 2416 + 2417 + if (!f2fs_has_extra_attr(inode)) 2418 + return -EOPNOTSUPP; 2419 + 2420 + kprojid = make_kprojid(&init_user_ns, (projid_t)projid); 2421 + 2422 + if (projid_eq(kprojid, F2FS_I(inode)->i_projid)) 2423 + return 0; 2424 + 2425 + err = mnt_want_write_file(filp); 2426 + if (err) 2427 + return err; 2428 + 2429 + err = -EPERM; 2430 + inode_lock(inode); 2431 + 2432 + /* Is it quota file? Do not allow user to mess with it */ 2433 + if (IS_NOQUOTA(inode)) 2434 + goto out_unlock; 2435 + 2436 + ipage = get_node_page(sbi, inode->i_ino); 2437 + if (IS_ERR(ipage)) { 2438 + err = PTR_ERR(ipage); 2439 + goto out_unlock; 2440 + } 2441 + 2442 + if (!F2FS_FITS_IN_INODE(F2FS_INODE(ipage), fi->i_extra_isize, 2443 + i_projid)) { 2444 + err = -EOVERFLOW; 2445 + f2fs_put_page(ipage, 1); 2446 + goto out_unlock; 2447 + } 2448 + f2fs_put_page(ipage, 1); 2449 + 2450 + dquot_initialize(inode); 2451 + 2452 + transfer_to[PRJQUOTA] = dqget(sb, make_kqid_projid(kprojid)); 2453 + if (!IS_ERR(transfer_to[PRJQUOTA])) { 2454 + err = __dquot_transfer(inode, transfer_to); 2455 + dqput(transfer_to[PRJQUOTA]); 2456 + if (err) 2457 + goto out_dirty; 2458 + } 2459 + 2460 + F2FS_I(inode)->i_projid = kprojid; 2461 + inode->i_ctime = current_time(inode); 2462 + out_dirty: 2463 + f2fs_mark_inode_dirty_sync(inode, true); 2464 + out_unlock: 2465 + inode_unlock(inode); 2466 + mnt_drop_write_file(filp); 2467 + return err; 2468 + } 2469 + #else 2470 + static int f2fs_ioc_setproject(struct file *filp, __u32 projid) 2471 + { 2472 + if (projid != F2FS_DEF_PROJID) 2473 + return -EOPNOTSUPP; 2474 + return 0; 2475 + } 2476 + #endif 2477 + 2478 + /* Transfer internal flags to xflags */ 2479 + static inline __u32 f2fs_iflags_to_xflags(unsigned long iflags) 2480 + { 2481 + __u32 xflags = 0; 2482 + 2483 + if (iflags & FS_SYNC_FL) 2484 + xflags |= FS_XFLAG_SYNC; 2485 + if (iflags & FS_IMMUTABLE_FL) 2486 + xflags |= FS_XFLAG_IMMUTABLE; 2487 + if (iflags & FS_APPEND_FL) 2488 + xflags |= FS_XFLAG_APPEND; 2489 + if (iflags & FS_NODUMP_FL) 2490 + xflags |= FS_XFLAG_NODUMP; 2491 + if (iflags & FS_NOATIME_FL) 2492 + xflags |= FS_XFLAG_NOATIME; 2493 + if (iflags & FS_PROJINHERIT_FL) 2494 + xflags |= FS_XFLAG_PROJINHERIT; 2495 + return xflags; 2496 + } 2497 + 2498 + #define F2FS_SUPPORTED_FS_XFLAGS (FS_XFLAG_SYNC | FS_XFLAG_IMMUTABLE | \ 2499 + FS_XFLAG_APPEND | FS_XFLAG_NODUMP | \ 2500 + FS_XFLAG_NOATIME | FS_XFLAG_PROJINHERIT) 2501 + 2502 + /* Flags we can manipulate with through EXT4_IOC_FSSETXATTR */ 2503 + #define F2FS_FL_XFLAG_VISIBLE (FS_SYNC_FL | \ 2504 + FS_IMMUTABLE_FL | \ 2505 + FS_APPEND_FL | \ 2506 + FS_NODUMP_FL | \ 2507 + FS_NOATIME_FL | \ 2508 + FS_PROJINHERIT_FL) 2509 + 2510 + /* Transfer xflags flags to internal */ 2511 + static inline unsigned long f2fs_xflags_to_iflags(__u32 xflags) 2512 + { 2513 + unsigned long iflags = 0; 2514 + 2515 + if (xflags & FS_XFLAG_SYNC) 2516 + iflags |= FS_SYNC_FL; 2517 + if (xflags & FS_XFLAG_IMMUTABLE) 2518 + iflags |= FS_IMMUTABLE_FL; 2519 + if (xflags & FS_XFLAG_APPEND) 2520 + iflags |= FS_APPEND_FL; 2521 + if (xflags & FS_XFLAG_NODUMP) 2522 + iflags |= FS_NODUMP_FL; 2523 + if (xflags & FS_XFLAG_NOATIME) 2524 + iflags |= FS_NOATIME_FL; 2525 + if (xflags & FS_XFLAG_PROJINHERIT) 2526 + iflags |= FS_PROJINHERIT_FL; 2527 + 2528 + return iflags; 2529 + } 2530 + 2531 + static int f2fs_ioc_fsgetxattr(struct file *filp, unsigned long arg) 2532 + { 2533 + struct inode *inode = file_inode(filp); 2534 + struct f2fs_inode_info *fi = F2FS_I(inode); 2535 + struct fsxattr fa; 2536 + 2537 + memset(&fa, 0, sizeof(struct fsxattr)); 2538 + fa.fsx_xflags = f2fs_iflags_to_xflags(fi->i_flags & 2539 + (FS_FL_USER_VISIBLE | FS_PROJINHERIT_FL)); 2540 + 2541 + if (f2fs_sb_has_project_quota(inode->i_sb)) 2542 + fa.fsx_projid = (__u32)from_kprojid(&init_user_ns, 2543 + fi->i_projid); 2544 + 2545 + if (copy_to_user((struct fsxattr __user *)arg, &fa, sizeof(fa))) 2546 + return -EFAULT; 2547 + return 0; 2548 + } 2549 + 2550 + static int f2fs_ioc_fssetxattr(struct file *filp, unsigned long arg) 2551 + { 2552 + struct inode *inode = file_inode(filp); 2553 + struct f2fs_inode_info *fi = F2FS_I(inode); 2554 + struct fsxattr fa; 2555 + unsigned int flags; 2556 + int err; 2557 + 2558 + if (copy_from_user(&fa, (struct fsxattr __user *)arg, sizeof(fa))) 2559 + return -EFAULT; 2560 + 2561 + /* Make sure caller has proper permission */ 2562 + if (!inode_owner_or_capable(inode)) 2563 + return -EACCES; 2564 + 2565 + if (fa.fsx_xflags & ~F2FS_SUPPORTED_FS_XFLAGS) 2566 + return -EOPNOTSUPP; 2567 + 2568 + flags = f2fs_xflags_to_iflags(fa.fsx_xflags); 2569 + if (f2fs_mask_flags(inode->i_mode, flags) != flags) 2570 + return -EOPNOTSUPP; 2571 + 2572 + err = mnt_want_write_file(filp); 2573 + if (err) 2574 + return err; 2575 + 2576 + inode_lock(inode); 2577 + flags = (fi->i_flags & ~F2FS_FL_XFLAG_VISIBLE) | 2578 + (flags & F2FS_FL_XFLAG_VISIBLE); 2579 + err = __f2fs_ioc_setflags(inode, flags); 2580 + inode_unlock(inode); 2581 + mnt_drop_write_file(filp); 2582 + if (err) 2583 + return err; 2584 + 2585 + err = f2fs_ioc_setproject(filp, fa.fsx_projid); 2586 + if (err) 2587 + return err; 2588 + 2589 + return 0; 2590 + } 2591 2592 long f2fs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) 2593 { ··· 2426 return f2fs_ioc_move_range(filp, arg); 2427 case F2FS_IOC_FLUSH_DEVICE: 2428 return f2fs_ioc_flush_device(filp, arg); 2429 + case F2FS_IOC_GET_FEATURES: 2430 + return f2fs_ioc_get_features(filp, arg); 2431 + case F2FS_IOC_FSGETXATTR: 2432 + return f2fs_ioc_fsgetxattr(filp, arg); 2433 + case F2FS_IOC_FSSETXATTR: 2434 + return f2fs_ioc_fssetxattr(filp, arg); 2435 default: 2436 return -ENOTTY; 2437 } ··· 2455 ret = __generic_file_write_iter(iocb, from); 2456 blk_finish_plug(&plug); 2457 clear_inode_flag(inode, FI_NO_PREALLOC); 2458 + 2459 + if (ret > 0) 2460 + f2fs_update_iostat(F2FS_I_SB(inode), APP_WRITE_IO, ret); 2461 } 2462 inode_unlock(inode); 2463 ··· 2491 case F2FS_IOC_DEFRAGMENT: 2492 case F2FS_IOC_MOVE_RANGE: 2493 case F2FS_IOC_FLUSH_DEVICE: 2494 + case F2FS_IOC_GET_FEATURES: 2495 + case F2FS_IOC_FSGETXATTR: 2496 + case F2FS_IOC_FSSETXATTR: 2497 break; 2498 default: 2499 return -ENOIOCTLCMD; ··· 2506 .open = f2fs_file_open, 2507 .release = f2fs_release_file, 2508 .mmap = f2fs_file_mmap, 2509 + .flush = f2fs_file_flush, 2510 .fsync = f2fs_sync_file, 2511 .fallocate = f2fs_fallocate, 2512 .unlocked_ioctl = f2fs_ioctl,
+78 -37
fs/f2fs/gc.c
··· 28 struct f2fs_sb_info *sbi = data; 29 struct f2fs_gc_kthread *gc_th = sbi->gc_thread; 30 wait_queue_head_t *wq = &sbi->gc_thread->gc_wait_queue_head; 31 - long wait_ms; 32 33 wait_ms = gc_th->min_sleep_time; 34 35 set_freezable(); 36 do { 37 wait_event_interruptible_timeout(*wq, 38 - kthread_should_stop() || freezing(current), 39 msecs_to_jiffies(wait_ms)); 40 41 if (try_to_freeze()) 42 continue; ··· 60 } 61 #endif 62 63 /* 64 * [GC triggering condition] 65 * 0. GC is not conducted currently. ··· 77 * So, I'd like to wait some time to collect dirty segments. 78 */ 79 if (!mutex_trylock(&sbi->gc_mutex)) 80 - continue; 81 82 if (!is_idle(sbi)) { 83 increase_sleep_time(gc_th, &wait_ms); 84 mutex_unlock(&sbi->gc_mutex); 85 - continue; 86 } 87 88 if (has_enough_invalid_blocks(sbi)) 89 decrease_sleep_time(gc_th, &wait_ms); 90 else 91 increase_sleep_time(gc_th, &wait_ms); 92 - 93 stat_inc_bggc_count(sbi); 94 95 /* if return value is not zero, no victim was selected */ ··· 106 107 /* balancing f2fs's metadata periodically */ 108 f2fs_balance_fs_bg(sbi); 109 110 } while (!kthread_should_stop()); 111 return 0; ··· 125 goto out; 126 } 127 128 gc_th->min_sleep_time = DEF_GC_THREAD_MIN_SLEEP_TIME; 129 gc_th->max_sleep_time = DEF_GC_THREAD_MAX_SLEEP_TIME; 130 gc_th->no_gc_sleep_time = DEF_GC_THREAD_NOGC_SLEEP_TIME; 131 132 gc_th->gc_idle = 0; 133 134 sbi->gc_thread = gc_th; 135 init_waitqueue_head(&sbi->gc_thread->gc_wait_queue_head); ··· 277 valid_blocks * 2 : valid_blocks; 278 } 279 280 - static unsigned int get_ssr_cost(struct f2fs_sb_info *sbi, 281 - unsigned int segno) 282 - { 283 - struct seg_entry *se = get_seg_entry(sbi, segno); 284 - 285 - return se->ckpt_valid_blocks > se->valid_blocks ? 286 - se->ckpt_valid_blocks : se->valid_blocks; 287 - } 288 - 289 static inline unsigned int get_gc_cost(struct f2fs_sb_info *sbi, 290 unsigned int segno, struct victim_sel_policy *p) 291 { 292 if (p->alloc_mode == SSR) 293 - return get_ssr_cost(sbi, segno); 294 295 /* alloc_mode == LFS */ 296 if (p->gc_mode == GC_GREEDY) ··· 591 } 592 593 *nofs = ofs_of_node(node_page); 594 - source_blkaddr = datablock_addr(node_page, ofs_in_node); 595 f2fs_put_page(node_page, 1); 596 597 if (source_blkaddr != blkaddr) ··· 599 return true; 600 } 601 602 - static void move_encrypted_block(struct inode *inode, block_t bidx, 603 - unsigned int segno, int off) 604 { 605 struct f2fs_io_info fio = { 606 .sbi = F2FS_I_SB(inode), ··· 697 fio.new_blkaddr = newaddr; 698 f2fs_submit_page_write(&fio); 699 700 f2fs_update_data_blkaddr(&dn, newaddr); 701 set_inode_flag(inode, FI_APPEND_WRITE); 702 if (page->index == 0) ··· 746 .page = page, 747 .encrypted_page = NULL, 748 .need_lock = LOCK_REQ, 749 }; 750 bool is_dirty = PageDirty(page); 751 int err; ··· 835 continue; 836 837 /* if encrypted inode, let's go phase 3 */ 838 - if (f2fs_encrypted_inode(inode) && 839 - S_ISREG(inode->i_mode)) { 840 add_gc_inode(gc_list, inode); 841 continue; 842 } ··· 869 continue; 870 } 871 locked = true; 872 } 873 874 start_bidx = start_bidx_of_node(nofs, inode) 875 + ofs_in_node; 876 - if (f2fs_encrypted_inode(inode) && S_ISREG(inode->i_mode)) 877 - move_encrypted_block(inode, start_bidx, segno, off); 878 else 879 - move_data_page(inode, start_bidx, gc_type, segno, off); 880 881 if (locked) { 882 up_write(&fi->dio_rwsem[WRITE]); ··· 917 struct blk_plug plug; 918 unsigned int segno = start_segno; 919 unsigned int end_segno = start_segno + sbi->segs_per_sec; 920 - int sec_freed = 0; 921 unsigned char type = IS_DATASEG(get_seg_entry(sbi, segno)->type) ? 922 SUM_TYPE_DATA : SUM_TYPE_NODE; 923 ··· 963 gc_type); 964 965 stat_inc_seg_count(sbi, type, gc_type); 966 next: 967 f2fs_put_page(sum_page, 0); 968 } ··· 977 978 blk_finish_plug(&plug); 979 980 - if (gc_type == FG_GC && 981 - get_valid_blocks(sbi, start_segno, true) == 0) 982 - sec_freed = 1; 983 - 984 stat_inc_call_count(sbi->stat_info); 985 986 - return sec_freed; 987 } 988 989 int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, 990 bool background, unsigned int segno) 991 { 992 int gc_type = sync ? FG_GC : BG_GC; 993 - int sec_freed = 0; 994 - int ret; 995 struct cp_control cpc; 996 unsigned int init_segno = segno; 997 struct gc_inode_list gc_list = { 998 .ilist = LIST_HEAD_INIT(gc_list.ilist), 999 .iroot = RADIX_TREE_INIT(GFP_NOFS), 1000 }; 1001 1002 cpc.reason = __get_cp_reason(sbi); 1003 gc_more: ··· 1030 gc_type = FG_GC; 1031 } 1032 1033 - ret = -EINVAL; 1034 /* f2fs_balance_fs doesn't need to do BG_GC in critical path. */ 1035 - if (gc_type == BG_GC && !background) 1036 goto stop; 1037 - if (!__get_victim(sbi, &segno, gc_type)) 1038 goto stop; 1039 - ret = 0; 1040 1041 - if (do_garbage_collect(sbi, segno, &gc_list, gc_type) && 1042 - gc_type == FG_GC) 1043 sec_freed++; 1044 1045 if (gc_type == FG_GC) 1046 sbi->cur_victim_sec = NULL_SEGNO; ··· 1060 stop: 1061 SIT_I(sbi)->last_victim[ALLOC_NEXT] = 0; 1062 SIT_I(sbi)->last_victim[FLUSH_DEVICE] = init_segno; 1063 mutex_unlock(&sbi->gc_mutex); 1064 1065 put_gc_inode(&gc_list);
··· 28 struct f2fs_sb_info *sbi = data; 29 struct f2fs_gc_kthread *gc_th = sbi->gc_thread; 30 wait_queue_head_t *wq = &sbi->gc_thread->gc_wait_queue_head; 31 + unsigned int wait_ms; 32 33 wait_ms = gc_th->min_sleep_time; 34 35 set_freezable(); 36 do { 37 wait_event_interruptible_timeout(*wq, 38 + kthread_should_stop() || freezing(current) || 39 + gc_th->gc_wake, 40 msecs_to_jiffies(wait_ms)); 41 + 42 + /* give it a try one time */ 43 + if (gc_th->gc_wake) 44 + gc_th->gc_wake = 0; 45 46 if (try_to_freeze()) 47 continue; ··· 55 } 56 #endif 57 58 + if (!sb_start_write_trylock(sbi->sb)) 59 + continue; 60 + 61 /* 62 * [GC triggering condition] 63 * 0. GC is not conducted currently. ··· 69 * So, I'd like to wait some time to collect dirty segments. 70 */ 71 if (!mutex_trylock(&sbi->gc_mutex)) 72 + goto next; 73 + 74 + if (gc_th->gc_urgent) { 75 + wait_ms = gc_th->urgent_sleep_time; 76 + goto do_gc; 77 + } 78 79 if (!is_idle(sbi)) { 80 increase_sleep_time(gc_th, &wait_ms); 81 mutex_unlock(&sbi->gc_mutex); 82 + goto next; 83 } 84 85 if (has_enough_invalid_blocks(sbi)) 86 decrease_sleep_time(gc_th, &wait_ms); 87 else 88 increase_sleep_time(gc_th, &wait_ms); 89 + do_gc: 90 stat_inc_bggc_count(sbi); 91 92 /* if return value is not zero, no victim was selected */ ··· 93 94 /* balancing f2fs's metadata periodically */ 95 f2fs_balance_fs_bg(sbi); 96 + next: 97 + sb_end_write(sbi->sb); 98 99 } while (!kthread_should_stop()); 100 return 0; ··· 110 goto out; 111 } 112 113 + gc_th->urgent_sleep_time = DEF_GC_THREAD_URGENT_SLEEP_TIME; 114 gc_th->min_sleep_time = DEF_GC_THREAD_MIN_SLEEP_TIME; 115 gc_th->max_sleep_time = DEF_GC_THREAD_MAX_SLEEP_TIME; 116 gc_th->no_gc_sleep_time = DEF_GC_THREAD_NOGC_SLEEP_TIME; 117 118 gc_th->gc_idle = 0; 119 + gc_th->gc_urgent = 0; 120 + gc_th->gc_wake= 0; 121 122 sbi->gc_thread = gc_th; 123 init_waitqueue_head(&sbi->gc_thread->gc_wait_queue_head); ··· 259 valid_blocks * 2 : valid_blocks; 260 } 261 262 static inline unsigned int get_gc_cost(struct f2fs_sb_info *sbi, 263 unsigned int segno, struct victim_sel_policy *p) 264 { 265 if (p->alloc_mode == SSR) 266 + return get_seg_entry(sbi, segno)->ckpt_valid_blocks; 267 268 /* alloc_mode == LFS */ 269 if (p->gc_mode == GC_GREEDY) ··· 582 } 583 584 *nofs = ofs_of_node(node_page); 585 + source_blkaddr = datablock_addr(NULL, node_page, ofs_in_node); 586 f2fs_put_page(node_page, 1); 587 588 if (source_blkaddr != blkaddr) ··· 590 return true; 591 } 592 593 + /* 594 + * Move data block via META_MAPPING while keeping locked data page. 595 + * This can be used to move blocks, aka LBAs, directly on disk. 596 + */ 597 + static void move_data_block(struct inode *inode, block_t bidx, 598 + unsigned int segno, int off) 599 { 600 struct f2fs_io_info fio = { 601 .sbi = F2FS_I_SB(inode), ··· 684 fio.new_blkaddr = newaddr; 685 f2fs_submit_page_write(&fio); 686 687 + f2fs_update_iostat(fio.sbi, FS_GC_DATA_IO, F2FS_BLKSIZE); 688 + 689 f2fs_update_data_blkaddr(&dn, newaddr); 690 set_inode_flag(inode, FI_APPEND_WRITE); 691 if (page->index == 0) ··· 731 .page = page, 732 .encrypted_page = NULL, 733 .need_lock = LOCK_REQ, 734 + .io_type = FS_GC_DATA_IO, 735 }; 736 bool is_dirty = PageDirty(page); 737 int err; ··· 819 continue; 820 821 /* if encrypted inode, let's go phase 3 */ 822 + if (f2fs_encrypted_file(inode)) { 823 add_gc_inode(gc_list, inode); 824 continue; 825 } ··· 854 continue; 855 } 856 locked = true; 857 + 858 + /* wait for all inflight aio data */ 859 + inode_dio_wait(inode); 860 } 861 862 start_bidx = start_bidx_of_node(nofs, inode) 863 + ofs_in_node; 864 + if (f2fs_encrypted_file(inode)) 865 + move_data_block(inode, start_bidx, segno, off); 866 else 867 + move_data_page(inode, start_bidx, gc_type, 868 + segno, off); 869 870 if (locked) { 871 up_write(&fi->dio_rwsem[WRITE]); ··· 898 struct blk_plug plug; 899 unsigned int segno = start_segno; 900 unsigned int end_segno = start_segno + sbi->segs_per_sec; 901 + int seg_freed = 0; 902 unsigned char type = IS_DATASEG(get_seg_entry(sbi, segno)->type) ? 903 SUM_TYPE_DATA : SUM_TYPE_NODE; 904 ··· 944 gc_type); 945 946 stat_inc_seg_count(sbi, type, gc_type); 947 + 948 + if (gc_type == FG_GC && 949 + get_valid_blocks(sbi, segno, false) == 0) 950 + seg_freed++; 951 next: 952 f2fs_put_page(sum_page, 0); 953 } ··· 954 955 blk_finish_plug(&plug); 956 957 stat_inc_call_count(sbi->stat_info); 958 959 + return seg_freed; 960 } 961 962 int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, 963 bool background, unsigned int segno) 964 { 965 int gc_type = sync ? FG_GC : BG_GC; 966 + int sec_freed = 0, seg_freed = 0, total_freed = 0; 967 + int ret = 0; 968 struct cp_control cpc; 969 unsigned int init_segno = segno; 970 struct gc_inode_list gc_list = { 971 .ilist = LIST_HEAD_INIT(gc_list.ilist), 972 .iroot = RADIX_TREE_INIT(GFP_NOFS), 973 }; 974 + 975 + trace_f2fs_gc_begin(sbi->sb, sync, background, 976 + get_pages(sbi, F2FS_DIRTY_NODES), 977 + get_pages(sbi, F2FS_DIRTY_DENTS), 978 + get_pages(sbi, F2FS_DIRTY_IMETA), 979 + free_sections(sbi), 980 + free_segments(sbi), 981 + reserved_segments(sbi), 982 + prefree_segments(sbi)); 983 984 cpc.reason = __get_cp_reason(sbi); 985 gc_more: ··· 1002 gc_type = FG_GC; 1003 } 1004 1005 /* f2fs_balance_fs doesn't need to do BG_GC in critical path. */ 1006 + if (gc_type == BG_GC && !background) { 1007 + ret = -EINVAL; 1008 goto stop; 1009 + } 1010 + if (!__get_victim(sbi, &segno, gc_type)) { 1011 + ret = -ENODATA; 1012 goto stop; 1013 + } 1014 1015 + seg_freed = do_garbage_collect(sbi, segno, &gc_list, gc_type); 1016 + if (gc_type == FG_GC && seg_freed == sbi->segs_per_sec) 1017 sec_freed++; 1018 + total_freed += seg_freed; 1019 1020 if (gc_type == FG_GC) 1021 sbi->cur_victim_sec = NULL_SEGNO; ··· 1029 stop: 1030 SIT_I(sbi)->last_victim[ALLOC_NEXT] = 0; 1031 SIT_I(sbi)->last_victim[FLUSH_DEVICE] = init_segno; 1032 + 1033 + trace_f2fs_gc_end(sbi->sb, ret, total_freed, sec_freed, 1034 + get_pages(sbi, F2FS_DIRTY_NODES), 1035 + get_pages(sbi, F2FS_DIRTY_DENTS), 1036 + get_pages(sbi, F2FS_DIRTY_IMETA), 1037 + free_sections(sbi), 1038 + free_segments(sbi), 1039 + reserved_segments(sbi), 1040 + prefree_segments(sbi)); 1041 + 1042 mutex_unlock(&sbi->gc_mutex); 1043 1044 put_gc_inode(&gc_list);
+19 -8
fs/f2fs/gc.h
··· 13 * whether IO subsystem is idle 14 * or not 15 */ 16 #define DEF_GC_THREAD_MIN_SLEEP_TIME 30000 /* milliseconds */ 17 #define DEF_GC_THREAD_MAX_SLEEP_TIME 60000 18 #define DEF_GC_THREAD_NOGC_SLEEP_TIME 300000 /* wait 5 min */ ··· 28 wait_queue_head_t gc_wait_queue_head; 29 30 /* for gc sleep time */ 31 unsigned int min_sleep_time; 32 unsigned int max_sleep_time; 33 unsigned int no_gc_sleep_time; 34 35 /* for changing gc mode */ 36 unsigned int gc_idle; 37 }; 38 39 struct gc_inode_list { ··· 69 } 70 71 static inline void increase_sleep_time(struct f2fs_gc_kthread *gc_th, 72 - long *wait) 73 { 74 if (*wait == gc_th->no_gc_sleep_time) 75 return; 76 77 - *wait += gc_th->min_sleep_time; 78 - if (*wait > gc_th->max_sleep_time) 79 - *wait = gc_th->max_sleep_time; 80 } 81 82 static inline void decrease_sleep_time(struct f2fs_gc_kthread *gc_th, 83 - long *wait) 84 { 85 if (*wait == gc_th->no_gc_sleep_time) 86 *wait = gc_th->max_sleep_time; 87 88 - *wait -= gc_th->min_sleep_time; 89 - if (*wait <= gc_th->min_sleep_time) 90 - *wait = gc_th->min_sleep_time; 91 } 92 93 static inline bool has_enough_invalid_blocks(struct f2fs_sb_info *sbi)
··· 13 * whether IO subsystem is idle 14 * or not 15 */ 16 + #define DEF_GC_THREAD_URGENT_SLEEP_TIME 500 /* 500 ms */ 17 #define DEF_GC_THREAD_MIN_SLEEP_TIME 30000 /* milliseconds */ 18 #define DEF_GC_THREAD_MAX_SLEEP_TIME 60000 19 #define DEF_GC_THREAD_NOGC_SLEEP_TIME 300000 /* wait 5 min */ ··· 27 wait_queue_head_t gc_wait_queue_head; 28 29 /* for gc sleep time */ 30 + unsigned int urgent_sleep_time; 31 unsigned int min_sleep_time; 32 unsigned int max_sleep_time; 33 unsigned int no_gc_sleep_time; 34 35 /* for changing gc mode */ 36 unsigned int gc_idle; 37 + unsigned int gc_urgent; 38 + unsigned int gc_wake; 39 }; 40 41 struct gc_inode_list { ··· 65 } 66 67 static inline void increase_sleep_time(struct f2fs_gc_kthread *gc_th, 68 + unsigned int *wait) 69 { 70 + unsigned int min_time = gc_th->min_sleep_time; 71 + unsigned int max_time = gc_th->max_sleep_time; 72 + 73 if (*wait == gc_th->no_gc_sleep_time) 74 return; 75 76 + if ((long long)*wait + (long long)min_time > (long long)max_time) 77 + *wait = max_time; 78 + else 79 + *wait += min_time; 80 } 81 82 static inline void decrease_sleep_time(struct f2fs_gc_kthread *gc_th, 83 + unsigned int *wait) 84 { 85 + unsigned int min_time = gc_th->min_sleep_time; 86 + 87 if (*wait == gc_th->no_gc_sleep_time) 88 *wait = gc_th->max_sleep_time; 89 90 + if ((long long)*wait - (long long)min_time < (long long)min_time) 91 + *wait = min_time; 92 + else 93 + *wait -= min_time; 94 } 95 96 static inline bool has_enough_invalid_blocks(struct f2fs_sb_info *sbi)
+78 -64
fs/f2fs/inline.c
··· 22 if (!S_ISREG(inode->i_mode) && !S_ISLNK(inode->i_mode)) 23 return false; 24 25 - if (i_size_read(inode) > MAX_INLINE_DATA) 26 return false; 27 28 - if (f2fs_encrypted_inode(inode) && S_ISREG(inode->i_mode)) 29 return false; 30 31 return true; ··· 44 45 void read_inline_data(struct page *page, struct page *ipage) 46 { 47 void *src_addr, *dst_addr; 48 49 if (PageUptodate(page)) ··· 52 53 f2fs_bug_on(F2FS_P_SB(page), page->index); 54 55 - zero_user_segment(page, MAX_INLINE_DATA, PAGE_SIZE); 56 57 /* Copy the whole inline data block */ 58 - src_addr = inline_data_addr(ipage); 59 dst_addr = kmap_atomic(page); 60 - memcpy(dst_addr, src_addr, MAX_INLINE_DATA); 61 flush_dcache_page(page); 62 kunmap_atomic(dst_addr); 63 if (!PageUptodate(page)) ··· 68 { 69 void *addr; 70 71 - if (from >= MAX_INLINE_DATA) 72 return; 73 74 - addr = inline_data_addr(ipage); 75 76 f2fs_wait_on_page_writeback(ipage, NODE, true); 77 - memset(addr + from, 0, MAX_INLINE_DATA - from); 78 set_page_dirty(ipage); 79 80 if (from == 0) ··· 117 .op_flags = REQ_SYNC | REQ_PRIO, 118 .page = page, 119 .encrypted_page = NULL, 120 }; 121 int dirty, err; 122 ··· 202 { 203 void *src_addr, *dst_addr; 204 struct dnode_of_data dn; 205 int err; 206 207 set_new_dnode(&dn, inode, NULL, NULL, 0); ··· 220 221 f2fs_wait_on_page_writeback(dn.inode_page, NODE, true); 222 src_addr = kmap_atomic(page); 223 - dst_addr = inline_data_addr(dn.inode_page); 224 - memcpy(dst_addr, src_addr, MAX_INLINE_DATA); 225 kunmap_atomic(src_addr); 226 set_page_dirty(dn.inode_page); 227 228 set_inode_flag(inode, FI_APPEND_WRITE); 229 set_inode_flag(inode, FI_DATA_EXIST); ··· 264 265 f2fs_wait_on_page_writeback(ipage, NODE, true); 266 267 - src_addr = inline_data_addr(npage); 268 - dst_addr = inline_data_addr(ipage); 269 - memcpy(dst_addr, src_addr, MAX_INLINE_DATA); 270 271 set_inode_flag(inode, FI_INLINE_DATA); 272 set_inode_flag(inode, FI_DATA_EXIST); ··· 294 struct fscrypt_name *fname, struct page **res_page) 295 { 296 struct f2fs_sb_info *sbi = F2FS_SB(dir->i_sb); 297 - struct f2fs_inline_dentry *inline_dentry; 298 struct qstr name = FSTR_TO_QSTR(&fname->disk_name); 299 struct f2fs_dir_entry *de; 300 struct f2fs_dentry_ptr d; 301 struct page *ipage; 302 f2fs_hash_t namehash; 303 304 ipage = get_node_page(sbi, dir->i_ino); ··· 309 310 namehash = f2fs_dentry_hash(&name, fname); 311 312 - inline_dentry = inline_data_addr(ipage); 313 314 - make_dentry_ptr_inline(NULL, &d, inline_dentry); 315 de = find_target_dentry(fname, namehash, NULL, &d); 316 unlock_page(ipage); 317 if (de) ··· 325 int make_empty_inline_dir(struct inode *inode, struct inode *parent, 326 struct page *ipage) 327 { 328 - struct f2fs_inline_dentry *inline_dentry; 329 struct f2fs_dentry_ptr d; 330 331 - inline_dentry = inline_data_addr(ipage); 332 333 - make_dentry_ptr_inline(NULL, &d, inline_dentry); 334 do_make_empty_dir(inode, parent, &d); 335 336 set_page_dirty(ipage); 337 338 /* update i_size to MAX_INLINE_DATA */ 339 - if (i_size_read(inode) < MAX_INLINE_DATA) 340 - f2fs_i_size_write(inode, MAX_INLINE_DATA); 341 return 0; 342 } 343 ··· 346 * release ipage in this function. 347 */ 348 static int f2fs_move_inline_dirents(struct inode *dir, struct page *ipage, 349 - struct f2fs_inline_dentry *inline_dentry) 350 { 351 struct page *page; 352 struct dnode_of_data dn; 353 struct f2fs_dentry_block *dentry_blk; 354 int err; 355 356 page = f2fs_grab_cache_page(dir->i_mapping, 0, false); ··· 366 goto out; 367 368 f2fs_wait_on_page_writeback(page, DATA, true); 369 - zero_user_segment(page, MAX_INLINE_DATA, PAGE_SIZE); 370 371 dentry_blk = kmap_atomic(page); 372 373 /* copy data from inline dentry block to new dentry block */ 374 - memcpy(dentry_blk->dentry_bitmap, inline_dentry->dentry_bitmap, 375 - INLINE_DENTRY_BITMAP_SIZE); 376 - memset(dentry_blk->dentry_bitmap + INLINE_DENTRY_BITMAP_SIZE, 0, 377 - SIZE_OF_DENTRY_BITMAP - INLINE_DENTRY_BITMAP_SIZE); 378 /* 379 * we do not need to zero out remainder part of dentry and filename 380 * field, since we have used bitmap for marking the usage status of 381 * them, besides, we can also ignore copying/zeroing reserved space 382 * of dentry block, because them haven't been used so far. 383 */ 384 - memcpy(dentry_blk->dentry, inline_dentry->dentry, 385 - sizeof(struct f2fs_dir_entry) * NR_INLINE_DENTRY); 386 - memcpy(dentry_blk->filename, inline_dentry->filename, 387 - NR_INLINE_DENTRY * F2FS_SLOT_LEN); 388 389 kunmap_atomic(dentry_blk); 390 if (!PageUptodate(page)) ··· 404 return err; 405 } 406 407 - static int f2fs_add_inline_entries(struct inode *dir, 408 - struct f2fs_inline_dentry *inline_dentry) 409 { 410 struct f2fs_dentry_ptr d; 411 unsigned long bit_pos = 0; 412 int err = 0; 413 414 - make_dentry_ptr_inline(NULL, &d, inline_dentry); 415 416 while (bit_pos < d.max) { 417 struct f2fs_dir_entry *de; ··· 452 } 453 454 static int f2fs_move_rehashed_dirents(struct inode *dir, struct page *ipage, 455 - struct f2fs_inline_dentry *inline_dentry) 456 { 457 - struct f2fs_inline_dentry *backup_dentry; 458 int err; 459 460 backup_dentry = f2fs_kmalloc(F2FS_I_SB(dir), 461 - sizeof(struct f2fs_inline_dentry), GFP_F2FS_ZERO); 462 if (!backup_dentry) { 463 f2fs_put_page(ipage, 1); 464 return -ENOMEM; 465 } 466 467 - memcpy(backup_dentry, inline_dentry, MAX_INLINE_DATA); 468 truncate_inline_inode(dir, ipage, 0); 469 470 unlock_page(ipage); ··· 481 return 0; 482 recover: 483 lock_page(ipage); 484 - memcpy(inline_dentry, backup_dentry, MAX_INLINE_DATA); 485 f2fs_i_depth_write(dir, 0); 486 - f2fs_i_size_write(dir, MAX_INLINE_DATA); 487 set_page_dirty(ipage); 488 f2fs_put_page(ipage, 1); 489 ··· 492 } 493 494 static int f2fs_convert_inline_dir(struct inode *dir, struct page *ipage, 495 - struct f2fs_inline_dentry *inline_dentry) 496 { 497 if (!F2FS_I(dir)->i_dir_level) 498 return f2fs_move_inline_dirents(dir, ipage, inline_dentry); ··· 508 struct page *ipage; 509 unsigned int bit_pos; 510 f2fs_hash_t name_hash; 511 - struct f2fs_inline_dentry *inline_dentry = NULL; 512 struct f2fs_dentry_ptr d; 513 int slots = GET_DENTRY_SLOTS(new_name->len); 514 struct page *page = NULL; ··· 518 if (IS_ERR(ipage)) 519 return PTR_ERR(ipage); 520 521 - inline_dentry = inline_data_addr(ipage); 522 - bit_pos = room_for_filename(&inline_dentry->dentry_bitmap, 523 - slots, NR_INLINE_DENTRY); 524 - if (bit_pos >= NR_INLINE_DENTRY) { 525 err = f2fs_convert_inline_dir(dir, ipage, inline_dentry); 526 if (err) 527 return err; ··· 543 f2fs_wait_on_page_writeback(ipage, NODE, true); 544 545 name_hash = f2fs_dentry_hash(new_name, NULL); 546 - make_dentry_ptr_inline(NULL, &d, inline_dentry); 547 f2fs_update_dentry(ino, mode, &d, new_name, name_hash, bit_pos); 548 549 set_page_dirty(ipage); ··· 565 void f2fs_delete_inline_entry(struct f2fs_dir_entry *dentry, struct page *page, 566 struct inode *dir, struct inode *inode) 567 { 568 - struct f2fs_inline_dentry *inline_dentry; 569 int slots = GET_DENTRY_SLOTS(le16_to_cpu(dentry->name_len)); 570 unsigned int bit_pos; 571 int i; ··· 574 lock_page(page); 575 f2fs_wait_on_page_writeback(page, NODE, true); 576 577 - inline_dentry = inline_data_addr(page); 578 - bit_pos = dentry - inline_dentry->dentry; 579 for (i = 0; i < slots; i++) 580 - __clear_bit_le(bit_pos + i, 581 - &inline_dentry->dentry_bitmap); 582 583 set_page_dirty(page); 584 f2fs_put_page(page, 1); ··· 596 struct f2fs_sb_info *sbi = F2FS_I_SB(dir); 597 struct page *ipage; 598 unsigned int bit_pos = 2; 599 - struct f2fs_inline_dentry *inline_dentry; 600 601 ipage = get_node_page(sbi, dir->i_ino); 602 if (IS_ERR(ipage)) 603 return false; 604 605 - inline_dentry = inline_data_addr(ipage); 606 - bit_pos = find_next_bit_le(&inline_dentry->dentry_bitmap, 607 - NR_INLINE_DENTRY, 608 - bit_pos); 609 610 f2fs_put_page(ipage, 1); 611 612 - if (bit_pos < NR_INLINE_DENTRY) 613 return false; 614 615 return true; ··· 620 struct fscrypt_str *fstr) 621 { 622 struct inode *inode = file_inode(file); 623 - struct f2fs_inline_dentry *inline_dentry = NULL; 624 struct page *ipage = NULL; 625 struct f2fs_dentry_ptr d; 626 int err; 627 628 - if (ctx->pos == NR_INLINE_DENTRY) 629 return 0; 630 631 ipage = get_node_page(F2FS_I_SB(inode), inode->i_ino); 632 if (IS_ERR(ipage)) 633 return PTR_ERR(ipage); 634 635 - inline_dentry = inline_data_addr(ipage); 636 637 make_dentry_ptr_inline(inode, &d, inline_dentry); 638 639 err = f2fs_fill_dentries(ctx, &d, 0, fstr); 640 if (!err) 641 - ctx->pos = NR_INLINE_DENTRY; 642 643 f2fs_put_page(ipage, 1); 644 return err < 0 ? err : 0; ··· 665 goto out; 666 } 667 668 - ilen = min_t(size_t, MAX_INLINE_DATA, i_size_read(inode)); 669 if (start >= ilen) 670 goto out; 671 if (start + len < ilen) ··· 674 675 get_node_info(F2FS_I_SB(inode), inode->i_ino, &ni); 676 byteaddr = (__u64)ni.blk_addr << inode->i_sb->s_blocksize_bits; 677 - byteaddr += (char *)inline_data_addr(ipage) - (char *)F2FS_INODE(ipage); 678 err = fiemap_fill_next_extent(fieinfo, start, byteaddr, ilen, flags); 679 out: 680 f2fs_put_page(ipage, 1);
··· 22 if (!S_ISREG(inode->i_mode) && !S_ISLNK(inode->i_mode)) 23 return false; 24 25 + if (i_size_read(inode) > MAX_INLINE_DATA(inode)) 26 return false; 27 28 + if (f2fs_encrypted_file(inode)) 29 return false; 30 31 return true; ··· 44 45 void read_inline_data(struct page *page, struct page *ipage) 46 { 47 + struct inode *inode = page->mapping->host; 48 void *src_addr, *dst_addr; 49 50 if (PageUptodate(page)) ··· 51 52 f2fs_bug_on(F2FS_P_SB(page), page->index); 53 54 + zero_user_segment(page, MAX_INLINE_DATA(inode), PAGE_SIZE); 55 56 /* Copy the whole inline data block */ 57 + src_addr = inline_data_addr(inode, ipage); 58 dst_addr = kmap_atomic(page); 59 + memcpy(dst_addr, src_addr, MAX_INLINE_DATA(inode)); 60 flush_dcache_page(page); 61 kunmap_atomic(dst_addr); 62 if (!PageUptodate(page)) ··· 67 { 68 void *addr; 69 70 + if (from >= MAX_INLINE_DATA(inode)) 71 return; 72 73 + addr = inline_data_addr(inode, ipage); 74 75 f2fs_wait_on_page_writeback(ipage, NODE, true); 76 + memset(addr + from, 0, MAX_INLINE_DATA(inode) - from); 77 set_page_dirty(ipage); 78 79 if (from == 0) ··· 116 .op_flags = REQ_SYNC | REQ_PRIO, 117 .page = page, 118 .encrypted_page = NULL, 119 + .io_type = FS_DATA_IO, 120 }; 121 int dirty, err; 122 ··· 200 { 201 void *src_addr, *dst_addr; 202 struct dnode_of_data dn; 203 + struct address_space *mapping = page_mapping(page); 204 + unsigned long flags; 205 int err; 206 207 set_new_dnode(&dn, inode, NULL, NULL, 0); ··· 216 217 f2fs_wait_on_page_writeback(dn.inode_page, NODE, true); 218 src_addr = kmap_atomic(page); 219 + dst_addr = inline_data_addr(inode, dn.inode_page); 220 + memcpy(dst_addr, src_addr, MAX_INLINE_DATA(inode)); 221 kunmap_atomic(src_addr); 222 set_page_dirty(dn.inode_page); 223 + 224 + spin_lock_irqsave(&mapping->tree_lock, flags); 225 + radix_tree_tag_clear(&mapping->page_tree, page_index(page), 226 + PAGECACHE_TAG_DIRTY); 227 + spin_unlock_irqrestore(&mapping->tree_lock, flags); 228 229 set_inode_flag(inode, FI_APPEND_WRITE); 230 set_inode_flag(inode, FI_DATA_EXIST); ··· 255 256 f2fs_wait_on_page_writeback(ipage, NODE, true); 257 258 + src_addr = inline_data_addr(inode, npage); 259 + dst_addr = inline_data_addr(inode, ipage); 260 + memcpy(dst_addr, src_addr, MAX_INLINE_DATA(inode)); 261 262 set_inode_flag(inode, FI_INLINE_DATA); 263 set_inode_flag(inode, FI_DATA_EXIST); ··· 285 struct fscrypt_name *fname, struct page **res_page) 286 { 287 struct f2fs_sb_info *sbi = F2FS_SB(dir->i_sb); 288 struct qstr name = FSTR_TO_QSTR(&fname->disk_name); 289 struct f2fs_dir_entry *de; 290 struct f2fs_dentry_ptr d; 291 struct page *ipage; 292 + void *inline_dentry; 293 f2fs_hash_t namehash; 294 295 ipage = get_node_page(sbi, dir->i_ino); ··· 300 301 namehash = f2fs_dentry_hash(&name, fname); 302 303 + inline_dentry = inline_data_addr(dir, ipage); 304 305 + make_dentry_ptr_inline(dir, &d, inline_dentry); 306 de = find_target_dentry(fname, namehash, NULL, &d); 307 unlock_page(ipage); 308 if (de) ··· 316 int make_empty_inline_dir(struct inode *inode, struct inode *parent, 317 struct page *ipage) 318 { 319 struct f2fs_dentry_ptr d; 320 + void *inline_dentry; 321 322 + inline_dentry = inline_data_addr(inode, ipage); 323 324 + make_dentry_ptr_inline(inode, &d, inline_dentry); 325 do_make_empty_dir(inode, parent, &d); 326 327 set_page_dirty(ipage); 328 329 /* update i_size to MAX_INLINE_DATA */ 330 + if (i_size_read(inode) < MAX_INLINE_DATA(inode)) 331 + f2fs_i_size_write(inode, MAX_INLINE_DATA(inode)); 332 return 0; 333 } 334 ··· 337 * release ipage in this function. 338 */ 339 static int f2fs_move_inline_dirents(struct inode *dir, struct page *ipage, 340 + void *inline_dentry) 341 { 342 struct page *page; 343 struct dnode_of_data dn; 344 struct f2fs_dentry_block *dentry_blk; 345 + struct f2fs_dentry_ptr src, dst; 346 int err; 347 348 page = f2fs_grab_cache_page(dir->i_mapping, 0, false); ··· 356 goto out; 357 358 f2fs_wait_on_page_writeback(page, DATA, true); 359 + zero_user_segment(page, MAX_INLINE_DATA(dir), PAGE_SIZE); 360 361 dentry_blk = kmap_atomic(page); 362 363 + make_dentry_ptr_inline(dir, &src, inline_dentry); 364 + make_dentry_ptr_block(dir, &dst, dentry_blk); 365 + 366 /* copy data from inline dentry block to new dentry block */ 367 + memcpy(dst.bitmap, src.bitmap, src.nr_bitmap); 368 + memset(dst.bitmap + src.nr_bitmap, 0, dst.nr_bitmap - src.nr_bitmap); 369 /* 370 * we do not need to zero out remainder part of dentry and filename 371 * field, since we have used bitmap for marking the usage status of 372 * them, besides, we can also ignore copying/zeroing reserved space 373 * of dentry block, because them haven't been used so far. 374 */ 375 + memcpy(dst.dentry, src.dentry, SIZE_OF_DIR_ENTRY * src.max); 376 + memcpy(dst.filename, src.filename, src.max * F2FS_SLOT_LEN); 377 378 kunmap_atomic(dentry_blk); 379 if (!PageUptodate(page)) ··· 395 return err; 396 } 397 398 + static int f2fs_add_inline_entries(struct inode *dir, void *inline_dentry) 399 { 400 struct f2fs_dentry_ptr d; 401 unsigned long bit_pos = 0; 402 int err = 0; 403 404 + make_dentry_ptr_inline(dir, &d, inline_dentry); 405 406 while (bit_pos < d.max) { 407 struct f2fs_dir_entry *de; ··· 444 } 445 446 static int f2fs_move_rehashed_dirents(struct inode *dir, struct page *ipage, 447 + void *inline_dentry) 448 { 449 + void *backup_dentry; 450 int err; 451 452 backup_dentry = f2fs_kmalloc(F2FS_I_SB(dir), 453 + MAX_INLINE_DATA(dir), GFP_F2FS_ZERO); 454 if (!backup_dentry) { 455 f2fs_put_page(ipage, 1); 456 return -ENOMEM; 457 } 458 459 + memcpy(backup_dentry, inline_dentry, MAX_INLINE_DATA(dir)); 460 truncate_inline_inode(dir, ipage, 0); 461 462 unlock_page(ipage); ··· 473 return 0; 474 recover: 475 lock_page(ipage); 476 + memcpy(inline_dentry, backup_dentry, MAX_INLINE_DATA(dir)); 477 f2fs_i_depth_write(dir, 0); 478 + f2fs_i_size_write(dir, MAX_INLINE_DATA(dir)); 479 set_page_dirty(ipage); 480 f2fs_put_page(ipage, 1); 481 ··· 484 } 485 486 static int f2fs_convert_inline_dir(struct inode *dir, struct page *ipage, 487 + void *inline_dentry) 488 { 489 if (!F2FS_I(dir)->i_dir_level) 490 return f2fs_move_inline_dirents(dir, ipage, inline_dentry); ··· 500 struct page *ipage; 501 unsigned int bit_pos; 502 f2fs_hash_t name_hash; 503 + void *inline_dentry = NULL; 504 struct f2fs_dentry_ptr d; 505 int slots = GET_DENTRY_SLOTS(new_name->len); 506 struct page *page = NULL; ··· 510 if (IS_ERR(ipage)) 511 return PTR_ERR(ipage); 512 513 + inline_dentry = inline_data_addr(dir, ipage); 514 + make_dentry_ptr_inline(dir, &d, inline_dentry); 515 + 516 + bit_pos = room_for_filename(d.bitmap, slots, d.max); 517 + if (bit_pos >= d.max) { 518 err = f2fs_convert_inline_dir(dir, ipage, inline_dentry); 519 if (err) 520 return err; ··· 534 f2fs_wait_on_page_writeback(ipage, NODE, true); 535 536 name_hash = f2fs_dentry_hash(new_name, NULL); 537 f2fs_update_dentry(ino, mode, &d, new_name, name_hash, bit_pos); 538 539 set_page_dirty(ipage); ··· 557 void f2fs_delete_inline_entry(struct f2fs_dir_entry *dentry, struct page *page, 558 struct inode *dir, struct inode *inode) 559 { 560 + struct f2fs_dentry_ptr d; 561 + void *inline_dentry; 562 int slots = GET_DENTRY_SLOTS(le16_to_cpu(dentry->name_len)); 563 unsigned int bit_pos; 564 int i; ··· 565 lock_page(page); 566 f2fs_wait_on_page_writeback(page, NODE, true); 567 568 + inline_dentry = inline_data_addr(dir, page); 569 + make_dentry_ptr_inline(dir, &d, inline_dentry); 570 + 571 + bit_pos = dentry - d.dentry; 572 for (i = 0; i < slots; i++) 573 + __clear_bit_le(bit_pos + i, d.bitmap); 574 575 set_page_dirty(page); 576 f2fs_put_page(page, 1); ··· 586 struct f2fs_sb_info *sbi = F2FS_I_SB(dir); 587 struct page *ipage; 588 unsigned int bit_pos = 2; 589 + void *inline_dentry; 590 + struct f2fs_dentry_ptr d; 591 592 ipage = get_node_page(sbi, dir->i_ino); 593 if (IS_ERR(ipage)) 594 return false; 595 596 + inline_dentry = inline_data_addr(dir, ipage); 597 + make_dentry_ptr_inline(dir, &d, inline_dentry); 598 + 599 + bit_pos = find_next_bit_le(d.bitmap, d.max, bit_pos); 600 601 f2fs_put_page(ipage, 1); 602 603 + if (bit_pos < d.max) 604 return false; 605 606 return true; ··· 609 struct fscrypt_str *fstr) 610 { 611 struct inode *inode = file_inode(file); 612 struct page *ipage = NULL; 613 struct f2fs_dentry_ptr d; 614 + void *inline_dentry = NULL; 615 int err; 616 617 + make_dentry_ptr_inline(inode, &d, inline_dentry); 618 + 619 + if (ctx->pos == d.max) 620 return 0; 621 622 ipage = get_node_page(F2FS_I_SB(inode), inode->i_ino); 623 if (IS_ERR(ipage)) 624 return PTR_ERR(ipage); 625 626 + inline_dentry = inline_data_addr(inode, ipage); 627 628 make_dentry_ptr_inline(inode, &d, inline_dentry); 629 630 err = f2fs_fill_dentries(ctx, &d, 0, fstr); 631 if (!err) 632 + ctx->pos = d.max; 633 634 f2fs_put_page(ipage, 1); 635 return err < 0 ? err : 0; ··· 652 goto out; 653 } 654 655 + ilen = min_t(size_t, MAX_INLINE_DATA(inode), i_size_read(inode)); 656 if (start >= ilen) 657 goto out; 658 if (start + len < ilen) ··· 661 662 get_node_info(F2FS_I_SB(inode), inode->i_ino, &ni); 663 byteaddr = (__u64)ni.blk_addr << inode->i_sb->s_blocksize_bits; 664 + byteaddr += (char *)inline_data_addr(inode, ipage) - 665 + (char *)F2FS_INODE(ipage); 666 err = fiemap_fill_next_extent(fieinfo, start, byteaddr, ilen, flags); 667 out: 668 f2fs_put_page(ipage, 1);
+119 -13
fs/f2fs/inode.c
··· 49 50 static void __get_inode_rdev(struct inode *inode, struct f2fs_inode *ri) 51 { 52 if (S_ISCHR(inode->i_mode) || S_ISBLK(inode->i_mode) || 53 S_ISFIFO(inode->i_mode) || S_ISSOCK(inode->i_mode)) { 54 - if (ri->i_addr[0]) 55 - inode->i_rdev = 56 - old_decode_dev(le32_to_cpu(ri->i_addr[0])); 57 else 58 - inode->i_rdev = 59 - new_decode_dev(le32_to_cpu(ri->i_addr[1])); 60 } 61 } 62 63 static bool __written_first_block(struct f2fs_inode *ri) 64 { 65 - block_t addr = le32_to_cpu(ri->i_addr[0]); 66 67 if (addr != NEW_ADDR && addr != NULL_ADDR) 68 return true; ··· 73 74 static void __set_inode_rdev(struct inode *inode, struct f2fs_inode *ri) 75 { 76 if (S_ISCHR(inode->i_mode) || S_ISBLK(inode->i_mode)) { 77 if (old_valid_dev(inode->i_rdev)) { 78 - ri->i_addr[0] = 79 cpu_to_le32(old_encode_dev(inode->i_rdev)); 80 - ri->i_addr[1] = 0; 81 } else { 82 - ri->i_addr[0] = 0; 83 - ri->i_addr[1] = 84 cpu_to_le32(new_encode_dev(inode->i_rdev)); 85 - ri->i_addr[2] = 0; 86 } 87 } 88 } 89 90 static void __recover_inline_status(struct inode *inode, struct page *ipage) 91 { 92 - void *inline_data = inline_data_addr(ipage); 93 __le32 *start = inline_data; 94 - __le32 *end = start + MAX_INLINE_DATA / sizeof(__le32); 95 96 while (start < end) { 97 if (*start++) { ··· 108 return; 109 } 110 111 static int do_read_inode(struct inode *inode) 112 { 113 struct f2fs_sb_info *sbi = F2FS_I_SB(inode); 114 struct f2fs_inode_info *fi = F2FS_I(inode); 115 struct page *node_page; 116 struct f2fs_inode *ri; 117 118 /* Check if ino is within scope */ 119 if (check_nid_range(sbi, inode->i_ino)) { ··· 229 230 get_inline_info(inode, ri); 231 232 /* check data exist */ 233 if (f2fs_has_inline_data(inode) && !f2fs_exist_data(inode)) 234 __recover_inline_status(inode, node_page); ··· 244 245 if (!need_inode_block_update(sbi, inode->i_ino)) 246 fi->last_disk_size = inode->i_size; 247 248 f2fs_put_page(node_page, 1); 249 ··· 381 ri->i_generation = cpu_to_le32(inode->i_generation); 382 ri->i_dir_level = F2FS_I(inode)->i_dir_level; 383 384 __set_inode_rdev(inode, ri); 385 set_cold_node(inode, node_page); 386 ··· 518 stat_dec_inline_xattr(inode); 519 stat_dec_inline_dir(inode); 520 stat_dec_inline_inode(inode); 521 522 /* ino == 0, if f2fs_new_inode() was failed t*/ 523 if (inode->i_ino)
··· 49 50 static void __get_inode_rdev(struct inode *inode, struct f2fs_inode *ri) 51 { 52 + int extra_size = get_extra_isize(inode); 53 + 54 if (S_ISCHR(inode->i_mode) || S_ISBLK(inode->i_mode) || 55 S_ISFIFO(inode->i_mode) || S_ISSOCK(inode->i_mode)) { 56 + if (ri->i_addr[extra_size]) 57 + inode->i_rdev = old_decode_dev( 58 + le32_to_cpu(ri->i_addr[extra_size])); 59 else 60 + inode->i_rdev = new_decode_dev( 61 + le32_to_cpu(ri->i_addr[extra_size + 1])); 62 } 63 } 64 65 static bool __written_first_block(struct f2fs_inode *ri) 66 { 67 + block_t addr = le32_to_cpu(ri->i_addr[offset_in_addr(ri)]); 68 69 if (addr != NEW_ADDR && addr != NULL_ADDR) 70 return true; ··· 71 72 static void __set_inode_rdev(struct inode *inode, struct f2fs_inode *ri) 73 { 74 + int extra_size = get_extra_isize(inode); 75 + 76 if (S_ISCHR(inode->i_mode) || S_ISBLK(inode->i_mode)) { 77 if (old_valid_dev(inode->i_rdev)) { 78 + ri->i_addr[extra_size] = 79 cpu_to_le32(old_encode_dev(inode->i_rdev)); 80 + ri->i_addr[extra_size + 1] = 0; 81 } else { 82 + ri->i_addr[extra_size] = 0; 83 + ri->i_addr[extra_size + 1] = 84 cpu_to_le32(new_encode_dev(inode->i_rdev)); 85 + ri->i_addr[extra_size + 2] = 0; 86 } 87 } 88 } 89 90 static void __recover_inline_status(struct inode *inode, struct page *ipage) 91 { 92 + void *inline_data = inline_data_addr(inode, ipage); 93 __le32 *start = inline_data; 94 + __le32 *end = start + MAX_INLINE_DATA(inode) / sizeof(__le32); 95 96 while (start < end) { 97 if (*start++) { ··· 104 return; 105 } 106 107 + static bool f2fs_enable_inode_chksum(struct f2fs_sb_info *sbi, struct page *page) 108 + { 109 + struct f2fs_inode *ri = &F2FS_NODE(page)->i; 110 + int extra_isize = le32_to_cpu(ri->i_extra_isize); 111 + 112 + if (!f2fs_sb_has_inode_chksum(sbi->sb)) 113 + return false; 114 + 115 + if (!RAW_IS_INODE(F2FS_NODE(page)) || !(ri->i_inline & F2FS_EXTRA_ATTR)) 116 + return false; 117 + 118 + if (!F2FS_FITS_IN_INODE(ri, extra_isize, i_inode_checksum)) 119 + return false; 120 + 121 + return true; 122 + } 123 + 124 + static __u32 f2fs_inode_chksum(struct f2fs_sb_info *sbi, struct page *page) 125 + { 126 + struct f2fs_node *node = F2FS_NODE(page); 127 + struct f2fs_inode *ri = &node->i; 128 + __le32 ino = node->footer.ino; 129 + __le32 gen = ri->i_generation; 130 + __u32 chksum, chksum_seed; 131 + __u32 dummy_cs = 0; 132 + unsigned int offset = offsetof(struct f2fs_inode, i_inode_checksum); 133 + unsigned int cs_size = sizeof(dummy_cs); 134 + 135 + chksum = f2fs_chksum(sbi, sbi->s_chksum_seed, (__u8 *)&ino, 136 + sizeof(ino)); 137 + chksum_seed = f2fs_chksum(sbi, chksum, (__u8 *)&gen, sizeof(gen)); 138 + 139 + chksum = f2fs_chksum(sbi, chksum_seed, (__u8 *)ri, offset); 140 + chksum = f2fs_chksum(sbi, chksum, (__u8 *)&dummy_cs, cs_size); 141 + offset += cs_size; 142 + chksum = f2fs_chksum(sbi, chksum, (__u8 *)ri + offset, 143 + F2FS_BLKSIZE - offset); 144 + return chksum; 145 + } 146 + 147 + bool f2fs_inode_chksum_verify(struct f2fs_sb_info *sbi, struct page *page) 148 + { 149 + struct f2fs_inode *ri; 150 + __u32 provided, calculated; 151 + 152 + if (!f2fs_enable_inode_chksum(sbi, page) || 153 + PageDirty(page) || PageWriteback(page)) 154 + return true; 155 + 156 + ri = &F2FS_NODE(page)->i; 157 + provided = le32_to_cpu(ri->i_inode_checksum); 158 + calculated = f2fs_inode_chksum(sbi, page); 159 + 160 + if (provided != calculated) 161 + f2fs_msg(sbi->sb, KERN_WARNING, 162 + "checksum invalid, ino = %x, %x vs. %x", 163 + ino_of_node(page), provided, calculated); 164 + 165 + return provided == calculated; 166 + } 167 + 168 + void f2fs_inode_chksum_set(struct f2fs_sb_info *sbi, struct page *page) 169 + { 170 + struct f2fs_inode *ri = &F2FS_NODE(page)->i; 171 + 172 + if (!f2fs_enable_inode_chksum(sbi, page)) 173 + return; 174 + 175 + ri->i_inode_checksum = cpu_to_le32(f2fs_inode_chksum(sbi, page)); 176 + } 177 + 178 static int do_read_inode(struct inode *inode) 179 { 180 struct f2fs_sb_info *sbi = F2FS_I_SB(inode); 181 struct f2fs_inode_info *fi = F2FS_I(inode); 182 struct page *node_page; 183 struct f2fs_inode *ri; 184 + projid_t i_projid; 185 186 /* Check if ino is within scope */ 187 if (check_nid_range(sbi, inode->i_ino)) { ··· 153 154 get_inline_info(inode, ri); 155 156 + fi->i_extra_isize = f2fs_has_extra_attr(inode) ? 157 + le16_to_cpu(ri->i_extra_isize) : 0; 158 + 159 /* check data exist */ 160 if (f2fs_has_inline_data(inode) && !f2fs_exist_data(inode)) 161 __recover_inline_status(inode, node_page); ··· 165 166 if (!need_inode_block_update(sbi, inode->i_ino)) 167 fi->last_disk_size = inode->i_size; 168 + 169 + if (fi->i_flags & FS_PROJINHERIT_FL) 170 + set_inode_flag(inode, FI_PROJ_INHERIT); 171 + 172 + if (f2fs_has_extra_attr(inode) && f2fs_sb_has_project_quota(sbi->sb) && 173 + F2FS_FITS_IN_INODE(ri, fi->i_extra_isize, i_projid)) 174 + i_projid = (projid_t)le32_to_cpu(ri->i_projid); 175 + else 176 + i_projid = F2FS_DEF_PROJID; 177 + fi->i_projid = make_kprojid(&init_user_ns, i_projid); 178 179 f2fs_put_page(node_page, 1); 180 ··· 292 ri->i_generation = cpu_to_le32(inode->i_generation); 293 ri->i_dir_level = F2FS_I(inode)->i_dir_level; 294 295 + if (f2fs_has_extra_attr(inode)) { 296 + ri->i_extra_isize = cpu_to_le16(F2FS_I(inode)->i_extra_isize); 297 + 298 + if (f2fs_sb_has_project_quota(F2FS_I_SB(inode)->sb) && 299 + F2FS_FITS_IN_INODE(ri, F2FS_I(inode)->i_extra_isize, 300 + i_projid)) { 301 + projid_t i_projid; 302 + 303 + i_projid = from_kprojid(&init_user_ns, 304 + F2FS_I(inode)->i_projid); 305 + ri->i_projid = cpu_to_le32(i_projid); 306 + } 307 + } 308 + 309 __set_inode_rdev(inode, ri); 310 set_cold_node(inode, node_page); 311 ··· 415 stat_dec_inline_xattr(inode); 416 stat_dec_inline_dir(inode); 417 stat_dec_inline_inode(inode); 418 + 419 + if (!is_set_ckpt_flags(sbi, CP_ERROR_FLAG)) 420 + f2fs_bug_on(sbi, is_inode_flag_set(inode, FI_DIRTY_INODE)); 421 422 /* ino == 0, if f2fs_new_inode() was failed t*/ 423 if (inode->i_ino)
+43
fs/f2fs/namei.c
··· 58 goto fail; 59 } 60 61 err = dquot_initialize(inode); 62 if (err) 63 goto fail_drop; ··· 79 80 set_inode_flag(inode, FI_NEW_INODE); 81 82 if (test_opt(sbi, INLINE_XATTR)) 83 set_inode_flag(inode, FI_INLINE_XATTR); 84 if (test_opt(sbi, INLINE_DATA) && f2fs_may_inline_data(inode)) ··· 96 stat_inc_inline_xattr(inode); 97 stat_inc_inline_inode(inode); 98 stat_inc_inline_dir(inode); 99 100 trace_f2fs_new_inode(inode, 0); 101 return inode; ··· 225 !fscrypt_has_permitted_context(dir, inode)) 226 return -EPERM; 227 228 err = dquot_initialize(dir); 229 if (err) 230 return err; ··· 286 "in readonly mountpoint", dir->i_ino, pino); 287 return 0; 288 } 289 290 f2fs_balance_fs(sbi, true); 291 ··· 754 goto out; 755 } 756 757 err = dquot_initialize(old_dir); 758 if (err) 759 goto out; ··· 946 (!fscrypt_has_permitted_context(new_dir, old_inode) || 947 !fscrypt_has_permitted_context(old_dir, new_inode))) 948 return -EPERM; 949 950 err = dquot_initialize(old_dir); 951 if (err)
··· 58 goto fail; 59 } 60 61 + if (f2fs_sb_has_project_quota(sbi->sb) && 62 + (F2FS_I(dir)->i_flags & FS_PROJINHERIT_FL)) 63 + F2FS_I(inode)->i_projid = F2FS_I(dir)->i_projid; 64 + else 65 + F2FS_I(inode)->i_projid = make_kprojid(&init_user_ns, 66 + F2FS_DEF_PROJID); 67 + 68 err = dquot_initialize(inode); 69 if (err) 70 goto fail_drop; ··· 72 73 set_inode_flag(inode, FI_NEW_INODE); 74 75 + if (f2fs_sb_has_extra_attr(sbi->sb)) { 76 + set_inode_flag(inode, FI_EXTRA_ATTR); 77 + F2FS_I(inode)->i_extra_isize = F2FS_TOTAL_EXTRA_ATTR_SIZE; 78 + } 79 + 80 if (test_opt(sbi, INLINE_XATTR)) 81 set_inode_flag(inode, FI_INLINE_XATTR); 82 if (test_opt(sbi, INLINE_DATA) && f2fs_may_inline_data(inode)) ··· 84 stat_inc_inline_xattr(inode); 85 stat_inc_inline_inode(inode); 86 stat_inc_inline_dir(inode); 87 + 88 + F2FS_I(inode)->i_flags = 89 + f2fs_mask_flags(mode, F2FS_I(dir)->i_flags & F2FS_FL_INHERITED); 90 + 91 + if (S_ISDIR(inode->i_mode)) 92 + F2FS_I(inode)->i_flags |= FS_INDEX_FL; 93 + 94 + if (F2FS_I(inode)->i_flags & FS_PROJINHERIT_FL) 95 + set_inode_flag(inode, FI_PROJ_INHERIT); 96 97 trace_f2fs_new_inode(inode, 0); 98 return inode; ··· 204 !fscrypt_has_permitted_context(dir, inode)) 205 return -EPERM; 206 207 + if (is_inode_flag_set(dir, FI_PROJ_INHERIT) && 208 + (!projid_eq(F2FS_I(dir)->i_projid, 209 + F2FS_I(old_dentry->d_inode)->i_projid))) 210 + return -EXDEV; 211 + 212 err = dquot_initialize(dir); 213 if (err) 214 return err; ··· 260 "in readonly mountpoint", dir->i_ino, pino); 261 return 0; 262 } 263 + 264 + err = dquot_initialize(dir); 265 + if (err) 266 + return err; 267 268 f2fs_balance_fs(sbi, true); 269 ··· 724 goto out; 725 } 726 727 + if (is_inode_flag_set(new_dir, FI_PROJ_INHERIT) && 728 + (!projid_eq(F2FS_I(new_dir)->i_projid, 729 + F2FS_I(old_dentry->d_inode)->i_projid))) 730 + return -EXDEV; 731 + 732 err = dquot_initialize(old_dir); 733 if (err) 734 goto out; ··· 911 (!fscrypt_has_permitted_context(new_dir, old_inode) || 912 !fscrypt_has_permitted_context(old_dir, new_inode))) 913 return -EPERM; 914 + 915 + if ((is_inode_flag_set(new_dir, FI_PROJ_INHERIT) && 916 + !projid_eq(F2FS_I(new_dir)->i_projid, 917 + F2FS_I(old_dentry->d_inode)->i_projid)) || 918 + (is_inode_flag_set(new_dir, FI_PROJ_INHERIT) && 919 + !projid_eq(F2FS_I(old_dir)->i_projid, 920 + F2FS_I(new_dentry->d_inode)->i_projid))) 921 + return -EXDEV; 922 923 err = dquot_initialize(old_dir); 924 if (err)
+52 -27
fs/f2fs/node.c
··· 19 #include "f2fs.h" 20 #include "node.h" 21 #include "segment.h" 22 #include "trace.h" 23 #include <trace/events/f2fs.h> 24 ··· 555 level = 3; 556 goto got; 557 } else { 558 - BUG(); 559 } 560 got: 561 return level; ··· 579 int err = 0; 580 581 level = get_node_path(dn->inode, index, offset, noffset); 582 583 nids[0] = dn->inode->i_ino; 584 npage[0] = dn->inode_page; ··· 616 } 617 618 dn->nid = nids[i]; 619 - npage[i] = new_node_page(dn, noffset[i], NULL); 620 if (IS_ERR(npage[i])) { 621 alloc_nid_failed(sbi, nids[i]); 622 err = PTR_ERR(npage[i]); ··· 657 dn->nid = nids[level]; 658 dn->ofs_in_node = offset[level]; 659 dn->node_page = npage[level]; 660 - dn->data_blkaddr = datablock_addr(dn->node_page, dn->ofs_in_node); 661 return 0; 662 663 release_pages: ··· 880 trace_f2fs_truncate_inode_blocks_enter(inode, from); 881 882 level = get_node_path(inode, from, offset, noffset); 883 884 page = get_node_page(sbi, inode->i_ino); 885 if (IS_ERR(page)) { ··· 1028 set_new_dnode(&dn, inode, NULL, NULL, inode->i_ino); 1029 1030 /* caller should f2fs_put_page(page, 1); */ 1031 - return new_node_page(&dn, 0, NULL); 1032 } 1033 1034 - struct page *new_node_page(struct dnode_of_data *dn, 1035 - unsigned int ofs, struct page *ipage) 1036 { 1037 struct f2fs_sb_info *sbi = F2FS_I_SB(dn->inode); 1038 struct node_info new_ni; ··· 1175 err = -EIO; 1176 goto out_err; 1177 } 1178 page_hit: 1179 if(unlikely(nid != nid_of_node(page))) { 1180 f2fs_msg(sbi->sb, KERN_WARNING, "inconsistent node block, " ··· 1187 nid, nid_of_node(page), ino_of_node(page), 1188 ofs_of_node(page), cpver_of_node(page), 1189 next_blkaddr_of_node(page)); 1190 - ClearPageUptodate(page); 1191 err = -EINVAL; 1192 out_err: 1193 f2fs_put_page(page, 1); 1194 return ERR_PTR(err); 1195 } ··· 1336 } 1337 1338 static int __write_node_page(struct page *page, bool atomic, bool *submitted, 1339 - struct writeback_control *wbc) 1340 { 1341 struct f2fs_sb_info *sbi = F2FS_P_SB(page); 1342 nid_t nid; ··· 1350 .page = page, 1351 .encrypted_page = NULL, 1352 .submitted = false, 1353 }; 1354 1355 trace_f2fs_writepage(page, NODE); ··· 1407 if (submitted) 1408 *submitted = fio.submitted; 1409 1410 return 0; 1411 1412 redirty_out: ··· 1419 static int f2fs_write_node_page(struct page *page, 1420 struct writeback_control *wbc) 1421 { 1422 - return __write_node_page(page, false, NULL, wbc); 1423 } 1424 1425 int fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode, ··· 1507 1508 ret = __write_node_page(page, atomic && 1509 page == last_page, 1510 - &submitted, wbc); 1511 if (ret) { 1512 unlock_page(page); 1513 f2fs_put_page(last_page, 0); ··· 1545 return ret ? -EIO: 0; 1546 } 1547 1548 - int sync_node_pages(struct f2fs_sb_info *sbi, struct writeback_control *wbc) 1549 { 1550 pgoff_t index, end; 1551 struct pagevec pvec; ··· 1624 set_fsync_mark(page, 0); 1625 set_dentry_mark(page, 0); 1626 1627 - ret = __write_node_page(page, false, &submitted, wbc); 1628 if (ret) 1629 unlock_page(page); 1630 else if (submitted) ··· 1714 diff = nr_pages_to_write(sbi, NODE, wbc); 1715 wbc->sync_mode = WB_SYNC_NONE; 1716 blk_start_plug(&plug); 1717 - sync_node_pages(sbi, wbc); 1718 blk_finish_plug(&plug); 1719 wbc->nr_to_write = max((long)0, wbc->nr_to_write - diff); 1720 return 0; ··· 2208 { 2209 struct f2fs_sb_info *sbi = F2FS_I_SB(inode); 2210 nid_t prev_xnid = F2FS_I(inode)->i_xattr_nid; 2211 - nid_t new_xnid = nid_of_node(page); 2212 struct node_info ni; 2213 struct page *xpage; 2214 ··· 2225 2226 recover_xnid: 2227 /* 2: update xattr nid in inode */ 2228 - remove_free_nid(sbi, new_xnid); 2229 - f2fs_i_xnid_write(inode, new_xnid); 2230 - if (unlikely(inc_valid_node_count(sbi, inode, false))) 2231 - f2fs_bug_on(sbi, 1); 2232 update_inode_page(inode); 2233 2234 /* 3: update and set xattr node page dirty */ 2235 - xpage = grab_cache_page(NODE_MAPPING(sbi), new_xnid); 2236 - if (!xpage) 2237 - return -ENOMEM; 2238 2239 - memcpy(F2FS_NODE(xpage), F2FS_NODE(page), PAGE_SIZE); 2240 - 2241 - get_node_info(sbi, new_xnid, &ni); 2242 - ni.ino = inode->i_ino; 2243 - set_node_addr(sbi, &ni, NEW_ADDR, false); 2244 set_page_dirty(xpage); 2245 f2fs_put_page(xpage, 1); 2246 ··· 2280 dst->i_blocks = cpu_to_le64(1); 2281 dst->i_links = cpu_to_le32(1); 2282 dst->i_xattr_nid = 0; 2283 - dst->i_inline = src->i_inline & F2FS_INLINE_XATTR; 2284 2285 new_ni = old_ni; 2286 new_ni.ino = ino;
··· 19 #include "f2fs.h" 20 #include "node.h" 21 #include "segment.h" 22 + #include "xattr.h" 23 #include "trace.h" 24 #include <trace/events/f2fs.h> 25 ··· 554 level = 3; 555 goto got; 556 } else { 557 + return -E2BIG; 558 } 559 got: 560 return level; ··· 578 int err = 0; 579 580 level = get_node_path(dn->inode, index, offset, noffset); 581 + if (level < 0) 582 + return level; 583 584 nids[0] = dn->inode->i_ino; 585 npage[0] = dn->inode_page; ··· 613 } 614 615 dn->nid = nids[i]; 616 + npage[i] = new_node_page(dn, noffset[i]); 617 if (IS_ERR(npage[i])) { 618 alloc_nid_failed(sbi, nids[i]); 619 err = PTR_ERR(npage[i]); ··· 654 dn->nid = nids[level]; 655 dn->ofs_in_node = offset[level]; 656 dn->node_page = npage[level]; 657 + dn->data_blkaddr = datablock_addr(dn->inode, 658 + dn->node_page, dn->ofs_in_node); 659 return 0; 660 661 release_pages: ··· 876 trace_f2fs_truncate_inode_blocks_enter(inode, from); 877 878 level = get_node_path(inode, from, offset, noffset); 879 + if (level < 0) 880 + return level; 881 882 page = get_node_page(sbi, inode->i_ino); 883 if (IS_ERR(page)) { ··· 1022 set_new_dnode(&dn, inode, NULL, NULL, inode->i_ino); 1023 1024 /* caller should f2fs_put_page(page, 1); */ 1025 + return new_node_page(&dn, 0); 1026 } 1027 1028 + struct page *new_node_page(struct dnode_of_data *dn, unsigned int ofs) 1029 { 1030 struct f2fs_sb_info *sbi = F2FS_I_SB(dn->inode); 1031 struct node_info new_ni; ··· 1170 err = -EIO; 1171 goto out_err; 1172 } 1173 + 1174 + if (!f2fs_inode_chksum_verify(sbi, page)) { 1175 + err = -EBADMSG; 1176 + goto out_err; 1177 + } 1178 page_hit: 1179 if(unlikely(nid != nid_of_node(page))) { 1180 f2fs_msg(sbi->sb, KERN_WARNING, "inconsistent node block, " ··· 1177 nid, nid_of_node(page), ino_of_node(page), 1178 ofs_of_node(page), cpver_of_node(page), 1179 next_blkaddr_of_node(page)); 1180 err = -EINVAL; 1181 out_err: 1182 + ClearPageUptodate(page); 1183 f2fs_put_page(page, 1); 1184 return ERR_PTR(err); 1185 } ··· 1326 } 1327 1328 static int __write_node_page(struct page *page, bool atomic, bool *submitted, 1329 + struct writeback_control *wbc, bool do_balance, 1330 + enum iostat_type io_type) 1331 { 1332 struct f2fs_sb_info *sbi = F2FS_P_SB(page); 1333 nid_t nid; ··· 1339 .page = page, 1340 .encrypted_page = NULL, 1341 .submitted = false, 1342 + .io_type = io_type, 1343 }; 1344 1345 trace_f2fs_writepage(page, NODE); ··· 1395 if (submitted) 1396 *submitted = fio.submitted; 1397 1398 + if (do_balance) 1399 + f2fs_balance_fs(sbi, false); 1400 return 0; 1401 1402 redirty_out: ··· 1405 static int f2fs_write_node_page(struct page *page, 1406 struct writeback_control *wbc) 1407 { 1408 + return __write_node_page(page, false, NULL, wbc, false, FS_NODE_IO); 1409 } 1410 1411 int fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode, ··· 1493 1494 ret = __write_node_page(page, atomic && 1495 page == last_page, 1496 + &submitted, wbc, true, 1497 + FS_NODE_IO); 1498 if (ret) { 1499 unlock_page(page); 1500 f2fs_put_page(last_page, 0); ··· 1530 return ret ? -EIO: 0; 1531 } 1532 1533 + int sync_node_pages(struct f2fs_sb_info *sbi, struct writeback_control *wbc, 1534 + bool do_balance, enum iostat_type io_type) 1535 { 1536 pgoff_t index, end; 1537 struct pagevec pvec; ··· 1608 set_fsync_mark(page, 0); 1609 set_dentry_mark(page, 0); 1610 1611 + ret = __write_node_page(page, false, &submitted, 1612 + wbc, do_balance, io_type); 1613 if (ret) 1614 unlock_page(page); 1615 else if (submitted) ··· 1697 diff = nr_pages_to_write(sbi, NODE, wbc); 1698 wbc->sync_mode = WB_SYNC_NONE; 1699 blk_start_plug(&plug); 1700 + sync_node_pages(sbi, wbc, true, FS_NODE_IO); 1701 blk_finish_plug(&plug); 1702 wbc->nr_to_write = max((long)0, wbc->nr_to_write - diff); 1703 return 0; ··· 2191 { 2192 struct f2fs_sb_info *sbi = F2FS_I_SB(inode); 2193 nid_t prev_xnid = F2FS_I(inode)->i_xattr_nid; 2194 + nid_t new_xnid; 2195 + struct dnode_of_data dn; 2196 struct node_info ni; 2197 struct page *xpage; 2198 ··· 2207 2208 recover_xnid: 2209 /* 2: update xattr nid in inode */ 2210 + if (!alloc_nid(sbi, &new_xnid)) 2211 + return -ENOSPC; 2212 + 2213 + set_new_dnode(&dn, inode, NULL, NULL, new_xnid); 2214 + xpage = new_node_page(&dn, XATTR_NODE_OFFSET); 2215 + if (IS_ERR(xpage)) { 2216 + alloc_nid_failed(sbi, new_xnid); 2217 + return PTR_ERR(xpage); 2218 + } 2219 + 2220 + alloc_nid_done(sbi, new_xnid); 2221 update_inode_page(inode); 2222 2223 /* 3: update and set xattr node page dirty */ 2224 + memcpy(F2FS_NODE(xpage), F2FS_NODE(page), VALID_XATTR_BLOCK_SIZE); 2225 2226 set_page_dirty(xpage); 2227 f2fs_put_page(xpage, 1); 2228 ··· 2262 dst->i_blocks = cpu_to_le64(1); 2263 dst->i_links = cpu_to_le32(1); 2264 dst->i_xattr_nid = 0; 2265 + dst->i_inline = src->i_inline & (F2FS_INLINE_XATTR | F2FS_EXTRA_ATTR); 2266 + if (dst->i_inline & F2FS_EXTRA_ATTR) { 2267 + dst->i_extra_isize = src->i_extra_isize; 2268 + if (f2fs_sb_has_project_quota(sbi->sb) && 2269 + F2FS_FITS_IN_INODE(src, le16_to_cpu(src->i_extra_isize), 2270 + i_projid)) 2271 + dst->i_projid = src->i_projid; 2272 + } 2273 2274 new_ni = old_ni; 2275 new_ni.ino = ino;
+69 -14
fs/f2fs/recovery.c
··· 69 } 70 71 static struct fsync_inode_entry *add_fsync_inode(struct f2fs_sb_info *sbi, 72 - struct list_head *head, nid_t ino) 73 { 74 struct inode *inode; 75 struct fsync_inode_entry *entry; 76 77 inode = f2fs_iget_retry(sbi->sb, ino); 78 if (IS_ERR(inode)) 79 return ERR_CAST(inode); 80 81 entry = f2fs_kmem_cache_alloc(fsync_entry_slab, GFP_F2FS_ZERO); 82 entry->inode = inode; 83 list_add_tail(&entry->list, head); 84 85 return entry; 86 } 87 88 static void del_fsync_inode(struct fsync_inode_entry *entry) ··· 121 122 entry = get_fsync_inode(dir_list, pino); 123 if (!entry) { 124 - entry = add_fsync_inode(F2FS_I_SB(inode), dir_list, pino); 125 if (IS_ERR(entry)) { 126 dir = ERR_CAST(entry); 127 err = PTR_ERR(entry); ··· 155 err = -EEXIST; 156 goto out_unmap_put; 157 } 158 err = acquire_orphan_inode(F2FS_I_SB(inode)); 159 if (err) { 160 iput(einode); ··· 248 249 entry = get_fsync_inode(head, ino_of_node(page)); 250 if (!entry) { 251 if (!check_only && 252 IS_INODE(page) && is_dent_dnode(page)) { 253 err = recover_inode_page(sbi, page); 254 if (err) 255 break; 256 } 257 258 /* 259 * CP | dnode(F) | inode(DF) 260 * For this case, we should not give up now. 261 */ 262 - entry = add_fsync_inode(sbi, head, ino_of_node(page)); 263 if (IS_ERR(entry)) { 264 err = PTR_ERR(entry); 265 if (err == -ENOENT) { ··· 317 return 0; 318 319 /* Get the previous summary */ 320 - for (i = CURSEG_WARM_DATA; i <= CURSEG_COLD_DATA; i++) { 321 struct curseg_info *curseg = CURSEG_I(sbi, i); 322 if (curseg->segno == segno) { 323 sum = curseg->sum_blk->entries[blkoff]; ··· 354 f2fs_put_page(node_page, 1); 355 356 if (ino != dn->inode->i_ino) { 357 /* Deallocate previous index in the node page */ 358 inode = f2fs_iget_retry(sbi->sb, ino); 359 if (IS_ERR(inode)) 360 return PTR_ERR(inode); 361 } else { 362 inode = dn->inode; 363 } ··· 395 return 0; 396 397 truncate_out: 398 - if (datablock_addr(tdn.node_page, tdn.ofs_in_node) == blkaddr) 399 truncate_data_blocks_range(&tdn, 1); 400 if (dn->inode->i_ino == nid && !dn->inode_page_locked) 401 unlock_page(dn->inode_page); ··· 449 for (; start < end; start++, dn.ofs_in_node++) { 450 block_t src, dest; 451 452 - src = datablock_addr(dn.node_page, dn.ofs_in_node); 453 - dest = datablock_addr(page, dn.ofs_in_node); 454 455 /* skip recovering if dest is the same as src */ 456 if (src == dest) ··· 592 struct list_head dir_list; 593 int err; 594 int ret = 0; 595 bool need_writecp = false; 596 597 fsync_entry_slab = f2fs_kmem_cache_create("f2fs_fsync_inode_entry", 598 sizeof(struct fsync_inode_entry)); 599 - if (!fsync_entry_slab) 600 - return -ENOMEM; 601 602 INIT_LIST_HEAD(&inode_list); 603 INIT_LIST_HEAD(&dir_list); ··· 623 /* step #1: find fsynced inode numbers */ 624 err = find_fsync_dnodes(sbi, &inode_list, check_only); 625 if (err || list_empty(&inode_list)) 626 - goto out; 627 628 if (check_only) { 629 ret = 1; 630 - goto out; 631 } 632 633 need_writecp = true; ··· 636 err = recover_data(sbi, &inode_list, &dir_list); 637 if (!err) 638 f2fs_bug_on(sbi, !list_empty(&inode_list)); 639 - out: 640 destroy_fsync_dnodes(&inode_list); 641 642 /* truncate meta pages to be used by the recovery */ ··· 649 } 650 651 clear_sbi_flag(sbi, SBI_POR_DOING); 652 - if (err) 653 - set_ckpt_flags(sbi, CP_ERROR_FLAG); 654 mutex_unlock(&sbi->cp_mutex); 655 656 /* let's drop all the directory inodes for clean checkpoint */ ··· 662 } 663 664 kmem_cache_destroy(fsync_entry_slab); 665 return ret ? ret: err; 666 }
··· 69 } 70 71 static struct fsync_inode_entry *add_fsync_inode(struct f2fs_sb_info *sbi, 72 + struct list_head *head, nid_t ino, bool quota_inode) 73 { 74 struct inode *inode; 75 struct fsync_inode_entry *entry; 76 + int err; 77 78 inode = f2fs_iget_retry(sbi->sb, ino); 79 if (IS_ERR(inode)) 80 return ERR_CAST(inode); 81 + 82 + err = dquot_initialize(inode); 83 + if (err) 84 + goto err_out; 85 + 86 + if (quota_inode) { 87 + err = dquot_alloc_inode(inode); 88 + if (err) 89 + goto err_out; 90 + } 91 92 entry = f2fs_kmem_cache_alloc(fsync_entry_slab, GFP_F2FS_ZERO); 93 entry->inode = inode; 94 list_add_tail(&entry->list, head); 95 96 return entry; 97 + err_out: 98 + iput(inode); 99 + return ERR_PTR(err); 100 } 101 102 static void del_fsync_inode(struct fsync_inode_entry *entry) ··· 107 108 entry = get_fsync_inode(dir_list, pino); 109 if (!entry) { 110 + entry = add_fsync_inode(F2FS_I_SB(inode), dir_list, 111 + pino, false); 112 if (IS_ERR(entry)) { 113 dir = ERR_CAST(entry); 114 err = PTR_ERR(entry); ··· 140 err = -EEXIST; 141 goto out_unmap_put; 142 } 143 + 144 + err = dquot_initialize(einode); 145 + if (err) { 146 + iput(einode); 147 + goto out_unmap_put; 148 + } 149 + 150 err = acquire_orphan_inode(F2FS_I_SB(inode)); 151 if (err) { 152 iput(einode); ··· 226 227 entry = get_fsync_inode(head, ino_of_node(page)); 228 if (!entry) { 229 + bool quota_inode = false; 230 + 231 if (!check_only && 232 IS_INODE(page) && is_dent_dnode(page)) { 233 err = recover_inode_page(sbi, page); 234 if (err) 235 break; 236 + quota_inode = true; 237 } 238 239 /* 240 * CP | dnode(F) | inode(DF) 241 * For this case, we should not give up now. 242 */ 243 + entry = add_fsync_inode(sbi, head, ino_of_node(page), 244 + quota_inode); 245 if (IS_ERR(entry)) { 246 err = PTR_ERR(entry); 247 if (err == -ENOENT) { ··· 291 return 0; 292 293 /* Get the previous summary */ 294 + for (i = CURSEG_HOT_DATA; i <= CURSEG_COLD_DATA; i++) { 295 struct curseg_info *curseg = CURSEG_I(sbi, i); 296 if (curseg->segno == segno) { 297 sum = curseg->sum_blk->entries[blkoff]; ··· 328 f2fs_put_page(node_page, 1); 329 330 if (ino != dn->inode->i_ino) { 331 + int ret; 332 + 333 /* Deallocate previous index in the node page */ 334 inode = f2fs_iget_retry(sbi->sb, ino); 335 if (IS_ERR(inode)) 336 return PTR_ERR(inode); 337 + 338 + ret = dquot_initialize(inode); 339 + if (ret) { 340 + iput(inode); 341 + return ret; 342 + } 343 } else { 344 inode = dn->inode; 345 } ··· 361 return 0; 362 363 truncate_out: 364 + if (datablock_addr(tdn.inode, tdn.node_page, 365 + tdn.ofs_in_node) == blkaddr) 366 truncate_data_blocks_range(&tdn, 1); 367 if (dn->inode->i_ino == nid && !dn->inode_page_locked) 368 unlock_page(dn->inode_page); ··· 414 for (; start < end; start++, dn.ofs_in_node++) { 415 block_t src, dest; 416 417 + src = datablock_addr(dn.inode, dn.node_page, dn.ofs_in_node); 418 + dest = datablock_addr(dn.inode, page, dn.ofs_in_node); 419 420 /* skip recovering if dest is the same as src */ 421 if (src == dest) ··· 557 struct list_head dir_list; 558 int err; 559 int ret = 0; 560 + unsigned long s_flags = sbi->sb->s_flags; 561 bool need_writecp = false; 562 + 563 + if (s_flags & MS_RDONLY) { 564 + f2fs_msg(sbi->sb, KERN_INFO, "orphan cleanup on readonly fs"); 565 + sbi->sb->s_flags &= ~MS_RDONLY; 566 + } 567 + 568 + #ifdef CONFIG_QUOTA 569 + /* Needed for iput() to work correctly and not trash data */ 570 + sbi->sb->s_flags |= MS_ACTIVE; 571 + /* Turn on quotas so that they are updated correctly */ 572 + f2fs_enable_quota_files(sbi); 573 + #endif 574 575 fsync_entry_slab = f2fs_kmem_cache_create("f2fs_fsync_inode_entry", 576 sizeof(struct fsync_inode_entry)); 577 + if (!fsync_entry_slab) { 578 + err = -ENOMEM; 579 + goto out; 580 + } 581 582 INIT_LIST_HEAD(&inode_list); 583 INIT_LIST_HEAD(&dir_list); ··· 573 /* step #1: find fsynced inode numbers */ 574 err = find_fsync_dnodes(sbi, &inode_list, check_only); 575 if (err || list_empty(&inode_list)) 576 + goto skip; 577 578 if (check_only) { 579 ret = 1; 580 + goto skip; 581 } 582 583 need_writecp = true; ··· 586 err = recover_data(sbi, &inode_list, &dir_list); 587 if (!err) 588 f2fs_bug_on(sbi, !list_empty(&inode_list)); 589 + skip: 590 destroy_fsync_dnodes(&inode_list); 591 592 /* truncate meta pages to be used by the recovery */ ··· 599 } 600 601 clear_sbi_flag(sbi, SBI_POR_DOING); 602 mutex_unlock(&sbi->cp_mutex); 603 604 /* let's drop all the directory inodes for clean checkpoint */ ··· 614 } 615 616 kmem_cache_destroy(fsync_entry_slab); 617 + out: 618 + #ifdef CONFIG_QUOTA 619 + /* Turn quotas off */ 620 + f2fs_quota_off_umount(sbi->sb); 621 + #endif 622 + sbi->sb->s_flags = s_flags; /* Restore MS_RDONLY status */ 623 + 624 return ret ? ret: err; 625 }
+237 -59
fs/f2fs/segment.c
··· 17 #include <linux/swap.h> 18 #include <linux/timer.h> 19 #include <linux/freezer.h> 20 21 #include "f2fs.h" 22 #include "segment.h" 23 #include "node.h" 24 #include "trace.h" 25 #include <trace/events/f2fs.h> 26 ··· 169 return result - size + __reverse_ffz(tmp); 170 } 171 172 void register_inmem_page(struct inode *inode, struct page *page) 173 { 174 struct f2fs_inode_info *fi = F2FS_I(inode); ··· 230 struct node_info ni; 231 232 trace_f2fs_commit_inmem_page(page, INMEM_REVOKE); 233 - 234 set_new_dnode(&dn, inode, NULL, NULL, 0); 235 - if (get_dnode_of_data(&dn, page->index, LOOKUP_NODE)) { 236 err = -EAGAIN; 237 goto next; 238 } ··· 271 mutex_unlock(&fi->inmem_lock); 272 273 clear_inode_flag(inode, FI_ATOMIC_FILE); 274 stat_dec_atomic_write(inode); 275 } 276 ··· 316 .type = DATA, 317 .op = REQ_OP_WRITE, 318 .op_flags = REQ_SYNC | REQ_PRIO, 319 }; 320 pgoff_t last_idx = ULONG_MAX; 321 int err = 0; ··· 334 inode_dec_dirty_pages(inode); 335 remove_dirty_inode(inode); 336 } 337 - 338 fio.page = page; 339 fio.old_blkaddr = NULL_ADDR; 340 fio.encrypted_page = NULL; 341 fio.need_lock = LOCK_DONE; 342 err = do_write_data_page(&fio); 343 if (err) { 344 unlock_page(page); 345 break; 346 } 347 - 348 /* record old blkaddr for revoking */ 349 cur->old_addr = fio.old_blkaddr; 350 last_idx = page->index; ··· 510 if (kthread_should_stop()) 511 return 0; 512 513 if (!llist_empty(&fcc->issue_list)) { 514 struct flush_cmd *cmd, *next; 515 int ret; ··· 529 } 530 fcc->dispatch_list = NULL; 531 } 532 533 wait_event_interruptible(*q, 534 kthread_should_stop() || !llist_empty(&fcc->issue_list)); ··· 552 return ret; 553 } 554 555 - if (!atomic_read(&fcc->issing_flush)) { 556 - atomic_inc(&fcc->issing_flush); 557 ret = submit_flush_wait(sbi); 558 atomic_dec(&fcc->issing_flush); 559 ··· 562 563 init_completion(&cmd.wait); 564 565 - atomic_inc(&fcc->issing_flush); 566 llist_add(&cmd.llnode, &fcc->issue_list); 567 568 - if (!fcc->dispatch_list) 569 wake_up(&fcc->flush_wait_queue); 570 571 if (fcc->f2fs_issue_flush) { 572 wait_for_completion(&cmd.wait); 573 atomic_dec(&fcc->issing_flush); 574 } else { 575 - llist_del_all(&fcc->issue_list); 576 - atomic_set(&fcc->issing_flush, 0); 577 } 578 579 return cmd.ret; ··· 831 sentry = get_seg_entry(sbi, segno); 832 offset = GET_BLKOFF_FROM_SEG0(sbi, blk); 833 834 - size = min((unsigned long)(end - blk), max_blocks); 835 map = (unsigned long *)(sentry->cur_valid_map); 836 offset = __find_rev_next_bit(map, size, offset); 837 f2fs_bug_on(sbi, offset != size); 838 - blk += size; 839 } 840 #endif 841 } ··· 871 submit_bio(bio); 872 list_move_tail(&dc->list, &dcc->wait_list); 873 __check_sit_bitmap(sbi, dc->start, dc->start + dc->len); 874 } 875 } else { 876 __remove_discard_cmd(sbi, dc); ··· 1054 return 0; 1055 } 1056 1057 - static void __issue_discard_cmd(struct f2fs_sb_info *sbi, bool issue_cond) 1058 { 1059 struct discard_cmd_control *dcc = SM_I(sbi)->dcc_info; 1060 struct list_head *pend_list; 1061 struct discard_cmd *dc, *tmp; 1062 struct blk_plug plug; 1063 - int i, iter = 0; 1064 1065 mutex_lock(&dcc->cmd_lock); 1066 f2fs_bug_on(sbi, 1067 !__check_rb_tree_consistence(sbi, &dcc->root)); 1068 blk_start_plug(&plug); 1069 - for (i = MAX_PLIST_NUM - 1; i >= 0; i--) { 1070 pend_list = &dcc->pend_list[i]; 1071 list_for_each_entry_safe(dc, tmp, pend_list, list) { 1072 f2fs_bug_on(sbi, dc->state != D_PREP); 1073 1074 - if (!issue_cond || is_idle(sbi)) 1075 __submit_discard_cmd(sbi, dc); 1076 - if (issue_cond && iter++ > DISCARD_ISSUE_RATE) 1077 goto out; 1078 } 1079 } 1080 out: 1081 blk_finish_plug(&plug); 1082 mutex_unlock(&dcc->cmd_lock); 1083 } 1084 ··· 1209 } 1210 } 1211 1212 - /* This comes from f2fs_put_super */ 1213 void f2fs_wait_discard_bios(struct f2fs_sb_info *sbi) 1214 { 1215 __issue_discard_cmd(sbi, false); 1216 __wait_discard_cmd(sbi, false); 1217 } 1218 1219 static int issue_discard_thread(void *data) ··· 1233 struct f2fs_sb_info *sbi = data; 1234 struct discard_cmd_control *dcc = SM_I(sbi)->dcc_info; 1235 wait_queue_head_t *q = &dcc->discard_wait_queue; 1236 1237 set_freezable(); 1238 1239 do { 1240 - wait_event_interruptible(*q, kthread_should_stop() || 1241 - freezing(current) || 1242 - atomic_read(&dcc->discard_cmd_cnt)); 1243 if (try_to_freeze()) 1244 continue; 1245 if (kthread_should_stop()) 1246 return 0; 1247 1248 - __issue_discard_cmd(sbi, true); 1249 - __wait_discard_cmd(sbi, true); 1250 1251 - congestion_wait(BLK_RW_SYNC, HZ/50); 1252 } while (!kthread_should_stop()); 1253 return 0; 1254 } ··· 1456 1457 void clear_prefree_segments(struct f2fs_sb_info *sbi, struct cp_control *cpc) 1458 { 1459 - struct list_head *head = &(SM_I(sbi)->dcc_info->entry_list); 1460 struct discard_entry *entry, *this; 1461 struct dirty_seglist_info *dirty_i = DIRTY_I(sbi); 1462 unsigned long *prefree_map = dirty_i->dirty_segmap[PRE]; ··· 1539 goto find_next; 1540 1541 list_del(&entry->list); 1542 - SM_I(sbi)->dcc_info->nr_discards -= total_len; 1543 kmem_cache_free(discard_entry_slab, entry); 1544 } 1545 1546 - wake_up(&SM_I(sbi)->dcc_info->discard_wait_queue); 1547 } 1548 1549 static int create_discard_cmd_control(struct f2fs_sb_info *sbi) ··· 1561 if (!dcc) 1562 return -ENOMEM; 1563 1564 INIT_LIST_HEAD(&dcc->entry_list); 1565 - for (i = 0; i < MAX_PLIST_NUM; i++) 1566 INIT_LIST_HEAD(&dcc->pend_list[i]); 1567 INIT_LIST_HEAD(&dcc->wait_list); 1568 mutex_init(&dcc->cmd_lock); 1569 atomic_set(&dcc->issued_discard, 0); ··· 1632 struct seg_entry *se; 1633 unsigned int segno, offset; 1634 long int new_vblocks; 1635 1636 segno = GET_SEGNO(sbi, blkaddr); 1637 ··· 1652 1653 /* Update valid block bitmap */ 1654 if (del > 0) { 1655 - if (f2fs_test_and_set_bit(offset, se->cur_valid_map)) { 1656 #ifdef CONFIG_F2FS_CHECK_FS 1657 - if (f2fs_test_and_set_bit(offset, 1658 - se->cur_valid_map_mir)) 1659 - f2fs_bug_on(sbi, 1); 1660 - else 1661 - WARN_ON(1); 1662 - #else 1663 f2fs_bug_on(sbi, 1); 1664 - #endif 1665 } 1666 if (f2fs_discard_en(sbi) && 1667 !f2fs_test_and_set_bit(offset, se->discard_map)) 1668 sbi->discard_blks--; ··· 1681 se->ckpt_valid_blocks++; 1682 } 1683 } else { 1684 - if (!f2fs_test_and_clear_bit(offset, se->cur_valid_map)) { 1685 #ifdef CONFIG_F2FS_CHECK_FS 1686 - if (!f2fs_test_and_clear_bit(offset, 1687 - se->cur_valid_map_mir)) 1688 - f2fs_bug_on(sbi, 1); 1689 - else 1690 - WARN_ON(1); 1691 - #else 1692 f2fs_bug_on(sbi, 1); 1693 - #endif 1694 } 1695 if (f2fs_discard_en(sbi) && 1696 f2fs_test_and_clear_bit(offset, se->discard_map)) 1697 sbi->discard_blks++; ··· 2061 * This function always allocates a used segment(from dirty seglist) by SSR 2062 * manner, so it should recover the existing segment information of valid blocks 2063 */ 2064 - static void change_curseg(struct f2fs_sb_info *sbi, int type, bool reuse) 2065 { 2066 struct dirty_seglist_info *dirty_i = DIRTY_I(sbi); 2067 struct curseg_info *curseg = CURSEG_I(sbi, type); ··· 2082 curseg->alloc_type = SSR; 2083 __next_free_blkoff(sbi, curseg, 0); 2084 2085 - if (reuse) { 2086 - sum_page = get_sum_page(sbi, new_segno); 2087 - sum_node = (struct f2fs_summary_block *)page_address(sum_page); 2088 - memcpy(curseg->sum_blk, sum_node, SUM_ENTRY_SIZE); 2089 - f2fs_put_page(sum_page, 1); 2090 - } 2091 } 2092 2093 static int get_ssr_segment(struct f2fs_sb_info *sbi, int type) ··· 2149 else if (curseg->alloc_type == LFS && is_next_segment_free(sbi, type)) 2150 new_curseg(sbi, type, false); 2151 else if (need_SSR(sbi) && get_ssr_segment(sbi, type)) 2152 - change_curseg(sbi, type, true); 2153 else 2154 new_curseg(sbi, type, false); 2155 ··· 2242 2243 schedule(); 2244 } 2245 out: 2246 range->len = F2FS_BLK_TO_BYTES(cpc.trimmed); 2247 return err; ··· 2364 2365 mutex_unlock(&sit_i->sentry_lock); 2366 2367 - if (page && IS_NODESEG(type)) 2368 fill_node_footer_blkaddr(page, NEXT_FREE_BLKADDR(sbi, curseg)); 2369 2370 if (add_list) { 2371 struct f2fs_bio_info *io; ··· 2401 } 2402 } 2403 2404 - void write_meta_page(struct f2fs_sb_info *sbi, struct page *page) 2405 { 2406 struct f2fs_io_info fio = { 2407 .sbi = sbi, ··· 2421 2422 set_page_writeback(page); 2423 f2fs_submit_page_write(&fio); 2424 } 2425 2426 void write_node_page(unsigned int nid, struct f2fs_io_info *fio) ··· 2431 2432 set_summary(&sum, nid, 0, 0); 2433 do_write_page(&sum, fio); 2434 } 2435 2436 void write_data_page(struct dnode_of_data *dn, struct f2fs_io_info *fio) ··· 2446 set_summary(&sum, dn->nid, dn->ofs_in_node, ni.version); 2447 do_write_page(&sum, fio); 2448 f2fs_update_data_blkaddr(dn, fio->new_blkaddr); 2449 } 2450 2451 int rewrite_data_page(struct f2fs_io_info *fio) 2452 { 2453 fio->new_blkaddr = fio->old_blkaddr; 2454 stat_inc_inplace_blocks(fio->sbi); 2455 - return f2fs_submit_page_bio(fio); 2456 } 2457 2458 void __f2fs_replace_block(struct f2fs_sb_info *sbi, struct f2fs_summary *sum, ··· 2503 /* change the current segment */ 2504 if (segno != curseg->segno) { 2505 curseg->next_segno = segno; 2506 - change_curseg(sbi, type, true); 2507 } 2508 2509 curseg->next_blkoff = GET_BLKOFF_FROM_SEG0(sbi, new_blkaddr); ··· 2522 if (recover_curseg) { 2523 if (old_cursegno != curseg->segno) { 2524 curseg->next_segno = old_cursegno; 2525 - change_curseg(sbi, type, true); 2526 } 2527 curseg->next_blkoff = old_blkoff; 2528 } ··· 2561 } 2562 } 2563 2564 - void f2fs_wait_on_encrypted_page_writeback(struct f2fs_sb_info *sbi, 2565 - block_t blkaddr) 2566 { 2567 struct page *cpage; 2568
··· 17 #include <linux/swap.h> 18 #include <linux/timer.h> 19 #include <linux/freezer.h> 20 + #include <linux/sched/signal.h> 21 22 #include "f2fs.h" 23 #include "segment.h" 24 #include "node.h" 25 + #include "gc.h" 26 #include "trace.h" 27 #include <trace/events/f2fs.h> 28 ··· 167 return result - size + __reverse_ffz(tmp); 168 } 169 170 + bool need_SSR(struct f2fs_sb_info *sbi) 171 + { 172 + int node_secs = get_blocktype_secs(sbi, F2FS_DIRTY_NODES); 173 + int dent_secs = get_blocktype_secs(sbi, F2FS_DIRTY_DENTS); 174 + int imeta_secs = get_blocktype_secs(sbi, F2FS_DIRTY_IMETA); 175 + 176 + if (test_opt(sbi, LFS)) 177 + return false; 178 + if (sbi->gc_thread && sbi->gc_thread->gc_urgent) 179 + return true; 180 + 181 + return free_sections(sbi) <= (node_secs + 2 * dent_secs + imeta_secs + 182 + 2 * reserved_sections(sbi)); 183 + } 184 + 185 void register_inmem_page(struct inode *inode, struct page *page) 186 { 187 struct f2fs_inode_info *fi = F2FS_I(inode); ··· 213 struct node_info ni; 214 215 trace_f2fs_commit_inmem_page(page, INMEM_REVOKE); 216 + retry: 217 set_new_dnode(&dn, inode, NULL, NULL, 0); 218 + err = get_dnode_of_data(&dn, page->index, LOOKUP_NODE); 219 + if (err) { 220 + if (err == -ENOMEM) { 221 + congestion_wait(BLK_RW_ASYNC, HZ/50); 222 + cond_resched(); 223 + goto retry; 224 + } 225 err = -EAGAIN; 226 goto next; 227 } ··· 248 mutex_unlock(&fi->inmem_lock); 249 250 clear_inode_flag(inode, FI_ATOMIC_FILE); 251 + clear_inode_flag(inode, FI_HOT_DATA); 252 stat_dec_atomic_write(inode); 253 } 254 ··· 292 .type = DATA, 293 .op = REQ_OP_WRITE, 294 .op_flags = REQ_SYNC | REQ_PRIO, 295 + .io_type = FS_DATA_IO, 296 }; 297 pgoff_t last_idx = ULONG_MAX; 298 int err = 0; ··· 309 inode_dec_dirty_pages(inode); 310 remove_dirty_inode(inode); 311 } 312 + retry: 313 fio.page = page; 314 fio.old_blkaddr = NULL_ADDR; 315 fio.encrypted_page = NULL; 316 fio.need_lock = LOCK_DONE; 317 err = do_write_data_page(&fio); 318 if (err) { 319 + if (err == -ENOMEM) { 320 + congestion_wait(BLK_RW_ASYNC, HZ/50); 321 + cond_resched(); 322 + goto retry; 323 + } 324 unlock_page(page); 325 break; 326 } 327 /* record old blkaddr for revoking */ 328 cur->old_addr = fio.old_blkaddr; 329 last_idx = page->index; ··· 481 if (kthread_should_stop()) 482 return 0; 483 484 + sb_start_intwrite(sbi->sb); 485 + 486 if (!llist_empty(&fcc->issue_list)) { 487 struct flush_cmd *cmd, *next; 488 int ret; ··· 498 } 499 fcc->dispatch_list = NULL; 500 } 501 + 502 + sb_end_intwrite(sbi->sb); 503 504 wait_event_interruptible(*q, 505 kthread_should_stop() || !llist_empty(&fcc->issue_list)); ··· 519 return ret; 520 } 521 522 + if (atomic_inc_return(&fcc->issing_flush) == 1) { 523 ret = submit_flush_wait(sbi); 524 atomic_dec(&fcc->issing_flush); 525 ··· 530 531 init_completion(&cmd.wait); 532 533 llist_add(&cmd.llnode, &fcc->issue_list); 534 535 + /* update issue_list before we wake up issue_flush thread */ 536 + smp_mb(); 537 + 538 + if (waitqueue_active(&fcc->flush_wait_queue)) 539 wake_up(&fcc->flush_wait_queue); 540 541 if (fcc->f2fs_issue_flush) { 542 wait_for_completion(&cmd.wait); 543 atomic_dec(&fcc->issing_flush); 544 } else { 545 + struct llist_node *list; 546 + 547 + list = llist_del_all(&fcc->issue_list); 548 + if (!list) { 549 + wait_for_completion(&cmd.wait); 550 + atomic_dec(&fcc->issing_flush); 551 + } else { 552 + struct flush_cmd *tmp, *next; 553 + 554 + ret = submit_flush_wait(sbi); 555 + 556 + llist_for_each_entry_safe(tmp, next, list, llnode) { 557 + if (tmp == &cmd) { 558 + cmd.ret = ret; 559 + atomic_dec(&fcc->issing_flush); 560 + continue; 561 + } 562 + tmp->ret = ret; 563 + complete(&tmp->wait); 564 + } 565 + } 566 } 567 568 return cmd.ret; ··· 778 sentry = get_seg_entry(sbi, segno); 779 offset = GET_BLKOFF_FROM_SEG0(sbi, blk); 780 781 + if (end < START_BLOCK(sbi, segno + 1)) 782 + size = GET_BLKOFF_FROM_SEG0(sbi, end); 783 + else 784 + size = max_blocks; 785 map = (unsigned long *)(sentry->cur_valid_map); 786 offset = __find_rev_next_bit(map, size, offset); 787 f2fs_bug_on(sbi, offset != size); 788 + blk = START_BLOCK(sbi, segno + 1); 789 } 790 #endif 791 } ··· 815 submit_bio(bio); 816 list_move_tail(&dc->list, &dcc->wait_list); 817 __check_sit_bitmap(sbi, dc->start, dc->start + dc->len); 818 + 819 + f2fs_update_iostat(sbi, FS_DISCARD, 1); 820 } 821 } else { 822 __remove_discard_cmd(sbi, dc); ··· 996 return 0; 997 } 998 999 + static int __issue_discard_cmd(struct f2fs_sb_info *sbi, bool issue_cond) 1000 { 1001 struct discard_cmd_control *dcc = SM_I(sbi)->dcc_info; 1002 struct list_head *pend_list; 1003 struct discard_cmd *dc, *tmp; 1004 struct blk_plug plug; 1005 + int iter = 0, issued = 0; 1006 + int i; 1007 + bool io_interrupted = false; 1008 1009 mutex_lock(&dcc->cmd_lock); 1010 f2fs_bug_on(sbi, 1011 !__check_rb_tree_consistence(sbi, &dcc->root)); 1012 blk_start_plug(&plug); 1013 + for (i = MAX_PLIST_NUM - 1; 1014 + i >= 0 && plist_issue(dcc->pend_list_tag[i]); i--) { 1015 pend_list = &dcc->pend_list[i]; 1016 list_for_each_entry_safe(dc, tmp, pend_list, list) { 1017 f2fs_bug_on(sbi, dc->state != D_PREP); 1018 1019 + /* Hurry up to finish fstrim */ 1020 + if (dcc->pend_list_tag[i] & P_TRIM) { 1021 __submit_discard_cmd(sbi, dc); 1022 + issued++; 1023 + 1024 + if (fatal_signal_pending(current)) 1025 + break; 1026 + continue; 1027 + } 1028 + 1029 + if (!issue_cond) { 1030 + __submit_discard_cmd(sbi, dc); 1031 + issued++; 1032 + continue; 1033 + } 1034 + 1035 + if (is_idle(sbi)) { 1036 + __submit_discard_cmd(sbi, dc); 1037 + issued++; 1038 + } else { 1039 + io_interrupted = true; 1040 + } 1041 + 1042 + if (++iter >= DISCARD_ISSUE_RATE) 1043 goto out; 1044 } 1045 + if (list_empty(pend_list) && dcc->pend_list_tag[i] & P_TRIM) 1046 + dcc->pend_list_tag[i] &= (~P_TRIM); 1047 } 1048 out: 1049 blk_finish_plug(&plug); 1050 + mutex_unlock(&dcc->cmd_lock); 1051 + 1052 + if (!issued && io_interrupted) 1053 + issued = -1; 1054 + 1055 + return issued; 1056 + } 1057 + 1058 + static void __drop_discard_cmd(struct f2fs_sb_info *sbi) 1059 + { 1060 + struct discard_cmd_control *dcc = SM_I(sbi)->dcc_info; 1061 + struct list_head *pend_list; 1062 + struct discard_cmd *dc, *tmp; 1063 + int i; 1064 + 1065 + mutex_lock(&dcc->cmd_lock); 1066 + for (i = MAX_PLIST_NUM - 1; i >= 0; i--) { 1067 + pend_list = &dcc->pend_list[i]; 1068 + list_for_each_entry_safe(dc, tmp, pend_list, list) { 1069 + f2fs_bug_on(sbi, dc->state != D_PREP); 1070 + __remove_discard_cmd(sbi, dc); 1071 + } 1072 + } 1073 mutex_unlock(&dcc->cmd_lock); 1074 } 1075 ··· 1102 } 1103 } 1104 1105 + /* This comes from f2fs_put_super and f2fs_trim_fs */ 1106 void f2fs_wait_discard_bios(struct f2fs_sb_info *sbi) 1107 { 1108 __issue_discard_cmd(sbi, false); 1109 + __drop_discard_cmd(sbi); 1110 __wait_discard_cmd(sbi, false); 1111 + } 1112 + 1113 + static void mark_discard_range_all(struct f2fs_sb_info *sbi) 1114 + { 1115 + struct discard_cmd_control *dcc = SM_I(sbi)->dcc_info; 1116 + int i; 1117 + 1118 + mutex_lock(&dcc->cmd_lock); 1119 + for (i = 0; i < MAX_PLIST_NUM; i++) 1120 + dcc->pend_list_tag[i] |= P_TRIM; 1121 + mutex_unlock(&dcc->cmd_lock); 1122 } 1123 1124 static int issue_discard_thread(void *data) ··· 1114 struct f2fs_sb_info *sbi = data; 1115 struct discard_cmd_control *dcc = SM_I(sbi)->dcc_info; 1116 wait_queue_head_t *q = &dcc->discard_wait_queue; 1117 + unsigned int wait_ms = DEF_MIN_DISCARD_ISSUE_TIME; 1118 + int issued; 1119 1120 set_freezable(); 1121 1122 do { 1123 + wait_event_interruptible_timeout(*q, 1124 + kthread_should_stop() || freezing(current) || 1125 + dcc->discard_wake, 1126 + msecs_to_jiffies(wait_ms)); 1127 if (try_to_freeze()) 1128 continue; 1129 if (kthread_should_stop()) 1130 return 0; 1131 1132 + if (dcc->discard_wake) { 1133 + dcc->discard_wake = 0; 1134 + if (sbi->gc_thread && sbi->gc_thread->gc_urgent) 1135 + mark_discard_range_all(sbi); 1136 + } 1137 1138 + sb_start_intwrite(sbi->sb); 1139 + 1140 + issued = __issue_discard_cmd(sbi, true); 1141 + if (issued) { 1142 + __wait_discard_cmd(sbi, true); 1143 + wait_ms = DEF_MIN_DISCARD_ISSUE_TIME; 1144 + } else { 1145 + wait_ms = DEF_MAX_DISCARD_ISSUE_TIME; 1146 + } 1147 + 1148 + sb_end_intwrite(sbi->sb); 1149 + 1150 } while (!kthread_should_stop()); 1151 return 0; 1152 } ··· 1320 1321 void clear_prefree_segments(struct f2fs_sb_info *sbi, struct cp_control *cpc) 1322 { 1323 + struct discard_cmd_control *dcc = SM_I(sbi)->dcc_info; 1324 + struct list_head *head = &dcc->entry_list; 1325 struct discard_entry *entry, *this; 1326 struct dirty_seglist_info *dirty_i = DIRTY_I(sbi); 1327 unsigned long *prefree_map = dirty_i->dirty_segmap[PRE]; ··· 1402 goto find_next; 1403 1404 list_del(&entry->list); 1405 + dcc->nr_discards -= total_len; 1406 kmem_cache_free(discard_entry_slab, entry); 1407 } 1408 1409 + wake_up_discard_thread(sbi, false); 1410 } 1411 1412 static int create_discard_cmd_control(struct f2fs_sb_info *sbi) ··· 1424 if (!dcc) 1425 return -ENOMEM; 1426 1427 + dcc->discard_granularity = DEFAULT_DISCARD_GRANULARITY; 1428 INIT_LIST_HEAD(&dcc->entry_list); 1429 + for (i = 0; i < MAX_PLIST_NUM; i++) { 1430 INIT_LIST_HEAD(&dcc->pend_list[i]); 1431 + if (i >= dcc->discard_granularity - 1) 1432 + dcc->pend_list_tag[i] |= P_ACTIVE; 1433 + } 1434 INIT_LIST_HEAD(&dcc->wait_list); 1435 mutex_init(&dcc->cmd_lock); 1436 atomic_set(&dcc->issued_discard, 0); ··· 1491 struct seg_entry *se; 1492 unsigned int segno, offset; 1493 long int new_vblocks; 1494 + bool exist; 1495 + #ifdef CONFIG_F2FS_CHECK_FS 1496 + bool mir_exist; 1497 + #endif 1498 1499 segno = GET_SEGNO(sbi, blkaddr); 1500 ··· 1507 1508 /* Update valid block bitmap */ 1509 if (del > 0) { 1510 + exist = f2fs_test_and_set_bit(offset, se->cur_valid_map); 1511 #ifdef CONFIG_F2FS_CHECK_FS 1512 + mir_exist = f2fs_test_and_set_bit(offset, 1513 + se->cur_valid_map_mir); 1514 + if (unlikely(exist != mir_exist)) { 1515 + f2fs_msg(sbi->sb, KERN_ERR, "Inconsistent error " 1516 + "when setting bitmap, blk:%u, old bit:%d", 1517 + blkaddr, exist); 1518 f2fs_bug_on(sbi, 1); 1519 } 1520 + #endif 1521 + if (unlikely(exist)) { 1522 + f2fs_msg(sbi->sb, KERN_ERR, 1523 + "Bitmap was wrongly set, blk:%u", blkaddr); 1524 + f2fs_bug_on(sbi, 1); 1525 + se->valid_blocks--; 1526 + del = 0; 1527 + } 1528 + 1529 if (f2fs_discard_en(sbi) && 1530 !f2fs_test_and_set_bit(offset, se->discard_map)) 1531 sbi->discard_blks--; ··· 1528 se->ckpt_valid_blocks++; 1529 } 1530 } else { 1531 + exist = f2fs_test_and_clear_bit(offset, se->cur_valid_map); 1532 #ifdef CONFIG_F2FS_CHECK_FS 1533 + mir_exist = f2fs_test_and_clear_bit(offset, 1534 + se->cur_valid_map_mir); 1535 + if (unlikely(exist != mir_exist)) { 1536 + f2fs_msg(sbi->sb, KERN_ERR, "Inconsistent error " 1537 + "when clearing bitmap, blk:%u, old bit:%d", 1538 + blkaddr, exist); 1539 f2fs_bug_on(sbi, 1); 1540 } 1541 + #endif 1542 + if (unlikely(!exist)) { 1543 + f2fs_msg(sbi->sb, KERN_ERR, 1544 + "Bitmap was wrongly cleared, blk:%u", blkaddr); 1545 + f2fs_bug_on(sbi, 1); 1546 + se->valid_blocks++; 1547 + del = 0; 1548 + } 1549 + 1550 if (f2fs_discard_en(sbi) && 1551 f2fs_test_and_clear_bit(offset, se->discard_map)) 1552 sbi->discard_blks++; ··· 1900 * This function always allocates a used segment(from dirty seglist) by SSR 1901 * manner, so it should recover the existing segment information of valid blocks 1902 */ 1903 + static void change_curseg(struct f2fs_sb_info *sbi, int type) 1904 { 1905 struct dirty_seglist_info *dirty_i = DIRTY_I(sbi); 1906 struct curseg_info *curseg = CURSEG_I(sbi, type); ··· 1921 curseg->alloc_type = SSR; 1922 __next_free_blkoff(sbi, curseg, 0); 1923 1924 + sum_page = get_sum_page(sbi, new_segno); 1925 + sum_node = (struct f2fs_summary_block *)page_address(sum_page); 1926 + memcpy(curseg->sum_blk, sum_node, SUM_ENTRY_SIZE); 1927 + f2fs_put_page(sum_page, 1); 1928 } 1929 1930 static int get_ssr_segment(struct f2fs_sb_info *sbi, int type) ··· 1990 else if (curseg->alloc_type == LFS && is_next_segment_free(sbi, type)) 1991 new_curseg(sbi, type, false); 1992 else if (need_SSR(sbi) && get_ssr_segment(sbi, type)) 1993 + change_curseg(sbi, type); 1994 else 1995 new_curseg(sbi, type, false); 1996 ··· 2083 2084 schedule(); 2085 } 2086 + /* It's time to issue all the filed discards */ 2087 + mark_discard_range_all(sbi); 2088 + f2fs_wait_discard_bios(sbi); 2089 out: 2090 range->len = F2FS_BLK_TO_BYTES(cpc.trimmed); 2091 return err; ··· 2202 2203 mutex_unlock(&sit_i->sentry_lock); 2204 2205 + if (page && IS_NODESEG(type)) { 2206 fill_node_footer_blkaddr(page, NEXT_FREE_BLKADDR(sbi, curseg)); 2207 + 2208 + f2fs_inode_chksum_set(sbi, page); 2209 + } 2210 2211 if (add_list) { 2212 struct f2fs_bio_info *io; ··· 2236 } 2237 } 2238 2239 + void write_meta_page(struct f2fs_sb_info *sbi, struct page *page, 2240 + enum iostat_type io_type) 2241 { 2242 struct f2fs_io_info fio = { 2243 .sbi = sbi, ··· 2255 2256 set_page_writeback(page); 2257 f2fs_submit_page_write(&fio); 2258 + 2259 + f2fs_update_iostat(sbi, io_type, F2FS_BLKSIZE); 2260 } 2261 2262 void write_node_page(unsigned int nid, struct f2fs_io_info *fio) ··· 2263 2264 set_summary(&sum, nid, 0, 0); 2265 do_write_page(&sum, fio); 2266 + 2267 + f2fs_update_iostat(fio->sbi, fio->io_type, F2FS_BLKSIZE); 2268 } 2269 2270 void write_data_page(struct dnode_of_data *dn, struct f2fs_io_info *fio) ··· 2276 set_summary(&sum, dn->nid, dn->ofs_in_node, ni.version); 2277 do_write_page(&sum, fio); 2278 f2fs_update_data_blkaddr(dn, fio->new_blkaddr); 2279 + 2280 + f2fs_update_iostat(sbi, fio->io_type, F2FS_BLKSIZE); 2281 } 2282 2283 int rewrite_data_page(struct f2fs_io_info *fio) 2284 { 2285 + int err; 2286 + 2287 fio->new_blkaddr = fio->old_blkaddr; 2288 stat_inc_inplace_blocks(fio->sbi); 2289 + 2290 + err = f2fs_submit_page_bio(fio); 2291 + 2292 + f2fs_update_iostat(fio->sbi, fio->io_type, F2FS_BLKSIZE); 2293 + 2294 + return err; 2295 } 2296 2297 void __f2fs_replace_block(struct f2fs_sb_info *sbi, struct f2fs_summary *sum, ··· 2324 /* change the current segment */ 2325 if (segno != curseg->segno) { 2326 curseg->next_segno = segno; 2327 + change_curseg(sbi, type); 2328 } 2329 2330 curseg->next_blkoff = GET_BLKOFF_FROM_SEG0(sbi, new_blkaddr); ··· 2343 if (recover_curseg) { 2344 if (old_cursegno != curseg->segno) { 2345 curseg->next_segno = old_cursegno; 2346 + change_curseg(sbi, type); 2347 } 2348 curseg->next_blkoff = old_blkoff; 2349 } ··· 2382 } 2383 } 2384 2385 + void f2fs_wait_on_block_writeback(struct f2fs_sb_info *sbi, block_t blkaddr) 2386 { 2387 struct page *cpage; 2388
+29 -18
fs/f2fs/segment.h
··· 492 return SM_I(sbi)->ovp_segments; 493 } 494 495 - static inline int overprovision_sections(struct f2fs_sb_info *sbi) 496 - { 497 - return GET_SEC_FROM_SEG(sbi, (unsigned int)overprovision_segments(sbi)); 498 - } 499 - 500 static inline int reserved_sections(struct f2fs_sb_info *sbi) 501 { 502 return GET_SEC_FROM_SEG(sbi, (unsigned int)reserved_segments(sbi)); 503 - } 504 - 505 - static inline bool need_SSR(struct f2fs_sb_info *sbi) 506 - { 507 - int node_secs = get_blocktype_secs(sbi, F2FS_DIRTY_NODES); 508 - int dent_secs = get_blocktype_secs(sbi, F2FS_DIRTY_DENTS); 509 - int imeta_secs = get_blocktype_secs(sbi, F2FS_DIRTY_IMETA); 510 - 511 - if (test_opt(sbi, LFS)) 512 - return false; 513 - 514 - return free_sections(sbi) <= (node_secs + 2 * dent_secs + imeta_secs + 515 - 2 * reserved_sections(sbi)); 516 } 517 518 static inline bool has_not_enough_free_secs(struct f2fs_sb_info *sbi, ··· 558 559 if (test_opt(sbi, LFS)) 560 return false; 561 562 if (policy & (0x1 << F2FS_IPU_FORCE)) 563 return true; ··· 784 785 wbc->nr_to_write = desired; 786 return desired - nr_to_write; 787 }
··· 492 return SM_I(sbi)->ovp_segments; 493 } 494 495 static inline int reserved_sections(struct f2fs_sb_info *sbi) 496 { 497 return GET_SEC_FROM_SEG(sbi, (unsigned int)reserved_segments(sbi)); 498 } 499 500 static inline bool has_not_enough_free_secs(struct f2fs_sb_info *sbi, ··· 576 577 if (test_opt(sbi, LFS)) 578 return false; 579 + 580 + /* if this is cold file, we should overwrite to avoid fragmentation */ 581 + if (file_is_cold(inode)) 582 + return true; 583 584 if (policy & (0x1 << F2FS_IPU_FORCE)) 585 return true; ··· 798 799 wbc->nr_to_write = desired; 800 return desired - nr_to_write; 801 + } 802 + 803 + static inline void wake_up_discard_thread(struct f2fs_sb_info *sbi, bool force) 804 + { 805 + struct discard_cmd_control *dcc = SM_I(sbi)->dcc_info; 806 + bool wakeup = false; 807 + int i; 808 + 809 + if (force) 810 + goto wake_up; 811 + 812 + mutex_lock(&dcc->cmd_lock); 813 + for (i = MAX_PLIST_NUM - 1; 814 + i >= 0 && plist_issue(dcc->pend_list_tag[i]); i--) { 815 + if (!list_empty(&dcc->pend_list[i])) { 816 + wakeup = true; 817 + break; 818 + } 819 + } 820 + mutex_unlock(&dcc->cmd_lock); 821 + if (!wakeup) 822 + return; 823 + wake_up: 824 + dcc->discard_wake = 1; 825 + wake_up_interruptible_all(&dcc->discard_wait_queue); 826 }
+401 -32
fs/f2fs/super.c
··· 25 #include <linux/quotaops.h> 26 #include <linux/f2fs_fs.h> 27 #include <linux/sysfs.h> 28 29 #include "f2fs.h" 30 #include "node.h" ··· 108 Opt_fault_injection, 109 Opt_lazytime, 110 Opt_nolazytime, 111 Opt_usrquota, 112 Opt_grpquota, 113 Opt_err, 114 }; 115 ··· 157 {Opt_fault_injection, "fault_injection=%u"}, 158 {Opt_lazytime, "lazytime"}, 159 {Opt_nolazytime, "nolazytime"}, 160 {Opt_usrquota, "usrquota"}, 161 {Opt_grpquota, "grpquota"}, 162 {Opt_err, NULL}, 163 }; 164 ··· 182 va_start(args, fmt); 183 vaf.fmt = fmt; 184 vaf.va = &args; 185 - printk("%sF2FS-fs (%s): %pV\n", level, sb->s_id, &vaf); 186 va_end(args); 187 } 188 ··· 193 inode_init_once(&fi->vfs_inode); 194 } 195 196 static int parse_options(struct super_block *sb, char *options) 197 { 198 struct f2fs_sb_info *sbi = F2FS_SB(sb); ··· 298 substring_t args[MAX_OPT_ARGS]; 299 char *p, *name; 300 int arg = 0; 301 302 if (!options) 303 return 0; ··· 512 sb->s_flags &= ~MS_LAZYTIME; 513 break; 514 #ifdef CONFIG_QUOTA 515 case Opt_usrquota: 516 set_opt(sbi, USRQUOTA); 517 break; 518 case Opt_grpquota: 519 set_opt(sbi, GRPQUOTA); 520 break; 521 #else 522 case Opt_usrquota: 523 case Opt_grpquota: 524 f2fs_msg(sb, KERN_INFO, 525 "quota operations not supported"); 526 break; ··· 593 return -EINVAL; 594 } 595 } 596 597 if (F2FS_IO_SIZE_BITS(sbi) && !test_opt(sbi, LFS)) { 598 f2fs_msg(sb, KERN_ERR, ··· 630 init_rwsem(&fi->dio_rwsem[READ]); 631 init_rwsem(&fi->dio_rwsem[WRITE]); 632 init_rwsem(&fi->i_mmap_sem); 633 634 #ifdef CONFIG_QUOTA 635 memset(&fi->i_dquot, 0, sizeof(fi->i_dquot)); ··· 638 #endif 639 /* Will be used by directory only */ 640 fi->i_dir_level = F2FS_SB(sb)->dir_level; 641 return &fi->vfs_inode; 642 } 643 ··· 777 kfree(sbi->devs); 778 } 779 780 - static void f2fs_quota_off_umount(struct super_block *sb); 781 static void f2fs_put_super(struct super_block *sb) 782 { 783 struct f2fs_sb_info *sbi = F2FS_SB(sb); ··· 834 835 kfree(sbi->ckpt); 836 837 - f2fs_exit_sysfs(sbi); 838 839 sb->s_fs_info = NULL; 840 if (sbi->s_chksum_driver) ··· 843 844 destroy_device_list(sbi); 845 mempool_destroy(sbi->write_io_dummy); 846 destroy_percpu_info(sbi); 847 for (i = 0; i < NR_PAGE_TYPE; i++) 848 kfree(sbi->write_io[i]); ··· 859 int err = 0; 860 861 trace_f2fs_sync_fs(sb, sync); 862 863 if (sync) { 864 struct cp_control cpc; ··· 896 { 897 return 0; 898 } 899 900 static int f2fs_statfs(struct dentry *dentry, struct kstatfs *buf) 901 { ··· 974 buf->f_fsid.val[0] = (u32)id; 975 buf->f_fsid.val[1] = (u32)(id >> 32); 976 977 return 0; 978 } 979 980 static int f2fs_show_options(struct seq_file *seq, struct dentry *root) ··· 1090 sbi->fault_info.inject_rate); 1091 #endif 1092 #ifdef CONFIG_QUOTA 1093 if (test_opt(sbi, USRQUOTA)) 1094 seq_puts(seq, ",usrquota"); 1095 if (test_opt(sbi, GRPQUOTA)) 1096 seq_puts(seq, ",grpquota"); 1097 #endif 1098 1099 return 0; 1100 } ··· 1148 #ifdef CONFIG_F2FS_FAULT_INJECTION 1149 struct f2fs_fault_info ffi = sbi->fault_info; 1150 #endif 1151 1152 /* 1153 * Save the old mount options in case we ··· 1161 org_mount_opt = sbi->mount_opt; 1162 old_sb_flags = sb->s_flags; 1163 active_logs = sbi->active_logs; 1164 1165 /* recover superblocks we couldn't write due to previous RO mount */ 1166 if (!(*flags & MS_RDONLY) && is_sbi_flag_set(sbi, SBI_NEED_SB_WRITE)) { ··· 1260 goto restore_gc; 1261 } 1262 skip: 1263 /* Update the POSIXACL Flag */ 1264 sb->s_flags = (sb->s_flags & ~MS_POSIXACL) | 1265 (test_opt(sbi, POSIX_ACL) ? MS_POSIXACL : 0); ··· 1279 stop_gc_thread(sbi); 1280 } 1281 restore_opts: 1282 sbi->mount_opt = org_mount_opt; 1283 sbi->active_logs = active_logs; 1284 sb->s_flags = old_sb_flags; ··· 1385 } 1386 1387 if (len == towrite) 1388 - return err; 1389 inode->i_version++; 1390 inode->i_mtime = inode->i_ctime = current_time(inode); 1391 f2fs_mark_inode_dirty_sync(inode, false); ··· 1400 static qsize_t *f2fs_get_reserved_space(struct inode *inode) 1401 { 1402 return &F2FS_I(inode)->i_reserved_quota; 1403 } 1404 1405 static int f2fs_quota_sync(struct super_block *sb, int type) ··· 1460 struct inode *inode; 1461 int err; 1462 1463 - err = f2fs_quota_sync(sb, -1); 1464 if (err) 1465 return err; 1466 ··· 1488 if (!inode || !igrab(inode)) 1489 return dquot_quota_off(sb, type); 1490 1491 - f2fs_quota_sync(sb, -1); 1492 1493 err = dquot_quota_off(sb, type); 1494 if (err) ··· 1504 return err; 1505 } 1506 1507 - static void f2fs_quota_off_umount(struct super_block *sb) 1508 { 1509 int type; 1510 1511 for (type = 0; type < MAXQUOTAS; type++) 1512 f2fs_quota_off(sb, type); 1513 } 1514 1515 static const struct dquot_operations f2fs_quota_operations = { ··· 1527 .write_info = dquot_commit_info, 1528 .alloc_dquot = dquot_alloc, 1529 .destroy_dquot = dquot_destroy, 1530 .get_next_id = dquot_get_next_id, 1531 }; 1532 ··· 1542 .get_nextdqblk = dquot_get_next_dqblk, 1543 }; 1544 #else 1545 - static inline void f2fs_quota_off_umount(struct super_block *sb) 1546 { 1547 } 1548 #endif 1549 1550 - static struct super_operations f2fs_sops = { 1551 .alloc_inode = f2fs_alloc_inode, 1552 .drop_inode = f2fs_drop_inode, 1553 .destroy_inode = f2fs_destroy_inode, ··· 1651 1652 static loff_t max_file_blocks(void) 1653 { 1654 - loff_t result = (DEF_ADDRS_PER_INODE - F2FS_INLINE_XATTR_ADDRS); 1655 loff_t leaf_count = ADDRS_PER_BLOCK; 1656 1657 /* two direct node blocks */ 1658 result += (leaf_count * 2); ··· 2277 sb->s_fs_info = sbi; 2278 sbi->raw_super = raw_super; 2279 2280 /* 2281 * The BLKZONED feature indicates that the drive was formatted with 2282 * zone alignment optimization. This is optional for host-aware ··· 2316 #ifdef CONFIG_QUOTA 2317 sb->dq_op = &f2fs_quota_operations; 2318 sb->s_qcop = &f2fs_quotactl_ops; 2319 - sb->s_quota_types = QTYPE_MASK_USR | QTYPE_MASK_GRP; 2320 #endif 2321 2322 sb->s_op = &f2fs_sops; ··· 2339 /* disallow all the data/node/meta page writes */ 2340 set_sbi_flag(sbi, SBI_POR_DOING); 2341 spin_lock_init(&sbi->stat_lock); 2342 2343 for (i = 0; i < NR_PAGE_TYPE; i++) { 2344 int n = (i == META) ? 1: NR_TEMP_TYPE; ··· 2462 if (err) 2463 goto free_nm; 2464 2465 - /* if there are nt orphan nodes free them */ 2466 - err = recover_orphan_inodes(sbi); 2467 - if (err) 2468 - goto free_node_inode; 2469 - 2470 /* read root inode and dentry */ 2471 root = f2fs_iget(sb, F2FS_ROOT_INO(sbi)); 2472 if (IS_ERR(root)) { ··· 2481 goto free_root_inode; 2482 } 2483 2484 - err = f2fs_init_sysfs(sbi); 2485 if (err) 2486 goto free_root_inode; 2487 2488 /* recover fsynced data */ 2489 if (!test_opt(sbi, DISABLE_ROLL_FORWARD)) { ··· 2499 if (bdev_read_only(sb->s_bdev) && 2500 !is_set_ckpt_flags(sbi, CP_UMOUNT_FLAG)) { 2501 err = -EROFS; 2502 - goto free_sysfs; 2503 } 2504 2505 if (need_fsck) ··· 2513 need_fsck = true; 2514 f2fs_msg(sb, KERN_ERR, 2515 "Cannot recover all fsync data errno=%d", err); 2516 - goto free_sysfs; 2517 } 2518 } else { 2519 err = recover_fsync_data(sbi, true); ··· 2537 /* After POR, we can run background GC thread.*/ 2538 err = start_gc_thread(sbi); 2539 if (err) 2540 - goto free_sysfs; 2541 } 2542 kfree(options); 2543 ··· 2555 f2fs_update_time(sbi, REQ_TIME); 2556 return 0; 2557 2558 - free_sysfs: 2559 f2fs_sync_inode_meta(sbi); 2560 - f2fs_exit_sysfs(sbi); 2561 free_root_inode: 2562 dput(sb->s_root); 2563 sb->s_root = NULL; ··· 2574 mutex_lock(&sbi->umount_mutex); 2575 release_ino_entry(sbi, true); 2576 f2fs_leave_shrinker(sbi); 2577 - /* 2578 - * Some dirty meta pages can be produced by recover_orphan_inodes() 2579 - * failed by EIO. Then, iput(node_inode) can trigger balance_fs_bg() 2580 - * followed by write_checkpoint() through f2fs_write_node_pages(), which 2581 - * falls into an infinite loop in sync_meta_pages(). 2582 - */ 2583 - truncate_inode_pages_final(META_MAPPING(sbi)); 2584 iput(sbi->node_inode); 2585 mutex_unlock(&sbi->umount_mutex); 2586 f2fs_destroy_stats(sbi); ··· 2593 for (i = 0; i < NR_PAGE_TYPE; i++) 2594 kfree(sbi->write_io[i]); 2595 destroy_percpu_info(sbi); 2596 kfree(options); 2597 free_sb_buf: 2598 kfree(raw_super); ··· 2680 err = create_extent_cache(); 2681 if (err) 2682 goto free_checkpoint_caches; 2683 - err = f2fs_register_sysfs(); 2684 if (err) 2685 goto free_extent_cache; 2686 err = register_shrinker(&f2fs_shrinker_info); ··· 2699 free_shrinker: 2700 unregister_shrinker(&f2fs_shrinker_info); 2701 free_sysfs: 2702 - f2fs_unregister_sysfs(); 2703 free_extent_cache: 2704 destroy_extent_cache(); 2705 free_checkpoint_caches: ··· 2719 f2fs_destroy_root_stats(); 2720 unregister_filesystem(&f2fs_fs_type); 2721 unregister_shrinker(&f2fs_shrinker_info); 2722 - f2fs_unregister_sysfs(); 2723 destroy_extent_cache(); 2724 destroy_checkpoint_caches(); 2725 destroy_segment_manager_caches();
··· 25 #include <linux/quotaops.h> 26 #include <linux/f2fs_fs.h> 27 #include <linux/sysfs.h> 28 + #include <linux/quota.h> 29 30 #include "f2fs.h" 31 #include "node.h" ··· 107 Opt_fault_injection, 108 Opt_lazytime, 109 Opt_nolazytime, 110 + Opt_quota, 111 + Opt_noquota, 112 Opt_usrquota, 113 Opt_grpquota, 114 + Opt_prjquota, 115 + Opt_usrjquota, 116 + Opt_grpjquota, 117 + Opt_prjjquota, 118 + Opt_offusrjquota, 119 + Opt_offgrpjquota, 120 + Opt_offprjjquota, 121 + Opt_jqfmt_vfsold, 122 + Opt_jqfmt_vfsv0, 123 + Opt_jqfmt_vfsv1, 124 Opt_err, 125 }; 126 ··· 144 {Opt_fault_injection, "fault_injection=%u"}, 145 {Opt_lazytime, "lazytime"}, 146 {Opt_nolazytime, "nolazytime"}, 147 + {Opt_quota, "quota"}, 148 + {Opt_noquota, "noquota"}, 149 {Opt_usrquota, "usrquota"}, 150 {Opt_grpquota, "grpquota"}, 151 + {Opt_prjquota, "prjquota"}, 152 + {Opt_usrjquota, "usrjquota=%s"}, 153 + {Opt_grpjquota, "grpjquota=%s"}, 154 + {Opt_prjjquota, "prjjquota=%s"}, 155 + {Opt_offusrjquota, "usrjquota="}, 156 + {Opt_offgrpjquota, "grpjquota="}, 157 + {Opt_offprjjquota, "prjjquota="}, 158 + {Opt_jqfmt_vfsold, "jqfmt=vfsold"}, 159 + {Opt_jqfmt_vfsv0, "jqfmt=vfsv0"}, 160 + {Opt_jqfmt_vfsv1, "jqfmt=vfsv1"}, 161 {Opt_err, NULL}, 162 }; 163 ··· 157 va_start(args, fmt); 158 vaf.fmt = fmt; 159 vaf.va = &args; 160 + printk_ratelimited("%sF2FS-fs (%s): %pV\n", level, sb->s_id, &vaf); 161 va_end(args); 162 } 163 ··· 168 inode_init_once(&fi->vfs_inode); 169 } 170 171 + #ifdef CONFIG_QUOTA 172 + static const char * const quotatypes[] = INITQFNAMES; 173 + #define QTYPE2NAME(t) (quotatypes[t]) 174 + static int f2fs_set_qf_name(struct super_block *sb, int qtype, 175 + substring_t *args) 176 + { 177 + struct f2fs_sb_info *sbi = F2FS_SB(sb); 178 + char *qname; 179 + int ret = -EINVAL; 180 + 181 + if (sb_any_quota_loaded(sb) && !sbi->s_qf_names[qtype]) { 182 + f2fs_msg(sb, KERN_ERR, 183 + "Cannot change journaled " 184 + "quota options when quota turned on"); 185 + return -EINVAL; 186 + } 187 + qname = match_strdup(args); 188 + if (!qname) { 189 + f2fs_msg(sb, KERN_ERR, 190 + "Not enough memory for storing quotafile name"); 191 + return -EINVAL; 192 + } 193 + if (sbi->s_qf_names[qtype]) { 194 + if (strcmp(sbi->s_qf_names[qtype], qname) == 0) 195 + ret = 0; 196 + else 197 + f2fs_msg(sb, KERN_ERR, 198 + "%s quota file already specified", 199 + QTYPE2NAME(qtype)); 200 + goto errout; 201 + } 202 + if (strchr(qname, '/')) { 203 + f2fs_msg(sb, KERN_ERR, 204 + "quotafile must be on filesystem root"); 205 + goto errout; 206 + } 207 + sbi->s_qf_names[qtype] = qname; 208 + set_opt(sbi, QUOTA); 209 + return 0; 210 + errout: 211 + kfree(qname); 212 + return ret; 213 + } 214 + 215 + static int f2fs_clear_qf_name(struct super_block *sb, int qtype) 216 + { 217 + struct f2fs_sb_info *sbi = F2FS_SB(sb); 218 + 219 + if (sb_any_quota_loaded(sb) && sbi->s_qf_names[qtype]) { 220 + f2fs_msg(sb, KERN_ERR, "Cannot change journaled quota options" 221 + " when quota turned on"); 222 + return -EINVAL; 223 + } 224 + kfree(sbi->s_qf_names[qtype]); 225 + sbi->s_qf_names[qtype] = NULL; 226 + return 0; 227 + } 228 + 229 + static int f2fs_check_quota_options(struct f2fs_sb_info *sbi) 230 + { 231 + /* 232 + * We do the test below only for project quotas. 'usrquota' and 233 + * 'grpquota' mount options are allowed even without quota feature 234 + * to support legacy quotas in quota files. 235 + */ 236 + if (test_opt(sbi, PRJQUOTA) && !f2fs_sb_has_project_quota(sbi->sb)) { 237 + f2fs_msg(sbi->sb, KERN_ERR, "Project quota feature not enabled. " 238 + "Cannot enable project quota enforcement."); 239 + return -1; 240 + } 241 + if (sbi->s_qf_names[USRQUOTA] || sbi->s_qf_names[GRPQUOTA] || 242 + sbi->s_qf_names[PRJQUOTA]) { 243 + if (test_opt(sbi, USRQUOTA) && sbi->s_qf_names[USRQUOTA]) 244 + clear_opt(sbi, USRQUOTA); 245 + 246 + if (test_opt(sbi, GRPQUOTA) && sbi->s_qf_names[GRPQUOTA]) 247 + clear_opt(sbi, GRPQUOTA); 248 + 249 + if (test_opt(sbi, PRJQUOTA) && sbi->s_qf_names[PRJQUOTA]) 250 + clear_opt(sbi, PRJQUOTA); 251 + 252 + if (test_opt(sbi, GRPQUOTA) || test_opt(sbi, USRQUOTA) || 253 + test_opt(sbi, PRJQUOTA)) { 254 + f2fs_msg(sbi->sb, KERN_ERR, "old and new quota " 255 + "format mixing"); 256 + return -1; 257 + } 258 + 259 + if (!sbi->s_jquota_fmt) { 260 + f2fs_msg(sbi->sb, KERN_ERR, "journaled quota format " 261 + "not specified"); 262 + return -1; 263 + } 264 + } 265 + return 0; 266 + } 267 + #endif 268 + 269 static int parse_options(struct super_block *sb, char *options) 270 { 271 struct f2fs_sb_info *sbi = F2FS_SB(sb); ··· 175 substring_t args[MAX_OPT_ARGS]; 176 char *p, *name; 177 int arg = 0; 178 + #ifdef CONFIG_QUOTA 179 + int ret; 180 + #endif 181 182 if (!options) 183 return 0; ··· 386 sb->s_flags &= ~MS_LAZYTIME; 387 break; 388 #ifdef CONFIG_QUOTA 389 + case Opt_quota: 390 case Opt_usrquota: 391 set_opt(sbi, USRQUOTA); 392 break; 393 case Opt_grpquota: 394 set_opt(sbi, GRPQUOTA); 395 break; 396 + case Opt_prjquota: 397 + set_opt(sbi, PRJQUOTA); 398 + break; 399 + case Opt_usrjquota: 400 + ret = f2fs_set_qf_name(sb, USRQUOTA, &args[0]); 401 + if (ret) 402 + return ret; 403 + break; 404 + case Opt_grpjquota: 405 + ret = f2fs_set_qf_name(sb, GRPQUOTA, &args[0]); 406 + if (ret) 407 + return ret; 408 + break; 409 + case Opt_prjjquota: 410 + ret = f2fs_set_qf_name(sb, PRJQUOTA, &args[0]); 411 + if (ret) 412 + return ret; 413 + break; 414 + case Opt_offusrjquota: 415 + ret = f2fs_clear_qf_name(sb, USRQUOTA); 416 + if (ret) 417 + return ret; 418 + break; 419 + case Opt_offgrpjquota: 420 + ret = f2fs_clear_qf_name(sb, GRPQUOTA); 421 + if (ret) 422 + return ret; 423 + break; 424 + case Opt_offprjjquota: 425 + ret = f2fs_clear_qf_name(sb, PRJQUOTA); 426 + if (ret) 427 + return ret; 428 + break; 429 + case Opt_jqfmt_vfsold: 430 + sbi->s_jquota_fmt = QFMT_VFS_OLD; 431 + break; 432 + case Opt_jqfmt_vfsv0: 433 + sbi->s_jquota_fmt = QFMT_VFS_V0; 434 + break; 435 + case Opt_jqfmt_vfsv1: 436 + sbi->s_jquota_fmt = QFMT_VFS_V1; 437 + break; 438 + case Opt_noquota: 439 + clear_opt(sbi, QUOTA); 440 + clear_opt(sbi, USRQUOTA); 441 + clear_opt(sbi, GRPQUOTA); 442 + clear_opt(sbi, PRJQUOTA); 443 + break; 444 #else 445 + case Opt_quota: 446 case Opt_usrquota: 447 case Opt_grpquota: 448 + case Opt_prjquota: 449 + case Opt_usrjquota: 450 + case Opt_grpjquota: 451 + case Opt_prjjquota: 452 + case Opt_offusrjquota: 453 + case Opt_offgrpjquota: 454 + case Opt_offprjjquota: 455 + case Opt_jqfmt_vfsold: 456 + case Opt_jqfmt_vfsv0: 457 + case Opt_jqfmt_vfsv1: 458 + case Opt_noquota: 459 f2fs_msg(sb, KERN_INFO, 460 "quota operations not supported"); 461 break; ··· 406 return -EINVAL; 407 } 408 } 409 + #ifdef CONFIG_QUOTA 410 + if (f2fs_check_quota_options(sbi)) 411 + return -EINVAL; 412 + #endif 413 414 if (F2FS_IO_SIZE_BITS(sbi) && !test_opt(sbi, LFS)) { 415 f2fs_msg(sb, KERN_ERR, ··· 439 init_rwsem(&fi->dio_rwsem[READ]); 440 init_rwsem(&fi->dio_rwsem[WRITE]); 441 init_rwsem(&fi->i_mmap_sem); 442 + init_rwsem(&fi->i_xattr_sem); 443 444 #ifdef CONFIG_QUOTA 445 memset(&fi->i_dquot, 0, sizeof(fi->i_dquot)); ··· 446 #endif 447 /* Will be used by directory only */ 448 fi->i_dir_level = F2FS_SB(sb)->dir_level; 449 + 450 return &fi->vfs_inode; 451 } 452 ··· 584 kfree(sbi->devs); 585 } 586 587 static void f2fs_put_super(struct super_block *sb) 588 { 589 struct f2fs_sb_info *sbi = F2FS_SB(sb); ··· 642 643 kfree(sbi->ckpt); 644 645 + f2fs_unregister_sysfs(sbi); 646 647 sb->s_fs_info = NULL; 648 if (sbi->s_chksum_driver) ··· 651 652 destroy_device_list(sbi); 653 mempool_destroy(sbi->write_io_dummy); 654 + #ifdef CONFIG_QUOTA 655 + for (i = 0; i < MAXQUOTAS; i++) 656 + kfree(sbi->s_qf_names[i]); 657 + #endif 658 destroy_percpu_info(sbi); 659 for (i = 0; i < NR_PAGE_TYPE; i++) 660 kfree(sbi->write_io[i]); ··· 663 int err = 0; 664 665 trace_f2fs_sync_fs(sb, sync); 666 + 667 + if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING))) 668 + return -EAGAIN; 669 670 if (sync) { 671 struct cp_control cpc; ··· 697 { 698 return 0; 699 } 700 + 701 + #ifdef CONFIG_QUOTA 702 + static int f2fs_statfs_project(struct super_block *sb, 703 + kprojid_t projid, struct kstatfs *buf) 704 + { 705 + struct kqid qid; 706 + struct dquot *dquot; 707 + u64 limit; 708 + u64 curblock; 709 + 710 + qid = make_kqid_projid(projid); 711 + dquot = dqget(sb, qid); 712 + if (IS_ERR(dquot)) 713 + return PTR_ERR(dquot); 714 + spin_lock(&dq_data_lock); 715 + 716 + limit = (dquot->dq_dqb.dqb_bsoftlimit ? 717 + dquot->dq_dqb.dqb_bsoftlimit : 718 + dquot->dq_dqb.dqb_bhardlimit) >> sb->s_blocksize_bits; 719 + if (limit && buf->f_blocks > limit) { 720 + curblock = dquot->dq_dqb.dqb_curspace >> sb->s_blocksize_bits; 721 + buf->f_blocks = limit; 722 + buf->f_bfree = buf->f_bavail = 723 + (buf->f_blocks > curblock) ? 724 + (buf->f_blocks - curblock) : 0; 725 + } 726 + 727 + limit = dquot->dq_dqb.dqb_isoftlimit ? 728 + dquot->dq_dqb.dqb_isoftlimit : 729 + dquot->dq_dqb.dqb_ihardlimit; 730 + if (limit && buf->f_files > limit) { 731 + buf->f_files = limit; 732 + buf->f_ffree = 733 + (buf->f_files > dquot->dq_dqb.dqb_curinodes) ? 734 + (buf->f_files - dquot->dq_dqb.dqb_curinodes) : 0; 735 + } 736 + 737 + spin_unlock(&dq_data_lock); 738 + dqput(dquot); 739 + return 0; 740 + } 741 + #endif 742 743 static int f2fs_statfs(struct dentry *dentry, struct kstatfs *buf) 744 { ··· 733 buf->f_fsid.val[0] = (u32)id; 734 buf->f_fsid.val[1] = (u32)(id >> 32); 735 736 + #ifdef CONFIG_QUOTA 737 + if (is_inode_flag_set(dentry->d_inode, FI_PROJ_INHERIT) && 738 + sb_has_quota_limits_enabled(sb, PRJQUOTA)) { 739 + f2fs_statfs_project(sb, F2FS_I(dentry->d_inode)->i_projid, buf); 740 + } 741 + #endif 742 return 0; 743 + } 744 + 745 + static inline void f2fs_show_quota_options(struct seq_file *seq, 746 + struct super_block *sb) 747 + { 748 + #ifdef CONFIG_QUOTA 749 + struct f2fs_sb_info *sbi = F2FS_SB(sb); 750 + 751 + if (sbi->s_jquota_fmt) { 752 + char *fmtname = ""; 753 + 754 + switch (sbi->s_jquota_fmt) { 755 + case QFMT_VFS_OLD: 756 + fmtname = "vfsold"; 757 + break; 758 + case QFMT_VFS_V0: 759 + fmtname = "vfsv0"; 760 + break; 761 + case QFMT_VFS_V1: 762 + fmtname = "vfsv1"; 763 + break; 764 + } 765 + seq_printf(seq, ",jqfmt=%s", fmtname); 766 + } 767 + 768 + if (sbi->s_qf_names[USRQUOTA]) 769 + seq_show_option(seq, "usrjquota", sbi->s_qf_names[USRQUOTA]); 770 + 771 + if (sbi->s_qf_names[GRPQUOTA]) 772 + seq_show_option(seq, "grpjquota", sbi->s_qf_names[GRPQUOTA]); 773 + 774 + if (sbi->s_qf_names[PRJQUOTA]) 775 + seq_show_option(seq, "prjjquota", sbi->s_qf_names[PRJQUOTA]); 776 + #endif 777 } 778 779 static int f2fs_show_options(struct seq_file *seq, struct dentry *root) ··· 809 sbi->fault_info.inject_rate); 810 #endif 811 #ifdef CONFIG_QUOTA 812 + if (test_opt(sbi, QUOTA)) 813 + seq_puts(seq, ",quota"); 814 if (test_opt(sbi, USRQUOTA)) 815 seq_puts(seq, ",usrquota"); 816 if (test_opt(sbi, GRPQUOTA)) 817 seq_puts(seq, ",grpquota"); 818 + if (test_opt(sbi, PRJQUOTA)) 819 + seq_puts(seq, ",prjquota"); 820 #endif 821 + f2fs_show_quota_options(seq, sbi->sb); 822 823 return 0; 824 } ··· 862 #ifdef CONFIG_F2FS_FAULT_INJECTION 863 struct f2fs_fault_info ffi = sbi->fault_info; 864 #endif 865 + #ifdef CONFIG_QUOTA 866 + int s_jquota_fmt; 867 + char *s_qf_names[MAXQUOTAS]; 868 + int i, j; 869 + #endif 870 871 /* 872 * Save the old mount options in case we ··· 870 org_mount_opt = sbi->mount_opt; 871 old_sb_flags = sb->s_flags; 872 active_logs = sbi->active_logs; 873 + 874 + #ifdef CONFIG_QUOTA 875 + s_jquota_fmt = sbi->s_jquota_fmt; 876 + for (i = 0; i < MAXQUOTAS; i++) { 877 + if (sbi->s_qf_names[i]) { 878 + s_qf_names[i] = kstrdup(sbi->s_qf_names[i], 879 + GFP_KERNEL); 880 + if (!s_qf_names[i]) { 881 + for (j = 0; j < i; j++) 882 + kfree(s_qf_names[j]); 883 + return -ENOMEM; 884 + } 885 + } else { 886 + s_qf_names[i] = NULL; 887 + } 888 + } 889 + #endif 890 891 /* recover superblocks we couldn't write due to previous RO mount */ 892 if (!(*flags & MS_RDONLY) && is_sbi_flag_set(sbi, SBI_NEED_SB_WRITE)) { ··· 952 goto restore_gc; 953 } 954 skip: 955 + #ifdef CONFIG_QUOTA 956 + /* Release old quota file names */ 957 + for (i = 0; i < MAXQUOTAS; i++) 958 + kfree(s_qf_names[i]); 959 + #endif 960 /* Update the POSIXACL Flag */ 961 sb->s_flags = (sb->s_flags & ~MS_POSIXACL) | 962 (test_opt(sbi, POSIX_ACL) ? MS_POSIXACL : 0); ··· 966 stop_gc_thread(sbi); 967 } 968 restore_opts: 969 + #ifdef CONFIG_QUOTA 970 + sbi->s_jquota_fmt = s_jquota_fmt; 971 + for (i = 0; i < MAXQUOTAS; i++) { 972 + kfree(sbi->s_qf_names[i]); 973 + sbi->s_qf_names[i] = s_qf_names[i]; 974 + } 975 + #endif 976 sbi->mount_opt = org_mount_opt; 977 sbi->active_logs = active_logs; 978 sb->s_flags = old_sb_flags; ··· 1065 } 1066 1067 if (len == towrite) 1068 + return 0; 1069 inode->i_version++; 1070 inode->i_mtime = inode->i_ctime = current_time(inode); 1071 f2fs_mark_inode_dirty_sync(inode, false); ··· 1080 static qsize_t *f2fs_get_reserved_space(struct inode *inode) 1081 { 1082 return &F2FS_I(inode)->i_reserved_quota; 1083 + } 1084 + 1085 + static int f2fs_quota_on_mount(struct f2fs_sb_info *sbi, int type) 1086 + { 1087 + return dquot_quota_on_mount(sbi->sb, sbi->s_qf_names[type], 1088 + sbi->s_jquota_fmt, type); 1089 + } 1090 + 1091 + void f2fs_enable_quota_files(struct f2fs_sb_info *sbi) 1092 + { 1093 + int i, ret; 1094 + 1095 + for (i = 0; i < MAXQUOTAS; i++) { 1096 + if (sbi->s_qf_names[i]) { 1097 + ret = f2fs_quota_on_mount(sbi, i); 1098 + if (ret < 0) 1099 + f2fs_msg(sbi->sb, KERN_ERR, 1100 + "Cannot turn on journaled " 1101 + "quota: error %d", ret); 1102 + } 1103 + } 1104 } 1105 1106 static int f2fs_quota_sync(struct super_block *sb, int type) ··· 1119 struct inode *inode; 1120 int err; 1121 1122 + err = f2fs_quota_sync(sb, type); 1123 if (err) 1124 return err; 1125 ··· 1147 if (!inode || !igrab(inode)) 1148 return dquot_quota_off(sb, type); 1149 1150 + f2fs_quota_sync(sb, type); 1151 1152 err = dquot_quota_off(sb, type); 1153 if (err) ··· 1163 return err; 1164 } 1165 1166 + void f2fs_quota_off_umount(struct super_block *sb) 1167 { 1168 int type; 1169 1170 for (type = 0; type < MAXQUOTAS; type++) 1171 f2fs_quota_off(sb, type); 1172 + } 1173 + 1174 + int f2fs_get_projid(struct inode *inode, kprojid_t *projid) 1175 + { 1176 + *projid = F2FS_I(inode)->i_projid; 1177 + return 0; 1178 } 1179 1180 static const struct dquot_operations f2fs_quota_operations = { ··· 1180 .write_info = dquot_commit_info, 1181 .alloc_dquot = dquot_alloc, 1182 .destroy_dquot = dquot_destroy, 1183 + .get_projid = f2fs_get_projid, 1184 .get_next_id = dquot_get_next_id, 1185 }; 1186 ··· 1194 .get_nextdqblk = dquot_get_next_dqblk, 1195 }; 1196 #else 1197 + void f2fs_quota_off_umount(struct super_block *sb) 1198 { 1199 } 1200 #endif 1201 1202 + static const struct super_operations f2fs_sops = { 1203 .alloc_inode = f2fs_alloc_inode, 1204 .drop_inode = f2fs_drop_inode, 1205 .destroy_inode = f2fs_destroy_inode, ··· 1303 1304 static loff_t max_file_blocks(void) 1305 { 1306 + loff_t result = 0; 1307 loff_t leaf_count = ADDRS_PER_BLOCK; 1308 + 1309 + /* 1310 + * note: previously, result is equal to (DEF_ADDRS_PER_INODE - 1311 + * F2FS_INLINE_XATTR_ADDRS), but now f2fs try to reserve more 1312 + * space in inode.i_addr, it will be more safe to reassign 1313 + * result as zero. 1314 + */ 1315 1316 /* two direct node blocks */ 1317 result += (leaf_count * 2); ··· 1922 sb->s_fs_info = sbi; 1923 sbi->raw_super = raw_super; 1924 1925 + /* precompute checksum seed for metadata */ 1926 + if (f2fs_sb_has_inode_chksum(sb)) 1927 + sbi->s_chksum_seed = f2fs_chksum(sbi, ~0, raw_super->uuid, 1928 + sizeof(raw_super->uuid)); 1929 + 1930 /* 1931 * The BLKZONED feature indicates that the drive was formatted with 1932 * zone alignment optimization. This is optional for host-aware ··· 1956 #ifdef CONFIG_QUOTA 1957 sb->dq_op = &f2fs_quota_operations; 1958 sb->s_qcop = &f2fs_quotactl_ops; 1959 + sb->s_quota_types = QTYPE_MASK_USR | QTYPE_MASK_GRP | QTYPE_MASK_PRJ; 1960 #endif 1961 1962 sb->s_op = &f2fs_sops; ··· 1979 /* disallow all the data/node/meta page writes */ 1980 set_sbi_flag(sbi, SBI_POR_DOING); 1981 spin_lock_init(&sbi->stat_lock); 1982 + 1983 + /* init iostat info */ 1984 + spin_lock_init(&sbi->iostat_lock); 1985 + sbi->iostat_enable = false; 1986 1987 for (i = 0; i < NR_PAGE_TYPE; i++) { 1988 int n = (i == META) ? 1: NR_TEMP_TYPE; ··· 2098 if (err) 2099 goto free_nm; 2100 2101 /* read root inode and dentry */ 2102 root = f2fs_iget(sb, F2FS_ROOT_INO(sbi)); 2103 if (IS_ERR(root)) { ··· 2122 goto free_root_inode; 2123 } 2124 2125 + err = f2fs_register_sysfs(sbi); 2126 if (err) 2127 goto free_root_inode; 2128 + 2129 + /* if there are nt orphan nodes free them */ 2130 + err = recover_orphan_inodes(sbi); 2131 + if (err) 2132 + goto free_sysfs; 2133 2134 /* recover fsynced data */ 2135 if (!test_opt(sbi, DISABLE_ROLL_FORWARD)) { ··· 2135 if (bdev_read_only(sb->s_bdev) && 2136 !is_set_ckpt_flags(sbi, CP_UMOUNT_FLAG)) { 2137 err = -EROFS; 2138 + goto free_meta; 2139 } 2140 2141 if (need_fsck) ··· 2149 need_fsck = true; 2150 f2fs_msg(sb, KERN_ERR, 2151 "Cannot recover all fsync data errno=%d", err); 2152 + goto free_meta; 2153 } 2154 } else { 2155 err = recover_fsync_data(sbi, true); ··· 2173 /* After POR, we can run background GC thread.*/ 2174 err = start_gc_thread(sbi); 2175 if (err) 2176 + goto free_meta; 2177 } 2178 kfree(options); 2179 ··· 2191 f2fs_update_time(sbi, REQ_TIME); 2192 return 0; 2193 2194 + free_meta: 2195 f2fs_sync_inode_meta(sbi); 2196 + /* 2197 + * Some dirty meta pages can be produced by recover_orphan_inodes() 2198 + * failed by EIO. Then, iput(node_inode) can trigger balance_fs_bg() 2199 + * followed by write_checkpoint() through f2fs_write_node_pages(), which 2200 + * falls into an infinite loop in sync_meta_pages(). 2201 + */ 2202 + truncate_inode_pages_final(META_MAPPING(sbi)); 2203 + free_sysfs: 2204 + f2fs_unregister_sysfs(sbi); 2205 free_root_inode: 2206 dput(sb->s_root); 2207 sb->s_root = NULL; ··· 2202 mutex_lock(&sbi->umount_mutex); 2203 release_ino_entry(sbi, true); 2204 f2fs_leave_shrinker(sbi); 2205 iput(sbi->node_inode); 2206 mutex_unlock(&sbi->umount_mutex); 2207 f2fs_destroy_stats(sbi); ··· 2228 for (i = 0; i < NR_PAGE_TYPE; i++) 2229 kfree(sbi->write_io[i]); 2230 destroy_percpu_info(sbi); 2231 + #ifdef CONFIG_QUOTA 2232 + for (i = 0; i < MAXQUOTAS; i++) 2233 + kfree(sbi->s_qf_names[i]); 2234 + #endif 2235 kfree(options); 2236 free_sb_buf: 2237 kfree(raw_super); ··· 2311 err = create_extent_cache(); 2312 if (err) 2313 goto free_checkpoint_caches; 2314 + err = f2fs_init_sysfs(); 2315 if (err) 2316 goto free_extent_cache; 2317 err = register_shrinker(&f2fs_shrinker_info); ··· 2330 free_shrinker: 2331 unregister_shrinker(&f2fs_shrinker_info); 2332 free_sysfs: 2333 + f2fs_exit_sysfs(); 2334 free_extent_cache: 2335 destroy_extent_cache(); 2336 free_checkpoint_caches: ··· 2350 f2fs_destroy_root_stats(); 2351 unregister_filesystem(&f2fs_fs_type); 2352 unregister_shrinker(&f2fs_shrinker_info); 2353 + f2fs_exit_sysfs(); 2354 destroy_extent_cache(); 2355 destroy_checkpoint_caches(); 2356 destroy_segment_manager_caches();
+221 -30
fs/f2fs/sysfs.c
··· 18 #include "gc.h" 19 20 static struct proc_dir_entry *f2fs_proc_root; 21 - static struct kset *f2fs_kset; 22 23 /* Sysfs support for f2fs */ 24 enum { ··· 40 const char *, size_t); 41 int struct_type; 42 int offset; 43 }; 44 45 static unsigned char *__struct_ptr(struct f2fs_sb_info *sbi, int struct_type) ··· 74 return snprintf(buf, PAGE_SIZE, "%llu\n", 75 (unsigned long long)(sbi->kbytes_written + 76 BD_PART_WRITTEN(sbi))); 77 } 78 79 static ssize_t f2fs_sbi_show(struct f2fs_attr *a, ··· 152 spin_unlock(&sbi->stat_lock); 153 return count; 154 } 155 *ui = t; 156 return count; 157 } 158 ··· 215 complete(&sbi->s_kobj_unregister); 216 } 217 218 #define F2FS_ATTR_OFFSET(_struct_type, _name, _mode, _show, _store, _offset) \ 219 static struct f2fs_attr f2fs_attr_##_name = { \ 220 .attr = {.name = __stringify(_name), .mode = _mode }, \ ··· 256 #define F2FS_GENERAL_RO_ATTR(name) \ 257 static struct f2fs_attr f2fs_attr_##name = __ATTR(name, 0444, name##_show, NULL) 258 259 F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_min_sleep_time, min_sleep_time); 260 F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_max_sleep_time, max_sleep_time); 261 F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_no_gc_sleep_time, no_gc_sleep_time); 262 F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_idle, gc_idle); 263 F2FS_RW_ATTR(SM_INFO, f2fs_sm_info, reclaim_segments, rec_prefree_segments); 264 F2FS_RW_ATTR(DCC_INFO, discard_cmd_control, max_small_discards, max_discards); 265 F2FS_RW_ATTR(RESERVED_BLOCKS, f2fs_sb_info, reserved_blocks, reserved_blocks); 266 F2FS_RW_ATTR(SM_INFO, f2fs_sm_info, batched_trim_sections, trim_sections); 267 F2FS_RW_ATTR(SM_INFO, f2fs_sm_info, ipu_policy, ipu_policy); ··· 286 F2FS_RW_ATTR(F2FS_SBI, f2fs_sb_info, dir_level, dir_level); 287 F2FS_RW_ATTR(F2FS_SBI, f2fs_sb_info, cp_interval, interval_time[CP_TIME]); 288 F2FS_RW_ATTR(F2FS_SBI, f2fs_sb_info, idle_interval, interval_time[REQ_TIME]); 289 #ifdef CONFIG_F2FS_FAULT_INJECTION 290 F2FS_RW_ATTR(FAULT_INFO_RATE, f2fs_fault_info, inject_rate, inject_rate); 291 F2FS_RW_ATTR(FAULT_INFO_TYPE, f2fs_fault_info, inject_type, inject_type); 292 #endif 293 F2FS_GENERAL_RO_ATTR(lifetime_write_kbytes); 294 295 #define ATTR_LIST(name) (&f2fs_attr_##name.attr) 296 static struct attribute *f2fs_attrs[] = { 297 ATTR_LIST(gc_min_sleep_time), 298 ATTR_LIST(gc_max_sleep_time), 299 ATTR_LIST(gc_no_gc_sleep_time), 300 ATTR_LIST(gc_idle), 301 ATTR_LIST(reclaim_segments), 302 ATTR_LIST(max_small_discards), 303 ATTR_LIST(batched_trim_sections), 304 ATTR_LIST(ipu_policy), 305 ATTR_LIST(min_ipu_util), ··· 328 ATTR_LIST(dirty_nats_ratio), 329 ATTR_LIST(cp_interval), 330 ATTR_LIST(idle_interval), 331 #ifdef CONFIG_F2FS_FAULT_INJECTION 332 ATTR_LIST(inject_rate), 333 ATTR_LIST(inject_type), 334 #endif 335 ATTR_LIST(lifetime_write_kbytes), 336 ATTR_LIST(reserved_blocks), 337 NULL, 338 }; 339 ··· 358 .store = f2fs_attr_store, 359 }; 360 361 - static struct kobj_type f2fs_ktype = { 362 .default_attrs = f2fs_attrs, 363 .sysfs_ops = &f2fs_attr_ops, 364 .release = f2fs_sb_release, 365 }; 366 367 static int segment_info_seq_show(struct seq_file *seq, void *offset) ··· 432 return 0; 433 } 434 435 #define F2FS_PROC_FILE_DEF(_name) \ 436 static int _name##_open_fs(struct inode *inode, struct file *file) \ 437 { \ ··· 489 490 F2FS_PROC_FILE_DEF(segment_info); 491 F2FS_PROC_FILE_DEF(segment_bits); 492 493 - int __init f2fs_register_sysfs(void) 494 { 495 - f2fs_proc_root = proc_mkdir("fs/f2fs", NULL); 496 497 - f2fs_kset = kset_create_and_add("f2fs", NULL, fs_kobj); 498 - if (!f2fs_kset) 499 - return -ENOMEM; 500 - return 0; 501 } 502 503 - void f2fs_unregister_sysfs(void) 504 { 505 - kset_unregister(f2fs_kset); 506 remove_proc_entry("fs/f2fs", NULL); 507 } 508 509 - int f2fs_init_sysfs(struct f2fs_sb_info *sbi) 510 { 511 struct super_block *sb = sbi->sb; 512 int err; 513 514 if (f2fs_proc_root) 515 sbi->s_proc = proc_mkdir(sb->s_id, f2fs_proc_root); ··· 538 &f2fs_seq_segment_info_fops, sb); 539 proc_create_data("segment_bits", S_IRUGO, sbi->s_proc, 540 &f2fs_seq_segment_bits_fops, sb); 541 } 542 - 543 - sbi->s_kobj.kset = f2fs_kset; 544 - init_completion(&sbi->s_kobj_unregister); 545 - err = kobject_init_and_add(&sbi->s_kobj, &f2fs_ktype, NULL, 546 - "%s", sb->s_id); 547 - if (err) 548 - goto err_out; 549 return 0; 550 - err_out: 551 - if (sbi->s_proc) { 552 - remove_proc_entry("segment_info", sbi->s_proc); 553 - remove_proc_entry("segment_bits", sbi->s_proc); 554 - remove_proc_entry(sb->s_id, f2fs_proc_root); 555 - } 556 - return err; 557 } 558 559 - void f2fs_exit_sysfs(struct f2fs_sb_info *sbi) 560 { 561 - kobject_del(&sbi->s_kobj); 562 - kobject_put(&sbi->s_kobj); 563 - wait_for_completion(&sbi->s_kobj_unregister); 564 - 565 if (sbi->s_proc) { 566 remove_proc_entry("segment_info", sbi->s_proc); 567 remove_proc_entry("segment_bits", sbi->s_proc); 568 remove_proc_entry(sbi->sb->s_id, f2fs_proc_root); 569 } 570 }
··· 18 #include "gc.h" 19 20 static struct proc_dir_entry *f2fs_proc_root; 21 22 /* Sysfs support for f2fs */ 23 enum { ··· 41 const char *, size_t); 42 int struct_type; 43 int offset; 44 + int id; 45 }; 46 47 static unsigned char *__struct_ptr(struct f2fs_sb_info *sbi, int struct_type) ··· 74 return snprintf(buf, PAGE_SIZE, "%llu\n", 75 (unsigned long long)(sbi->kbytes_written + 76 BD_PART_WRITTEN(sbi))); 77 + } 78 + 79 + static ssize_t features_show(struct f2fs_attr *a, 80 + struct f2fs_sb_info *sbi, char *buf) 81 + { 82 + struct super_block *sb = sbi->sb; 83 + int len = 0; 84 + 85 + if (!sb->s_bdev->bd_part) 86 + return snprintf(buf, PAGE_SIZE, "0\n"); 87 + 88 + if (f2fs_sb_has_crypto(sb)) 89 + len += snprintf(buf, PAGE_SIZE - len, "%s", 90 + "encryption"); 91 + if (f2fs_sb_mounted_blkzoned(sb)) 92 + len += snprintf(buf + len, PAGE_SIZE - len, "%s%s", 93 + len ? ", " : "", "blkzoned"); 94 + if (f2fs_sb_has_extra_attr(sb)) 95 + len += snprintf(buf + len, PAGE_SIZE - len, "%s%s", 96 + len ? ", " : "", "extra_attr"); 97 + if (f2fs_sb_has_project_quota(sb)) 98 + len += snprintf(buf + len, PAGE_SIZE - len, "%s%s", 99 + len ? ", " : "", "projquota"); 100 + if (f2fs_sb_has_inode_chksum(sb)) 101 + len += snprintf(buf + len, PAGE_SIZE - len, "%s%s", 102 + len ? ", " : "", "inode_checksum"); 103 + len += snprintf(buf + len, PAGE_SIZE - len, "\n"); 104 + return len; 105 } 106 107 static ssize_t f2fs_sbi_show(struct f2fs_attr *a, ··· 124 spin_unlock(&sbi->stat_lock); 125 return count; 126 } 127 + 128 + if (!strcmp(a->attr.name, "discard_granularity")) { 129 + struct discard_cmd_control *dcc = SM_I(sbi)->dcc_info; 130 + int i; 131 + 132 + if (t == 0 || t > MAX_PLIST_NUM) 133 + return -EINVAL; 134 + if (t == *ui) 135 + return count; 136 + 137 + mutex_lock(&dcc->cmd_lock); 138 + for (i = 0; i < MAX_PLIST_NUM; i++) { 139 + if (i >= t - 1) 140 + dcc->pend_list_tag[i] |= P_ACTIVE; 141 + else 142 + dcc->pend_list_tag[i] &= (~P_ACTIVE); 143 + } 144 + mutex_unlock(&dcc->cmd_lock); 145 + 146 + *ui = t; 147 + return count; 148 + } 149 + 150 *ui = t; 151 + 152 + if (!strcmp(a->attr.name, "iostat_enable") && *ui == 0) 153 + f2fs_reset_iostat(sbi); 154 + if (!strcmp(a->attr.name, "gc_urgent") && t == 1 && sbi->gc_thread) { 155 + sbi->gc_thread->gc_wake = 1; 156 + wake_up_interruptible_all(&sbi->gc_thread->gc_wait_queue_head); 157 + wake_up_discard_thread(sbi, true); 158 + } 159 + 160 return count; 161 } 162 ··· 155 complete(&sbi->s_kobj_unregister); 156 } 157 158 + enum feat_id { 159 + FEAT_CRYPTO = 0, 160 + FEAT_BLKZONED, 161 + FEAT_ATOMIC_WRITE, 162 + FEAT_EXTRA_ATTR, 163 + FEAT_PROJECT_QUOTA, 164 + FEAT_INODE_CHECKSUM, 165 + }; 166 + 167 + static ssize_t f2fs_feature_show(struct f2fs_attr *a, 168 + struct f2fs_sb_info *sbi, char *buf) 169 + { 170 + switch (a->id) { 171 + case FEAT_CRYPTO: 172 + case FEAT_BLKZONED: 173 + case FEAT_ATOMIC_WRITE: 174 + case FEAT_EXTRA_ATTR: 175 + case FEAT_PROJECT_QUOTA: 176 + case FEAT_INODE_CHECKSUM: 177 + return snprintf(buf, PAGE_SIZE, "supported\n"); 178 + } 179 + return 0; 180 + } 181 + 182 #define F2FS_ATTR_OFFSET(_struct_type, _name, _mode, _show, _store, _offset) \ 183 static struct f2fs_attr f2fs_attr_##_name = { \ 184 .attr = {.name = __stringify(_name), .mode = _mode }, \ ··· 172 #define F2FS_GENERAL_RO_ATTR(name) \ 173 static struct f2fs_attr f2fs_attr_##name = __ATTR(name, 0444, name##_show, NULL) 174 175 + #define F2FS_FEATURE_RO_ATTR(_name, _id) \ 176 + static struct f2fs_attr f2fs_attr_##_name = { \ 177 + .attr = {.name = __stringify(_name), .mode = 0444 }, \ 178 + .show = f2fs_feature_show, \ 179 + .id = _id, \ 180 + } 181 + 182 + F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_urgent_sleep_time, 183 + urgent_sleep_time); 184 F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_min_sleep_time, min_sleep_time); 185 F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_max_sleep_time, max_sleep_time); 186 F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_no_gc_sleep_time, no_gc_sleep_time); 187 F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_idle, gc_idle); 188 + F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_urgent, gc_urgent); 189 F2FS_RW_ATTR(SM_INFO, f2fs_sm_info, reclaim_segments, rec_prefree_segments); 190 F2FS_RW_ATTR(DCC_INFO, discard_cmd_control, max_small_discards, max_discards); 191 + F2FS_RW_ATTR(DCC_INFO, discard_cmd_control, discard_granularity, discard_granularity); 192 F2FS_RW_ATTR(RESERVED_BLOCKS, f2fs_sb_info, reserved_blocks, reserved_blocks); 193 F2FS_RW_ATTR(SM_INFO, f2fs_sm_info, batched_trim_sections, trim_sections); 194 F2FS_RW_ATTR(SM_INFO, f2fs_sm_info, ipu_policy, ipu_policy); ··· 191 F2FS_RW_ATTR(F2FS_SBI, f2fs_sb_info, dir_level, dir_level); 192 F2FS_RW_ATTR(F2FS_SBI, f2fs_sb_info, cp_interval, interval_time[CP_TIME]); 193 F2FS_RW_ATTR(F2FS_SBI, f2fs_sb_info, idle_interval, interval_time[REQ_TIME]); 194 + F2FS_RW_ATTR(F2FS_SBI, f2fs_sb_info, iostat_enable, iostat_enable); 195 #ifdef CONFIG_F2FS_FAULT_INJECTION 196 F2FS_RW_ATTR(FAULT_INFO_RATE, f2fs_fault_info, inject_rate, inject_rate); 197 F2FS_RW_ATTR(FAULT_INFO_TYPE, f2fs_fault_info, inject_type, inject_type); 198 #endif 199 F2FS_GENERAL_RO_ATTR(lifetime_write_kbytes); 200 + F2FS_GENERAL_RO_ATTR(features); 201 + 202 + #ifdef CONFIG_F2FS_FS_ENCRYPTION 203 + F2FS_FEATURE_RO_ATTR(encryption, FEAT_CRYPTO); 204 + #endif 205 + #ifdef CONFIG_BLK_DEV_ZONED 206 + F2FS_FEATURE_RO_ATTR(block_zoned, FEAT_BLKZONED); 207 + #endif 208 + F2FS_FEATURE_RO_ATTR(atomic_write, FEAT_ATOMIC_WRITE); 209 + F2FS_FEATURE_RO_ATTR(extra_attr, FEAT_EXTRA_ATTR); 210 + F2FS_FEATURE_RO_ATTR(project_quota, FEAT_PROJECT_QUOTA); 211 + F2FS_FEATURE_RO_ATTR(inode_checksum, FEAT_INODE_CHECKSUM); 212 213 #define ATTR_LIST(name) (&f2fs_attr_##name.attr) 214 static struct attribute *f2fs_attrs[] = { 215 + ATTR_LIST(gc_urgent_sleep_time), 216 ATTR_LIST(gc_min_sleep_time), 217 ATTR_LIST(gc_max_sleep_time), 218 ATTR_LIST(gc_no_gc_sleep_time), 219 ATTR_LIST(gc_idle), 220 + ATTR_LIST(gc_urgent), 221 ATTR_LIST(reclaim_segments), 222 ATTR_LIST(max_small_discards), 223 + ATTR_LIST(discard_granularity), 224 ATTR_LIST(batched_trim_sections), 225 ATTR_LIST(ipu_policy), 226 ATTR_LIST(min_ipu_util), ··· 217 ATTR_LIST(dirty_nats_ratio), 218 ATTR_LIST(cp_interval), 219 ATTR_LIST(idle_interval), 220 + ATTR_LIST(iostat_enable), 221 #ifdef CONFIG_F2FS_FAULT_INJECTION 222 ATTR_LIST(inject_rate), 223 ATTR_LIST(inject_type), 224 #endif 225 ATTR_LIST(lifetime_write_kbytes), 226 + ATTR_LIST(features), 227 ATTR_LIST(reserved_blocks), 228 + NULL, 229 + }; 230 + 231 + static struct attribute *f2fs_feat_attrs[] = { 232 + #ifdef CONFIG_F2FS_FS_ENCRYPTION 233 + ATTR_LIST(encryption), 234 + #endif 235 + #ifdef CONFIG_BLK_DEV_ZONED 236 + ATTR_LIST(block_zoned), 237 + #endif 238 + ATTR_LIST(atomic_write), 239 + ATTR_LIST(extra_attr), 240 + ATTR_LIST(project_quota), 241 + ATTR_LIST(inode_checksum), 242 NULL, 243 }; 244 ··· 231 .store = f2fs_attr_store, 232 }; 233 234 + static struct kobj_type f2fs_sb_ktype = { 235 .default_attrs = f2fs_attrs, 236 .sysfs_ops = &f2fs_attr_ops, 237 .release = f2fs_sb_release, 238 + }; 239 + 240 + static struct kobj_type f2fs_ktype = { 241 + .sysfs_ops = &f2fs_attr_ops, 242 + }; 243 + 244 + static struct kset f2fs_kset = { 245 + .kobj = {.ktype = &f2fs_ktype}, 246 + }; 247 + 248 + static struct kobj_type f2fs_feat_ktype = { 249 + .default_attrs = f2fs_feat_attrs, 250 + .sysfs_ops = &f2fs_attr_ops, 251 + }; 252 + 253 + static struct kobject f2fs_feat = { 254 + .kset = &f2fs_kset, 255 }; 256 257 static int segment_info_seq_show(struct seq_file *seq, void *offset) ··· 288 return 0; 289 } 290 291 + static int iostat_info_seq_show(struct seq_file *seq, void *offset) 292 + { 293 + struct super_block *sb = seq->private; 294 + struct f2fs_sb_info *sbi = F2FS_SB(sb); 295 + time64_t now = ktime_get_real_seconds(); 296 + 297 + if (!sbi->iostat_enable) 298 + return 0; 299 + 300 + seq_printf(seq, "time: %-16llu\n", now); 301 + 302 + /* print app IOs */ 303 + seq_printf(seq, "app buffered: %-16llu\n", 304 + sbi->write_iostat[APP_BUFFERED_IO]); 305 + seq_printf(seq, "app direct: %-16llu\n", 306 + sbi->write_iostat[APP_DIRECT_IO]); 307 + seq_printf(seq, "app mapped: %-16llu\n", 308 + sbi->write_iostat[APP_MAPPED_IO]); 309 + 310 + /* print fs IOs */ 311 + seq_printf(seq, "fs data: %-16llu\n", 312 + sbi->write_iostat[FS_DATA_IO]); 313 + seq_printf(seq, "fs node: %-16llu\n", 314 + sbi->write_iostat[FS_NODE_IO]); 315 + seq_printf(seq, "fs meta: %-16llu\n", 316 + sbi->write_iostat[FS_META_IO]); 317 + seq_printf(seq, "fs gc data: %-16llu\n", 318 + sbi->write_iostat[FS_GC_DATA_IO]); 319 + seq_printf(seq, "fs gc node: %-16llu\n", 320 + sbi->write_iostat[FS_GC_NODE_IO]); 321 + seq_printf(seq, "fs cp data: %-16llu\n", 322 + sbi->write_iostat[FS_CP_DATA_IO]); 323 + seq_printf(seq, "fs cp node: %-16llu\n", 324 + sbi->write_iostat[FS_CP_NODE_IO]); 325 + seq_printf(seq, "fs cp meta: %-16llu\n", 326 + sbi->write_iostat[FS_CP_META_IO]); 327 + seq_printf(seq, "fs discard: %-16llu\n", 328 + sbi->write_iostat[FS_DISCARD]); 329 + 330 + return 0; 331 + } 332 + 333 #define F2FS_PROC_FILE_DEF(_name) \ 334 static int _name##_open_fs(struct inode *inode, struct file *file) \ 335 { \ ··· 303 304 F2FS_PROC_FILE_DEF(segment_info); 305 F2FS_PROC_FILE_DEF(segment_bits); 306 + F2FS_PROC_FILE_DEF(iostat_info); 307 308 + int __init f2fs_init_sysfs(void) 309 { 310 + int ret; 311 312 + kobject_set_name(&f2fs_kset.kobj, "f2fs"); 313 + f2fs_kset.kobj.parent = fs_kobj; 314 + ret = kset_register(&f2fs_kset); 315 + if (ret) 316 + return ret; 317 + 318 + ret = kobject_init_and_add(&f2fs_feat, &f2fs_feat_ktype, 319 + NULL, "features"); 320 + if (ret) 321 + kset_unregister(&f2fs_kset); 322 + else 323 + f2fs_proc_root = proc_mkdir("fs/f2fs", NULL); 324 + return ret; 325 } 326 327 + void f2fs_exit_sysfs(void) 328 { 329 + kobject_put(&f2fs_feat); 330 + kset_unregister(&f2fs_kset); 331 remove_proc_entry("fs/f2fs", NULL); 332 + f2fs_proc_root = NULL; 333 } 334 335 + int f2fs_register_sysfs(struct f2fs_sb_info *sbi) 336 { 337 struct super_block *sb = sbi->sb; 338 int err; 339 + 340 + sbi->s_kobj.kset = &f2fs_kset; 341 + init_completion(&sbi->s_kobj_unregister); 342 + err = kobject_init_and_add(&sbi->s_kobj, &f2fs_sb_ktype, NULL, 343 + "%s", sb->s_id); 344 + if (err) 345 + return err; 346 347 if (f2fs_proc_root) 348 sbi->s_proc = proc_mkdir(sb->s_id, f2fs_proc_root); ··· 333 &f2fs_seq_segment_info_fops, sb); 334 proc_create_data("segment_bits", S_IRUGO, sbi->s_proc, 335 &f2fs_seq_segment_bits_fops, sb); 336 + proc_create_data("iostat_info", S_IRUGO, sbi->s_proc, 337 + &f2fs_seq_iostat_info_fops, sb); 338 } 339 return 0; 340 } 341 342 + void f2fs_unregister_sysfs(struct f2fs_sb_info *sbi) 343 { 344 if (sbi->s_proc) { 345 + remove_proc_entry("iostat_info", sbi->s_proc); 346 remove_proc_entry("segment_info", sbi->s_proc); 347 remove_proc_entry("segment_bits", sbi->s_proc); 348 remove_proc_entry(sbi->sb->s_id, f2fs_proc_root); 349 } 350 + kobject_del(&sbi->s_kobj); 351 }
+7 -1
fs/f2fs/xattr.c
··· 442 } else { 443 struct dnode_of_data dn; 444 set_new_dnode(&dn, inode, NULL, NULL, new_nid); 445 - xpage = new_node_page(&dn, XATTR_NODE_OFFSET, ipage); 446 if (IS_ERR(xpage)) { 447 alloc_nid_failed(sbi, new_nid); 448 return PTR_ERR(xpage); ··· 473 if (len > F2FS_NAME_LEN) 474 return -ERANGE; 475 476 error = lookup_all_xattrs(inode, ipage, index, len, name, 477 &entry, &base_addr); 478 if (error) 479 return error; 480 ··· 505 int error = 0; 506 size_t rest = buffer_size; 507 508 error = read_all_xattrs(inode, NULL, &base_addr); 509 if (error) 510 return error; 511 ··· 690 f2fs_lock_op(sbi); 691 /* protect xattr_ver */ 692 down_write(&F2FS_I(inode)->i_sem); 693 err = __f2fs_setxattr(inode, index, name, value, size, ipage, flags); 694 up_write(&F2FS_I(inode)->i_sem); 695 f2fs_unlock_op(sbi); 696
··· 442 } else { 443 struct dnode_of_data dn; 444 set_new_dnode(&dn, inode, NULL, NULL, new_nid); 445 + xpage = new_node_page(&dn, XATTR_NODE_OFFSET); 446 if (IS_ERR(xpage)) { 447 alloc_nid_failed(sbi, new_nid); 448 return PTR_ERR(xpage); ··· 473 if (len > F2FS_NAME_LEN) 474 return -ERANGE; 475 476 + down_read(&F2FS_I(inode)->i_xattr_sem); 477 error = lookup_all_xattrs(inode, ipage, index, len, name, 478 &entry, &base_addr); 479 + up_read(&F2FS_I(inode)->i_xattr_sem); 480 if (error) 481 return error; 482 ··· 503 int error = 0; 504 size_t rest = buffer_size; 505 506 + down_read(&F2FS_I(inode)->i_xattr_sem); 507 error = read_all_xattrs(inode, NULL, &base_addr); 508 + up_read(&F2FS_I(inode)->i_xattr_sem); 509 if (error) 510 return error; 511 ··· 686 f2fs_lock_op(sbi); 687 /* protect xattr_ver */ 688 down_write(&F2FS_I(inode)->i_sem); 689 + down_write(&F2FS_I(inode)->i_xattr_sem); 690 err = __f2fs_setxattr(inode, index, name, value, size, ipage, flags); 691 + up_write(&F2FS_I(inode)->i_xattr_sem); 692 up_write(&F2FS_I(inode)->i_sem); 693 f2fs_unlock_op(sbi); 694
+16 -24
include/linux/f2fs_fs.h
··· 186 #define F2FS_NAME_LEN 255 187 #define F2FS_INLINE_XATTR_ADDRS 50 /* 200 bytes for inline xattrs */ 188 #define DEF_ADDRS_PER_INODE 923 /* Address Pointers in an Inode */ 189 #define DEF_NIDS_PER_INODE 5 /* Node IDs in an Inode */ 190 #define ADDRS_PER_INODE(inode) addrs_per_inode(inode) 191 #define ADDRS_PER_BLOCK 1018 /* Address Pointers in a Direct Block */ ··· 207 #define F2FS_INLINE_DENTRY 0x04 /* file inline dentry flag */ 208 #define F2FS_DATA_EXIST 0x08 /* file inline data exist flag */ 209 #define F2FS_INLINE_DOTS 0x10 /* file having implicit dot dentries */ 210 - 211 - #define MAX_INLINE_DATA (sizeof(__le32) * (DEF_ADDRS_PER_INODE - \ 212 - F2FS_INLINE_XATTR_ADDRS - 1)) 213 214 struct f2fs_inode { 215 __le16 i_mode; /* file mode */ ··· 235 236 struct f2fs_extent i_ext; /* caching a largest extent */ 237 238 - __le32 i_addr[DEF_ADDRS_PER_INODE]; /* Pointers to data blocks */ 239 - 240 __le32 i_nid[DEF_NIDS_PER_INODE]; /* direct(2), indirect(2), 241 double_indirect(1) node id */ 242 } __packed; ··· 473 #define MAX_DIR_BUCKETS (1 << ((MAX_DIR_HASH_DEPTH / 2) - 1)) 474 475 /* 476 - * space utilization of regular dentry and inline dentry 477 * regular dentry inline dentry 478 * bitmap 1 * 27 = 27 1 * 23 = 23 479 * reserved 1 * 3 = 3 1 * 7 = 7 ··· 509 __u8 filename[NR_DENTRY_IN_BLOCK][F2FS_SLOT_LEN]; 510 } __packed; 511 512 - /* for inline dir */ 513 - #define NR_INLINE_DENTRY (MAX_INLINE_DATA * BITS_PER_BYTE / \ 514 - ((SIZE_OF_DIR_ENTRY + F2FS_SLOT_LEN) * \ 515 - BITS_PER_BYTE + 1)) 516 - #define INLINE_DENTRY_BITMAP_SIZE ((NR_INLINE_DENTRY + \ 517 - BITS_PER_BYTE - 1) / BITS_PER_BYTE) 518 - #define INLINE_RESERVED_SIZE (MAX_INLINE_DATA - \ 519 - ((SIZE_OF_DIR_ENTRY + F2FS_SLOT_LEN) * \ 520 - NR_INLINE_DENTRY + INLINE_DENTRY_BITMAP_SIZE)) 521 - 522 - /* inline directory entry structure */ 523 - struct f2fs_inline_dentry { 524 - __u8 dentry_bitmap[INLINE_DENTRY_BITMAP_SIZE]; 525 - __u8 reserved[INLINE_RESERVED_SIZE]; 526 - struct f2fs_dir_entry dentry[NR_INLINE_DENTRY]; 527 - __u8 filename[NR_INLINE_DENTRY][F2FS_SLOT_LEN]; 528 - } __packed; 529 - 530 /* file types used in inode_info->flags */ 531 enum { 532 F2FS_FT_UNKNOWN, ··· 523 }; 524 525 #define S_SHIFT 12 526 527 #endif /* _LINUX_F2FS_FS_H */
··· 186 #define F2FS_NAME_LEN 255 187 #define F2FS_INLINE_XATTR_ADDRS 50 /* 200 bytes for inline xattrs */ 188 #define DEF_ADDRS_PER_INODE 923 /* Address Pointers in an Inode */ 189 + #define CUR_ADDRS_PER_INODE(inode) (DEF_ADDRS_PER_INODE - \ 190 + get_extra_isize(inode)) 191 #define DEF_NIDS_PER_INODE 5 /* Node IDs in an Inode */ 192 #define ADDRS_PER_INODE(inode) addrs_per_inode(inode) 193 #define ADDRS_PER_BLOCK 1018 /* Address Pointers in a Direct Block */ ··· 205 #define F2FS_INLINE_DENTRY 0x04 /* file inline dentry flag */ 206 #define F2FS_DATA_EXIST 0x08 /* file inline data exist flag */ 207 #define F2FS_INLINE_DOTS 0x10 /* file having implicit dot dentries */ 208 + #define F2FS_EXTRA_ATTR 0x20 /* file having extra attribute */ 209 210 struct f2fs_inode { 211 __le16 i_mode; /* file mode */ ··· 235 236 struct f2fs_extent i_ext; /* caching a largest extent */ 237 238 + union { 239 + struct { 240 + __le16 i_extra_isize; /* extra inode attribute size */ 241 + __le16 i_padding; /* padding */ 242 + __le32 i_projid; /* project id */ 243 + __le32 i_inode_checksum;/* inode meta checksum */ 244 + __le32 i_extra_end[0]; /* for attribute size calculation */ 245 + }; 246 + __le32 i_addr[DEF_ADDRS_PER_INODE]; /* Pointers to data blocks */ 247 + }; 248 __le32 i_nid[DEF_NIDS_PER_INODE]; /* direct(2), indirect(2), 249 double_indirect(1) node id */ 250 } __packed; ··· 465 #define MAX_DIR_BUCKETS (1 << ((MAX_DIR_HASH_DEPTH / 2) - 1)) 466 467 /* 468 + * space utilization of regular dentry and inline dentry (w/o extra reservation) 469 * regular dentry inline dentry 470 * bitmap 1 * 27 = 27 1 * 23 = 23 471 * reserved 1 * 3 = 3 1 * 7 = 7 ··· 501 __u8 filename[NR_DENTRY_IN_BLOCK][F2FS_SLOT_LEN]; 502 } __packed; 503 504 /* file types used in inode_info->flags */ 505 enum { 506 F2FS_FT_UNKNOWN, ··· 533 }; 534 535 #define S_SHIFT 12 536 + 537 + #define F2FS_DEF_PROJID 0 /* default project ID */ 538 539 #endif /* _LINUX_F2FS_FS_H */
+110 -3
include/trace/events/f2fs.h
··· 543 544 TRACE_EVENT(f2fs_background_gc, 545 546 - TP_PROTO(struct super_block *sb, long wait_ms, 547 unsigned int prefree, unsigned int free), 548 549 TP_ARGS(sb, wait_ms, prefree, free), 550 551 TP_STRUCT__entry( 552 __field(dev_t, dev) 553 - __field(long, wait_ms) 554 __field(unsigned int, prefree) 555 __field(unsigned int, free) 556 ), ··· 562 __entry->free = free; 563 ), 564 565 - TP_printk("dev = (%d,%d), wait_ms = %ld, prefree = %u, free = %u", 566 show_dev(__entry->dev), 567 __entry->wait_ms, 568 __entry->prefree, 569 __entry->free) 570 ); 571 572 TRACE_EVENT(f2fs_get_victim,
··· 543 544 TRACE_EVENT(f2fs_background_gc, 545 546 + TP_PROTO(struct super_block *sb, unsigned int wait_ms, 547 unsigned int prefree, unsigned int free), 548 549 TP_ARGS(sb, wait_ms, prefree, free), 550 551 TP_STRUCT__entry( 552 __field(dev_t, dev) 553 + __field(unsigned int, wait_ms) 554 __field(unsigned int, prefree) 555 __field(unsigned int, free) 556 ), ··· 562 __entry->free = free; 563 ), 564 565 + TP_printk("dev = (%d,%d), wait_ms = %u, prefree = %u, free = %u", 566 show_dev(__entry->dev), 567 __entry->wait_ms, 568 __entry->prefree, 569 __entry->free) 570 + ); 571 + 572 + TRACE_EVENT(f2fs_gc_begin, 573 + 574 + TP_PROTO(struct super_block *sb, bool sync, bool background, 575 + long long dirty_nodes, long long dirty_dents, 576 + long long dirty_imeta, unsigned int free_sec, 577 + unsigned int free_seg, int reserved_seg, 578 + unsigned int prefree_seg), 579 + 580 + TP_ARGS(sb, sync, background, dirty_nodes, dirty_dents, dirty_imeta, 581 + free_sec, free_seg, reserved_seg, prefree_seg), 582 + 583 + TP_STRUCT__entry( 584 + __field(dev_t, dev) 585 + __field(bool, sync) 586 + __field(bool, background) 587 + __field(long long, dirty_nodes) 588 + __field(long long, dirty_dents) 589 + __field(long long, dirty_imeta) 590 + __field(unsigned int, free_sec) 591 + __field(unsigned int, free_seg) 592 + __field(int, reserved_seg) 593 + __field(unsigned int, prefree_seg) 594 + ), 595 + 596 + TP_fast_assign( 597 + __entry->dev = sb->s_dev; 598 + __entry->sync = sync; 599 + __entry->background = background; 600 + __entry->dirty_nodes = dirty_nodes; 601 + __entry->dirty_dents = dirty_dents; 602 + __entry->dirty_imeta = dirty_imeta; 603 + __entry->free_sec = free_sec; 604 + __entry->free_seg = free_seg; 605 + __entry->reserved_seg = reserved_seg; 606 + __entry->prefree_seg = prefree_seg; 607 + ), 608 + 609 + TP_printk("dev = (%d,%d), sync = %d, background = %d, nodes = %lld, " 610 + "dents = %lld, imeta = %lld, free_sec:%u, free_seg:%u, " 611 + "rsv_seg:%d, prefree_seg:%u", 612 + show_dev(__entry->dev), 613 + __entry->sync, 614 + __entry->background, 615 + __entry->dirty_nodes, 616 + __entry->dirty_dents, 617 + __entry->dirty_imeta, 618 + __entry->free_sec, 619 + __entry->free_seg, 620 + __entry->reserved_seg, 621 + __entry->prefree_seg) 622 + ); 623 + 624 + TRACE_EVENT(f2fs_gc_end, 625 + 626 + TP_PROTO(struct super_block *sb, int ret, int seg_freed, 627 + int sec_freed, long long dirty_nodes, 628 + long long dirty_dents, long long dirty_imeta, 629 + unsigned int free_sec, unsigned int free_seg, 630 + int reserved_seg, unsigned int prefree_seg), 631 + 632 + TP_ARGS(sb, ret, seg_freed, sec_freed, dirty_nodes, dirty_dents, 633 + dirty_imeta, free_sec, free_seg, reserved_seg, prefree_seg), 634 + 635 + TP_STRUCT__entry( 636 + __field(dev_t, dev) 637 + __field(int, ret) 638 + __field(int, seg_freed) 639 + __field(int, sec_freed) 640 + __field(long long, dirty_nodes) 641 + __field(long long, dirty_dents) 642 + __field(long long, dirty_imeta) 643 + __field(unsigned int, free_sec) 644 + __field(unsigned int, free_seg) 645 + __field(int, reserved_seg) 646 + __field(unsigned int, prefree_seg) 647 + ), 648 + 649 + TP_fast_assign( 650 + __entry->dev = sb->s_dev; 651 + __entry->ret = ret; 652 + __entry->seg_freed = seg_freed; 653 + __entry->sec_freed = sec_freed; 654 + __entry->dirty_nodes = dirty_nodes; 655 + __entry->dirty_dents = dirty_dents; 656 + __entry->dirty_imeta = dirty_imeta; 657 + __entry->free_sec = free_sec; 658 + __entry->free_seg = free_seg; 659 + __entry->reserved_seg = reserved_seg; 660 + __entry->prefree_seg = prefree_seg; 661 + ), 662 + 663 + TP_printk("dev = (%d,%d), ret = %d, seg_freed = %d, sec_freed = %d, " 664 + "nodes = %lld, dents = %lld, imeta = %lld, free_sec:%u, " 665 + "free_seg:%u, rsv_seg:%d, prefree_seg:%u", 666 + show_dev(__entry->dev), 667 + __entry->ret, 668 + __entry->seg_freed, 669 + __entry->sec_freed, 670 + __entry->dirty_nodes, 671 + __entry->dirty_dents, 672 + __entry->dirty_imeta, 673 + __entry->free_sec, 674 + __entry->free_seg, 675 + __entry->reserved_seg, 676 + __entry->prefree_seg) 677 ); 678 679 TRACE_EVENT(f2fs_get_victim,