tjh.dev
Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
1Overview
2========
3
4EROFS file-system stands for Enhanced Read-Only File System. Different
5from other read-only file systems, it aims to be designed for flexibility,
6scalability, but be kept simple and high performance.
7
8It is designed as a better filesystem solution for the following scenarios:
9 - read-only storage media or
10
11 - part of a fully trusted read-only solution, which means it needs to be
12 immutable and bit-for-bit identical to the official golden image for
13 their releases due to security and other considerations and
14
15 - hope to save some extra storage space with guaranteed end-to-end performance
16 by using reduced metadata and transparent file compression, especially
17 for those embedded devices with limited memory (ex, smartphone);
18
19Here is the main features of EROFS:
20 - Little endian on-disk design;
21
22 - Currently 4KB block size (nobh) and therefore maximum 16TB address space;
23
24 - Metadata & data could be mixed by design;
25
26 - 2 inode versions for different requirements:
27 compact (v1) extended (v2)
28 Inode metadata size: 32 bytes 64 bytes
29 Max file size: 4 GB 16 EB (also limited by max. vol size)
30 Max uids/gids: 65536 4294967296
31 File change time: no yes (64 + 32-bit timestamp)
32 Max hardlinks: 65536 4294967296
33 Metadata reserved: 4 bytes 14 bytes
34
35 - Support extended attributes (xattrs) as an option;
36
37 - Support xattr inline and tail-end data inline for all files;
38
39 - Support POSIX.1e ACLs by using xattrs;
40
41 - Support transparent file compression as an option:
42 LZ4 algorithm with 4 KB fixed-sized output compression for high performance.
43
44The following git tree provides the file system user-space tools under
45development (ex, formatting tool mkfs.erofs):
46>> git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git
47
48Bugs and patches are welcome, please kindly help us and send to the following
49linux-erofs mailing list:
50>> linux-erofs mailing list <linux-erofs@lists.ozlabs.org>
51
52Mount options
53=============
54
55(no)user_xattr Setup Extended User Attributes. Note: xattr is enabled
56 by default if CONFIG_EROFS_FS_XATTR is selected.
57(no)acl Setup POSIX Access Control List. Note: acl is enabled
58 by default if CONFIG_EROFS_FS_POSIX_ACL is selected.
59cache_strategy=%s Select a strategy for cached decompression from now on:
60 disabled: In-place I/O decompression only;
61 readahead: Cache the last incomplete compressed physical
62 cluster for further reading. It still does
63 in-place I/O decompression for the rest
64 compressed physical clusters;
65 readaround: Cache the both ends of incomplete compressed
66 physical clusters for further reading.
67 It still does in-place I/O decompression
68 for the rest compressed physical clusters.
69
70On-disk details
71===============
72
73Summary
74-------
75Different from other read-only file systems, an EROFS volume is designed
76to be as simple as possible:
77
78 |-> aligned with the block size
79 ____________________________________________________________
80 | |SB| | ... | Metadata | ... | Data | Metadata | ... | Data |
81 |_|__|_|_____|__________|_____|______|__________|_____|______|
82 0 +1K
83
84All data areas should be aligned with the block size, but metadata areas
85may not. All metadatas can be now observed in two different spaces (views):
86 1. Inode metadata space
87 Each valid inode should be aligned with an inode slot, which is a fixed
88 value (32 bytes) and designed to be kept in line with compact inode size.
89
90 Each inode can be directly found with the following formula:
91 inode offset = meta_blkaddr * block_size + 32 * nid
92
93 |-> aligned with 8B
94 |-> followed closely
95 + meta_blkaddr blocks |-> another slot
96 _____________________________________________________________________
97 | ... | inode | xattrs | extents | data inline | ... | inode ...
98 |________|_______|(optional)|(optional)|__(optional)_|_____|__________
99 |-> aligned with the inode slot size
100 . .
101 . .
102 . .
103 . .
104 . .
105 . .
106 .____________________________________________________|-> aligned with 4B
107 | xattr_ibody_header | shared xattrs | inline xattrs |
108 |____________________|_______________|_______________|
109 |-> 12 bytes <-|->x * 4 bytes<-| .
110 . . .
111 . . .
112 . . .
113 ._______________________________.______________________.
114 | id | id | id | id | ... | id | ent | ... | ent| ... |
115 |____|____|____|____|______|____|_____|_____|____|_____|
116 |-> aligned with 4B
117 |-> aligned with 4B
118
119 Inode could be 32 or 64 bytes, which can be distinguished from a common
120 field which all inode versions have -- i_format:
121
122 __________________ __________________
123 | i_format | | i_format |
124 |__________________| |__________________|
125 | ... | | ... |
126 | | | |
127 |__________________| 32 bytes | |
128 | |
129 |__________________| 64 bytes
130
131 Xattrs, extents, data inline are followed by the corresponding inode with
132 proper alignment, and they could be optional for different data mappings.
133 _currently_ total 4 valid data mappings are supported:
134
135 0 flat file data without data inline (no extent);
136 1 fixed-sized output data compression (with non-compacted indexes);
137 2 flat file data with tail packing data inline (no extent);
138 3 fixed-sized output data compression (with compacted indexes, v5.3+).
139
140 The size of the optional xattrs is indicated by i_xattr_count in inode
141 header. Large xattrs or xattrs shared by many different files can be
142 stored in shared xattrs metadata rather than inlined right after inode.
143
144 2. Shared xattrs metadata space
145 Shared xattrs space is similar to the above inode space, started with
146 a specific block indicated by xattr_blkaddr, organized one by one with
147 proper align.
148
149 Each share xattr can also be directly found by the following formula:
150 xattr offset = xattr_blkaddr * block_size + 4 * xattr_id
151
152 |-> aligned by 4 bytes
153 + xattr_blkaddr blocks |-> aligned with 4 bytes
154 _________________________________________________________________________
155 | ... | xattr_entry | xattr data | ... | xattr_entry | xattr data ...
156 |________|_____________|_____________|_____|______________|_______________
157
158Directories
159-----------
160All directories are now organized in a compact on-disk format. Note that
161each directory block is divided into index and name areas in order to support
162random file lookup, and all directory entries are _strictly_ recorded in
163alphabetical order in order to support improved prefix binary search
164algorithm (could refer to the related source code).
165
166 ___________________________
167 / |
168 / ______________|________________
169 / / | nameoff1 | nameoffN-1
170 ____________.______________._______________v________________v__________
171| dirent | dirent | ... | dirent | filename | filename | ... | filename |
172|___.0___|____1___|_____|___N-1__|____0_____|____1_____|_____|___N-1____|
173 \ ^
174 \ | * could have
175 \ | trailing '\0'
176 \________________________| nameoff0
177
178 Directory block
179
180Note that apart from the offset of the first filename, nameoff0 also indicates
181the total number of directory entries in this block since it is no need to
182introduce another on-disk field at all.
183
184Compression
185-----------
186Currently, EROFS supports 4KB fixed-sized output transparent file compression,
187as illustrated below:
188
189 |---- Variant-Length Extent ----|-------- VLE --------|----- VLE -----
190 clusterofs clusterofs clusterofs
191 | | | logical data
192_________v_______________________________v_____________________v_______________
193... | . | | . | | . | ...
194____|____.________|_____________|________.____|_____________|__.__________|____
195 |-> cluster <-|-> cluster <-|-> cluster <-|-> cluster <-|-> cluster <-|
196 size size size size size
197 . . . .
198 . . . .
199 . . . .
200 _______._____________._____________._____________._____________________
201 ... | | | | ... physical data
202 _______|_____________|_____________|_____________|_____________________
203 |-> cluster <-|-> cluster <-|-> cluster <-|
204 size size size
205
206Currently each on-disk physical cluster can contain 4KB (un)compressed data
207at most. For each logical cluster, there is a corresponding on-disk index to
208describe its cluster type, physical cluster address, etc.
209
210See "struct z_erofs_vle_decompressed_index" in erofs_fs.h for more details.
211