From 75a5221630fe5aa3fedba7a06be618db0f79ba1e Mon Sep 17 00:00:00 2001 From: Gao Xiang Date: Thu, 7 Sep 2023 13:05:42 +0800 Subject: [PATCH 1/3] erofs: fix memory leak of LZMA global compressed deduplication When stressing microLZMA EROFS images with the new global compressed deduplication feature enabled (`-Ededupe`), I found some short-lived temporary pages weren't properly released, which could slowly cause unexpected OOMs hours later. Let's fix it now (LZ4 and DEFLATE don't have this issue.) Fixes: 5c2a64252c5d ("erofs: introduce partial-referenced pclusters") Signed-off-by: Gao Xiang Link: https://lore.kernel.org/r/20230907050542.97152-1-hsiangkao@linux.alibaba.com --- fs/erofs/decompressor_lzma.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/erofs/decompressor_lzma.c b/fs/erofs/decompressor_lzma.c index 73091fbe3ea4..dee10d22ada9 100644 --- a/fs/erofs/decompressor_lzma.c +++ b/fs/erofs/decompressor_lzma.c @@ -217,9 +217,12 @@ int z_erofs_lzma_decompress(struct z_erofs_decompress_req *rq, strm->buf.out_size = min_t(u32, outlen, PAGE_SIZE - pageofs); outlen -= strm->buf.out_size; - if (!rq->out[no] && rq->fillgaps) /* deduped */ + if (!rq->out[no] && rq->fillgaps) { /* deduped */ rq->out[no] = erofs_allocpage(pagepool, GFP_KERNEL | __GFP_NOFAIL); + set_page_private(rq->out[no], + Z_EROFS_SHORTLIVED_PAGE); + } if (rq->out[no]) strm->buf.out = kmap(rq->out[no]) + pageofs; pageofs = 0; From f939aeea7ab7d96cd321e7ac107f5a070836b66f Mon Sep 17 00:00:00 2001 From: Jingbo Xu Date: Fri, 15 Sep 2023 16:27:28 +0800 Subject: [PATCH 2/3] erofs: allow empty device tags in flatdev mode Device tags aren't actually required in flatdev mode, thus fix mount failure due to empty device tags in flatdev mode. Signed-off-by: Jingbo Xu Fixes: 8b465fecc35a ("erofs: support flattened block device for multi-blob images") Reviewed-by: Jia Zhu Reviewed-by: Gao Xiang Link: https://lore.kernel.org/r/20230915082728.56588-1-jefflexu@linux.alibaba.com Signed-off-by: Gao Xiang --- fs/erofs/super.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/erofs/super.c b/fs/erofs/super.c index 44a24d573f1f..3700af9ee173 100644 --- a/fs/erofs/super.c +++ b/fs/erofs/super.c @@ -235,7 +235,7 @@ static int erofs_init_device(struct erofs_buf *buf, struct super_block *sb, return PTR_ERR(ptr); dis = ptr + erofs_blkoff(sb, *pos); - if (!dif->path) { + if (!sbi->devs->flatdev && !dif->path) { if (!dis->tag[0]) { erofs_err(sb, "empty device tag @ pos %llu", *pos); return -EINVAL; From 3048102d9d68008e948decbd730f0748dd7bdc31 Mon Sep 17 00:00:00 2001 From: Jingbo Xu Date: Thu, 28 Sep 2023 21:48:52 +0800 Subject: [PATCH 3/3] erofs: update documentation - update new features like bloom filter and DEFLATE. - add documentation for the long xattr name prefixes, which was landed upstream since v6.4. Signed-off-by: Jingbo Xu Reviewed-by: Gao Xiang Link: https://lore.kernel.org/r/20230928134852.31118-1-jefflexu@linux.alibaba.com Signed-off-by: Gao Xiang --- Documentation/filesystems/erofs.rst | 40 ++++++++++++++++++++++++++--- 1 file changed, 37 insertions(+), 3 deletions(-) diff --git a/Documentation/filesystems/erofs.rst b/Documentation/filesystems/erofs.rst index 4654ee57c1d5..f200d7874495 100644 --- a/Documentation/filesystems/erofs.rst +++ b/Documentation/filesystems/erofs.rst @@ -58,12 +58,14 @@ Here are the main features of EROFS: - Support extended attributes as an option; + - Support a bloom filter that speeds up negative extended attribute lookups; + - Support POSIX.1e ACLs by using extended attributes; - Support transparent data compression as an option: - LZ4 and MicroLZMA algorithms can be used on a per-file basis; In addition, - inplace decompression is also supported to avoid bounce compressed buffers - and page cache thrashing. + LZ4, MicroLZMA and DEFLATE algorithms can be used on a per-file basis; In + addition, inplace decompression is also supported to avoid bounce compressed + buffers and unnecessary page cache thrashing. - Support chunk-based data deduplication and rolling-hash compressed data deduplication; @@ -268,6 +270,38 @@ details.) By the way, chunk-based files are all uncompressed for now. +Long extended attribute name prefixes +------------------------------------- +There are use cases where extended attributes with different values can have +only a few common prefixes (such as overlayfs xattrs). The predefined prefixes +work inefficiently in both image size and runtime performance in such cases. + +The long xattr name prefixes feature is introduced to address this issue. The +overall idea is that, apart from the existing predefined prefixes, the xattr +entry could also refer to user-specified long xattr name prefixes, e.g. +"trusted.overlay.". + +When referring to a long xattr name prefix, the highest bit (bit 7) of +erofs_xattr_entry.e_name_index is set, while the lower bits (bit 0-6) as a whole +represent the index of the referred long name prefix among all long name +prefixes. Therefore, only the trailing part of the name apart from the long +xattr name prefix is stored in erofs_xattr_entry.e_name, which could be empty if +the full xattr name matches exactly as its long xattr name prefix. + +All long xattr prefixes are stored one by one in the packed inode as long as +the packed inode is valid, or in the meta inode otherwise. The +xattr_prefix_count (of the on-disk superblock) indicates the total number of +long xattr name prefixes, while (xattr_prefix_start * 4) indicates the start +offset of long name prefixes in the packed/meta inode. Note that, long extended +attribute name prefixes are disabled if xattr_prefix_count is 0. + +Each long name prefix is stored in the format: ALIGN({__le16 len, data}, 4), +where len represents the total size of the data part. The data part is actually +represented by 'struct erofs_xattr_long_prefix', where base_index represents the +index of the predefined xattr name prefix, e.g. EROFS_XATTR_INDEX_TRUSTED for +"trusted.overlay." long name prefix, while the infix string keeps the string +after stripping the short prefix, e.g. "overlay." for the example above. + Data compression ---------------- EROFS implements fixed-sized output compression which generates fixed-sized