Kernel/linux-stable
1
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git synced 2024-12-29 09:16:33 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge patch series "timekeeping/fs: multigrain timestamp redux"

Browse Source 
Jeff Layton <jlayton@kernel.org> says:

The VFS has always used coarse-grained timestamps when updating the
ctime and mtime after a change. This has the benefit of allowing
filesystems to optimize away a lot metadata updates, down to around 1
per jiffy, even when a file is under heavy writes.

Unfortunately, this has always been an issue when we're exporting via
NFSv3, which relies on timestamps to validate caches. A lot of changes
can happen in a jiffy, so timestamps aren't sufficient to help the
client decide when to invalidate the cache. Even with NFSv4, a lot of
exported filesystems don't properly support a change attribute and are
subject to the same problems with timestamp granularity. Other
applications have similar issues with timestamps (e.g backup
applications).

If we were to always use fine-grained timestamps, that would improve the
situation, but that becomes rather expensive, as the underlying
filesystem would have to log a lot more metadata updates.

What we need is a way to only use fine-grained timestamps when they are
being actively queried. Use the (unused) top bit in inode->i_ctime_nsec
as a flag that indicates whether the current timestamps have been
queried via stat() or the like. When it's set, we allow the kernel to
use a fine-grained timestamp iff it's necessary to make the ctime show
a different value.

This solves the problem of being able to distinguish the timestamp
between updates, but introduces a new problem: it's now possible for a
file being changed to get a fine-grained timestamp. A file that is
altered just a bit later can then get a coarse-grained one that appears
older than the earlier fine-grained time. This violates timestamp
ordering guarantees.

To remedy this, keep a global monotonic atomic64_t value that acts as a
timestamp floor.  When we go to stamp a file, we first get the latter of
the current floor value and the current coarse-grained time. If the
inode ctime hasn't been queried then we just attempt to stamp it with
that value.

If it has been queried, then first see whether the current coarse time
is later than the existing ctime. If it is, then we accept that value.
If it isn't, then we get a fine-grained time and try to swap that into
the global floor. Whether that succeeds or fails, we take the resulting
floor time, convert it to realtime and try to swap that into the ctime.

We take the result of the ctime swap whether it succeeds or fails, since
either is just as valid.

Filesystems can opt into this by setting the FS_MGTIME fstype flag.
Others should be unaffected (other than being subject to the same floor
value as multigrain filesystems).

* patches from https://lore.kernel.org/r/20241002-mgtime-v10-0-d1c4717f5284@kernel.org:
  tmpfs: add support for multigrain timestamps
  btrfs: convert to multigrain timestamps
  ext4: switch to multigrain timestamps
  xfs: switch to multigrain timestamps
  Documentation: add a new file documenting multigrain timestamps
  fs: add percpu counters for significant multigrain timestamp events
  fs: tracepoints around multigrain timestamp events
  fs: handle delegated timestamps in setattr_copy_mgtime
  fs: have setattr_copy handle multigrain timestamps appropriately
  fs: add infrastructure for multigrain timestamps

Link: https://lore.kernel.org/r/20241002-mgtime-v10-0-d1c4717f5284@kernel.org
Signed-off-by: Christian Brauner <brauner@kernel.org>
This commit is contained in:
Christian Brauner 2024-10-07 12:49:12 +02:00
commit b40508ca5d
No known key found for this signature in database
GPG Key ID: 91C61BC06578DCA2
14 changed files with 656 additions and 77 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
Documentation/filesystems/index.rst
View File

@ -29,6 +29,7 @@ algorithms work.
fiemap
files
locks
multigrain-ts
mount_api
quota
seq_file

125
Documentation/filesystems/multigrain-ts.rst Normal file
View File

@ -0,0 +1,125 @@
.. SPDX-License-Identifier: GPL-2.0
=====================
Multigrain Timestamps
=====================
Introduction
============
Historically, the kernel has always used coarse time values to stamp inodes.
This value is updated every jiffy, so any change that happens within that jiffy
will end up with the same timestamp.
When the kernel goes to stamp an inode (due to a read or write), it first gets
the current time and then compares it to the existing timestamp(s) to see
whether anything will change. If nothing changed, then it can avoid updating
the inode's metadata.
Coarse timestamps are therefore good from a performance standpoint, since they
reduce the need for metadata updates, but bad from the standpoint of
determining whether anything has changed, since a lot of things can happen in a
jiffy.
They are particularly troublesome with NFSv3, where unchanging timestamps can
make it difficult to tell whether to invalidate caches. NFSv4 provides a
dedicated change attribute that should always show a visible change, but not
all filesystems implement this properly, causing the NFS server to substitute
the ctime in many cases.
Multigrain timestamps aim to remedy this by selectively using fine-grained
timestamps when a file has had its timestamps queried recently, and the current
coarse-grained time does not cause a change.
Inode Timestamps
================
There are currently 3 timestamps in the inode that are updated to the current
wallclock time on different activity:
ctime:
The inode change time. This is stamped with the current time whenever
the inode's metadata is changed. Note that this value is not settable
from userland.
mtime:
The inode modification time. This is stamped with the current time
any time a file's contents change.
atime:
The inode access time. This is stamped whenever an inode's contents are
read. Widely considered to be a terrible mistake. Usually avoided with
options like noatime or relatime.
Updating the mtime always implies a change to the ctime, but updating the
atime due to a read request does not.
Multigrain timestamps are only tracked for the ctime and the mtime. atimes are
not affected and always use the coarse-grained value (subject to the floor).
Inode Timestamp Ordering
========================
In addition to just providing info about changes to individual files, file
timestamps also serve an important purpose in applications like "make". These
programs measure timestamps in order to determine whether source files might be
newer than cached objects.
Userland applications like make can only determine ordering based on
operational boundaries. For a syscall those are the syscall entry and exit
points. For io_uring or nfsd operations, that's the request submission and
response. In the case of concurrent operations, userland can make no
determination about the order in which things will occur.
For instance, if a single thread modifies one file, and then another file in
sequence, the second file must show an equal or later mtime than the first. The
same is true if two threads are issuing similar operations that do not overlap
in time.
If however, two threads have racing syscalls that overlap in time, then there
is no such guarantee, and the second file may appear to have been modified
before, after or at the same time as the first, regardless of which one was
submitted first.
Note that the above assumes that the system doesn't experience a backward jump
of the realtime clock. If that occurs at an inopportune time, then timestamps
can appear to go backward, even on a properly functioning system.
Multigrain Timestamp Implementation
===================================
Multigrain timestamps are aimed at ensuring that changes to a single file are
always recognizable, without violating the ordering guarantees when multiple
different files are modified. This affects the mtime and the ctime, but the
atime will always use coarse-grained timestamps.
It uses an unused bit in the i_ctime_nsec field to indicate whether the mtime
or ctime has been queried. If either or both have, then the kernel takes
special care to ensure the next timestamp update will display a visible change.
This ensures tight cache coherency for use-cases like NFS, without sacrificing
the benefits of reduced metadata updates when files aren't being watched.
The Ctime Floor Value
=====================
It's not sufficient to simply use fine or coarse-grained timestamps based on
whether the mtime or ctime has been queried. A file could get a fine grained
timestamp, and then a second file modified later could get a coarse-grained one
that appears earlier than the first, which would break the kernel's timestamp
ordering guarantees.
To mitigate this problem, maintain a global floor value that ensures that
this can't happen. The two files in the above example may appear to have been
modified at the same time in such a case, but they will never show the reverse
order. To avoid problems with realtime clock jumps, the floor is managed as a
monotonic ktime_t, and the values are converted to realtime clock values as
needed.
Implementation Notes
====================
Multigrain timestamps are intended for use by local filesystems that get
ctime values from the local clock. This is in contrast to network filesystems
and the like that just mirror timestamp values from a server.
For most filesystems, it's sufficient to just set the FS_MGTIME flag in the
fstype->fs_flags in order to opt-in, providing the ctime is only ever set via
inode_set_ctime_current(). If the filesystem has a ->getattr routine that
doesn't call generic_fillattr, then it should call fill_mg_cmtime() to
fill those values. For setattr, it should use setattr_copy() to update the
timestamps, or otherwise mimic its behavior.

61
fs/attr.c
View File

@ -271,6 +271,47 @@ int inode_newsize_ok(const struct inode *inode, loff_t offset)
}
EXPORT_SYMBOL(inode_newsize_ok);
/**
* setattr_copy_mgtime - update timestamps for mgtime inodes
* @inode: inode timestamps to be updated
* @attr: attrs for the update
*
* With multigrain timestamps, take more care to prevent races when
* updating the ctime. Always update the ctime to the very latest using
* the standard mechanism, and use that to populate the atime and mtime
* appropriately (unless those are being set to specific values).
*/
static void setattr_copy_mgtime(struct inode *inode, const struct iattr *attr)
{
unsigned int ia_valid = attr->ia_valid;
struct timespec64 now;
if (ia_valid & ATTR_CTIME) {
/*
* In the case of an update for a write delegation, we must respect
* the value in ia_ctime and not use the current time.
*/
if (ia_valid & ATTR_DELEG)
now = inode_set_ctime_deleg(inode, attr->ia_ctime);
else
now = inode_set_ctime_current(inode);
} else {
/* If ATTR_CTIME isn't set, then ATTR_MTIME shouldn't be either. */
WARN_ON_ONCE(ia_valid & ATTR_MTIME);
now = current_time(inode);
}
if (ia_valid & ATTR_ATIME_SET)
inode_set_atime_to_ts(inode, attr->ia_atime);
else if (ia_valid & ATTR_ATIME)
inode_set_atime_to_ts(inode, now);
if (ia_valid & ATTR_MTIME_SET)
inode_set_mtime_to_ts(inode, attr->ia_mtime);
else if (ia_valid & ATTR_MTIME)
inode_set_mtime_to_ts(inode, now);
}
/**
* setattr_copy - copy simple metadata updates into the generic inode
* @idmap: idmap of the mount the inode was found from
@ -303,12 +344,6 @@ void setattr_copy(struct mnt_idmap *idmap, struct inode *inode,
i_uid_update(idmap, attr, inode);
i_gid_update(idmap, attr, inode);
if (ia_valid & ATTR_ATIME)
inode_set_atime_to_ts(inode, attr->ia_atime);
if (ia_valid & ATTR_MTIME)
inode_set_mtime_to_ts(inode, attr->ia_mtime);
if (ia_valid & ATTR_CTIME)
inode_set_ctime_to_ts(inode, attr->ia_ctime);
if (ia_valid & ATTR_MODE) {
umode_t mode = attr->ia_mode;
if (!in_group_or_capable(idmap, inode,
@ -316,6 +351,20 @@ void setattr_copy(struct mnt_idmap *idmap, struct inode *inode,
mode &= ~S_ISGID;
inode->i_mode = mode;
}
if (is_mgtime(inode))
return setattr_copy_mgtime(inode, attr);
if (ia_valid & ATTR_ATIME)
inode_set_atime_to_ts(inode, attr->ia_atime);
if (ia_valid & ATTR_MTIME)
inode_set_mtime_to_ts(inode, attr->ia_mtime);
if (ia_valid & ATTR_CTIME) {
if (ia_valid & ATTR_DELEG)
inode_set_ctime_deleg(inode, attr->ia_ctime);
else
inode_set_ctime_to_ts(inode, attr->ia_ctime);
}
}
EXPORT_SYMBOL(setattr_copy);

25
fs/btrfs/file.c
View File

@ -1120,26 +1120,6 @@ void btrfs_check_nocow_unlock(struct btrfs_inode *inode)
btrfs_drew_write_unlock(&inode->root->snapshot_lock);
}
static void update_time_for_write(struct inode *inode)
{
struct timespec64 now, ts;
if (IS_NOCMTIME(inode))
return;
now = current_time(inode);
ts = inode_get_mtime(inode);
if (!timespec64_equal(&ts, &now))
inode_set_mtime_to_ts(inode, now);
ts = inode_get_ctime(inode);
if (!timespec64_equal(&ts, &now))
inode_set_ctime_to_ts(inode, now);
if (IS_I_VERSION(inode))
inode_inc_iversion(inode);
}
int btrfs_write_check(struct kiocb *iocb, struct iov_iter *from, size_t count)
{
struct file *file = iocb->ki_filp;
@ -1170,7 +1150,10 @@ int btrfs_write_check(struct kiocb *iocb, struct iov_iter *from, size_t count)
* need to start yet another transaction to update the inode as we will
* update the inode when we finish writing whatever data we write.
*/
update_time_for_write(inode);
if (!IS_NOCMTIME(inode)) {
inode_set_mtime_to_ts(inode, inode_set_ctime_current(inode));
inode_inc_iversion(inode);
}
start_pos = round_down(pos, fs_info->sectorsize);
oldsize = i_size_read(inode);

3
fs/btrfs/super.c
View File

@ -2198,7 +2198,8 @@ static struct file_system_type btrfs_fs_type = {
.init_fs_context = btrfs_init_fs_context,
.parameters = btrfs_fs_parameters,
.kill_sb = btrfs_kill_super,
.fs_flags = FS_REQUIRES_DEV | FS_BINARY_MOUNTDATA | FS_ALLOW_IDMAP,
.fs_flags = FS_REQUIRES_DEV | FS_BINARY_MOUNTDATA |
FS_ALLOW_IDMAP | FS_MGTIME,
};
MODULE_ALIAS_FS("btrfs");

2
fs/ext4/super.c
View File

@ -7329,7 +7329,7 @@ static struct file_system_type ext4_fs_type = {
.init_fs_context = ext4_init_fs_context,
.parameters = ext4_param_specs,
.kill_sb = ext4_kill_sb,
.fs_flags = FS_REQUIRES_DEV | FS_ALLOW_IDMAP,
.fs_flags = FS_REQUIRES_DEV | FS_ALLOW_IDMAP | FS_MGTIME,
};
MODULE_ALIAS_FS("ext4");

290
fs/inode.c
View File

@ -21,7 +21,12 @@
#include <linux/list_lru.h>
#include <linux/iversion.h>
#include <linux/rw_hint.h>
#include <linux/seq_file.h>
#include <linux/debugfs.h>
#include <trace/events/writeback.h>
#define CREATE_TRACE_POINTS
#include <trace/events/timestamp.h>
#include "internal.h"
/*
@ -98,6 +103,70 @@ long get_nr_dirty_inodes(void)
return nr_dirty > 0 ? nr_dirty : 0;
}
#ifdef CONFIG_DEBUG_FS
static DEFINE_PER_CPU(long, mg_ctime_updates);
static DEFINE_PER_CPU(long, mg_fine_stamps);
static DEFINE_PER_CPU(long, mg_ctime_swaps);
static unsigned long get_mg_ctime_updates(void)
{
unsigned long sum = 0;
int i;
for_each_possible_cpu(i)
sum += data_race(per_cpu(mg_ctime_updates, i));
return sum;
}
static unsigned long get_mg_fine_stamps(void)
{
unsigned long sum = 0;
int i;
for_each_possible_cpu(i)
sum += data_race(per_cpu(mg_fine_stamps, i));
return sum;
}
static unsigned long get_mg_ctime_swaps(void)
{
unsigned long sum = 0;
int i;
for_each_possible_cpu(i)
sum += data_race(per_cpu(mg_ctime_swaps, i));
return sum;
}
#define mgtime_counter_inc(__var) this_cpu_inc(__var)
static int mgts_show(struct seq_file *s, void *p)
{
unsigned long ctime_updates = get_mg_ctime_updates();
unsigned long ctime_swaps = get_mg_ctime_swaps();
unsigned long fine_stamps = get_mg_fine_stamps();
unsigned long floor_swaps = timekeeping_get_mg_floor_swaps();
seq_printf(s, "%lu %lu %lu %lu\n",
ctime_updates, ctime_swaps, fine_stamps, floor_swaps);
return 0;
}
DEFINE_SHOW_ATTRIBUTE(mgts);
static int __init mg_debugfs_init(void)
{
debugfs_create_file("multigrain_timestamps", S_IFREG | S_IRUGO, NULL, NULL, &mgts_fops);
return 0;
}
late_initcall(mg_debugfs_init);
#else /* ! CONFIG_DEBUG_FS */
#define mgtime_counter_inc(__var) do { } while (0)
#endif /* CONFIG_DEBUG_FS */
/*
* Handle nr_inode sysctl
*/
@ -2209,19 +2278,58 @@ int file_remove_privs(struct file *file)
}
EXPORT_SYMBOL(file_remove_privs);
/**
* current_time - Return FS time (possibly fine-grained)
* @inode: inode.
*
* Return the current time truncated to the time granularity supported by
* the fs, as suitable for a ctime/mtime change. If the ctime is flagged
* as having been QUERIED, get a fine-grained timestamp, but don't update
* the floor.
*
* For a multigrain inode, this is effectively an estimate of the timestamp
* that a file would receive. An actual update must go through
* inode_set_ctime_current().
*/
struct timespec64 current_time(struct inode *inode)
{
struct timespec64 now;
u32 cns;
ktime_get_coarse_real_ts64_mg(&now);
if (!is_mgtime(inode))
goto out;
/* If nothing has queried it, then coarse time is fine */
cns = smp_load_acquire(&inode->i_ctime_nsec);
if (cns & I_CTIME_QUERIED) {
/*
* If there is no apparent change, then get a fine-grained
* timestamp.
*/
if (now.tv_nsec == (cns & ~I_CTIME_QUERIED))
ktime_get_real_ts64(&now);
}
out:
return timestamp_truncate(now, inode);
}
EXPORT_SYMBOL(current_time);
static int inode_needs_update_time(struct inode *inode)
{
struct timespec64 now, ts;
int sync_it = 0;
struct timespec64 now = current_time(inode);
struct timespec64 ts;
/* First try to exhaust all avenues to not sync */
if (IS_NOCMTIME(inode))
return 0;
now = current_time(inode);
ts = inode_get_mtime(inode);
if (!timespec64_equal(&ts, &now))
sync_it = S_MTIME;
sync_it |= S_MTIME;
ts = inode_get_ctime(inode);
if (!timespec64_equal(&ts, &now))
@ -2598,6 +2706,16 @@ void inode_nohighmem(struct inode *inode)
}
EXPORT_SYMBOL(inode_nohighmem);
struct timespec64 inode_set_ctime_to_ts(struct inode *inode, struct timespec64 ts)
{
trace_inode_set_ctime_to_ts(inode, &ts);
set_normalized_timespec64(&ts, ts.tv_sec, ts.tv_nsec);
inode->i_ctime_sec = ts.tv_sec;
inode->i_ctime_nsec = ts.tv_nsec;
return ts;
}
EXPORT_SYMBOL(inode_set_ctime_to_ts);
/**
* timestamp_truncate - Truncate timespec to a granularity
* @t: Timespec
@ -2629,41 +2747,161 @@ struct timespec64 timestamp_truncate(struct timespec64 t, struct inode *inode)
}
EXPORT_SYMBOL(timestamp_truncate);
/**
* current_time - Return FS time
* @inode: inode.
*
* Return the current time truncated to the time granularity supported by
* the fs.
*
* Note that inode and inode->sb cannot be NULL.
* Otherwise, the function warns and returns time without truncation.
*/
struct timespec64 current_time(struct inode *inode)
{
struct timespec64 now;
ktime_get_coarse_real_ts64(&now);
return timestamp_truncate(now, inode);
}
EXPORT_SYMBOL(current_time);
/**
* inode_set_ctime_current - set the ctime to current_time
* @inode: inode
*
* Set the inode->i_ctime to the current value for the inode. Returns
* the current value that was assigned to i_ctime.
* Set the inode's ctime to the current value for the inode. Returns the
* current value that was assigned. If this is not a multigrain inode, then we
* set it to the later of the coarse time and floor value.
*
* If it is multigrain, then we first see if the coarse-grained timestamp is
* distinct from what is already there. If so, then use that. Otherwise, get a
* fine-grained timestamp.
*
* After that, try to swap the new value into i_ctime_nsec. Accept the
* resulting ctime, regardless of the outcome of the swap. If it has
* already been replaced, then that timestamp is later than the earlier
* unacceptable one, and is thus acceptable.
*/
struct timespec64 inode_set_ctime_current(struct inode *inode)
{
struct timespec64 now = current_time(inode);
struct timespec64 now;
u32 cns, cur;
inode_set_ctime_to_ts(inode, now);
ktime_get_coarse_real_ts64_mg(&now);
now = timestamp_truncate(now, inode);
/* Just return that if this is not a multigrain fs */
if (!is_mgtime(inode)) {
inode_set_ctime_to_ts(inode, now);
goto out;
}
/*
* A fine-grained time is only needed if someone has queried
* for timestamps, and the current coarse grained time isn't
* later than what's already there.
*/
cns = smp_load_acquire(&inode->i_ctime_nsec);
if (cns & I_CTIME_QUERIED) {
struct timespec64 ctime = { .tv_sec = inode->i_ctime_sec,
.tv_nsec = cns & ~I_CTIME_QUERIED };
if (timespec64_compare(&now, &ctime) <= 0) {
ktime_get_real_ts64_mg(&now);
now = timestamp_truncate(now, inode);
mgtime_counter_inc(mg_fine_stamps);
}
}
mgtime_counter_inc(mg_ctime_updates);
/* No need to cmpxchg if it's exactly the same */
if (cns == now.tv_nsec && inode->i_ctime_sec == now.tv_sec) {
trace_ctime_xchg_skip(inode, &now);
goto out;
}
cur = cns;
retry:
/* Try to swap the nsec value into place. */
if (try_cmpxchg(&inode->i_ctime_nsec, &cur, now.tv_nsec)) {
/* If swap occurred, then we're (mostly) done */
inode->i_ctime_sec = now.tv_sec;
trace_ctime_ns_xchg(inode, cns, now.tv_nsec, cur);
mgtime_counter_inc(mg_ctime_swaps);
} else {
/*
* Was the change due to someone marking the old ctime QUERIED?
* If so then retry the swap. This can only happen once since
* the only way to clear I_CTIME_QUERIED is to stamp the inode
* with a new ctime.
*/
if (!(cns & I_CTIME_QUERIED) && (cns | I_CTIME_QUERIED) == cur) {
cns = cur;
goto retry;
}
/* Otherwise, keep the existing ctime */
now.tv_sec = inode->i_ctime_sec;
now.tv_nsec = cur & ~I_CTIME_QUERIED;
}
out:
return now;
}
EXPORT_SYMBOL(inode_set_ctime_current);
/**
* inode_set_ctime_deleg - try to update the ctime on a delegated inode
* @inode: inode to update
* @update: timespec64 to set the ctime
*
* Attempt to atomically update the ctime on behalf of a delegation holder.
*
* The nfs server can call back the holder of a delegation to get updated
* inode attributes, including the mtime. When updating the mtime, update
* the ctime to a value at least equal to that.
*
* This can race with concurrent updates to the inode, in which
* case the update is skipped.
*
* Note that this works even when multigrain timestamps are not enabled,
* so it is used in either case.
*/
struct timespec64 inode_set_ctime_deleg(struct inode *inode, struct timespec64 update)
{
struct timespec64 now, cur_ts;
u32 cur, old;
/* pairs with try_cmpxchg below */
cur = smp_load_acquire(&inode->i_ctime_nsec);
cur_ts.tv_nsec = cur & ~I_CTIME_QUERIED;
cur_ts.tv_sec = inode->i_ctime_sec;
/* If the update is older than the existing value, skip it. */
if (timespec64_compare(&update, &cur_ts) <= 0)
return cur_ts;
ktime_get_coarse_real_ts64_mg(&now);
/* Clamp the update to "now" if it's in the future */
if (timespec64_compare(&update, &now) > 0)
update = now;
update = timestamp_truncate(update, inode);
/* No need to update if the values are already the same */
if (timespec64_equal(&update, &cur_ts))
return cur_ts;
/*
* Try to swap the nsec value into place. If it fails, that means
* it raced with an update due to a write or similar activity. That
* stamp takes precedence, so just skip the update.
*/
retry:
old = cur;
if (try_cmpxchg(&inode->i_ctime_nsec, &cur, update.tv_nsec)) {
inode->i_ctime_sec = update.tv_sec;
mgtime_counter_inc(mg_ctime_swaps);
return update;
}
/*
* Was the change due to another task marking the old ctime QUERIED?
*
* If so, then retry the swap. This can only happen once since
* the only way to clear I_CTIME_QUERIED is to stamp the inode
* with a new ctime.
*/
if (!(old & I_CTIME_QUERIED) && (cur == (old | I_CTIME_QUERIED)))
goto retry;
/* Otherwise, it was a new timestamp. */
cur_ts.tv_sec = inode->i_ctime_sec;
cur_ts.tv_nsec = cur & ~I_CTIME_QUERIED;
return cur_ts;
}
EXPORT_SYMBOL(inode_set_ctime_deleg);
/**
* in_group_or_capable - check whether caller is CAP_FSETID privileged
* @idmap: idmap of the mount @inode was found from

46
fs/stat.c
View File

@ -23,9 +23,45 @@
#include <linux/uaccess.h>
#include <asm/unistd.h>
#include <trace/events/timestamp.h>
#include "internal.h"
#include "mount.h"
/**
* fill_mg_cmtime - Fill in the mtime and ctime and flag ctime as QUERIED
* @stat: where to store the resulting values
* @request_mask: STATX_* values requested
* @inode: inode from which to grab the c/mtime
*
* Given @inode, grab the ctime and mtime out if it and store the result
* in @stat. When fetching the value, flag it as QUERIED (if not already)
* so the next write will record a distinct timestamp.
*
* NB: The QUERIED flag is tracked in the ctime, but we set it there even
* if only the mtime was requested, as that ensures that the next mtime
* change will be distinct.
*/
void fill_mg_cmtime(struct kstat *stat, u32 request_mask, struct inode *inode)
{
atomic_t *pcn = (atomic_t *)&inode->i_ctime_nsec;
/* If neither time was requested, then don't report them */
if (!(request_mask & (STATX_CTIME|STATX_MTIME))) {
stat->result_mask &= ~(STATX_CTIME|STATX_MTIME);
return;
}
stat->mtime = inode_get_mtime(inode);
stat->ctime.tv_sec = inode->i_ctime_sec;
stat->ctime.tv_nsec = (u32)atomic_read(pcn);
if (!(stat->ctime.tv_nsec & I_CTIME_QUERIED))
stat->ctime.tv_nsec = ((u32)atomic_fetch_or(I_CTIME_QUERIED, pcn));
stat->ctime.tv_nsec &= ~I_CTIME_QUERIED;
trace_fill_mg_cmtime(inode, &stat->ctime, &stat->mtime);
}
EXPORT_SYMBOL(fill_mg_cmtime);
/**
* generic_fillattr - Fill in the basic attributes from the inode struct
* @idmap: idmap of the mount the inode was found from
@ -58,8 +94,14 @@ void generic_fillattr(struct mnt_idmap *idmap, u32 request_mask,
stat->rdev = inode->i_rdev;
stat->size = i_size_read(inode);
stat->atime = inode_get_atime(inode);
stat->mtime = inode_get_mtime(inode);
stat->ctime = inode_get_ctime(inode);
if (is_mgtime(inode)) {
fill_mg_cmtime(stat, request_mask, inode);
} else {
stat->ctime = inode_get_ctime(inode);
stat->mtime = inode_get_mtime(inode);
}
stat->blksize = i_blocksize(inode);
stat->blocks = inode->i_blocks;

6
fs/xfs/libxfs/xfs_trans_inode.c
View File

@ -62,12 +62,12 @@ xfs_trans_ichgtime(
ASSERT(tp);
xfs_assert_ilocked(ip, XFS_ILOCK_EXCL);
tv = current_time(inode);
/* If the mtime changes, then ctime must also change */
ASSERT(flags & XFS_ICHGTIME_CHG);
tv = inode_set_ctime_current(inode);
if (flags & XFS_ICHGTIME_MOD)
inode_set_mtime_to_ts(inode, tv);
if (flags & XFS_ICHGTIME_CHG)
inode_set_ctime_to_ts(inode, tv);
if (flags & XFS_ICHGTIME_ACCESS)
inode_set_atime_to_ts(inode, tv);
if (flags & XFS_ICHGTIME_CREATE)

10
fs/xfs/xfs_iops.c
View File

@ -597,8 +597,9 @@ xfs_vn_getattr(
stat->gid = vfsgid_into_kgid(vfsgid);
stat->ino = ip->i_ino;
stat->atime = inode_get_atime(inode);
stat->mtime = inode_get_mtime(inode);
stat->ctime = inode_get_ctime(inode);
fill_mg_cmtime(stat, request_mask, inode);
stat->blocks = XFS_FSB_TO_BB(mp, ip->i_nblocks + ip->i_delayed_blks);
if (xfs_has_v3inodes(mp)) {
@ -608,11 +609,6 @@ xfs_vn_getattr(
}
}
if ((request_mask & STATX_CHANGE_COOKIE) && IS_I_VERSION(inode)) {
stat->change_cookie = inode_query_iversion(inode);
stat->result_mask |= STATX_CHANGE_COOKIE;
}
/*
* Note: If you add another clause to set an attribute flag, please
* update attributes_mask below.

2
fs/xfs/xfs_super.c
View File

@ -2063,7 +2063,7 @@ static struct file_system_type xfs_fs_type = {
.init_fs_context = xfs_init_fs_context,
.parameters = xfs_fs_parameters,
.kill_sb = xfs_kill_sb,
.fs_flags = FS_REQUIRES_DEV | FS_ALLOW_IDMAP,
.fs_flags = FS_REQUIRES_DEV | FS_ALLOW_IDMAP | FS_MGTIME,
};
MODULE_ALIAS_FS("xfs");

36
include/linux/fs.h
View File

@ -1584,6 +1584,8 @@ static inline bool fsuidgid_has_mapping(struct super_block *sb,
struct timespec64 current_time(struct inode *inode);
struct timespec64 inode_set_ctime_current(struct inode *inode);
struct timespec64 inode_set_ctime_deleg(struct inode *inode,
struct timespec64 update);
static inline time64_t inode_get_atime_sec(const struct inode *inode)
{
@ -1653,6 +1655,17 @@ static inline struct timespec64 inode_set_mtime(struct inode *inode,
return inode_set_mtime_to_ts(inode, ts);
}
/*
* Multigrain timestamps
*
* Conditionally use fine-grained ctime and mtime timestamps when there
* are users actively observing them via getattr. The primary use-case
* for this is NFS clients that use the ctime to distinguish between
* different states of the file, and that are often fooled by multiple
* operations that occur in the same coarse-grained timer tick.
*/
#define I_CTIME_QUERIED ((u32)BIT(31))
static inline time64_t inode_get_ctime_sec(const struct inode *inode)
{
return inode->i_ctime_sec;
@ -1660,7 +1673,7 @@ static inline time64_t inode_get_ctime_sec(const struct inode *inode)
static inline long inode_get_ctime_nsec(const struct inode *inode)
{
return inode->i_ctime_nsec;
return inode->i_ctime_nsec & ~I_CTIME_QUERIED;
}
static inline struct timespec64 inode_get_ctime(const struct inode *inode)
@ -1671,13 +1684,7 @@ static inline struct timespec64 inode_get_ctime(const struct inode *inode)
return ts;
}
static inline struct timespec64 inode_set_ctime_to_ts(struct inode *inode,
struct timespec64 ts)
{
inode->i_ctime_sec = ts.tv_sec;
inode->i_ctime_nsec = ts.tv_nsec;
return ts;
}
struct timespec64 inode_set_ctime_to_ts(struct inode *inode, struct timespec64 ts);
/**
* inode_set_ctime - set the ctime in the inode
@ -2542,6 +2549,7 @@ struct file_system_type {
#define FS_USERNS_MOUNT 8 /* Can be mounted by userns root */
#define FS_DISALLOW_NOTIFY_PERM 16 /* Disable fanotify permission events */
#define FS_ALLOW_IDMAP 32 /* FS has been updated to handle vfs idmappings. */
#define FS_MGTIME 64 /* FS uses multigrain timestamps */
#define FS_RENAME_DOES_D_MOVE 32768 /* FS will handle d_move() during rename() internally. */
int (*init_fs_context)(struct fs_context *);
const struct fs_parameter_spec *parameters;
@ -2565,6 +2573,17 @@ struct file_system_type {
#define MODULE_ALIAS_FS(NAME) MODULE_ALIAS("fs-" NAME)
/**
* is_mgtime: is this inode using multigrain timestamps
* @inode: inode to test for multigrain timestamps
*
* Return true if the inode uses multigrain timestamps, false otherwise.
*/
static inline bool is_mgtime(const struct inode *inode)
{
return inode->i_sb->s_type->fs_flags & FS_MGTIME;
}
extern struct dentry *mount_bdev(struct file_system_type *fs_type,
int flags, const char *dev_name, void *data,
int (*fill_super)(struct super_block *, void *, int));
@ -3321,6 +3340,7 @@ extern void page_put_link(void *);
extern int page_symlink(struct inode *inode, const char *symname, int len);
extern const struct inode_operations page_symlink_inode_operations;
extern void kfree_link(void *);
void fill_mg_cmtime(struct kstat *stat, u32 request_mask, struct inode *inode);
void generic_fillattr(struct mnt_idmap *, u32, struct inode *, struct kstat *);
void generic_fill_statx_attr(struct inode *inode, struct kstat *stat);
void generic_fill_statx_atomic_writes(struct kstat *stat,

124
include/trace/events/timestamp.h Normal file
View File

@ -0,0 +1,124 @@
/* SPDX-License-Identifier: GPL-2.0 */
#undef TRACE_SYSTEM
#define TRACE_SYSTEM timestamp
#if !defined(_TRACE_TIMESTAMP_H) || defined(TRACE_HEADER_MULTI_READ)
#define _TRACE_TIMESTAMP_H
#include <linux/tracepoint.h>
#include <linux/fs.h>
#define CTIME_QUERIED_FLAGS \
{ I_CTIME_QUERIED, "Q" }
DECLARE_EVENT_CLASS(ctime,
TP_PROTO(struct inode *inode,
struct timespec64 *ctime),
TP_ARGS(inode, ctime),
TP_STRUCT__entry(
__field(dev_t, dev)
__field(ino_t, ino)
__field(time64_t, ctime_s)
__field(u32, ctime_ns)
__field(u32, gen)
),
TP_fast_assign(
__entry->dev = inode->i_sb->s_dev;
__entry->ino = inode->i_ino;
__entry->gen = inode->i_generation;
__entry->ctime_s = ctime->tv_sec;
__entry->ctime_ns = ctime->tv_nsec;
),
TP_printk("ino=%d:%d:%ld:%u ctime=%lld.%u",
MAJOR(__entry->dev), MINOR(__entry->dev), __entry->ino, __entry->gen,
__entry->ctime_s, __entry->ctime_ns
)
);
DEFINE_EVENT(ctime, inode_set_ctime_to_ts,
TP_PROTO(struct inode *inode,
struct timespec64 *ctime),
TP_ARGS(inode, ctime));
DEFINE_EVENT(ctime, ctime_xchg_skip,
TP_PROTO(struct inode *inode,
struct timespec64 *ctime),
TP_ARGS(inode, ctime));
TRACE_EVENT(ctime_ns_xchg,
TP_PROTO(struct inode *inode,
u32 old,
u32 new,
u32 cur),
TP_ARGS(inode, old, new, cur),
TP_STRUCT__entry(
__field(dev_t, dev)
__field(ino_t, ino)
__field(u32, gen)
__field(u32, old)
__field(u32, new)
__field(u32, cur)
),
TP_fast_assign(
__entry->dev = inode->i_sb->s_dev;
__entry->ino = inode->i_ino;
__entry->gen = inode->i_generation;
__entry->old = old;
__entry->new = new;
__entry->cur = cur;
),
TP_printk("ino=%d:%d:%ld:%u old=%u:%s new=%u cur=%u:%s",
MAJOR(__entry->dev), MINOR(__entry->dev), __entry->ino, __entry->gen,
__entry->old & ~I_CTIME_QUERIED,
__print_flags(__entry->old & I_CTIME_QUERIED, "|", CTIME_QUERIED_FLAGS),
__entry->new,
__entry->cur & ~I_CTIME_QUERIED,
__print_flags(__entry->cur & I_CTIME_QUERIED, "|", CTIME_QUERIED_FLAGS)
)
);
TRACE_EVENT(fill_mg_cmtime,
TP_PROTO(struct inode *inode,
struct timespec64 *ctime,
struct timespec64 *mtime),
TP_ARGS(inode, ctime, mtime),
TP_STRUCT__entry(
__field(dev_t, dev)
__field(ino_t, ino)
__field(time64_t, ctime_s)
__field(time64_t, mtime_s)
__field(u32, ctime_ns)
__field(u32, mtime_ns)
__field(u32, gen)
),
TP_fast_assign(
__entry->dev = inode->i_sb->s_dev;
__entry->ino = inode->i_ino;
__entry->gen = inode->i_generation;
__entry->ctime_s = ctime->tv_sec;
__entry->mtime_s = mtime->tv_sec;
__entry->ctime_ns = ctime->tv_nsec;
__entry->mtime_ns = mtime->tv_nsec;
),
TP_printk("ino=%d:%d:%ld:%u ctime=%lld.%u mtime=%lld.%u",
MAJOR(__entry->dev), MINOR(__entry->dev), __entry->ino, __entry->gen,
__entry->ctime_s, __entry->ctime_ns,
__entry->mtime_s, __entry->mtime_ns
)
);
#endif /* _TRACE_TIMESTAMP_H */
/* This part must be outside protection */
#include <trace/define_trace.h>

2
mm/shmem.c
View File

@ -4951,7 +4951,7 @@ static struct file_system_type shmem_fs_type = {
.parameters = shmem_fs_parameters,
#endif
.kill_sb = kill_litter_super,
.fs_flags = FS_USERNS_MOUNT | FS_ALLOW_IDMAP,
.fs_flags = FS_USERNS_MOUNT | FS_ALLOW_IDMAP | FS_MGTIME,
};
void __init shmem_init(void)