linux/fs/nfs
Chuck Lever d869da91cc nfs/blocklayout: Fix premature PR key unregistration
During generic/069 runs with pNFS SCSI layouts, the NFS client emits
the following in the system journal:

kernel: pNFS: failed to open device /dev/disk/by-id/dm-uuid-mpath-0x6001405e3366f045b7949eb8e4540b51 (-2)
kernel: pNFS: using block device sdb (reservation key 0x666b60901e7b26b3)
kernel: pNFS: failed to open device /dev/disk/by-id/dm-uuid-mpath-0x6001405e3366f045b7949eb8e4540b51 (-2)
kernel: pNFS: using block device sdb (reservation key 0x666b60901e7b26b3)
kernel: sd 6:0:0:1: reservation conflict
kernel: sd 6:0:0:1: [sdb] tag#16 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
kernel: sd 6:0:0:1: [sdb] tag#16 CDB: Write(10) 2a 00 00 00 00 50 00 00 08 00
kernel: reservation conflict error, dev sdb, sector 80 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
kernel: sd 6:0:0:1: reservation conflict
kernel: sd 6:0:0:1: reservation conflict
kernel: sd 6:0:0:1: [sdb] tag#18 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
kernel: sd 6:0:0:1: [sdb] tag#17 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
kernel: sd 6:0:0:1: [sdb] tag#18 CDB: Write(10) 2a 00 00 00 00 60 00 00 08 00
kernel: sd 6:0:0:1: [sdb] tag#17 CDB: Write(10) 2a 00 00 00 00 58 00 00 08 00
kernel: reservation conflict error, dev sdb, sector 96 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 0
kernel: reservation conflict error, dev sdb, sector 88 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 0
systemd[1]: fstests-generic-069.scope: Deactivated successfully.
systemd[1]: fstests-generic-069.scope: Consumed 5.092s CPU time.
systemd[1]: media-test.mount: Deactivated successfully.
systemd[1]: media-scratch.mount: Deactivated successfully.
kernel: sd 6:0:0:1: reservation conflict
kernel: failed to unregister PR key.

This appears to be due to a race. bl_alloc_lseg() calls this:

561 static struct nfs4_deviceid_node *
562 bl_find_get_deviceid(struct nfs_server *server,
563                 const struct nfs4_deviceid *id, const struct cred *cred,
564                 gfp_t gfp_mask)
565 {
566         struct nfs4_deviceid_node *node;
567         unsigned long start, end;
568
569 retry:
570         node = nfs4_find_get_deviceid(server, id, cred, gfp_mask);
571         if (!node)
572                 return ERR_PTR(-ENODEV);

nfs4_find_get_deviceid() does a lookup without the spin lock first.
If it can't find a matching deviceid, it creates a new device_info
(which calls bl_alloc_deviceid_node, and that registers the device's
PR key).

Then it takes the nfs4_deviceid_lock and looks up the deviceid again.
If it finds it this time, bl_find_get_deviceid() frees the spare
(new) device_info, which unregisters the PR key for the same device.

Any subsequent I/O from this client on that device gets EBADE.

The umount later unregisters the device's PR key again.

To prevent this problem, register the PR key after the deviceid_node
lookup.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Benjamin Coddington <bcodding@redhat.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
2024-07-08 13:47:27 -04:00
..
blocklayout nfs/blocklayout: Fix premature PR key unregistration 2024-07-08 13:47:27 -04:00
filelayout pNFS: rework pnfs_generic_pg_check_layout to check IO range 2024-05-21 08:34:15 -04:00
flexfilelayout NFSv4/pNFS: Do layout state recovery upon reboot 2024-07-08 13:47:26 -04:00
cache_lib.c NFS client updates for Linux 4.15 2017-11-17 14:18:00 -08:00
cache_lib.h NFS client updates for Linux 4.15 2017-11-17 14:18:00 -08:00
callback_proc.c pNFS: Add a flag argument to pnfs_destroy_layouts_byclid() 2024-07-08 13:47:26 -04:00
callback_xdr.c NFSv4: Add CB_GETATTR support for delegated attributes 2024-07-08 13:47:25 -04:00
callback.c nfsd: stop setting ->pg_stats for unused stats 2024-03-01 09:12:08 -05:00
callback.h NFSv4: Add CB_GETATTR support for delegated attributes 2024-07-08 13:47:25 -04:00
client.c NFS: remove unused variable nfs_rpcstat 2024-03-09 09:14:51 -05:00
delegation.c NFSv4: Delegreturn must set m/atime when they are delegated 2024-07-08 13:47:25 -04:00
delegation.h NFSv4: Delegreturn must set m/atime when they are delegated 2024-07-08 13:47:25 -04:00
dir.c NFSv4: Add a flags argument to the 'have_delegation' callback 2024-07-08 13:47:25 -04:00
direct.c nfs: drop the incorrect assertion in nfs_swap_rw() 2024-06-24 20:52:11 -07:00
dns_resolve.c NFS: Move common includes outside ifdef 2023-08-24 13:24:15 -04:00
dns_resolve.h NFS: Avoid memcpy() run-time warning for struct sockaddr overflows 2022-10-27 15:52:10 -04:00
export.c nfsd: allow reaping files still under writeback 2023-04-26 09:04:59 -04:00
file.c NFSv4: Add a flags argument to the 'have_delegation' callback 2024-07-08 13:47:25 -04:00
fs_context.c nfs: keep server info for remounts 2024-05-20 11:37:15 -04:00
fscache.c NFS: Fix nfs_netfs_issue_read() xarray locking for writeback interrupt 2024-03-09 09:14:38 -05:00
fscache.h mm: Remove the PG_fscache alias for PG_private_2 2024-04-29 15:01:42 +01:00
getroot.c NFS: Remove the nfs4_label argument from nfs_setsecurity 2021-11-05 14:54:40 -04:00
inode.c NFSv4: Fix up delegated attributes in nfs_setattr 2024-07-08 13:47:25 -04:00
internal.h nfs: fix undefined behavior in nfs_block_bits() 2024-05-21 08:34:15 -04:00
io.c NFS: Fix up incorrect documentation 2021-04-05 09:04:20 -04:00
iostat.h mm: change inlined allocation helpers to account at the call site 2024-04-25 20:55:59 -07:00
Kconfig NFS: Don't enable NFS v2 by default 2024-05-20 11:37:15 -04:00
Makefile nfs: Convert to new fscache volume/cookie API 2022-01-10 11:53:25 +00:00
mount_clnt.c NFS: remove unused struct 'mnt_fhstatus' 2024-07-08 13:47:24 -04:00
namespace.c fs: pass the request_mask to generic_fillattr 2023-08-09 08:56:36 +02:00
netns.h nfs: make the rpc_stat per net namespace 2024-03-09 09:14:51 -05:00
nfs2super.c fs: nfs: add missing MODULE_DESCRIPTION() macros 2024-07-08 13:47:24 -04:00
nfs2xdr.c NFS: Guard against READDIR loop when entry names exceed MAXNAMELEN 2023-08-30 11:08:27 -04:00
nfs3_fs.h fs: drop unused posix acl handlers 2023-03-06 09:57:12 +01:00
nfs3acl.c Mainly singleton patches all over the place. Series of note are: 2023-04-27 19:57:00 -07:00
nfs3client.c NFS: enable nconnect for RDMA 2024-03-09 09:14:51 -05:00
nfs3proc.c Return the delegation when deleting sillyrenamed files 2024-07-08 13:47:26 -04:00
nfs3super.c fs: nfs: add missing MODULE_DESCRIPTION() macros 2024-07-08 13:47:24 -04:00
nfs3xdr.c NFS: Guard against READDIR loop when entry names exceed MAXNAMELEN 2023-08-30 11:08:27 -04:00
nfs4_fs.h NFSv4.1: constify the stateid argument in nfs41_test_stateid() 2024-07-08 13:47:26 -04:00
nfs4client.c NFS: enable nconnect for RDMA 2024-03-09 09:14:51 -05:00
nfs4file.c filelock: split leases out of struct file_lock 2024-02-05 13:11:44 +01:00
nfs4getroot.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
nfs4idmap.c cred: Do not default to init_cred in prepare_kernel_cred() 2022-11-01 10:04:52 -07:00
nfs4idmap.h NFS: Move nfs_idmap.h into fs/nfs/ 2015-04-23 15:16:14 -04:00
nfs4namespace.c NFS: Avoid memcpy() run-time warning for struct sockaddr overflows 2022-10-27 15:52:10 -04:00
nfs4proc.c NFSv4/pnfs: Give nfs4_proc_layoutreturn() a flags argument 2024-07-08 13:47:26 -04:00
nfs4renewd.c treewide: remove editor modelines and cruft 2021-05-07 00:26:34 -07:00
nfs4session.c NFSv4: Sanity check the parameters in nfs41_update_target_slotid() 2021-11-07 09:23:14 -05:00
nfs4session.h NFSv4: Sanity check the parameters in nfs41_update_target_slotid() 2021-11-07 09:23:14 -05:00
nfs4state.c NFSv4/pNFS: Do layout state recovery upon reboot 2024-07-08 13:47:26 -04:00
nfs4super.c fs: nfs: add missing MODULE_DESCRIPTION() macros 2024-07-08 13:47:24 -04:00
nfs4sysctl.c fs: Remove the now superfluous sentinel elements from ctl_table array 2023-12-28 04:57:57 -08:00
nfs4trace.c pnfs/filelayout: add tracepoint to getdeviceinfo 2024-02-28 16:18:19 -05:00
nfs4trace.h tracing/treewide: Remove second parameter of __assign_str() 2024-05-22 20:14:47 -04:00
nfs4xdr.c NFSv4: Clean up encode_nfs4_stateid() 2024-07-08 13:47:26 -04:00
nfs42.h NFSv4.2: fix listxattr maximum XDR buffer size 2024-02-28 16:18:18 -05:00
nfs42proc.c nfs42: client needs to strip file mode's suid/sgid bit after ALLOCATE op 2023-10-11 09:37:48 -04:00
nfs42xattr.c mm, slab: remove last vestiges of SLAB_MEM_SPREAD 2024-03-12 20:32:19 -07:00
nfs42xdr.c NFSv4.2: Rework scratch handling for READ_PLUS (again) 2023-08-23 15:58:47 -04:00
nfs.h nfs: move nfs4_xattr_handlers to .rodata 2023-10-09 16:24:20 +02:00
nfsroot.c NFS: Fix an off by one in root_nfs_cat() 2024-03-09 09:14:51 -05:00
nfstrace.c NFSv4: Catch and trace server filehandle encoding errors 2021-04-14 09:36:29 -04:00
nfstrace.h tracing/treewide: Remove second parameter of __assign_str() 2024-05-22 20:14:47 -04:00
pagelist.c nfs: Avoid flushing many pages with NFS_FILE_SYNC 2024-05-24 12:26:14 -04:00
pnfs_dev.c NFSv4/pnfs: minor fix for cleanup path in nfs4_get_device_info 2023-08-24 13:24:15 -04:00
pnfs_nfs.c NFSv4.1/pnfs: fix NFS with TLS in pnfs 2024-03-09 09:14:51 -05:00
pnfs.c NFSv4/pNFS: Do layout state recovery upon reboot 2024-07-08 13:47:26 -04:00
pnfs.h NFSv4/pNFS: Do layout state recovery upon reboot 2024-07-08 13:47:26 -04:00
proc.c Return the delegation when deleting sillyrenamed files 2024-07-08 13:47:26 -04:00
read.c NFSv4: Add support for delegated atime and mtime attributes 2024-07-08 13:47:25 -04:00
super.c NFS: make sure lock/nolock overriding local_lock mount option 2024-05-20 11:10:27 -04:00
symlink.c nfs: propagate readlink errors in nfs_symlink_filler 2024-05-22 19:25:00 -04:00
sysctl.c fs: Remove the now superfluous sentinel elements from ctl_table array 2023-12-28 04:57:57 -08:00
sysfs.c NFS: Fix sysfs server name memory leak 2023-08-19 10:26:29 -04:00
sysfs.h NFS: Add sysfs links to sunrpc clients for nfs_clients 2023-06-19 15:04:13 -04:00
unlink.c Return the delegation when deleting sillyrenamed files 2024-07-08 13:47:26 -04:00
write.c NFSv4: Add support for delegated atime and mtime attributes 2024-07-08 13:47:25 -04:00