linux-next/block
Tejun Heo e08d02aa5f blk-iocost: implement Andy's method for donation weight updates
iocost implements work conservation by reducing iocg->inuse and propagating
the adjustment upwards proportionally. However, while I knew the target
absolute hierarchical proportion - adjusted hweight_inuse, I couldn't figure
out how to determine the iocg->inuse adjustment to achieve that and
approximated the adjustment by scaling iocg->inuse using the proportion of
the needed hweight_inuse changes.

When nested, these scalings aren't accurate even when adjusting a single
node as the donating node also receives the benefit of the donated portion.
When multiple nodes are donating as they often do, they can be wildly wrong.

iocost employed various safety nets to combat the inaccuracies. There are
ample buffers in determining how much to donate, the adjustments are
conservative and gradual. While it can achieve a reasonable level of work
conservation in simple scenarios, the inaccuracies can easily add up leading
to significant loss of total work. This in turn makes it difficult to
closely cap vrate as vrate adjustment is needed to compensate for the loss
of work. The combination of inaccurate donation calculations and vrate
adjustments can lead to wide fluctuations and clunky overall behaviors.

Andy Newell devised a method to calculate the needed ->inuse updates to
achieve the target hweight_inuse's. The method is compatible with the
proportional inuse adjustment propagation which allows all hot path
operations to be local to each iocg.

To roughly summarize, Andy's method divides the tree into donating and
non-donating parts, calculates global donation rate which is used to
determine the target hweight_inuse for each node, and then derives per-level
proportions. There's non-trivial amount of math involved. Please refer to
the following pdfs for detailed descriptions.

  https://drive.google.com/file/d/1PsJwxPFtjUnwOY1QJ5AeICCcsL7BM3bo
  https://drive.google.com/file/d/1vONz1-fzVO7oY5DXXsLjSxEtYYQbOvsE
  https://drive.google.com/file/d/1WcrltBOSPN0qXVdBgnKm4mdp9FhuEFQN

This patch implements Andy's method in transfer_surpluses(). This makes the
donation calculations accurate per cycle and enables further improvements in
other parts of the donation logic.

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Andy Newell <newella@fb.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-09-01 19:38:32 -06:00
..
partitions block: remove the disk argument to delete_partition 2020-09-01 19:38:25 -06:00
badblocks.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
bfq-cgroup.c bfq: fix blkio cgroup leakage v4 2020-08-18 07:48:08 -07:00
bfq-iosched.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
bfq-iosched.h bfq: fix blkio cgroup leakage v4 2020-08-18 07:48:08 -07:00
bfq-wf2q.c bfq: fix blkio cgroup leakage v4 2020-08-18 07:48:08 -07:00
bio-integrity.c block: make function __bio_integrity_free() static 2020-07-02 12:38:18 -06:00
bio.c block: Fix page_is_mergeable() for compound pages 2020-08-17 19:35:53 -07:00
blk-cgroup-rwstat.c blk-cgroup: separate out blkg_rwstat under CONFIG_BLK_CGROUP_RWSTAT 2019-11-07 12:28:13 -07:00
blk-cgroup-rwstat.h blk-cgroup: separate out blkg_rwstat under CONFIG_BLK_CGROUP_RWSTAT 2019-11-07 12:28:13 -07:00
blk-cgroup.c blkcg: fix memleak for iolatency 2020-08-21 17:14:27 -06:00
blk-core.c blk-mq: use BLK_MQ_NO_TAG for no tag 2020-09-01 16:49:26 -06:00
blk-crypto-fallback.c block: rename generic_make_request to submit_bio_noacct 2020-07-01 07:27:24 -06:00
blk-crypto-internal.h block: blk-crypto-fallback for Inline Encryption 2020-05-14 09:48:03 -06:00
blk-crypto.c block: rename generic_make_request to submit_bio_noacct 2020-07-01 07:27:24 -06:00
blk-exec.c block: add a blk_account_io_merge_bio helper 2020-05-27 05:21:23 -06:00
blk-flush.c block: fix double account of flush request's driver tag 2020-08-11 13:53:32 -06:00
blk-integrity.c block: Make blk-integrity preclude hardware inline encryption 2020-05-14 09:48:03 -06:00
blk-ioc.c block: remove retry loop in ioc_release_fn() 2020-07-16 10:22:15 -06:00
blk-iocost.c blk-iocost: implement Andy's method for donation weight updates 2020-09-01 19:38:32 -06:00
blk-iolatency.c blk-iolatency: only call ktime_get() if needed 2020-07-01 08:02:38 -06:00
blk-lib.c block: check queue's limits.discard_granularity in __blkdev_issue_discard() 2020-08-05 17:15:47 -06:00
blk-map.c block: remove the BIO_USER_MAPPED flag 2020-09-01 16:49:26 -06:00
blk-merge.c block: Add a new helper to attempt to merge a bio 2020-09-01 16:49:26 -06:00
blk-mq-cpumap.c blk-mq: balance mapping between present CPUs and queues 2019-08-04 21:43:12 -06:00
blk-mq-debugfs-zoned.c block: Cleanup license notice 2019-01-17 21:21:40 -07:00
blk-mq-debugfs.c for-5.9/block-20200802 2020-08-03 11:57:03 -07:00
blk-mq-debugfs.h blk-mq: no need to check return value of debugfs_create functions 2019-06-13 03:00:30 -06:00
blk-mq-pci.c block: Fix blk_mq_*_map_queues() kernel-doc headers 2019-05-31 15:12:34 -06:00
blk-mq-rdma.c block: Fix blk_mq_*_map_queues() kernel-doc headers 2019-05-31 15:12:34 -06:00
blk-mq-sched.c blk-mq: use BLK_MQ_NO_TAG for no tag 2020-09-01 16:49:26 -06:00
blk-mq-sched.h block: blk-mq: Remove blk_mq_sched_started_request and started_request 2019-07-23 07:25:09 -06:00
blk-mq-sysfs.c blk-mq: make sure that line break can be printed 2019-11-04 07:14:10 -07:00
blk-mq-tag.c blk-mq: move blk_mq_get_driver_tag into blk-mq.c 2020-06-30 12:57:59 -06:00
blk-mq-tag.h blk-mq: centralise related handling into blk_mq_get_driver_tag 2020-07-08 16:06:42 -06:00
blk-mq-virtio.c blk-mq: Fix typo in comment 2020-03-17 20:55:21 +01:00
blk-mq.c blk-mq: insert request not through ->queue_rq into sw/scheduler queue 2020-08-21 17:09:22 -06:00
blk-mq.h Revert "blk-mq: put driver tag when this request is completed" 2020-07-01 22:58:32 -06:00
blk-pm.c scsi: block: pm: Simplify resume handling 2020-07-24 22:09:55 -04:00
blk-pm.h block: remove the queue_lock indirection 2018-11-15 12:17:28 -07:00
blk-rq-qos.c Revert "blk-rq-qos: remove redundant finish_wait to rq_qos_wait." 2020-07-15 09:33:37 -06:00
blk-rq-qos.h blk-rq-qos: fix first node deletion of rq_qos_del() 2019-10-15 10:13:13 -06:00
blk-settings.c block: remove blk_queue_stack_limits 2020-07-20 15:38:52 -06:00
blk-stat.c blk-stat: make q->stats->lock irqsafe 2020-09-01 16:48:46 -06:00
blk-stat.h block: deactivate blk_stat timer in wbt_disable_default() 2018-12-12 06:47:51 -07:00
blk-sysfs.c block: add max_active_zones to blk-sysfs 2020-07-15 14:26:11 -06:00
blk-throttle.c block: rename generic_make_request to submit_bio_noacct 2020-07-01 07:27:24 -06:00
blk-timeout.c block: blk-timeout: delete duplicated word 2020-07-31 16:29:47 -06:00
blk-wbt.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
blk-wbt.h blk-wbt: remove wbt_update_limits 2020-05-29 16:30:39 -06:00
blk-zoned.c block: don't do revalidate zones on invalid devices 2020-08-03 09:24:04 -06:00
blk.h block: remove the disk argument to delete_partition 2020-09-01 19:38:25 -06:00
bounce.c block: rename generic_make_request to submit_bio_noacct 2020-07-01 07:27:24 -06:00
bsg-lib.c bsg-lib: convert comma to semicolon 2020-08-16 20:07:12 -07:00
bsg.c compat_ioctl: bsg: add handler 2020-01-03 09:33:21 +01:00
cmdline-parser.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
elevator.c block: elevator: delete duplicated word and fix typos 2020-07-31 16:29:47 -06:00
genhd.c block: remove the unused q argument to part_in_flight and part_in_flight_rw 2020-09-01 19:38:31 -06:00
ioctl.c block: Fix type of first compat_put_{,u}long() argument 2020-05-19 09:40:29 -06:00
ioprio.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
Kconfig blk-wbt: Remove obsolete multiqueue I/O scheduling comment 2020-09-01 16:49:26 -06:00
Kconfig.iosched treewide: replace '---help---' in Kconfig files with 'help' 2020-06-14 01:57:21 +09:00
keyslot-manager.c block/keyslot-manager: use kvfree_sensitive() 2020-06-29 13:24:05 -06:00
kyber-iosched.c block: Move blk_mq_bio_list_merge() into blk-merge.c 2020-09-01 16:49:26 -06:00
Makefile blk-mq: merge blk-softirq.c into blk-mq.c 2020-06-24 09:15:56 -06:00
mq-deadline.c blk-mq: remove the bio argument to ->prepare_request 2020-05-29 10:23:24 -06:00
opal_proto.h block: sed-opal: Change the check condition for regular session validity 2020-03-12 08:00:10 -06:00
scsi_ioctl.c scsi: core: Allow non-root users to perform ZBC commands 2020-03-16 18:26:31 -04:00
sed-opal.c block: sed-opal: Change the check condition for regular session validity 2020-03-12 08:00:10 -06:00
t10-pi.c block: Allow t10-pi to be modular 2020-01-06 20:59:04 -07:00