2019-06-01 08:08:55 +00:00
|
|
|
// SPDX-License-Identifier: GPL-2.0-only
|
2015-10-02 14:04:01 +00:00
|
|
|
/*
|
|
|
|
|
* CPPC (Collaborative Processor Performance Control) driver for
|
|
|
|
|
* interfacing with the CPUfreq layer and governors. See
|
|
|
|
|
* cppc_acpi.c for CPPC specific methods.
|
|
|
|
|
*
|
|
|
|
|
* (C) Copyright 2014, 2015 Linaro Ltd.
|
|
|
|
|
* Author: Ashwin Chaugule <ashwin.chaugule@linaro.org>
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#define pr_fmt(fmt) "CPPC Cpufreq:" fmt
|
|
|
|
|
|
2020-06-23 10:19:40 +00:00
|
|
|
#include <linux/arch_topology.h>
|
2015-10-02 14:04:01 +00:00
|
|
|
#include <linux/kernel.h>
|
|
|
|
|
#include <linux/module.h>
|
|
|
|
|
#include <linux/delay.h>
|
|
|
|
|
#include <linux/cpu.h>
|
|
|
|
|
#include <linux/cpufreq.h>
|
2020-06-23 10:19:40 +00:00
|
|
|
#include <linux/irq_work.h>
|
|
|
|
|
#include <linux/kthread.h>
|
2018-03-23 10:30:31 +00:00
|
|
|
#include <linux/time.h>
|
2015-10-02 14:04:01 +00:00
|
|
|
#include <linux/vmalloc.h>
|
2020-06-23 10:19:40 +00:00
|
|
|
#include <uapi/linux/sched/types.h>
|
2015-10-02 14:04:01 +00:00
|
|
|
|
2024-10-01 19:35:57 +00:00
|
|
|
#include <linux/unaligned.h>
|
cpufreq: CPPC: Force reporting values in KHz to fix user space interface
When CPPC is being used by ACPI on arm64, user space tools such as
cpupower report CPU frequency values from sysfs that are incorrect.
What the driver was doing was reporting the values given by ACPI tables
in whatever scale was used to provide them. However, the ACPI spec
defines the CPPC values as unitless abstract numbers. Internal kernel
structures such as struct perf_cap, in contrast, expect these values
to be in KHz. When these struct values get reported via sysfs, the
user space tools also assume they are in KHz, causing them to report
incorrect values (for example, reporting a CPU frequency of 1MHz when
it should be 1.8GHz).
The downside is that this approach has some assumptions:
(1) It relies on SMBIOS3 being used, *and* that the Max Frequency
value for a processor is set to a non-zero value.
(2) It assumes that all processors run at the same speed, or that
the CPPC values have all been scaled to reflect relative speed.
This patch retrieves the largest CPU Max Frequency from a type 4 DMI
record that it can find. This may not be an issue, however, as a
sampling of DMI data on x86 and arm64 indicates there is often only
one such record regardless. Since CPPC is relatively new, it is
unclear if the ACPI ASL will always be written to reflect any sort
of relative performance of processors of differing speeds.
(3) It assumes that performance and frequency both scale linearly.
For arm64 servers, this may be sufficient, but it does rely on
firmware values being set correctly. Hence, other approaches will
be considered in the future.
This has been tested on three arm64 servers, with and without DMI, with
and without CPPC support.
Signed-off-by: Al Stone <ahs3@redhat.com>
Signed-off-by: Prashanth Prakash <pprakash@codeaurora.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2016-07-20 21:10:04 +00:00
|
|
|
|
2015-10-02 14:04:01 +00:00
|
|
|
#include <acpi/cppc_acpi.h>
|
|
|
|
|
|
|
|
|
|
/*
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
* This list contains information parsed from per CPU ACPI _CPC and _PSD
|
|
|
|
|
* structures: e.g. the highest and lowest supported performance, capabilities,
|
|
|
|
|
* desired performance, level requested etc. Depending on the share_type, not
|
|
|
|
|
* all CPUs will have an entry in the list.
|
2015-10-02 14:04:01 +00:00
|
|
|
*/
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
static LIST_HEAD(cpu_data_list);
|
|
|
|
|
|
cpufreq: CPPC: add SW BOOST support
To add SW BOOST support for CPPC, we need to get the max frequency of
boost mode and non-boost mode. ACPI spec 6.2 section 8.4.7.1 describes
the following two CPC registers.
"Highest performance is the absolute maximum performance an individual
processor may reach, assuming ideal conditions. This performance level
may not be sustainable for long durations, and may only be achievable if
other platform components are in a specific state; for example, it may
require other processors be in an idle state.
Nominal Performance is the maximum sustained performance level of the
processor, assuming ideal operating conditions. In absence of an
external constraint (power, thermal, etc.) this is the performance level
the platform is expected to be able to maintain continuously. All
processors are expected to be able to sustain their nominal performance
state simultaneously."
To add SW BOOST support for CPPC, we can use Highest Performance as the
max performance in boost mode and Nominal Performance as the max
performance in non-boost mode. If the Highest Performance is greater
than the Nominal Performance, we assume SW BOOST is supported.
The current CPPC driver does not support SW BOOST and use 'Highest
Performance' as the max performance the CPU can achieve. 'Nominal
Performance' is used to convert 'performance' to 'frequency'. That
means, if firmware enable boost and provide a value for Highest
Performance which is greater than Nominal Performance, boost feature is
enabled by default.
Because SW BOOST is disabled by default, so, after this patch, boost
feature is disabled by default even if boost is enabled by firmware.
Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
Suggested-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
[ rjw: Subject ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-05-30 02:08:31 +00:00
|
|
|
static bool boost_supported;
|
2015-10-02 14:04:01 +00:00
|
|
|
|
2022-05-21 03:24:38 +00:00
|
|
|
static struct cpufreq_driver cppc_cpufreq_driver;
|
|
|
|
|
|
2024-09-29 03:32:14 +00:00
|
|
|
#ifdef CONFIG_ACPI_CPPC_CPUFREQ_FIE
|
2022-09-12 20:37:22 +00:00
|
|
|
static enum {
|
|
|
|
|
FIE_UNSET = -1,
|
|
|
|
|
FIE_ENABLED,
|
|
|
|
|
FIE_DISABLED
|
|
|
|
|
} fie_disabled = FIE_UNSET;
|
|
|
|
|
|
|
|
|
|
module_param(fie_disabled, int, 0444);
|
|
|
|
|
MODULE_PARM_DESC(fie_disabled, "Disable Frequency Invariance Engine (FIE)");
|
2020-06-23 10:19:40 +00:00
|
|
|
|
|
|
|
|
/* Frequency invariance support */
|
|
|
|
|
struct cppc_freq_invariance {
|
|
|
|
|
int cpu;
|
|
|
|
|
struct irq_work irq_work;
|
|
|
|
|
struct kthread_work work;
|
|
|
|
|
struct cppc_perf_fb_ctrs prev_perf_fb_ctrs;
|
|
|
|
|
struct cppc_cpudata *cpu_data;
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
static DEFINE_PER_CPU(struct cppc_freq_invariance, cppc_freq_inv);
|
|
|
|
|
static struct kthread_worker *kworker_fie;
|
|
|
|
|
|
|
|
|
|
static int cppc_perf_from_fbctrs(struct cppc_cpudata *cpu_data,
|
|
|
|
|
struct cppc_perf_fb_ctrs *fb_ctrs_t0,
|
|
|
|
|
struct cppc_perf_fb_ctrs *fb_ctrs_t1);
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* cppc_scale_freq_workfn - CPPC arch_freq_scale updater for frequency invariance
|
|
|
|
|
* @work: The work item.
|
|
|
|
|
*
|
|
|
|
|
* The CPPC driver register itself with the topology core to provide its own
|
|
|
|
|
* implementation (cppc_scale_freq_tick()) of topology_scale_freq_tick() which
|
|
|
|
|
* gets called by the scheduler on every tick.
|
|
|
|
|
*
|
|
|
|
|
* Note that the arch specific counters have higher priority than CPPC counters,
|
|
|
|
|
* if available, though the CPPC driver doesn't need to have any special
|
|
|
|
|
* handling for that.
|
|
|
|
|
*
|
|
|
|
|
* On an invocation of cppc_scale_freq_tick(), we schedule an irq work (since we
|
|
|
|
|
* reach here from hard-irq context), which then schedules a normal work item
|
|
|
|
|
* and cppc_scale_freq_workfn() updates the per_cpu arch_freq_scale variable
|
|
|
|
|
* based on the counter updates since the last tick.
|
|
|
|
|
*/
|
|
|
|
|
static void cppc_scale_freq_workfn(struct kthread_work *work)
|
|
|
|
|
{
|
|
|
|
|
struct cppc_freq_invariance *cppc_fi;
|
|
|
|
|
struct cppc_perf_fb_ctrs fb_ctrs = {0};
|
|
|
|
|
struct cppc_cpudata *cpu_data;
|
|
|
|
|
unsigned long local_freq_scale;
|
|
|
|
|
u64 perf;
|
|
|
|
|
|
|
|
|
|
cppc_fi = container_of(work, struct cppc_freq_invariance, work);
|
|
|
|
|
cpu_data = cppc_fi->cpu_data;
|
|
|
|
|
|
|
|
|
|
if (cppc_get_perf_ctrs(cppc_fi->cpu, &fb_ctrs)) {
|
|
|
|
|
pr_warn("%s: failed to read perf counters\n", __func__);
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
perf = cppc_perf_from_fbctrs(cpu_data, &cppc_fi->prev_perf_fb_ctrs,
|
|
|
|
|
&fb_ctrs);
|
2024-09-29 03:32:13 +00:00
|
|
|
if (!perf)
|
|
|
|
|
return;
|
|
|
|
|
|
2020-06-23 10:19:40 +00:00
|
|
|
cppc_fi->prev_perf_fb_ctrs = fb_ctrs;
|
|
|
|
|
|
|
|
|
|
perf <<= SCHED_CAPACITY_SHIFT;
|
|
|
|
|
local_freq_scale = div64_u64(perf, cpu_data->perf_caps.highest_perf);
|
|
|
|
|
|
|
|
|
|
/* This can happen due to counter's overflow */
|
|
|
|
|
if (unlikely(local_freq_scale > 1024))
|
|
|
|
|
local_freq_scale = 1024;
|
|
|
|
|
|
|
|
|
|
per_cpu(arch_freq_scale, cppc_fi->cpu) = local_freq_scale;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void cppc_irq_work(struct irq_work *irq_work)
|
|
|
|
|
{
|
|
|
|
|
struct cppc_freq_invariance *cppc_fi;
|
|
|
|
|
|
|
|
|
|
cppc_fi = container_of(irq_work, struct cppc_freq_invariance, irq_work);
|
|
|
|
|
kthread_queue_work(kworker_fie, &cppc_fi->work);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void cppc_scale_freq_tick(void)
|
|
|
|
|
{
|
|
|
|
|
struct cppc_freq_invariance *cppc_fi = &per_cpu(cppc_freq_inv, smp_processor_id());
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* cppc_get_perf_ctrs() can potentially sleep, call that from the right
|
|
|
|
|
* context.
|
|
|
|
|
*/
|
|
|
|
|
irq_work_queue(&cppc_fi->irq_work);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static struct scale_freq_data cppc_sftd = {
|
|
|
|
|
.source = SCALE_FREQ_SOURCE_CPPC,
|
|
|
|
|
.set_freq_scale = cppc_scale_freq_tick,
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
static void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy)
|
|
|
|
|
{
|
|
|
|
|
struct cppc_freq_invariance *cppc_fi;
|
|
|
|
|
int cpu, ret;
|
|
|
|
|
|
2022-09-12 20:37:22 +00:00
|
|
|
if (fie_disabled)
|
2020-06-23 10:19:40 +00:00
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
for_each_cpu(cpu, policy->cpus) {
|
|
|
|
|
cppc_fi = &per_cpu(cppc_freq_inv, cpu);
|
|
|
|
|
cppc_fi->cpu = cpu;
|
|
|
|
|
cppc_fi->cpu_data = policy->driver_data;
|
|
|
|
|
kthread_init_work(&cppc_fi->work, cppc_scale_freq_workfn);
|
|
|
|
|
init_irq_work(&cppc_fi->irq_work, cppc_irq_work);
|
|
|
|
|
|
|
|
|
|
ret = cppc_get_perf_ctrs(cpu, &cppc_fi->prev_perf_fb_ctrs);
|
|
|
|
|
if (ret) {
|
|
|
|
|
pr_warn("%s: failed to read perf counters for cpu:%d: %d\n",
|
|
|
|
|
__func__, cpu, ret);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Don't abort if the CPU was offline while the driver
|
|
|
|
|
* was getting registered.
|
|
|
|
|
*/
|
|
|
|
|
if (cpu_online(cpu))
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* Register for freq-invariance */
|
|
|
|
|
topology_set_scale_freq_source(&cppc_sftd, policy->cpus);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We free all the resources on policy's removal and not on CPU removal as the
|
|
|
|
|
* irq-work are per-cpu and the hotplug core takes care of flushing the pending
|
|
|
|
|
* irq-works (hint: smpcfd_dying_cpu()) on CPU hotplug. Even if the kthread-work
|
|
|
|
|
* fires on another CPU after the concerned CPU is removed, it won't harm.
|
|
|
|
|
*
|
|
|
|
|
* We just need to make sure to remove them all on policy->exit().
|
|
|
|
|
*/
|
|
|
|
|
static void cppc_cpufreq_cpu_fie_exit(struct cpufreq_policy *policy)
|
|
|
|
|
{
|
|
|
|
|
struct cppc_freq_invariance *cppc_fi;
|
|
|
|
|
int cpu;
|
|
|
|
|
|
2022-09-12 20:37:22 +00:00
|
|
|
if (fie_disabled)
|
2020-06-23 10:19:40 +00:00
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
/* policy->cpus will be empty here, use related_cpus instead */
|
|
|
|
|
topology_clear_scale_freq_source(SCALE_FREQ_SOURCE_CPPC, policy->related_cpus);
|
|
|
|
|
|
|
|
|
|
for_each_cpu(cpu, policy->related_cpus) {
|
|
|
|
|
cppc_fi = &per_cpu(cppc_freq_inv, cpu);
|
|
|
|
|
irq_work_sync(&cppc_fi->irq_work);
|
|
|
|
|
kthread_cancel_work_sync(&cppc_fi->work);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void __init cppc_freq_invariance_init(void)
|
|
|
|
|
{
|
|
|
|
|
struct sched_attr attr = {
|
|
|
|
|
.size = sizeof(struct sched_attr),
|
|
|
|
|
.sched_policy = SCHED_DEADLINE,
|
|
|
|
|
.sched_nice = 0,
|
|
|
|
|
.sched_priority = 0,
|
|
|
|
|
/*
|
|
|
|
|
* Fake (unused) bandwidth; workaround to "fix"
|
|
|
|
|
* priority inheritance.
|
|
|
|
|
*/
|
2024-08-13 14:43:47 +00:00
|
|
|
.sched_runtime = NSEC_PER_MSEC,
|
|
|
|
|
.sched_deadline = 10 * NSEC_PER_MSEC,
|
|
|
|
|
.sched_period = 10 * NSEC_PER_MSEC,
|
2020-06-23 10:19:40 +00:00
|
|
|
};
|
|
|
|
|
int ret;
|
|
|
|
|
|
2022-09-12 20:37:22 +00:00
|
|
|
if (fie_disabled != FIE_ENABLED && fie_disabled != FIE_DISABLED) {
|
|
|
|
|
fie_disabled = FIE_ENABLED;
|
|
|
|
|
if (cppc_perf_ctrs_in_pcc()) {
|
|
|
|
|
pr_info("FIE not enabled on systems with registers in PCC\n");
|
|
|
|
|
fie_disabled = FIE_DISABLED;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (fie_disabled)
|
2020-06-23 10:19:40 +00:00
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
kworker_fie = kthread_create_worker(0, "cppc_fie");
|
2023-08-17 07:47:56 +00:00
|
|
|
if (IS_ERR(kworker_fie)) {
|
|
|
|
|
pr_warn("%s: failed to create kworker_fie: %ld\n", __func__,
|
|
|
|
|
PTR_ERR(kworker_fie));
|
|
|
|
|
fie_disabled = FIE_DISABLED;
|
2020-06-23 10:19:40 +00:00
|
|
|
return;
|
2023-08-17 07:47:56 +00:00
|
|
|
}
|
2020-06-23 10:19:40 +00:00
|
|
|
|
|
|
|
|
ret = sched_setattr_nocheck(kworker_fie->task, &attr);
|
|
|
|
|
if (ret) {
|
|
|
|
|
pr_warn("%s: failed to set SCHED_DEADLINE: %d\n", __func__,
|
|
|
|
|
ret);
|
|
|
|
|
kthread_destroy_worker(kworker_fie);
|
2023-08-17 07:47:56 +00:00
|
|
|
fie_disabled = FIE_DISABLED;
|
2020-06-23 10:19:40 +00:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void cppc_freq_invariance_exit(void)
|
|
|
|
|
{
|
2022-09-12 20:37:22 +00:00
|
|
|
if (fie_disabled)
|
2020-06-23 10:19:40 +00:00
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
kthread_destroy_worker(kworker_fie);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#else
|
|
|
|
|
static inline void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy)
|
|
|
|
|
{
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static inline void cppc_cpufreq_cpu_fie_exit(struct cpufreq_policy *policy)
|
|
|
|
|
{
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static inline void cppc_freq_invariance_init(void)
|
|
|
|
|
{
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static inline void cppc_freq_invariance_exit(void)
|
|
|
|
|
{
|
|
|
|
|
}
|
|
|
|
|
#endif /* CONFIG_ACPI_CPPC_CPUFREQ_FIE */
|
|
|
|
|
|
2015-10-02 14:04:01 +00:00
|
|
|
static int cppc_cpufreq_set_target(struct cpufreq_policy *policy,
|
2020-11-05 12:55:17 +00:00
|
|
|
unsigned int target_freq,
|
|
|
|
|
unsigned int relation)
|
2015-10-02 14:04:01 +00:00
|
|
|
{
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
struct cppc_cpudata *cpu_data = policy->driver_data;
|
2020-12-14 12:38:20 +00:00
|
|
|
unsigned int cpu = policy->cpu;
|
2015-10-02 14:04:01 +00:00
|
|
|
struct cpufreq_freqs freqs;
|
|
|
|
|
int ret = 0;
|
|
|
|
|
|
2024-06-12 11:46:31 +00:00
|
|
|
cpu_data->perf_ctrls.desired_perf =
|
|
|
|
|
cppc_khz_to_perf(&cpu_data->perf_caps, target_freq);
|
2015-10-02 14:04:01 +00:00
|
|
|
freqs.old = policy->cur;
|
|
|
|
|
freqs.new = target_freq;
|
|
|
|
|
|
|
|
|
|
cpufreq_freq_transition_begin(policy, &freqs);
|
2020-12-14 12:38:20 +00:00
|
|
|
ret = cppc_set_perf(cpu, &cpu_data->perf_ctrls);
|
2015-10-02 14:04:01 +00:00
|
|
|
cpufreq_freq_transition_end(policy, &freqs, ret != 0);
|
|
|
|
|
|
|
|
|
|
if (ret)
|
|
|
|
|
pr_debug("Failed to set target on CPU:%d. ret:%d\n",
|
2020-12-14 12:38:20 +00:00
|
|
|
cpu, ret);
|
2015-10-02 14:04:01 +00:00
|
|
|
|
|
|
|
|
return ret;
|
|
|
|
|
}
|
|
|
|
|
|
2022-05-18 09:09:00 +00:00
|
|
|
static unsigned int cppc_cpufreq_fast_switch(struct cpufreq_policy *policy,
|
|
|
|
|
unsigned int target_freq)
|
|
|
|
|
{
|
|
|
|
|
struct cppc_cpudata *cpu_data = policy->driver_data;
|
|
|
|
|
unsigned int cpu = policy->cpu;
|
|
|
|
|
u32 desired_perf;
|
|
|
|
|
int ret;
|
|
|
|
|
|
2023-12-11 10:48:53 +00:00
|
|
|
desired_perf = cppc_khz_to_perf(&cpu_data->perf_caps, target_freq);
|
2022-05-18 09:09:00 +00:00
|
|
|
cpu_data->perf_ctrls.desired_perf = desired_perf;
|
|
|
|
|
ret = cppc_set_perf(cpu, &cpu_data->perf_ctrls);
|
|
|
|
|
|
|
|
|
|
if (ret) {
|
|
|
|
|
pr_debug("Failed to set target on CPU:%d. ret:%d\n",
|
|
|
|
|
cpu, ret);
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return target_freq;
|
|
|
|
|
}
|
|
|
|
|
|
cpufreq: Avoid creating excessively large stack frames
In the process of modifying a cpufreq policy, the cpufreq core makes
a copy of it including all of the internals which is stored on the
CPU stack. Because struct cpufreq_policy is relatively large, this
may cause the size of the stack frame to exceed the 2 KB limit and
so the GCC complains when -Wframe-larger-than= is used.
In fact, it is not necessary to copy the entire policy structure
in order to modify it, however.
First, because cpufreq_set_policy() obtains the min and max policy
limits from frequency QoS now, it is not necessary to pass the limits
to it from the callers. The only things that need to be passed to it
from there are the new governor pointer or (if there is a built-in
governor in the driver) the "policy" value representing the governor
choice. They both can be passed as individual arguments, though, so
make cpufreq_set_policy() take them this way and rework its callers
accordingly. This avoids making copies of cpufreq policies in the
callers of cpufreq_set_policy().
Second, cpufreq_set_policy() still needs to pass the new policy
data to the ->verify() callback of the cpufreq driver whose task
is to sanitize the min and max policy limits. It still does not
need to make a full copy of struct cpufreq_policy for this purpose,
but it needs to pass a few items from it to the driver in case they
are needed (different drivers have different needs in that respect
and all of them have to be covered). For this reason, introduce
struct cpufreq_policy_data to hold copies of the members of
struct cpufreq_policy used by the existing ->verify() driver
callbacks and pass a pointer to a temporary structure of that
type to ->verify() (instead of passing a pointer to full struct
cpufreq_policy to it).
While at it, notice that intel_pstate and longrun don't really need
to verify the "policy" value in struct cpufreq_policy, so drop those
check from them to avoid copying "policy" into struct
cpufreq_policy_data (which allows it to be slightly smaller).
Also while at it fix up white space in a couple of places and make
cpufreq_set_policy() static (as it can be so).
Fixes: 3000ce3c52f8 ("cpufreq: Use per-policy frequency QoS")
Link: https://lore.kernel.org/linux-pm/CAMuHMdX6-jb1W8uC2_237m8ctCpsnGp=JCxqt8pCWVqNXHmkVg@mail.gmail.com
Reported-by: kbuild test robot <lkp@intel.com>
Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: 5.4+ <stable@vger.kernel.org> # 5.4+
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
2020-01-26 22:40:11 +00:00
|
|
|
static int cppc_verify_policy(struct cpufreq_policy_data *policy)
|
2015-10-02 14:04:01 +00:00
|
|
|
{
|
|
|
|
|
cpufreq_verify_within_cpu_limits(policy);
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
|
2018-04-27 17:35:27 +00:00
|
|
|
/*
|
|
|
|
|
* The PCC subspace describes the rate at which platform can accept commands
|
|
|
|
|
* on the shared PCC channel (including READs which do not count towards freq
|
2020-11-05 12:55:17 +00:00
|
|
|
* transition requests), so ideally we need to use the PCC values as a fallback
|
2018-04-27 17:35:27 +00:00
|
|
|
* if we don't have a platform specific transition_delay_us
|
|
|
|
|
*/
|
|
|
|
|
#ifdef CONFIG_ARM64
|
|
|
|
|
#include <asm/cputype.h>
|
|
|
|
|
|
2020-11-05 12:55:18 +00:00
|
|
|
static unsigned int cppc_cpufreq_get_transition_delay_us(unsigned int cpu)
|
2018-04-27 17:35:27 +00:00
|
|
|
{
|
|
|
|
|
unsigned long implementor = read_cpuid_implementor();
|
|
|
|
|
unsigned long part_num = read_cpuid_part_number();
|
|
|
|
|
|
|
|
|
|
switch (implementor) {
|
|
|
|
|
case ARM_CPU_IMP_QCOM:
|
|
|
|
|
switch (part_num) {
|
|
|
|
|
case QCOM_CPU_PART_FALKOR_V1:
|
|
|
|
|
case QCOM_CPU_PART_FALKOR:
|
2021-03-13 02:50:06 +00:00
|
|
|
return 10000;
|
2018-04-27 17:35:27 +00:00
|
|
|
}
|
|
|
|
|
}
|
2021-03-13 02:50:06 +00:00
|
|
|
return cppc_get_transition_latency(cpu) / NSEC_PER_USEC;
|
2018-04-27 17:35:27 +00:00
|
|
|
}
|
2022-05-30 10:04:24 +00:00
|
|
|
#else
|
|
|
|
|
static unsigned int cppc_cpufreq_get_transition_delay_us(unsigned int cpu)
|
|
|
|
|
{
|
|
|
|
|
return cppc_get_transition_latency(cpu) / NSEC_PER_USEC;
|
|
|
|
|
}
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
#if defined(CONFIG_ARM64) && defined(CONFIG_ENERGY_MODEL)
|
2018-04-27 17:35:27 +00:00
|
|
|
|
cpufreq: CPPC: Add per_cpu efficiency_class
In ACPI, describing power efficiency of CPUs can be done through the
following arm specific field:
ACPI 6.4, s5.2.12.14 'GIC CPU Interface (GICC) Structure',
'Processor Power Efficiency Class field':
Describes the relative power efficiency of the associated pro-
cessor. Lower efficiency class numbers are more efficient than
higher ones (e.g. efficiency class 0 should be treated as more
efficient than efficiency class 1). However, absolute values
of this number have no meaning: 2 isn’t necessarily half as
efficient as 1.
The efficiency_class field is stored in the GicC structure of the
ACPI MADT table and it's currently supported in Linux for arm64 only.
Thus, this new functionality is introduced for arm64 only.
To allow the cppc_cpufreq driver to know and preprocess the
efficiency_class values of all the CPUs, add a per_cpu efficiency_class
variable to store them.
At least 2 different efficiency classes must be present,
otherwise there is no use in creating an Energy Model.
The efficiency_class values are squeezed in [0:#efficiency_class-1]
while conserving the order. For instance, efficiency classes of:
[111, 212, 250]
will be mapped to:
[0 (was 111), 1 (was 212), 2 (was 250)].
Each policy being independently registered in the driver, populating
the per_cpu efficiency_class is done only once at the driver
initialization. This prevents from having each policy re-searching the
efficiency_class values of other CPUs. The EM will be registered in a
following patch.
The patch also exports acpi_cpu_get_madt_gicc() to fetch the GicC
structure of the ACPI MADT table for each CPU.
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:07 +00:00
|
|
|
static DEFINE_PER_CPU(unsigned int, efficiency_class);
|
cpufreq: CPPC: Register EM based on efficiency class information
Performance states and energy consumption values are not advertised
in ACPI. In the GicC structure of the MADT table, the "Processor
Power Efficiency Class field" (called efficiency class from now)
allows to describe the relative energy efficiency of CPUs.
To leverage the EM and EAS, the CPPC driver creates a set of
artificial performance states and registers them in the Energy Model
(EM), such as:
- Every 20 capacity unit, a performance state is created.
- The energy cost of each performance state gradually increases.
No power value is generated as only the cost is used in the EM.
During task placement, a task can raise the frequency of its whole
pd. This can make EAS place a task on a pd with CPUs that are
individually less energy efficient.
As cost values are artificial, and to place tasks on CPUs with the
lower efficiency class, a gap in cost values is generated for adjacent
efficiency classes.
E.g.:
- efficiency class = 0, capacity is in [0-1024], so cost values
are in [0: 51] (one performance state every 20 capacity unit)
- efficiency class = 1, capacity is in [0-1024], cost values
are in [1*gap+0: 1*gap+51].
The value of the cost gap is chosen to absorb a the energy of 4 CPUs
at their maximum capacity. This means that between:
1- a pd of 4 CPUs, each of them being used at almost their full
capacity. Their efficiency class is N.
2- a CPU using almost none of its capacity. Its efficiency class is
N+1
EAS will choose the first option.
This patch also populates the (struct cpufreq_driver).register_em
callback if the valid efficiency_class ACPI values are provided.
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:08 +00:00
|
|
|
static void cppc_cpufreq_register_em(struct cpufreq_policy *policy);
|
|
|
|
|
|
|
|
|
|
/* Create an artificial performance state every CPPC_EM_CAP_STEP capacity unit. */
|
|
|
|
|
#define CPPC_EM_CAP_STEP (20)
|
|
|
|
|
/* Increase the cost value by CPPC_EM_COST_STEP every performance state. */
|
|
|
|
|
#define CPPC_EM_COST_STEP (1)
|
|
|
|
|
/* Add a cost gap correspnding to the energy of 4 CPUs. */
|
|
|
|
|
#define CPPC_EM_COST_GAP (4 * SCHED_CAPACITY_SCALE * CPPC_EM_COST_STEP \
|
|
|
|
|
/ CPPC_EM_CAP_STEP)
|
|
|
|
|
|
|
|
|
|
static unsigned int get_perf_level_count(struct cpufreq_policy *policy)
|
|
|
|
|
{
|
|
|
|
|
struct cppc_perf_caps *perf_caps;
|
|
|
|
|
unsigned int min_cap, max_cap;
|
|
|
|
|
struct cppc_cpudata *cpu_data;
|
|
|
|
|
int cpu = policy->cpu;
|
|
|
|
|
|
|
|
|
|
cpu_data = policy->driver_data;
|
|
|
|
|
perf_caps = &cpu_data->perf_caps;
|
|
|
|
|
max_cap = arch_scale_cpu_capacity(cpu);
|
2022-12-20 10:12:25 +00:00
|
|
|
min_cap = div_u64((u64)max_cap * perf_caps->lowest_perf,
|
|
|
|
|
perf_caps->highest_perf);
|
cpufreq: CPPC: Register EM based on efficiency class information
Performance states and energy consumption values are not advertised
in ACPI. In the GicC structure of the MADT table, the "Processor
Power Efficiency Class field" (called efficiency class from now)
allows to describe the relative energy efficiency of CPUs.
To leverage the EM and EAS, the CPPC driver creates a set of
artificial performance states and registers them in the Energy Model
(EM), such as:
- Every 20 capacity unit, a performance state is created.
- The energy cost of each performance state gradually increases.
No power value is generated as only the cost is used in the EM.
During task placement, a task can raise the frequency of its whole
pd. This can make EAS place a task on a pd with CPUs that are
individually less energy efficient.
As cost values are artificial, and to place tasks on CPUs with the
lower efficiency class, a gap in cost values is generated for adjacent
efficiency classes.
E.g.:
- efficiency class = 0, capacity is in [0-1024], so cost values
are in [0: 51] (one performance state every 20 capacity unit)
- efficiency class = 1, capacity is in [0-1024], cost values
are in [1*gap+0: 1*gap+51].
The value of the cost gap is chosen to absorb a the energy of 4 CPUs
at their maximum capacity. This means that between:
1- a pd of 4 CPUs, each of them being used at almost their full
capacity. Their efficiency class is N.
2- a CPU using almost none of its capacity. Its efficiency class is
N+1
EAS will choose the first option.
This patch also populates the (struct cpufreq_driver).register_em
callback if the valid efficiency_class ACPI values are provided.
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:08 +00:00
|
|
|
if ((min_cap == 0) || (max_cap < min_cap))
|
|
|
|
|
return 0;
|
|
|
|
|
return 1 + max_cap / CPPC_EM_CAP_STEP - min_cap / CPPC_EM_CAP_STEP;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The cost is defined as:
|
|
|
|
|
* cost = power * max_frequency / frequency
|
|
|
|
|
*/
|
|
|
|
|
static inline unsigned long compute_cost(int cpu, int step)
|
|
|
|
|
{
|
|
|
|
|
return CPPC_EM_COST_GAP * per_cpu(efficiency_class, cpu) +
|
|
|
|
|
step * CPPC_EM_COST_STEP;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static int cppc_get_cpu_power(struct device *cpu_dev,
|
|
|
|
|
unsigned long *power, unsigned long *KHz)
|
|
|
|
|
{
|
|
|
|
|
unsigned long perf_step, perf_prev, perf, perf_check;
|
|
|
|
|
unsigned int min_step, max_step, step, step_check;
|
|
|
|
|
unsigned long prev_freq = *KHz;
|
|
|
|
|
unsigned int min_cap, max_cap;
|
|
|
|
|
struct cpufreq_policy *policy;
|
|
|
|
|
|
|
|
|
|
struct cppc_perf_caps *perf_caps;
|
|
|
|
|
struct cppc_cpudata *cpu_data;
|
|
|
|
|
|
|
|
|
|
policy = cpufreq_cpu_get_raw(cpu_dev->id);
|
2024-10-30 01:20:19 +00:00
|
|
|
if (!policy)
|
2024-11-06 01:01:11 +00:00
|
|
|
return -EINVAL;
|
2024-10-30 01:20:19 +00:00
|
|
|
|
cpufreq: CPPC: Register EM based on efficiency class information
Performance states and energy consumption values are not advertised
in ACPI. In the GicC structure of the MADT table, the "Processor
Power Efficiency Class field" (called efficiency class from now)
allows to describe the relative energy efficiency of CPUs.
To leverage the EM and EAS, the CPPC driver creates a set of
artificial performance states and registers them in the Energy Model
(EM), such as:
- Every 20 capacity unit, a performance state is created.
- The energy cost of each performance state gradually increases.
No power value is generated as only the cost is used in the EM.
During task placement, a task can raise the frequency of its whole
pd. This can make EAS place a task on a pd with CPUs that are
individually less energy efficient.
As cost values are artificial, and to place tasks on CPUs with the
lower efficiency class, a gap in cost values is generated for adjacent
efficiency classes.
E.g.:
- efficiency class = 0, capacity is in [0-1024], so cost values
are in [0: 51] (one performance state every 20 capacity unit)
- efficiency class = 1, capacity is in [0-1024], cost values
are in [1*gap+0: 1*gap+51].
The value of the cost gap is chosen to absorb a the energy of 4 CPUs
at their maximum capacity. This means that between:
1- a pd of 4 CPUs, each of them being used at almost their full
capacity. Their efficiency class is N.
2- a CPU using almost none of its capacity. Its efficiency class is
N+1
EAS will choose the first option.
This patch also populates the (struct cpufreq_driver).register_em
callback if the valid efficiency_class ACPI values are provided.
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:08 +00:00
|
|
|
cpu_data = policy->driver_data;
|
|
|
|
|
perf_caps = &cpu_data->perf_caps;
|
|
|
|
|
max_cap = arch_scale_cpu_capacity(cpu_dev->id);
|
2022-12-20 10:12:25 +00:00
|
|
|
min_cap = div_u64((u64)max_cap * perf_caps->lowest_perf,
|
|
|
|
|
perf_caps->highest_perf);
|
|
|
|
|
perf_step = div_u64((u64)CPPC_EM_CAP_STEP * perf_caps->highest_perf,
|
|
|
|
|
max_cap);
|
cpufreq: CPPC: Register EM based on efficiency class information
Performance states and energy consumption values are not advertised
in ACPI. In the GicC structure of the MADT table, the "Processor
Power Efficiency Class field" (called efficiency class from now)
allows to describe the relative energy efficiency of CPUs.
To leverage the EM and EAS, the CPPC driver creates a set of
artificial performance states and registers them in the Energy Model
(EM), such as:
- Every 20 capacity unit, a performance state is created.
- The energy cost of each performance state gradually increases.
No power value is generated as only the cost is used in the EM.
During task placement, a task can raise the frequency of its whole
pd. This can make EAS place a task on a pd with CPUs that are
individually less energy efficient.
As cost values are artificial, and to place tasks on CPUs with the
lower efficiency class, a gap in cost values is generated for adjacent
efficiency classes.
E.g.:
- efficiency class = 0, capacity is in [0-1024], so cost values
are in [0: 51] (one performance state every 20 capacity unit)
- efficiency class = 1, capacity is in [0-1024], cost values
are in [1*gap+0: 1*gap+51].
The value of the cost gap is chosen to absorb a the energy of 4 CPUs
at their maximum capacity. This means that between:
1- a pd of 4 CPUs, each of them being used at almost their full
capacity. Their efficiency class is N.
2- a CPU using almost none of its capacity. Its efficiency class is
N+1
EAS will choose the first option.
This patch also populates the (struct cpufreq_driver).register_em
callback if the valid efficiency_class ACPI values are provided.
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:08 +00:00
|
|
|
min_step = min_cap / CPPC_EM_CAP_STEP;
|
|
|
|
|
max_step = max_cap / CPPC_EM_CAP_STEP;
|
|
|
|
|
|
2023-12-11 10:48:53 +00:00
|
|
|
perf_prev = cppc_khz_to_perf(perf_caps, *KHz);
|
cpufreq: CPPC: Register EM based on efficiency class information
Performance states and energy consumption values are not advertised
in ACPI. In the GicC structure of the MADT table, the "Processor
Power Efficiency Class field" (called efficiency class from now)
allows to describe the relative energy efficiency of CPUs.
To leverage the EM and EAS, the CPPC driver creates a set of
artificial performance states and registers them in the Energy Model
(EM), such as:
- Every 20 capacity unit, a performance state is created.
- The energy cost of each performance state gradually increases.
No power value is generated as only the cost is used in the EM.
During task placement, a task can raise the frequency of its whole
pd. This can make EAS place a task on a pd with CPUs that are
individually less energy efficient.
As cost values are artificial, and to place tasks on CPUs with the
lower efficiency class, a gap in cost values is generated for adjacent
efficiency classes.
E.g.:
- efficiency class = 0, capacity is in [0-1024], so cost values
are in [0: 51] (one performance state every 20 capacity unit)
- efficiency class = 1, capacity is in [0-1024], cost values
are in [1*gap+0: 1*gap+51].
The value of the cost gap is chosen to absorb a the energy of 4 CPUs
at their maximum capacity. This means that between:
1- a pd of 4 CPUs, each of them being used at almost their full
capacity. Their efficiency class is N.
2- a CPU using almost none of its capacity. Its efficiency class is
N+1
EAS will choose the first option.
This patch also populates the (struct cpufreq_driver).register_em
callback if the valid efficiency_class ACPI values are provided.
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:08 +00:00
|
|
|
step = perf_prev / perf_step;
|
|
|
|
|
|
|
|
|
|
if (step > max_step)
|
|
|
|
|
return -EINVAL;
|
|
|
|
|
|
|
|
|
|
if (min_step == max_step) {
|
|
|
|
|
step = max_step;
|
|
|
|
|
perf = perf_caps->highest_perf;
|
|
|
|
|
} else if (step < min_step) {
|
|
|
|
|
step = min_step;
|
|
|
|
|
perf = perf_caps->lowest_perf;
|
|
|
|
|
} else {
|
|
|
|
|
step++;
|
|
|
|
|
if (step == max_step)
|
|
|
|
|
perf = perf_caps->highest_perf;
|
|
|
|
|
else
|
|
|
|
|
perf = step * perf_step;
|
|
|
|
|
}
|
|
|
|
|
|
2023-12-11 10:48:53 +00:00
|
|
|
*KHz = cppc_perf_to_khz(perf_caps, perf);
|
|
|
|
|
perf_check = cppc_khz_to_perf(perf_caps, *KHz);
|
cpufreq: CPPC: Register EM based on efficiency class information
Performance states and energy consumption values are not advertised
in ACPI. In the GicC structure of the MADT table, the "Processor
Power Efficiency Class field" (called efficiency class from now)
allows to describe the relative energy efficiency of CPUs.
To leverage the EM and EAS, the CPPC driver creates a set of
artificial performance states and registers them in the Energy Model
(EM), such as:
- Every 20 capacity unit, a performance state is created.
- The energy cost of each performance state gradually increases.
No power value is generated as only the cost is used in the EM.
During task placement, a task can raise the frequency of its whole
pd. This can make EAS place a task on a pd with CPUs that are
individually less energy efficient.
As cost values are artificial, and to place tasks on CPUs with the
lower efficiency class, a gap in cost values is generated for adjacent
efficiency classes.
E.g.:
- efficiency class = 0, capacity is in [0-1024], so cost values
are in [0: 51] (one performance state every 20 capacity unit)
- efficiency class = 1, capacity is in [0-1024], cost values
are in [1*gap+0: 1*gap+51].
The value of the cost gap is chosen to absorb a the energy of 4 CPUs
at their maximum capacity. This means that between:
1- a pd of 4 CPUs, each of them being used at almost their full
capacity. Their efficiency class is N.
2- a CPU using almost none of its capacity. Its efficiency class is
N+1
EAS will choose the first option.
This patch also populates the (struct cpufreq_driver).register_em
callback if the valid efficiency_class ACPI values are provided.
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:08 +00:00
|
|
|
step_check = perf_check / perf_step;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* To avoid bad integer approximation, check that new frequency value
|
|
|
|
|
* increased and that the new frequency will be converted to the
|
|
|
|
|
* desired step value.
|
|
|
|
|
*/
|
|
|
|
|
while ((*KHz == prev_freq) || (step_check != step)) {
|
|
|
|
|
perf++;
|
2023-12-11 10:48:53 +00:00
|
|
|
*KHz = cppc_perf_to_khz(perf_caps, perf);
|
|
|
|
|
perf_check = cppc_khz_to_perf(perf_caps, *KHz);
|
cpufreq: CPPC: Register EM based on efficiency class information
Performance states and energy consumption values are not advertised
in ACPI. In the GicC structure of the MADT table, the "Processor
Power Efficiency Class field" (called efficiency class from now)
allows to describe the relative energy efficiency of CPUs.
To leverage the EM and EAS, the CPPC driver creates a set of
artificial performance states and registers them in the Energy Model
(EM), such as:
- Every 20 capacity unit, a performance state is created.
- The energy cost of each performance state gradually increases.
No power value is generated as only the cost is used in the EM.
During task placement, a task can raise the frequency of its whole
pd. This can make EAS place a task on a pd with CPUs that are
individually less energy efficient.
As cost values are artificial, and to place tasks on CPUs with the
lower efficiency class, a gap in cost values is generated for adjacent
efficiency classes.
E.g.:
- efficiency class = 0, capacity is in [0-1024], so cost values
are in [0: 51] (one performance state every 20 capacity unit)
- efficiency class = 1, capacity is in [0-1024], cost values
are in [1*gap+0: 1*gap+51].
The value of the cost gap is chosen to absorb a the energy of 4 CPUs
at their maximum capacity. This means that between:
1- a pd of 4 CPUs, each of them being used at almost their full
capacity. Their efficiency class is N.
2- a CPU using almost none of its capacity. Its efficiency class is
N+1
EAS will choose the first option.
This patch also populates the (struct cpufreq_driver).register_em
callback if the valid efficiency_class ACPI values are provided.
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:08 +00:00
|
|
|
step_check = perf_check / perf_step;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* With an artificial EM, only the cost value is used. Still the power
|
|
|
|
|
* is populated such as 0 < power < EM_MAX_POWER. This allows to add
|
|
|
|
|
* more sense to the artificial performance states.
|
|
|
|
|
*/
|
|
|
|
|
*power = compute_cost(cpu_dev->id, step);
|
|
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static int cppc_get_cpu_cost(struct device *cpu_dev, unsigned long KHz,
|
|
|
|
|
unsigned long *cost)
|
|
|
|
|
{
|
|
|
|
|
unsigned long perf_step, perf_prev;
|
|
|
|
|
struct cppc_perf_caps *perf_caps;
|
|
|
|
|
struct cpufreq_policy *policy;
|
|
|
|
|
struct cppc_cpudata *cpu_data;
|
|
|
|
|
unsigned int max_cap;
|
|
|
|
|
int step;
|
|
|
|
|
|
|
|
|
|
policy = cpufreq_cpu_get_raw(cpu_dev->id);
|
2024-10-30 08:24:49 +00:00
|
|
|
if (!policy)
|
2024-11-06 01:12:38 +00:00
|
|
|
return -EINVAL;
|
2024-10-30 08:24:49 +00:00
|
|
|
|
cpufreq: CPPC: Register EM based on efficiency class information
Performance states and energy consumption values are not advertised
in ACPI. In the GicC structure of the MADT table, the "Processor
Power Efficiency Class field" (called efficiency class from now)
allows to describe the relative energy efficiency of CPUs.
To leverage the EM and EAS, the CPPC driver creates a set of
artificial performance states and registers them in the Energy Model
(EM), such as:
- Every 20 capacity unit, a performance state is created.
- The energy cost of each performance state gradually increases.
No power value is generated as only the cost is used in the EM.
During task placement, a task can raise the frequency of its whole
pd. This can make EAS place a task on a pd with CPUs that are
individually less energy efficient.
As cost values are artificial, and to place tasks on CPUs with the
lower efficiency class, a gap in cost values is generated for adjacent
efficiency classes.
E.g.:
- efficiency class = 0, capacity is in [0-1024], so cost values
are in [0: 51] (one performance state every 20 capacity unit)
- efficiency class = 1, capacity is in [0-1024], cost values
are in [1*gap+0: 1*gap+51].
The value of the cost gap is chosen to absorb a the energy of 4 CPUs
at their maximum capacity. This means that between:
1- a pd of 4 CPUs, each of them being used at almost their full
capacity. Their efficiency class is N.
2- a CPU using almost none of its capacity. Its efficiency class is
N+1
EAS will choose the first option.
This patch also populates the (struct cpufreq_driver).register_em
callback if the valid efficiency_class ACPI values are provided.
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:08 +00:00
|
|
|
cpu_data = policy->driver_data;
|
|
|
|
|
perf_caps = &cpu_data->perf_caps;
|
|
|
|
|
max_cap = arch_scale_cpu_capacity(cpu_dev->id);
|
|
|
|
|
|
2023-12-11 10:48:53 +00:00
|
|
|
perf_prev = cppc_khz_to_perf(perf_caps, KHz);
|
cpufreq: CPPC: Register EM based on efficiency class information
Performance states and energy consumption values are not advertised
in ACPI. In the GicC structure of the MADT table, the "Processor
Power Efficiency Class field" (called efficiency class from now)
allows to describe the relative energy efficiency of CPUs.
To leverage the EM and EAS, the CPPC driver creates a set of
artificial performance states and registers them in the Energy Model
(EM), such as:
- Every 20 capacity unit, a performance state is created.
- The energy cost of each performance state gradually increases.
No power value is generated as only the cost is used in the EM.
During task placement, a task can raise the frequency of its whole
pd. This can make EAS place a task on a pd with CPUs that are
individually less energy efficient.
As cost values are artificial, and to place tasks on CPUs with the
lower efficiency class, a gap in cost values is generated for adjacent
efficiency classes.
E.g.:
- efficiency class = 0, capacity is in [0-1024], so cost values
are in [0: 51] (one performance state every 20 capacity unit)
- efficiency class = 1, capacity is in [0-1024], cost values
are in [1*gap+0: 1*gap+51].
The value of the cost gap is chosen to absorb a the energy of 4 CPUs
at their maximum capacity. This means that between:
1- a pd of 4 CPUs, each of them being used at almost their full
capacity. Their efficiency class is N.
2- a CPU using almost none of its capacity. Its efficiency class is
N+1
EAS will choose the first option.
This patch also populates the (struct cpufreq_driver).register_em
callback if the valid efficiency_class ACPI values are provided.
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:08 +00:00
|
|
|
perf_step = CPPC_EM_CAP_STEP * perf_caps->highest_perf / max_cap;
|
|
|
|
|
step = perf_prev / perf_step;
|
|
|
|
|
|
|
|
|
|
*cost = compute_cost(cpu_dev->id, step);
|
|
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
cpufreq: CPPC: Add per_cpu efficiency_class
In ACPI, describing power efficiency of CPUs can be done through the
following arm specific field:
ACPI 6.4, s5.2.12.14 'GIC CPU Interface (GICC) Structure',
'Processor Power Efficiency Class field':
Describes the relative power efficiency of the associated pro-
cessor. Lower efficiency class numbers are more efficient than
higher ones (e.g. efficiency class 0 should be treated as more
efficient than efficiency class 1). However, absolute values
of this number have no meaning: 2 isn’t necessarily half as
efficient as 1.
The efficiency_class field is stored in the GicC structure of the
ACPI MADT table and it's currently supported in Linux for arm64 only.
Thus, this new functionality is introduced for arm64 only.
To allow the cppc_cpufreq driver to know and preprocess the
efficiency_class values of all the CPUs, add a per_cpu efficiency_class
variable to store them.
At least 2 different efficiency classes must be present,
otherwise there is no use in creating an Energy Model.
The efficiency_class values are squeezed in [0:#efficiency_class-1]
while conserving the order. For instance, efficiency classes of:
[111, 212, 250]
will be mapped to:
[0 (was 111), 1 (was 212), 2 (was 250)].
Each policy being independently registered in the driver, populating
the per_cpu efficiency_class is done only once at the driver
initialization. This prevents from having each policy re-searching the
efficiency_class values of other CPUs. The EM will be registered in a
following patch.
The patch also exports acpi_cpu_get_madt_gicc() to fetch the GicC
structure of the ACPI MADT table for each CPU.
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:07 +00:00
|
|
|
|
|
|
|
|
static int populate_efficiency_class(void)
|
|
|
|
|
{
|
|
|
|
|
struct acpi_madt_generic_interrupt *gicc;
|
|
|
|
|
DECLARE_BITMAP(used_classes, 256) = {};
|
|
|
|
|
int class, cpu, index;
|
|
|
|
|
|
|
|
|
|
for_each_possible_cpu(cpu) {
|
|
|
|
|
gicc = acpi_cpu_get_madt_gicc(cpu);
|
|
|
|
|
class = gicc->efficiency_class;
|
|
|
|
|
bitmap_set(used_classes, class, 1);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (bitmap_weight(used_classes, 256) <= 1) {
|
|
|
|
|
pr_debug("Efficiency classes are all equal (=%d). "
|
|
|
|
|
"No EM registered", class);
|
|
|
|
|
return -EINVAL;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Squeeze efficiency class values on [0:#efficiency_class-1].
|
|
|
|
|
* Values are per spec in [0:255].
|
|
|
|
|
*/
|
|
|
|
|
index = 0;
|
|
|
|
|
for_each_set_bit(class, used_classes, 256) {
|
|
|
|
|
for_each_possible_cpu(cpu) {
|
|
|
|
|
gicc = acpi_cpu_get_madt_gicc(cpu);
|
|
|
|
|
if (gicc->efficiency_class == class)
|
|
|
|
|
per_cpu(efficiency_class, cpu) = index;
|
|
|
|
|
}
|
|
|
|
|
index++;
|
|
|
|
|
}
|
cpufreq: CPPC: Register EM based on efficiency class information
Performance states and energy consumption values are not advertised
in ACPI. In the GicC structure of the MADT table, the "Processor
Power Efficiency Class field" (called efficiency class from now)
allows to describe the relative energy efficiency of CPUs.
To leverage the EM and EAS, the CPPC driver creates a set of
artificial performance states and registers them in the Energy Model
(EM), such as:
- Every 20 capacity unit, a performance state is created.
- The energy cost of each performance state gradually increases.
No power value is generated as only the cost is used in the EM.
During task placement, a task can raise the frequency of its whole
pd. This can make EAS place a task on a pd with CPUs that are
individually less energy efficient.
As cost values are artificial, and to place tasks on CPUs with the
lower efficiency class, a gap in cost values is generated for adjacent
efficiency classes.
E.g.:
- efficiency class = 0, capacity is in [0-1024], so cost values
are in [0: 51] (one performance state every 20 capacity unit)
- efficiency class = 1, capacity is in [0-1024], cost values
are in [1*gap+0: 1*gap+51].
The value of the cost gap is chosen to absorb a the energy of 4 CPUs
at their maximum capacity. This means that between:
1- a pd of 4 CPUs, each of them being used at almost their full
capacity. Their efficiency class is N.
2- a CPU using almost none of its capacity. Its efficiency class is
N+1
EAS will choose the first option.
This patch also populates the (struct cpufreq_driver).register_em
callback if the valid efficiency_class ACPI values are provided.
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:08 +00:00
|
|
|
cppc_cpufreq_driver.register_em = cppc_cpufreq_register_em;
|
cpufreq: CPPC: Add per_cpu efficiency_class
In ACPI, describing power efficiency of CPUs can be done through the
following arm specific field:
ACPI 6.4, s5.2.12.14 'GIC CPU Interface (GICC) Structure',
'Processor Power Efficiency Class field':
Describes the relative power efficiency of the associated pro-
cessor. Lower efficiency class numbers are more efficient than
higher ones (e.g. efficiency class 0 should be treated as more
efficient than efficiency class 1). However, absolute values
of this number have no meaning: 2 isn’t necessarily half as
efficient as 1.
The efficiency_class field is stored in the GicC structure of the
ACPI MADT table and it's currently supported in Linux for arm64 only.
Thus, this new functionality is introduced for arm64 only.
To allow the cppc_cpufreq driver to know and preprocess the
efficiency_class values of all the CPUs, add a per_cpu efficiency_class
variable to store them.
At least 2 different efficiency classes must be present,
otherwise there is no use in creating an Energy Model.
The efficiency_class values are squeezed in [0:#efficiency_class-1]
while conserving the order. For instance, efficiency classes of:
[111, 212, 250]
will be mapped to:
[0 (was 111), 1 (was 212), 2 (was 250)].
Each policy being independently registered in the driver, populating
the per_cpu efficiency_class is done only once at the driver
initialization. This prevents from having each policy re-searching the
efficiency_class values of other CPUs. The EM will be registered in a
following patch.
The patch also exports acpi_cpu_get_madt_gicc() to fetch the GicC
structure of the ACPI MADT table for each CPU.
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:07 +00:00
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
|
cpufreq: CPPC: Register EM based on efficiency class information
Performance states and energy consumption values are not advertised
in ACPI. In the GicC structure of the MADT table, the "Processor
Power Efficiency Class field" (called efficiency class from now)
allows to describe the relative energy efficiency of CPUs.
To leverage the EM and EAS, the CPPC driver creates a set of
artificial performance states and registers them in the Energy Model
(EM), such as:
- Every 20 capacity unit, a performance state is created.
- The energy cost of each performance state gradually increases.
No power value is generated as only the cost is used in the EM.
During task placement, a task can raise the frequency of its whole
pd. This can make EAS place a task on a pd with CPUs that are
individually less energy efficient.
As cost values are artificial, and to place tasks on CPUs with the
lower efficiency class, a gap in cost values is generated for adjacent
efficiency classes.
E.g.:
- efficiency class = 0, capacity is in [0-1024], so cost values
are in [0: 51] (one performance state every 20 capacity unit)
- efficiency class = 1, capacity is in [0-1024], cost values
are in [1*gap+0: 1*gap+51].
The value of the cost gap is chosen to absorb a the energy of 4 CPUs
at their maximum capacity. This means that between:
1- a pd of 4 CPUs, each of them being used at almost their full
capacity. Their efficiency class is N.
2- a CPU using almost none of its capacity. Its efficiency class is
N+1
EAS will choose the first option.
This patch also populates the (struct cpufreq_driver).register_em
callback if the valid efficiency_class ACPI values are provided.
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:08 +00:00
|
|
|
static void cppc_cpufreq_register_em(struct cpufreq_policy *policy)
|
|
|
|
|
{
|
|
|
|
|
struct cppc_cpudata *cpu_data;
|
|
|
|
|
struct em_data_callback em_cb =
|
|
|
|
|
EM_ADV_DATA_CB(cppc_get_cpu_power, cppc_get_cpu_cost);
|
|
|
|
|
|
|
|
|
|
cpu_data = policy->driver_data;
|
|
|
|
|
em_dev_register_perf_domain(get_cpu_device(policy->cpu),
|
|
|
|
|
get_perf_level_count(policy), &em_cb,
|
|
|
|
|
cpu_data->shared_cpu_map, 0);
|
|
|
|
|
}
|
|
|
|
|
|
2018-04-27 17:35:27 +00:00
|
|
|
#else
|
cpufreq: CPPC: Add per_cpu efficiency_class
In ACPI, describing power efficiency of CPUs can be done through the
following arm specific field:
ACPI 6.4, s5.2.12.14 'GIC CPU Interface (GICC) Structure',
'Processor Power Efficiency Class field':
Describes the relative power efficiency of the associated pro-
cessor. Lower efficiency class numbers are more efficient than
higher ones (e.g. efficiency class 0 should be treated as more
efficient than efficiency class 1). However, absolute values
of this number have no meaning: 2 isn’t necessarily half as
efficient as 1.
The efficiency_class field is stored in the GicC structure of the
ACPI MADT table and it's currently supported in Linux for arm64 only.
Thus, this new functionality is introduced for arm64 only.
To allow the cppc_cpufreq driver to know and preprocess the
efficiency_class values of all the CPUs, add a per_cpu efficiency_class
variable to store them.
At least 2 different efficiency classes must be present,
otherwise there is no use in creating an Energy Model.
The efficiency_class values are squeezed in [0:#efficiency_class-1]
while conserving the order. For instance, efficiency classes of:
[111, 212, 250]
will be mapped to:
[0 (was 111), 1 (was 212), 2 (was 250)].
Each policy being independently registered in the driver, populating
the per_cpu efficiency_class is done only once at the driver
initialization. This prevents from having each policy re-searching the
efficiency_class values of other CPUs. The EM will be registered in a
following patch.
The patch also exports acpi_cpu_get_madt_gicc() to fetch the GicC
structure of the ACPI MADT table for each CPU.
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:07 +00:00
|
|
|
static int populate_efficiency_class(void)
|
|
|
|
|
{
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
2018-04-27 17:35:27 +00:00
|
|
|
#endif
|
|
|
|
|
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
static struct cppc_cpudata *cppc_cpufreq_get_cpu_data(unsigned int cpu)
|
2015-10-02 14:04:01 +00:00
|
|
|
{
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
struct cppc_cpudata *cpu_data;
|
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
|
|
cpu_data = kzalloc(sizeof(struct cppc_cpudata), GFP_KERNEL);
|
|
|
|
|
if (!cpu_data)
|
|
|
|
|
goto out;
|
2015-10-02 14:04:01 +00:00
|
|
|
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
if (!zalloc_cpumask_var(&cpu_data->shared_cpu_map, GFP_KERNEL))
|
|
|
|
|
goto free_cpu;
|
2015-10-02 14:04:01 +00:00
|
|
|
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
ret = acpi_get_psd_map(cpu, cpu_data);
|
2015-10-02 14:04:01 +00:00
|
|
|
if (ret) {
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
pr_debug("Err parsing CPU%d PSD data: ret:%d\n", cpu, ret);
|
|
|
|
|
goto free_mask;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
ret = cppc_get_perf_caps(cpu, &cpu_data->perf_caps);
|
|
|
|
|
if (ret) {
|
|
|
|
|
pr_debug("Err reading CPU%d perf caps: ret:%d\n", cpu, ret);
|
|
|
|
|
goto free_mask;
|
2015-10-02 14:04:01 +00:00
|
|
|
}
|
|
|
|
|
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
list_add(&cpu_data->node, &cpu_data_list);
|
|
|
|
|
|
|
|
|
|
return cpu_data;
|
|
|
|
|
|
|
|
|
|
free_mask:
|
|
|
|
|
free_cpumask_var(cpu_data->shared_cpu_map);
|
|
|
|
|
free_cpu:
|
|
|
|
|
kfree(cpu_data);
|
|
|
|
|
out:
|
|
|
|
|
return NULL;
|
|
|
|
|
}
|
|
|
|
|
|
2021-06-18 08:01:27 +00:00
|
|
|
static void cppc_cpufreq_put_cpu_data(struct cpufreq_policy *policy)
|
|
|
|
|
{
|
|
|
|
|
struct cppc_cpudata *cpu_data = policy->driver_data;
|
|
|
|
|
|
|
|
|
|
list_del(&cpu_data->node);
|
|
|
|
|
free_cpumask_var(cpu_data->shared_cpu_map);
|
|
|
|
|
kfree(cpu_data);
|
|
|
|
|
policy->driver_data = NULL;
|
|
|
|
|
}
|
|
|
|
|
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
static int cppc_cpufreq_cpu_init(struct cpufreq_policy *policy)
|
|
|
|
|
{
|
|
|
|
|
unsigned int cpu = policy->cpu;
|
|
|
|
|
struct cppc_cpudata *cpu_data;
|
|
|
|
|
struct cppc_perf_caps *caps;
|
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
|
|
cpu_data = cppc_cpufreq_get_cpu_data(cpu);
|
|
|
|
|
if (!cpu_data) {
|
|
|
|
|
pr_err("Error in acquiring _CPC/_PSD data for CPU%d.\n", cpu);
|
|
|
|
|
return -ENODEV;
|
|
|
|
|
}
|
|
|
|
|
caps = &cpu_data->perf_caps;
|
|
|
|
|
policy->driver_data = cpu_data;
|
cpufreq: CPPC: Force reporting values in KHz to fix user space interface
When CPPC is being used by ACPI on arm64, user space tools such as
cpupower report CPU frequency values from sysfs that are incorrect.
What the driver was doing was reporting the values given by ACPI tables
in whatever scale was used to provide them. However, the ACPI spec
defines the CPPC values as unitless abstract numbers. Internal kernel
structures such as struct perf_cap, in contrast, expect these values
to be in KHz. When these struct values get reported via sysfs, the
user space tools also assume they are in KHz, causing them to report
incorrect values (for example, reporting a CPU frequency of 1MHz when
it should be 1.8GHz).
The downside is that this approach has some assumptions:
(1) It relies on SMBIOS3 being used, *and* that the Max Frequency
value for a processor is set to a non-zero value.
(2) It assumes that all processors run at the same speed, or that
the CPPC values have all been scaled to reflect relative speed.
This patch retrieves the largest CPU Max Frequency from a type 4 DMI
record that it can find. This may not be an issue, however, as a
sampling of DMI data on x86 and arm64 indicates there is often only
one such record regardless. Since CPPC is relatively new, it is
unclear if the ACPI ASL will always be written to reflect any sort
of relative performance of processors of differing speeds.
(3) It assumes that performance and frequency both scale linearly.
For arm64 servers, this may be sufficient, but it does rely on
firmware values being set correctly. Hence, other approaches will
be considered in the future.
This has been tested on three arm64 servers, with and without DMI, with
and without CPPC support.
Signed-off-by: Al Stone <ahs3@redhat.com>
Signed-off-by: Prashanth Prakash <pprakash@codeaurora.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2016-07-20 21:10:04 +00:00
|
|
|
|
2017-05-11 22:39:44 +00:00
|
|
|
/*
|
|
|
|
|
* Set min to lowest nonlinear perf to avoid any efficiency penalty (see
|
|
|
|
|
* Section 8.4.7.1.1.5 of ACPI 6.1 spec)
|
|
|
|
|
*/
|
2023-12-11 10:48:53 +00:00
|
|
|
policy->min = cppc_perf_to_khz(caps, caps->lowest_nonlinear_perf);
|
|
|
|
|
policy->max = cppc_perf_to_khz(caps, caps->nominal_perf);
|
2017-05-11 22:39:44 +00:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Set cpuinfo.min_freq to Lowest to make the full range of performance
|
|
|
|
|
* available if userspace wants to use any perf between lowest & lowest
|
|
|
|
|
* nonlinear perf
|
|
|
|
|
*/
|
2023-12-11 10:48:53 +00:00
|
|
|
policy->cpuinfo.min_freq = cppc_perf_to_khz(caps, caps->lowest_perf);
|
|
|
|
|
policy->cpuinfo.max_freq = cppc_perf_to_khz(caps, caps->nominal_perf);
|
2017-05-11 22:39:44 +00:00
|
|
|
|
2020-11-05 12:55:18 +00:00
|
|
|
policy->transition_delay_us = cppc_cpufreq_get_transition_delay_us(cpu);
|
|
|
|
|
policy->shared_type = cpu_data->shared_type;
|
2015-10-02 14:04:01 +00:00
|
|
|
|
2020-12-14 12:38:21 +00:00
|
|
|
switch (policy->shared_type) {
|
|
|
|
|
case CPUFREQ_SHARED_TYPE_HW:
|
|
|
|
|
case CPUFREQ_SHARED_TYPE_NONE:
|
|
|
|
|
/* Nothing to be done - we'll have a policy for each CPU */
|
|
|
|
|
break;
|
|
|
|
|
case CPUFREQ_SHARED_TYPE_ANY:
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
/*
|
|
|
|
|
* All CPUs in the domain will share a policy and all cpufreq
|
|
|
|
|
* operations will use a single cppc_cpudata structure stored
|
|
|
|
|
* in policy->driver_data.
|
|
|
|
|
*/
|
2020-11-05 12:55:18 +00:00
|
|
|
cpumask_copy(policy->cpus, cpu_data->shared_cpu_map);
|
2020-12-14 12:38:21 +00:00
|
|
|
break;
|
|
|
|
|
default:
|
|
|
|
|
pr_debug("Unsupported CPU co-ord type: %d\n",
|
|
|
|
|
policy->shared_type);
|
2021-06-18 08:01:27 +00:00
|
|
|
ret = -EFAULT;
|
|
|
|
|
goto out;
|
2015-10-02 14:04:01 +00:00
|
|
|
}
|
|
|
|
|
|
2022-05-18 09:09:00 +00:00
|
|
|
policy->fast_switch_possible = cppc_allow_fast_switch();
|
2022-05-18 09:09:01 +00:00
|
|
|
policy->dvfs_possible_from_any_cpu = true;
|
2022-05-18 09:09:00 +00:00
|
|
|
|
cpufreq: CPPC: add SW BOOST support
To add SW BOOST support for CPPC, we need to get the max frequency of
boost mode and non-boost mode. ACPI spec 6.2 section 8.4.7.1 describes
the following two CPC registers.
"Highest performance is the absolute maximum performance an individual
processor may reach, assuming ideal conditions. This performance level
may not be sustainable for long durations, and may only be achievable if
other platform components are in a specific state; for example, it may
require other processors be in an idle state.
Nominal Performance is the maximum sustained performance level of the
processor, assuming ideal operating conditions. In absence of an
external constraint (power, thermal, etc.) this is the performance level
the platform is expected to be able to maintain continuously. All
processors are expected to be able to sustain their nominal performance
state simultaneously."
To add SW BOOST support for CPPC, we can use Highest Performance as the
max performance in boost mode and Nominal Performance as the max
performance in non-boost mode. If the Highest Performance is greater
than the Nominal Performance, we assume SW BOOST is supported.
The current CPPC driver does not support SW BOOST and use 'Highest
Performance' as the max performance the CPU can achieve. 'Nominal
Performance' is used to convert 'performance' to 'frequency'. That
means, if firmware enable boost and provide a value for Highest
Performance which is greater than Nominal Performance, boost feature is
enabled by default.
Because SW BOOST is disabled by default, so, after this patch, boost
feature is disabled by default even if boost is enabled by firmware.
Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
Suggested-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
[ rjw: Subject ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-05-30 02:08:31 +00:00
|
|
|
/*
|
|
|
|
|
* If 'highest_perf' is greater than 'nominal_perf', we assume CPU Boost
|
|
|
|
|
* is supported.
|
|
|
|
|
*/
|
2020-11-05 12:55:19 +00:00
|
|
|
if (caps->highest_perf > caps->nominal_perf)
|
cpufreq: CPPC: add SW BOOST support
To add SW BOOST support for CPPC, we need to get the max frequency of
boost mode and non-boost mode. ACPI spec 6.2 section 8.4.7.1 describes
the following two CPC registers.
"Highest performance is the absolute maximum performance an individual
processor may reach, assuming ideal conditions. This performance level
may not be sustainable for long durations, and may only be achievable if
other platform components are in a specific state; for example, it may
require other processors be in an idle state.
Nominal Performance is the maximum sustained performance level of the
processor, assuming ideal operating conditions. In absence of an
external constraint (power, thermal, etc.) this is the performance level
the platform is expected to be able to maintain continuously. All
processors are expected to be able to sustain their nominal performance
state simultaneously."
To add SW BOOST support for CPPC, we can use Highest Performance as the
max performance in boost mode and Nominal Performance as the max
performance in non-boost mode. If the Highest Performance is greater
than the Nominal Performance, we assume SW BOOST is supported.
The current CPPC driver does not support SW BOOST and use 'Highest
Performance' as the max performance the CPU can achieve. 'Nominal
Performance' is used to convert 'performance' to 'frequency'. That
means, if firmware enable boost and provide a value for Highest
Performance which is greater than Nominal Performance, boost feature is
enabled by default.
Because SW BOOST is disabled by default, so, after this patch, boost
feature is disabled by default even if boost is enabled by firmware.
Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
Suggested-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
[ rjw: Subject ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-05-30 02:08:31 +00:00
|
|
|
boost_supported = true;
|
|
|
|
|
|
2015-10-02 14:04:01 +00:00
|
|
|
/* Set policy->cur to max now. The governors will adjust later. */
|
2023-12-11 10:48:53 +00:00
|
|
|
policy->cur = cppc_perf_to_khz(caps, caps->highest_perf);
|
2020-11-05 12:55:19 +00:00
|
|
|
cpu_data->perf_ctrls.desired_perf = caps->highest_perf;
|
2015-10-02 14:04:01 +00:00
|
|
|
|
2020-11-05 12:55:18 +00:00
|
|
|
ret = cppc_set_perf(cpu, &cpu_data->perf_ctrls);
|
2021-06-18 08:01:27 +00:00
|
|
|
if (ret) {
|
2015-10-02 14:04:01 +00:00
|
|
|
pr_debug("Err setting perf value:%d on CPU:%d. ret:%d\n",
|
2020-11-05 12:55:19 +00:00
|
|
|
caps->highest_perf, cpu, ret);
|
2021-06-18 08:01:27 +00:00
|
|
|
goto out;
|
|
|
|
|
}
|
|
|
|
|
|
2020-06-23 10:19:40 +00:00
|
|
|
cppc_cpufreq_cpu_fie_init(policy);
|
2021-06-18 08:01:27 +00:00
|
|
|
return 0;
|
2015-10-02 14:04:01 +00:00
|
|
|
|
2021-06-18 08:01:27 +00:00
|
|
|
out:
|
|
|
|
|
cppc_cpufreq_put_cpu_data(policy);
|
2015-10-02 14:04:01 +00:00
|
|
|
return ret;
|
|
|
|
|
}
|
|
|
|
|
|
2024-07-04 06:53:55 +00:00
|
|
|
static void cppc_cpufreq_cpu_exit(struct cpufreq_policy *policy)
|
2021-06-23 04:24:39 +00:00
|
|
|
{
|
|
|
|
|
struct cppc_cpudata *cpu_data = policy->driver_data;
|
|
|
|
|
struct cppc_perf_caps *caps = &cpu_data->perf_caps;
|
|
|
|
|
unsigned int cpu = policy->cpu;
|
|
|
|
|
int ret;
|
|
|
|
|
|
2020-06-23 10:19:40 +00:00
|
|
|
cppc_cpufreq_cpu_fie_exit(policy);
|
|
|
|
|
|
2021-06-23 04:24:39 +00:00
|
|
|
cpu_data->perf_ctrls.desired_perf = caps->lowest_perf;
|
|
|
|
|
|
|
|
|
|
ret = cppc_set_perf(cpu, &cpu_data->perf_ctrls);
|
|
|
|
|
if (ret)
|
|
|
|
|
pr_debug("Err setting perf value:%d on CPU:%d. ret:%d\n",
|
|
|
|
|
caps->lowest_perf, cpu, ret);
|
|
|
|
|
|
2021-06-18 08:01:27 +00:00
|
|
|
cppc_cpufreq_put_cpu_data(policy);
|
2021-06-23 04:24:39 +00:00
|
|
|
}
|
|
|
|
|
|
2018-07-12 06:07:55 +00:00
|
|
|
static inline u64 get_delta(u64 t1, u64 t0)
|
|
|
|
|
{
|
|
|
|
|
if (t1 > t0 || t0 > ~(u32)0)
|
|
|
|
|
return t1 - t0;
|
|
|
|
|
|
|
|
|
|
return (u32)t1 - (u32)t0;
|
|
|
|
|
}
|
|
|
|
|
|
2020-06-23 10:19:40 +00:00
|
|
|
static int cppc_perf_from_fbctrs(struct cppc_cpudata *cpu_data,
|
|
|
|
|
struct cppc_perf_fb_ctrs *fb_ctrs_t0,
|
|
|
|
|
struct cppc_perf_fb_ctrs *fb_ctrs_t1)
|
2018-07-12 06:07:55 +00:00
|
|
|
{
|
|
|
|
|
u64 delta_reference, delta_delivered;
|
2020-06-23 10:19:40 +00:00
|
|
|
u64 reference_perf;
|
2018-07-12 06:07:55 +00:00
|
|
|
|
2021-06-18 08:12:23 +00:00
|
|
|
reference_perf = fb_ctrs_t0->reference_perf;
|
2018-07-12 06:07:55 +00:00
|
|
|
|
2021-06-18 08:12:23 +00:00
|
|
|
delta_reference = get_delta(fb_ctrs_t1->reference,
|
|
|
|
|
fb_ctrs_t0->reference);
|
|
|
|
|
delta_delivered = get_delta(fb_ctrs_t1->delivered,
|
|
|
|
|
fb_ctrs_t0->delivered);
|
2018-07-12 06:07:55 +00:00
|
|
|
|
2024-09-29 03:32:13 +00:00
|
|
|
/*
|
|
|
|
|
* Avoid divide-by zero and unchanged feedback counters.
|
|
|
|
|
* Leave it for callers to handle.
|
|
|
|
|
*/
|
2020-06-23 10:19:40 +00:00
|
|
|
if (!delta_reference || !delta_delivered)
|
2024-09-29 03:32:13 +00:00
|
|
|
return 0;
|
2018-07-12 06:07:55 +00:00
|
|
|
|
2020-06-23 10:19:40 +00:00
|
|
|
return (reference_perf * delta_delivered) / delta_reference;
|
2018-07-12 06:07:55 +00:00
|
|
|
}
|
|
|
|
|
|
2024-09-29 03:32:13 +00:00
|
|
|
static int cppc_get_perf_ctrs_sample(int cpu,
|
|
|
|
|
struct cppc_perf_fb_ctrs *fb_ctrs_t0,
|
|
|
|
|
struct cppc_perf_fb_ctrs *fb_ctrs_t1)
|
|
|
|
|
{
|
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
|
|
ret = cppc_get_perf_ctrs(cpu, fb_ctrs_t0);
|
|
|
|
|
if (ret)
|
|
|
|
|
return ret;
|
|
|
|
|
|
|
|
|
|
udelay(2); /* 2usec delay between sampling */
|
|
|
|
|
|
|
|
|
|
return cppc_get_perf_ctrs(cpu, fb_ctrs_t1);
|
|
|
|
|
}
|
|
|
|
|
|
2020-11-05 12:55:18 +00:00
|
|
|
static unsigned int cppc_cpufreq_get_rate(unsigned int cpu)
|
2018-07-12 06:07:55 +00:00
|
|
|
{
|
|
|
|
|
struct cppc_perf_fb_ctrs fb_ctrs_t0 = {0}, fb_ctrs_t1 = {0};
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
struct cpufreq_policy *policy = cpufreq_cpu_get(cpu);
|
2024-04-08 09:35:36 +00:00
|
|
|
struct cppc_cpudata *cpu_data;
|
2020-06-23 10:19:40 +00:00
|
|
|
u64 delivered_perf;
|
2018-07-12 06:07:55 +00:00
|
|
|
int ret;
|
|
|
|
|
|
2024-04-08 09:35:36 +00:00
|
|
|
if (!policy)
|
|
|
|
|
return -ENODEV;
|
|
|
|
|
|
|
|
|
|
cpu_data = policy->driver_data;
|
|
|
|
|
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
cpufreq_cpu_put(policy);
|
|
|
|
|
|
2024-09-29 03:32:13 +00:00
|
|
|
ret = cppc_get_perf_ctrs_sample(cpu, &fb_ctrs_t0, &fb_ctrs_t1);
|
|
|
|
|
if (ret) {
|
|
|
|
|
if (ret == -EFAULT)
|
|
|
|
|
/* Any of the associated CPPC regs is 0. */
|
|
|
|
|
goto out_invalid_counters;
|
|
|
|
|
else
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
2018-07-12 06:07:55 +00:00
|
|
|
|
2020-06-23 10:19:40 +00:00
|
|
|
delivered_perf = cppc_perf_from_fbctrs(cpu_data, &fb_ctrs_t0,
|
|
|
|
|
&fb_ctrs_t1);
|
2024-09-29 03:32:13 +00:00
|
|
|
if (!delivered_perf)
|
|
|
|
|
goto out_invalid_counters;
|
|
|
|
|
|
|
|
|
|
return cppc_perf_to_khz(&cpu_data->perf_caps, delivered_perf);
|
|
|
|
|
|
|
|
|
|
out_invalid_counters:
|
|
|
|
|
/*
|
|
|
|
|
* Feedback counters could be unchanged or 0 when a cpu enters a
|
|
|
|
|
* low-power idle state, e.g. clock-gated or power-gated.
|
|
|
|
|
* Use desired perf for reflecting frequency. Get the latest register
|
|
|
|
|
* value first as some platforms may update the actual delivered perf
|
|
|
|
|
* there; if failed, resort to the cached desired perf.
|
|
|
|
|
*/
|
|
|
|
|
if (cppc_get_desired_perf(cpu, &delivered_perf))
|
|
|
|
|
delivered_perf = cpu_data->perf_ctrls.desired_perf;
|
2020-06-23 10:19:40 +00:00
|
|
|
|
2023-12-11 10:48:53 +00:00
|
|
|
return cppc_perf_to_khz(&cpu_data->perf_caps, delivered_perf);
|
2018-07-12 06:07:55 +00:00
|
|
|
}
|
|
|
|
|
|
cpufreq: CPPC: add SW BOOST support
To add SW BOOST support for CPPC, we need to get the max frequency of
boost mode and non-boost mode. ACPI spec 6.2 section 8.4.7.1 describes
the following two CPC registers.
"Highest performance is the absolute maximum performance an individual
processor may reach, assuming ideal conditions. This performance level
may not be sustainable for long durations, and may only be achievable if
other platform components are in a specific state; for example, it may
require other processors be in an idle state.
Nominal Performance is the maximum sustained performance level of the
processor, assuming ideal operating conditions. In absence of an
external constraint (power, thermal, etc.) this is the performance level
the platform is expected to be able to maintain continuously. All
processors are expected to be able to sustain their nominal performance
state simultaneously."
To add SW BOOST support for CPPC, we can use Highest Performance as the
max performance in boost mode and Nominal Performance as the max
performance in non-boost mode. If the Highest Performance is greater
than the Nominal Performance, we assume SW BOOST is supported.
The current CPPC driver does not support SW BOOST and use 'Highest
Performance' as the max performance the CPU can achieve. 'Nominal
Performance' is used to convert 'performance' to 'frequency'. That
means, if firmware enable boost and provide a value for Highest
Performance which is greater than Nominal Performance, boost feature is
enabled by default.
Because SW BOOST is disabled by default, so, after this patch, boost
feature is disabled by default even if boost is enabled by firmware.
Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
Suggested-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
[ rjw: Subject ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-05-30 02:08:31 +00:00
|
|
|
static int cppc_cpufreq_set_boost(struct cpufreq_policy *policy, int state)
|
|
|
|
|
{
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
struct cppc_cpudata *cpu_data = policy->driver_data;
|
2020-11-05 12:55:19 +00:00
|
|
|
struct cppc_perf_caps *caps = &cpu_data->perf_caps;
|
cpufreq: CPPC: add SW BOOST support
To add SW BOOST support for CPPC, we need to get the max frequency of
boost mode and non-boost mode. ACPI spec 6.2 section 8.4.7.1 describes
the following two CPC registers.
"Highest performance is the absolute maximum performance an individual
processor may reach, assuming ideal conditions. This performance level
may not be sustainable for long durations, and may only be achievable if
other platform components are in a specific state; for example, it may
require other processors be in an idle state.
Nominal Performance is the maximum sustained performance level of the
processor, assuming ideal operating conditions. In absence of an
external constraint (power, thermal, etc.) this is the performance level
the platform is expected to be able to maintain continuously. All
processors are expected to be able to sustain their nominal performance
state simultaneously."
To add SW BOOST support for CPPC, we can use Highest Performance as the
max performance in boost mode and Nominal Performance as the max
performance in non-boost mode. If the Highest Performance is greater
than the Nominal Performance, we assume SW BOOST is supported.
The current CPPC driver does not support SW BOOST and use 'Highest
Performance' as the max performance the CPU can achieve. 'Nominal
Performance' is used to convert 'performance' to 'frequency'. That
means, if firmware enable boost and provide a value for Highest
Performance which is greater than Nominal Performance, boost feature is
enabled by default.
Because SW BOOST is disabled by default, so, after this patch, boost
feature is disabled by default even if boost is enabled by firmware.
Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
Suggested-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
[ rjw: Subject ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-05-30 02:08:31 +00:00
|
|
|
int ret;
|
|
|
|
|
|
|
|
|
|
if (!boost_supported) {
|
|
|
|
|
pr_err("BOOST not supported by CPU or firmware\n");
|
|
|
|
|
return -EINVAL;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (state)
|
2023-12-11 10:48:53 +00:00
|
|
|
policy->max = cppc_perf_to_khz(caps, caps->highest_perf);
|
cpufreq: CPPC: add SW BOOST support
To add SW BOOST support for CPPC, we need to get the max frequency of
boost mode and non-boost mode. ACPI spec 6.2 section 8.4.7.1 describes
the following two CPC registers.
"Highest performance is the absolute maximum performance an individual
processor may reach, assuming ideal conditions. This performance level
may not be sustainable for long durations, and may only be achievable if
other platform components are in a specific state; for example, it may
require other processors be in an idle state.
Nominal Performance is the maximum sustained performance level of the
processor, assuming ideal operating conditions. In absence of an
external constraint (power, thermal, etc.) this is the performance level
the platform is expected to be able to maintain continuously. All
processors are expected to be able to sustain their nominal performance
state simultaneously."
To add SW BOOST support for CPPC, we can use Highest Performance as the
max performance in boost mode and Nominal Performance as the max
performance in non-boost mode. If the Highest Performance is greater
than the Nominal Performance, we assume SW BOOST is supported.
The current CPPC driver does not support SW BOOST and use 'Highest
Performance' as the max performance the CPU can achieve. 'Nominal
Performance' is used to convert 'performance' to 'frequency'. That
means, if firmware enable boost and provide a value for Highest
Performance which is greater than Nominal Performance, boost feature is
enabled by default.
Because SW BOOST is disabled by default, so, after this patch, boost
feature is disabled by default even if boost is enabled by firmware.
Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
Suggested-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
[ rjw: Subject ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-05-30 02:08:31 +00:00
|
|
|
else
|
2023-12-11 10:48:53 +00:00
|
|
|
policy->max = cppc_perf_to_khz(caps, caps->nominal_perf);
|
cpufreq: CPPC: add SW BOOST support
To add SW BOOST support for CPPC, we need to get the max frequency of
boost mode and non-boost mode. ACPI spec 6.2 section 8.4.7.1 describes
the following two CPC registers.
"Highest performance is the absolute maximum performance an individual
processor may reach, assuming ideal conditions. This performance level
may not be sustainable for long durations, and may only be achievable if
other platform components are in a specific state; for example, it may
require other processors be in an idle state.
Nominal Performance is the maximum sustained performance level of the
processor, assuming ideal operating conditions. In absence of an
external constraint (power, thermal, etc.) this is the performance level
the platform is expected to be able to maintain continuously. All
processors are expected to be able to sustain their nominal performance
state simultaneously."
To add SW BOOST support for CPPC, we can use Highest Performance as the
max performance in boost mode and Nominal Performance as the max
performance in non-boost mode. If the Highest Performance is greater
than the Nominal Performance, we assume SW BOOST is supported.
The current CPPC driver does not support SW BOOST and use 'Highest
Performance' as the max performance the CPU can achieve. 'Nominal
Performance' is used to convert 'performance' to 'frequency'. That
means, if firmware enable boost and provide a value for Highest
Performance which is greater than Nominal Performance, boost feature is
enabled by default.
Because SW BOOST is disabled by default, so, after this patch, boost
feature is disabled by default even if boost is enabled by firmware.
Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
Suggested-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
[ rjw: Subject ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-05-30 02:08:31 +00:00
|
|
|
policy->cpuinfo.max_freq = policy->max;
|
|
|
|
|
|
|
|
|
|
ret = freq_qos_update_request(policy->max_freq_req, policy->max);
|
|
|
|
|
if (ret < 0)
|
|
|
|
|
return ret;
|
|
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
|
2020-12-14 12:38:22 +00:00
|
|
|
static ssize_t show_freqdomain_cpus(struct cpufreq_policy *policy, char *buf)
|
|
|
|
|
{
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
struct cppc_cpudata *cpu_data = policy->driver_data;
|
2020-12-14 12:38:22 +00:00
|
|
|
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
return cpufreq_show_cpus(cpu_data->shared_cpu_map, buf);
|
2020-12-14 12:38:22 +00:00
|
|
|
}
|
|
|
|
|
cpufreq_freq_attr_ro(freqdomain_cpus);
|
|
|
|
|
|
|
|
|
|
static struct freq_attr *cppc_cpufreq_attr[] = {
|
|
|
|
|
&freqdomain_cpus,
|
|
|
|
|
NULL,
|
|
|
|
|
};
|
|
|
|
|
|
2015-10-02 14:04:01 +00:00
|
|
|
static struct cpufreq_driver cppc_cpufreq_driver = {
|
|
|
|
|
.flags = CPUFREQ_CONST_LOOPS,
|
|
|
|
|
.verify = cppc_verify_policy,
|
|
|
|
|
.target = cppc_cpufreq_set_target,
|
2018-07-12 06:07:55 +00:00
|
|
|
.get = cppc_cpufreq_get_rate,
|
2022-05-18 09:09:00 +00:00
|
|
|
.fast_switch = cppc_cpufreq_fast_switch,
|
2015-10-02 14:04:01 +00:00
|
|
|
.init = cppc_cpufreq_cpu_init,
|
2021-06-23 04:24:39 +00:00
|
|
|
.exit = cppc_cpufreq_cpu_exit,
|
cpufreq: CPPC: add SW BOOST support
To add SW BOOST support for CPPC, we need to get the max frequency of
boost mode and non-boost mode. ACPI spec 6.2 section 8.4.7.1 describes
the following two CPC registers.
"Highest performance is the absolute maximum performance an individual
processor may reach, assuming ideal conditions. This performance level
may not be sustainable for long durations, and may only be achievable if
other platform components are in a specific state; for example, it may
require other processors be in an idle state.
Nominal Performance is the maximum sustained performance level of the
processor, assuming ideal operating conditions. In absence of an
external constraint (power, thermal, etc.) this is the performance level
the platform is expected to be able to maintain continuously. All
processors are expected to be able to sustain their nominal performance
state simultaneously."
To add SW BOOST support for CPPC, we can use Highest Performance as the
max performance in boost mode and Nominal Performance as the max
performance in non-boost mode. If the Highest Performance is greater
than the Nominal Performance, we assume SW BOOST is supported.
The current CPPC driver does not support SW BOOST and use 'Highest
Performance' as the max performance the CPU can achieve. 'Nominal
Performance' is used to convert 'performance' to 'frequency'. That
means, if firmware enable boost and provide a value for Highest
Performance which is greater than Nominal Performance, boost feature is
enabled by default.
Because SW BOOST is disabled by default, so, after this patch, boost
feature is disabled by default even if boost is enabled by firmware.
Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
Suggested-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
[ rjw: Subject ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-05-30 02:08:31 +00:00
|
|
|
.set_boost = cppc_cpufreq_set_boost,
|
2020-12-14 12:38:22 +00:00
|
|
|
.attr = cppc_cpufreq_attr,
|
2015-10-02 14:04:01 +00:00
|
|
|
.name = "cppc_cpufreq",
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
static int __init cppc_cpufreq_init(void)
|
|
|
|
|
{
|
2020-06-23 10:19:40 +00:00
|
|
|
int ret;
|
|
|
|
|
|
2022-08-14 16:35:48 +00:00
|
|
|
if (!acpi_cpc_valid())
|
2015-10-02 14:04:01 +00:00
|
|
|
return -ENODEV;
|
|
|
|
|
|
2020-06-23 10:19:40 +00:00
|
|
|
cppc_freq_invariance_init();
|
cpufreq: CPPC: Add per_cpu efficiency_class
In ACPI, describing power efficiency of CPUs can be done through the
following arm specific field:
ACPI 6.4, s5.2.12.14 'GIC CPU Interface (GICC) Structure',
'Processor Power Efficiency Class field':
Describes the relative power efficiency of the associated pro-
cessor. Lower efficiency class numbers are more efficient than
higher ones (e.g. efficiency class 0 should be treated as more
efficient than efficiency class 1). However, absolute values
of this number have no meaning: 2 isn’t necessarily half as
efficient as 1.
The efficiency_class field is stored in the GicC structure of the
ACPI MADT table and it's currently supported in Linux for arm64 only.
Thus, this new functionality is introduced for arm64 only.
To allow the cppc_cpufreq driver to know and preprocess the
efficiency_class values of all the CPUs, add a per_cpu efficiency_class
variable to store them.
At least 2 different efficiency classes must be present,
otherwise there is no use in creating an Energy Model.
The efficiency_class values are squeezed in [0:#efficiency_class-1]
while conserving the order. For instance, efficiency classes of:
[111, 212, 250]
will be mapped to:
[0 (was 111), 1 (was 212), 2 (was 250)].
Each policy being independently registered in the driver, populating
the per_cpu efficiency_class is done only once at the driver
initialization. This prevents from having each policy re-searching the
efficiency_class values of other CPUs. The EM will be registered in a
following patch.
The patch also exports acpi_cpu_get_madt_gicc() to fetch the GicC
structure of the ACPI MADT table for each CPU.
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2022-04-25 12:38:07 +00:00
|
|
|
populate_efficiency_class();
|
2019-02-17 03:54:15 +00:00
|
|
|
|
2020-06-23 10:19:40 +00:00
|
|
|
ret = cpufreq_register_driver(&cppc_cpufreq_driver);
|
|
|
|
|
if (ret)
|
|
|
|
|
cppc_freq_invariance_exit();
|
|
|
|
|
|
|
|
|
|
return ret;
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
}
|
2015-10-02 14:04:01 +00:00
|
|
|
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
static inline void free_cpu_data(void)
|
|
|
|
|
{
|
|
|
|
|
struct cppc_cpudata *iter, *tmp;
|
2015-10-02 14:04:01 +00:00
|
|
|
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
list_for_each_entry_safe(iter, tmp, &cpu_data_list, node) {
|
|
|
|
|
free_cpumask_var(iter->shared_cpu_map);
|
|
|
|
|
list_del(&iter->node);
|
|
|
|
|
kfree(iter);
|
2018-03-05 05:40:38 +00:00
|
|
|
}
|
2015-10-02 14:04:01 +00:00
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
2016-04-15 00:45:53 +00:00
|
|
|
static void __exit cppc_cpufreq_exit(void)
|
|
|
|
|
{
|
|
|
|
|
cpufreq_unregister_driver(&cppc_cpufreq_driver);
|
2020-06-23 10:19:40 +00:00
|
|
|
cppc_freq_invariance_exit();
|
2016-04-15 00:45:53 +00:00
|
|
|
|
cppc_cpufreq: replace per-cpu data array with a list
The cppc_cpudata per-cpu storage was inefficient (1) additional to causing
functional issues (2) when CPUs are hotplugged out, due to per-cpu data
being improperly initialised.
(1) The amount of information needed for CPPC performance control in its
cpufreq driver depends on the domain (PSD) coordination type:
ANY: One set of CPPC control and capability data (e.g desired
performance, highest/lowest performance, etc) applies to all
CPUs in the domain.
ALL: Same as ANY. To be noted that this type is not currently
supported. When supported, information about which CPUs
belong to a domain is needed in order for frequency change
requests to be sent to each of them.
HW: It's necessary to store CPPC control and capability
information for all the CPUs. HW will then coordinate the
performance state based on their limitations and requests.
NONE: Same as HW. No HW coordination is expected.
Despite this, the previous initialisation code would indiscriminately
allocate memory for all CPUs (all_cpu_data) and unnecessarily
duplicate performance capabilities and the domain sharing mask and type
for each possible CPU.
(2) With the current per-cpu structure, when having ANY coordination,
the cppc_cpudata cpu information is not initialised (will remain 0)
for all CPUs in a policy, other than policy->cpu. When policy->cpu is
hotplugged out, the driver will incorrectly use the uninitialised (0)
value of the other CPUs when making frequency changes. Additionally,
the previous values stored in the perf_ctrls.desired_perf will be
lost when policy->cpu changes.
Therefore replace the array of per cpu data with a list. The memory for
each structure is allocated at policy init, where a single structure
can be allocated per policy, not per cpu. In order to accommodate the
struct list_head node in the cppc_cpudata structure, the now unused cpu
and cur_policy variables are removed.
For example, on a arm64 Juno platform with 6 CPUs: (0, 1, 2, 3) in PSD1,
(4, 5) in PSD2 - ANY coordination, the memory allocation comparison shows:
Before patch:
- ANY coordination:
total slack req alloc/free caller
0 0 0 0/1 _kernel_size_le_hi32+0x0xffff800008ff7810
0 0 0 0/6 _kernel_size_le_hi32+0x0xffff800008ff7808
128 80 48 1/0 _kernel_size_le_hi32+0x0xffff800008ffc070
768 0 768 6/0 _kernel_size_le_hi32+0x0xffff800008ffc0e4
After patch:
- ANY coordination:
total slack req alloc/free caller
256 0 256 2/0 _kernel_size_le_hi32+0x0xffff800008fed410
0 0 0 0/2 _kernel_size_le_hi32+0x0xffff800008fed274
Additional notes:
- A pointer to the policy's cppc_cpudata is stored in policy->driver_data
- Driver registration is skipped if _CPC entries are not present.
Signed-off-by: Ionela Voinescu <ionela.voinescu@arm.com>
Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2020-12-14 12:38:23 +00:00
|
|
|
free_cpu_data();
|
2016-04-15 00:45:53 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
module_exit(cppc_cpufreq_exit);
|
|
|
|
|
MODULE_AUTHOR("Ashwin Chaugule");
|
|
|
|
|
MODULE_DESCRIPTION("CPUFreq driver based on the ACPI CPPC v5.0+ spec");
|
|
|
|
|
MODULE_LICENSE("GPL");
|
|
|
|
|
|
2015-10-02 14:04:01 +00:00
|
|
|
late_initcall(cppc_cpufreq_init);
|
2016-10-14 18:00:23 +00:00
|
|
|
|
2018-10-02 22:34:57 +00:00
|
|
|
static const struct acpi_device_id cppc_acpi_ids[] __used = {
|
2016-10-14 18:00:23 +00:00
|
|
|
{ACPI_PROCESSOR_DEVICE_HID, },
|
|
|
|
|
{}
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
MODULE_DEVICE_TABLE(acpi, cppc_acpi_ids);
|