summaryrefslogtreecommitdiff
path: root/kernel/rcutree.h
diff options
context:
space:
mode:
authorPaul E. McKenney <paulmck@linux.vnet.ibm.com>2012-12-03 13:52:00 -0800
committerPaul E. McKenney <paulmck@linux.vnet.ibm.com>2013-01-08 14:15:57 -0800
commitdc35c8934eba959b690921615fcd987e8bc17e4a (patch)
treec8020ec8ac191505e35905e89e58f02aa4e5fc31 /kernel/rcutree.h
parent1b0048a44c502c5ab850203e6e0a6498d7d8676d (diff)
rcu: Tag callback lists with corresponding grace-period number
Currently, callbacks are advanced each time the corresponding CPU notices a change in its leaf rcu_node structure's ->completed value (this value counts grace-period completions). This approach has worked quite well, but with the advent of RCU_FAST_NO_HZ, we cannot count on a given CPU seeing all the grace-period completions. When a CPU misses a grace-period completion that occurs while it is in dyntick-idle mode, this will delay invocation of its callbacks. In addition, acceleration of callbacks (when RCU realizes that a given callback need only wait until the end of the next grace period, rather than having to wait for a partial grace period followed by a full grace period) must be carried out extremely carefully. Insufficient acceleration will result in unnecessarily long grace-period latencies, while excessive acceleration will result in premature callback invocation. Changes that involve this tradeoff are therefore among the most nerve-wracking changes to RCU. This commit therefore explicitly tags groups of callbacks with the number of the grace period that they are waiting for. This means that callback-advancement and callback-acceleration functions are idempotent, so that excessive acceleration will merely waste a few CPU cycles. This also allows a CPU to take full advantage of any grace periods that have elapsed while it has been in dyntick-idle mode. It should also enable simulataneous simplifications to and optimizations of RCU_FAST_NO_HZ. Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Diffstat (limited to 'kernel/rcutree.h')
-rw-r--r--kernel/rcutree.h2
1 files changed, 2 insertions, 0 deletions
diff --git a/kernel/rcutree.h b/kernel/rcutree.h
index 4b69291b093d..c9f362e9b13d 100644
--- a/kernel/rcutree.h
+++ b/kernel/rcutree.h
@@ -282,6 +282,8 @@ struct rcu_data {
*/
struct rcu_head *nxtlist;
struct rcu_head **nxttail[RCU_NEXT_SIZE];
+ unsigned long nxtcompleted[RCU_NEXT_SIZE];
+ /* grace periods for sublists. */
long qlen_lazy; /* # of lazy queued callbacks */
long qlen; /* # of queued callbacks, incl lazy */
long qlen_last_fqs_check;