diff options
authorSimon Horman <>2013-05-22 14:50:31 +0900
committerMaciej Wereski <>2014-12-29 09:31:34 +0100
commitd71cd852d88a36a25b0e8ad05a5f72f540bdd085 (patch)
parent960e9f6d75ac7720fc7af5770369d67af6adb575 (diff)
sched: add cond_resched_rcu() helper
This is intended for use in loops which read data protected by RCU and may have a large number of iterations. Such an example is dumping the list of connections known to IPVS: ip_vs_conn_array() and ip_vs_conn_seq_next(). The benefits are for CONFIG_PREEMPT_RCU=y where we save CPU cycles by moving rcu_read_lock and rcu_read_unlock out of large loops but still allowing the current task to be preempted after every loop iteration for the CONFIG_PREEMPT_RCU=n case. The call to cond_resched() is not needed when CONFIG_PREEMPT_RCU=y. Thanks to Paul E. McKenney for explaining this and for the final version that checks the context with CONFIG_DEBUG_ATOMIC_SLEEP=y for all possible configurations. The function can be empty in the CONFIG_PREEMPT_RCU case, rcu_read_lock and rcu_read_unlock are not needed in this case because the task can be preempted on indication from scheduler. Thanks to Peter Zijlstra for catching this and for his help in trying a solution that changes __might_sleep. Initial cond_resched_rcu_lock() function suggested by Eric Dumazet. Tested-by: Julian Anastasov <> Signed-off-by: Julian Anastasov <> Signed-off-by: Simon Horman <> Acked-by: Peter Zijlstra <> Signed-off-by: Pablo Neira Ayuso <> Change-Id: I5f36f86484198f9064725d424c3d91d5fac8e1d4 Origin: Backported-by: Maciej Wereski <> Signed-off-by: Maciej Wereski <>
1 files changed, 9 insertions, 0 deletions
diff --git a/include/linux/sched.h b/include/linux/sched.h
index f87e9a8d364..e12df9e68e3 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -2462,6 +2462,15 @@ extern int __cond_resched_softirq(void);
__cond_resched_softirq(); \
+static inline void cond_resched_rcu(void)
+ rcu_read_unlock();
+ cond_resched();
+ rcu_read_lock();
* Does a critical section need to be broken due to another
* task waiting?: (technically does not depend on CONFIG_PREEMPT,