diff options
| author | Peter Zijlstra <peterz@infradead.org> | 2013-08-14 14:51:00 +0200 |
|---|---|---|
| committer | Ingo Molnar <mingo@kernel.org> | 2013-09-25 14:07:57 +0200 |
| commit | c2daa3bed53a81171cf8c1a36db798e82b91afe8 (patch) | |
| tree | a7a7b1676e3101b986276f9ec0eab0ab5183e1ed /arch/x86/kernel/entry_64.S | |
| parent | a233f1120c37724938f7201fe2353b2577adaaf9 (diff) | |
| download | kernel-common-c2daa3bed53a81171cf8c1a36db798e82b91afe8.tar.gz kernel-common-c2daa3bed53a81171cf8c1a36db798e82b91afe8.tar.bz2 kernel-common-c2daa3bed53a81171cf8c1a36db798e82b91afe8.zip | |
sched, x86: Provide a per-cpu preempt_count implementation
Convert x86 to use a per-cpu preemption count. The reason for doing so
is that accessing per-cpu variables is a lot cheaper than accessing
thread_info variables.
We still need to save/restore the actual preemption count due to
PREEMPT_ACTIVE so we place the per-cpu __preempt_count variable in the
same cache-line as the other hot __switch_to() variables such as
current_task.
NOTE: this save/restore is required even for !PREEMPT kernels as
cond_resched() also relies on preempt_count's PREEMPT_ACTIVE to ignore
task_struct::state.
Also rename thread_info::preempt_count to ensure nobody is
'accidentally' still poking at it.
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/n/tip-gzn5rfsf8trgjoqx8hyayy3q@git.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Diffstat (limited to 'arch/x86/kernel/entry_64.S')
| -rw-r--r-- | arch/x86/kernel/entry_64.S | 4 |
1 files changed, 1 insertions, 3 deletions
diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S index 1b69951a81e2..6a43e7d29fe7 100644 --- a/arch/x86/kernel/entry_64.S +++ b/arch/x86/kernel/entry_64.S @@ -1118,10 +1118,8 @@ retint_signal: /* Returning to kernel space. Check if we need preemption */ /* rcx: threadinfo. interrupts off. */ ENTRY(retint_kernel) - cmpl $0,TI_preempt_count(%rcx) + cmpl $0,PER_CPU_VAR(__preempt_count) jnz retint_restore_args - bt $TIF_NEED_RESCHED,TI_flags(%rcx) - jnc retint_restore_args bt $9,EFLAGS-ARGOFFSET(%rsp) /* interrupts off? */ jnc retint_restore_args call preempt_schedule_irq |