]> CyberLeo.Net >> Repos - FreeBSD/FreeBSD.git/commit
amd64 pmap: rework delayed invalidation, removing global mutex.
authorkib <kib@FreeBSD.org>
Thu, 16 May 2019 13:28:48 +0000 (13:28 +0000)
committerkib <kib@FreeBSD.org>
Thu, 16 May 2019 13:28:48 +0000 (13:28 +0000)
commitb10ca253846a115d49a36a112d6a578bfc8bc8d2
treedb3a21ac2f601c06c225e3a9439a5aa7374a4613
parentd20c4c3cb26a46121e793f89972a09dfce67eb74
amd64 pmap: rework delayed invalidation, removing global mutex.

For machines having cmpxcgh16b instruction, i.e. everything but very
early Athlons, provide lockless implementation of delayed
invalidation.

The implementation maintains lock-less single-linked list with the
trick from the T.L. Harris article about volatile mark of the elements
being removed. Double-CAS is used to atomically update both link and
generation.  New thread starting DI appends itself to the end of the
queue, setting the generation to the generation of the last element
+1.  On DI finish, thread donates its generation to the previous
element.  The generation of the fake head of the list is the last
passed DI generation.  Basically, the implementation is a queued
spinlock but without spinlock.

Many thanks both to Peter Holm and Mark Johnson for keeping with me
while I produced intermediate versions of the patch.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
MFC note: td_md.md_invl_gen should go to the end of struct thread
Differential revision: https://reviews.freebsd.org/D19630
sys/amd64/amd64/pmap.c
sys/amd64/amd64/trap.c
sys/amd64/amd64/vm_machdep.c
sys/amd64/include/pmap.h
sys/amd64/include/proc.h
sys/kern/kern_thread.c