1 .\" Copyright (c) 2000-2001 John H. Baldwin <jhb@FreeBSD.org>
2 .\" All rights reserved.
4 .\" Redistribution and use in source and binary forms, with or without
5 .\" modification, are permitted provided that the following conditions
7 .\" 1. Redistributions of source code must retain the above copyright
8 .\" notice, this list of conditions and the following disclaimer.
9 .\" 2. Redistributions in binary form must reproduce the above copyright
10 .\" notice, this list of conditions and the following disclaimer in the
11 .\" documentation and/or other materials provided with the distribution.
13 .\" THIS SOFTWARE IS PROVIDED BY THE DEVELOPERS ``AS IS'' AND ANY EXPRESS OR
14 .\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
15 .\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
16 .\" IN NO EVENT SHALL THE DEVELOPERS BE LIABLE FOR ANY DIRECT, INDIRECT,
17 .\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
18 .\" NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
19 .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
20 .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
21 .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
22 .\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
35 .Nm atomic_readandclear ,
44 .Fn atomic_add_[acq_|rel_]<type> "volatile <type> *p" "<type> v"
46 .Fn atomic_clear_[acq_|rel_]<type> "volatile <type> *p" "<type> v"
48 .Fo atomic_cmpset_[acq_|rel_]<type>
49 .Fa "volatile <type> *dst"
54 .Fn atomic_fetchadd_<type> "volatile <type> *p" "<type> v"
56 .Fn atomic_load_acq_<type> "volatile <type> *p"
58 .Fn atomic_readandclear_<type> "volatile <type> *p"
60 .Fn atomic_set_[acq_|rel_]<type> "volatile <type> *p" "<type> v"
62 .Fn atomic_subtract_[acq_|rel_]<type> "volatile <type> *p" "<type> v"
64 .Fn atomic_store_rel_<type> "volatile <type> *p" "<type> v"
66 .Fn atomic_swap_<type> "volatile <type> *p" "<type> v"
68 .Fn atomic_testandset_<type> "volatile <type> *p" "u_int v"
70 Each of the atomic operations is guaranteed to be atomic across multiple
71 threads and in the presence of interrupts.
72 They can be used to implement reference counts or as building blocks for more
73 advanced synchronization primitives such as mutexes.
75 Each atomic operation operates on a specific
77 The type to use is indicated in the function name.
78 The available types that can be used are:
80 .Bl -tag -offset indent -width short -compact
86 unsigned integer the size of a pointer
88 unsigned 32-bit integer
90 unsigned 64-bit integer
93 For example, the function to atomically add two integers is called
96 Certain architectures also provide operations for types smaller than
99 .Bl -tag -offset indent -width short -compact
103 unsigned short integer
105 unsigned 8-bit integer
107 unsigned 16-bit integer
110 These must not be used in MI code because the instructions to implement them
111 efficiently might not be available.
112 .Ss Acquire and Release Operations
113 By default, a thread's accesses to different memory locations might not be
116 that is, the order in which the accesses appear in the source code.
117 To optimize the program's execution, both the compiler and processor might
118 reorder the thread's accesses.
119 However, both ensure that their reordering of the accesses is not visible to
121 Otherwise, the traditional memory model that is expected by single-threaded
122 programs would be violated.
123 Nonetheless, other threads in a multithreaded program, such as the
125 kernel, might observe the reordering.
126 Moreover, in some cases, such as the implementation of synchronization between
127 threads, arbitrary reordering might result in the incorrect execution of the
129 To constrain the reordering that both the compiler and processor might perform
130 on a thread's accesses, the thread should use atomic operations with
136 Most of the atomic operations on memory have three variants.
137 The first variant performs the operation without imposing any ordering
138 constraints on memory accesses to other locations.
139 The second variant has acquire semantics, and the third variant has release
141 In effect, operations with acquire and release semantics establish one-way
142 barriers to reordering.
144 When an atomic operation has acquire semantics, the effects of the operation
145 must have completed before any subsequent load or store (by program order) is
147 Conversely, acquire semantics do not require that prior loads or stores have
148 completed before the atomic operation is performed.
149 To denote acquire semantics, the suffix
151 is inserted into the function name immediately prior to the
152 .Dq Li _ Ns Aq Fa type
154 For example, to subtract two integers ensuring that subsequent loads and
155 stores happen after the subtraction is performed, use
156 .Fn atomic_subtract_acq_int .
158 When an atomic operation has release semantics, the effects of all prior
159 loads or stores (by program order) must have completed before the operation
161 Conversely, release semantics do not require that the effects of the
162 atomic operation must have completed before any subsequent load or store is
164 To denote release semantics, the suffix
166 is inserted into the function name immediately prior to the
167 .Dq Li _ Ns Aq Fa type
169 For example, to add two long integers ensuring that all prior loads and
170 stores happen before the addition, use
171 .Fn atomic_add_rel_long .
173 The one-way barriers provided by acquire and release operations allow the
174 implementations of common synchronization primitives to express their
175 ordering requirements without also imposing unnecessary ordering.
176 For example, for a critical section guarded by a mutex, an acquire operation
177 when the mutex is locked and a release operation when the mutex is unlocked
178 will prevent any loads or stores from moving outside of the critical
180 However, they will not prevent the compiler or processor from moving loads
181 or stores into the critical section, which does not violate the semantics of
183 .Ss Multiple Processors
184 In multiprocessor systems, the atomicity of the atomic operations on memory
185 depends on support for cache coherence in the underlying architecture.
186 In general, cache coherence on the default memory type,
187 .Dv VM_MEMATTR_DEFAULT ,
188 is guaranteed by all architectures that are supported by
190 For example, cache coherence is guaranteed on write-back memory by the
195 However, on some architectures, cache coherence might not be enabled on all
197 To determine if cache coherence is enabled for a non-default memory type,
198 consult the architecture's documentation.
201 architecture, coherency is only guaranteed for pages that are configured to
202 using a caching policy of either uncached or write back.
204 This section describes the semantics of each operation using a C like notation.
206 .It Fn atomic_add p v
207 .Bd -literal -compact
210 .It Fn atomic_clear p v
211 .Bd -literal -compact
214 .It Fn atomic_cmpset dst old new
215 .Bd -literal -compact
226 functions are not implemented for the types
233 .It Fn atomic_fetchadd p v
234 .Bd -literal -compact
243 functions are only implemented for the types
248 and do not have any variants with memory barriers at this time.
251 .Bd -literal -compact
258 functions are only provided with acquire memory barriers.
260 .It Fn atomic_readandclear p
261 .Bd -literal -compact
269 .Fn atomic_readandclear
270 functions are not implemented for the types
277 and do not have any variants with memory barriers at this time.
279 .It Fn atomic_set p v
280 .Bd -literal -compact
283 .It Fn atomic_subtract p v
284 .Bd -literal -compact
287 .It Fn atomic_store p v
288 .Bd -literal -compact
295 functions are only provided with release memory barriers.
297 .It Fn atomic_swap p v
298 .Bd -literal -compact
307 functions are not implemented for the types
314 and do not have any variants with memory barriers at this time.
316 .It Fn atomic_testandset p v
317 .Bd -literal -compact
318 bit = 1 << (v % (sizeof(*p) * NBBY));
319 tmp = (*p & bit) != 0;
326 .Fn atomic_testandset
327 functions are only implemented for the types
332 and do not have any variants with memory barriers at this time.
336 is currently not implemented for any of the atomic operations on the
345 function returns the result of the compare operation.
347 .Fn atomic_fetchadd ,
349 .Fn atomic_readandclear ,
352 functions return the value at the specified address.
354 .Fn atomic_testandset
355 function returns the result of the test operation.
357 This example uses the
358 .Fn atomic_cmpset_acq_ptr
361 functions to obtain a sleep mutex and handle recursion.
370 /* Try to obtain mtx_lock once. */
371 #define _obtain_lock(mp, tid) \\
372 atomic_cmpset_acq_ptr(&(mp)->mtx_lock, MTX_UNOWNED, (tid))
374 /* Get a sleep lock, deal with recursion inline. */
375 #define _get_sleep_lock(mp, tid, opts, file, line) do { \\
376 uintptr_t _tid = (uintptr_t)(tid); \\
378 if (!_obtain_lock(mp, tid)) { \\
379 if (((mp)->mtx_lock & MTX_FLAGMASK) != _tid) \\
380 _mtx_lock_sleep((mp), _tid, (opts), (file), (line));\\
382 atomic_set_ptr(&(mp)->mtx_lock, MTX_RECURSE); \\
383 (mp)->mtx_recurse++; \\
395 operations were first introduced in
397 This first set only supported the types
406 .Fn atomic_readandclear ,
409 operations were added in
418 and all of the acquire and release variants
424 operations were added in
429 .Fn atomic_testandset
430 operations were added in