rlibby [Wed, 27 Nov 2019 19:49:55 +0000 (19:49 +0000)]
uma: trash memory when ctor/dtor supplied too
On INVARIANTS kernels, UMA has a use-after-free detection mechanism.
This mechanism previously required that all of the ctor/dtor/uminit/fini
arguments to uma_zcreate() be NULL in order to function. Now, it only
requires that uminit and fini be NULL; now, the trash ctor and dtor will
be called in addition to any supplied ctor or dtor.
Also do a little refactoring for readability of the resulting logic.
This enables use-after-free detection for more zones, and will allow for
simplification of some callers that worked around the previous
restriction (see kern_mbuf.c).
tuexen [Wed, 27 Nov 2019 19:32:29 +0000 (19:32 +0000)]
Plug two mbuf leaks during INIT-ACK handling.
One leak happens when there is not enough memory to allocate the
the resources for streams. The other leak happens if the are
unknown parameters in the received INIT-ACK chunk which require
reporting and the INIT-ACK requires sending an ABORT due to illegal
parameter combinations.
Hopefully this fixes
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=19083
andrew [Wed, 27 Nov 2019 16:52:46 +0000 (16:52 +0000)]
Support kernels larger than EFI_STAGING_SIZE in loader.efi
With a very large kernel or module the staging area may be too small to
hold it. When this is the case try to allocate more space before failing
in the efi copyin/copyout/readin functions.
rlibby [Wed, 27 Nov 2019 01:54:39 +0000 (01:54 +0000)]
witness: sleepable rm locks are not sleepable in read mode
There are two classes of rm lock, one "sleepable" and one not. But even
a "sleepable" rm lock is only sleepable in write mode, and is
non-sleepable when taken in read mode.
Warn about sleepable rm locks in read mode as non-sleepable locks. Do
this by defining a new lock operation flag, LOP_NOSLEEP, to indicate
that a lock is non-sleepable despite what the LO_SLEEPABLE flag would
indicate, and defining a new witness lock instance flag, LI_SLEEPABLE,
to track the product of LO_SLEEPABLE and LOP_NOSLEEP on the lock
instance.
mjg [Wed, 27 Nov 2019 01:20:55 +0000 (01:20 +0000)]
cache: fix numcache accounting on entry
. entries are never created and .. can reuse existing entries,
meaning the early count bump is both spurious and leading to
overcounting in certain cases.
jeff [Wed, 27 Nov 2019 00:39:23 +0000 (00:39 +0000)]
Use atomics in more cases for object references. We now can completely
omit the object lock if we are above a certain threshold. Hold only a
single vnode reference when the vnode object has any ref > 0. This
allows us to only lock the object and vnode on 0-1 and 1-0 transitions.
jeff [Tue, 26 Nov 2019 22:17:02 +0000 (22:17 +0000)]
Refactor uma_zalloc_arg(). It is a mess of gotos and code which doesn't
make sense after many partial refactors. Attempt to make a smaller cache
footprint for the fast path.
Summary:
Various paths through hypot(x, y) will multiply x and y by a power of
two, perform the calculation in a range where IEEE-754 provides greater
precision, then undo the multiplication to determine the true result.
Undoing that multiplication is implemented as t1*w, where t1=2**k.
2**k is often computed by taking the high word of 1.0, then adding k<<20
(for doubles or long doubles) or k<<23 (for floats) to it, then
overwriting that high word. But when k is negative this left-shifts a
negative value -- and that's undefined behavior in many editions of C
and C++.
This patch should fix all hypot implementations to compute 2**k without
triggering this particular bit of undefined behavior.
Test Plan: I've only very lightly tested out the hypot(double, double)
change, in SpiderMonkey's JavaScript engine, for consistency with prior
behavior. The other functions' changes have more or less only been
eyeballed. Careful examination appreciated! Do note, however, that an
error in any of these changes would most likely produce a value that is
incorrect by a factor of two, so any mistake would most likely be
glaring if invoked.
emaste [Tue, 26 Nov 2019 20:46:20 +0000 (20:46 +0000)]
stop building arm LINT-V5 kernel
r354290 removed arm.arm from universe, but arm.arm kernels were still
found and built during the kernel stage. r354934 tagged armv5 kernel
configs as NO_UNIVERSE, but LINT-V5 remained. Stop building it as well.
Leave the clean rule in place for now so folks don't end up with a stale
LINT-V5.
Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22560
mmel [Tue, 26 Nov 2019 17:56:39 +0000 (17:56 +0000)]
Finish implementation of RK3299 clocks.
- implement of all but mmc clocks. MMC clocks will be added later by own commit.
- use 'link' clock type for external clocks.
- use macros for initialization of structure's named members.
MFC after: 3 weeks
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D22441
scottl [Tue, 26 Nov 2019 17:25:49 +0000 (17:25 +0000)]
Revert r355021. In my haste to grep for Giant, I missed that it was in
conditional ifdefs for this driver. We will consider removing those ifdefs
in the future.
np [Tue, 26 Nov 2019 05:54:25 +0000 (05:54 +0000)]
cxgbe(4): Allow the driver to specify multiple FECs that the firmware
should try in order to link up with the peer.
Various FEC variables within the driver can now have multiple bits set
instead of being powers of 2. 0 and -1 in the user knobs still mean no
FEC and auto (driver decides) respectively for backward compatibility,
but no-FEC and auto now have their own bits in the internal
representation. There is a new bit that can be set to request the FEC
recommended by the cable/transceiver module.
Add sysctls to display link related capabilities of the local side as
well as the link partner.
Note that all this needs a new firmware and the documentation for the
driver FEC knobs will be updated after that firmware is added to the
driver.
ian [Mon, 25 Nov 2019 19:59:53 +0000 (19:59 +0000)]
Allow opt-out of automatic ntpd leapfile checking/fetching.
When a system has no internet connection, or when it is configured to obtain
ntpd leapfiles from some source other than the internet, or even when the
sysadmin has decided for some reason to customize ntp.conf to eliminate use
of the leapfile, the rc.d/ntpd script emits various error messages related
to the file.
This change allows setting the rc var ntp_db_leapfile to NONE to disable all
automatic processing related to that file in rc.d/ntpd.
chs [Mon, 25 Nov 2019 19:31:38 +0000 (19:31 +0000)]
In ffs_freefile(), use a separate variable to hold the inode number within
the cg rather than reusuing "ino" for this purpose. This reduces the diff
for an upcoming change that improves handling of I/O errors.
oshogbo [Mon, 25 Nov 2019 18:33:21 +0000 (18:33 +0000)]
procdesc: allow to collect status through wait(1) if process is traced
The debugger like truss(1) depends on the wait(2) syscall. This syscall
waits for ALL children. When it is waiting for ALL child's the children
created by process descriptors are not returned. This behavior was
introduced because we want to implement libraries which may pdfork(1).
The behavior of process descriptor brakes truss(1) because it will
not be able to collect the status of processes with process descriptors.
To address this problem the status is returned to parent when the
child is traced. While the process is traced the debugger is the new parent.
In case the original parent and debugger are the same process it means the
debugger explicitly used pdfork() to create the child. In that case the debugger
should be using kqueue()/pdwait() instead of wait().
Add test case to verify that. The test case was implemented by markj@.
emaste [Mon, 25 Nov 2019 18:18:28 +0000 (18:18 +0000)]
remove armv6 LLVM workaround introduced in r341812
r341812 enabled only arm target support in LLVM on arm and armv6,
because ld.bfd 2.17.50 lacked support for range extensions required for
linking such large binaries/libraries. r341812 indicated that the
workaround should be removed once the userland can be linked by lld.
r354289 switched armv6 to use lld by default, so remove the workaround
on armv6. The workaround remains in place for arm (v5), and will
presumably be removed when arm is retired.
rlibby [Mon, 25 Nov 2019 07:38:27 +0000 (07:38 +0000)]
sysctl sysctls: wire old buf before output with sysctl lock
Several sysctl sysctls output to a user buffer while holding a
non-sleepable lock that protects the sysctl topology. They need to wire
the output buffer, or else they may try to sleep on a page fault.
dougm [Mon, 25 Nov 2019 02:19:47 +0000 (02:19 +0000)]
Where 'current' is used to index over vm_map entries, use
'entry'. Where 'entry' is used to identify the starting point for
iteration, use 'first_entry'. These are the naming conventions used in
most of the vm_map.c code. Where VM_MAP_ENTRY_FOREACH can be used, do
so. Squeeze a few lines to fit in 80 columns. Where lines are being
modified for these reasons, look to remove style(9) violations.
bz [Sun, 24 Nov 2019 23:21:47 +0000 (23:21 +0000)]
Allow kernel to compile without BPF.
r297816 added some bpf magic for VIMAGE unconditionally which no longer
allows kernels to compile without bpf (but with other networking).
Add the missing ifdef checks and allow a kernel to compile without bpf
again.
PR: 242136
Reported by: dave mischler.com
MFC after: 2 weeks
ian [Sun, 24 Nov 2019 21:08:56 +0000 (21:08 +0000)]
When doing ARM stack unwinding as part of stack_save(9), do not search
loaded modules (pass 0/false for the can_lock arg). Searching the unwind
info in modules acquires an exclusive sxlock, and the stack(9) functions can
be called in a context where unbounded sleeps are forbidden (such as from
the witness checkorder code).
Just ignoring the existence of modules in stack_save() is not ideal, so I'm
looking for a better solution, but this commit will make it possible to boot
an ARM kernel with WITNESS enabled again, until I get something better.
wulf [Sun, 24 Nov 2019 20:47:40 +0000 (20:47 +0000)]
Linux epoll: Register events with zero event mask
Such an events are legal and should be interpreted as EPOLLERR | EPOLLHUP.
Register a disabled kqueue event in that case as we do not support EPOLLHUP yet.
Required by Linux Steam client.
PR: 240590
Reported by: Alex S <iwtcex@gmail.com>
Reviewed by: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D22516
wulf [Sun, 24 Nov 2019 20:44:14 +0000 (20:44 +0000)]
Linux epoll: Check both read and write kqueue events existence in EPOLL_CTL_ADD
Linux epoll EPOLL_CTL_ADD op handler should always check registration
of both EVFILT_READ and EVFILT_WRITE kevents to deceide if supplied
file descriptor fd is already registered with epoll instance.
wulf [Sun, 24 Nov 2019 20:41:47 +0000 (20:41 +0000)]
Linux epoll: Don't deregister file descriptor after EPOLLONESHOT is fired
Linux epoll does not remove descriptor after one-shot event has been triggered.
Set EV_DISPATCH kqueue flag rather then EV_ONESHOT to get the same behavior.
Required by Linux Steam client.
PR: 240590
Reported by: Alex S <iwtcex@gmail.com>
Reviewed by: emaste, imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D22513
kib [Sun, 24 Nov 2019 19:12:23 +0000 (19:12 +0000)]
Record part of the owner struct thread pointer into busy_lock.
Record as much bits from curthread into busy_lock as fits. Low bits
for struct thread * representation are zero due to struct and zone
alignment, and they leave space for busy flags (perhaps except
statically allocated thread0). Upper bits are not very interesting
for assert, and in most practical situations recorded value should
allow to manually identify the owner with certainity.
Assert that unbusy is performed by the owner, except few places where
unbusy is done in io completion handler. For this case, add
_unchecked variants of asserts and unbusy primitives.
Reviewed by: markj (previous version)
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D22298
kib [Sun, 24 Nov 2019 19:06:38 +0000 (19:06 +0000)]
tmpfs: resolve deadlock between rename and unmount.
Top-level kern_renameat() increases the writecount on the mount point,
which, together with tmpfs unmount suspending the mount, already
ensures that unmount cannot proceed while rename unlocks and relocks
all operated vnodes.
Remove vfs_busy() call from tmpfs_rename() which was done while
holding a vnode lock, creating the deadlock. The only intent of the
busy operation seems to be the prevention of unmount, which is already
ensured.
Reported and tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
imp [Sun, 24 Nov 2019 15:37:19 +0000 (15:37 +0000)]
Don't need giant for these drivers dev nodes.
Also, Giant isn't required to busy / unbusy a device, so drop that too while I'm
here. It's not done elsewhere in the tree and in the future will likely be
handled by a node lock to ensure consistency. Leave Giant in place for attach
and removing childing, as that's actually still needed, even if imperfect.
Remove stale comment about contigmalloc taking Giant and calling w/o the lock
held. Neither of these is still true.
imp [Sun, 24 Nov 2019 15:37:14 +0000 (15:37 +0000)]
Hoist locking giant back up into the ioctl handler
Move the locking back into the ioctl handler. This "fixes" the race where we hve
a hot plug event just after the dropping of Giant in pci_find_dbsf, assuming the
driver doesn't then call anything that drops and picks up Giant again... It's a
little safer since don't think it doesn't, but we lack the tools to know for
sure.
imp [Sun, 24 Nov 2019 15:24:05 +0000 (15:24 +0000)]
Fix leak in state machine for commands.
When we get a device departed message from the firmware, we send a TARGET_REST
to the device to let the firmware know we're done and as part of the recovery
process. This will abort all the commands. While the documentation says the IOC
is responsible for writing the completion message for all the commands pending
with an aborted status, we sometimes have queued commands for the target that
haven't been completed so are in the INQUEUE state. So, when we later complete
the pending CCB as aborted, these commands are freed and we hit the "state not
busy" panic.
Elsewhere where we dequeue commands, we move the state to BUSY from INQUEUE. Do
that here as well. In talking to Ken, Scott and Justin, they recommended a
series of tests to see if this is 100% safe. Those tests are ongoing, but
preliminary tests suggest this is safe as we see no duplicate completions when
we hit this case at work. We have a machine that has a dodgy powersupply which
usually doesn't apply power to a few drives, but sometimes does when the machine
is under heavy load so we get a rash of the connect / disconnect messages over
half an hour. Without this change, we'd see state not busy panic. With this
change, the drives just annoyingly come and go without affecting the rest of the
machine, but without a complete error injection test suite, it's hard to know if
all edge cases are now covered or not.
lwhsu [Sun, 24 Nov 2019 15:03:35 +0000 (15:03 +0000)]
Fix gcc build
We have -Werror=strict-overflow so gcc complains:
In file included from /tmp/obj/workspace/src/amd64.amd64/tmp/usr/include/bitstring.h:36:0,
from /workspace/src/tests/sys/sys/bitstring_test.c:34:
/workspace/src/tests/sys/sys/bitstring_test.c: In function 'bit_ffc_at_test':
/workspace/src/sys/sys/bitstring.h:239:5: error: assuming signed overflow does not occur when assuming that (X + c) >= X is always true [-Werror=strict-overflow]
if (_start >= _nbits) {
^
Disable assuming overflow of signed integer will never happen by specifying
-fno-strict-overflow
jhibbits [Sun, 24 Nov 2019 04:35:29 +0000 (04:35 +0000)]
rtld/powerpc: Fix _rtld_bind_start for powerpcspe
Summary:
We need to save off the full 64-bit register, not just the low 32 bits,
of all registers getting saved off in _rtld_bind_start. Additionally,
we need to save off the other SPE registers (SPEFSCR and accumulator),
so that their program state is not affected by the PLT resolver.
imp [Sat, 23 Nov 2019 23:57:26 +0000 (23:57 +0000)]
Add a warning about Giant Locked devices
Add a warning when a device registers with devfs and requests
D_NEEDGIANT. The warning says the device will go away before
13.0. This is needed to flush out the devices in the tree that are
still Giant locked. This warning, or some variant of it, should have
gone into the tree a long time ago...
The intention is to require all devices be converted to not use
automatic giant in this way, or remove any such devices that remain
that we don't have the hardware to test a conversion of.
kbd so far is the only device that can't leave the tree, yet needs
something sensible done to avoid the auto giant lock (even if it is
just doing the wrapping itself). There may be others added to this
list... Any discussions of this topic will take place on arch@.
imp [Sat, 23 Nov 2019 23:44:00 +0000 (23:44 +0000)]
We don't even need Giant here. It isn't protecting anything internal
to geom, and nothing we call requires it to be held. It's left over
from a time when the latter wasn't the case. Retire it.
imp [Sat, 23 Nov 2019 23:43:52 +0000 (23:43 +0000)]
Push Giant down one layer
The /dev/pci device doesn't need GIANT, per se. However, one routine
that it calls, pci_find_dbsf implicitly does. It walks a list that can
change when PCI scans a new bus. With hotplug, this means we could
have a race with that scanning. To prevent that, take out Giant around
scanning the list.
However, given that we have places in the tree that drop giant, if
held when we call into them, the whole use of Giant to protect newbus
may be less effective that we desire, so add a comment about why we're
talking it out, and we'll address the issue when we lock newbus with
something other than Giant.
bdragon [Sat, 23 Nov 2019 21:18:55 +0000 (21:18 +0000)]
[PowerPC] Use QEMU-compatible version of SPE accumulator save
Switch from "evaddumiaaw 0,0" to "evmwumiaa 0,0,0" when persisting the
accumulator. This has the benefit of actually being implemented in QEMU
as it is the form Linux uses for the same task.
Both instructions are functionally equivilent, as we are using them for
their side effect of copying the accumulator to GPRs rather than for the
actual math operation that they are performing.
dim [Sat, 23 Nov 2019 19:35:09 +0000 (19:35 +0000)]
libclang_rt: enable on powerpc*
Summary:
Enable on powerpc64 and in lib/libclang_rt/Makefile change
MACHINE_CPUARCH to MACHINE_ARCH because on powerpc64
MACHINE_ARCH==MACHINE_CPUARCH so the 32-bit library overwrites 64-bit
library during installworld.
This patch doesn't enable any other libclang_rt libraries because they
need to be separately ported.
I have verified that games/julius (which fails on powerpc64 elfv2
without this change because of no libclang_rt profiling library) builds.
Test Plan: Ship it, test on powerpc and powerpcspe
kevans [Sat, 23 Nov 2019 03:38:26 +0000 (03:38 +0000)]
arm64/NOTES: add SOC_BRCM_BCM2838
This should have been done back when it was added, but it was not. It only
really adds an extra entry for memory mapping bits in bcm2835_vcbus.c, so
nothing too extensive yet.
kevans [Sat, 23 Nov 2019 01:47:17 +0000 (01:47 +0000)]
bcm2835_dma: rip out the "use_dma" flag, make it non-optional
Now that it works for the Raspberry Pi 4, we can discontinue our workarounds
that were put in place to at least get a bootable kernel for other testing.
kevans [Sat, 23 Nov 2019 01:46:02 +0000 (01:46 +0000)]
bcm2835_sdhci: "fix" DMA on the RPi 4
According to the documentation I have, DREQ pacing should be required here.
The DREQ# hasn't changed since the BCM2835. As soon as we attempt to setup
DREQ, DMA stalls and there's no clear reason why as of yet. Setting this
back to NONE seems to work just as well, though it's yet to be determined if
this is a sustainable model in high-throughput scenarios.
cem [Fri, 22 Nov 2019 23:23:40 +0000 (23:23 +0000)]
Add explicit SI_SUB_EPOCH
Add explicit SI_SUB_EPOCH, after SI_SUB_TASKQ and before SI_SUB_SMP
(EARLY_AP_STARTUP). Rename existing "SI_SUB_TASKQ + 1" to SI_SUB_EPOCH.
epoch(9) consumers cannot epoch_alloc() before SI_SUB_EPOCH:SI_ORDER_SECOND,
but likely should allocate before SI_SUB_SMP. Prior to this change,
consumers (well, epoch itself, and net/if.c) just open-coded the
SI_SUB_TASKQ + 1 order to match epoch.c, but this was fragile.
mav [Fri, 22 Nov 2019 21:31:59 +0000 (21:31 +0000)]
Do not retry long ready waits if previous gave nothing.
I have some disks reporting "Logical unit is in process of becoming ready"
for about half an hour before finally reporting failure. During that time
CAM waits for the readiness during ~2 minutes for each request, that makes
system boot take very long time.
This change reduces wait times for the following requests to ~1 second if
previously long wait for that device has timed out.
cem [Fri, 22 Nov 2019 20:22:29 +0000 (20:22 +0000)]
random(4): Use ordinary sysctl definitions
There's no need to dynamically populate them; the SYSCTL_ macros take care
of load/unload appropriately already (and random_harvestq is 'standard' and
cannot be unloaded anyway).
cem [Fri, 22 Nov 2019 19:30:31 +0000 (19:30 +0000)]
random/ivy: Provide mechanism to read independent seed values from rdrand
On x86 platforms with the intrinsic, rdrand is a deterministic bit generator
(AES-CTR) seeded from an entropic source. On x86 platforms with rdseed, it
is something closer to the upstream entropic source. (There is more nuance;
a block diagram is provided in [1].)
On devices with rdrand and without rdseed, there is no good intrinsic for
acecssing the good entropic soure directly. However, the DRBG is guaranteed
to reseed every 8 kB on these platforms. As a conservative option, on such
hardware we can read an extra 7.99kB samples every time we want a sample
from an independent seed.
As one can imagine, this drastically slows the effective read rate of
RDRAND (a factor of 1024 on amd64 and 2048 on ia32). Microbenchmarks on AMD
Zen (has RDSEED) show an RDRAND rate of 25 MB/s and Intel Haswell (no
RDSEED) show RDRAND of 170 MB/s. This would reduce the read rate on Haswell
to ~170 kB/s (at 100% CPU). random(4)'s harvestq thread periodically
"feeds" from pure sources in amounts of 128-1024 bytes. On Haswell,
enabling this feature increases the CPU time of RDRAND in each "feed" from
approximately 0.7-6 µs to 0.7-6 ms.
Because there is some performance penalty to this more conservative option,
a knob is provided to enable the change. The change does not affect
platforms with RDSEED.
mav [Fri, 22 Nov 2019 18:39:51 +0000 (18:39 +0000)]
Make CAM use root_mount_hold_token() to delay boot.
Before this change CAM used config_intrhook_establish() for this purpose,
but that approach does not allow to delay it again after releasing once.
USB stack uses root_mount_hold() to delay boot until bus scan is complete.
But once it is, CAM had no time to scan SCSI bus, registered by umass(4),
if it already done other scans and called config_intrhook_disestablish().
The new approach makes it work smooth, assuming the USB device is found
during the initial bus scan. Devices appearing on USB bus later may still
require setting kern.cam.boot_delay, but hopefully those are minority.
markj [Fri, 22 Nov 2019 16:31:30 +0000 (16:31 +0000)]
Reclaim memory from UMA if the page daemon is struggling.
Use the UMA reclaim thread to asynchronously drain all caches if
there is a severe shortage in a domain. Otherwise we only trigger UMA
reclamation every 10s even when the system has completely run out of
memory.
Stop entirely draining the caches when one domain falls below its min
threshold. In some workloads it is normal for one NUMA domain to end
up being nearly depleted by kernel memory allocations, for example for
the ZFS ARC. The domainset iterators skip domains below the
vmd_min_free theshold on the first iteration, so we should allow that
mechanism to limit further depletion of the domain's free pages before
taking the extreme step of calling uma_reclaim(UMA_RECLAIM_DRAIN_CPU).
Discussed with: jeff
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22395
markj [Fri, 22 Nov 2019 16:31:10 +0000 (16:31 +0000)]
Update the checks in vm_page_zone_import().
- Remove the cnt == 1 check. UMA passes cnt == 1 when it has disabled
per-CPU caching. In this case we might as well just allocate a single
page and return it to the caller, since the caller is going to do
exactly that anyway if the UMA cache allocation attempt fails.
- Don't replenish caches if the domain is severely short on free pages.
With large buckets we may otherwise quickly exacerbate a situation
where the page daemon is failing to keep up.
- Don't replenish caches if the calling thread belongs to the page
daemon, which should avoid creating extra memory pressure when it is
trying to free memory. Virtually all such allocations while occur in
the context of laundering, where the laundry thread must allocate
slabs for various swap and I/O-related UMA zones.
Reviewed by: kib
Discussed with: alc, jeff
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22394
markj [Fri, 22 Nov 2019 16:30:47 +0000 (16:30 +0000)]
Revise the page cache size policy.
In r353734 the use of the page caches was limited to systems with a
relatively large amount of RAM per CPU. This was to mitigate some
issues reported with the system not able to keep up with memory pressure
in cases where it had been able to do so prior to the addition of the
direct free pool cache. This change re-enables those caches.
The change modifies uma_zone_set_maxcache(), which was introduced
specifically for the page cache zones. Rather than using it to limit
only the full bucket cache, have it also set uz_count_max to provide an
upper bound on the per-CPU cache size that is consistent with the number
of items requested. Remove its return value since it has no use.
Enable the page cache zones unconditionally, and limit them to 0.1% of
the domain's pages. The limit can be overridden by the
vm.pgcache_zone_max tunable as before.
Change the item size parameter passed to uma_zcache_create() to the
correct size, and stop setting UMA_ZONE_MAXBUCKET. This allows the page
cache buckets to be adaptively sized, like the rest of UMA's caches.
This also causes the initial bucket size to be small, so only systems
which benefit from large caches will get them.
Reviewed by: gallatin, jeff
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22393
markj [Fri, 22 Nov 2019 16:28:52 +0000 (16:28 +0000)]
Fix locking in vm_reserv_reclaim_contig().
We were not properly handling the case where the trylock of the
reservaton fails, in which case we could leak reservation lock.
Introduce a marker reservation to implement precise scanning in
vm_reserv_reclaim_contig(). Before, a race could result in early
termination of the scan in rare situations. Use the marker's lock to
serialize scans of the partpop queue so that a global marker structure
can be used. Modify vm_reserv_reclaim_inactive() to handle the presence
of a marker while minimizing the hold time of domain-global locks.
Reviewed by: alc, jeff, kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22392