mjg [Sat, 8 Dec 2018 10:22:12 +0000 (10:22 +0000)]
Fix a corner case in ID bitmap management.
If all IDs from trypid to pid_max were used as pids, the code would enter
a loop which would be infinite if none of the IDs could become free (e.g.
they all belong to processes which did not transitioned to zombie).
Fixes: r341684 ("Manage process-related IDs with bitmaps")
kib [Fri, 7 Dec 2018 23:05:12 +0000 (23:05 +0000)]
Fix expression evaluation.
Braces were put in the wrong place, causing failing EAGAIN check to
return zero result. Remove the problematic assignment from the
conditional expression at all.
While there, remove used once variable vp, and wrap too long line.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
mav [Fri, 7 Dec 2018 20:30:00 +0000 (20:30 +0000)]
Fix several iov handling bugs in bhyve virtio-scsi backend.
- buf_to_iov() does not use buflen parameter, allowing out of bound read.
- buf_to_iov() leaks memory if seek argument > 0.
- iov_to_buf() doesn't need to reallocate buffer for every segment.
- there is no point to use size_t for iov counts, int is more then enough.
- some iov function arguments can be constified.
- pci_vtscsi_request_handle() used truncate_iov() incorrectly, allowing
getting out of buffer and possibly corrupting data.
- pci_vtscsi_controlq_notify() written returned status at wrong offset.
- pci_vtscsi_controlq_notify() leaked one buffer per event.
Reported by: wg
Reviewed by: araujo
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D18465
mjg [Fri, 7 Dec 2018 12:32:25 +0000 (12:32 +0000)]
proc: when exiting move to zombproc before taking proctree
The kernel was already doing this prior to r329615. It was changed
to reduce contention on allproc. However, introduction of pidhash
locks and removal of proctree -> allproc ordering from fork thanks
to bitmaps fixed things enough to make this change pessimal.
waitpid takes proctree on each call and this change (now) causes
avoidable stalls if allproc is held.
mjg [Fri, 7 Dec 2018 12:22:32 +0000 (12:22 +0000)]
Manage process-related IDs with bitmaps
Currently unique pid allocation on fork often requires a full walk of
process, group, session lists to make sure it is not used by anything.
This has a side effect of requiring proctree to be held along with allproc,
which adds more contention in poudriere -j 128.
The patch below implements trivial bitmaps which gets rid of the problem.
Dedicated lock is introduced to manage IDs.
While here a bug was discovered: all processes would inherit reap id from
the first process spawned by init. This had a side effect of keeping the
ID used and when allocation rolls over to the beginning it keeps being
skipped.
The patch is loosely based on initial work by mjoras@.
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
vmaffione [Fri, 7 Dec 2018 11:47:42 +0000 (11:47 +0000)]
netmap: remove dead code obsoleted by iflib
The iflib subsystem implements netmap support in a driver-independent
way (sys/net/iflib.c). We can therefore remove the headers that
used to implement netmap support for all the drivers now supported
by iflib (em, igb, ixl, ixgbe, lem).
pjd [Fri, 7 Dec 2018 03:13:36 +0000 (03:13 +0000)]
Consider the following situation:
The sender has .not_terminated file. It gets disconnected. The last trail
file is then terminated without adding new data (this can happen for example
when auditd is being stopped on the sender). After reconnect the .not_terminated
was not renamed on the receiver as it should.
We were already handling similar situation where the sender crashed and the
.not_terminated trail file was renamed to .crash_recovery. Extend this case to
handle the situation above.
cem [Fri, 7 Dec 2018 02:44:04 +0000 (02:44 +0000)]
gmirror: Evaluate mirror components against newest metadata copy
Re-apply r341665 with format strings fixed.
If we happen to taste a stale mirror component first, don't reject valid,
newer components that have differing metadata from the stale component
(during STARTING). Instead, update our view of the most recent metadata as
we taste components.
Like mediasize beforehand, remove some checks from g_mirror_check_metadata
which would evict valid components due to metadata that can change over a
mirror's lifetime. g_mirror_check_metadata is invoked long before we check
genid/syncid and decide which component(s) are newest and whether or not we
have quorum.
Before checking if we can enter RUNNING (i.e., we have quorum) after a NEW
component is added, first remove any known stale or inconsistent disks from
the mirrorset, rather than removing them *after* deciding we have quorum.
Check if we have quorum after removing these components.
Additionally, add a knob, kern.geom.mirror.launch_mirror_before_timeout, to
force gmirrors to wait out the full timeout (kern.geom.mirror.timeout)
before transitioning from STARTING to RUNNING. This is a kludge to help
ensure all eligible, boot-time available mirror components are tasted before
RUNNING a gmirror.
Add a basic test case for STARTING -> RUNNING startup behavior around stale
genids.
asomers [Fri, 7 Dec 2018 00:27:38 +0000 (00:27 +0000)]
geom tests: Fix cleanup of ATF tests since r341392
r341392 changed common test cleanup routines in a way that allowed them to
be used by TAP tests as well as ATF tests. However, a late change made
during code review resulted in cleanup being broken for ATF tests, which
source geom_subr.sh separately during the body and cleanup phases of the
test. The result was that md(4) devices wouldn't get cleaned up.
cem [Thu, 6 Dec 2018 23:55:39 +0000 (23:55 +0000)]
gmirror: Evaluate mirror components against newest metadata copy
If we happen to taste a stale mirror component first, don't reject valid,
newer components that have differing metadata from the stale component
(during STARTING). Instead, update our view of the most recent metadata as
we taste components.
Like mediasize beforehand, remove some checks from g_mirror_check_metadata
which would evict valid components due to metadata that can change over a
mirror's lifetime. g_mirror_check_metadata is invoked long before we check
genid/syncid and decide which component(s) are newest and whether or not we
have quorum.
Before checking if we can enter RUNNING (i.e., we have quorum) after a NEW
component is added, first remove any known stale or inconsistent disks from
the mirrorset, rather than removing them *after* deciding we have quorum.
Check if we have quorum after removing these components.
Additionally, add a knob, kern.geom.mirror.launch_mirror_before_timeout, to
force gmirrors to wait out the full timeout (kern.geom.mirror.timeout)
before transitioning from STARTING to RUNNING. This is a kludge to help
ensure all eligible, boot-time available mirror components are tasted before
RUNNING a gmirror.
When we are instructed to forget mirror components, bump the generation id
to avoid confusion with such stale components later.
Add a basic test case for STARTING -> RUNNING startup behavior around stale
genids.
imp [Thu, 6 Dec 2018 22:58:26 +0000 (22:58 +0000)]
Dynamically load .so modules to expand functionality
o Dynamically load all the .so files found in /libexec/nvmecontrol and
/usr/local/libexec/nvmecontrol.
o Link nvmecontrol -rdynamic so that its symbols are visible to the
libraries we load.
o Create concatinated linker sets that we dynamically expand.
o Add the linked-in top and logpage linker sets to the mirrors for them
and add those sets to the mirrors when we load a new .so.
o Add some macros to help hide the names of the linker sets.
o Update the man page.
jhb [Thu, 6 Dec 2018 22:35:07 +0000 (22:35 +0000)]
Rename riscv64-freebsd.c to riscv-freebsd.c.
This fixes truss when built as part of a riscv64sf world. Additionally,
if FreeBSD ever supports RV32 RISC-V most of this file can be used as-is
just as a single file is used for all of the MIPS ABIs.
cxgbe(4): Get Linux cxgb4vf working in bhyve VMs with VFs passed
through.
cxgb4vf doesn't own the buffer size list but still expects the first two
entries to be 4K and some power of 2 respectively. The BSD cxgbe
doesn't care where its preferred buffer sizes are as long as they're in
the list somewhere, so just move its entries towards the end as a
workaround.
pfsync code is called for every new state, state update and state
deletion in pf. While pf itself can operate on multiple states at the
same time (on different cores, assuming the states hash to a different
hashrow), pfsync only had a single lock.
This greatly reduced throughput on multicore systems.
Address this by splitting the pfsync queues into buckets, based on the
state id. This ensures that updates for a given connection always end up
in the same bucket, which allows pfsync to still collapse multiple
updates into one, while allowing multiple cores to proceed at the same
time.
The number of buckets is tunable, but defaults to 2 x number of cpus.
Benchmarking has shown improvement, depending on hardware and setup, from ~30%
to ~100%.
sef [Thu, 6 Dec 2018 18:21:48 +0000 (18:21 +0000)]
Reduce number of DNS queries in mountd.
As reported by a FreeNAS user (see https://redmine.ixsystems.com/issues/55728),
mountd does more calls to getnameinfo() than it needs to; this changes it to
only call it for the RPC calls it needs the name information for.
sobomax [Thu, 6 Dec 2018 18:12:50 +0000 (18:12 +0000)]
Apparently @portmgr does not feel like my current level of contribution
warrants me a repo access to the ports tree anymore. Move myself into alumni
at least for a time being.
yuripv [Thu, 6 Dec 2018 13:46:18 +0000 (13:46 +0000)]
termcap: add italics and tmux entries
- add "ecma+italics" cap
- use it in "xterm-basic"
- add "tmux"/"tmux-256color" caps, fixing "so" and "se" caps,
and sourcing "ecma+italics" and "screen"/"screen-256color"
Specifically, assume that the device is present if evaluation of _STA
method fails.
Before r330957 we ignored any _STA evaluation failure (which was
performed by AcpiGetObjectInfo in ACPICA contrib code) for the purpose
of acpi_DeviceIsPresent and acpi_BatteryIsPresent. ACPICA 20180313
removed evaluation of _STA from AcpiGetObjectInfo. So, we added
evaluation of _STA to acpi_DeviceIsPresent and acpi_BatteryIsPresent.
One important difference is that the new code ignored a failure only if
_STA did not exist (AE_NOT_FOUND). Any other kind of failure was
treated as a fatal failure. Apparently, on some systems we can get
AE_NOT_EXIST when evaluating _STA. And that error is not an evil twin
of AE_NOT_FOUND, despite a very similar name, but a distinct error
related to a missing handler for an ACPI operation region.
It's possible that for some people the problem was already fixed by
changes in ACPICA and/or in acpi_ec driver (or even in BIOS) that fixed
the AE_NOT_EXIST failure related to EC operation region.
This work is based on a great analysis by cem and an earlier patch by
Ali Abdallah <aliovx@gmail.com>.
jhibbits [Thu, 6 Dec 2018 04:36:02 +0000 (04:36 +0000)]
powerpc: Set very low priority mode while waiting for AP unleash event
The POWER9 does not recognize 'or 27,27,27' as a thread priority NOP. On
earlier POWER architectures, this NOP would note to the processor to give up
resources if able, to improve performance of other threads.
All processors that support the thread priority NOPs recognize the
'or 31,31,31' NOP as very low priority, so use this to perform a similar
function, and not burn cycles on POWER9.
jhibbits [Thu, 6 Dec 2018 04:30:24 +0000 (04:30 +0000)]
powerpc: Fix ELFv2 JMP_SLOT relocation fixup
The jump slot is a function pointer, not a descriptor pointer, in ELFv2. Just
write the pointer itself over, not the contents of the pointer, which would be
the first instruction of the function.
The 'interrupts' property is actually 2 words, not one, on macgpio child
nodes. Open Firmware's getprop function might be returning the value
copied, not the total size of the property, but FDT's returns the total
size. Prior to this patch, this would cause the SYS_RES_IRQ resource list
to not be populated when running with the 'usefdt' loader variable set, to
convert the OFW device tree to a FDT. Since the property is always 2 words,
read both words, and ignore the second.
Tested by: Dennis Clarke (previous attempt)
MFC after: 2 weeks
mckusick [Thu, 6 Dec 2018 01:04:56 +0000 (01:04 +0000)]
If the vfs.ffs.dotrimcons sysctl option is enabled while a file
deletion is active, specifically after a call to ffs_blkrelease_start()
but before the call to ffs_blkrelease_finish(), ffs_blkrelease_start()
will have handed out SINGLETON_KEY rather than starting a collection
sequence. Thus if we get a SINGLETON_KEY passed to ffs_blkrelease_finish(),
we just return rather than trying to finish the nonexistent sequence.
Reported by: Warner Losh (imp@)
Sponsored by: Netflix
mckusick [Thu, 6 Dec 2018 00:09:39 +0000 (00:09 +0000)]
Normally when an attempt is made to mount a UFS/FFS filesystem whose
superblock has a check-hash error, an error message noting the
superblock check-hash failure is printed and the mount fails. The
administrator then runs fsck to repair the filesystem and when
successful, the filesystem can once again be mounted.
This approach fails if the filesystem in question is a root filesystem
from which you are trying to boot. Here, the loader fails when trying
to access the filesystem to get the kernel to boot. So it is necessary
to allow the loader to ignore the superblock check-hash error and make
a best effort to read the kernel. The filesystem may be suffiently
corrupted that the read attempt fails, but there is no harm in trying
since the loader makes no attempt to write to the filesystem.
Once the kernel is loaded and starts to run, it attempts to mount its
root filesystem. Once again, failure means that it breaks to its prompt
to ask where to get its root filesystem. Unless you have an alternate
root filesystem, you are stuck.
Since the root filesystem is initially mounted read-only, it is
safe to make an attempt to mount the root filesystem with the failed
superblock check-hash. Thus, when asked to mount a root filesystem
with a failed superblock check-hash, the kernel prints a warning
message that the root filesystem superblock check-hash needs repair,
but notes that it is ignoring the error and proceeding. It does
mark the filesystem as needing an fsck which prevents it from being
enabled for writing until fsck has been run on it. The net effect
is that the reboot fails to single user, but at least at that point
the administrator has the tools at hand to fix the problem.
Reported by: Rick Macklem (rmacklem@)
Discussed with: Warner Losh (imp@)
Sponsored by: Netflix
brooks [Wed, 5 Dec 2018 19:18:16 +0000 (19:18 +0000)]
Further simplify arguments to init.
With the removal of BOOTCDROM and fastboot support, this code always
passed "-s" or "--". The latter simply terminates getopt(3) processing
in init so we only need to pass "-s" in the single user case, or nothing
in other cases.
The passing of "--" seems to have been done to ensure that the number of
arguments passed to init was always the same and thus that argc was the
same.
Also GC the write-only variable pathlen (not in reviewed version).
alc [Wed, 5 Dec 2018 18:26:40 +0000 (18:26 +0000)]
Terminate a blist_alloc search when a blst_meta_alloc call fails with
cursor == 0.
Every call to blst_meta_alloc but the one at the root is made only when the
meta-node is known to include a free block, so that either the allocation
will succeed, the node hint will be updated, or the last block of the meta-
node range is, and remains, free. But the call at the root is made without
checking that there is a free block, so in the case that every block is
allocated, there is no hint update to prevent the current code from looping
forever.
brooks [Wed, 5 Dec 2018 17:29:14 +0000 (17:29 +0000)]
Remove ifdef BOOTCDROM option to start init.
When BOOTCDROM is defined (via CFLAGS as there is no config option)
it causes -C to be passed to init, but our init and the version of
sysinstall I glanced at in 6.x don't support -C. The last plausibly
related support was removed from the tree in 1995.
asomers [Wed, 5 Dec 2018 17:28:40 +0000 (17:28 +0000)]
stat(2): clarify which syscalls modify file timestamps
The list of syscalls that modify st_atim, st_mtim, and st_ctim was quite out
of date and probably not accurate to begin with. Update it, and make it
clear that the list is open-ended.
markj [Wed, 5 Dec 2018 17:06:00 +0000 (17:06 +0000)]
Clamp the INPCB port hash tables to IPPORT_MAX + 1 chains.
Memory beyond that limit was previously unused, wasting roughly 1MB per
8GB of RAM. Also retire INP_PCBLBGROUP_PORTHASH, which was identical to
INP_PCBPORTHASH.
slavash [Wed, 5 Dec 2018 14:25:03 +0000 (14:25 +0000)]
mlx5en: Implement backpressure indication.
The backpressure indication is implemented using an unlimited rate type of
mbuf send tag. When the upper layers typically the socket layer has obtained such
a tag, it can then query the destination driver queue for the current
amount of space available in the send queue.
A single mbuf send tag may be referenced multiple times and a refcount has been added
to the mlx5e_priv structure to track its usage. Because the send tag resides
in the mlx5e_channel structure, there is no need to wait for refcounts to reach
zero until the mlx4en(4) driver is detached. The channels structure is persistant
during the lifetime of the mlx5en(4) driver it belongs to and can so be accessed
without any need of synchronization.
The mlx5e_snd_tag structure was extended to contain a type field, because there are now
two different tag types which end up in the driver which need to be distinguished.
slavash [Wed, 5 Dec 2018 14:24:33 +0000 (14:24 +0000)]
mlx5en: Improve configuration of HW LRO.
In order to enable HW LRO, both the "hw_lro" sysctl in the mlx5en(4) config
space must be set, and the ifconfig(8) LRO capability must be set. Any other
settings will disable HW LRO.
slavash [Wed, 5 Dec 2018 14:24:02 +0000 (14:24 +0000)]
mlx5en: Count all transmitted and received bytes.
Add counter for all transmitted and received bytes. Currently only all
transmitted and received packets were counted. Fix description of RX LRO
counters while at it.
slavash [Wed, 5 Dec 2018 14:23:31 +0000 (14:23 +0000)]
mlx5en: Statically allocate and free the channel structure(s).
By allocating the worst case size channel structure array
at attach time we can eliminate various NULL checks in the
fast path. And also reduce the chance for use-after-free
issues in the transmit fast path.
This change is also a requirement for implementing
backpressure support.
slavash [Wed, 5 Dec 2018 14:23:01 +0000 (14:23 +0000)]
mlx5en: Fix race in mlx5e_ethtool_debug_stats().
Writing to the debug stats variable must be locked,
else serialization will be lost which might cause
various kernel panics due to creating and destroying
sysctls out of order.
Make sure the sysctl context is initialized after freeing
the sysctl nodes, else they can be freed twice.
projects / FreeBSD / FreeBSD.git / log