Alan Cox [Sun, 10 Jan 2021 08:51:33 +0000 (02:51 -0600)]
Prefer the use of vm_page_domain() to vm_phys_domain().
When we already have the vm page in hand, use vm_page_domain() instead
of vm_phys_domain(). The former has a trivial constant-time
implementation whereas the latter iterates over the mem_affinity array.
iflib: add assert to prevent out-of-bounds array access
The iflib_queues_alloc() allocates isc_nrxqs iflib_dma_info structs
for each rxqset, and links each struct to a different free list.
As a result, it must be isc_nrxqs >= isc_nfl (plus the completion
queue, if present).
Add an assertion to make this constraint explicit.
Robert Watson [Fri, 1 Jan 2021 13:04:46 +0000 (13:04 +0000)]
Track pipe(2) reads and writes as rusage message receives and sends, a
feature misplaced during the transition from BSD 4.4's socket implementation
to the optimised FreeBSD pipe implementation.
netmap: iflib: enable/disable krings on any interface reinit
Since 1d238b07d5d4d9660ae0e0, krings are disabled before
a reinit cycle triggered by iflib_netmap_register.
However, this operation is actually necessary also for
any interface reinit triggered by other causes (i.e.,
ifconfig commands).
We achieve this goal by moving the krings enable/disable
operation inside iflib_stop() and iflib_init_locked().
Once here, this change also removes some redundant operations
from iflib_netmap_register(), that are already performed by
iflib_stop().
Jamie Gritton [Sun, 10 Jan 2021 05:05:06 +0000 (21:05 -0800)]
jail: Simplify handling of prison_deref()
Track the the current lock/reference state in a single variable,
rather than deducing the proper prison_deref() flags from a
combination of equations and hard-coded values.
Instead of providing ifuncs for each kind of fence, define ifuncs
that combine fence and invocation of RDTSC. This refactoring makes
introduction of RDTSCP use possible.
Reviewed by: gallatin, markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27986
tsc: add RDTSCP or faster variants of get_timecount()
Use it in preference of Xfenced RDTSC if RDTSCP is supported. It is
recommended by both Intel and AMD. But, on AMD Zens and newer use
LFENCE, as recommended by AMD [*]. In particular, this means that now
AMD CPUs use more appropriate fence instead of too harsh MFENCe.
Add comment explaining the intent of the selection logic.
Instead of trying to maintain pg_jobc counter on each process group
update (and sometimes before), just calculate the counter when needed.
Still, for the benefit of the signal delivery code, explicitly mark
orphaned groups as such with the new process group flag.
This way we prevent bugs in the corner cases where updates to the counter
were missed due to complicated configuration of p_pptr/p_opptr/real_parent
(debugger).
Since we need to iterate over all children of the process on exit, this
change mostly affects the process group entry and leave, where we need
to iterate all process group members to detect orpaned status.
Proctree lock is needed for correct calculation and collection of the
job-control related data in kinfo_proc. There was even an XXX comment
about it.
Satisfy locking and lock ordering requirements by taking proctree lock
around pass over each bucket in proc_iterate(), and in sysctl_kern_proc()
and note_procstat_proc() for individual process reporting.
Reviewed by: jilles
Tested by: pho
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27871
Increase the scope of the process group lock ownership. This ensures that
we are consistent in returning EIO for tty write from an orphan and delivery
of TTYOUT signals.
Reviewed by: jilles
Tested by: pho
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27871
Often, we have a process locked and need to get locked process group.
In this case, because progress group lock is before process lock,
unlocking process allows the group to be freed. See for instance
tty_wait_background().
Make pgrp structures allocated from nofree zone, and ensure type stability
of the pgrp mutex.
Reviewed by: jilles
Tested by: pho
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27871
Similarly to what done for iflib in 1d238b07d5d4d9660ae0e,
this patch prevents access to the krings during the interface
reset triggered by netmap_register().
The netmap_reset() function is meant to be called by the driver
when they initialize (or re-initialize) a hardware ring.
However, since the introduction of support for opening (in
netmap mode) a subset of the available rings, netmap_reset()
may be called multiple times on actively used rings, causing
both kring and netmap ring to transition to an inconsistent
state.
This changes improves the situation by resetting all the
indices fields of the kring to 0, as expected after the
reinitialization of a hardware ring.
When netmap_fl_refill() is called at initialization time (e.g.,
during netmap_iflib_register()), nic_i must be 0, since the
free list is reinitialized. At the end of the refill cycle, nic_i
must still be zero, because exactly N descriptors (N is the ring size)
are refilled.
This patch therefore fixes the assertions to check on nic_i rather
than on nm_i. The current netmap_reset() may in fact cause nm_i
to be != 0 while the device is resetting: this may happen when
multiple non-cooperating processes open different subsets of the
available netmap rings.
When different processes open separate subsets of the
available rings of a same netmap interface, a device
reset may be performed while one of the processes
is actively using some rings (e.g., caused by another
process executing a nmport_open()).
With this patch, such situation will cause the
active process to get a POLLERR, so that it can
have a chance to detect the situation.
We also guarantee that no process is running a txsync
or rxsync (ioctl or poll) while an iflib device reset
is in progress.
This imposes a fairly severe limitation on space available for mmap that
was not noticed prior to commit. Unfixed mmap will only map from
[data + MAXSIZE, end of user VA space], bringing the amount of usable space
down way too low for non-trivial link jobs (for instance).
Ed Maste [Sat, 9 Jan 2021 18:34:58 +0000 (13:34 -0500)]
ldd: renumber executable type constants
ldd had #defines for AOUT, ELF, and ELF32. The removal of AOUT left a
possibly confusing gap. These are not used anywhere but this file so
renumber to avoid the gap.
Ed Maste [Sat, 9 Jan 2021 16:22:28 +0000 (11:22 -0500)]
diff: honour flags with -q
Previously -q (just print a line when files differ) ignored flags like
-w (ignore whitespace). Avoid the D_BRIEF short-circuit when flags are
in effect.
PR: 252515
Reported by: Scott Aitken
Reviewed by: kevans
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D28064
This module intended to measure performance of routing lookups.
Uses a list of IP addresses specified by sysctl one-by-one.
Performance testing is triggered by changing sysctl OID with a number of lookups to execute.
Lookups are done by the chunks of 10K routes, entering/exiting epoch on
chunk granularity to amortise cost.
Example:
make -C sys/modules/test/fib_lookup unload load
for i in `cat ~/ip4.txt`; do sysctl net.route.test.add_inet_addr=$i; done
for i in `cat ~/ip6.txt`; do sysctl net.route.test.add_inet6_addr=$i; done
Dec 13 23:24:05 current kernel: 10000000 packets in 417240173 nanoseconds, 23967011 pps
Dec 13 23:24:06 current kernel: run: 10000000 packets vnet 0xfffff80003073f00
Dec 13 23:24:07 current kernel: 10000000 packets in 423086254 nanoseconds, 23635842 pps
This change introduces loadable fib lookup modules based on
DPDK rte_lpm lib targeted for high-speed lookups in large-scale tables.
It is based on the lookup framework described in D27401.
IPv4 module is called dpdk_lpm4. It wraps around rte_lpm [1] library.
This library implements variation of DIR24-8 [2] lookup algorithm.
Module provide lockless route lookups and in-place incremental updates,
allowing for good RIB performance.
IPv6 module is called dpdk_lpm6. It wraps around rte_lpm6 [3] library.
Implementation can be seen as multi-bit trie where the stride or number of bits
inspected on each level varies from level to level.
It can vary from 1 to 14 memory accesses, with 5 being the average value
for the lengths that are most commonly used in IPv6.
Module provide lockless route lookups for global unicast addresses
and in-place incremental updates, allowing for good RIB performance.
Implementation details:
* wrapper code lives in `sys/contrib/dpdk_rte_lpm/dpdk_lpm[6].c`.
* rte_lpm[6] implementation contains both RIB and FIB code.
. RIB ("rule_") code, backed by array of hash tables part has been commented out,
as base radix already provides all the necessary primitives.
* link-local lookups are currently implemented as base radix lookup.
This part should be converted to something like read-only radix trie.
Usage detail:
Compile kernel with option FIB_ALGO and load dpdk_lpm4/dpdk_lpm6
module at any time. They will be picked up automatically when
amount of routes raises to several thousand.
The NVMe byte-swap routines for big-endian platforms used memcpy() to
move the unaligned 64-bit value into a temp register to byte swap it.
Instead of introducing a dependency, manually byte-swap the values in
place.
The general style in sbin/nvmecontrol apppears to print uint64_t types
using %j, so I'm using that instead of the more general (but admittedly
ugly) PRIu64.
Bryan Drewery [Fri, 18 Dec 2020 17:58:03 +0000 (09:58 -0800)]
dtrace: Fix /"string" == NULL/ comparisons using an uninitialized value.
A test of this is funcs/tst.strtok.d which has this filter:
BEGIN
/(this->field = strtok(this->str, ",")) == NULL/
{
exit(1);
}
The test will randomly fail with exit status of 1 indicating that this->field
was NULL even though printing it out shows it is not.
This is compiled to the DTrace instruction set:
// Pushed arguments not shown here
// call strtok() and set result into %r1
07: 2f001f01 call DIF_SUBR(31), %r1 ! strtok
// set thread local scalar this->field from %r1
08: 39050101 stls %r1, DT_VAR(1281) ! DT_VAR(1281) = "field"
// Prepare for the == comparison
// Set right side of %r2 to NULL
09: 25000102 setx DT_INTEGER[1], %r2 ! 0x0
// string compare %r1 (strtok result) to %r2
10: 27010200 scmp %r1, %r2
In this case only %r1 is loaded with a string limit set to lim1. %r2 being
NULL does not get loaded and does not set lim2. Then we call dtrace_strncmp()
with MIN(lim1, lim2) resulting in passing 0 and comparing neither side.
dtrace_strncmp() handles this case fine and it already has been while
being lucky with what lim2 was [un]initialized as.
Reviewed by: markj, Don Morris <dgmorris AT earthlink.net>
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D27671
Currently only libexec/rtld-elf32 uses internal LIBC_NOSSP_PIC during
the build but it gets it directly from the objdir rather than a sysroot.
For example, /usr/obj/usr/src/amd64.amd64/obj-lib32/lib/libc/libc_nossp_pic.a.
We don't stage lib32 libraries in WORLDTMP/usr/lib32 and doing so doesn't
buy much. If we want to use a staged lib32 library then we need to look in
LIBCOMPATTMP where they were staged. For example if LIBC_PIC were wanted then
look for /usr/obj/usr/src/amd64.amd64/obj-lib32/tmp/usr/lib32/libc_pic.a.
Bryan Drewery [Wed, 16 Dec 2020 20:34:38 +0000 (12:34 -0800)]
Makefile.inc1: Avoid using release/Makefile for VERSION.
release/Makefile.inc1 has git executions that were being ran for each of
these lookups. The results were not needed so just lookup what we want
directly instead.
Mitchell Horne [Fri, 8 Jan 2021 21:06:11 +0000 (17:06 -0400)]
sifive_uart: quiet GCC -Werror=parentheses
Add an additional set of braces to clarify intention. The '&' operator
has a higher precedence than '|', but the reader may not always remember
this. No functional change.
Warner Losh [Fri, 8 Jan 2021 16:34:31 +0000 (09:34 -0700)]
sysctl: implement debug.kdb.panic_str
This is just like debug.kdb.panic, except the string that's passed in
is reported in the panic message. This allows people with automated
systems to collect kernel panics over a large fleet of machines to
flag panics better. Strings like "Warner look at this hang" or "see
JIRA ABC-1234 for details" allow these automated systems to route the
forced panic to the appropriate engineers like you can with other
types of panics. Other users are likely possible.
Stop trying to manually calculate RID, which cannot be done correctly
by PCI_DEVFN(). Use PCI_GET_RID() method instead.
Do not use pci_find_dbsf() to go from the linux pci_dev to freebsd
device_t. First, device is readily available as dev.bsddev. Second,
using pci_find_dbsf() fails for ARI-enabled functions with large
function numbers, because PCI_SLOT()/PCI_FUNC() are for non-ARI.
Kyle Evans [Fri, 8 Jan 2021 19:57:32 +0000 (13:57 -0600)]
libregex: re-enable `make check`
The tests are generally expected to pass, uncomment the annotation that
lets `make check` work. Note that `make check` currently requires kyua
from ports or an appropriate symlink into /usr/local/bin.
Miod Vallat [Fri, 8 Jan 2021 18:59:00 +0000 (12:59 -0600)]
libc: regex: rework unsafe pointer arithmetic
regcomp.c uses the "start + count < end" idiom to check that there are
"count" bytes available in an array of char "start" and "end" both point to.
This is fine, unless "start + count" goes beyond the last element of the
array. In this case, pedantic interpretation of the C standard makes the
comparison of such a pointer against "end" undefined, and optimizers from
hell will happily remove as much code as possible because of this.
An example of this occurs in regcomp.c's bothcases(), which defines
bracket[3], sets "next" to "bracket" and "end" to "bracket + 2". Then it
invokes p_bracket(), which starts with "if (p->next + 5 < p->end)"...
Because bothcases() and p_bracket() are static functions in regcomp.c, there
is a real risk of miscompilation if aggressive inlining happens.
The following diff rewrites the "start + count < end" constructs into "end -
start > count". Assuming "end" and "start" are always pointing in the array
(such as "bracket[3]" above), "end - start" is well-defined and can be
compared without trouble.
As a bonus, MORE2() implies MORE() therefore SEETWO() can be simplified a
bit.
mhorne [Wed, 9 Dec 2020 20:38:42 +0000 (16:38 -0400)]
arm64: gdb(4) machine-dependent bits
Everything required for remote kernel debugging over a serial
connection. For FDT-based systems, a debug port can be specified by
setting hw.fdt.dbgport to the desired device tree node in loader.conf.
For example, hw.fdt.dbgport="uart1", or
hw.fdt.dbgport="serial@ff1a0000".
Looks good: emaste
Tested by: rwatson
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27727
mhorne [Mon, 21 Dec 2020 16:16:09 +0000 (12:16 -0400)]
arm64: remove pcb_pc
The program counter field in the PCB is written in exactly one place,
makectx(), upon entry to the debugger. For threads other than curthread,
its value will be empty, or bogus. Rather than writing to this field in
more places, it can be removed in favor of using the value in the link
register.
To make this clearer, pcb->pcb_x[30] is renamed to pcb->pcb_lr, similar
to what already exists in struct trapframe. Also, prefer lr to x30 in
assembly, as it better conveys intention.
This improves PC_REGS() for kdb_thread != curthread. It is required for
a functional gdb(4) stub, fixing the output of `info threads`, in
particular.
The space occupied by pcb_pc is retained, for compatibility with kgdb.
Reviewed by: markj, jhb
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27720
mhorne [Fri, 18 Dec 2020 18:43:52 +0000 (14:43 -0400)]
arm64: don't pass user trapframe to kdb_trap()
This effectively undoes the changes made in r321571. While useful, it is
inconsistent with how other architectures pass trapframes to kdb. This
change is also required to get a working gdb(4) stub on arm64, as
otherwise the backtrace will begin too early.
As of 088a7eef95b1, this information can still be obtained via
"show registers/u".
Reviewed by: jhb (slightly earlier version)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Pull Request: https://reviews.freebsd.org/D27719
mhorne [Fri, 18 Dec 2020 20:06:46 +0000 (16:06 -0400)]
ddb: add ability to print user registers
The debugger is always entered after some kind of kernel trap, often a
breakpoint in kdb_enter(). This means that the most recent trapframe
will include kernel state at the time of the trap, when often it is
desirable to the developer to view the contents of the previous
trapframe. This trapframe often corresponds to the entry from userspace.
The ddb(4) man page claims the ability to display user register state
via the 'u' modifier to `show registers`, but this appears untrue. It is
not obvious from a quick search of the history when this feature was
added, or when it was removed. (Re)implement this feature in
db_show_regs, noting that it is not necessarily populated with userspace
state.
Reviewed by: jhb (earlier version), markj, bcr (manpages)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27705
Andrew Gallatin [Fri, 8 Jan 2021 18:18:42 +0000 (13:18 -0500)]
mbuf: enable ext_pgs ("unmapped") mbufs by default
Ext_pg mbufs allow carrying multiple pages per mbuf. This
reduces mbuf linked list traversals, especially in socket
buffers, thereby reducing cache misses and CPU use for
applications using sendfile. Note that ext_pages use
unmapped pages, eliminating KVA mapping costs on 32-bit
platforms.
Ext_pg mbufs are also required for ktls (KERN_TLS), and having
them disabled by default is a stumbling block for those
wishing to enable ktls.
Mark Johnston [Fri, 8 Jan 2021 18:32:04 +0000 (13:32 -0500)]
mpr, mps: Fix a stack buffer overflow in the user passthru ioctl
Previously we copied in the request into a stack-allocated structure
that could be smaller than the request size. Furthermore, we checked
the request size only after doing the copyin.
Fix this by allocating a buffer to hold the request, then copying the
buffer's contents into a command descriptor. This is a bit heavy-handed
but I expect the overhead will not be noticeable. The approach of
coping the header in first is susceptible to TOCTOU problems.
Reviewed by: imp
Reported by: maxpl0it@protonmail.com
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27963
Mark Johnston [Fri, 8 Jan 2021 18:32:04 +0000 (13:32 -0500)]
safexcel: Fix a race around unblocking of crypto ops
safexcel_ring_intr() could fail to observed that sc_blocked is set after
completing all outstanding ops for a ring, in which case blocked ops
would be deferred forever.
Request structures are managed by individual rings, so move the
"blocked" flag into the per-ring state block and use the ring lock to
synchronize with safexcel_process(). Remove sc_mtx since it is now
unused.
MFC after: 3 days
Sponsored by: Rubicon Communications, LLC (Netgate)
Mark Johnston [Fri, 8 Jan 2021 18:32:04 +0000 (13:32 -0500)]
safexcel: Stop using a stack buffer for the ring lock name
mtx_init() does not make a copy of the name so the buffer must be valid
for the lifetime of the driver instance. Store each ring's lock's name
in the ring structure.
MFC after: 3 days
Sponsored by: Rubicon Communications, LLC (Netgate)
The new UAR API already offsets the UAR map pointer the mlx5en(4) is using.
While at it remove some no longer needed variables for keeping track
of the current BF offset.
This fixes a regression issue after the new UAR allocation APIs
were introduced.
Gleb Smirnoff [Fri, 8 Jan 2021 17:23:16 +0000 (09:23 -0800)]
When be_activate() turns on a new boot environment, it always deactivates
the current one first. And if it fails to do so, it abandons activation.
However, with the new bootonce feature, there is a legitimate case when
a pool doesn't have "bootfs" property set. Check for this case before
calling be_deactivate().
Kyle Evans [Fri, 8 Jan 2021 15:18:46 +0000 (09:18 -0600)]
build: add WITHOUT_CLEAN workaround for 821aa63a0940
The *w variants of ncurses directories went away, and the remaining names
build the widechar variants instead of non-widechar variants. As such, the
entire ncurses tree should be regenerated.
Key off of lib/ncurses/ncursesw being present and remove the whole ncurses
hierarchy if it is.
Kyle Evans [Thu, 7 Jan 2021 21:51:18 +0000 (15:51 -0600)]
efidev: remove EFIIOC_GET_TABLE ioctl
This ioctl would instantly induce a panic, likely since near inception, up
until 0861c7d3e048. Lack of previous interest in fixing it combined with
the problematic interface (exports a pointer, really a physical address)
brings us to the natural conclusion: remove it until a useful consumer
forward.
If it eventually gets resurrected, the interface should definitely not
return in this exact form and likely needs to be reimagined.
The associated KPI, efi_get_table, is left intact for the time being.
Andrew Turner [Mon, 4 Jan 2021 13:50:17 +0000 (13:50 +0000)]
Update hwpmc on armv7 to handle overflow better
When testing hwpmc on arm64 we found the counter could overflow while
reading the event count. Handle this case in the armv7 code by also
checking if the overflow bit is set and incrementing the overflow
cound as needed.
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D27969
Update user access region, UAR, APIs in the core in mlx5core.
This change include several changes as listed below all related to UAR.
UAR is a special PCI memory area where the so-called doorbell register and
blue flame register live. Blue flame is a feature for sending small packets
more efficiently via a PCI memory page, instead of using PCI DMA.
- All structures and functions named xxx_uuars were renamed into xxx_bfreg.
- Remove partially implemented Blueflame support from mlx5en(4) and mlx5ib.
- Implement blue flame register allocator.
- Use blue flame register allocator in mlx5ib.
- A common UAR page is now allocated by the core to support doorbell register
writes for all of mlx5en and mlx5ib, instead of allocating one UAR per
sendqueue.
- Add support for DEVX query UAR.
- Add support for 4K UAR for libmlx5.
projects / FreeBSD / FreeBSD.git / log