Michal Meloun [Sat, 25 Jul 2020 06:32:23 +0000 (06:32 +0000)]
Revert r363123.
As Emanuel poited me the Linux processes these clock assignments in forward
order, not in reversed. I misread the original code.
Tha problem with wrong order for assigned clocks found in tegra (and some imx)
DT should be reanalyzed and solved by different way.
Rick Macklem [Fri, 24 Jul 2020 23:17:09 +0000 (23:17 +0000)]
Add support for ext_pgs mbufs to nfsm_uiombuflist() and nfsm_split().
This patch uses a slightly different algorithm for nfsm_uiombuflist() for
the non-ext_pgs case, where a variable called "mcp" is maintained, pointing to
the current location that mbuf data can be filled into. This avoids use of
mtod(mp, char *) + mp->m_len to calculate the location, since this does
not work for ext_pgs mbufs and I think it makes the algorithm more readable.
This change should not result in semantic changes for the non-ext_pgs case.
The patch also deletes come unneeded code.
It also adds support for anonymous page ext_pgs mbufs to nfsm_split().
This is another in the series of commits that add support to the NFS client
and server for building RPC messages in ext_pgs mbufs with anonymous pages.
This is useful so that the entire mbuf list does not need to be
copied before calling sosend() when NFS over TLS is enabled.
At this time for this case, use of ext_pgs mbufs cannot be enabled, since
ktls_encrypt() replaces the unencrypted data with encrypted data in place.
Until such time as this can be enabled, there should be no semantic change.
Also, note that this code is only used by the NFS client for a mirrored pNFS
server.
Make it possible to get/set MMC frequency from camcontrol
Enhance camcontrol(8) so that it's possible to manually set frequency for SD/MMC cards.
While here, display more information about the current controller, such as
supported operating modes and VCCQ voltages, as well as current VCCQ voltage.
Reviewed by: manu
Approved by: imp (mentor)
Differential Revision: https://reviews.freebsd.org/D25795
Alexander Motin [Fri, 24 Jul 2020 19:54:15 +0000 (19:54 +0000)]
Make lapic_ipi_vectored(APIC_IPI_DEST_SELF) NMI safe.
Sending IPI to self or all CPUs does not require write into upper part of
the ICR, prone to races. Previously the code disabled interrupts, but it
was not enough for NMIs. Instead of that when possible write only lower
part of the register, or use special SELF IPI register in x2APIC mode.
This also removes ICR reads used to preserve reserved bits on write.
It was there from the beginning, but I failed to find explanation why,
neither I see Linux doing it. Specification even tells that ICR content
may be lost in deep C-states, so if hardware does not bother to preserve
it, why should we?
Convert the bufobj tries to an SMR zone/PCTRIE and add a gbincore_unlocked()
API wrapping this functionality. Use it for a fast path in getblkx(),
falling back to locked lookup if we raced a thread changing the buf's
identity.
Use SMR to provide safe unlocked lookup for pctries from SMR zones
Adapt r358130, for the almost identical vm_radix, to the pctrie subsystem.
Like that change, the tree is kept correct for readers with store barriers
and careful ordering. Existing locks serialize writers.
Add a PCTRIE_DEFINE_SMR() wrapper that takes an additional smr_t parameter
and instantiates a FOO_PCTRIE_LOOKUP_UNLOCKED() function, in addition to the
usual definitions created by PCTRIE_DEFINE().
Interface consumers will be introduced in later commits.
As future work, it might be nice to add vm_radix algorithms missing from
generic pctrie to the pctrie interface, and then adapt vm_radix to use
pctrie.
Juli Mallett [Fri, 24 Jul 2020 16:58:13 +0000 (16:58 +0000)]
Remove reference to nlist(3) missed in SCCS revision 5.26 by mckusick
when converting rwhod(8) to using kern.boottime ather than extracting
the boot time from kernel memory directly.
Document that force_depend() supports only /etc/rc.d scripts
Currently, force_depend() from rc.subr(8) does not support depending on
scripts outside of /etc/rc.d (like /usr/local/etc/rc.d). The /etc/rc.d path
is hard-coded into force_depend().
vm: fix swap reservation leak and clean up surrounding code
The code did not subtract from the global counter if per-uid reservation
failed.
Cleanup highlights:
- load overcommit once
- move per-uid manipulation to dedicated routines
- don't fetch wire count if requested size is below the limit
- convert return type from int to bool
- ifdef the routines with _KERNEL to keep vm.h compilable by userspace
Alex Richardson [Fri, 24 Jul 2020 08:40:04 +0000 (08:40 +0000)]
Include TMPFS in all the GENERIC kernel configs
Being able to use tmpfs without kernel modules is very useful when building
small MFS_ROOT kernels without a real file system.
Including TMPFS also matches arm/GENERIC and the MIPS std.MALTA configs.
Compiling TMPFS only adds 4 .c files so this should not make much of a
difference to NO_MODULES build times (as we do for our minimal RISC-V
images).
Reviewed By: br (earlier version for riscv), brooks, emaste
Differential Revision: https://reviews.freebsd.org/D25317
John Baldwin [Thu, 23 Jul 2020 23:48:18 +0000 (23:48 +0000)]
Add support for KTLS RX via software decryption.
Allow TLS records to be decrypted in the kernel after being received
by a NIC. At a high level this is somewhat similar to software KTLS
for the transmit path except in reverse. Protocols enqueue mbufs
containing encrypted TLS records (or portions of records) into the
tail of a socket buffer and the KTLS layer decrypts those records
before returning them to userland applications. However, there is an
important difference:
- In the transmit case, the socket buffer is always a single "record"
holding a chain of mbufs. Not-yet-encrypted mbufs are marked not
ready (M_NOTREADY) and released to protocols for transmit by marking
mbufs ready once their data is encrypted.
- In the receive case, incoming (encrypted) data appended to the
socket buffer is still a single stream of data from the protocol,
but decrypted TLS records are stored as separate records in the
socket buffer and read individually via recvmsg().
Initially I tried to make this work by marking incoming mbufs as
M_NOTREADY, but there didn't seemed to be a non-gross way to deal with
picking a portion of the mbuf chain and turning it into a new record
in the socket buffer after decrypting the TLS record it contained
(along with prepending a control message). Also, such mbufs would
also need to be "pinned" in some way while they are being decrypted
such that a concurrent sbcut() wouldn't free them out from under the
thread performing decryption.
As such, I settled on the following solution:
- Socket buffers now contain an additional chain of mbufs (sb_mtls,
sb_mtlstail, and sb_tlscc) containing encrypted mbufs appended by
the protocol layer. These mbufs are still marked M_NOTREADY, but
soreceive*() generally don't know about them (except that they will
block waiting for data to be decrypted for a blocking read).
- Each time a new mbuf is appended to this TLS mbuf chain, the socket
buffer peeks at the TLS record header at the head of the chain to
determine the encrypted record's length. If enough data is queued
for the TLS record, the socket is placed on a per-CPU TLS workqueue
(reusing the existing KTLS workqueues and worker threads).
- The worker thread loops over the TLS mbuf chain decrypting records
until it runs out of data. Each record is detached from the TLS
mbuf chain while it is being decrypted to keep the mbufs "pinned".
However, a new sb_dtlscc field tracks the character count of the
detached record and sbcut()/sbdrop() is updated to account for the
detached record. After the record is decrypted, the worker thread
first checks to see if sbcut() dropped the record. If so, it is
freed (can happen when a socket is closed with pending data).
Otherwise, the header and trailer are stripped from the original
mbufs, a control message is created holding the decrypted TLS
header, and the decrypted TLS record is appended to the "normal"
socket buffer chain.
(Side note: the SBCHECK() infrastucture was very useful as I was
able to add assertions there about the TLS chain that caught several
bugs during development.)
This avoids injecting errors into the test system's mirrors.
gnop seems like a good solution here but it injects errors at the wrong
place vs where these tests expect and does not support a 'max global count'
like the failpoints do with 'n*' syntax.
John Baldwin [Thu, 23 Jul 2020 21:33:10 +0000 (21:33 +0000)]
Pass the right size to memcpy() when copying the array of FP registers.
The size of the containing structure was passed instead of the size of
the array. This happened to be harmless as the extra word copied is
one we copy in the next line anyway.
Michael Tuexen [Thu, 23 Jul 2020 19:43:49 +0000 (19:43 +0000)]
Clear the pointer to the socket when closing it also in case of
an ungraceful operation.
This fixes a use-after-free bug found and reported by Taylor
Brandstetter of Google by testing the userland stack.
Doug Moore [Thu, 23 Jul 2020 17:16:20 +0000 (17:16 +0000)]
Rank balanced (RB) trees are a class of balanced trees that includes
AVL trees, red-black trees, and others. Weak AVL (wavl) trees are a
recently discovered member of that class. This change replaces
red-black rebalancing with weak AVL rebalancing in the RB tree macros.
Wavl trees sit between AVL and red-black trees in terms of how
strictly balance is enforced. They have the stricter balance of AVL
trees as the tree is built - a wavl tree is an AVL tree until the
first deletion. Once removals start, wavl trees are lazier about
rebalancing than AVL trees, so that removals can be fast, but the
balance of the tree can decay to that of a red-black tree. Subsequent
insertions can push balance back toward the stricter AVL conditions.
Removing a node from a wavl tree never requires more than two
rotations, which is better than either red-black or AVL
trees. Inserting a node into a wavl tree never requires more than two
rotations, which matches red-black and AVL trees. The only
disadvantage of wavl trees to red-black trees is that more insertions
are likely to adjust the tree a bit. That's the cost of keeping the
tree more balanced.
Testing has shown that for the cases where red-black trees do worst,
wavl trees better balance leads to faster lookups, so that if lookups
outnumber insertions by a nontrivial amount, lookup time saved exceeds
the extra cost of balancing.
Mark Johnston [Thu, 23 Jul 2020 14:21:45 +0000 (14:21 +0000)]
MFOpenZFS: Fix zpool history unbounded memory usage
In original implementation, zpool history will read the whole history
before printing anything, causing memory usage goes unbounded. We fix
this by breaking it into read-print iterations.
Reviewed-by: Tom Caputi <tcaputi@datto.com> Reviewed-by: Matt Ahrens <matt@delphix.com> Reviewed-by: Igor Kozhukhov <igor@dilos.org> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Chunwei Chen <david.chen@nutanix.com>
Closes #9516
Note, this change changes the libzfs.so ABI by modifying the prototype
of zpool_get_history(). Since libzfs is effectively private to the base
system it is anticipated that this will not be a problem.
vm: annotate swap_reserved with __exclusive_cache_line
The counter keeps being updated all the time and variables read afterwards
share the cacheline. Note this still fundamentally does not scale and needs
to be replaced, in the meantime gets a bandaid.
Brooks Davis [Wed, 22 Jul 2020 23:39:58 +0000 (23:39 +0000)]
Correct a type-mismatch between xdr_long and the variable "bad".
Way back in r28911 (August 1997, CVS rev 1.22) we imported a NetBSD
information leak fix via OpenBSD. Unfortunatly we failed to track the
followup commit that fixed the type of the error code. Apply the change
from int to long now.
Reviewed by: emaste
Found by: CHERI
Obtained from: CheriBSD
MFC after: 3 days
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D25779
Rick Macklem [Wed, 22 Jul 2020 23:33:37 +0000 (23:33 +0000)]
Modify writing to mirrored pNFS DSs to prepare for use of ext_pgs mbufs.
This patch modifies writing to mirrored pNFS DSs slightly so that there is
only one m_copym() call for a mirrored pair instead of two of them.
This call replaces the custom nfsm_copym() call, which is no longer needed
and deleted by this patch. The patch does introduce a new nfsm_split()
function that only calls m_split() for the non-ext_pgs case.
The semantics of nfsm_uiombuflist() is changed to include code that nul
pads the generated mbuf list. This was done by nfsm_copym() prior to this patch.
The main reason for this change is that it allows the data to be a list
of ext_pgs mbufs, since the m_copym() is for the entire mbuf list.
This support will be added in a future commit.
This patch only affects writing to mirrored flexible file layout pNFS servers.
Brooks Davis [Wed, 22 Jul 2020 21:44:51 +0000 (21:44 +0000)]
Avoid reading one byte before the path buffer.
This happens when there's only one component (e.g. "/foo"). This
(mostly-harmless) bug has been present since June 1990 when it was
commited to mountd.c SCCS version 5.9.
Note: the bug is on the second changed line, the first line is changed
for visual consistency.
pkg-bootstrap: complain on improper `pkg bootstrap` usage
Right now, the bootstrap will gloss over things like pkg bootstrap -x or
pkg bootstrap -f pkg. Make it more clear that this is incorrect, and hint
at the correct formatting.
Reported by: jhb (IIRC via IRC)
Approved by: bapt, jhb, manu
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D24750
Tom Jones [Wed, 22 Jul 2020 13:49:54 +0000 (13:49 +0000)]
Add tests for "add", "change" and "delete" functionality of /sbin/route.
Add tests to cover "add", "change" and "delete" functionality of /sbin/route
for ipv4 and ipv6. These tests for the existing route tool are the first step
towards creating libroute.
It is very conservative. Only spinning when LK_ADAPTIVE is passed, only on
exclusive lock and never when any waiters are present. buffer cache is remains
not spinning.
This reduces total sleep times during buildworld etc., but it does not shorten
total real time (culprits are contention in the vm subsystem along with slock +
upgrade which is not covered).
For microbenchmarks: open3_processes -t 52 (open/close of the same file for
writing) ops/s:
before: 258845
after: 801638
libbe: annotate lbh as __unused in be_is_auto_snapshot_name
lbh is included for consistency with other functions and in case
future work needs to use it, but it is currently unused. Mark it,
and a post-OpenZFS-import world will be able to raise WARNS of
libbe to the default (pending some minor changes to openzfs libzfs).
Warner Losh [Wed, 22 Jul 2020 00:44:47 +0000 (00:44 +0000)]
getty appears to date from 3rd edition research unix. That's the oldest man page
on TUHS and its 'unix 1972' restoration effort has assembler sources that look
like simpler version of what's in the 5th edition.
During device attachment, all interrupt sources will bind to the BSP,
as it is the only processor online. This means interrupts must be
redistributed ("shuffled") later, during SI_SUB_SMP.
For the EARLY_AP_STARTUP case, this is no longer true. SI_SUB_SMP will
execute much earlier, meaning APs will be online and available before
devices begin attachment, and there will therefore be nothing to
shuffle.
All PIC-conforming interrupt controllers will handle this early
distribution properly, except for RISC-V's PLIC. Make the necessary
tweak to the PLIC driver.
While here, convert irq_assign_cpu from a boolean_t to a bool.
Remove all variations of rtrequest <rtrequest1_fib, rtrequest_fib,
in6_rtrequest, rtrequest_fib> and their uses and switch to
to rib_action(). This is part of the new routing KPI.
Fix style and comment around concave/convex regions in TCP cubic.
In cubic, the concave region is when snd_cwnd starts growing slower
towards max_cwnd (cwnd at the time of the congestion event), and
the convex region is when snd_cwnd starts to grow faster and
eventually appearing like slow-start like growth.
Mark Johnston [Tue, 21 Jul 2020 15:03:36 +0000 (15:03 +0000)]
traceroute6: Fix most warnings at the default WARNS level.
Fix some style issues as well. Leave -Wno-cast-aligned set for now, as
most of the warnings come casts of CMSG_DATA(), which does provide
sufficient alignment in practice.
arg0 should be an offset of the return point within the function, arg1
should be the return value. Previously the return probe had arguments as
if for the entry probe.
Tested on armv7.
andrew noted that the same problem seems to be present on arm64, mips,
and riscv.
I am not sure if I will get around to fixing those. So, platform users
or anyone looking to make a contribution please be aware of this
opportunity.
I2C communication is done by a combination of driving a line low or
letting it float, so that it is either pulled up or driven low by
another party.
r355276 besides the stated goal of the change -- using the new GPIO API
-- also changed the logic, so that active state is signaled by actively
driving a line.
That worked with iicbb prior to r362042, but stopped working after that
commit on at least some hardware. My guess that the breakage was
related to getting an ACK bit. A device expected to be able to drive
SDA actively low, but controller was actively driving it high for some
time.
Anyway, this change seems to fix the problem.
Tested using gpioiic on Orange Pi PC Plus with HTU21 sensor.
Reported by: Nick Kostirya <nikolay.kostirya@i11.co>
Reviewed by: manu
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25684
- Handle whitespace with long flags that take arguments:
echo 'foo bar' > test
zgrep --regexp='foo bar' test
- Do not hang reading from stdin with patterns in a file:
echo foobar > test
echo foo > pattern
zgrep -f pattern test
zgrep --file=pattern test
- Handle any flags after -e:
echo foobar > test
zgrep -e foo --ignore-case < test
These two are still outstanding problems:
- Does not handle flags that take an argument if there is no
whitespace:
zgrep -enfs /etc/rpc
- When more than one -e pattern used matching should occur for all
patterns (similar to multiple patterns supplied with -f file).
Instead only the last pattern is used for matching:
zgrep -e rex -e nfs /etc/rpc
(This problem is masked in the unpatched version by the "any
flags after -e" problem.)
Add tests for the above problems.
Update the mange and add references to gzip(1) and zstd(1) and also
document the remaining known problems.
Add MODULE_VERSION to TCP loadable congestion control modules.
Without versioning information, using preexisting loader /
linker code is not easily possible when another module may
have dependencies on pre-loaded modules, and also doesn't
allow the automatic loading of dependent modules.
iflib: initialize netmap with the correct number of descriptors
In case the network device has a RX or TX control queue, the correct
number of TX/RX descriptors is contained in the second entry of the
isc_ntxd (or isc_nrxd) array, rather than in the first entry.
This case is correctly handled by iflib_device_register() and
iflib_pseudo_register(), but not by iflib_netmap_attach().
If the first entry is larger than the second, this can result in a
panic. This change fixes the bug by introducing two helper functions
that also lead to some code simplification.
Mark Johnston [Mon, 20 Jul 2020 18:22:38 +0000 (18:22 +0000)]
libdwarf: Hide SHT_NOBITS sections.
gnu_debuglink external debug files will contain an .eh_frame section of
type SHT_NOBITS. libdwarf does not handle such sections (or rather, it
expects all debug sections to not have type SHT_NOBITS). Avoid loading
SHT_NOBITS sections, to be consistent with SGI libdwarf's handling of
this case.
projects / FreeBSD / FreeBSD.git / log