Kristof Provost [Tue, 29 Jun 2021 08:26:40 +0000 (10:26 +0200)]
pf: Reduce the data returned in DIOCGETSTATESNV
This call is particularly slow due to the large amount of data it
returns. Remove all fields pfctl does not use. There is no functional
impact to pfctl, but it somewhat speeds up the call.
It might affect other (i.e. non-FreeBSD) code that uses the new
interface, but this call is very new, so there's unlikely to be any. No
releases contained the previous version, so we choose to live with the
ABI modification.
Kristof Provost [Mon, 28 Jun 2021 10:48:20 +0000 (12:48 +0200)]
pf tests: Stress state retrieval
Create and retrieve 20.000 states. There have been issues with nvlists
causing very slow state retrieval. We don't impose a specific limit on
the time required to retrieve the states, but do log it. In excessive
cases the Kyua timeout will fail this test.
Alexander Motin [Fri, 25 Jun 2021 17:52:58 +0000 (13:52 -0400)]
Allow sleepq_signal() to drop the lock.
Introduce SLEEPQ_DROP sleepq_signal() flag, allowing one to drop the
sleep queue chain lock before returning. Reduced lock scope allows
significantly reduce lock contention inside taskqueue_enqueue() for
ZFS worker threads doing ~350K disk reads/s on 40-thread system.
Olivier Houchard [Wed, 30 Jun 2021 20:56:50 +0000 (22:56 +0200)]
arm: Make sure we can handle a thumb entry point.
Similarly to what's been done on arm64 with commit 712c060c94fd447c91b0e6218c12a431206b487a, when executing a binary, if the
entry point is a thumb symbol, then make sure we set the PSL_T flag, otherwise
the CPU will interpret it in ARM mode, and that will likely leads to an
undefined instruction.
Olivier Houchard [Wed, 30 Jun 2021 12:50:47 +0000 (14:50 +0200)]
arm64: Make sure COMPAT_FREEBSD32 handles thumb entry point.
If the entry point for the binary executed is a thumb 2 entry point, make
sure we set the PSR_T bit, or the CPU will interpret it as arm32 code and
bad things will happen.
Ed Maste [Sun, 27 Jun 2021 17:21:26 +0000 (13:21 -0400)]
Add deprecation notice for WITH_PROFILE option
As discussed on freebsd-current [1] and freebsd-arch [2] and review
D30833, FreeBSD 14 will ship without the _p.a libraries built with -pg.
Both upstream and base system (in commit b762974cf4b9) Clang have been
modified to remove the special case for linking against these libraries.
Clang's -pg support and mcount() remain, so building with -pg can still
be used on code that the user builds; we just do not provide prebuilt
libraries compiled with -pg. A similar change is still needed for GCC.
Mariusz Zaborski [Fri, 11 Jun 2021 15:35:36 +0000 (17:35 +0200)]
libnv: optimize nvlist size calculation
If we had a multiple nvlist, during nvlist_pack, we calculated the size
of every nvlist separately. For example, if we had a nvlist with three
nodes each containing another (A contains B, and B contains C), we first
calculated the size of nvlist A (which contains B, C), then we calculate
the size of B (which contains C, notice that we already did the
calculation of B, when we calculate A), and finally C. This means that
this calculation was O(N!). This was done because each time we pack
nvlist, we have to put its size in the header
(the separate header for A, B, and C).
To not break the ABI and to reduce the complexity of nvlist_size,
instead of calculating the nvlist size when requested,
we track the size of each nvlist.
Mariusz Zaborski [Sun, 17 Feb 2019 18:26:27 +0000 (18:26 +0000)]
libnv: fix double free
In r343986 we introduced a double free. The structure was already
freed fixed in the r302966. This problem was introduced
because the GitHub version was out of sync with the FreeBSD one.
Submitted by: Mindaugas Rasiukevicius <rmind@netbsd.org>
MFC with: r343986
Mariusz Zaborski [Sun, 10 Feb 2019 23:28:55 +0000 (23:28 +0000)]
libnv: fix memory leaks
nvpair_create_stringv: free the temporary string; this fix affects
nvlist_add_stringf() and nvlist_add_stringv().
nvpair_remove_nvlist_array (NV_TYPE_NVLIST_ARRAY case): free the chain
of nvpairs (as resetting it prevents nvlist_destroy() from freeing it).
Note: freeing the chain in nvlist_destroy() is not sufficient, because
it would still leak through nvlist_take_nvlist_array(). This affects
all nvlist_*_nvlist_array() use
Submitted by: Mindaugas Rasiukevicius <rmind@netbsd.org>
Reported by: clang/gcc ASAN
MFC after: 2 weeks
Stefan Eßer [Tue, 29 Jun 2021 07:01:13 +0000 (09:01 +0200)]
md5: Improve compatibility with coreutils and format fix
The previous changes that added support for the coreutils -c option
modified the output generated by passing -r to match that of the
coreutils versions. The difference is that coreutils separates the
hash from the file name by two blanks " " (or a blank followed by
an asterisk " *" with the -b option denoting).
While most scripts or users will not notice the difference, it might
be considered a violation of POLA and this commit reverts the change
for the non-sum programs. These will print a single blank " " as the
separator, as they die before the previous commit.
In order to still generate output that is identical to that of the
coreutils programs, this commit generates the " " or " *" separator
used by them for the -sum versions, depending on the presence of the
-b option.
Lutz Donnerhacke [Thu, 27 May 2021 21:42:54 +0000 (23:42 +0200)]
libalias: Switch to SPLAY trees
Current data structure is using a hash of unordered lists. Those
unordered lists are quite efficient, because the least recently
inserted entries are most likely to be used again. In order to avoid
long search times in other cases, the lists are hashed into many
buckets. Unfortunatly a search for a miss needs an exhaustive
inspection and a careful definition of the hash.
Splay trees offer a similar feature - almost O(1) for access of the
least recently used entries), and amortized O(ln(n) - for almost all
other cases. Get rid of the hash.
Now the data structure should able to quickly react to external
packets without eating CPU cycles for breakfast, preventing a DoS.
Lutz Donnerhacke [Tue, 25 May 2021 13:03:34 +0000 (15:03 +0200)]
libalias: Restructure
Clean up the database handling in order to switch to more efficient
data structures. The development of this patch was artificially split
in to many small steps to ease reviewing.
- Common search terms
- Separate fully qualified search
- Separate table for partial links
- Cleanup _FindLinkIn
- Factor out the outgoing search function
- Factor out a common idiom to return found links
- Reorder incoming links by grouping of common search terms
- Remove LSNAT from outgoing search
- Group internal structure semantically
- Separate table for PPTP
- Use AliasRange instead of PORT_BASE
- Remove temporary state deleteAllLinks from global struct
- Avoid uninitialized expiration
Lutz Donnerhacke [Tue, 25 May 2021 07:07:21 +0000 (09:07 +0200)]
libalias: Promote per instance global variable timeStamp
Summary:
- Use LibAliasTime as a real global variable for central timekeeping.
- Reduce number of syscalls in user space considerably.
- Dynamically adjust the packet counters to match the second resolution.
- Only check the first few packets after a time increase for expiry.
Lutz Donnerhacke [Sat, 15 May 2021 15:35:36 +0000 (17:35 +0200)]
libalias: tidy up housekeeping
Replace current expensive, but sparsly called housekeeping
by a single, repetive action.
This is part of a larger restructure of libalias in order to switch to
more efficient data structures. The whole restructure process is
split into 15 reviews to ease reviewing. All those steps will be
squashed into a single commit for MFC in order to hide the
intermediate states from production systems.
Mike Karels [Sun, 20 Jun 2021 17:50:31 +0000 (12:50 -0500)]
genet: pullup minimum header amount for IPv4
The genet driver (RPi4 Ethernet) had code to pull headers into the
first mbuf if there was only an Ethernet header there. This was
originally needed for ICMPv6 replies, then for forwarded IPv6/TCP.
Now a situation has been found where it is needed for IPv4, when
using NAT with IPFW. Generalize to do this for all protocols.
Rather than using an IPv6-related definition for the length, move
the length to a variable that can be set with sysctl
(hw.genet.tx_hdr_min). Move an old tunable to a new RDTUN variable
with a better name.
Mike Karels [Sun, 20 Jun 2021 16:10:26 +0000 (11:10 -0500)]
genet: Fix potential crash during attach
As pointed out in the bug, the genet driver (RPi4 Ethernet) was
attaching the interrupts before the data structures were fully
initialized, causing a crash if an interrupt came in during the
attach. Fix by reordering code blocks.
PR: 256334
Reported by: < ghuckriede at blackberry.com >
Reviewed by: < ghuckriede at blackberry.com > (informally)
Mateusz Guzik [Wed, 30 Jun 2021 14:15:25 +0000 (16:15 +0200)]
mbuf: add m_free_raw to be used instead of directly calling uma_zfree
The intent is to remove all direct zone_mbuf consumers so that ctor/dtor
from that zone can be reimplemented as wrappers around uma, avoiding an
indirect function call.
As for example pfctl -ss keeps calling it, it saves a lot of overhead
from elided parsing of /etc/nsswitch.conf and /etc/protocols.
Sample result when running a pre-nvlist binary with nfs root and dumping
7 mln states:
before: 24.817u 62.993s 1:28.52 99.1%
after: 8.064u 1.117s 0:18.87 48.5%
Alex Richardson [Tue, 22 Jun 2021 08:09:44 +0000 (09:09 +0100)]
rtld-elf: Check the return value of obj_enforce_relro()
The mprotect() call was failing on CheriBSD when changing rtld's relro
page permissions due to missing CHERI capability permissions on the
mprotect() argument but did not report an error since the return value
was being ignored. It should never fail on any supported FreeBSD
architecture, but checking the return value seems like a good
sanity check to me.
Alex Richardson [Wed, 16 Jun 2021 15:35:55 +0000 (16:35 +0100)]
truss: minor cleanup and pedantic warning fixes
Noticed while porting the recent truss compat32 changes to CheriBSD.
This also fixes i386 tracing by zero-extending user addresses instead
of sign-extending them.
Reviewed By: jhb
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D30211
Alex Richardson [Wed, 16 Jun 2021 15:29:57 +0000 (16:29 +0100)]
truss: fix handling of 64-bit arguments/return values for compat32
Deciding whether to combine two values to a 64-bit one should be based on
the process ABI, and not dependent on whether truss is compiled for an
LP64 ABI. This is a follow-up cleanup for D27625. I found this while
looking for uses of the `__LP64__` macro (since using this is wrong for
CHERI systems).
Test Plan: truss still works. Since I tested on AMD64 and all syscalls
in the trace have their quad argument last there was no difference
in the output. Should fix output for compat32 on MIPS64 though.
Reviewed By: jhb
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D27637
Alex Richardson [Wed, 16 Jun 2021 15:28:27 +0000 (16:28 +0100)]
kdump: use VIS_NOLOCALE when printing output
Attempting to parse the output as UTF-8 exposed an out-of-bounds read
in vis(3) (see https://reviews.freebsd.org/D30771). However, I don't
think it makes much sense to decode this output at all. Additionally,
passing one byte at a time will almost certainly result in incorrect
output for multi-byte characters.
Reviewed By: brooks
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D30772
Alex Richardson [Wed, 16 Jun 2021 15:27:13 +0000 (16:27 +0100)]
vis(3): avoid out-of-bounds stack buffer reads
I found this while running kdump(1) on a CheriBSD system due to a
capability length violation when printing the /etc/libmap.conf read()
system call: it crashed immediately after printing the first line.
Found by: CHERI
Reviewed By: jhb
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D30771
Alex Richardson [Wed, 3 Feb 2021 15:27:17 +0000 (15:27 +0000)]
Expose clang's alignment builtins and use them for roundup2/rounddown2
This makes roundup2/rounddown2 type- and const-preserving and allows
using it on pointer types without casting to uintptr_t first. Not
performing pointer-to-integer conversions also helps the compiler's
optimization passes and can therefore result in better code generation.
When using it with integer values there should be no change other than
the compiler checking that the alignment value is a valid power-of-two.
I originally implemented these builtins for CHERI a few years ago and
they have been very useful for CheriBSD. However, they are also useful
for non-CHERI code so I was able to upstream them for Clang 10.0.
Rationale from the clang documentation:
Clang provides builtins to support checking and adjusting alignment
of pointers and integers. These builtins can be used to avoid relying
on implementation-defined behavior of arithmetic on integers derived
from pointers. Additionally, these builtins retain type information
and, unlike bitwise arithmetic, they can perform semantic checking on
the alignment value.
There is also a feature request for GCC, so GCC may also support it in
the future: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98641
John Baldwin [Wed, 17 Feb 2021 21:12:06 +0000 (13:12 -0800)]
fwohci: Cast bitfield to uint32_t before passing it to roundup2().
The fallback for __align_up() used by roundup2() uses __typeof__()
which doesn't work for bitfields. This fixes the build on GCC which
uses the fallback.
Alan Somers [Fri, 18 Jun 2021 00:04:59 +0000 (18:04 -0600)]
fusefs: ensure that FUSE ops' headers' unique values are actually unique
Every FUSE operation has a unique value in its header. As the name
implies, these values are supposed to be unique among all outstanding
operations. And since FUSE_INTERRUPT is asynchronous and racy, it is
desirable that the unique values be unique among all operations that are
"close in time".
Ensure that they are actually unique by incrementing them whenever we
reuse a fuse_dispatcher object, for example during fsync, write, and
listextattr.
Alan Somers [Tue, 15 Jun 2021 20:24:05 +0000 (14:24 -0600)]
fusefs: improve warnings about buggy FUSE servers
The fusefs driver will print warning messages about FUSE servers that
commit protocol violations. Previously it would print those warnings on
every violation, but that could spam the console. Now it will print
each warning no more than once per lifetime of the mount. There is also
now a dtrace probe for each violation.
Ryan Libby [Wed, 7 Apr 2021 19:39:05 +0000 (12:39 -0700)]
shared shadow vm object invalidation regression test
Add a regression test for a scenario where a shadow vm object is shared
by multiple mappings. If a page COW occurs through one of the mappings,
then the virtual-to-physical mapping may become invalidated.
Ryan Libby [Wed, 24 Feb 2021 23:56:16 +0000 (15:56 -0800)]
ddb: reliably fail with ambiguous commands
db_cmd_match had an even/odd bug, where if a third command was partially
matched (or any odd number greater than one) the search result would be
set back from CMD_AMBIGUOUS to CMD_FOUND, causing the last command in
the list to be executed instead of failing the match.
Ryan Libby [Wed, 24 Feb 2021 23:56:16 +0000 (15:56 -0800)]
ofed: quiet gcc -Wint-in-bool-context
The int in the argument to the ternary triggered -Wint-in-bool-context
from gcc. Upstream linux has a larger and more entangled patch, 12f727721eee61b3d19dedb95cb893b2baa9fe41, which doesn't apply cleanly.
When we eventually sync that, we can just drop this change.
Emmanuel Vadot [Mon, 3 May 2021 08:12:26 +0000 (10:12 +0200)]
pkgbase: Move librt to clibs
librt implement the POSIX realtime extension library.
Move it to clibs instead of utilities as a number of ports uses it
so avoid a dependancy on FreeBSD-utilities.
Emmanuel Vadot [Tue, 16 Mar 2021 06:12:56 +0000 (07:12 +0100)]
pkgbase: Remove case for runtime and jail package ucl generation
They aren't needed and produce wrong package comments :
We use to have "runtime-dev package" instead of
"FreeBSD Base System (Development Files)" for example
Rick Macklem [Wed, 16 Jun 2021 00:42:13 +0000 (17:42 -0700)]
nfscl: Make NFSv4.0 client acquisition NFSv4.1/4.2 compatible
When the NFSv4.0 client was implemented, acquisition of a clientid
via SetClientID/SetClientIDConfirm was done upon the first Open,
since that was when it was needed. NFSv4.1/4.2 acquires the clientid
during mount (via ExchangeID/CreateSession), since the associated
session is required during mount.
This patch modifies the NFSv4.0 mount so that it acquires the
clientid during mount. This simplifies the code and makes it
easy to implement "find the highest minor version supported by
the NFSv4 server", which will be done for the default minorversion
in a future commit.
The "start_renewthread" argument for nfscl_getcl() is replaced
by "tryminvers", which will be used by the aforementioned
future commit.
Brian Behlendorf [Tue, 29 Jun 2021 20:16:38 +0000 (13:16 -0700)]
Linux 5.13 compat: META
Increase the Linux-Maximum version in the META file to 5.13.
All of the required compatibility patches have been merged
and the 5.13 kernel has been officially released.
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Alexander Motin [Tue, 29 Jun 2021 12:59:14 +0000 (08:59 -0400)]
Avoid 64bit division in multilist index functions
The number of sublists in a multilist is relatively small. We dont need
64 bits to calculate an index. 32 bits is sufficient and makes the
code more efficient.
Reviewed-by: Matthew Ahrens <mahrens@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Mark Maybee <mark.maybee@delphix.com> Signed-off-by: Alexander Motin <mav@FreeBSD.org> Sponsored-By: iXsystems, Inc.
Closes #12288
projects / FreeBSD / FreeBSD.git / log