sbin/md5: improve compatibility with coreutils -c option
The -c option expects a digest file in either BSD or coreutils format.
The output for matched and mismatched files is identical to that
of the coreutils version.
The review of these changes included test cases that have already
been committed for the functionality that existed before.
Another test script is added to cover the coreutils compatible
extension implemented by this patch.
This commit contains a tests/Makefile that has been cleaned up
compared to the review version, using an implicit rule to apply the
TESTBASE path at build time (and the scripts have been renamed to
have an extension of .SH instead of .sh to trigger this rule).
md5: Improve compatibility with coreutils and format fix
The previous changes that added support for the coreutils -c option
modified the output generated by passing -r to match that of the
coreutils versions. The difference is that coreutils separates the
hash from the file name by two blanks " " (or a blank followed by
an asterisk " *" with the -b option denoting).
While most scripts or users will not notice the difference, it might
be considered a violation of POLA and this commit reverts the change
for the non-sum programs. These will print a single blank " " as the
separator, as they die before the previous commit.
In order to still generate output that is identical to that of the
coreutils programs, this commit generates the " " or " *" separator
used by them for the -sum versions, depending on the presence of the
-b option.
Warner Losh [Thu, 20 May 2021 17:26:46 +0000 (11:26 -0600)]
md5: portability fix -- include stdbool.h explicitly
stdbool.h needs to be included to use type bool variables. Due to
namespace pollution, this gets brought in on FreeBSD, but not on
other systems. Include it explicilty.
Warner Losh [Wed, 19 May 2021 17:26:20 +0000 (11:26 -0600)]
md5: Create md5sum, etc compatible programs
On Linux, there's a similar set of programs to ours, but that end in the
letters 'sum'. These act basically like FreeBSD versions run with the -r
option. Add code so that when the program ends in 'sum' you get the
linux -r behavior. This is enough to make most things that use sha*sum
work correctly (the -c / --check options, as well as the long args are
not implemented). When running with the -sum programs, ignore -t instead
of running internal speed tests and make -c an error.
Reviewed by: sef, and kp and allanjude (earlier version)
Relnotes: yes
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D30309
While the correctness of the supported hash algorithms can be tested
with the built-in self-test feature, these test cases are meant to
detect changes in the output format.
A follow-up commit will improve the compatibility with the coreutils
versions of the hash programs, and these tests should detect any
unintended side-effects of such a change on existing functionality.
sbin/md5: fix test case to not write outside temp directory
The bsd-p-test did create files 1.out .. 8.out in the test directory.
This has been fixed and the temporary output files are now written
to the temporary directory.
Lutz Donnerhacke [Thu, 27 May 2021 21:42:54 +0000 (23:42 +0200)]
libalias: Switch to SPLAY trees
Current data structure is using a hash of unordered lists. Those
unordered lists are quite efficient, because the least recently
inserted entries are most likely to be used again. In order to avoid
long search times in other cases, the lists are hashed into many
buckets. Unfortunatly a search for a miss needs an exhaustive
inspection and a careful definition of the hash.
Splay trees offer a similar feature - almost O(1) for access of the
least recently used entries), and amortized O(ln(n) - for almost all
other cases. Get rid of the hash.
Now the data structure should able to quickly react to external
packets without eating CPU cycles for breakfast, preventing a DoS.
Lutz Donnerhacke [Tue, 25 May 2021 13:03:34 +0000 (15:03 +0200)]
libalias: Restructure
Clean up the database handling in order to switch to more efficient
data structures. The development of this patch was artificially split
in to many small steps to ease reviewing.
- Common search terms
- Separate fully qualified search
- Separate table for partial links
- Cleanup _FindLinkIn
- Factor out the outgoing search function
- Factor out a common idiom to return found links
- Reorder incoming links by grouping of common search terms
- Remove LSNAT from outgoing search
- Group internal structure semantically
- Separate table for PPTP
- Use AliasRange instead of PORT_BASE
- Remove temporary state deleteAllLinks from global struct
- Avoid uninitialized expiration
Lutz Donnerhacke [Tue, 25 May 2021 07:07:21 +0000 (09:07 +0200)]
libalias: Promote per instance global variable timeStamp
Summary:
- Use LibAliasTime as a real global variable for central timekeeping.
- Reduce number of syscalls in user space considerably.
- Dynamically adjust the packet counters to match the second resolution.
- Only check the first few packets after a time increase for expiry.
Lutz Donnerhacke [Sat, 15 May 2021 15:35:36 +0000 (17:35 +0200)]
libalias: tidy up housekeeping
Replace current expensive, but sparsly called housekeeping
by a single, repetive action.
This is part of a larger restructure of libalias in order to switch to
more efficient data structures. The whole restructure process is
split into 15 reviews to ease reviewing. All those steps will be
squashed into a single commit for MFC in order to hide the
intermediate states from production systems.
As for example pfctl -ss keeps calling it, it saves a lot of overhead
from elided parsing of /etc/nsswitch.conf and /etc/protocols.
Sample result when running a pre-nvlist binary with nfs root and dumping
7 mln states:
before: 24.817u 62.993s 1:28.52 99.1%
after: 8.064u 1.117s 0:18.87 48.5%
Rick Macklem [Sat, 26 Jun 2021 23:47:59 +0000 (16:47 -0700)]
param.h: Bump __FreeBSD_version for commit 6ae32cc8182f
Commit 6ae32cc8182f changed the internal KAPI between the krpc and
nfsd modules. As such, both need to be rebuilt from sources.
Bump __FreeBSD_version to 1202508.
Rick Macklem [Fri, 11 Jun 2021 23:57:14 +0000 (16:57 -0700)]
krpc: Acquire ref count of CLIENT for backchannel use
Michael Dexter <editor@callfortesting.org> reported
a crash in FreeNAS, where the first argument to
clnt_bck_svccall() was no longer valid.
This argument is a pointer to the callback CLIENT
structure, which is free'd when the associated
NFSv4 ClientID is free'd.
This appears to have occurred because a callback
reply was still in the socket receive queue when
the CLIENT structure was free'd.
This patch acquires a reference count on the CLIENT
that is not CLNT_RELEASE()'d until the socket structure
is destroyed. This should guarantee that the CLIENT
structure is still valid when clnt_bck_svccall() is called.
It also adds a check for closed or closing to
clnt_bck_svccall() so that it will not process the callback
RPC reply message after the ClientID is free'd.
Rick Macklem [Mon, 7 Jun 2021 20:48:25 +0000 (13:48 -0700)]
nfscl: Fix generation of va_fsid for a tree of NFSv4 server file systems
Pre-r318997 the code looked like:
if (vp->v_mount->mnt_stat.f_fsid.val[0] != (uint32_t)np->n_vattr.na_filesid[0])
vap->va_fsid = (uint32_t)np->n_vattr.na_filesid[0];
Doing this assignment got lost by r318997 and, as such, NFSv4 mounts
of servers with trees of file systems on the server is broken, due to duplicate
fileno values for the same st_dev/va_fsid.
Although I could have re-introduced the assignment, since the value of
na_filesid[0] is not guaranteed to be unique across the server file systems,
I felt it was better to always do the hash for na_filesid[0,1].
Since dev_t (st_dev/va_fsid) is now 64bits, I switched to a 64bit hash.
There is a slight chance of a hash conflict where 2 different na_filesid
values map to same va_fsid, which will be documented in the BUGS
section of the man page for mount_nfs(8). Using a table to keep track
of mappings to catch conflicts would not easily scale to 10,000+ server file
systems and, when the conflict occurs, it only results in fts(3) reporting
a "directory cycle" under certain circumstances.
Alex Richardson [Tue, 30 Mar 2021 13:53:41 +0000 (14:53 +0100)]
sys/dev/md: Drop unncessary __GLOBL(mfs_root)
LLVM12 complains if you change the symbol binding:
error: mfs_root_end changed binding to STB_WEAK [-Werror,-Winline-asm]
error: mfs_root changed binding to STB_WEAK [-Werror,-Winline-asm]
Kristof Provost [Fri, 4 Jun 2021 08:52:07 +0000 (10:52 +0200)]
pf: store L4 headers in pf_pdesc
Rather than pointers to the headers store full copies. This brings us
slightly closer to what OpenBSD does, and also makes more sense than
storing pointers to stack variable copies of the headers.
Dimitry Andric [Fri, 25 Jun 2021 18:42:38 +0000 (20:42 +0200)]
Work around bogus old gcc "initializer element is not constant" error
After df3b437c1e073eb83e9a93af1c417f3ee8d0de3b, older gcc's such as
4.2.1 (still used on earlier branches for e.g. mips and powerpc) and
6.3.0 (still used for some cross-builds) started throwing bogus errors
like:
In file included from /workspace/src/lib/msun/src/s_llround.c:11:0:
/workspace/src/lib/msun/src/s_lround.c:54:31: error: initializer element is not constant
static const type dtype_min = type_min - 0.5;
^~~~~~~~
/workspace/src/lib/msun/src/s_lround.c:55:31: error: initializer element is not constant
static const type dtype_max = type_max + 0.5;
^~~~~~~~
Since 'type_min' and 'type_max' are constants declared just above these
lines this error is nonsensical, but older gcc's are not smart enough.
Work around the error by reusing the (type)DTYPE_MIN and (type)DTYPE_MAX
macros, so I can MFC this right away, unbreaking a few stable builds.
Dimitry Andric [Tue, 22 Jun 2021 16:38:27 +0000 (18:38 +0200)]
Fix failures in libm's lround_test after clang 12 import
It turned out that the (type)DTYPE_MAX conversions at the top of
s_lround.c are now emitted as cvtsi2sd instructions, at least on SSE
capable CPUs. This caused the FE_INEXACT flag to always be set, at least
for the double and float variants. Under clang 11, the whole INRANGE()
comparisons were still optimized away, but this has "improved" in clang
12, due to stricter adherence to the -ffp-exception-behavior=maytrap
compiler flag.
To avoid run-time integer to float conversions, use static constants
instead, so they are computed at compile time, and the INRANGE()
statements are optimized away again, if applicable.
While here, use an integer instead of a floating type to store the test
results in lround_test.c, as this is more appropriate, and we can also
drop the volatile hack.
Dimitry Andric [Tue, 22 Jun 2021 20:26:13 +0000 (22:26 +0200)]
Disable llvm generating 128-bit multiply libcalls on 32-bit ARM
Merge commit 789708617d20 from llvm git (Koutheir Attouchi):
Do not generate calls to the 128-bit function __multi3() on 32-bit ARM
Re-applying this patch after bots failures. Should be fine now.
The function __multi3() is undefined on 32-bit ARM, so a call to it should
never be emitted. Instead, plain instructions need to be generated to
perform 128-bit multiplications.
Mark Johnston [Wed, 19 Dec 2018 04:54:32 +0000 (04:54 +0000)]
Remove a use of a negative array index from fxp(4).
This fixes a warning seen when compiling amd64 GENERIC with clang 7.
Also remove the workaround added in r337324. clang 7 and gcc 4.2
generate the same code with or without the code change.
Dimitry Andric [Mon, 14 Jun 2021 19:17:05 +0000 (21:17 +0200)]
Export various 128 bit long double functions from libgcc_s.so.1
These were already compiled for some time on aarch64 and riscv, by
including lib/libcompiler_rt/Makefile.inc, but never exported in the
shared library. Since gcc exports these under version GCC_4.6.0, we do
the same.
This review should replace D11482 for now. For e.g. amd64 more work is
still to be done, as compiler-rt does not seem to support 128 bit long
double math for that architecture.
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D28690
Kristof Provost [Fri, 21 May 2021 12:26:49 +0000 (14:26 +0200)]
dummynet: Fix schedlist and aqmlist locking
These are global (i.e. shared across vnets) structures, so we need
global lock to protect them. However, we look up entries in these lists
(find_aqm_type(), find_sched_type()) and return them. We must ensure
that the returned structures cannot go away while we are using them.
Resolve this by using NET_EPOCH(). The structures can be safely accessed
under it, and we postpone their cleanup until we're sure they're no
longer used.
Test functionality of ng_vlan_rotate(4):
- Rotate 1 to 9 stagged vlans in any possible direction and length
- Rotate random combinations of ethertypes (8100, 88a8, 9100)
- Automatic reverse rotating for backward data flow
- Test too many and too few vlans
Test functionality of ng_hub(4):
- replicting traffic to anything but the sending hook
- persistence
- an unrestricted loop
- implementation limits with many hooks.
Test functionality of ng_bridge(4):
- replicating traffic to anything but the sending hook
- persistence
- detect loops
- unicast to only one link of many
- stretch to implementation limits on broadcast
tests/netgraph: Inital framework for testing libnetgraph
Provide a framework of functions to test various netgraph modules.
Tests contain:
- creating, renaming, and destroying nodes
- connecting and removing hooks
- sending and receiving data
- sending ASCII messages and receiving binary responses
- errors can be passed for indiviual inspection or fail the test
Also contains some fixups:
- indent all files correctly
- finish factoring out
- remove debugging code
- check for renaming issues reported in PR241954
Casper services expect that the first 3 descriptors (stdin/stdout/stderr)
will point to /dev/null. Which Casper will ensure later. The Casper
services are forked from the original process. If the initial process
closes one of those descriptors, Casper may reuse one of them for it on
purpose. If this is the case, then renumarate the descriptors used by
Casper to higher numbers. This is done already after the fork, so it
doesn't break the parent process.
Kristof Provost [Sat, 15 May 2021 11:45:55 +0000 (13:45 +0200)]
pf: Convenience function for optional (numeric) arguments
Add _opt() variants for the uint* functions. These functions set the
provided default value if the nvlist doesn't contain the relevant value.
This is helpful for optional values (e.g. when the API is extended to
add new fields).
While here simplify the header by also using macros to create the
prototypes for the macro-generated function implementations.
netgraph/ng_base: Renaming a node to the same name is a noop
Detailed analysis in https://github.com/genneko/freebsd-vimage-jails/issues/2
brought the problem down to a double call of ng_node_name() before and
after a vnet move. Because the name of the node is already known
(occupied by itself), the second call fails.
PR: 241954
Reported by: Paul Armstrong
Differential Revision: https://reviews.freebsd.org/D30110
Ed Maste [Thu, 7 Nov 2019 15:51:44 +0000 (15:51 +0000)]
linux_renameat2: improve flag checks
In the cases where Linux returns an error (e.g. passing in an undefined
flag) there's no need for us to emit a message. (The target of this
message is a developer working on the linuxulatorm, not the author of
presumably broken Linux software).
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21606
Dmitry Chagin [Wed, 26 May 2021 05:34:32 +0000 (08:34 +0300)]
linux_common: retire extra module version.
The second 'linuxcommon' line was added by c66f5b079d2a259c3a65b1efe0f2143cd030dc52
but Linuxulator's modules dependend on 'linux_common'.
To avoid such mistakes in the future rename moduledata name and module
name to 'linux_common' and retire 'linuxcommon' line.
Tijl Coosemans [Sun, 6 Dec 2020 10:58:55 +0000 (10:58 +0000)]
Move V4L feature declarations and DTrace provider definitions from
linux_common.c to linux_util.c so they become available on i386.
linux_common.c defines the linux_common kernel module but this module does
not exist on i386 and linux_common.c is not included in the linux module.
linux_util.c is included in the linux_common module on amd64 and the linux
module on i386.
Remove linux_common.c from files.i386 again. It was added recently in
r367433 when the DTrace provider definitions were moved.
The V4L feature declarations were moved to linux_common in r283423.
Tijl Coosemans [Sat, 5 Dec 2020 14:53:24 +0000 (14:53 +0000)]
Fix i386 linux module after r367395.
In r367395 parts of machine dependent linux_dummy.c were moved to a new
machine independent file sys/compat/linux/linux_dummy.c and the existing
linux_dummy.c was renamed to linux_dummy_machdep.c.
Add linux_dummy_machdep.c to the linux module for i386.
Rename sys/amd64/linux32/linux_dummy.c for consistency.
Add the new linux_dummy.c to the linux module for i386.
Conrad Meyer [Fri, 6 Nov 2020 22:04:57 +0000 (22:04 +0000)]
linux(4): Fix loadable modules after r367395
Move dtrace SDT definitions into linux_common module code. Also, build
linux_dummy.c into the linux_common kld -- we don't need separate
versions of these stubs for 32- and 64-bit emulation.
John Baldwin [Tue, 20 Apr 2021 20:22:11 +0000 (13:22 -0700)]
etcupdate: Gracefully handle SIGINT when building trees.
Run the 'build_tree' function inside of a subshell and trap SIGINT to
return an error to the caller. This allows callers to gracefully
cleanup a partially created tree.
While here, redirect stdout/stderr of the subshell to the log file
instead of applying redirections individually to each command executed
while building the tree.
John Baldwin [Tue, 20 Apr 2021 20:21:42 +0000 (13:21 -0700)]
etcupdate: Always extract to a temporary tree.
etcupdate has had a somewhat nasty race condition since its creation
in that its state machine can get very confused if it is interrupted
while building the tree to compare against. This is exacerbated by
the fact that etcupdate doesn't emit any output while building the
tree which can take several seconds (especially in recent years with
the addition of the tree-wide buildconfig/installconfig passes).
To mitigate this, always install a new tree into a temporary directory
created via mktemp as was previously done only for dry-runs via -n.
The existing trees are only rotated and the new tree installed as
/var/db/etcupdate/current after the update command has completed.
David Bright [Mon, 24 May 2021 19:02:43 +0000 (14:02 -0500)]
pciconf: Fix up pciconf -lc output
The pciconf command fails to emit newlines when particular ecap field
values are seen. Fix them up. This has been seen on several systems at
$JOB. The documentation for PCI capabilities says that capability
type 0 should not be used once the spec for PCI capabilities was
published, but that seems more wishful-thinking than reality. pciconf
also chooses not to print fields related to field values that are
zero, but it seems several of these fields are zero on actual
hardware.
Sponsored by: Dell EMC Isilon
Submitted by: Robert Herndon (Robert.Herndon@dell.com)
David Bright [Mon, 24 May 2021 17:12:15 +0000 (12:12 -0500)]
libsa: Fix infinite loop in bzipfs & gzipfs
A bug in the loader's bzipfs & gzipfs filesystems caused compressed
kernel and modules not to work on EFI systems with a veriexec-enabled
loader. Since the size of files in these filesystems are not known
_a priori_ `stat` would initialize the size to -1 and the loader would
then hang in an infinite loop while trying to seek (read) to the end
of file since the loop termination condition compares the current
offset to that negative target position.
projects / FreeBSD / FreeBSD.git / log