Ian Lepore [Tue, 22 Sep 2020 14:59:05 +0000 (14:59 +0000)]
MFC r365729:
Add product ID strings for a couple Microchip usb hubs. Also, update the
vendor ID string to say just "Microchip Technology" -- the buyout of
Standard Microsystems happened in 2012 and the SMC/SMSC names are pretty
much retired at this point.
MFC r365829, r365837, r365852: certctl rehash upon install/distribute
r365829:
installworld: run `certctl rehash` after installation completes
This was originally introduced back in r360833, and subsequently reverted
because it was broken for -DNO_ROOT builds and it may not have been the
correct place for it.
While debatably this may still not be 'the correct place,' it's much cleaner
than scattering rehashes all throughout the tree. brooks has fixed the issue
with -DNO_ROOT by properly writing to the METALOG in r361397.
Do note that this is different than what was originally committed; brooks
had revisions in D24932 that made it actually use the revised unprivileged
mode and write to METALOG, along with being a little more friendly to
foreign crossbuilds and just using the certctl in-tree.
With this change, I believe we should now have a populated /etc/ssl/certs in
the VM images.
r365837:
Promote the installworld `certctl rehash` to distributeworld
Contrary to my belief, installworld is not sufficient for getting certs
installed into VM images. Promote the rehash to both installworld and
distributeworld (notably: not stageworld) and rehash the base distdir so we
end up with /etc/ssl/certs populated in the base dist archive. A future
commit will remove the rehash from bsdinstall, which doesn't really need to
happen if they're installed into base.txz.
While here, fix a minor typo: s/CERTCLTFLAGS/CERTCTLFLAGS/
r365852:
Revert r361257: bsdinstall: do a `certctl rehash` upon installation [...]
As of r365829, any given base distribution set will now include the /etc/ssl
symlinks that this rehash would've otherwise installed. This extra step is
no longer required.
MFC r365237:
Micro optimise _callout_stop_safe() by removing dead code.
The CS_DRAIN flag cannot be set at the same time like the async-drain function
pointer is set. These are orthogonal features. Assert this at the beginning
of the function.
Before:
if (flags & CS_DRAIN) {
/* FALLTHROUGH */
} else if (xxx) {
return yyy;
}
if (drain) {
zzz = drain;
}
After:
if (flags & CS_DRAIN) {
/* FALLTHROUGH */
} else if (xxx) {
return yyy;
} else {
if (drain) {
zzz = drain;
}
}
r363887:
Use the right factor when finding the best frequency and compare the
absolute value of the result.
Submitted by: kibab
r365395:
aw_clk_nm: fix incorrect use of abs()
abs() takes a (signed) int as input.
Instead, it was used with unsigned 64-bit integers.
So, add and use a new helper function to calculate a difference between
two uint64_t-s.
Rick Macklem [Mon, 21 Sep 2020 00:50:32 +0000 (00:50 +0000)]
MFC: r365703
Fix a case where the NFSv4.0 server might crash if delegations are enabled.
asomers@ reported a crash on an NFSv4.0 server with a backtrace of:
kdb_backtrace
vpanic
panic
nfsrv_docallback
nfsrv_checkgetattr
nfsrvd_getattr
nfsrvd_dorpc
nfssvc_program
svc_run_internal
svc_thread_start
fork_exit
fork_trampoline
where the panic message was "docallb", which indicates that a callback
was attempted when the ClientID is unconfirmed.
This would not normally occur, but it is possible to have an unconfirmed
ClientID structure with delegation structure(s) chained off it if the
client were to issue a SetClientID with the same "id" but different
"verifier" after acquiring delegations on the previously confirmed ClientID.
The bug appears to be that nfsrv_checkgetattr() failed to check for
this uncommon case of an unconfirmed ClientID with a delegation structure
that no longer refers to a delegation the client knows about.
This patch adds a check for this case, handling it as if no delegation
exists, which is the case when the above occurs.
Although difficult to reproduce, this change should avoid the panic().
MFC r365830: make it possible recovering broken GPT after some LBAs cut off
If pre-formatted device has GPT and a partition covering
last available LBAs and the device is attached using
a bridge reducing amount of LBAs, then it could be not enough
forcing GEOM to use primary GPT. Also, we should make it possible
to recover GPT and this requires either deleting or resizing the partition.
This change enables "gpart delete" and "gpart resize" commands
on corrupted GPT with following "gpart recover".
It still does not allow modifying corrupted GPT without
preliminary setting sysctl kern.geom.part.check_integrity=0
Alan Somers [Sat, 19 Sep 2020 19:48:15 +0000 (19:48 +0000)]
MFC r365391, r365415
r365391:
nsswitch.conf(5): recommend placing cache after files
When cache precedes files, and nscd is configured to allow negative caching,
commands like "pw groupadd" can fail. The sequence of events looks like:
1. A command like pkg(8) looks up the group, and finds it absent.
2. pkg invokes pw(8) to add the group
3. pkg queries the group, but nscd says it doesn't exist, since it has a
negative cache entry for that group.
See also: https://lists.freebsd.org/pipermail/freebsd-current/2012-January/031595.html
The first issue was lack of quoting around INSTALLFLAGS, which set it
incorrectly and produced an error on -M.
The second issue was that we weren't actually doing the install in
unprivileged mode, making it effectively useless. This was designed to pass
through the proper metalog/unpriv flags to install(1), so just let it
happen.
Allan Jude [Thu, 17 Sep 2020 21:51:05 +0000 (21:51 +0000)]
Update naming of per-dataset counters introduced in r365689
Upstream OpenZFS is changing the structure of the sysctl mibs to avoid
having a / in the element name. Chase this change here to avoid introducing
a new sysctl that will change named between 12.2 and 13.0
MFC-with: 365689
Relnotes: yes (fix example)
Sponsored by: Klara Inc.
MFC r365646, r365720:
r365646:
Enclose BRANCH_OVERRIDE in quotes in order to fix an issue with
freebsd-update(8) builds, where BRANCH is suffixed with -p0 for
builds.
r365720 (gordon):
Partially revert r346018 and use the if/then construct instead of
shell.
This would allow interested parties to do experimental runs with an
environment set appropriately to raise all the warnings throughout the
build; e.g. env WARNS=6 NO_WERROR=yes buildworld.
Not currently touching the numerous instances in ^/tools.
MFC r365493-r365494, r365600, r365602, r365637: various WARNS fixes
r365493:
libc/resolv: attempt to fix the test under WARNS=6
In a side-change that I'm working on to start defaulting src builds to
WARNS=6 where WARNS isn't otherwise specified, GCC6 (and clang, to a lesser
extent) pointed out a number of issues with the resolv tests:
- Global method variable that gets shadowed in run_tests()
- Signed/unsigned comparison between i in run_tests() and hosts->sl_cur
The shadowed variable looks like it might actually be bogus as written, as
we pass it to RUN_TESTS -> run_tests, but other parts use the global method
instead. This change is mainly geared towards correcting that by removing
the global and plumbing the method through from run_tests -> run into the
new thread.
For the signed/unsigned comparison, there's no compelling reason to not just
switch i/nthreads/nhosts to size_t.
The review also included a change to the load() function that was better
addressed by jhb in r365302.
r365494:
libc tests: dynthr_mod: fix some WARNS issues
This is being addressed as part of a side-patch I'm working on that builds
all the things with WARNS=6, instead of relying on it being supplied in just
shallow parts of the build with higher-level Makefile.inc.
Provide a prototype for mod_main and annotate the thread function argument
as unused.
r365600:
MFV r365599: import fix for a libexecinfo warning at higher WARNS
v1.17 of this file included a fix that I just submitted upstream to fix a
warning about prevent_inline with external linkage not having been
previously declared.
r365602:
librt: tests: fix minor issues with higher WARNS
got_sigalrm is a global with external linkage and must therefore have a
previous extern declaration. There's no reason to maintain the status quo
there, so just make it static.
The result var is unused.
This part of the test has not been upstreamed, presumably because it exists
solely for sem_clockwait_np. We should perhaps consider moving it into its
own test file outside of ^/contrib/netbsd-tests, but this can happen later.
r365637:
MFV r365636: libarchive: import fix for WARNS=6 builds in testing bits
Two more cases of explicitly marking globals for internal linkage where they
need not be shared. Committed upstream as of a38e62314a1f.
John Baldwin [Wed, 16 Sep 2020 22:55:27 +0000 (22:55 +0000)]
MFC 365276: Compute the correct size of the string to move forward.
Previously this was counting the amount of spare room at the start of
the buffer that the string needed to move forward and passing that as
the number of bytes to copy to memmove rather than the length of the
string to be copied.
In the strfmon test in the test suite this caused the memmove to
overflow the allocated buffer by one byte which CHERI caught.
Remove documentation of RELEASE_CRUNCH here. It's obsolete and no longer a
good example.
r364166:
Fix crunchgen usage of mkstemp()
On Glibc systems mkstemp can only be used once with the same template
string since it will be modified in-place and no longer contain any 'X'
chars. It is fine to reuse the same file here but we need to be explicit and
use open() instead of mkstemp() on the second use.
While touching this file also avoid a hardcoded /bin/pwd since that may not
work when building on non-FreeBSD systems.
r364174:
Use env pwd instead of pwd in crunchgen.c
In r364166 I changed /bin/pwd to pwd, but pwd can be shell builtin that
may not correctly return a real path. To ensure that all symlinks are
resolved use `env pwd -P` instead (the -P flag is part of POSIX so
should be supported everywhere).
r364234:
crunchgen: use pwd -P without env
The -P flag is required by POSIX so we don't have to care whether pwd is
a shell builtin or not. This also allows removing pwd from the list of
bootstrap tools since all shells we care about for building have a
builtin pwd command. This effectively reverts r364190.
r364646:
Re-indent crunched_main.c in preparation for D25998
r364647:
Correctly determine the real executable in crunched binaries
This should fix cases like su setting argv[0] to _su for /bin/sh.
Previously cheribsdbox (a crunched tool we use in CheriBSD to reduce the
size of our minimal disk images to allow loading them onto FPGAs without
waiting forever for the transfer) would complain about _su not being
compiled in, but now that we also look at AT_EXECPATH it correctly
invokes the sh tool.
Note: we use use AT_EXECPATH instead of the KERN_PROC_PATHNAME sysctl to get
the crunchgen binary name since it seems like KERN_PROC_PATHNAME just
returns the last cached path for a given hardlink.
When using `su`, instead of invoking /bin/csh this would invoke the last
used hardlink to cheribsdbox. This caused weird test failures when running
tests due to `id` being executed instead of `echo`:
$ id # id is a hardlink to /bin/cheribsdbox
$ su postgres -c 'echo 1' # su is also a hardlink
uid=1001(postgres) gid=1001(postgres) groups=1001(postgres)
r365605:
crunchgen(8): fix crunched application build with WARNS=6
This was revealed by the rescue build with a patch I'm working on to default
WARNS=6 everywhere. The issues resolved were:
- Missing prototype for _crunched_${ident}_stub in the *_stub.c generated
bits
- Missing prototype for crunched_main
- Incomplete prototype for _crunched_${ident}_stub in the generated parts of
crunched_main
- Literal strings in the stub table must drop const qualifier, unless we
const'ify name
- f field in struct stub didn't have a proper prototype
Most of these issues are minor formalities and easily addressed.
I note that if my patch to eventually raise WARNS for the rescue build
lands, we'll need to bump the __FreeBSD_version requirement for
bootstrapping crunchgen and wipe out the rescue .OBJDIR if it's stale, which
we should be able to detect pretty easily from a couple of the issues that
have been fixed here.
r365705:
__FreeBSD_version bump for r365605 (crunchgen producing WARNS-clean)
The change in D26397 will need a __FreeBSD_version to base off of for
bootstrapping crunchgen, to avoid avoidable build failures just because the
host has an outdated crunchgen.
o Enhance dependency loop logging: print full chain instead of the
last link competing the loop;
o Add -g option to generate dependency graph suitable for GraphViz
visualization, loops and other graph generation issues are highlighted
automatically;
o Add -p option that enables grouping items that can be processed in
parallel.
Submitted by: Boris Lytochkin <lytboris at gmail>
Differential Revision: https://reviews.freebsd.org/D25389
Allan Jude [Sun, 13 Sep 2020 16:28:25 +0000 (16:28 +0000)]
MFOpenZFS: Introduce read/write kstats per dataset
The following patch introduces a few statistics on reads and writes
grouped by dataset. These statistics are implemented as kstats
(backed by aggregate sums for performance) and can be retrieved by
using the dataset objset ID number. The motivation for this change is
to provide some preliminary analytics on dataset usage/performance.
Reviewed-by: Richard Elling <Richard.Elling@RichardElling.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Matthew Ahrens <mahrens@delphix.com> Signed-off-by: Serapheim Dimitropoulos <serapheim@delphix.com>
openzfs/zfs@a448a2557ec4938ed6944c7766fe0b8e6e5f6456
Also contains parts of:
MFOpenZFS: Connect dataset_kstats for FreeBSD
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed by: Sean Eric Fagan <sef@ixsystems.com> Reviewed-by: Serapheim Dimitropoulos <serapheim@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Allan Jude <allan@klarasystems.com>
openzfs/zfs@4547fc4e071ceb1818b3a46c3035b923e06e5390
This is a direct commit to stable/12 because they do not exist in
illumos upstream ZFS and needed to be heavily modified to work in
stable/12
getlogin_r is specified by POSIX to to take a size_t len, not int. Fix our
version to do the same, bump the symbol version due to ABI change and
provide compat.
This was reported to break compilation of Ruby 2.8.
Some discussion about the necessity of the ABI compat did take place in the
review. While many 64-bit platforms would likely be passing it in a 64-bit
register and zero-extended and thus, not notice ABI breakage, some do
sign-extend (e.g. mips).
MFC r365500: certctl: fix hashed link generation with duplicate subjects
Currently, certctl rehash will just keep clobbering .0 rather than
incrementing the suffix upon encountering a duplicate. Do this, and do it
for blacklisted certs as well.
This also improves the situation with the blacklist to be a little less
flakey, comparing cert fingerprints for all certs with a matching subject
hash in the blacklist to determine if the cert we're looking at can be
installed.
Future work needs to completely revamp the blacklist to align more with how
it's described in PR 246614. In particular, /etc/ssl/blacklisted should go
away to avoid potential confusion -- OpenSSL will not read it, it's
basically certctl internal.
Follow-up r365662 (MFC of r365371 and r365373) by also removing the
header hack from jemalloc_FreeBSD.h, which rendered any make.conf
MALLOC_PRODUCTION or src.conf WITH/WITHOUT_MALLOC_PRODUCTION irrelevant.
Direct commit to stable/{11,12} as this does not apply to head.
Follow-up r365662 (MFC of r365371 and r365373) by correctly setting
WITH_MALLOC_PRODUCTION for stable branches. Also add a note to UPDATING,
to inform users about the new setting.
Direct commit to stable/{11,12} as this does not apply to head.
Noticed by: imp, Ronald Klop <ronald-lists@klop.ws>
Turn MALLOC_PRODUCTION into a regular src.conf(5) option
For historical reasons, defining MALLOC_PRODUCTION in /etc/make.conf has
been used to turn off potentially expensive debug checks and statistics
gathering in the implementation of malloc(3).
It seems more consistent to turn this into a regular src.conf(5) option,
e.g. WITH_MALLOC_PRODUCTION / WITHOUT_MALLOC_PRODUCTION. This can then
be toggled similar to any other source build option, and turned on or
off by default for e.g. stable branches.
There have been several mentions on our mailing lists about missing
atomic functions in our system libraries (e.g. __atomic_load_8 and
friends), and recently I saw __bswapdi2 and __bswapsi2 mentioned too.
To address this, add implementations for the functions from compiler-rt
to the system compiler support libraries, e.g. libcompiler_rt.a and and
libgcc_s.so.
This also needs a small fixup in compiler-rt's atomic.c, to ensure that
32-bit mips can build correctly.
Bump __FreeBSD_version to make it easier for port maintainers to detect
when these functions were added.
After r364753, there should be no need to suppress -Watomic-alignment
warnings anymore for compiler-rt's atomic.c. This occurred because the
IS_LOCK_FREE_8 macro was not correctly defined to 0 for mips, and this
caused the compiler to emit a runtime call to __atomic_is_lock_free(),
and that triggers the warning.
MFC r365509:
Follow-up r364753 by enabling compiler-rt's atomic implementation only
for clang, as it uses clang specific builtins, and does not compile
correctly with gcc. Note that gcc packages usually come with their own
libatomic, providing these primitives.
MFC r365588:
Follow-up r364753 by only using arm's stdatomic.c implementation, as it
already covers the functions in compiler-rt's atomic.c, leading to
conflicts when linking.
There's a race where dying vnets move their interfaces back to their original
vnet, and if_epair cleanup (where deleting one interface also deletes the other
end of the epair). This is commonly triggered by the pf tests, but also by
cleanup of vnet jails.
As we've not yet been able to fix the root cause of the issue work around the
panic by not dereferencing a NULL softc in epair_qflush() and by not
re-attaching DYING interfaces.
This isn't a full fix, but makes a very common panic far less likely.
This should have been MFC'd as part of r353800, but was not because caroot
had not been MFC'd at that time. As such, this is a direct commit to
stable/12.
MFC r365490-r365491: Remove FREEBSD_UPDATE dependency on PORTSNAP
r365490:
phttpget: move out of portsnap
Currently, WITHOUT_PORTSNAP forces WITHOUT_FREEBSD_UPDATE because the
latter relies on phttpget, which lives inside the portsnap build bits.
Remove the dependency between these two options by moving phttpget out into
^/libexec and building/installing it if either WITH_PORTSNAP or
WITH_FREEBSD_UPDATE.
Future work could remove the conditional if it's decided that users will use
it independently of either the current in-base consumers.
r365491:
opts: FREEBSD_UPDATE no longer relies on PORTSNAP
phttpget is no longer tied to the portsnap build as of r365490.
Alexander Motin [Fri, 11 Sep 2020 14:00:10 +0000 (14:00 +0000)]
MFC r342852 (by cem): powerpc: Fix regression introduced in r342771
In r342771, I introduced a regression in Power by abusing the platform
smp_topo() method as a shortcut for providing the MI information needed for
the stated sysctls. The smp_topo() method was already called later by
sched_ule (under the name cpu_topo()), and initializes a static array of
scheduler topology information. I had skimmed the smp_topo_foo() functions
and assumed they were idempotent; empirically, they are not (or at least,
detect re-initialization and panic).
Do the cleaner thing I should have done in the first place and add a
platform method specifically for core- and thread-count probing.
Allan Jude [Thu, 10 Sep 2020 21:01:22 +0000 (21:01 +0000)]
MFC r360229, r363255
r360229:
Add VIRTIO_BLK_T_DISCARD (TRIM) support to the bhyve virtio-blk backend
This will advertise support for TRIM to the guest virtio-blk driver and
perform the DIOCGDELETE ioctl on the backing storage if it supports it.
Thanks to Jason King and others at Joyent and illumos for expanding on
my original patch, adding improvements including better error handling
and making sure to following the virtio spec.
r363255:
Add VIRTIO_BLK_T_DISCARD support to the virtio-blk driver
If the hypervisor advertises support for the DISCARD command then the
guest can perform TRIM commands, freeing space on the backing store.
If VIRTIO_BLK_F_DISCARD is enabled, advertise DISKFLAG_CANDELETE
Eric Joyner [Thu, 10 Sep 2020 20:46:16 +0000 (20:46 +0000)]
MFC r361541, r362038, r364240
These MFCs add the ice(4) driver to the kernel for Intel 800 Series
Ethernet adapters, a couple fixes for the ice_ddp module makefile, and
remove some redeclarations, respectively.
John Baldwin [Thu, 10 Sep 2020 20:34:44 +0000 (20:34 +0000)]
MFC 363459:
Pass the right size to memcpy() when copying the array of FP registers.
The size of the containing structure was passed instead of the size of
the array. This happened to be harmless as the extra word copied is
one we copy in the next line anyway.
Merge WiFi net80211, drivers, and management in order to support better 11n
and upcoming 11ac.
This includes an ath(4) update, some run(4) 11n support, 11n for otus(4),
A-MPDU, A-MSDU, A-MPDU+A-MSDU and Fast frames options, scanning fixes,
enahnced PRIV checks for jails, restored parent device name printing,
improvements for upcoming VHT support, lots of under-the-hood infrastructure
improvements, new device ID, debug tools updates, some whistespace changes
(to make future MFCs easier).
This does not include (nost) epoch(9) related changes as too much other
infrastructure was not merged for that.
Bump __FreeBSD_veresion as this changes the priv(9) names (not know to be
used externally), and net80211 structures.
Tested on: some ath(4) AP, run(4) STA, and rtwn(4) STA
Discussed with: adrian (extremly briefly)
Sponsored by: Rubicon Communications, LLC (d/b/a "Netgate") [partially]
Relnotes: yes
Michael Tuexen [Thu, 10 Sep 2020 18:04:34 +0000 (18:04 +0000)]
MFC r350061:
Fix compilation on platforms using gcc.
When compiling RACK on platforms using gcc, a warning that tcp_outflags
is defined but not used is issued and terminates compilation on PPC64,
for example. So don't indicate that tcp_outflags is used.
Michael Tuexen [Thu, 10 Sep 2020 17:41:23 +0000 (17:41 +0000)]
MFC r352661 (from rrs):
lets put (void) in a couple of functions to keep older platforms that
are stuck with gcc happy (ppc). The changes are needed in both bbr and
rack.
Michael Tuexen [Thu, 10 Sep 2020 17:31:34 +0000 (17:31 +0000)]
MFC r351782:
Fix two TCP RACK issues:
* Convert the TCP delayed ACK timer from ms to ticks as required.
This fixes the timer on platforms with hz != 1000.
* Don't delay acknowledgements which report duplicate data using
DSACKs.
Michael Tuexen [Thu, 10 Sep 2020 17:29:20 +0000 (17:29 +0000)]
MFC r351328 (by rrs):
Fix an issue when TSO and Rack play together. Basically
an retransmission of the initial SYN (with data) would
cause us to strip the SYN and decrement/increase offset/len
which then caused us a -1 offset and a panic.
Michael Tuexen [Thu, 10 Sep 2020 17:26:16 +0000 (17:26 +0000)]
MFC r350973 (from rrs):
Place back in the dependency on HPTS via module depends versus
a fatal error in compiling. This was taken out by mistake
when I mis-merged from the 18q22p2 sources of rack in NF. Opps.
Michael Tuexen [Thu, 10 Sep 2020 17:12:42 +0000 (17:12 +0000)]
MFC r364754:
RFC 3465 defines a limit L used in TCP slow start for limiting the number
of acked bytes as described in Section 2.2 of that document.
This patch ensures that this limit is not also applied in congestion
avoidance. Applying this limit also in congestion avoidance can result in
using less bandwidth than allowed.
Michael Tuexen [Thu, 10 Sep 2020 16:59:54 +0000 (16:59 +0000)]
MFC r357100:
The server side of TCP fast open relies on the delayed ACK timer to allow
including user data in the SYN-ACK. When DSACK support was added in
r347382, an immediate ACK was sent even for the received SYN with
user data. This patch fixes that and allows again to send user data with
the SYN-ACK.
Michael Tuexen [Thu, 10 Sep 2020 16:47:08 +0000 (16:47 +0000)]
MFC r358023:
Don't use uninitialised stack memory if the sysctl variable
net.inet.tcp.hostcache.enable is set to 0.
The bug resulted in using possibly a too small MSS value or wrong
initial retransmission timer settings. Possibly the value used
for ssthresh was also wrong.
projects / FreeBSD / FreeBSD.git / log