Simon J. Gerraty [Sat, 22 Apr 2023 22:07:53 +0000 (15:07 -0700)]
Extract some of local.meta.sys.mk to local.meta.sys.env.mk
Latest meta.sys.mk follows sys.mk in including local.meta.sys.env.mk
and local.meta.sys.mk late, but we have things we need done early
so move them to local.meta.sys.env.mk
Simon J. Gerraty [Sat, 22 Apr 2023 19:01:49 +0000 (12:01 -0700)]
Simplify building host tools during DIRDEPS_BUILD
The whole point of the DIRDEPS_BUILD is to avoid tree walks
and basically build everything in a single pass.
We use the pseudo MACHINE "host" to represent the build host.
When the build host is not FreeBSD or is an older version of FreeBSD
it may need some help to build host-tools.
The directory tools/build does this - building libegacy.
local.sys.mk: create a pseudo option MK_host_egacy to indicate
if tools/build needs to be built for "host".
local.dirdeps.mk: set MK_host_egacy.host to ${MK_host_egacy}
all other DEP_MACHINES will get "no"
This allows a Makefile.depend.options in makefs etc to cause tools/build
to be built for host but only if necessary.
local.init.mk: use ISYSTEM as arg to -isystem so that it can be overridden.
The default remains ${STAGE_INCLUDEDIR}
src.init.mk: if MACHINE is host and we are not FreeBSD
set some MK_ flags the same as tools/build/mk/Makefile.boot.pre and
include src.init.${.MAKE.OS:tl}.mk if it exists.
For older versions of FreeBSD add libegacy when building PROGs for "host"
Also instead of -isystem${STAGE_INCLUDEDIR} we want
-I${STAGE_INCLUDEDIR} and -isystem/usr/include so we override ISYSTEM.
This means any headers we stage for "host" will take precedence over
system headers but #include_next will DTRT.
src.init.linux.mk: add
-I${SRCTOP}/tools/build/cross-build/include/linux
and generally deal with building host tools on Linux.
Eg. static linking does not work so set NO_SHARED= no
Override some HAVE_ flags.
src.sys.env.mk: on linux awk throws an warning about # in newvers.sh
just send stderr to /dev/null
The powerpc asm from openzfs assumes that big-endian is always ELFv1 and
ELFv2 is always little-endian, while FreeBSD uses ELFv2 everywhere. Add
the necessary bits to the checksum asm to work on big-endian ELFv2.
Ed Maste [Fri, 21 Apr 2023 13:11:45 +0000 (09:11 -0400)]
llvm: fix some llvm-* tool links when WITH_LLVM_BINUTILS is set
A few Makefiles overrode LINKS and MLINKS when WITH_LLVM_BINUTILS was
set, which caused some llvm- prefixed tools to disappear. One such case
was llvm-ranlib, which some ports invoke explicitly.
Use += when adding to LINKS and MLINKS under WITH_LLVM_BINUTILS.
PR: 270955, 270956, 270959
Submitted by: jbeich
Reviewed by: arichardson Fixes: 021385aba562 ("Add WITH_LLVM_BINUTILS to install LLVM binuti...")
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39747
Warner Losh [Fri, 21 Apr 2023 16:14:41 +0000 (10:14 -0600)]
newvers: Use correct regexp
There's no need to quote the # here. Inside of regexp, it's not treated
like a comment from an awk perspective. And inside if '' it's not
treated as special by the shell. gawk also warns.
Randall Stewart [Fri, 21 Apr 2023 11:12:25 +0000 (07:12 -0400)]
tcp: hpts needs to still call output even after input.
The other stacks it turns out actually expect the output to be called and can become stuck if it is
not. This is because they run there timer code from there and the input routine does not always
assure a timer is running. The real longterm fix here might be to go into the other stacks (rack and bbr)
and make sure that a timer is running after input if you don't do output.. as well as call the timer functions.
This would cut down on calls from hpts. But I think its too dramatic of a change for the immediate time.
* Align 'on <interface>' parameter with the BNF, so use 'on <ifspec>'
* Clarify etherprotospec BNF, to make it clearer that only numbers are
supported.
Suggested by: Christian McDonald
Sponsored by: Rubicon Communications, LLC ("Netgate")
Before the commit 6cc44223cb6717795afdac4348bbe7e2a968a07d the
field event_mask was fully copied to the EventMasks field.
After this commit the event_mask (uint8_t) is 4 times casted to
EventMask (uint32_t). Because of that 24 bits of each event_mask array
is lost.
This commits brings back simple copying of field, and after words
converting 32 bits field to the requested endian.
I don't think we need more sophisticated method,
as the array is of size 4 (for 32 bits version).
Commit 3e0856b63fe0e375a0951e05c2ef98bb2ebd9421 updated
__sg_alloc_table_from_pages to use the same API as linux, but modified
the loop condition when going over the pages in a sg list. Part of the
change included moving the sg_next call out of the for loop and into the
body, which causes an off by one error when traversing the list. Since
sg_next is called before the loop body it will skip the first element
and read one past the last element.
This caused panics when running PRIME with nvidia-drm as the off-by-one
issue causes a NULL dereference.
Reviewed by: bz, hselasky
Differential Revision: https://reviews.freebsd.org/D39628
Fixes: 3e0856b63fe0 ("linuxkpi: Fix `sg_alloc_table_from_pages()` to have the same API as Linux")
Warner Losh [Fri, 21 Apr 2023 05:05:30 +0000 (23:05 -0600)]
syscall.master: Remove stray 4.2
Back in 4.3BSD, the system call table wasn't generated, and there was an
entry:
"4.2 sigreturn", /* 139 = old 4.2 sigreturn */
This got converted to
139 OBSOL 0 4.2 sigreturn
in 4.3 RENO. Since it was obsolete, nothing bad happened. In fact,
there was code in makeyscalls.sh to cope:
{ comment = $4
for (i = 5; i <= NF; i++)
comment = comment " " $i
if (NF < 5)
$5 = $4
}
so the generated comment in syscalls.c was almost correct:
"obs_4.2", /* 139 = obsolete 4.2 sigreturn */
a bug that we have to this very day, despite makesyscalls.sh being
rewritten in lua.
However, this historical wart is the only place in our current
syscalls.master file where we have an extra field for the 'not
generated' class of system calls. Remove the historical wart so that the
re-write of makesyscalls.lua can be simpler (so, I hope, qemu's bsd-user
can large swathes of code automatically generated too). This should help
make things more understandable (changes to simplify makesyscalls.lue
aren't quite debugged, so have to wait for another day).
There's 3 different obsolete sigreturns (but only 1 that was ever in
FreeBSD 2.x and newer).
Simon J. Gerraty [Fri, 21 Apr 2023 05:00:40 +0000 (22:00 -0700)]
bsdinstall avoid subdir depending on parent
When not doing tree walks, it is bad for sub-dirs to depend on
parents. Move the generation of opt_osname.h to distextract
and have others that need that depend on it.
In usr.sbin/bsdinstall use SUBDIR_DEPEND_ so tree walking still works.
These are the changes since the last update (copy-pasted from the
release notes for Chelsio Unified Wire v3.18.0.0):
====================
Version : 1.27.3.0
Date : 04/07/2023
Fixes
-----
BASE:
- Fixed a hang if module eeprom reads gives invalid data.
- KR backlplane no-fec link problem fixed.
OFLD:
- iscsi ddp errors fixed.
- iwarp connection abort in rare cases causing NIC traffic hang fixed.
ENHANCEMENTS
------------
BASE:
- Cisco GLC-TE 1G modules support added.
====================
Version : 1.27.1.0
Date : 12/02/2022
Fixes
-----
BASE:
- memwrite dsgl cannot be used for T5.
OFLD:
- Enabled FCoE in SO adapters.
- TOE-TLS crash fixed.
- iscsi hang fixed.
Simon J. Gerraty [Fri, 21 Apr 2023 03:38:53 +0000 (20:38 -0700)]
Update meta.sys.mk and related local*mk
Move the setting of TARGET_SPEC_VARS to local.sys.env.mk
so meta.sys.mk can do the processing, and include local.meta.sys.mk
later.
Move the setting of GENDIRDEPS_FILTER*_VARS from local.gendirdeps.mk
to local.meta.sys.mk so we can automatically set DEP_* at level 1+
to avoid syntax errors when DEP_* variables are used in conditionals
in Makefile.depend files.
Update gendirdeps.mk just to get the documentation about the above.
No functional change.
local.dirdeps.mk be more careful about adding to DIRDEPS to avoid
unnecessary overhead, and introducing cycles in the graph.
Also set DEP_MACHINE_CPUARCH.
Warner Losh [Thu, 20 Apr 2023 22:16:21 +0000 (16:16 -0600)]
syscall.master: Fix comments
Have more accruate comments. While #if, #else, etc are copied to the
header files, lines that don't start with # are not. And #include files
are only output to sysinc (which winds up at the front of init_sysent.c
which seems a bit odd). This is all radically undocumented, and likely
has drifted somewhat from 4.4BSD and what other systems do (they've
drifted too, fwiw).
This change touches both kernel and netstat(1), but either of the changes
will fix printing pcb addresses with -A.
The thing is that historically netstat(1) treated TCP differently, and
printed tcpcb address instead of inpcb address. This is not documented
anywhere! With e68b3792440 these two addresses became the same. It is
highly likely they will be the same for a long time, but it might be they
will start to differ again in a far future. My proposal is to stop
treating TCP differently with netstat(1) and right now is a good opportunity
to do that, since there will be no behavior change at all. The kernel
change to tcp_inptoxtp() will go into stable/14 to make it compatible with
netstat(1) binary from stable/13. We can drop it later, probably together
with in_ppcb pointer from inpcb. The in_ppcb in xinpcb will stay for size
compatibility.
Bjoern A. Zeeb [Tue, 14 Feb 2023 15:53:13 +0000 (15:53 +0000)]
dpaa2: add console support for FDT based systems
Add DPAA2 console support for MC and AIOP (latter untested) for FDT
systems. ACPI systems are prepared but need some proper bus function
in order to get the address from MC (and likely a file splitup then).
This will come at a later stage once other ACPI/FDT bus parts are
cleared up.
The work was originally done in July 2022 and finally switched to
bus_space[1] lately to be ready for main.
Revert "libc: Implement bsort(3) a bitonic type of sorting algorithm."
Some points for the future:
- libc is not the right place for sorting algorithms.
Probably libutil is better suited for this purpose or
a dedicated libsort. Should move all sorting algorithms
away from libc eventually.
- CheriBSD uses capabilities for memory access, and could
benefit from a standard memswap() function.
- Do something about qsort() in FreeBSD's libc like:
- Mark it deprecated on FreeBSD, as a first step,
due to missing limits on CPU time.
- Audit the use of qsort() in the FreeBSD base system
and consider swapping to other existing sorting
algorithms.
Simon J. Gerraty [Thu, 20 Apr 2023 17:05:43 +0000 (10:05 -0700)]
Fix building host tools for host
Several makefile depend on tools built for host.
At least when using DIRDEPS_BUILD we can build these for the
pseudo machine "host" to facilitate building on older host versions.
Ideally we would build these tools in their own directories to avoid
building more than needed.
For now, setting an appropriate default for BTOOLSPATH will suffice
Ed Maste [Tue, 18 Apr 2023 13:57:29 +0000 (09:57 -0400)]
makefs: set cd9660 Rock Ridge timestamps for . and ..
DOT and DOTDOT entries have special handling, and previously only Rock
Ridge PX (POSIX attributes) entries were attached. Add TF (timestamp)
entries as well.
PR: 203531
Reported by: Thomas Schmitt <scdbackup@gmx.net>
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39662
Mark Johnston [Thu, 20 Apr 2023 15:48:33 +0000 (11:48 -0400)]
inpcb: Release the inpcb cred reference before freeing the structure
Now that the inp_cred pointer is accessed only while the inpcb lock is
held, we can avoid deferring a crfree() call when freeing an inpcb.
This fixes a problem introduced when inpcb hash tables started being
synchronized with SMR: the credential reference previously could not be
released until all lockless readers have drained, and there is no
mechanism to explicitly purge cached, freed UMA items. Thus, ucred
references could linger indefinitely, and since ucreds hold a jail
reference, the jail would linger indefinitely as well. This manifests
as jails getting stuck in the DYING state.
Mark Johnston [Thu, 20 Apr 2023 15:48:19 +0000 (11:48 -0400)]
inpcb: Avoid inp_cred dereferences in SMR-protected lookup
The SMR-protected inpcb lookup algorithm currently has to check whether
a matching inpcb belongs to a jail, in order to prioritize jailed
bound sockets. To do this it has to maintain a ucred reference, and for
this to be safe, the reference can't be released until the UMA
destructor is called, and this will not happen within any bounded time
period.
Changing SMR to periodically recycle garbage is not trivial. Instead,
let's implement SMR-synchronized lookup without needing to dereference
inp_cred. This will allow the inpcb code to free the inp_cred reference
immediately when a PCB is freed, ensuring that ucred (and thus jail)
references are released promptly.
Commit 220d89212943 ("inpcb: immediately return matching pcb on lookup")
gets us part of the way there. This patch goes further to handle
lookups of unconnected sockets. Here, the strategy is to maintain a
well-defined order of items within a hash chain so that a wild lookup
can simply return the first match and preserve existing semantics. This
makes insertion of listening sockets more complicated in order to make
lookup simpler, which seems like the right tradeoff anyway given that
bind() is already a fairly expensive operation and lookups are more
common.
In particular, when inserting an unconnected socket, in_pcbinhash() now
keeps the following ordering:
- jailed sockets before non-jailed sockets,
- specified local addresses before unspecified local addresses.
Most of the change adds a separate SMR-based lookup path for inpcb hash
lookups. When a match is found, we try to lock the inpcb and
re-validate its connection info. In the common case, this works well
and we can simply return the inpcb. If this fails, typically because
something is concurrently modifying the inpcb, we go to the slow path,
which performs a serialized lookup.
Note, I did not touch lbgroup lookup, since there the credential
reference is formally synchronized by net_epoch, not SMR. In
particular, lbgroups are rarely allocated or freed.
I think it is possible to simplify in_pcblookup_hash_wild_locked() now,
but I didn't do it in this patch.
Mark Johnston [Thu, 20 Apr 2023 15:48:01 +0000 (11:48 -0400)]
inpcb: Split PCB hash tables
Currently we use a single hash table per PCB database for connected and
bound PCBs. Since we started using net_epoch to synchronize hash table
lookups, there's been a bug, noted in a comment above in_pcbrehash():
connecting a socket can cause an inpcb to move between hash chains, and
this can cause a concurrent lookup to follow the wrong linkage pointers.
I believe this could cause rare, spurious ECONNREFUSED errors in the
worse case.
Address the problem by introducing a second hash table and adding more
linkage pointers to struct inpcb. Now the database has one table each
for connected and unconnected sockets.
When inserting an inpcb into the hash table, in_pcbinhash() now looks at
the foreign address of the inpcb to figure out which table to use. This
ensures that queue linkage pointers are stable until the socket is
disconnected, so the problem described above goes away. There is also a
small benefit in that in_pcblookup_*() can now search just one of the
two possible hash buckets.
I also made the "rehash" parameter of in(6)_pcbconnect() unused. This
parameter seems confusing and it is simpler to let the inpcb code figure
out what to do using the existing INP_INHASHLIST flag.
UDP sockets pose a special problem since they can be connected and
disconnected multiple times during their lifecycle. To handle this, the
patch plugs a hole in the inpcb structure and uses it to store an SMR
sequence number. When an inpcb is disconnected - an operation which
requires the global PCB database hash lock - the write sequence number
is advanced, and in order to reconnect, the connecting thread must wait
for readers to drain before reusing the inpcb's hash chain linkage
pointers.
raw_ip (ab)uses the hash table without using the corresponding
accessors. Since there are now two hash tables, it arbitrarily uses the
"connected" table for all of its PCBs. This will be addressed in some
way in the future.
inp interators which specify a hash bucket will only visit connected
PCBs. This is not really correct, but nothing in the tree uses that
functionality except raw_ip, which as mentioned above places all of its
PCBs in the "connected" table and so is unaffected.
Move a KASSERT out of a function and make it a CTASSERT with
appropriate comments.
Skeleton implement two tkip functions, still left TODO, initializing
variables with dummy values to quiten compiler warnings. It is
unclear to me if we should still ever properly implement TKIP
compat code at this point. If so the current code gives a good
idea what needs to be done in addition to allocating references
to real state along with keyconf.
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Quieten some more (valid) gcc warnings and disable dead code.
There are more warnings, some probably a compiler problem, the
other related to firmware structs which I do not want to adjust
just locally. Leave a comment to revisit after a next driver
update.
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
network.subr: adjust regex for wlans_xxxxx rc.conf entries
Drivers like ath1[012]k will not match the current wlans_*-regex as
they have digits followed by letters. Adjust the regex to allow
this combination in order to be able to configure interfaces with
names like wlans_ath11k0="..."
MFC after: 3 days
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D39674
Rather than using ACCESS_ONCE() in READ_ONCE() add a missing cast
to const in order to satisfy -Wcast-equal by gcc.
Sadly we cannot do the same to WRITE_ONCE() which still is very
noisy.
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Reviewed by: hselasky
Differential Revision: https://reviews.freebsd.org/D39706
Harmonize sk_buff_head and sk_buff further and fix -Warray-bounds
warnings reports by gcc. At the same time simplify some code by
re-using other functions or factoring some code out.
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
A one-bit wide bit-field can take only the values 0 and -1. Clang 16
introduced a warning that "implicit truncation from 'int' to a one-bit
wide bit-field changes value from 1 to -1". Fix by using c99 bool.
Randall Stewart [Wed, 19 Apr 2023 18:02:12 +0000 (14:02 -0400)]
tcp: rack the request level logging is a bit too noisy when doing point logging.
When doing request level BB logging the hybrid_bw_log() does not have proper screening to minimize logging
when point level logging is in use. Lets fix it properly so you have to have the proper knobs set to get the
more noisy logging.
loader: Change version calculation to be more consistent.
Use 1000 * major + minor when calculating the version number that
gets set in the Ficl environment or lua loader property. This allows
for more room if the minor number needs to go above 9.
Randall Stewart [Wed, 19 Apr 2023 17:17:04 +0000 (13:17 -0400)]
tcp: Rack can crash with the new non-TSO fix..
Turns out the location of the check to see if we can do output is in the wrong place. We need
to jump off to the compressed acks before handling that case since th is NULL in the
compressed ack case which is handled differently anyway.
Randall Stewart [Wed, 19 Apr 2023 16:54:25 +0000 (12:54 -0400)]
We have a TCP_LOG_CONNEND log that should come out at the very last log of every connection. This
holds some nice stats about why/how the connection ended. Though with the current code it does not
come out without accounting due to the placement of the ifdefs. Also we need to make sure the stacks
fini has ran before calling in from tcp_subr so we get all logs the stack may make at its ending.
cxgbe/iw_cxgbe: debug routines to dump STAG (steering tag) entries.
t4_dump_stag to dump hw state for a known STAG.
t4_dump_all_stag to dump hw state for all valid STAGs. This routine
walks the entire STAG region looking for valid entries and this can take
a while for some configurations.
Brooks Davis [Wed, 19 Apr 2023 15:58:46 +0000 (16:58 +0100)]
Cirrus-CI: Check that make sysent was run
Run the `make sysent` target and verify that the repo isn't modified
afterwards. This ensures that a pushed branch contains all the
required bits after a change to syscall.master.