Test prioritisation and dummynet queues.
We need to give the pipe sufficient bandwidth for dummynet to work.
Given that we can't rely on the TCP connection failing alltogether, but
we can measure the effect of dummynet by imposing a time limit on a
larger data transfer.
If TCP is prioritised it'll get most of the pipe bandwidth and easily
manage to transfer the data in 3 seconds or less. When not prioritised
this will not succeed.
Kristof Provost [Tue, 25 May 2021 14:54:32 +0000 (16:54 +0200)]
ipfw: Introduce dnctl
Introduce a link to the ipfw command, dnctl, for dummynet configuration.
dnctl only handles dummynet configuration, and is part of the effort to
support dummynet in pf.
/sbin/ipfw continues to accept pipe, queue and sched commands, but these can
now also be issued via the new dnctl command.
Mark Johnston [Tue, 7 Sep 2021 13:44:57 +0000 (09:44 -0400)]
sctp: Fix iterator synchronization in sctp_sendall()
- The SCTP_PCB_FLAGS_SND_ITERATOR_UP check was racy, since two threads
could observe that the flag is not set and then both set it. I'm not
sure if this is actually a problem in practice, i.e., maybe there's no
problem having multiple sends for a single PCB in the iterator list?
- sctp_sendall() was modifying sctp_flags without the inp lock held.
The change simply acquires the PCB write lock before toggling the flag,
fixing both problems.
Reviewed by: tuexen
Sponsored by: The FreeBSD Foundation
Mark Johnston [Tue, 7 Sep 2021 13:44:12 +0000 (09:44 -0400)]
sctp: Fix races around sctp_inpcb_free()
sctp_close() and sctp_abort() disassociate the PCB from its socket.
As a part of this, they attempt to free the PCB, which may end up
lingering. Fix some bugs in this area:
- For some reason, sctp_close() and sctp_abort() set
SCTP_PCB_FLAGS_SOCKET_GONE using an atomic compare-and-set without the
PCB lock held. This is racy since sctp_flags is normally updated
without atomics, using the PCB lock to synchronize. So, the update
can be lost, which can cause all sort of races with other SCTP
components which look for the _GONE flag. Fix the problem simply by
acquiring the PCB lock in order to set the flag. Note that we have to
drop and re-acquire the lock again in sctp_inpcb_free(), but I don't
see a good way around that for now. If it's a real problem, the _GONE
flag could be split out of sctp_flags and into a dedicated sctp_inpcb
field.
- In sctp_inpcb_free(), load sctp_socket after acquiring the PCB lock,
to avoid possible races with parallel sctp_inpcb_free() calls.
- Add an assertion sctp_inpcb_free() to verify that _ALLGONE is not set.
Reviewed by: tuexen
Sponsored by: The FreeBSD Foundation
Mark Johnston [Fri, 16 Jul 2021 02:35:28 +0000 (22:35 -0400)]
libc: Use the initial-exec TLS model
This permits more efficient accesses of thread-local variables, which
are heavily used at least by jemalloc and locale-aware code. Note that
on amd64 and i386, jemalloc's thread-local variables already have their
TLS model overridden by defining JEMALLOC_TLS_MODEL.
For now the change is applied only to tested platforms, but should in
principle be enabled everywhere.
PR: 255840
Suggested by: jrtc27
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Mark Johnston [Tue, 7 Sep 2021 13:46:58 +0000 (09:46 -0400)]
g_label: Handle small sector sizes when tasting
Make sure that the provider sector size is large enough to contain a
valid label before trying to read it. We performed this check already
for most label types, but not for several filesystem labels.
Reported by: syzbot+f52918174cdf193ae29c@syzkaller.appspotmail.com
Sponsored by: The FreeBSD Foundation
Guinan Sun [Thu, 9 Jul 2020 08:00:33 +0000 (08:00 +0000)]
ixgbe: cleanup spelling mistakes in comments
Several functions in the driver code have a weird function comment
formatting which uses two spaces instead of only one space for the main
function body.
This formatting will be mechanically fixed by sed in a future patch, but
doing so leads to some spelling warnings on that patch. Cleanup the
spelling mistakes that will be detected first. This way, it is easier to
verify the mechanical transformation done by sed in the following patch.
Guinan Sun [Thu, 9 Jul 2020 08:00:35 +0000 (08:00 +0000)]
ixgbe: move increments after evaluations
The retry variable was being incremented before it was evaluated by the
subsequent conditional against the maximum retries to figure out which
message to print. So we'll move the increment op to the end.
Xiaoyun Li [Mon, 17 Sep 2018 09:28:12 +0000 (17:28 +0800)]
ixgbe: update X550 SFP identification
Use ixgbe_identify_sfp_module_X550em to update SFP identification
flow. ixgbe_identify_sfp_module_X550em includes specific checks for
X550 about supported SFP modules.
Ashijeet Acharya [Fri, 24 May 2019 09:30:15 +0000 (11:30 +0200)]
ixgbe: wait for link after copper MAC setup
After setting up the link on x552/X557-AT 10GBASE-T NICs, sometimes the
link does not get set up properly and as a result all the subsequent
calls to ixgbe_check_link() from ixgbe_dev_link_update_share() fail.
Introduce a delay time of 1s in ixgbe_setup_mac_link_t_X550em() before
beginning to set up the external PHY link speed to ensure that the
controller can acquire the link.
Guinan Sun [Thu, 9 Jul 2020 08:00:40 +0000 (08:00 +0000)]
ixgbe: initialize data field in struct buffer
While sending request using ixgbe_hic_unlocked() the data field in
buffer struct is not used. It is set when the struct is overwritten by
FW to deliver the response. To not pass random data to FW the whole
structure should be zeroed before use.
Signed-off-by: Krzysztof Galazka <krzysztof.galazka@intel.com> Signed-off-by: Piotr Pietruszewski <piotr.pietruszewski@intel.com> Signed-off-by: Guinan Sun <guinanx.sun@intel.com> Reviewed-by: Wei Zhao <wei.zhao1@intel.com>
Approved by: imp
Obtained from: DPDK (40543be5376ca415b2a7e196315d0555725b8bdf)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D31621
Guinan Sun [Thu, 9 Jul 2020 08:00:30 +0000 (08:00 +0000)]
ixgbe: fix x550em 10G NIC link status
With the NVM image for x550em XFI will not report
the auto-negotiation feature correctly. The auto-negotiation
should be "No" for supports and advertised items.
At the same time update speed makes it support 1G and 10G.
Matthew Smith [Mon, 16 Jul 2018 16:36:04 +0000 (11:36 -0500)]
ixgbe: wait longer for link after fiber MAC setup
After setting up the link on a fiber port, the maximum wait time for
the link to come up is 500 ms in ixgbe_setup_mac_link_multispeed_fiber().
On an x550 SFP+ port, this is often not sufficiently long for the link
to come up. This can result in never being able to retrieve accurate
link status for the port using rte_eth_link_get_nowait().
Increase the maximum wait time in ixgbe_setup_mac_link_multispeed_fiber()
to 1 s.
Guinan Sun [Thu, 9 Jul 2020 08:00:41 +0000 (08:00 +0000)]
ixgbe: improve log about autoneg being disabled
On ESXi OS, when user disables auto negotiation, the following log
appears: "(unsupported) Flow control autoneg is disabled".
It is true that auto negotiation is disabled but it is
not necessarily true that it is not supported.
Guinan Sun [Thu, 9 Jul 2020 08:00:31 +0000 (08:00 +0000)]
ixgbe: fix infinite recursion on PCIe link down
In some corner cases the functions ixgbe_clear_rar_generic and
ixgbe_clear_vmdq_generic may call one another leading to infinite
recursion.
When ixgbe_clear_vmdq_generic is called with IXGBE_CLEAR_VMDQ_ALL
flag, it's going to clear MPSAR registers, and proceed to call
ixgbe_clear_rar_generic, which in turn will clear the RAR registers,
and recursively call back ixgbe_clear_vmdq_generic. Normally, the
latter would detect that MPSAR registers have already been cleared
and terminate the recursion.
However, when PCIe link is down, and before the driver has had the
opportunity to shut itself down, all register reads return 0xFFFFFFFF,
and all register writes fail silently. In such case, because
ixgbe_clear_vmdq_generic blindly assumes that clearing MPSAR registers
succeeded, it's going to always call ixgbe_clear_rar_generic, which
in turn will always call back ixgbe_clear_vmdq_generic, creating
infinite recursion.
This patch re-reads MPSAR register values after they had been cleared.
In case of PCIe link failure, the values read will be non-zero, which
will terminate the recursion. On the other hand, under normal
circumstances the value read from MPSAR registers is going to be equal
to the value previously written, so this patch is expected not to cause
any regressions.
Simon Ellmann [Thu, 17 Dec 2020 17:14:52 +0000 (18:14 +0100)]
ixgbe: clear all queues on VF reset
ixgbe devices support up to 8 Rx and Tx queues per virtual function.
Currently, the registers of only seven queues are set to default when
resetting a VF.
To fix, discard both positive and negative vhid values in
in_aifaddr_ioctl, if carp(4) is not loaded. This prevents NULL pointer
dereference and kernel panic.
Alexander Motin [Fri, 3 Sep 2021 00:58:02 +0000 (20:58 -0400)]
vmd(4): Major driver refactoring
- Re-implement pcib interface to use standard pci bus driver on top of
vmd(4) instead of custom one.
- Re-implement memory/bus resource allocation to properly handle even
complicated configurations.
- Re-implement interrupt handling to evenly distribute children's MSI/
MSI-X interrupts between available vmd(4) MSI-X vectors and setup them
to be handled by standard OS mechanisms with minimal overhead, except
sharing when unavoidable.
Successfully tested on Dell XPS 13 laptop with Core i7-1185G7 CPU (VMD
device ID 0x9a0b) and single NVMe SSD, dual-booting with Windows 10.
Successfully tested on Supermicro X11DPI-NT motherboard with Xeon(R)
Gold 6242R CPUs (VMD device ID 0x201d), simultaneously handling NVMe
SSD on one PCIe port and PLX bridge with 3 NVMe and 1 AHCI SSDs on
another. Handles SSD hot-plug (except Optane 905p for some reason,
which are not detected until manual bus rescan) and enabled IOMMU
(directly connected SSDs work, but ones connected to the PLX fail
without errors from IOMMU).
This fixes a kernel panic when probing for vmd_bus on Intel TigerLake on
14-CURRENT. Apparently, vmd_bus is a type of PCI bus, but was registered
as a separate device class.
Warner Losh [Wed, 11 Aug 2021 16:59:28 +0000 (10:59 -0600)]
stand: Add MK_PIE=no to defs.mk
There's no need to build both pie and non-pie .o's for stand. There's
some other build thing with MK_BEAR_SSL=yes and/or MK_LOADER_VERIEXEC=yes
that causes the pie build to fail that the 'ar' stage now. Since we don't
need the PIE stuff and the non-PIE stuff, disable PIE for the boot loader.
Warner Losh [Thu, 10 Jun 2021 00:10:12 +0000 (18:10 -0600)]
mk: WITH_FOO=no now generates a warning
Many people are used to gnu configure's behavior of changing
--with-foo=no to --without-foo. At the same time, several folks have
WITH_FOO=no in their config files to enable this ironic form of the
option because of an old meme from IRC, a mailing list or the forums (I
forget which). Add a warning to allow to alert people w/o breaking POLA.
Warner Losh [Fri, 30 Jul 2021 23:11:43 +0000 (17:11 -0600)]
clock_gettime: Add Linux aliases for CLOCK_*
Linux standardized what we call CLOCK_{REALTIME,MONOTONIC}_FAST as
CLOCK_{REALTIME,MONOTONIC}_COARSE. In addition, Linux spells
CLOCK_UPTIME as CLOCK_BOOTTIME.
Add aliases to time.h and document these new aliases in
clock_gettime(2).
Warner Losh [Fri, 30 Jul 2021 23:10:56 +0000 (17:10 -0600)]
time.h: reduce CLOCK_ namespace pollution, move to _clock_id.h
Attempt to comply with the strict namespace pollution requirements of
_POSIX_C_SOURCE. Add guards to limit visitbility of CLOCK_ and TIMER_
defines as appropriate. Only define the CLOCK_ variables relevant to the
specific standards. Move all the sharing to sys/_clock_id.h and make
time.h and sys/time.h both include that rather than copy due to the
now large number of clocks and compat defines.
Please note: The old time.h previously used these newer dates:
CLOCK_REALTIME 199506
CLOCK_MONOTONIC 200112
CLOCK_THREAD_CPUTIME_ID 200112
CLOCK_PROCESS_CPUTIME_ID 200112
but glibc defines all of these for 199309. glibc uses this date for all
these values, however, only CLOCK_REALTIME was in IEEE 1003.1b. Add a
comment about this to document it. A large number of programs and
libraries assume that these will be defined for _POSIX_C_SOURCE =
199309.
In addition, leak CLOCK_UPTIME_FAST for the pocl package until it can be
updated to use a simple CLOCK_MONOTONIC.
John Hood [Wed, 28 Jul 2021 19:43:02 +0000 (13:43 -0600)]
loader: support.4th resets the read buffer incorrectly
Large nextboot.conf files (over 80 bytes) are not read correctly by the
Forth loader, causing file parsing to abort, and nextboot configuration
fails to apply.
Simple repro:
nextboot -e foo=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
shutdown -r now
That will cause the bug to cause a parse failure but shouldn't otherwise
affect the boot. Depending on your loader configuration, you may also
have to set beastie_disable and/or reduce the number of modules loaded
to see the error on a small console screen. 12.0 or CURRENT users will
also have to explicitly use the Forth loader instead of the Lua loader.
The error will look something like:
Warning: syntax error on file /boot/loader.conf.local
foo="xxxxxxxxxxxxxxnextboot_enable="YES"
^
/boot/support.4th has crude file I/O buffering, which uses a buffer
'read_buffer', defined to be 80 bytes by the 'read_buffer_size'
constant. The loader first tastes nextboot.conf, reading and parsing
the first line in it for nextboot_enable="YES". If this is true, then
it reopens the file and parses it like other loader .conf files.
Unfortunately, the file I/O buffering code does not fully reset the
buffer state in the reset_line_reading word. If the last file was read
to the end, that doesn't matter; the file buffer is treated as empty
anyway. But in the nextboot.conf case, the loader will not read to the
end of file if it is over 80 bytes, and the file buffer may be reused
when reading the next file. When the file is reread, the corrupt text
may cause file parsing to abort on bad syntax (if the corrupt line has
<>2 quotes in it), the wrong variable to be set, no variable to be set
at all, or (if the splice happens to land at a line ending) something
approximating normal operation.
The bug is very old, dating back to at least 2000 if not before, and is
still present in 12.0 and CURRENT r345863 (though it is now hidden by
the Lua loader by default).
Suggested one-line attached. This does change the behavior of the
reset_line_reading word, which is exported in the line-reading
dictionary (though the export is not documented in loader man pages).
But repo history shows it was probably exported for the PNP support
code, which was never included in the loader build, and was removed 5
months ago.
One thing that puzzles me: how has this bug gone unnoticed/unfixed for
nearly 2 decades? I find it hard to believe that nobody's tried to do
something interesting with nextboot, like load a kernel and filesystem,
which is what I'm doing.
Warner Losh [Wed, 28 Jul 2021 19:47:05 +0000 (13:47 -0600)]
genoffset: simplify and rewrite in sh
genoffset used the fully generic ASSYM macro to generate the offsets
needed for the thread_lite structure. However, since these are offsets
into a structure, they will always be necessarily small and positive. As
such, just create a simple character array of the right size and use a
naming convention such that we can recover the field name, structure
name and type. Use nm -t d and sort -n to sort these into order, then
loop over the resutls to generate the thread_lite structure.
Warner Losh [Sat, 24 Jul 2021 15:03:53 +0000 (09:03 -0600)]
devctl: don't publish the mount options
Mount options aren't solely ASCII strings. In addition, experience to
date suggests that the mount options are much less useful than was
originally supposed and the mount flags suffice to make decisions. Drop
the reporting of options for the mount/remount/unmount events.
Warner Losh [Fri, 23 Jul 2021 21:21:02 +0000 (15:21 -0600)]
geom_disk: use a preallocated geom_event for disk destruction.
Preallocate a geom_event (using the new geom_alloc_event) when we create
a disk. When we create the disk, we're going to be in a sleepable
context, so we can always allocate this extra bit of memory. Then use
this preallocated memory to free the disk. CAM can try to free the disk
from an unsleepable context if there was I/O outstanding when the disk
was destroyted (say because the SIM said it had gone away). The I/O
context isn't sleepable. Rather than trying to invent a retry mechanism
and making sure all the other geom_disk consumers did it properly,
preallocating the event ensure that the geom_disk will be properly torn
down, even when there's memory pressure when the disk departs.
Warner Losh [Fri, 23 Jul 2021 21:16:57 +0000 (15:16 -0600)]
geom: create an API to allocate events, and use that storage to send them
g_alloc_event will allocate storage for an opaque event. g_post_event_ep
can use memory returned by g_alloc_event to send an event from a context
that might not be able to allocate the event. Occasionally, we can
alloate memory when we create an object, but not while we're destroy
it. This allows one to allocate at creation time memory to use when
destorying the object.
The behavior remains the same, but lualoader now uses the more concise
verbiage that forthloader used. This is particularly important because
the previous line would exceed the right boundary of the menu and run
straight into space that would typically be allowed for the logo.
This makes it slightly easier to port logos from forthloader to
lualoader.
Colin Percival [Fri, 27 Aug 2021 00:54:53 +0000 (17:54 -0700)]
Add support for recording EC2 AMI Ids in SSM
If SSMPREFIX is specified, AMI Ids will be recorded in the SSM
Parameter Store under the name
${SSMPREFIX}/${ARCH}/${FLAVOUR}/${ROOTFS}/${REVISION}/${BRANCH}
where ARCH is "amd64" or "arm64", FLAVOUR is "base" (but may have
other options in the future), ROOTFS is "ufs" (but may have other
options in the future), and REVISION and BRANCH have their normal
meanings.
FreeBSD will be using the public prefix "/aws/service/freebsd",
resulting in SSM Parameter names which look like
/aws/service/freebsd/amd64/base/ufs/14.0/CURRENT
pf: ensure states passed to pf_free_state() are always unlinked
In pf_create_state() we can end up deleting the state immediately. This
can happen if we fail to map the relevant addresses or fail
normalization or fail to insert it into the state table.
If that happens we delete the state again with pf_free_state(). However,
this asserts that the state must be unlinked.
It's correct to simply set the state to PFTM_UNLINKED because we've not
yet linked it.
Martin Matuska [Mon, 23 Aug 2021 00:54:15 +0000 (02:54 +0200)]
libarchive: import changes from upstream
Libarchive 3.5.2
New features:
PR #1502: Support for PWB and v7 binary cpio formats
PR #1509: Support of deflate algorithm in symbolic link decompression
for ZIP archives
Important bugfixes:
IS #1044: fix extraction of hardlinks to symlinks
PR #1480: Fix truncation of size values during 7zip archive
extraction on 32bit architectures
PR #1504: fix rar header skiming
PR #1514: ZIP excessive disk read - fix location of central directory
PR #1520: fix double-free in CAB reader
PR #1521: Fixed leak of rar before ending with error
PR #1530: Handle short writes from archive_write_callback
PR #1532: 7zip: Use compression settings from file also for file header
IS #1566: do not follow symlinks when processing the fixup list
Toomas Soome [Thu, 2 Sep 2021 21:17:32 +0000 (00:17 +0300)]
loader.efi: fix console output after BS off
When Boot Services (BS) are switched off, we can not use BS
functions any more. Since drawn console does implement our own
Blt(), we can use it to draw the console.
However, SimpleTextOutput protocol based console output must be
blocked.
Tested by inserting printf() after ExitBootServices() call.
Ed Maste [Thu, 2 Sep 2021 00:23:40 +0000 (20:23 -0400)]
openssh: clarify krb5 use in freebsd-configure
freebsd-configure.sh runs configure twice, --with-kerberos5 and
--without-kerberos5, in order to build a config.h that defaults to
kerberos5 disabled, and a small config file that represents the
differences.
Rename config.h.orig to config.h.kerberos5 to clarify the intent of this
script.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
llvm: Revert "[HardwareLoops] Change order of SCEV expression construction for InitLoopCount."
Reverts llvm commit 42eaf4fe0adef3344adfd9fbccd49f325cb549ef, pointed
from bisect as source of regression that causes liblzma to compress/
uncompress incorrectly. It's know to affect powerpc64 BE only.
The patch unbreaks FreeBSD powerpc64 installation media, since
bsdinstall can't uncompress the *.txz produced by FreeBSD CI. It's
probably miscompiling other software bas well.
Reviewed by: dim
MFC after: 2 days
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision: https://reviews.freebsd.org/D31804
Mark Johnston [Wed, 1 Sep 2021 14:27:31 +0000 (10:27 -0400)]
sctp: Release the socket reference when detaching an association
Later in sctp_free_assoc(), when we clean up chunk lists,
sctp_free_spbufspace() is used to reset the byte count in the socket
send buffer. However, if the PCB is going away, the socket may already
have been detached from the PCB, in which case this becomes a use-after
free. Clear the socket reference from the association before detaching
it from the PCB, if the PCB has already lost its socket reference.
Reviewed by: tuexen
Sponsored by: The FreeBSD Foundation
Mark Johnston [Wed, 1 Sep 2021 14:27:51 +0000 (10:27 -0400)]
sctp: Hold association locks across socket wakeups when freeing
At this point we do not hold the inpcb lock, so the only thing holding
the socket reference live is the TCB lock, which needs to be acquired by
sctp_inpcb_free() in order to destroy associations. Defer the unlock to
until after we dereference the socket reference.
Reported by: syzbot+1d0f2c4675de76a4cf1e@syzkaller.appspotmail.com
Reported by: syzbot+fabee77954fe69d3a5ad@syzkaller.appspotmail.com
Reviewed by: tuexen
Sponsored by: The FreeBSD Foundation
Mark Johnston [Wed, 1 Sep 2021 18:18:58 +0000 (14:18 -0400)]
kevent: Fix races between timer detach and kqtimer_proc_continue()
- When detaching a knote, we need to double check the enqueued flag
after acquiring the process lock, as kqtimer_proc_continue() may have
toggled it.
- kqtimer_proc_continue() could in principle reschedule a stopped
callout after filt_timerdetach() drains the callout. So, we need to
re-check.
Reported by: syzbot+4a4cebb3ec07892cb040@syzkaller.appspotmail.com
Reported by: syzbot+a9c04bc76078a3b7dd8d@syzkaller.appspotmail.com
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
to change a state's state (that term is overloaded in pf, protocol state
like ESTABLISHED for tcp here), don't do it directly, but go through a newly
introduced pf_set_protostate()
In implicit mode assignment of slot number and tracking id is performed
automatically on each synchronization requested by device driver.
This is done with creation of intermediate buffer for multitouch events.
This buffer holds untracked events until synchronization is requested by
device driver. It is needed as touch assigment requires
knowledges of all touch positions pushed in current and previous reports.
Touch tracking is a process of assignment of unique trackingID to each
initiated contact on the surface. Keeping the trackingIDs persistent
across multitouch reports requires solving of so called Euclidian
Bipartite Matching problem.
This commit imports EBM-solver implementation based on Dinitz-Kronrod
algorithm to find minimum cost matching between contacts listed in two
consecutive reports.
projects / FreeBSD / FreeBSD.git / log