This commit adds BBR (Bottleneck Bandwidth and RTT) congestion control. This
is a completely separate TCP stack (tcp_bbr.ko) that will be built only if
you add the make options WITH_EXTRA_TCP_STACKS=1 and also include the option
TCPHPTS. You can also include the RATELIMIT option if you have a NIC interface that
supports hardware pacing, BBR understands how to use such a feature.
Note that this commit also adds in a general purpose time-filter which
allows you to have a min-filter or max-filter. A filter allows you to
have a low (or high) value for some period of time and degrade slowly
to another value has time passes. You can find out the details of
BBR by looking at the original paper at:
or consult many other web resources you can find on the web
referenced by "BBR congestion control". It should be noted that
BBRv1 (which this is) does tend to unfairness in cases of small
buffered paths, and it will usually get less bandwidth in the case
of large BDP paths(when competing with new-reno or cubic flows). BBR
is still an active research area and we do plan on implementing V2
of BBR to see if it is an improvement over V1.
Sponsored by: Netflix Inc.
Differential Revision: https://reviews.freebsd.org/D21582
From Piotr:
r351152 introduced iflib_deregister() function calling
EVENTHANDLER_DEREGISTER() to unregister VLAN events. This patch removes
duplicate of EVENTHANDLER_DEREGISTER() calls placed in
iflib_device_deregister() as this function is now calling
iflib_deregister(). This is to avoid deregistering same event twice.
This patch also adds check in iflib_vlan_register() to prevent
registering VLAN while being in detach.
Patch co-authored by Krzysztof Galazka <krzysztof.galazka@intel.com>,
erj <erj@FreeBSD.org> and Jacob Keller <jacob.e.keller@intel.com>.
Signed-off-by: Piotr Pietruszewski <piotr.pietruszewski@intel.com>
Submitted by: Piotr Pietruszewski <piotr.pietruszewski@intel.com>
Reviewed by: gallatin@, erj@
MFC after: 3 days
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D21711
Plumb a memory leak.
Thnanks to Felix Weinrank for finding this issue using fuzz testing
and reporting it for the userland stack:
https://github.com/sctplab/usrsctp/issues/378
Replace all mtx_lock()/mtx_unlock() on n_mtx with the macros.
For a long time, some places in the NFS code have locked/unlocked the
NFS node lock with the macros NFSLOCKNODE()/NFSUNLOCKNODE() whereas
others have simply used mtx_lock()/mtx_unlock().
Since the NFS node mutex needs to change to an sx lock so it can be held when
vnode_pager_setsize() is called, replace all occurrences of mtx_lock/mtx_unlock
with the macros to simply making the change to an sx lock in future commit.
There is no semantic change as a result of this commit.
I am not sure if the change to an sx lock will be MFC'd soon, so I put
an MFC of 1 week on this commit so that it could be MFC'd with that commit.
- track the total count of hot entries
- pre-read the lock when shrinking since it is typically already taken
- place the lock in its own cacheline
- shorten the hold time of hot lock list when zapping
- For each queue pair precalculate CPU and domain it is bound to.
If queue pairs are not per-CPU, then use the domain of the device.
- Allocate most of queue pair memory from the domain it is bound to.
- Bind callouts to the same CPUs as queue pair to avoid migrations.
- Do not assign queue pairs to each SMT thread. It just wasted
resources and increased lock congestions.
- Remove fixed multiplier of CPUs per queue pair, spread them even.
This allows to use more queue pairs in some hardware configurations.
- If queue pair serves multiple CPUs, bind different NVMe devices to
different CPUs.
There is no reason for these routines to be written in assembly. In
the ports of DTrace to other platforms, they are already written in C.
No functional change intended.
Set NX on some non-leaf direct map page table entries.
The direct map is never used for execution of code, so we might as well
set NX in the direct map's PML4Es. Also clarify the intent of the code
in create_pagetables() that restricts access protections on the region
of the direct map mapping the kernel text.
Use elf_relocaddr() when handling R_X86_64_RELATIVE relocations.
This is required for DPCPU and VNET data variable definitions to work when
KLDs are linked as DSOs. R_X86_64_RELATIVE relocations should not appear
in object files, so assert this in elf_relocaddr().
If robust mutex' owner terminated, causing kernel-assisted state
recovery, and then pthread_mutex_destroy() is executed as the next
action, assert is triggered about mutex still being on the list.
Ignore the mutex linkage in pthread_mutex_destroy() for shared robust
mutexes with dead owner, same as for enqueue_mutex().
Reported by: avg
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
SYSINIT usage was added, but the <sys/kernel.h> dependency was not added.
This worked by coincidence, as most of the mips configs have DDB enabled and
pmap.c gets <sys/kernel.h> via ddb.h pollution.
sef [Mon, 23 Sep 2019 04:28:07 +0000 (04:28 +0000)]
Add two options to allow mount to avoid covering up existing mount points.
The two options are
* nocover/cover: Prevent/allow mounting over an existing root mountpoint.
E.g., "mount -t ufs -o nocover /dev/sd1a /usr/local" will fail if /usr/local
is already a mountpoint.
* emptydir/noemptydir: Prevent/allow mounting on a non-empty directory.
E.g., "mount -t ufs -o emptydir /dev/sd1a /usr" will fail.
Neither of these options is intended to be a default, for historical and
compatibility reasons.
octeon-sdk: suppress another set of warnings under clang
Clang sees this construct and warns that adding an int to a string like this
does not concatenate the two. Fortunately, this is not what octeon-sdk
actually intended to do, so we take the path towards remediation that clang
offers: use array indexing instead.
octeon1: suppress a couple of warnings under clang
These appear in octeon-sdk -- there are new releases, but they don't seem to
address the running issues in octeon-sdk. GCC4.2 is more than happy, but
clang is much less-so and most of them are fairly innocuous and perhaps a
by-product of their style guide, which may make some of the changes harder
to upstream (if this is even possible anymore).
Add support for kernel threads in kvm_getprocs() and the underlying
kvm_proclist() in libkvm when fetching from a kernel core file. This
has been missing/needed for several releases, when kernel threads became
normal threads. The loop over the processes now contains a sub-loop for
threads, which iterates beyond the first thread only when threads are
requested. Also set some fields such as tid that were previously
uninitialized.
Don't hold the info lock when calling sctp_select_a_tag().
This avoids a double lock bug in the NAT colliding state processing
of SCTP. Thanks to Felix Weinrank for finding and reporting this issue in
https://github.com/sctplab/usrsctp/issues/374
He found this bug using fuzz testing.
Cleanup the RTO calculation and perform some consistency checks
before computing the RTO.
This should fix an overflow issue reported by Felix Weinrank in
https://github.com/sctplab/usrsctp/issues/375
for the userland stack and found by running a fuzz tester.
MFZoL: Retire send space estimation via ZFS_IOC_SEND
Add a small wrapper around libzfs_core's lzc_send_space() to libzfs so
that every legacy ZFS_IOC_SEND consumer, along with their userland
counterpart estimate_ioctl(), can leverage ZFS_IOC_SEND_SPACE to
request send space estimation.
The legacy functionality in zfs_ioc_send() is left untouched for
compatibility purposes.
print summary line for space estimate of zfs send from bookmark
Although there is always a single stream and the total size in the
summary is always equal to the size reported for the stream, it's nice
to follow the usual output format.
In case a translation fault on the kernel address space occurs from
within a critical section, we must perform a lock-free check on the
faulting address.
jot.1: Explain default argument values more precisely
The way jot(1) defaults missing arguments doesn't match the behaviour
described in the manpage, which states that with fewer than 3 arguments
missing values are supplied from left to right.
In fact, with one or two arguments, the last (s which is step size or seed)
defaults to 1 (or -1 if begin and end specify a descending range), and then
omitted arguments are set to default starting with the leftmost until three
arguments are available.
This is why `jot 2 1000` prints 1000 and 1001 instead of 1000 and 100.
Fix build errors of test.c, which had been broken for a long time.
This is a temporary fix and should be converted to a complete
test scenarios by using this tool.
Add a workaround for servers which respond RPC_PROGNOTREGISTERED
to a clnt_create() call even when it is actually a program
version mismatch.
Normally the server is supposed to return RPC_PROGVERSMISMATCH
when it supports the specified program but not support
the specified version. Some filers return RPC_PROGNOTREGISTERED
to RQUOTA v2 calls and FreeBSD does not retry with the old
v1 calls. This change fixes this failure scenario.
When a file is unlinked, the denode is not reclaimed until the last
reference is dropped, but the directory entry is immediately up for reuse.
This is a problem later when createde goes to grab a denode for the newly
created entry -- we search the hash and find a dead denode, then return that
without even bumping the reference count and the data later gets truncated
when the the last reference to the unlinked file is dropped.
This manifested itself as a broken in-place strip(1) on msdosfs. elfcopy
will do a sequence incredibly roughly like this:
and the resulting file would be truncated, but the write succeeded, as long
as a reference to the unlinked file had not been closed.
Some archaeology indicates that this bug has likely existed since msdosfs
was converted to use vfs_hash instead of a home rolled hash implementation
in r143570. Prior to that point, the hashget implementation would do a
refcnt check while searching and explicitly only return a denode with
de_refcnt != 0. vfs_hash did not yet have the callback that it does today,
so this slipped away and did not come back when it later grew that
functionality.
The comment indicating that we want to skip these denodes has been updated
to reflect where this is actually done. My repo-diving session seems to
indicate that the refcnt check was likely never actually below the comment,
to be pedantic, but instead a detail wrapped up in the hashget
implementation since the beginning of its inclusion into FreeBSD.
This bug was the cause behind the issue addressed in r352557.
loader: Respect loader_color=YES for serial consoles
It's not uncommon these days for the terminals attached to serial consoles
to support ANSI escape sequences. However, we assume escape sequences may
break some serial consoles and default to not using them when boot_serial or
boot_multicons (or if console contains "comconsole" in the forth loader) for
broader compatibility. We also have loader_color which can be explicitly set
to "NO" to disable the use of ANSI escape sequences.
The problem is that loader_color=YES gets ignored when boot_serial=YES or
boot_multicons=YES (or when console contains "comconsole" in the forth
loader).
To fix, the existing default behavior remains unchanged when loader_color is
unset, loader_color=NO explicitly disables the use of ANSI escape sequences
still, and the change is that loader_color=YES can now be used to explicitly
allow ANSI escapes when a serial console is enabled.
top(1): support multibyte characters in command names (ARGV array)
depending on locale.
- add setlocale()
- remove printable() function
- add VIS_OCTAL and VIS_SAFE to the flag of strvisx() to display
non-printable characters that do not use C-style backslash sequences
in three digit octal sequence, or remove it
This change allows multibyte characters to be displayed according to
locale. If it is recognized as a non-display character according to the
locale, it is displayed in three digit octal sequence.
Summary:
Install's strip capability, by way of strip(1), doesn't seem to work
correctly on msdosfs, and instead ends up truncating the resulting
binary to 0-length. As a workaround, don't strip ubldr(8). This
fixes installworld on Book-E ubldr-based platforms, which prior to this
would need to manually install ubldr separately after installworld, in
order to have a functional ubldr.
The same thing could be done on PowerNV platforms that use msdosfs /boot
volumes, since loader and loader.kboot, etc, all get truncated to 0 on
install. However, PowerNV does not use loader, instead loading from
petitboot, so it's not really necessary at this time.
Add quirk for XHCI(4) controllers to support USB control transfers
above 1Kbyte. It might look like some XHCI(4) controllers do not
support when the USB control transfer is split using a link TRB. The
next NORMAL TRB after the link TRB is simply failing with XHCI error
code 4. The quirk ensures we allocate a 64Kbyte buffer so that the
data stage TRB is not broken with a link TRB.
Increase the maximum user-space buffer size from 256kBytes to 32MBytes for
libusb. This is useful for speeding up large data transfers while reducing
the interrupt rate.
r348628 added a definition of NT_GNU_BUILD_ID. Some software (Valgrind)
also expects a #define for the note name (ELF_NOTE_GNU) in the case that
NT_GNU_BUILD_ID is defined.
PR: 239669
Reported by: Yuichiro NAITO
Sponsored by: The FreeBSD Foundation
Event: EuroBSDCon FreeBSD DevSummit 2019
Fix the handling of invalid parameters in ASCONF chunks.
Thanks to Mark Wodrich from Google for reproting the issue in
https://github.com/sctplab/usrsctp/issues/376
for the userland stack.
Allocate ioat->ring memory from the device domain.
Schedule ioat->poll_timer to the first CPU of the device domain.
According to pcm-numa tool from intel-pcm port, this reduces number of
remote DRAM accesses while copying data by 75%. And unless it is a noise,
I've noticed some speed improvement when copying data to other domain.
Follow up on r352304 which disabled default mlockall() at startup.
Unfortunately though the original tarball supports this in ./configure
(for Linux), to fully support disabling of mlockall() by default requires
a little extra help otherwise the following is logged in syslog:
Cannot set RLIMIT_MEMLOCK: Operation not permitted
Apply r346792 (cperciva) from stable/12 to head. The original commit
message:
On non-x86 systems, use "quarterly" packages.
x86 architectures have "latest" package builds on stable/*, so keep using
those (they'll get switched over to "quarterly" during releases).
The original commit was a direct commit to stable/12, as at the time it
was presumed it would not be necessary for head. However, when it is time
to create a releng branch or switch from PRERELEASE/STABLE to BETA/RC, the
pkg(7) Makefile needs further adjusting. This commit includes those
further adjustments, evaluating the BRANCH variable from release/Makefile
to determine the pkg(7) repository to use.
igor follows American style guides in the belief that abbreviations i.e.
and e.g. are always followed by a comma. Make that change now so that
future updates to freebsd-update.8 do not complain about this.
When the RACK stack computes the space for user data in a TCP segment,
it wasn't taking the IP level options into account. This patch fixes this.
In addition, it also corrects a KASSERT and adds protection code to assure
that the IP header chain and the TCP head fit in the first fragment as
required by RFC 7112.
Reviewed by: rrs@
MFC after: 3 days
Sponsored by: Nertflix, Inc.
Differential Revision: https://reviews.freebsd.org/D21666
When processing an incoming IPv6 packet over the loopback interface which
contains Hop-by-Hop options, the mbuf chain is potentially changed in
ip6_hopopts_input(), called by ip6_input_hbh().
This can happen, because of the the use of IP6_EXTHDR_CHECK, which might
call m_pullup().
So provide the updated pointer back to the called of ip6_input_hbh() to
avoid using a freed mbuf chain in`ip6_input()`.
Reviewed by: markj@
MFC after: 3 days
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D21664
It was incorrect with respect to swapping dataset IDs both in the
on-disk ZAP object and the in-memory queue.
In both cases, if only ds1 was already present, then it would be first
replaced with ds2 and then ds2 would be replaced back with ds1. Also,
both cases did not properly handle a situation where both ds1 and ds2
are already queued. A duplicate insertion would be attempted and its
failure would result in a panic.
This change has also been submitted to ZoL as zfsonlinux/zfs@dd262c9
vt: fix problems with trying to switch to a closed VT
If there is an attempt to switch from a process-owned VT to a closed VT,
then vt(4) first requests the process to release its VT and only then
realizes that the target VT is closed and, so, the switch is not
possible. So, the driver does not actually do any switch, but at the
same time the owning process is not notified about that and it does not
re-acquire the VT.
This change adds an early check for the target VT state, so that the
switch can be refused before the process coordination dance.
On top of that, the code now checks for a failure of vt_window_switch()
and calls vt_window_postswitch() for the current VT if it is in the
process mode.
Test Plan:
- configure VT1 - VT8 (ttyv0 - ttyv7) to be text consoles (run getty)
- configure VT9 (ttyv8) to rn X server
- make sure that the X server configuration allows VT switching
- leave VT10 - VT12 unconfigured
- while in the X server press Ctrl+Alt+F10
- without the patch, observe strange screen content and problems with
keyboard input
- with the patch, observe that nothing happens
The problem has been observed and the fix has been tested with an nVidia
graphics card and the proprietary nvidia driver.
Not sure if that matters.
Reviewed by: ray
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D21704
BERI stands for Bluespec Extensible RISC Implementation, based on MIPS.
BERI has not implemented standard MIPS perfomance monitoring counters,
instead it provides statistical counters.
BERI statcounters have a several limitations:
- They can't be written
- They don't support start/stop operation
- None of hardware interrupt is provided on a counter overflow.
So make it separate to hwpmc_mips module and support process/system
counting mode only.
Note that old pkg will failed to build after this. A recent ports tree (one
providing pkg 1.12+) is required to build. Older already built pkg, should
continue working as expected
PR: 238797
Exp run by: antoine
Reviewed by: cem
Approved by: cem
Differential Revision: https://reviews.freebsd.org/D20752
googletest: default-disable on all of MIPS for now
Parts of the fusefs tests trigger a bug in current versions of llvm: IR
representation of some routine for the MIPS targets is a function with a
large number of arguments. This then leads the compiler on an hour+ long
goose chase, which is OK if you build the current tree but less-so if you're
trying external toolchain or doing a universe build involving mips when it
eventually gets switched over to LLVM.
Better, accurate details can be found in LLVM PR43263.
mips: ubldr: use truncated load address for mips32
BFD appears to silently truncate 0xffffffff80800000 when it processes the
ldscript for 32-bit mips, but LLD chokes on it as the linker script tries to
place elements above 32-bit range. It's unclear to me if silent truncation
is kosher or not and whether this patch is really what we want to do, but it
is one approach at least.
The ioctl(2) is intended to provide more details about the cause of
the down for the link.
Eventually we might define a comprehensive list of codes for the
situations. But interface also allows the driver to provide free-form
null-terminated ASCII string to provide arbitrary non-formalized
information. Sample implementation exists for mlx5(4), where the
string is fetched from firmware controlling the port.
Further refine r352393, only call vnode_pager_setsize() outside the
node lock when shrinking.
This is similar to r252528, applied to the above commit.
Apparently there is a race which makes necessary at least to keep the
n_size and pager size consistent when extending. Current suspect is
that iod threads perform vnode_pager_setsize() without taking the
vnode lock, which corrupts the file content.
Reported and tested by: Masachika ISHIZUKA <ish@amail.plala.or.jp>
Discussed with: rmacklem (related issues)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
See http://www.open-std.org/jtc1/sc22/wg14/www/docs/summary.htm#dr_400.
Upstream jemalloc issue is opened by emaste at
https://github.com/jemalloc/jemalloc/issues/1629.
Reviewed by: emaste
PR: 240456
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
DIfferential revision: https://reviews.freebsd.org/D21632
projects / FreeBSD / FreeBSD.git / log