MFC: r345866
Fix malloc stats for the RPCSEC_GSS server code when DEBUG is enabled.
The code enabled when "DEBUG" is defined uses mem_alloc(), which is a
malloc(.., M_RPC, M_WAITOK | M_ZERO), but then calls gss_release_buffer()
which does a free(.., M_GSSAPI) to free the memory.
This patch fixes the problem by replacing mem_alloc() with a
malloc(.., M_GSSAPI, M_WAITOK | M_ZERO).
This bug affects almost no one, since the sources are not normally built
with "DEBUG" defined.
dim [Wed, 17 Apr 2019 20:16:48 +0000 (20:16 +0000)]
After r346168, also merge build infrastructure for LLVM libomp.
MFC r345235:
Add lib/libomp, with a Makefile, and generated configuration headers.
Not connected to the main build yet, as there is still the issue of the
GNU omp.h header conflicting with the LLVM one. (That is, if MK_GCC is
enabled.)
PR: 236062
MFC r345236:
Connect lib/libomp to the build.
* Set MK_OPENMP to yes by default only on amd64, for now.
* Bump __FreeBSD_version to signal this addition.
* Ensure gcc's conflicting omp.h is not installed if MK_OPENMP is yes.
* Update OptionalObsoleteFiles.inc to cope with the conflicting omp.h.
* Regenerate src.conf(5) with new WITH/WITHOUT fragments.
Relnotes: yes
PR: 236062
MFC r345242:
Explicitly link libomp.so against -lpthread, as it depends on pthread
functionality. This should make example OpenMP programs work out of the
box.
Reported by: jbeich
PR: 236062, 236581
MFC r345278:
Also explicitly link libomp.so against -lm, as it transitively depends
on scalbn and a few other math functions, via libcompiler-rt. This
should allow OpenMP programs to link with BFD linkers too.
Reported by: jbeich
PR: 236062, 236581
MFC r345282:
Remove --as-needed from the linker flags for libomp.so, as these
actually prevent the transitive dependency on libm.
Reported by: jbeich
PR: 236062, 236581
MFC r345291:
Turn on MK_OPENMP for i386 by default, now that it can build.
MFC r345192-r345194: if_bridge(4): Drop pointless rtflush
r345192:
if_bridge(4): Drop pointless rtflush
At this point, all routes should've already been dropped by removing all
members from the bridge. This condition is in-fact KASSERT'd in the line
immediately above where this nop flush was added.
r345193:
Revert r345192: Too many trees in play for bridge(4) bits
An accidental appendage was committed that has not undergone review yet.
r345194:
if_bridge(4): Drop pointless rtflush
At this point, all routes should've already been dropped by removing all
members from the bridge. This condition is in-fact KASSERT'd in the line
immediately above where this nop flush was added.
MFC r346132: stand: refactor overlay loading a little bit
It was pointed out that manually loading a .dtb to be used rather than
relying on platform-specific method for loading .dtb will result in overlays
not being applied. This was true because overlay loading was hacked into
fdt_platform_load_dtb, rather than done in a way more independent from how
the .dtb is loaded.
Instead, push overlay loading (for now) out into an
fdt_platform_load_overlays. This method easily allows ubldr to pull in any
fdt_overlays specified in the ub env, and omits overlay-checking on
platforms where they're not tested and/or not desired (e.g. powerpc). If we
eventually stop caring about fdt_overlays from ubenv (if we ever cared),
this method should get chopped out in favor of just calling
fdt_load_dtb_overlays() directly.
bridge_rtnode_zone still has outstanding allocations at the time of
destruction in the current model because all of the interface teardown
happens in a VNET_SYSUNINIT, -after- the MOD_UNLOAD has already been
processed. The SYSUNINIT triggers destruction of the interfaces, which then
attempts to free the memory from the zone that's already been destroyed, and
we hit a panic.
Solve this by virtualizing the uma_zone we allocate the rtnodes from to fix
the ordering. bridge_rtable_fini should also take care to flush any
remaining routes that weren't taken care of when dynamic routes were flushed
in bridge_stop.
r345187: bridge: Fix STP-related panic
After r345180 we need to have the appropriate vnet context set to delete an
rtnode in bridge_rtnode_destroy().
That's usually the case, but not when it's called by the STP code (through
bstp_notify_rtage()).
We have to set the vnet context in bridge_rtable_expire() just as we do in the
other STP callback bridge_state_change().
When an exception is thrown the unwinder must unwind its own C source
(starting with _Unwind_RaiseException in UnwindLevel1.c), so it needs to
be built with unwinding data.
MFC r324998 (by bdrewery):
Prefix {TARGET,BUILD}_TRIPLE with LLVM_ to avoid Makefile.inc1 collision.
The Makefile.inc1 TARGET_TRIPLE is for specifying which -target is used
during the build of world.
Introduce WITH_/WITHOUT_LLVM_COV to match GCC's WITH_/WITHOUT_GCOV.
It is intended to provide a superset of the interface and functionality
of gcov.
It is enabled by default when building Clang, similarly to gcov and GCC.
This change moves one file in libllvm to be compiled unconditionally.
Previously it was included only when WITH_CLANG_EXTRAS was set, but the
complexity of a new special case for (CLANG_EXTRAS | LLVM_COV) is not
worth avoiding a tiny increase in build time.
Reviewed by: dim, imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D142645
MFC r331244 (by jhb):
Add support for MIPS to LLVM's libunwind.
This is originally based on a patch from David Chisnall for soft-float
N64 but has since been updated to support O32, N32, and hard-float ABIs.
The soft-float O32, N32, and N64 support has been committed upstream.
The hard-float changes are still in review upstream.
Enable LLVM_LIBUNWIND on mips when building with a suitable (C+11-capable)
toolchain. This has been tested with external GCC for all ABIs and
O32 and N64 with clang.
All supported FreeBSD build host versions have backtrace.h, so we can
just eliminate that test. For futimes() we can test the compiler's
built-in __FreeBSD__ major version rather than relying on including
osreldate.h. This should reduce the frequency with which Clang gets
rebuilt when building world.
Reviewed by: dim
Sponsored by: The FreeBSD Foundation
MFC r337379 (by andrew):
Default to armv5te in LINT on arm. This should fix building LINT there.
MFC r337552:
Add optional LLVM BPF target support
BPF (eBPF) is an independent instruction set architecture which is
introduced in Linux a few years ago. Originally, eBPF execute
environment was only inside Linux kernel. However, recent years there
are some user space implementation (https://github.com/iovisor/ubpf,
https://doc.dpdk.org/guides/prog_guide/bpf_lib.html) and kernel space
implementation for FreeBSD is going on
(https://github.com/YutaroHayakawa/generic-ebpf).
The BPF target support can be enabled using WITH_LLVM_TARGET_BPF, as it
is not built by default.
In r308100, an explicit -fexceptions flag was added for the C sources
from LLVM's libunwind, which end up in libgcc_eh.a and libgcc_s.so.
This is because the unwinder needs the unwinder data for its own
functions.
However, for the C++ sources in libunwind, -fexceptions is already the
default, and this can have the side effect of generating a reference to
__gxx_personality_v0, the so-called personality function, which is
normally provided by the C++ ABI library (libcxxrt or libsupc++).
If the reference ends up in the eventual libgcc_s.so, linking any
non-C++ programs against it will fail with "undefined reference to
`__gxx_personality_v0'".
Note that at high optimization levels, the reference is usually
optimized away, which is why we have never noticed this problem before.
With clang 7.0.0 though, higher optimization levels don't help anymore,
since the addition of address-significance tables [1] in
<https://reviews.llvm.org/rL337339>. Effectively, this always causes a
reference to __gxx_personality_v0.
After discussion with the upstream author of that change, it turns out
that we should compile libunwind sources with the -fno-exceptions
-funwind-tables flags instead. This ensures unwind tables are
generated, but no references to any personality functions are emitted.
llvm-cov: also install as gcov (if GNU gcov is disabled)
llvm-cov provides a gcov-compatible interface when invoked as gcov.
Reviewed by: dim, markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17923
MFC r340296 (by emaste):
Move llvm-profdata build into MK_LLVM_COV block
llvm-profdata is used with llvm-cov for code coverage (although llvm-cov
can also operate independently in a gcov-compatible mode).
Although llvm-profdata can be used independently of llvm-cov it makes
sense to group these under one option.
Also handle these in OptionalObsoleteFiles.inc while here.
Sponsored by: The FreeBSD Foundation
MFC r340300 (by emaste):
libllvm: Move SampleProfWriter to SRCS_MIN
It is required by llvm-profdata, now built by default under the
LLVM_COV knob. The additional complexity that would come from avoiding
building it if CLANG_EXTRAS and LLVM_COV are both disabled is not worth
the small savings in build time.
Sponsored by: The FreeBSD Foundation
MFC r340972 (by emaste):
llvm-objdump: initial man page
Based on llvm-objdump's online documentation and usage information.
This serves as a starting point; additional detail and cleanup still
required.
Also being submitted upstream in LLVM review D54864. I expect to use
this bespoke copy while we have LLVM 6.0 or 7.0 in FreeBSD; when we
update to LLVM 8.0 it should be upstream and we will switch to it.
PR: 233437
Reviewed by: bcr (man formatting)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18309
MFC r340973 (by emaste):
llvm-objdump.1: remove invalid options
Some options appear in llvm-objdump's usage information as a side effect
of its option parsing implementation and are not actually llvm-objdump
options. Reported in LLVM review https://reviews.llvm.org/D54864.
Reported by: Fangrui Song
Sponsored by: The FreeBSD Foundation
MFC r340975 (by emaste):
llvm-objdump.1: fix igor / mandoc -Tlint warnings
Accidentally omitted from r340972.
MFC r341055 (by emaste):
llvm-objdump.1: remove more unintentional options
Some options come from static constructors in LLVM libraries and are
automatically added to llvm's usage output. They're not really supposed
to be llvm-objdump options.
Reported by: Fangrui Song in LLVM review D54864
Sponsored by: The FreeBSD Foundation
MFC r344779:
Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to
the upstream release_80 branch r355313 (effectively, 8.0.0 rc3). The
release will follow very soon, but no more functional changes are
expected.
Release notes for llvm, clang and lld 8.0.0 will soon be available here:
<https://releases.llvm.org/8.0.0/docs/ReleaseNotes.html>
<https://releases.llvm.org/8.0.0/tools/clang/docs/ReleaseNotes.html>
<https://releases.llvm.org/8.0.0/tools/lld/docs/ReleaseNotes.html>
PR: 236062
Relnotes: yes
MFC r344798 (by emaste):
libllvm: promote WithColor and xxhash to SRCS_MIN
The armv6 build failed in CI due to missing symbols (from these two
source files) in the bootstrap Clang.
This affected only armv6 because other Clang-using archs are using LLD
as the bootstrap linker, and thus include SRCS_MIW via LLD_BOOTSTRAP.
Reported by: CI, via lwhsu
Sponsored by: The FreeBSD Foundation
MFC r344825:
Add a few missed files to the MK_LLVM_TARGET_BPF=yes case, otherwise
clang and various other executables will fail to link with undefined
symbols.
Reported by: O. Hartmann <ohartmann@walstatt.org>
MFC r344852:
Put in a temporary workaround for what is likely a gcc 6 bug (it does
not occur with gcc 7 or later). This should prevent the following error
from breaking the head-amd64-gcc CI builds:
In file included from /workspace/src/contrib/llvm/tools/lldb/source/API/SBMemoryRegionInfo.cpp:14:0:
/workspace/src/contrib/llvm/tools/lldb/include/lldb/Target/MemoryRegionInfo.h:128:54: error: 'template<class _InputIterator> lldb_private::MemoryRegionInfos::MemoryRegionInfos(_InputIterator, _InputIterator, const allocator_type&)' inherited from 'std::__1::vector<lldb_private::MemoryRegionInfo>'
using std::vector<lldb_private::MemoryRegionInfo>::vector;
^~~~~~
/workspace/src/contrib/llvm/tools/lldb/include/lldb/Target/MemoryRegionInfo.h:128:54: error: conflicts with version inherited from 'std::__1::vector<lldb_private::MemoryRegionInfo>'
Reported by: CI
MFC r344896:
Pull in r354937 from upstream clang trunk (by Jörg Sonnenberger):
Fix inline assembler constraint validation
The current constraint logic is both too lax and too strict. It fails
for input outside the [INT_MIN..INT_MAX] range, but it also
implicitly accepts 0 as value when it should not. Adjust logic to
handle both correctly.
Pull in r355491 from upstream clang trunk (by Hans Wennborg):
Inline asm constraints: allow ICE-like pointers for the "n"
constraint (PR40890)
Apparently GCC allows this, and there's code relying on it (see bug).
The idea is to allow expression that would have been allowed if they
were cast to int. So I based the code on how such a cast would be
done (the CK_PointerToIntegral case in
IntExprEvaluator::VisitCastExpr()).
These should fix assertions and errors when using the inline assembly
"n" constraint in certain ways.
In case of devel/valgrind, a pointer was used as the input for the
constraint, which lead to "Assertion failed: (isInt() && "Invalid
accessor"), function getInt".
In case of math/secp256k1, a very large integer value was used as input
for the constraint, which lead to "error: value '4624529908474429119'
out of range for constraint 'n'".
PR: 236216, 236194
MFC r344951:
Merge llvm, clang, compiler-rt, libc++, lld, and lldb release_80 branch
r355677 (effectively, 8.0.0 rc4), resolve conflicts, and bump version
numbers.
PR: 236062
MFC r345018:
Merge LLVM libunwind trunk r351319, from just before upstream's
release_80 branch point. Afterwards, we will merge the rest of the
changes in the actual release_80 branch.
Pull in r355854 from upstream llvm trunk (by Jonas Paulsson):
[RegAlloc] Avoid compile time regression with multiple copy hints.
As a fix for https://bugs.llvm.org/show_bug.cgi?id=40986 ("excessive
compile time building opencollada"), this patch makes sure that no
phys reg is hinted more than once from getRegAllocationHints().
This handles the case were many virtual registers are assigned to the
same physreg. The previous compile time fix (r343686) in
weightCalcHelper() only made sure that physical/virtual registers are
passed no more than once to addRegAllocationHint().
Revert r308867 (which was originally committed in the clang390-import
project branch):
Work around LLVM PR30879, which is about a bad interaction between
X86 Call Frame Optimization on i386 and libunwind, by disallowing the
optimization for i386-freebsd12.
This should fix some instances of broken exception handling when
frame pointers are omitted, in particular some unittests run during
the build of editors/libreoffice.
This hack will be removed as soon as upstream has implemented a more
permanent fix for this problem.
And indeed, after r345018 and r345019, which updated LLVM libunwind to
the most recent version, the above workaround is no longer needed. The
upstream commit which fixed this is:
Specifically, 32 bit (i386-freebsd) executables optimized with omitted
frame pointers and Call Frame Optimization should now behave correctly
when a C++ exception is thrown, and the stack is unwound.
Merge llvm, clang, compiler-rt, libc++, libunwind, lld, and lldb
release_80 branch r356034 (effectively, 8.0.0 rc5), resolve conflicts,
and bump version numbers.
PR: 236062
MFC r345231:
Add LLVM openmp trunk r351319 (just before the release_80 branch point)
to contrib/llvm. This is not yet connected to the build, the glue for
that will come in a follow-up commit.
Add openmp __kmp_gettid() wrapper, using pthread_getthreadid_np(3).
This has also been submitted upstream.
PR: 236062
MFC r345283:
Enable building libomp.so for 32-bit x86. This is done by selectively
enabling the functions that save and restore MXCSR, since access to this
register requires SSE support.
Note that you may run into other issues with OpenMP on i386, since this
*not* yet supported upstream, and certainly not extensively tested.
PR: 236062, 236582
MFC r345345:
Merge llvm, clang, compiler-rt, libc++, libunwind, lld, lldb and openmp
8.0.0 final release r356365. There were no functional changes since the
most recent merge, of 8.0.0 rc5.
Release notes for llvm, clang, lld and libc++ 8.0.0 are now available:
Pull in r352826 from upstream lld trunk (by Fangrui Song):
[ELF] Support --{,no-}allow-shlib-undefined
Summary:
In ld.bfd/gold, --no-allow-shlib-undefined is the default when
linking an executable. This patch implements a check to error on
undefined symbols in a shared object, if all of its DT_NEEDED entries
are seen.
Our approach resembles the one used in gold, achieves a good balance
to be useful but not too smart (ld.bfd traces all DSOs and emulates
the behavior of a dynamic linker to catch more cases).
The error is issued based on the symbol table, different from
undefined reference errors issued for relocations. It is most
effective when there are DSOs that were not linked with -z defs (e.g.
when static sanitizers runtime is used).
gold has a comment that some system libraries on GNU/Linux may have
spurious undefined references and thus system libraries should be
excluded (https://sourceware.org/bugzilla/show_bug.cgi?id=6811). The
story may have changed now but we make --allow-shlib-undefined the
default for now. Its interaction with -shared can be discussed in the
future.
Together, these add support for --no-allow-shlib-undefined, and make it
the default for executables, so they will fail to link if any symbols
from needed shared libraries are undefined.
Reported by: jbeich
PR: 236062, 236141
MFC r345449:
Pull in r356809 from upstream llvm trunk (by Eli Friedman):
[ARM] Don't form "ands" when it isn't scheduled correctly.
In r322972/r323136, the iteration here was changed to catch cases at
the beginning of a basic block... but we accidentally deleted an
important safety check. Restore that check to the way it was.
This should fix "Assertion failed: (LiveCPSR && "CPSR liveness tracking
is wrong!"), function UpdateCPSRUse" errors when building the devel/xwpe
port for armv7.
MFC: r345818, r345828
Fix a race in the RPCSEC_GSS server code that caused crashes.
When a new client structure was allocated, it was added to the list
so that it was visible to other threads before the expiry time was
initialized, with only a single reference count.
The caller would increment the reference count, but it was possible
for another thread to decrement the reference count to zero and free
the structure before the caller incremented the reference count.
This could occur because the expiry time was still set to zero when
the new client structure was inserted in the list and the list was
unlocked.
This patch fixes the race by initializing the reference count to two
and initializing all fields, including the expiry time, before inserting
it in the list.
1. Not all kernels have netmap(4) support. Check for netmap(4) support before
attempting to run the tests via the `PLAIN_REQUIRE_KERNEL_MODULE(..)` macro.
2. Libraries shouldn't be added to LDFLAGS; they should be added to LIBADD
instead. This allows the build system to evaluate dependencies for sanity.
3. Sort some of the Makefile variables per bsd.README.
1., in particular, will resolve failures when running this testcase on kernels
lacking netmap(4) support, e.g., the i386 GENERIC kernels on ^/stable/11 and
^/stable/12.
MFC r344642 (by imp):
Unconditionally support unmapped BIOs. This was another shim for
supporting older kernels. However, all supported versions of FreeBSD
have unmapped I/Os (as do several that have gone EOL), remove it. It's
unlikely the driver would work on the older kernels anyway at this
point.
MFC r344640 (by imp):
Remove #ifdef code to support FreeBSD versions that haven't been
supported in years. A number of changes have been made to the driver
that likely wouldn't work on those older versions that aren't properly
ifdef'd and it's project policy to GC such code once it is stale.
MFC r342862 (by chuck): Add NVMe drive to NOIOB quirk list
Dell-branded Intel P4600 NVMe drives benefit from NVMe 1.3's NOIOB
feature. Unfortunately just like Intel DC P4500s, they don't advertise
themselves as benefiting from this...
This changes adds P4600s to the existing list of old drives which
benefit from striping.
MFC r339775: Put a workaround in for command timeout malfunctioning
At least one NVMe drive has a bug that makeing the Command Time Out
PCIe feature unreliable. The workaround is to disable this
feature. The driver wouldn't deal correctly with a timeout anyway.
Only do this for drives that are known bad.
MFC r337273 (by jhibbits):
nvme(4): Add bus_dmamap_sync() at the end of the request path
Summary:
Some architectures, in this case powerpc64, need explicit synchronization
barriers vs device accesses.
Prior to this change, when running 'make buildworld -j72' on a 18-core
(72-thread) POWER9, I would see controller resets often. With this change, I
don't see these resets messages, though another tester still does, for yet to be
determined reasons, so this may not be a complete fix. Additionally, I see a
~5-10% speed up in buildworld times, likely due to not needing to reset the
controller.
MFC r345264:
Add NAT64 CLAT implementation as defined in RFC6877.
CLAT is customer-side translator that algorithmically translates 1:1
private IPv4 addresses to global IPv6 addresses, and vice versa.
It is implemented as part of ipfw_nat64 kernel module. When module
is loaded or compiled into the kernel, it registers "nat64clat" external
action. External action named instance can be created using `create`
command and then used in ipfw rules. The create command accepts two
IPv6 prefixes `plat_prefix` and `clat_prefix`. If plat_prefix is ommitted,
IPv6 NAT64 Well-Known prefix 64:ff9b::/96 will be used.
# ipfw nat64clat CLAT create clat_prefix SRC_PFX plat_prefix DST_PFX
# ipfw add nat64clat CLAT ip4 from IPv4_PFX to any out
# ipfw add nat64clat CLAT ip6 from DST_PFX to SRC_PFX in
Obtained from: Yandex LLC
Submitted by: Boris N. Lytochkin
Relnotes: yes
Sponsored by: Yandex LLC
Add second IPv6 prefix to generic config structure and rename another
fields to conform to RFC6877. Now it contains two prefixes and length:
PLAT is provider-side translator that translates N:1 global IPv6 addresses
to global IPv4 addresses. CLAT is customer-side translator (XLAT) that
algorithmically translates 1:1 IPv4 addresses to global IPv6 addresses.
Use PLAT prefix in stateless (nat64stl) and stateful (nat64lsn)
translators.
Modify nat64_extract_ip4() and nat64_embed_ip4() functions to accept
prefix length and use plat_plen to specify prefix length.
Retire net.inet.ip.fw.nat64_allow_private sysctl variable.
Add NAT64_ALLOW_PRIVATE flag and use "allow_private" config option to
configure this ability separately for each NAT64 instance.
MFC r339542:
Retire IPFIREWALL_NAT64_DIRECT_OUTPUT kernel option. And add ability
to switch the output method in run-time. Also document some sysctl
variables that can by changed for NAT64 module.
NAT64 had compile time option IPFIREWALL_NAT64_DIRECT_OUTPUT to use
if_output directly from nat64 module. By default is used netisr based
output method. Now both methods can be used, but they require different
handling by rules.
MFC r341471:
Reimplement how net.inet.ip.fw.dyn_keep_states works.
Turning on of this feature allows to keep dynamic states when parent
rule is deleted. But it works only when the default rule is
"allow from any to any".
Now when rule with dynamic opcode is going to be deleted, and
net.inet.ip.fw.dyn_keep_states is enabled, existing states will reference
named objects corresponding to this rule, and also reference the rule.
And when ipfw_dyn_lookup_state() will find state for deleted parent rule,
it will return the pointer to the deleted rule, that is still valid.
This implementation doesn't support O_LIMIT_PARENT rules.
The refcnt field was added to struct ip_fw to keep reference, also
next pointer added to be able iterate rules and not damage the content
when deleted rules are chained.
Named objects are referenced only when states are going to be deleted to
be able reuse kidx of named objects when new parent rules will be
installed.
ipfw_dyn_get_count() function was modified and now it also looks into
dynamic states and constructs maps of existing named objects. This is
needed to correctly export orphaned states into userland.
ipfw_free_rule() was changed to be global, since now dynamic state can
free rule, when it is expired and references counters becomes 1.
External actions subsystem also modified, since external actions can be
deregisterd and instances can be destroyed. In these cases deleted rules,
that are referenced by orphaned states, must be modified to prevent access
to freed memory. ipfw_dyn_reset_eaction(), ipfw_reset_eaction_instance()
functions added for these purposes.
MFC r341472:
Add ability to request listing and deleting only for dynamic states.
This can be useful, when net.inet.ip.fw.dyn_keep_states is enabled, but
after rules reloading some state must be deleted. Added new flag '-D'
for such purpose.
Retire '-e' flag, since there can not be expired states in the meaning
that this flag historically had.
Also add "verbose" mode for listing of dynamic states, it can be enabled
with '-v' flag and adds additional information to states list. This can
be useful for debugging.
MFC r344018:
Remove `set' field from state structure and use set from parent rule.
Initially it was introduced because parent rule pointer could be freed,
and rule's information could become inaccessible. In r341471 this was
changed. And now we don't need this information, and also it can become
stale. E.g. rule can be moved from one set to another. This can lead
to parent's set and state's set will not match. In this case it is
possible that static rule will be freed, but dynamic state will not.
This can happen when `ipfw delete set N` command is used to delete
rules, that were moved to another set.
To fix the problem we will use the set number from parent rule.
MFC r344870:
Fix the problem with O_LIMIT states introduced in r344018.
dyn_install_state() uses `rule` pointer when it creates state.
For O_LIMIT states this pointer actually is not struct ip_fw,
it is pointer to O_LIMIT_PARENT state, that keeps actual pointer
to ip_fw parent rule. Thus we need to cache rule id and number
before calling dyn_get_parent_state(), so we can use them later
when the `rule` pointer is overrided.
MFC r342908:
Reduce the size of struct ip_fw_args from 240 to 128 bytes on amd64.
And refactor the code to avoid unneeded initialization to reduce overhead
of per-packet processing.
ipfw(4) can be invoked by pfil(9) framework for each packet several times.
Each call uses on-stack variable of type struct ip_fw_args to keep the
state of ipfw(4) processing. Currently this variable has 240 bytes size
on amd64. Each time ipfw(4) does bzero() on it, and then it initializes
some fields.
glebius@ has reported that they at Netflix discovered, that initialization
of this variable produces significant overhead on packet processing.
After patching I managed to increase performance of packet processing on
simple routing with ipfw(4) firewalling to about 11% from 9.8Mpps up to
11Mpps (Xeon E5-2660 v4@ + Mellanox 100G card).
Introduced new field flags, it is used to keep track of what fields was
initialized. Some fields were moved into the anonymous union, to reduce
the size. They all are mutually exclusive. dummypar field was unused, and
therefore it is removed. The hopstore6 field type was changed from
sockaddr_in6 to a bit smaller struct ip_fw_nh6. And now the size of struct
ip_fw_args is 128 bytes.
ipfw_chk() was modified to properly handle ip_fw_args.flags instead of
rely on checking for NULL pointers.
MFC r343551:
Fix the bug introduced in r342908, that causes problems with dynamic
handling for protocols without ports numbers.
Since port numbers were uninitialized for protocols like ICMP/ICMPv6,
ipfw_chk() used some non-zero values to create dynamic states, and due
this it failed to match replies with created states.
Reported by: Oliver Hartmann, Boris Lytochkin
Obtained from: Yandex LLC
Release notes documentation:
- r336040, jail(8) name support for cpuset(1), sockstat(1), ipfw(8),
and ugidfw(8).
- r336328, newfs_msdos(8) '-T' (timestamp) option added.
- r337461, ipfw(8) new rule options added.
- r338364, dd(1) status=progress support.
- r338451, last(1) libxo(3) support.
- r339160, diff(1) '-B' and '--ignore-blank-lines' support.
- r341758, bhyve(8) support for NumLock, ScrollLock and keypad keys.
- r342706, ktrdump(8) '-l' (live) support.
- r343538, newfs(8) and tunefs(8) support for '_' in label names.
- r343251, gzip(1) '-l' xz(1) support.
- r344020, pfctl(8) reference to net.pf.request_maxcount if a table
definition fails.
- r344490, fdisk(8) support for sectors larger than 2048 bytes.
- r344052, newfs(8) and tunefs(8) support for '-' in label names.
- r345561, sh(1) '-o pipefail'.
- r345878, patch(1) exit successfully if fed a 0-length patch.
Release notes documentation:
- r340611, rc.initdiskless support for auxiliary RAM.
- r340966, rcorder(8) rc.resume support.
- r341792, jail.conf(5) definition moved in rc.d/jail script.
- r341794, 340.noid periodic(8) no longer decends to jail(8)s.
- r342103, PATH now set in system crontab(5).
- r343046, rc_service addition to rc.subr(8).
- r343469, devd.conf(5) update to prevent duplicate hostapd(8)
and wpa_supplicant(8) startup.
MFC r344936: MFV/ZoL: Disable LBA weighting on files and SSDs
The LBA weighting makes sense on rotational media where the outer tracks
have twice the bandwidth of the inner tracks. However, it is detrimental
on nonrotational media such as solid state disks, where the only effect
is to ensure that metaslabs enter the best-fit allocation behavior
sooner, which is detrimental to performance. It also makes no sense on
files where the underlying filesystem can arrange things however it
wants.
Author: Richard Yao <ryao@gentoo.org> Signed-off-by: Richard Yao <ryao@gentoo.org> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3712
zfsonlinux/zfs@fb40095f5f0853946f8150481ca22602d1334dfe
To reduce code divergence this merge replaces equivalent but different
FreeBSD code detecting non-rotating medium vdevs.
MFC r344934, r345014: Add separate aggregation limit for non-rotating media.
Before sequential scrub patches ZFS never aggregated I/Os above 128KB.
Sequential scrub bumped that to 1MB, which motivation I understand for
spinning disks, since it should reduce number of head seeks. But for
SSDs it makes much less sense to me, especially on FreeBSD, where due
to MAXPHYS limitation device will likely still see bunch of 128KB I/Os
instead of one large. Having more strict aggregation limit allows to
avoid allocation of large memory buffer and memcpy to/from it, that is
a serious problem when bandwidth reaches few GB/s.
Update the bounds checking for zfs_vdev_aggregation_limit so that
it has a floor of zero and a maximum value of the supported block
size for the pool.
Additionally add an early return when zfs_vdev_aggregation_limit
equals zero to disable aggregation. For very fast solid state or
memory devices it may be more expensive to perform the aggregation
than to issue the IO immediately.
Commit 8542ef8 allowed optional IOs to be aggregated beyond
the specified aggregation limit. Since the aggregation limit
was also used to enforce the maximum block size, setting
`zfs_vdev_aggregation_limit=16777216` could result in an
attempt to allocate an ABD larger than 16M.
Author: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #6259
Closes #6270
zfsonlinux/zfs@2d678f779aba26a93314c8ee1142c3985fa25cb6
mm [Wed, 10 Apr 2019 21:46:06 +0000 (21:46 +0000)]
MFC r345497:
Sync libarchive with vendor.
Relevant vendor changes:
PR #1153: fixed 2 bugs in ZIP reader [1]
PR #1143: ensure archive_read_disk_entry_from_file() uses ARCHIVE_READ_DISK
Changes to file flags code, support more file flags on FreeBSD:
UF_OFFLINE, UF_READONLY, UF_SPARSE, UF_REPARSE, UF_SYSTEM
UF_ARCHIVE is not supported by intention (yet)
MFC r344161: stand: dev_net: correct net_open's interpretation of params
net_open previously casted the first vararg to a char * and this was
half-OK: at first, it is passed to netif_open, which would cast it back to
the struct devdesc * that it really is and use it properly. It is then
strdup()d and used as the netdev_name, which is objectively wrong.
Correct it so that the first vararg is properly casted to a struct devdesc *
and the netdev_name gets set properly to make it more clear at a glance that
it's not doing something horribly wrong.
freebsd32: fix padding of computed control message length for recvmsg()
Each control message region must be aligned on a 4-byte boundary on 32-bit
architectures. The 32-bit compat shim for recvmsg() gets the actual layout
right, but doesn't pad the payload length when computing msg_controllen for
the output message header. If a control message contains an unaligned
payload, such as the 1-byte TTL field in the example attached to PR 236737,
this can produce control message payload boundaries that extend beyond
the boundary reported by msg_controllen.
Backport fixes from FreeBSD-12 to help the random(4) device thread
not overwhelm the OS:
a) Use the correct symbolic constant when calculating 10'ths of a
second. This means that expensive reseeds happen at ony 1/10 Hz,
not some kHz.
b) Rate limit internal high-rate harveting efforts. This stops the
harvesting thread from total overkilling the high-grade entropy-
gathering work, while still being very conservatively safe.
PR: 230808
Reported by: danilo,eugen
Tested by: eugen
Approved by: so (blanket permission granted as I am the authour of this code)
Relnotes: Yes
MFC r344243, r345517-r345518: lualoader: More intelligent screen clearing
r344243:
lualoader: only clear the screen before first password prompt
This was previously an unconditional screen clear, regardless of whether or
not we would be prompting for any passwords. This is pointless, given that
the screen clear is only there to put our screen into a consistent state
before we draw the prompts and do cursor manipulation.
This is also the only screen clear besides that to draw the menu. One can
now see early pre-loader and loader output with the menu disabled, which may
be useful for diagnostics.
r345517:
lualoader: Clear the screen before prompting for password
Assuming that the autoboot sequence was interrupted, we've done enough
cursor manipulation that the prompt for the password will be sufficiently
obscured a couple of lines up. Clear the screen and reset the cursor
position here, too.
r345518:
lualoader: Fix up some luacheck concerns
- Garbage collect an unused (removed because it was useless) constant
- Don't bother with vararg notation if args will not be used