emaste [Fri, 5 Oct 2018 20:49:54 +0000 (20:49 +0000)]
libcrypto: have buildinf.h depend on Makefile
So that it will be regenerated after Makefile changes affecting the
file's content - specifically, the OpenSSL 1.1.1 update adds a DATE
macro which did not exist previously.
kevans [Fri, 5 Oct 2018 17:07:10 +0000 (17:07 +0000)]
lualoader: Don't draw loader menu with autoboot_delay=-1
This was mostly a cosmetic issue. autoboot_delay=-1 is documented to bypass
the loader menu and immediately execute the boot command, but lualoader
would draw the menu and immediately execute the boot command. No interaction
was possible with the menu.
The fix lifts autoboot_delay processing out of menu.autoboot, which now
takes a delay and does nothing if no delay is specified. This lines up with
my expectations of menu.autoboot's usage from a third party, which may
want more control over the process than the default behavior.
jhb [Fri, 5 Oct 2018 16:35:24 +0000 (16:35 +0000)]
Update the existing heimdal implementation for OpenSSL 1.1.
Existing work is underway to import a newer version of heimdal, but
this patchset gets us to a fully working tree to enable more wide
spread testing of OpenSSL 1.1 for now.
I've also enabled WARNS=1 for kerberos (which is the reason for the
change in libroken). Having -Werror enabled was useful during the
1.1 updates and we probably should have warnings enabled by default
for kerberos anyway.
This passes make tinderbox, and I have also done some very light
runtime testing on amd64.
0mp [Fri, 5 Oct 2018 08:17:29 +0000 (08:17 +0000)]
bsdinstall(8): Fix a typo.
Reported by: Jose Luis Duran
Reviewed by: bcr
Approved by: re (gjb), krion (mentor, implicit), mat (mentor, implicit)
Differential Revision: https://reviews.freebsd.org/D17409
mmacy [Fri, 5 Oct 2018 05:55:56 +0000 (05:55 +0000)]
hwpmc: Refactor sample ring buffer handling to fix races
Refactor sample ring buffer ring handling to make it more robust to
long running callchain collection handling
r338112 introduced a (now fixed) regression that exposed a number of race
conditions within the management of the sample buffers. This
simplifies the handling and moves the decision to overwrite a
callchain sample that has taken too long out of the NMI in to the
hardlock handler. With this change the problem no longer shows up as a
ring corruption but as the code spending all of its time in callchain
collection.
- Makes the producer / consumer index incrementing monotonic, making it
easier (for me at least) to reason about.
- Moves the decision to overwrite a sample from NMI context to interrupt
context where we can enforce serialization.
- Puts a time limit on waiting to collect a user callchain - putting a
bound on head-of-line blocking causing samples to be dropped
- Removes the flush routine which was previously needed to purge
dangling references to the pmc from the sample buffers but now is only
a source of a race condition on unload.
Previously one could lock up or crash HEAD by running:
pmcstat -S inst_retired.any_p -T and then hitting ^C
mmacy [Fri, 5 Oct 2018 05:50:56 +0000 (05:50 +0000)]
eliminate locking surrounding ui_vmsize and swap reserve by using atomics
Change swap_reserve and swap_total to be in units of pages so that
swap reservations can be done using only atomics instead of using a single
global mutex for swap_reserve and a single mutex for all processes running
under the same uid for uid accounting.
Results in mmap speed up and a 70% increase in brk calls / second.
rstone [Thu, 4 Oct 2018 22:03:58 +0000 (22:03 +0000)]
Hold a write lock across udp_notify()
With the new route cache feature udp_notify() will modify the inp when it
needs to invalidate the route cache. Ensure that we hold a write lock on
the inp before calling the function to ensure that multiple threads don't
race while trying to invalidate the cache (which previously lead to a page
fault).
brooks [Thu, 4 Oct 2018 21:55:58 +0000 (21:55 +0000)]
Reduce NL_ARGMAX to 4096 to match Linux.
NL_ARGMAX is the maximum number of positional arguments supported by
printf(3). Prior to r308145 it was declared as 99 and not enforced.
r308145 added enforcement and increased the value to 64k.
Unfortunately, development versions of PostgreSQL used the system
definition to allocate and zero an NL_ARGMAX * 4 sized array on the
stack of its snprintf implementation with measurable performance
impacts. This has been fixed in new PostgreSQL versions, but it is
possible that other programs suffer from this problem.
A value of 4096 puts us on par with Linux and is certainly large enough
for any reasonable program.
- Extend the bsdinstall(8) man page with ZFS installation scripting
details. [1]
- Extend the bsdinstall(8) man page with the description of all the ZFS
variables involved in a scripted installation of ZFS-based systems. [1]
- Extend the SCRIPTING section with an example for a ZFS-based scripted
installation. [1]
- Create a new section explaining how ZFS datasets must be written into
a variable to get them set on the final system. [1]
While here:
- Add Roberto to the copyrights for recognition as changes to the manual
page are huge.
- Use "Dq" for default values.
- Use sysrc(8) instead of echo in examples.
pjd [Thu, 4 Oct 2018 05:54:57 +0000 (05:54 +0000)]
When the adist_free list is empty and we lose connection to the receiver we
move all elements from the adist_send and adist_recv lists back onto the
adist_free list, but we don't wake consumers waitings for the adist_free list
to become non-empty. This can lead to the sender process stopping audit trail
files distribution and waiting forever.
Fix the problem by adding the missing wakeup.
While here slow down spinning on CPU in case of a short race in
sender_disconnect() and add an explaination when it can occur.
PR: 201953
Reported by: peter
Approved by: re (kib)
pjd [Thu, 4 Oct 2018 05:48:09 +0000 (05:48 +0000)]
When we look for a new trail file there might be a race between find trail
file name and opening it. This race was not properly handled, because we were
copying new name before checking for openat(2) error and when we were trying
again we were starting with the next trail file. This could result in skipping
distribution of such a trail file.
Fix this problem by checking for ENOENT first (only for .not_terminated files)
and then updating (or not) tr_filename before restarting the search.
PR: 200139
Reported by: peter
Approved by: re (kib)
gjb [Thu, 4 Oct 2018 01:46:56 +0000 (01:46 +0000)]
Fix the hardware.{txt,html} build in the release/Makefile 'reldoc'
target.
The doc/share/mk/doc.commands.mk sets SVN to /usr/local/bin/svn
by default, which is not necessarily installed by the documentation
project textproc/docproj port.
Ensure SVN can be evaluated properly to include the hardware pages
by iterating through /usr/local/bin and /usr/bin and looking for
both svn and svnlite binaries, and pass the SVN variable explicitly
through env(1) in the reldoc target to avoid failures if it does not
exist.
Approved by: re (rgrimes)
Sponsored by: The FreeBSD Foundation
brooks [Wed, 3 Oct 2018 20:39:48 +0000 (20:39 +0000)]
Move 32-bit compat support for FIODGNAME to the right place.
ioctl(2) commands only have meaning in the context of a file descriptor
so translating them in the syscall layer is incorrect.
The new handler users an accessor to retrieve/construct a pointer from
the last member of the passed structure and relies on type punning to
access the other member which requires no translation.
glebius [Wed, 3 Oct 2018 17:40:04 +0000 (17:40 +0000)]
In PR 227259, a user is reporting that they have code which is using
shutdown() to wakeup another thread blocked on a stream listen socket.
This code is failing, while it used to work on FreeBSD 10 and still
works on Linux.
It seems reasonable to add another exception to support something users are
actually doing, which used to work on FreeBSD 10, and still works on Linux.
And, it seems like it should be acceptable to POSIX, as we still return
ENOTCONN.
This patch is different to what had been committed to stable/11, since
code around listening sockets is different. Patch in D15019 is written
by jtl@, slightly modified by me.
brooks [Tue, 2 Oct 2018 23:23:56 +0000 (23:23 +0000)]
Move 32-bit compat support for CDIOREADTOCENTRYS to the right place.
ioctl(2) commands only have meaning in the context of a file descriptor
so translating them in the syscall layer is incorrect.
The new handler users an accessor to retrieve/construct a pointer from
the last member of the passed structure and relies on type punning to
access the other members which require no translation.
jhb [Tue, 2 Oct 2018 21:40:57 +0000 (21:40 +0000)]
Update obsolete files list for OpenSSL 1.1.1.
This will need a real date once this is merged to head.
One weird thing to note: the 32-bit engines get dumped into /usr/lib32
rather than /usr/lib32/engines, and I bet the 32-bit libcrypto.so i
looking for the .so files in the wrong place. We should probably fix
both of those at some point.
After r338257 is was possible to trigger a KASSERT() in ud6_output()
using an application trying to use a v4mapped destination address on a
kernel without INET support or on a v6only socket.
Catch this case and prevent the packet from going anywhere;
else, without the KASSERT() armed, a v4mapped destination
address might go out on the wire or other undefined behaviour
might happen, while with the KASSERT() we panic.
PR: 231728
Reported by: Jeremy Faulkner (gldisater gmail.com)
Approved by: re (kib)
rwatson [Tue, 2 Oct 2018 15:58:17 +0000 (15:58 +0000)]
Rework the logic around quick checks for auditing that take place at
system-call entry and whenever audit arguments or return values are
captured:
1. Expose a single global, audit_syscalls_enabled, which controls
whether the audit framework is entered, rather than exposing
components of the policy -- e.g., if the trail is enabled,
suspended, etc.
2. Introduce a new function audit_syscalls_enabled_update(), which is
called to update audit_syscalls_enabled whenever an aspect of the
policy changes, so that the value can be updated.
3. Remove a check of trail enablement/suspension from audit_new() --
at the point where this function has been entered, we believe that
system-call auditing is already in force, or we wouldn't get here,
so simply proceed to more expensive policy checks.
4. Use an audit-provided global, audit_dtrace_enabled, rather than a
dtaudit-provided global, to provide policy indicating whether
dtaudit would like system calls to be audited.
5. Do some minor cosmetic renaming to clarify what various variables
are for.
These changes collectively arrange it so that traditional audit
(trail, pipes) or the DTrace audit provider can enable system-call
probes without the other configured. Otherwise, dtaudit cannot
capture system-call data without auditd(8) started.
emaste [Tue, 2 Oct 2018 15:08:41 +0000 (15:08 +0000)]
libelf: correct mips64el test to use ELF header
libelf maintains two views of endianness: e_byteorder, and
e_ident[EI_DATA] in the ELF header itself. e_byteorder is not always
kept in sync, so use the ELF header endianness to test for mips64el.
PR: 231790
Bisected by: sbruno
Reviewed by: jhb
Approved by: re (kib)
MFC with: r338478
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17380
0mp [Tue, 2 Oct 2018 08:13:54 +0000 (08:13 +0000)]
top(1): Rework DESCRIPTION OF MEMORY section.
Due to markup issues, the DESCRIPTION OF MEMORY section is rather
unreadable; rework it a bit, using subsections for different lines of the
top output, and move it closer to description.
While here, pet manlint ordering other sections as expected.
andreast [Mon, 1 Oct 2018 18:46:35 +0000 (18:46 +0000)]
This commit reverts 338930. The approach was wrong.
Fix the issue with subtracting the TLS_TCB_SIZE too when we are trying to get
the 'where' in the R_PPC_TPREL32 case. At allocation time we added an offset
and the TLS_TCB_SIZE. This has to be subtracted as well.
Now all the issues reported are fixed. Tests were done on G4 and G5 PowerMac's.
Additionally I ran the tls tests from the gcc test suite and made sure the
results are as good as pre 338486.
Thanks to tuexen for reporting the malfunction and for patient testing.
Also testing thanks goes to jhibbits.
Reported by: tuexen
Discussed with: jhibbits, nwhitehorn
Approved by: re (gjb)
Pointyhat to: andreast
gallatin [Mon, 1 Oct 2018 14:14:21 +0000 (14:14 +0000)]
Allow empty NUMA memory domains to support Threadripper2
The AMD Threadripper 2990WX is basically a slightly crippled Epyc.
Rather than having 4 memory controllers, one per NUMA domain, it has
only 2 memory controllers enabled. This means that only 2 of the
4 NUMA domains can be populated with physical memory, and the
others are empty.
Add support to FreeBSD for empty NUMA domains by:
- creating empty memory domains when parsing the SRAT table,
rather than failing to parse the table
- not running the pageout deamon threads in empty domains
- adding defensive code to UMA to avoid allocating from empty domains
- adding defensive code to cpuset to avoid binding to an empty domain
Thanks to Jeff for suggesting this strategy.
tuexen [Mon, 1 Oct 2018 14:05:31 +0000 (14:05 +0000)]
Mitigate providing a timing signal if the COOKIE or AUTH
validation fails.
Thanks to jmg@ for reporting the issue, which was discussed in
https://admbugs.freebsd.org/show_bug.cgi?id=878
andrew [Mon, 1 Oct 2018 14:02:29 +0000 (14:02 +0000)]
Add STT_GNU_IFUNC and R_AARCH64_IRELATIVE support on arm64.
This is based on the amd64 implementation. Support for both PLT and
non-PLT (e.g. a global variable initilised with a pointer to an ifunc)
cases are supported.
We don't pass anything to the resolver as it is expected they will read
the ID registers directly, with the number of registers with CPU info
likely to increase in the future.
Reviewed by: kib
Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D17341
tuexen [Mon, 1 Oct 2018 13:09:18 +0000 (13:09 +0000)]
After allocating chunks set the fields in a consistent way.
This removes two assignments for the flags field being done
twice and adds one, which was missing.
Thanks to Felix Weinrank for reporting the issue he found
by using fuzz testing of the userland stack.
Add INP_INFO_WUNLOCK_ASSERT() macro and use it instead of
INP_INFO_UNLOCK_ASSERT() in TCP-related code. For encapsulated traffic
it is possible, that the code is running in net_epoch_preempt section,
and INP_INFO_UNLOCK_ASSERT() is very strict assertion for such case.
Fix the MODULE_PNP_INFO() for iwm(4) where I got the bus and module
arguments wrong in r339020.
PR: 231625
Reported by: Yuri Pankov (yuripv yuripv.net)
Reviewed by: cem, Yuri Pankov (yuripv yuripv.net)
Approved by: re (kib)
Pointyhat to: bz (a rather big one for this one)
Fix the handling of ancillary data for SCTP socket. Implement
sctp_process_cmsgs_for_init() and sctp_findassociation_cmsgs()
similar to sctp_find_cmsg() to improve consistency and avoid
the signed/unsigned issues in sctp_process_cmsgs_for_init()
and sctp_findassociation_cmsgs().
Thanks to andrew@ for reporting the problem he found using
syzcaller.
Increment the corresponding UDP stats counter (udps_opackets) when
sending UDP encapsulated SCTP packets.
This is consistent with the behaviour that when such packets are received,
the corresponding UDP stats counter (udps_ipackets) is incremented.
Thanks to Peter Lei for making me aware of this inconsistency.
Previously Clang required ifunc resolution functions to take no
arguments, presumably because GCC documented ifunc resolvers as taking
no arguments. However, GCC accepts resolvers accepting arguments, and
our rtld passes CPU ID information (cpuid, hwcap, etc.) to ifunc
resolvers. Just remove the check from the in-tree compiler for our in-
tree compiler; a different (per-OS) approach may be required upstream.
Reported by: mjg
Approved by: re (rgrimes)
MFC after: 1 week
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
This fixes a regression in lld that made it stop emitting STT_FILE
symbols, which ctfmerge relies upon to uniquify function table entries
that reference STB_LOCAL symbols. Consequently, ctfmerge stopped
emitting entries for static functions into the function table, and
dtrace no longer gets type info for them.
Approved by: re (kib)
Reported by: markj
PR: 230444
MFC after: 3 days
For changing the MTU on tun/tap devices, it should not matter whether it
is done via using ifconfig, which uses a SIOCSIFMTU ioctl() command, or
doing it using a TUNSIFINFO/TAPSIFINFO ioctl() command.
Without this patch, for IPv6 the new MTU is not used when creating routes.
Especially, when initiating TCP connections after increasing the MTU,
the old MTU is still used to compute the MSS.
Thanks to ae@ and bz@ for helping to improve the patch.
Reviewed by: ae@, bz@
Approved by: re (kib@)
MFC after: 1 week
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D17180
This fixes an assert in vdev_queue_change_io_priority():
VERIFY3(zio->io_priority < ZIO_PRIORITY_NUM_QUEUEABLE) failed (7 < 6)
PANIC at vdev_queue.c:832:vdev_queue_change_io_priority()
Reviewed-by: Tom Caputi <tcaputi@datto.com> Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tony Hutter <hutter2@llnl.gov>
Mark various removed system calls as OBSOL instead of UNIMPL.
This is mostly a cosmetic change except that obsolete system calls are
assigned meaningful names in the names arrays which means that using
tools like kdump or truss against binaries invoking these system calls
will print out the name instead of the number. The script I use to
generate the XML list of syscalls for GDB also ignores UNIMPL but not
OBSOL entries. In general UNIMPL should only be used to reserve
placeholders for system calls that have never been implemented while
system calls that existed at one time in FreeBSD but were removed
should be marked OBSOL instead.
Correct vm_fault_copy_entry() handling of backing file truncation
after the file mapping was wired.
if a wired map entry is backed by vnode and the file is truncated,
corresponding pages are invalidated. vm_fault_copy_entry() should be
aware of it and allow for invalid pages past end of file. Also, such
pages should be not mapped into userspace. If userspace accesses the
truncated part of the mapping later, it gets a signal, there is no way
kernel can prevent the page fault.
Reported by: andrew using syzkaller
Reviewed by: alc
Sponsored by: The FreeBSD Foundation
Approved by: re (gjb)
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D17323
In vm_fault_copy_entry(), we should not assert that entry is charged
if the dst_object is not of swap type.
It can only happen when entry does not require copy, otherwise
vm_map_protect() already adds the charge. So the assert was right for
the case where swap object was allocated in the vm_fault_copy_entry(),
but not when it was just copied from src_entry and its type is not
swap.
Reported by: andrew using syzkaller
Reviewed by: alc
Sponsored by: The FreeBSD Foundation
Approved by: re (gjb)
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D17323
Revert part of the r338891 which reordered local invalidation and IPI.
For PCID case, there is a dependency between pm_gen zeroing and
reading pm_active for IPI target selection, to ensure that the
invalidation is not missed.
Reported and tested by: mjg
Sponsored by: The FreeBSD Foundation
Approved by: re (gjb)
andrew [Fri, 28 Sep 2018 11:57:40 +0000 (11:57 +0000)]
Export ID_AA64ISAR{0,1}_EL1 to userland.
As with r338962 also export the instruction set attribute register. This
will allow userland to identify optional instructions the hardware
supports, for example in a future ifunc handler to decide which
implementation of a function to return.
When multiple threads are involved receiving completion events in LibUSB
make sure there is always a master polling thread, by setting the "ctx_handler"
field in the context. Else the reception of completion events can stop.
This happens if event threads are created and destroyed during runtime.
Found by: Ludovic Rousseau <ludovic.rousseau+freebsd@gmail.com>
PR: 231742
MFC after: 1 week
Approved by: re (kib)
Sponsored by: Mellanox Technologies
The pre-7.x compat for both native and 32-bit code was already in
pci_user.c. Use this infrastructure to add implement 32-bit support.
This is more correct as ioctl(2) commands only have meaning in the
context of a file descriptor.
Add description, parameters, options, sysctl and examples of using AQMs to ipfw man page. CoDel, PIE, FQ-CoDel and FQ-PIE AQM for Dummynet exist in FreeBSD 11 and 10.3.
Don't clear DR6 for debug exceptions from userland.
This reverts part of r333368. The attempt to clear DR6 was occuring
too soon as trapsignal() does not pause to let the debugger notice the
SIGTRAP and query DR6. The signal exchange does not occur until much
later during ast(). As a result, GDB was no longer recognizing
hardware breakpoints and watchpoints on x86.
In addition, any userland programs that want to inspect DR6 in a
SIGTRAP handler don't have a way to do this if we clear DR6 in the
exception handler.
Instead of relying on the kernel to clear DR6, debuggers will have to
explicitly clear it after a trace trap (which they needed to do on
older kernels anyway).
Reviewed by: kib
Approved by: re (delphij)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D17319
amd64: reimplement libc memcmp and bcmp with kernel memcmp
Both are significantly slower than hand-coded loops. See r338963 for
kernel commit.
bcmp differs from memcmp by always returning 1 when a difference is
found, as opposed to going for a value bigger or lower than 0
depending on what it is. This means it can do less work. For now the
code is duplicated and modified. This will get deduplicated after
another round of optimization when memcmp will get a longer-term form.
Both tested with the glibc suite. While the suite does not have a test
for bcmp, I created a wrapper routine which verified that values match
(0 vs 0, 1 vs non-zero).
Reviewed by: kib
Approved by: re (gjb)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17336
In in6_pcbpurgeif0() called, e.g., from if_clone_destroy(),
once we have a lock, make sure the inp is not marked freed.
This can happen since the list traversal and locking was
converted to epoch(9). If the inp is marked "freed", skip it.
This prevents a NULL pointer deref panic later on.
Reported by: slavash (Mellanox)
Tested by: slavash (Mellanox)
Reviewed by: markj (no formal review but caught my unlock mistake)
Approved by: re (kib)
- move the PSL.AC comment to the fault handler
- stop testing for zero-sized ops. after several minutes of package
building there were no copyin calls with zero bytes and very few
copyout. the semantic of returning 0 in this case is preserved
- shorten exit paths by clearing %eax earlier
- replace xchg with 3 movs. this is what compilers do. a naive
benchmark on EPYC suggests about 1% increase in thoughput thanks to
this change.
- remove the useless movb %cl,%al from copyout. it looks like a
leftover from many years ago
Reviewed by: kib
Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D17286
- Fix "mandoc -Tlint" warnings.
- Use the "Er" macro for errors.
- Add an extra newline for readability.
- Reference m_pullup(9) and m_defrag(9).
- Use the "Fx" macro when talking about FreeBSD versions.
Remove -m (update) from ldconfig -32 & -soft invocation on startup.
Since r154114 which introduced ldconfig_local32_dirs, ldconfig -32 was
called with -m. This means that ld-elf32.so.hints paths set is not
cleared for compat32 on boot, unlike ld,so,hints. Same -m was used in
r294295 for ld-elf-soft.so.hints on arm. The patch fixes the
asymmetry.
Noted by: Andreas Longwitz <longwitz@incore.de>
Reviewed by: brooks, emaste, imp
Discussed with: bdrewery
Sponsored by: The FreeBSD Foundation
Approved by: re (gjb)
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D17331
Both the in-kernel C variant and libc asm variant have very poor performance.
The former compiles to a single byte comparison loop, which breaks down even
for small sizes. The latter uses rep cmpsq/b which turn out to have very poor
throughput and are slower than a hand-coded 32-byte comparison loop.
Depending on size this is about 3-4 times faster than the current routines.
Reviewed by: kib
Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D17328
projects / FreeBSD / FreeBSD.git / log