Warner Losh [Fri, 23 Aug 2019 22:52:58 +0000 (22:52 +0000)]
It turns out the duplication is only mostly harmless.
While it worked with the kenrel, it wasn't working with the loader.
It failed to handle dependencies correctly. The reason for that is
that we never created a nvme module with the DRIVER_MODULE, but
instead a nvme_pci and nvme_ahci module. Create a real nvme module
that nvd can be dependent on so it can import the nvme symbols it
needs from there.
Arguably, nvd should just be a simple child of nvme, but transitioning
to that (and winning that argument given why it was done this way) is
beyond the scope of this change.
Mark Johnston [Fri, 23 Aug 2019 19:53:11 +0000 (19:53 +0000)]
Stop clearing page flags in vm_page_pqbatch_submit().
All existing callers guarantee that the page does not have a
pre-existing dequeue pending. Thus, if the page is dequeued before
pqbatch_submit() acquires the page queue lock, we do not need to do
anything since vm_page_dequeue_complete() takes care of clearing all
page queue state flags for us.
With this change, vm_page_pqbatch_submit() has the nice property that it
does not directly modify any fields in the page structure.
Mark Johnston [Fri, 23 Aug 2019 19:49:29 +0000 (19:49 +0000)]
Make vm_pqbatch_submit_page() externally visible.
It will become useful for the page daemon to be able to directly create
a batch queue entry for a page, and without modifying the page
structure. Rename vm_pqbatch_submit_page() to vm_page_pqbatch_submit()
to keep the namespace consistent. No functional change intended.
De-commision the MNTK_NOINSMNTQ kernel mount flag.
After all the changes, its dynamic scope is same as for MNTK_UNMOUNT,
but to allow the syncer vnode to be re-installed on unmount failure.
But the case of syncer was already handled by using the VV_FORCEINSMQ
flag for quite some time.
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
John Baldwin [Fri, 23 Aug 2019 18:26:34 +0000 (18:26 +0000)]
Fix universe to include arm LINT kernel configs.
Strip comments from the NOTES.armv[57] files as is done for other
NOTES files when building the corresponding LINT configs. Without
this, the LINT configs contained the NO_UNIVERSE comment from the
NOTES.armv[57] files.
Warner Losh [Fri, 23 Aug 2019 16:42:39 +0000 (16:42 +0000)]
Turn off -Werror for gcc 4.2.1 for userland
As discussed on arch@, gcc 4.2.1 is on its way out. Turn off Werror on gcc
versions < 5.0 permantly. This will allow older platforms to continue to compile
w/o new errors once we take them out of universe by default. This will also free
developers from chasing down obsolete warnings that produce no beneficial
changes to the source.
Warner Losh [Fri, 23 Aug 2019 16:42:04 +0000 (16:42 +0000)]
Turn off -Werror for gcc 4.2.1
As part of marching gcc 4.2.1 out of the tree, turn off -Werror on gcc 4.2.1
compiles by default. It generates too many false positives and breaks CI
for no benefit.
Alan Somers [Fri, 23 Aug 2019 15:22:20 +0000 (15:22 +0000)]
ping6: Rename options for better consistency with ping
Now equivalent options have the same flags, and nonequivalent options have
different flags. This is a prelude to merging the two commands.
Submitted by: Ján Sučan <sucanjan@gmail.com>
MFC: Never
Sponsored by: Google LLC (Google Summer of Code 2019)
Differential Revision: https://reviews.freebsd.org/D21345
Xin LI [Fri, 23 Aug 2019 06:39:40 +0000 (06:39 +0000)]
INVARIANTS: treat LA_LOCKED as the same of LA_XLOCKED in mtx_assert.
The Linux lockdep API assumes LA_LOCKED semantic in lockdep_assert_held(),
meaning that either a shared lock or write lock is Ok. On the other hand,
the timeout code uses lc_assert() with LA_XLOCKED, and we need both to
work.
For mutexes, because they can not be shared (this is unique among all lock
classes, and it is unlikely that we would add new lock class anytime soon),
it is easier to simply extend mtx_assert to handle LA_LOCKED there, despite
the change itself can be viewed as a slight abstraction violation.
Eugene Grosbein [Fri, 23 Aug 2019 01:25:38 +0000 (01:25 +0000)]
last(1): unbreak for 8-bit locales
Ouput format of last(1) is broken for non UTF-8 locales
since it got libxo(3) support. It uses strftime(3) that produces
non UTF-8 strings passed to xo_emit(3) with wrong %s format -
it should be %hs in this case, so xo_emit(3) produces empty output.
This change is basically no-op when locale is of UTF-8 type,
f.e. en_GB.UTF-8 or ru_RU.UTF-8 or sr_RS.UTF-8@latin.
It fixes output for other locales.
Warner Losh [Thu, 22 Aug 2019 21:56:11 +0000 (21:56 +0000)]
When we have errors resetting the device before we allocate the
queues, don't try to tear them down in the ctrlr_destroy
path. Otherwise, we dereference queue structures that are NULL and we
trap.
This fix is incomplete: we leak IRQ and MSI resources when this
happens. That's preferable to a crash but still should be fixed.
Kyle Evans [Thu, 22 Aug 2019 21:43:21 +0000 (21:43 +0000)]
mips: hide regnum definitions behind _KERNEL/_WANT_MIPS_REGNUM
machine/regnum.h ends up being included by sys/procfs.h and sys/ptrace.h via
machine/reg.h. Many of the regnum definitions are too short and too generic
to be exposing to any userland application including one of these two
headers. Moreover, these actively cause build failures in googletest
(template <typename T1 ...> expanding to template <typename 9 ...>).
Hide the definitions behind _KERNEL or _WANT_MIPS_REGNUM, and patch all of
the userland consumers to define as needed.
Warner Losh [Thu, 22 Aug 2019 21:12:51 +0000 (21:12 +0000)]
We need to define version 1 of nvme, not nvme_foo. Otherwise nvd won't
load and people who pull in nvme/nvd from modules can't load nvd.ko
since it depends on nvme, not nvme_foo. The duplicate doesn't matter
since kldxref properly handles that case.
Warner Losh [Thu, 22 Aug 2019 20:09:32 +0000 (20:09 +0000)]
Move releasing of resources to later
Turn off bus master after we detach the device (to match the prior
order). Release MSI after we're done detaching and have turned off
all the interrupts. Otherwise this may cause problems as other threads
race nvme_detach. This more closely matches the old order.
Kyle Evans [Thu, 22 Aug 2019 19:10:31 +0000 (19:10 +0000)]
Fix the build with WITHOUT_GOOGLETEST
Attempting to build the fusefs tests WITHOUT_GOOGLETEST will result in an
error if the host system or sysroot doesn't already have googletest headers
in /usr/include/private (e.g. host built/installed WITHOUT_GOOGLETEST, clean
cross-buildworld WITHOUT_GOOGLETEST).
Alan Somers [Thu, 22 Aug 2019 18:57:24 +0000 (18:57 +0000)]
ping: By default, don't reverse lookup IP addresses
ping's default is now not to attempt reverse DNS lookups. The -H flag will
enable them. This change is not quite a reversion of r351330. That change
made the happy path and error path do reverse lookups consistently; this
change changes the default for both paths.
Submitted by: Ján Sučan <sucanjan@gmail.com>
Discussed with: cem
MFC after: 2 weeks
MFC-With: 351330
Sponsored by: Google LLC (Google Summer of Code 2019)
Differential Revision: https://reviews.freebsd.org/D21364
Cy Schubert [Thu, 22 Aug 2019 18:52:30 +0000 (18:52 +0000)]
MFV r346563:
Update wpa 2.8 --> 2.9
hostapd:
* SAE changes
- disable use of groups using Brainpool curves
- improved protection against side channel attacks
[https://w1.fi/security/2019-6/]
* EAP-pwd changes
- disable use of groups using Brainpool curves
- improved protection against side channel attacks
[https://w1.fi/security/2019-6/]
* fixed FT-EAP initial mobility domain association using PMKSA caching
* added configuration of airtime policy
* fixed FILS to and RSNE into (Re)Association Response frames
* fixed DPP bootstrapping URI parser of channel list
* added support for regulatory WMM limitation (for ETSI)
* added support for MACsec Key Agreement using IEEE 802.1X/PSK
* added experimental support for EAP-TEAP server (RFC 7170)
* added experimental support for EAP-TLS server with TLS v1.3
* added support for two server certificates/keys (RSA/ECC)
* added AKMSuiteSelector into "STA <addr>" control interface data to
determine with AKM was used for an association
* added eap_sim_id parameter to allow EAP-SIM/AKA server pseudonym and
fast reauthentication use to be disabled
* fixed an ECDH operation corner case with OpenSSL
wpa_supplicant:
* SAE changes
- disable use of groups using Brainpool curves
- improved protection against side channel attacks
[https://w1.fi/security/2019-6/]
* EAP-pwd changes
- disable use of groups using Brainpool curves
- allow the set of groups to be configured (eap_pwd_groups)
- improved protection against side channel attacks
[https://w1.fi/security/2019-6/]
* fixed FT-EAP initial mobility domain association using PMKSA caching
(disabled by default for backwards compatibility; can be enabled
with ft_eap_pmksa_caching=1)
* fixed a regression in OpenSSL 1.1+ engine loading
* added validation of RSNE in (Re)Association Response frames
* fixed DPP bootstrapping URI parser of channel list
* extended EAP-SIM/AKA fast re-authentication to allow use with FILS
* extended ca_cert_blob to support PEM format
* improved robustness of P2P Action frame scheduling
* added support for EAP-SIM/AKA using anonymous@realm identity
* fixed Hotspot 2.0 credential selection based on roaming consortium
to ignore credentials without a specific EAP method
* added experimental support for EAP-TEAP peer (RFC 7170)
* added experimental support for EAP-TLS peer with TLS v1.3
* fixed a regression in WMM parameter configuration for a TDLS peer
* fixed a regression in operation with drivers that offload 802.1X
4-way handshake
* fixed an ECDH operation corner case with OpenSSL
Ed Maste [Thu, 22 Aug 2019 16:06:52 +0000 (16:06 +0000)]
msdosfs_fat: reduce diffs with NetBSD and makefs
Use pointer arithmetic (as now done in makefs, and in NetBSD) instead of
taking the address of array element. No functional change, but this
makes it easier to compare different versions of this file.
Reviewed by: kib
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21365
Bjoern A. Zeeb [Thu, 22 Aug 2019 09:24:43 +0000 (09:24 +0000)]
usb: fix usb_fdt_support.c when altq enabled (usb_ehernet.h changes)
After r351243 when ALTQ was enabled in the kernel, the inline functions
in ifq.h would not have full type information as if_var.h was not
included.
Given usb_ethernet.h already includes all the various headers (which)
is the cause of the problem here, add if_var.h to it. This fixes the
builds again.
Reported by: CI system, e.g. FreeBSD-head-aarch64-LINT
Mike Karels [Thu, 22 Aug 2019 03:28:31 +0000 (03:28 +0000)]
Change w(1) to compute FROM (host) field size dynamically
It's nice to be able to display a full IPv6 host address if
needed, but it's also nice to display more than 3 characters of a command
line. Compute the needed size for the FROM column in an earlier pass,
and determine the maximum, then print what fits for the command.
Rick Macklem [Thu, 22 Aug 2019 01:15:06 +0000 (01:15 +0000)]
Map ENOTTY to EINVAL for lseek(SEEK_DATA/SEEK_HOLE).
Without this patch, when an application performed lseek(SEEK_DATA/SEEK_HOLE)
on a file in a file system that does not have its own VOP_IOCTL(), the
lseek(2) fails with errno ENOTTY. This didn't seem appropriate, since
ENOTTY is not listed as an error return by either the lseek(2) man page
nor the POSIX draft for lseek(2).
This was discussed on freebsd-current@ here:
http://docs.FreeBSD.org/cgi/mid.cgi?CAOtMX2iiQdv1+15e1N_r7V6aCx_VqAJCTP1AW+qs3Yg7sPg9wA
This trivial patch maps ENOTTY to EINVAL for lseek(SEEK_DATA/SEEK_HOLE).
Conrad Meyer [Thu, 22 Aug 2019 00:34:11 +0000 (00:34 +0000)]
gdb(4): Implement qXfer:threads:read
This streams out an XML document over several GDB packets describing all
threads in the system; their ids, name, and any loosely defined "extra info"
we feel like including. For now, I have included a string version of the run
state, similar to some of the DDB logic to stringify thread state.
The benefit of supporting this in addition to the qfThreadInfo/qsThreadInfo
packing is that in this mode, the host gdb does not ask for every thread's
"qThreadExtraInfo," saving per-thread round-trips on "info threads."
To use this feature, (k)gdb needs to be built with the --with-expat option.
I would encourage enabling this option by default in our GDB port, if it is
not already.
Finally, there is another optional attribute you can specify per-thread
called a "handle." Handles are arbitrarily long sequences of bytes,
represented in the XML as hexadecimal. It is unclear to me how or if GDB
actually uses handles for anything. So I have left them out.
Mark Johnston [Wed, 21 Aug 2019 23:43:58 +0000 (23:43 +0000)]
Add lockmgr(9) probes to the lockstat DTrace provider.
They follow the conventions set by rw and sx lock probes. There is
an additional lockstat:::lockmgr-disown probe.
Update lockstat(1) to report on contention and hold events for
lockmgr locks. Document the new probes in dtrace_lockstat.4, and
deduplicate some of the existing probe descriptions.
Reviewed by: mjg
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21355
Mateusz Guzik [Wed, 21 Aug 2019 23:24:40 +0000 (23:24 +0000)]
nullfs: lock the vnode with LK_SHARED in null_vptocnp
null_nodeget which follows almost always finds the target vnode in the hash,
avoiding insmntque1 altogether. Should it be needed, it already checks if the
lock needs to be upgraded.
Reviewed by: kib
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D20244
Warner Losh [Wed, 21 Aug 2019 22:18:01 +0000 (22:18 +0000)]
Create a AHCI attachment for nvme.
Intel has created RST and many laptops from vendors like Lenovo and Asus. It's a
mechanism for creating multiple boot devices under windows. It effectively hides
the nvme drive inside of the ahci controller. The details are supposed to be a
trade secret. However, there's a reverse engineered Linux driver, and this
implements similar operations to allow nvme drives to attach. The ahci driver
attaches nvme children that proxy the remapped resources to the child. nvme_ahci
is just like nvme_pci, except it doesn't do the PCI specific things. That's
moved into ahci where appropriate.
When the nvme drive is remapped, MSI-x interrupts aren't forwarded (the linux
driver doesn't know how to use this either). INTx interrupts are used
instead. This is suboptimal, but usually sufficient for the laptops these parts
are in.
This is based loosely on https://www.spinics.net/lists/linux-ide/msg53364.html
submitted, but not accepted by, Linux. It was written by Dan Williams. These
changes were written from scratch by Olivier Houchard.
Warner Losh [Wed, 21 Aug 2019 22:17:55 +0000 (22:17 +0000)]
Separate the pci attachment from the rest of nvme
Nvme drives can be attached in a number of different ways. Separate out the PCI
attachment so that we can have other attachment types, like ahci and various
types of NVMeoF.
Alexander Motin [Wed, 21 Aug 2019 20:17:30 +0000 (20:17 +0000)]
Improve NVMe hot unplug handling.
If device is unplugged from the system (CSTS register reads return
0xffffffff), it makes no sense to send any more recovery requests or
expect any responses back. If there is a detach call in such state,
just stop all activity and free resources. If there is no detach
call (hot-plug is not supported), rely on normal timeout handling,
but when it trigger controller reset, do not wait for impossible and
quickly report failure.
Mark Johnston [Wed, 21 Aug 2019 20:01:52 +0000 (20:01 +0000)]
Remove manual wire_count adjustments from the unmapped mbuf code.
The original code came from a desire to minimize the number of updates
to v_wire_count, which prior to r329187 was updated using atomics.
However, there is no significant benefit to batching today, so simply
allocate pages using VM_ALLOC_WIRED and rely on system accounting.
Mark Johnston [Wed, 21 Aug 2019 19:35:04 +0000 (19:35 +0000)]
Modify pipe_poll() to properly check for pending direct writes.
With r349546, it is a responsibility of the writer to clear PIPE_DIRECTW
after pinned data has been read. In particular, once a reader has
drained this data, there is a small window where the pipe is empty but
PIPE_DIRECTW is set. pipe_poll() was using the presence of PIPE_DIRECTW
to determine whether to return POLLIN, so in this window it would
claim that data was available to read when this was not the case.
Fix this by modifying several checks for PIPE_DIRECTW to instead look
at the number of residual bytes in data pinned by a direct writer. In
some cases we really do want to check for PIPE_DIRECTW, since the
presence of this flag indicates that any attempt to write to the pipe
will block on the existing direct writer.
Bisected and test case provided by: mav
Tested by: pho
Reviewed by: kib
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21333
Mark Johnston [Wed, 21 Aug 2019 16:11:12 +0000 (16:11 +0000)]
Simplify vm_page_dequeue() and fix an assertion.
- Add a vm_pagequeue_remove() function to physically remove a page
from its queue and update the queue length.
- Remove vm_page_pagequeue_lockptr() and let vm_page_pagequeue()
return NULL for dequeued pages.
- Avoid unnecessarily reloading the queue index if vm_page_dequeue()
loses a race with a concurrent queue operation.
- Correct an always-true assertion: vm_page_dequeue() may be called
from the page allocator with the page unlocked. The assertion
m->order == VM_NFREEORDER simply tests whether the page has been
removed from the vm_phys free lists; instead, check whether the
page belongs to an object.
Alan Somers [Wed, 21 Aug 2019 14:52:12 +0000 (14:52 +0000)]
ping: do reverse DNS lookup of the target address
When printing replies, ping will now attempt a reverse DNS lookup of the
target. That can be suppressed by using the "-n" option. Curiously, ping
has always done reverse lookups in certain error paths, but never in the
success path.
Submitted by: Ján Sučan <sucanjan@gmail.com>
MFC after: 2 weeks
Sponsored by: Google LLC (Google Summer of Code 2019)
Differential Revision: https://reviews.freebsd.org/D21351
Bjoern A. Zeeb [Wed, 21 Aug 2019 10:54:52 +0000 (10:54 +0000)]
gprof: disable building of a.out components
On arm64, riscv, and s390x disable building of aout components.
This allows gprof to build on these architectures which never supported
the legacy a.out binary format.
Randall Stewart [Wed, 21 Aug 2019 10:45:28 +0000 (10:45 +0000)]
Fix an issue when TSO and Rack play together. Basically
an retransmission of the initial SYN (with data) would
cause us to strip the SYN and decrement/increase offset/len
which then caused us a -1 offset and a panic.
Reported by: Larry Rosenman
(Michael Tuexen helped me debug this at the IETF)
Alexander Motin [Wed, 21 Aug 2019 02:17:39 +0000 (02:17 +0000)]
Formalize NVMe controller consumer life cycle.
This fixes possible double call of fail_fn, for example on hot removal.
It also allows ctrlr_fn to safely return NULL cookie in case of failure
and not get useless ns_fn or fail_fn call with NULL cookie later.
Toomas Soome [Tue, 20 Aug 2019 19:31:11 +0000 (19:31 +0000)]
loader.efi: efipart should be more careful about constructing block device lists
The cd handles should be collected as list of partitions. Some systems also
provide base name for block device (like PciRoot(0x0)/Pci(0x5,0x0)), we need
to be careful about those.
To make out life a bit easier, we prepare not just an array of handles, but we
allocate pdinfo struct for each handle and devicepath, then we can simplify
our work to sort the devices.
Ed Maste [Tue, 20 Aug 2019 18:20:45 +0000 (18:20 +0000)]
makefs: add msdosfs (FAT) support
Add FAT support to makefs by copying some files from sys/fs/msdosfs/ and
updating others with changes from NetBSD.
The six files copied from sys/fs/msdosfs at r348251 and modified are:
denode.h direntry.h fat.h msdosfs_fat.c msdosfs_lookup.c msdosfsmount.h
I would prefer to avoid the duplication, but reluctance to doing so was
expressed in a previous review (D11197); for now copy the files and
revisit in the future.
Mark Johnston [Tue, 20 Aug 2019 17:52:12 +0000 (17:52 +0000)]
Use a sleepable lock for midistat functions.
Otherwise the mutex needs to be dropped when copying out the midistat
sbuf, leading to a race which allows one to read kernel memory beyond
the end of the sbuf buffer.
Reported and tested by: pho
Security: CVE-2019-5612
Dimitry Andric [Tue, 20 Aug 2019 17:39:32 +0000 (17:39 +0000)]
Pull in r368867 from upstream libc++ trunk (by Marshall Clow):
Rework recursive_timed_mutex so that it uses __thread_id instead of
using the lower-level __libcpp_thread_id. This is prep for fixing
PR42918. Reviewed as https://reviews.llvm.org/D65895
Pull in r368916 from upstream libc++ trunk (by Marshall Clow):
Fix thread comparison by making sure we never pass our special 'not a
thread' value to the underlying implementation. Fixes PR#42918.
This should fix std::thread::id::operator==() attempting to call
pthread_equal(3) with zero values.
Reported by: andrew@tao11.riddles.org.uk
PR: 239038, 239550
MFC after: 3 days
Brooks Davis [Tue, 20 Aug 2019 16:07:17 +0000 (16:07 +0000)]
Remove some compatability with Seventh Edition UNIX realloc().
In Seventh Edition UNIX, the last pointer passed to free() was
guaranteed to not actually have been freed allowing memory to be
"compacted" via the following pattern:
free(foo);
foo = realloc(foo, newsize);
Further, Andrew Koenig reports in "C Traps and Pitfalls" that the
original realloc() implementation required this pattern.
The C standard is clear that this is Undefined Behavior. Modern
allocators don't support it and no portable code could rely on it so
remove this support.
Note: the removed implementation contains an off-by-one error and if
an item isn't found on the freelist, then twice as much memory as the
largest possible allocation will be copied.
Kristof Provost [Tue, 20 Aug 2019 14:31:22 +0000 (14:31 +0000)]
netpfil tests: Add the set ToS test for ipfw and pf
This test tests the following:
- The firewall is able to set the tos bits
- The firewall is able to set the DSCP bits when EN bits is already set and
the EN bits remains unchanged.
- The firewall is able to drop the packets based on ToS value
projects / FreeBSD / FreeBSD.git / log