mav [Thu, 8 Dec 2016 15:58:03 +0000 (15:58 +0000)]
Fix spa_alloc_tree sorting by offset in r305331.
Original commit "7090 zfs should improve allocation order" declares alloc
queue sorted by time and offset. But in practice io_offset is always zero,
so sorting happened only by time, while order of writes with equal time was
completely random. On Illumos this did not affected much thanks to using
high resolution timestamps. On FreeBSD due to using much faster but low
resolution timestamps it caused bad data placement on disks, affecting
further read performance.
This change switches zio_timestamp_compare() from comparing uninitialized
io_offset to really populated io_bookmark values. I haven't decided yet
what to do with timestampts, but on simple tests this change gives the
same peformance results by just making code to work as declared.
kib [Thu, 8 Dec 2016 11:39:08 +0000 (11:39 +0000)]
Use the populate() driver paging method for i915 driver.
In particular, the fault access type is accounted for when the
aperture page is moved to GTT domain. On the other hand, the current
pager structure is left intact, most important, only one page is
instantiated per populate call.
Reviewed by: alc
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks
kib [Thu, 8 Dec 2016 11:35:53 +0000 (11:35 +0000)]
Implement the populate() pager method for phys pager.
It allows to provide configurable agressive prefaulting and useful
hints to page daemon about memory allocations, on faults for pages
managed by phys pager. In fact, this implementation is superior to
the MAP_SHARED_PHYS hack from my Postgresql paper, while giving
similar benefits of reducing the page faults numbers on SysV shared
memory mappings.
Reviewed by: alc
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks
kib [Thu, 8 Dec 2016 11:26:11 +0000 (11:26 +0000)]
Add a new populate() pager method and extend device pager ops vector
with cdev_pg_populate() to provide device drivers access to it. It
gives drivers fine control of the pages ownership and allows drivers
to implement arbitrary prefault policies.
The populate method is called on a page fault and is supposed to
populate the vm object with the page at the fault location and some
amount of pages around it, at pager's discretion. VM provides the
pager with the hints about current range of the object mapping, to
avoid instantiation of immediately unused pages, if pager decides so.
Also, VM passes the fault type and map entry protection to the pager,
allowing it to force the optimal required ownership of the mapped
pages.
Installed pages must contiguously fill the returned region, be fully
valid and exclusively busied. Of course, the pages must be compatible
with the object' type.
After populate() successfully returned, VM fault handler installs as
many instantiated pages into the process page tables as it sees
reasonable, while still obeying the correct semantic for COW and vm
map locking.
The method is opt-in, pager sets OBJ_POPULATE flag to indicate that
the method can be called. If pager' vm objects can be shadowed, pager
must implement the traditional getpages() method in addition to the
populate(). Populate() might fall back to the getpages() on per-call
basis as well, by returning VM_PAGER_BAD error code.
For now for device pagers, the populate() method is only allowed to be
used by the managed device pagers, but the limitation is only made
because there is no unmanaged fault handlers which could use it right
now.
KPI designed together with, and reviewed by: alc
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks
alc [Thu, 8 Dec 2016 04:29:29 +0000 (04:29 +0000)]
Previously, vm_radix_remove() would panic if the radix trie didn't
contain a vm_page_t at the specified index. However, with this
change, vm_radix_remove() no longer panics. Instead, it returns NULL
if there is no vm_page_t at the specified index. Otherwise, it
returns the vm_page_t. The motivation for this change is that it
simplifies the use of radix tries in the amd64, arm64, and i386 pmap
implementations. Instead of performing a lookup before every remove,
the pmap can simply perform the remove.
gavin [Wed, 7 Dec 2016 22:52:12 +0000 (22:52 +0000)]
Switch if_run.c to use a bitmap for debug levels rather than arbitrary
values. This more closely matches other wifi drivers in the tree.
The bitmap levels have been based closely on other drivers (primarily
[u]rtwn(4)) in the hope that one day these can be unified into a shared
wifi-debug framework.
This is the first step of several pieces of work I'm planning on doing
with the run(4) driver. I may well adjust and refine some of the debug
bitmaps at a later date.
tuexen [Wed, 7 Dec 2016 19:30:59 +0000 (19:30 +0000)]
Cleanup the names of SSN, SID, TSN, FSN, PPID and MID.
This made a couple of bugs visible in handling SSN wrap-arounds
when using DATA chunks. Now bulk transfer seems to work fine...
This fixes the issue reported in
https://github.com/sctplab/usrsctp/issues/111
rwatson [Wed, 7 Dec 2016 16:11:55 +0000 (16:11 +0000)]
Replace spaces with tabs in definition of SCTP system calls, for consistency
with the remainder of the syscalls.master file. This problem does not occur
in the freebsd32 version of the same system calls.
vangyzen [Wed, 7 Dec 2016 15:04:22 +0000 (15:04 +0000)]
Export the whole thread name in kinfo_proc
kinfo_proc::ki_tdname is three characters shorter than
thread::td_name. Add a ki_moretdname field for these three
extra characters. Add the new field to kinfo_proc32, as well.
Update all in-tree consumers to read the new field and assemble
the full name, except for lldb's HostThreadFreeBSD.cpp, which
I will handle separately. Bump __FreeBSD_version.
andrew [Wed, 7 Dec 2016 14:35:05 +0000 (14:35 +0000)]
Add ACPI support to the arm64 mp code. We use the Multiple APIC Description
Table to find the CPUs to find the CPUs to start. Currently we assume PSCI,
however this assumption is shared with the FDT code.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
andrew [Wed, 7 Dec 2016 14:24:53 +0000 (14:24 +0000)]
Add ACPI support to the PSCI driver. This checks the Fixed ACPI Description
Table to find if the hardware supports PSCI, and if so what method the
kernel should use to interact with it.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
gnn [Wed, 7 Dec 2016 07:27:47 +0000 (07:27 +0000)]
Fix a kernel panic in DTrace's rw_iswriter subroutine.
On FreeBSD the sense of rw_write_held() and rw_iswriter() were reversed,
probably due to a cut and paste error. Using rw_iswriter() would cause
the kernel to panic.
Convert result of hash_packet6() into host byte order.
For IPv4 similar function uses addresses and ports in host byte order,
but for IPv6 it used network byte order. This led to very bad hash
distribution for IPv6 flows. Now the result looks similar to IPv4.
markj [Tue, 6 Dec 2016 22:52:45 +0000 (22:52 +0000)]
Provide dummy sysctls for v_cache_count and v_tcached.
Some utilities (notably top(1)) exit if any of their input sysctls don't
exist, and the removal of the above-mentioned PG_CACHE-related sysctls
makes it difficult to run such utilities on different versions of the
kernel without recompiling.
markj [Tue, 6 Dec 2016 22:48:28 +0000 (22:48 +0000)]
Require the STACK option for code that captures stacks of running threads.
stack_machdep.c is compiled if either of the DDB or STACK options is
specified, but stack_save_td_running() isn't useable from DDB. Moreover,
stack_save_td_running() works by raising an NMI on the CPU running the
target thread, and the corresponding handler is compiled only if STACK is
configured.
dim [Tue, 6 Dec 2016 20:44:40 +0000 (20:44 +0000)]
During the bootstrap phase, when building the minimal llvm library on
PowerPC, add lib/Support/Atomic.cpp. This is needed because upstream
llvm revision r271821 disabled the use of std::call_once, which causes
some fallback functions from Atomic.cpp to be used instead.
Reported by: Mark Millard
PR: 214902
X-MFC-With: 309124
Properly sign extend the result of jrand48() and mrand48().
These functions are supposed to return a value between [_2^31, 2^31).
This doesn't seem to work on 64-bit systems, where we return a value
between [0, 3^32). Patch up the function to use proper casts to int32_t.
While there, fix some other style bugs.
glebius [Tue, 6 Dec 2016 18:50:44 +0000 (18:50 +0000)]
Fix possible integer overflow in guest memory bounds checking, which could
lead to access from the virtual machine to the heap of the bhyve(8) process.
Submitted by: Felix Wilhelm <fwilhelm ernw.de>
Patch by: grehan
Security: FreeBSD-SA-16:38.bhyve
glebius [Tue, 6 Dec 2016 18:50:33 +0000 (18:50 +0000)]
Fix possible buffer overflow(s) in link_ntoa(3).
A specially crafted sockaddr_dl argument can trigger a static buffer overflow
in the libc library, with possibility to rewrite with arbitrary data following
static buffers that belong to other library functions.
glebius [Tue, 6 Dec 2016 18:50:22 +0000 (18:50 +0000)]
When telnetd(8) composes argument list for login(1), an unexpected sequence
of memory allocation failures combined with insufficient error checking
could result in the construction and execution of an argument sequence that
was not intended.
Fix that treating malloc(3) failures as fatal condition.
sobomax [Tue, 6 Dec 2016 18:22:25 +0000 (18:22 +0000)]
Auto-generate 2 test cases that differ only in structure and SCM_XXX constant
used. We can do it programmatically, but that would make code convoluted
and more complex. I have two more of those types coming for the CLOCK_REALTIME
and CLOCK_MONOTONIC. This seems like an elegant and scallable approach.
kib [Tue, 6 Dec 2016 17:13:17 +0000 (17:13 +0000)]
Do not leak curthread->inact_mtx when cancelling in pthread_cond_wait(3).
Leave robust-protected region before checking for cancellation by
calling _thr_testcancel(). Otherwise, if cancelling request was
pending, the cancel handler is called with the dandling inact_mtx,
which triggers an assert if any mutex operation is performed by the
handler.
Reported and tested by: Dimitri Staessens <dimitri.staessens@intec.ugent.be>
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
bdrewery [Tue, 6 Dec 2016 17:05:02 +0000 (17:05 +0000)]
strvis(3): Avoid internal state of multibyte functions being tainted.
The mbtoc(3) and wctomb(3) functions use internal state which may be
tainted before the call to strvis(3). In this context we can just use
the thread-safe versions mbrtoc(3) and wcrtomb(3) which allow passing
our own state from our stack.
andrew [Tue, 6 Dec 2016 15:12:14 +0000 (15:12 +0000)]
Create two GIC ivars to find the bus type and GIC hardware version. These
will be used by the gicv2m and ITS ACPI drivers to only attach to the
correct parent.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
tuexen [Tue, 6 Dec 2016 10:21:25 +0000 (10:21 +0000)]
Whitespace changes.
The tools using to generate the sources has been updated and produces
different whitespaces. Commit this seperately to avoid intermixing
these with real code changes.
markj [Tue, 6 Dec 2016 04:28:56 +0000 (04:28 +0000)]
libdtrace: Don't use a read-only handle for enumerating pid probes.
Enumeration of return probes involves disassembling subroutines in the
target process, and ptrace(2) is currently used to read from the target
process. libproc could read from the backing file instead to avoid this
problem, but in the common case libdtrace will have a writeable handle
on the process anyway. In particular, a writeable handle is needed to list
USDT probes, and libdtrace will cache such a handle for processes that it
controls via dtrace -c and -p.
markj [Tue, 6 Dec 2016 04:22:38 +0000 (04:22 +0000)]
libproc: Add support for some proc_attach() flags.
This change adds some handling for the equivalent of Solaris' PGRAB_*
flags. In particular, support for PGRAB_RDONLY is needed to avoid a
nasty deadlock: dtrace(1) may otherwise stop the master process for its
pseudo-terminal and end up blocking while writing to standard output.
markj [Tue, 6 Dec 2016 04:21:35 +0000 (04:21 +0000)]
libproc: Cache symbol tables for mapped objects upon access.
Extend the file handle cache entries to include symbol tables as well. An
index is used to implement binary search by symbol value. Lookups by
name are comparatively rare and are thus still implemented with a linear
search, but support for a binary search by name would be straightforward
to add if needed.
markj [Tue, 6 Dec 2016 04:20:32 +0000 (04:20 +0000)]
libproc: Match prefixes when looking up mapped object by name.
When looking up an object by name, allow prefix matches if no direct match
is found. This allows one to, for example, match libc entry probes with:
# dtrace -n 'pid$target:libc.so::entry' -c ./foo
instead of requiring "libc.so.7" or a glob.
Also remove proc_obj2map() as it currently just duplicates the
functionality of proc_name2map(). It's supposed to take a Solaris
link-map ID as a paramter, but support for this isn't implemented and
isn't required to support DTrace's pid provider.
markj [Tue, 6 Dec 2016 04:19:08 +0000 (04:19 +0000)]
libproc: Cache ELF handles for loaded objects.
libproc previously created a new handle for each symbol lookup, which
gives rather egregious performance for DTrace's ustack() action. With
this change libproc will cache the libelf descriptor upon access, making
lookups much faster in the common case.
markj [Tue, 6 Dec 2016 04:18:09 +0000 (04:18 +0000)]
libproc: Improve .gnu_debuglink support.
As of r278658 libproc looks for debug files under /usr/lib/debug and will
use them if available. This change fleshes out that support a bit further:
- Check for a .gnu_debuglink section and use the file name specified
there if one is present.
- Validate external debug files with the CRC in the .gnu_debuglink
section so as to avoid using stale or corrupt debug files.
- Search for debug files in the directory containing the referencing
object or in the .debug subdirectory, as GDB does.
markj [Tue, 6 Dec 2016 04:14:20 +0000 (04:14 +0000)]
libproc: Make proc_getpid() an accessor for struct proc_handle.
This allows librtld_db to fetch the PID from a handle without calling into
libproc. Together with r303531, this means that librtld_db no longer
references symbols from libproc.
libarchive #831:
Spelling fixes
libarchive #832:
Relax sanity checks of number fields in tar header even more
OSS-Fuzz #16:
Fix possible hang in uudecode_filter_read()
OSS-Fuzz #220:
Reject an 'ar' filename table larger than 1GB or a filename larger
than 1MB.
jhb [Tue, 6 Dec 2016 00:39:00 +0000 (00:39 +0000)]
Rework syscall structure lookups.
Avoid always using an O(n^2) loop over known syscall structures with
strcmp() on each system call. Instead, use a per-ABI cache indexed by
the system call number. The first 1024 system calls (which should cover
all of the normal system calls in currently-supported ABIs) use a flat array
indexed by the system call number to find system call structure. For other
system calls, a linked list of structures storing an integer to structure
mapping is stored in the ABI. The linked list isn't very smart, but it
should only be used by buggy applications invoking unknown system calls.
This also fixes handling of unknown system calls which currently trigger
a NULL pointer dereference.
jhb [Tue, 6 Dec 2016 00:36:02 +0000 (00:36 +0000)]
Don't attach to Host-PCI bridges with a bad bus number.
If the bus number assigned to a Host-PCI bridge doesn't match the first
bus number in the associated producer range from _CRS, print a warning and
fail to attach rather than panicking due to an assertion failure.
At least one single-socket Dell machine leaves a "ghost" Host-PCI bridge
device in the ACPI namespace that seems to correspond to the I/O hub in
the second socket of a two-socket machine. However, the BIOS doesn't
configure the settings for this "ghost" bridge correctly, nor does it have
any PCI devices behind it.
libarchive #831:
Spelling fixes
libarchive #832:
Relax sanity checks of number fields in tar header even more
OSS-Fuzz #16:
Fix possible hang in uudecode_filter_read()
OSS-Fuzz #220:
Reject an 'ar' filename table larger than 1GB or a filename larger
than 1MB.
avos [Tue, 6 Dec 2016 00:13:49 +0000 (00:13 +0000)]
rsu: add hardware crypto support (WEP, TKIP and CCMP).
This change includes firmware commands for key setup +
some additional checking via CAMREAD / CAMWRITE registers.
Nothing (except rsu_delete_key() for pairwise keys) is deferred;
to ensure that things are done in order rsu_set_key() will wait
until key deletion task will be finished.
Tested with Asus USB-N10 (all ciphers).
Differences from initial (reviewed) patch:
- Pause AC queues before disassociation - since CMD_DISCONNECT clears
crypto state all pending frames must be processed / dropped before it.
- Check sc_running flag before trying to set static keys.
- Clear key index from bitmap even when firmware command fails
(it will be invalidated via CAMWRITE anyway).
rmacklem [Mon, 5 Dec 2016 22:36:25 +0000 (22:36 +0000)]
Fix the NFSv4.1 server for Open reclaim after a reboot.
The NFSv4.1 server failed to update the nfs-stablerestart file for
a client when the client was issued its first Open. As such, recovery
of Opens after a server reboot failed with NFSERR_NOGRACE.
This patch fixes this.
It also changes the code so that it malloc()'s the 1024 byte array
instead of allocating it on the kernel stack for both NFSv4.0 and NFSv4.1.
Note that this bug only affected NFSv4.1 and only when clients attempted
to reclaim Opens after a server reboot.
sobomax [Mon, 5 Dec 2016 17:21:04 +0000 (17:21 +0000)]
Refactor the regression test code by splitting huge monolithic C
file into smaller pieces that are hopefully easier to understand
and extend. This is to pave the ground for adding few more
socket timestamp formats that I am working on here.
- Append RCR_APP_PHYSTS bit after firmware loading - otherwise
firmware will reset the register and this modification will be lost.
(without it Rx PHY descriptor section will contain garbage).
- Check if R92S_RXDW0_PHYST bit is set (like it is done in rtwn(4)) -
even if infosz is non-zero the section may not contain anything useful.
- In case, if descriptor is absent (A-MPDU?) use last calibrated RSSI
(rtwn(4) uses RSSI from the previous (sub)frame; probably, this
approach should be used here too).
kib [Sun, 4 Dec 2016 20:44:58 +0000 (20:44 +0000)]
Restructure the code to handle reporting of non-exited processes from
wait(2).
- Do not acquire the process spinlock if neither WTRAPPED nor WUNTRACED
options were passed [1].
- Extract the code to report alive process into a new helper
report_alive_proc() and use it for trapped, stopped and continued
childrens.
Note that the process spinlock is required around the WTRAPPED and
WUNTRACED tests, because P_STOPPED_TRACE and P_STOPPED_SIG flags are
set before other threads are stopped at the suspension point, and that
threads increment p_suspcount while owning only the process spinlock,
the process lock is dropped by them. If the spinlock is not taken for
tests, the syscall thread might miss both p_suspcount increment and
wakeup in wakeup in thread_suspend_switch().
Based on the submission by: mjg [1]
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
projects / FreeBSD / FreeBSD.git / log