https://www.illumos.org/issues/5695
In dmu_sync_ready(), a hole block pointer will have it's logical size
explicitly set as it's necessary for replay purposes. To "undo" this,
dmu_sync_done() will zero out any hole that it finds. This becomes a
problem when using the "hole_birth" feature, as this will also wipe out
any birth time that might have happened to be set on the hole.
...
As a fix, the logic to zero out a hole is only applied to old style
holes with a birth time of zero. Holes created with the "hole_birth"
feature enabled will have a non-zero birth time, and will be skipped
(thus preserving the ltime, type, and level information as well).
In addition, zdb was updated to also print the ltime, type, and level
information for these new style holes. Previously, only the logical
birth time would be printed.
Author: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Warner Losh [Wed, 12 Aug 2015 17:19:52 +0000 (17:19 +0000)]
Why on earth have we been building rescue as a build tool for the past
12 years? Nothing downstream in the build uses it. Eliminate it as a
build tool.
Andrew Turner [Wed, 12 Aug 2015 17:06:22 +0000 (17:06 +0000)]
Set the counter-timer virtual offset to a know value, it may not have been
set by the boot code and are reset to an implementation defined value that
may be unknown.
Warner Losh [Wed, 12 Aug 2015 16:43:15 +0000 (16:43 +0000)]
Crunchgen needs to be bootstrapped to pick up the STRIP->STRIPBIN
changes to prevent the 'rescue: not found' errors from happening.
Bump FreeBSD_version to 1100078 since there's been no version bumps
since this change was made. Only people that installed since r284356
really need to do this bootstrapping, but since crunchgen needs to
bootstrap for other reasons, bumping the number was the simplest.
Ed Schouten [Wed, 12 Aug 2015 16:17:00 +0000 (16:17 +0000)]
Properly return ENOTDIR when calling *at() on a non-vnode.
We already properly return ENOTDIR when calling *at() on a non-directory
vnode, but it turns out that if you call it on a socket, we see EINVAL.
Patch up namei to properly translate this to ENOTDIR.
Marcel Moolenaar [Wed, 12 Aug 2015 15:26:32 +0000 (15:26 +0000)]
Better support memory mapped console devices, such as VGA and EFI
frame buffers and memory mapped UARTs.
1. Delay calling cninit() until after pmap_bootstrap(). This makes
sure we have PMAP initialized enough to add translations. Keep
kdb_init() after cninit() so that we have console when we need
to break into the debugger on boot.
2. Unfortunately, the ATPIC code had be moved as well so as to
avoid a spurious trap #30. The reason for which is not known
at this time.
3. In pmap_mapdev_attr(), when we need to map a device prior to the
VM system being initialized, use virtual_avail as the KVA to map
the device at. In particular, avoid using the direct map on amd64
because we can't demote by virtue of not being able to allocate
yet. Keep track of the translation.
Re-use the translation after the VM has been initialized to not
waste KVA and to satisfy the assumption in uart(4) that the handle
returned for the low-level console is the same as later returned
when the device is probed and attached.
4. In pmap_unmapdev() remove the mapping from the table when called
pre-init. Otherwise keep the mapping. During bus probe and attach
device resources are mapped and unmapped multiple times, which
would have us destroy the mapping used by the low-level console.
5. In pmap_init(), set pmap_initialized to signal that we're not
pre-init anymore. On amd64, bring the direct map in sync with the
translations created at that time.
6. Implement bus_space_map() and bus_space_unmap() for real: when
the tag corresponds to memory space, call the corresponding
pmap_mapdev() and pmap_unmapdev() functions to construct and
actual handle.
7. In efifb.c and vt_vga.c, remove the crutches and hacks and simply
call pmap_mapdev_attr() or bus_space_map() as desired.
Notes:
1. uart(4) already used bus_space_map() during low-level console
setup but since serial ports have traditionally been I/O port
based, the lack of a proper implementation for said function
was not a problem. It has always supported memory mapped UARTs
for low-level consoles by setting hw.uart.console accordingly.
2. The use of the direct map on amd64 without setting caching
attributes has been a bigger problem than previously thought.
This change has the fortunate (and unexpected) side-effect of
fixing various EFI frame buffer problems (though not all).
PR: 191564, 194952
Special thanks to:
1. XipLink, Inc -- generously donated an Intel Bay Trail E3800
based eval board (ADLE3800PC).
2. The FreeBSD Foundation, in particular emaste@ -- for UEFI
support in general and testing.
3. Everyone who tested the proposed for PR 191564.
4. jhb@ and kib@ for being a soundboard and applying a clue bat
if so needed.
Ed Schouten [Wed, 12 Aug 2015 11:30:31 +0000 (11:30 +0000)]
Unignore signals when starting CloudABI processes.
As CloudABI processes cannot adjust their signal handlers, we need to
make sure that we start up CloudABI processes with consistent signal
masks. Though the POSIx standard signal behavior is all right, we do
need to make sure that we ignore SIGPIPE, as it would otherwise be
hard to interact with pipes and sockets.
Extend execsigs() to iterate over ps_sigignore and call sigdflt() for
each of the ignored signals.
In x2APIC mode, IPI generation is atomic because it is performed by
single ICR MSR write. This is in contrast with the xAPIC mode, where
we must read current ICR value, do bit fiddling and perform two 32-bit
register writes. As a consequence, there is no need to disable
interrupts around ICR value calculation and write.
Note that typical users of ipi_raw() and ipi_vectored() take spinlock,
which already disables interrupts. For them, the change removes
unneeded CLI and POPFL/Q instructions.
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Initialization of smp_tlb_wait does not require release semantic, no
data is synchronized by store/load to the variable. The
lapic_write_icr() function ensures that store buffers are flushed
before IPI command is issued.
Discussed with: bde
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Ed Schouten [Wed, 12 Aug 2015 08:41:48 +0000 (08:41 +0000)]
Make blocking CloudABI futex operations work.
Blocking on locks and condition variables can be accomplished by polling
and using the special filters CONDVAR, LOCK_RDLOCK and LOCK_WRLOCK.
For now it wouldn't make sense to implement this functionality into
kqueue() itself, for the reason that they are CloudABI specific and
would require us to resize 'struct kevent' to hold all of the parameters
of interest.
Add a bandaid to the CloudABI poll system call to call into the futex
code directly if it detects specific combinations of events that are
used by the C library.
Ed Schouten [Wed, 12 Aug 2015 07:59:00 +0000 (07:59 +0000)]
Make poll() and kqueue() on CloudABI work.
This change implements two functions, cloudabi64_kevent_copyin() and
cloudabi64_kevent_copyout(), that convert CloudABI structures to
FreeBSD's struct kevent. CloudABI uses two structures: subscription_t
and event_t. The former is used for input, whereas the latter is used
for output. Unlike struct kevent, fields aren't overloaded for multiple
purposes or for separate event types.
For poll() we call into the newly introduced kern_kevent_anonymous()
function that allows us to poll without a file descriptor. This function
is not only used by poll(), but also by functions such as
sleep() and clock_nanosleep().
Marcel Moolenaar [Wed, 12 Aug 2015 04:03:04 +0000 (04:03 +0000)]
Use bus_alloc_resource_any(), rather than bus_alloc_resource()
with start 0 and end ~0. This avoids confusion WRT to what the
value of length can or should be.
Marcelo Araujo [Wed, 12 Aug 2015 00:49:20 +0000 (00:49 +0000)]
Describe that bcopy(3) is deprecated and marked as LEGACY in
POSIX.1-2001 and removed from the specification in POSIX.1-2008.
New softwares shall use memcpy(3) or memmove(3).
Xin LI [Tue, 11 Aug 2015 22:43:28 +0000 (22:43 +0000)]
Disable building INDEX-9 and INDEX-10 because they are not useful for
users who uses only FreeBSD -CURRENT, and building these indexes are
only useful to those who share ports tree across different FreeBSD
releases, and system administrator with that need can always enable
this behavior.
MFC after: 2 weeks (only the principals, not actual change)
Mariusz Zaborski [Tue, 11 Aug 2015 18:01:10 +0000 (18:01 +0000)]
The nvlist_move_nvpair() function can fail in two cases, if:
- the nvlist error is set, or
- the nvlist case ignore flag is not set and there is attend to
add element with duplicated name.
In both cases the nvlist_move_nvpair() function free nvpair structure.
If library will try to unpack a binary blob which contains duplicated
names it will end up with using memory after free.
To prevent that, the nvlist_move_nvpair() function interface is changed
to report about failure and checks are added to the nvpair_xunpack()
function.
Koop Mast [Tue, 11 Aug 2015 16:51:44 +0000 (16:51 +0000)]
Instead of defining the actualy user and group id in the drmP.h files
define GID_VIDEO in sys/conf.h, and use it together with UID_ROOT
to define DRM_DEV_UID and DRM_DEV_GID in the drmP.h files.
So there is one place where the UID's and GID's are defined.
Ed Schouten [Tue, 11 Aug 2015 14:08:46 +0000 (14:08 +0000)]
Fall back to O_RDONLY -- not O_WRONLY.
If CloudABI processes open files with a set of requested rights that do
not match any of the privileges granted by O_RDONLY, O_WRONLY or O_RDWR,
we'd better fall back to O_RDONLY -- not O_WRONLY.
Ed Schouten [Tue, 11 Aug 2015 13:47:23 +0000 (13:47 +0000)]
Add support for anonymous kqueues.
CloudABI's polling system calls merge the concept of one-shot polling
(poll, select) and stateful polling (kqueue). They share the same data
structures.
Extend FreeBSD's kqueue to provide support for waiting for events on an
anonymous kqueue. Unlike stateful polling, there is no need to support
timeouts, as an additional timer event could be used instead.
Furthermore, it makes no sense to use a different number of input and
output kevents. Merge this into a single argument.
Andrew Turner [Tue, 11 Aug 2015 13:42:58 +0000 (13:42 +0000)]
Start to support PSCI 1.0. For all the functions we currently support this
can be seen as the same as 0.2. There are changes with the data passed to
CPU_SUSPEND, however we don't yet use this call.
Alexander Motin [Tue, 11 Aug 2015 10:39:19 +0000 (10:39 +0000)]
MFV r277425:
5376 arc_kmem_reap_now() should not result in clearing arc_no_grow
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Steven Hartland <killing@multiplay.co.uk>
Reviewed by: Richard Elling <richard.elling@richardelling.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Store addresses instead of sockaddrs inside llentry.
This permits us having all (not fully true yet) all the info
needed in lookup process in first 64 bytes of 'struct llentry'.
struct llentry layout:
BEFORE:
[rwlock .. state .. state .. MAC ] (lle+1) [sockaddr_in[6]]
AFTER
[ in[6]_addr MAC .. state .. rwlock ]
Currently, address part of struct llentry has only 16 bytes for the key.
However, lltable does not restrict any custom lltable consumers with long
keys use the previous approach (store key at (lle+1)).
* Split lltable_init() into lltable_allocate_htbl() (alloc
hash table with default callbacks) and lltable_link() (
links any lltable to the list).
* Switch from LLTBL_HASHTBL_SIZE to per-lltable hash size field.
* Move lltable setup to separate functions in in[6]_domifattach.
Alexander Motin [Mon, 10 Aug 2015 21:36:51 +0000 (21:36 +0000)]
MFV 286604: 5812 assertion failed in zrl_tryenter(): zr_owner==NULL
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Reviewed by: Will Andrews <will@freebsd.org>
Approved by: Gordon Ross <gwr@nexenta.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Alexander Motin [Mon, 10 Aug 2015 21:32:40 +0000 (21:32 +0000)]
MFV 286602: 5810 zdb should print details of bpobj
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Will Andrews <will@freebsd.org>
Reviewed by: Simon Klinkert <simon.klinkert@gmail.com>
Approved by: Gordon Ross <gwr@nexenta.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Alexander Motin [Mon, 10 Aug 2015 21:19:42 +0000 (21:19 +0000)]
MFV 286599: 5808 spa_check_logs is not necessary on readonly pools
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Paul Dagnelie <paul.dagnelie@delphix.com>
Reviewed by: Simon Klinkert <simon.klinkert@gmail.com>
Reviewed by: Will Andrews <will@freebsd.org>
Approved by: Gordon Ross <gwr@nexenta.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Ian Lepore [Mon, 10 Aug 2015 20:54:03 +0000 (20:54 +0000)]
Correct the polarity of the PPS assert and clear events with respect to the
electrical signals on the serial port. Virtually all devices which output a
PPS signal generate a brief higher-voltage pulse, the leading edge of which
is the on-time point.
Both DCD and CTS are active-low signals on the wire, meaning the assertion
of their status bits in the modem status register corresponds to the lower
voltage level on the wire. So when the status bit transitions to not-set,
create a PPS assert event; when the status bit transitions to set, create a
PPS clear event.
Ian Lepore [Mon, 10 Aug 2015 20:08:09 +0000 (20:08 +0000)]
Allow the choice of PPS signal captured by uart(4) to be runtime-configured,
eliminating the need to build a custom kernel to use the CTS signal.
The historical UART_PPS_ON_CTS kernel option is still honored, but now it
can be overridden at runtime using a tunable to configure all uart devices
(hw.uart.pps_mode) or specific devices (dev.uart.#.pps_mode). The per-
device config is both a tunable and a writable sysctl.
This syncs the PPS capabilities of uart(4) with the enhancements recently
recently added to ucom(4) for capturing from USB serial devices.
Alexander Motin [Mon, 10 Aug 2015 19:38:07 +0000 (19:38 +0000)]
MFV 286588: 5820 verify failed in zio_done(): BP_EQUAL(bp, io_bp_orig)
Reviewed by: Alex Reece <alex@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Steven Hartland <killing@multiplay.co.uk>
Approved by: Garrett D'Amore <garrett@damore.org>
Author: Matthew Ahrens <mahrens@delphix.com>
Alexander Motin [Mon, 10 Aug 2015 19:32:58 +0000 (19:32 +0000)]
MFV 286586: 5746 more checksumming in zfs send
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Albert Lee <trisk@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Make kstack_pages a tunable on arm, x86, and powepc. On i386, the
initial thread stack is not adjusted by the tunable, the stack is
allocated too early to get access to the kernel environment. See
TD0_KSTACK_PAGES for the thread0 stack sizing on i386.
The tunable was tested on x86 only. From the visual inspection, it
seems that it might work on arm and powerpc. The arm
USPACE_SVC_STACK_TOP and powerpc USPACE macros seems to be already
incorrect for the threads with non-default kstack size. I only
changed the macros to use variable instead of constant, since I cannot
test.
On arm64, mips and sparc64, some static data structures are sized by
KSTACK_PAGES, so the tunable is disabled.
Sponsored by: The FreeBSD Foundation
MFC after: 2 week
Zbigniew Bodek [Mon, 10 Aug 2015 17:16:49 +0000 (17:16 +0000)]
Avoid sign extension of value passed to kva_alloc from uma_zone_reserve_kva
Fixes "panic: vm_radix_reserve_kva: unable to reserve KVA" caused by sign
extention of "pages * UMA_SLAB_SIZE" value passed to kva_alloc() which
takes unsigned long argument.
In the erroneus case that triggered this bug, the number of pages
to allocate in uma_zone_reserve_kva() was 0x8ebe6, that gave the
total number of bytes to allocate equal to 0x8ebe6000 (int).
This was then sign extended in kva_alloc() to 0xffffffff8ebe6000
(unsigned long).
Reviewed by: alc, kib
Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Obtained from: Semihalf
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3346
Pre-resolve symbols required for the deferred signal processing. This
avoids recursion into rtld when leaving libthr critical section for
the deferred signal delivery.
For the same reason, use syscall(2) instead of referencing
__sys_sigreturn(2). Syscall() is already pre-resolved for fork()
interceptor.
Tested by: Andre Meiser <ortadur@web.de>
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Alexander Motin [Mon, 10 Aug 2015 12:39:10 +0000 (12:39 +0000)]
MFV r277430: 5313 Allow I/Os to be aggregated across ZIO priority classes
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Reviewed by: Will Andrews <willa@SpectraLogic.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Justin T. Gibbs <justing@spectralogic.com>
Partially merge r274887,r275334,r275577,r275578,r275586 to minimize
differences between projects/routing and HEAD.
This commit tries to keep code logic the same while changing underlying
code to use unified callbacks.
* Add llt_foreach_entry method to traverse all entries in given llt
* Add llt_dump_entry method to export particular lle entry in sysctl/rtsock
format (code is not indented properly to minimize diff). Will be fixed
in the next commits.
* Add llt_link_entry/llt_unlink_entry methods to link/unlink particular lle.
* Add llt_fill_sa_entry method to export address in the lle to sockaddr
format.
* Add llt_hash method to use in generic hash table support code.
* Add llt_free_entry method which is used in llt_prefix_free code.
* Prepare for fine-grained locking by separating lle unlink and deletion in
lltable_free() and lltable_prefix_free().
* Provide lltable_get<ifp|af>() functions to reduce direct 'struct lltable'
access by external callers.
* Remove @llt agrument from lle_free() lle callback since it was unused.
* Temporarily add L3_CADDR() macro for 'const' sockaddr typecasting.
* Switch to per-af hashing code.
* Rename LLE_FREE_LOCKED() callback from in[6]_lltable_free() to
in_[6]lltable_destroy() to avoid clashing with llt_free_entry() method.
Update description from these functions.
* Use unified lltable_free_entry() function instead of per-af one.
Alexander Motin [Mon, 10 Aug 2015 11:30:07 +0000 (11:30 +0000)]
MFV r277428: 5056 ZFS deadlock on db_mtx and dn_holds
Reviewed by: Will Andrews <willa@spectralogic.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Justin Gibbs <justing@spectralogic.com>
Currently, every buffer cached in the L2ARC is accompanied by a 240-byte
header in memory, leading to very high memory consumption when using very
large cache devices. These changes significantly reduce this overhead.
Trunk Optimized
+-----------------+
L1-only | 176 B | 176 B | (same)
+-----------------+
L1 & L2 | 240 B | 208 B | (saved 32 bytes)
+-----------------+
L2-only | 240 B | 128 B | (saved 116 bytes)
+-----------------+
For an average blocksize of 8KB, this means that for the L2ARC, the ratio
of metadata to data has gone down from about 2.92% to 1.56%. For a
'storage optimized' EC2 instance with 1600GB of SSD and 60GB of RAM, this
means that we expect a completely full L2ARC to use (1600 GB * 0.0156) /
60GB = 41% of the available memory, down from 78%.
Bjoern A. Zeeb [Mon, 10 Aug 2015 10:29:32 +0000 (10:29 +0000)]
Rather than hardcoding a string and limiting the comparison to these
characters use the defined constant so that in case of change this
would not break.