Eric van Gyzen [Sun, 28 May 2017 00:45:28 +0000 (00:45 +0000)]
Fix INSTALL_AS_USER with external nsswitch databases
The INSTALL_AS_USER option tells "install" to use the current
user name as the owner of the installed file. The "install"
command executed by the build is statically linked, so it does not
load nsswitch modules, such as nss_ldap.so, so it fails when
the user is only defined in such a database.
Fix it to use the current UID instead of user name. This works
for all users. I expect it is also slightly more efficient.
Enji Cooper [Sat, 27 May 2017 23:19:32 +0000 (23:19 +0000)]
kvm_geterr_test: Compile out the portions that require kvm_open2(3) on
systems that lack the libcall, based on __FreeBSD_version.
kvm_open2(3) wasn't made available until r291406, which is in ^/stable/11,
but not ^/stable/10. This makes some of kvm_geterr_test available for testing
on ^/stable/10.
Alan Cox [Sat, 27 May 2017 21:46:00 +0000 (21:46 +0000)]
After r118390, the variable "dmmax" was neither the correct strip size
nor the correct maximum block size. Moreover, after r318995, it serves
no purpose except to provide information to user space through a read-
sysctl.
This change eliminates the variable "dmmax" but retains the sysctl. It
also corrects the value returned by the sysctl.
Since ino64 expanded dev_t to 64bit, make VOP_GETATTR(9) provide all
bits of mnt_stat.f_fsid as va_fsid for vnodes on filesystems which use
f_fsid. In particular, NFSv3 and sometimes NFSv4, and ZFS use this
method or reporting st_dev by stat(2).
Provide a new helper vn_fsid() to avoid duplicating code to copy
f_fsid to va_fsid.
Note that the change is mostly cosmetic. Its motivation is to avoid
sign-extension of f_fsid[0] into 64bit dev_t value which happens after
dev_t becomes 64bit..
Reviewed by: avg(zfs), rmacklem (nfs) (both for previous version)
Sponsored by: The FreeBSD Foundation
Alan Cox [Sat, 27 May 2017 16:40:00 +0000 (16:40 +0000)]
In r118390, the swap pager's approach to striping swap allocation over
multiple devices was changed. However, swapoff_one() was not fully and
correctly converted. In particular, with r118390's introduction of a per-
device blist, the maximum swap block size, "dmmax", became irrelevant to
swapoff_one()'s operation. Moreover, swapoff_one() was performing out-of-
range operations on the per-device blist that were silently ignored by
blist_fill().
This change corrects both of these problems with swapoff_one(), which will
allow us to potentially increase MAX_PAGEOUT_CLUSTER. Previously,
swapoff_one() would panic inside of blist_fill() if you increased
MAX_PAGEOUT_CLUSTER.
While it sounds like a good idea to extract the RFC1048 data from PXE, in the
end it is not and it is causing lots of issues. Our pxeloader might need
options which are incompatible with other pxe servers (for example iPXE, but
not only).
Our pxe loaders are also now settings their own user class, so it is useful to
issue our own pxe request at startup
All the code are now only issueing one single dhcp request at startup of the
loader meaning we can always request a the PXE informations from the
dhcp server.
Previous code lost that information, meaning no option 55 anymore (meaning not
working with the kea dhcp server) and no request for rootpath etc, no user class
Remove the flags from the bootp function which is not needed anymore
Brooks Davis [Fri, 26 May 2017 21:10:01 +0000 (21:10 +0000)]
Add missing usage and getopt(3) options
- Add the missing option 'n' to the getopt(3) string
- Add the missing options 'libxo' and 'N' to the usage message
- Add the missing options 'M' and 'N' to the man-page
John Baldwin [Fri, 26 May 2017 20:20:40 +0000 (20:20 +0000)]
Fail large requests with EFBIG.
The adapter firmware in general does not accept PDUs larger than 64k - 1
bytes in size. Sending crypto requests larger than this size result in
hangs or incorrect output, so reject them with EFBIG. For requests
chaining an AES cipher with an HMAC, the firmware appears to require
slightly smaller requests (around 512 bytes).
Alexander Motin [Fri, 26 May 2017 20:15:33 +0000 (20:15 +0000)]
Improve applying unified capabilities to the lagg ports.
Some NICs have some capabilities dependent, so that disabling one require
disabling some other (TXCSUM/RXCSUM on em). This code tries to reach the
consensus more insistently.
Andriy Gapon [Fri, 26 May 2017 17:44:47 +0000 (17:44 +0000)]
Allow PROBE_SPINUP to fail in CAM ATA transport
The motivation for this is two-fold.
1. Some old WD SATA disks may appear as if they need to be spun up
when they are already spinning. Those disks would respond with
an error to the spin-up request.
2. Even if we really fail to spin up the disk, we still can try to
proceed to the subsequent phases. If we fail later on, then no
difference. Otherwise we get a chance to communicate with the
disk which is better than completely ignoring it, because a user
can try to recover the disk.
David Bright [Fri, 26 May 2017 16:36:30 +0000 (16:36 +0000)]
Add newsyslog capability to write RFC5424 compliant rotation message.
This modification adds the capability to newsyslog to write the
rotation message in a format that is compliant with RFC5424. This
capability is enabled on a per-log file basis through a new value
("T") in the flags field in newsyslog.conf. This is useful on systems
that use the RFC5424 format for log files so that the rotation message
format matches that of the other log messages. There has been recent
mention of adding an RFC5424 compliant mode to syslogd and at least
one alternative system log daemon (rsyslogd) that already has the
capability to use that format.
Dimitry Andric [Fri, 26 May 2017 16:29:55 +0000 (16:29 +0000)]
Define a new __INO64 macro in <sys/_types.h>, to indicate the system
uses 64-bit inode numbers. Programs can use this to avoid including
<sys/param.h>, with its associated namespace pollution.
Ed Maste [Fri, 26 May 2017 16:03:28 +0000 (16:03 +0000)]
rm stale ptrace dependencies after r305012
This is similar to r318912, except that ptrace.[sS] was previously a
file in the source tree, not a generated assembly wrapper.
Check for the existence of ptrace.[sS] in the .depend file to determine
if we have to clean it up. This is a bit hackish and will not be left
in place indefinitely, but provides a useful example case when
investigating a better solution in bmake.
Reviewed by: bdrewery
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10930
Eric van Gyzen [Fri, 26 May 2017 15:51:51 +0000 (15:51 +0000)]
libthr: prevent setcontext() from masking SIGTHR
__thr_setcontext() mistakenly tested for the presence of SIGCANCEL
in its local ucontext_t instead of the parameter. Therefore,
if a thread calls setcontext() with a context whose signal mask
contains SIGTHR (a.k.a. SIGCANCEL), that signal will be blocked,
preventing the thread from being cancelled or suspended.
https://www.illumos.org/issues/8166
If we do a scrub while a leaf device is offline (via "zpool offline"),
we will inadvertently clear the DTL (dirty time log) of the offline
device, even though it is still damaged. When the device comes back
online, we will incompletely resilver it, thinking that the scrub
repaired blocks written before the scrub was started. The incomplete
resilver can lead to data loss if there is a subsequent failure of a
different leaf device.
The fix is to never clear the DTL of offline devices. Note that if a
device is onlined while a scrub is in progress, the scrub will be
restarted.
The problem can be worked around by running "zpool scrub" after
"zpool online".
See also https://github.com/zfsonlinux/zfs/issues/5806
Reviewed by: George Wilson george.wilson@delphix.com
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Matthew Ahrens <mahrens@delphix.com>
https://www.illumos.org/issues/8166
If we do a scrub while a leaf device is offline (via "zpool offline"),
we will inadvertently clear the DTL (dirty time log) of the offline
device, even though it is still damaged. When the device comes back
online, we will incompletely resilver it, thinking that the scrub
repaired blocks written before the scrub was started. The incomplete
resilver can lead to data loss if there is a subsequent failure of a
different leaf device.
The fix is to never clear the DTL of offline devices. Note that if a
device is onlined while a scrub is in progress, the scrub will be
restarted.
The problem can be worked around by running "zpool scrub" after
"zpool online".
See also https://github.com/zfsonlinux/zfs/issues/5806
Reviewed by: George Wilson george.wilson@delphix.com
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Matthew Ahrens <mahrens@delphix.com>
https://www.illumos.org/issues/7446
Since we support whole-disk configuration for boot pool, we also will need
whole disk support with UEFI boot and for this, zpool create should create efi-
system partition.
I have borrowed the idea from oracle solaris, and introducing zpool create -
B switch to provide an way to specify that boot partition should be created.
However, there is still an question, how big should the system partition be.
For time being, I have set default size 256MB (thats minimum size for FAT32
with 4k blocks). To support custom size, the set on creation "bootsize"
property is created and so the custom size can be set as: zpool create B -
o bootsize=34MB rpool c0t0d0
After pool is created, the "bootsize" property is read only. When -B switch is
not used, the bootsize defaults to 0 and is shown in zpool get output with
value ''. Older zfs/zpool implementations are ignoring this property.
https://www.illumos.org/rb/r/219/
Reviewed by: Andrew Stormont <andyjstormont@gmail.com>
Reviewed by: Yuri Pankov <yuri.pankov@gmail.com>
Approved by: Dan McDonald <danmcd@kebe.com>
Author: Toomas Soome <tsoome@me.com>
https://www.illumos.org/issues/6418
An easy, direct means of sanitizing pool vdevs can be helpful for management
purposes.
FreeBSD has had a 'zpool labelclear' for some time, see: https://
svnweb.freebsd.org/base?view=revision&revision=224171
SpectraBSD has a slightly updated version, which I propose for inclusion.
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Will Andrews <will@firepipe.net>
Note: the bulk of the change has been already imported, this is a follow
up that imports zpool.1m changes.
https://www.illumos.org/issues/6781
cache
A device used to cache storage pool data. A cache device cannot
be cannot be configured as a mirror or raidz group. For more
information, see the "Cache Devices" section.
needs changed to
cache
A device used to cache storage pool data. A cache device cannot
be configured as a mirror or raidz group. For more
information, see the "Cache Devices" section.
Reviewed by: Toomas Soome <tsoome@me.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Matthew Ahrens <mahrens@delphix.com>
Author: Alexander Pyhalov <apyhalov@gmail.com>
https://www.illumos.org/issues/2897
Found this option in some Oracle documentation and wanted to check out the
zpool manpage on it in OI. Unfortunately it seems to be missing from the
manpage, so I first thought it was unsupported. However, "# zpool split" does
print the correct usage. Testing with the "-n" switch makes me believe it is
supported (I don't actually need to split my pool).
Reviewed by: Dan McDonald <danmcd@omniti.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Hans Rosenfeld <rosenfeld@grumpf.hope-2000.org>
Author: Steven Burgess <sburgess@datto.com>
Andriy Gapon [Fri, 26 May 2017 11:54:42 +0000 (11:54 +0000)]
4465 zpool(1M) is able to offline cache vdevs despite what man page says
5659 in the manual page for zpool(1M), one misuse of the word 'zpool' to describe a pool
https://www.illumos.org/issues/4465
zpool(1M) is able to offline cache vdevs despite man page saying that it isn't:
zpool offline [-t] pool device ...
Takes the specified physical device offline. While the device is
offline, no attempt is made to read or write to the device.
This command is not applicable to spares or cache devices.
altair:root:~# zpool create testoff c9t67d0 cache c9t71d0
altair:root:~# zpool status testoff
pool: testoff
state: ONLINE
scan: none requested
config:
errors: No known data errors
altair:root:~# zpool offline testoff c9t71d0
altair:root:~# zpool status testoff
pool: testoff
state: ONLINE
status: One or more devices has been taken offline by the administrator.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Online the device using 'zpool online' or replace the device with
'zpool replace'.
scan: none requested
https://www.illumos.org/issues/5659
At https://github.com/illumos/illumos-gate/blob/master/usr/src/man/man1m/
zpool.1m#L931
Do not add a disk that is currently configured as a quorum device to
a zpool.
– should be:
Do not add a disk that is currently configured as a quorum device to
a pool.
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Hans Rosenfeld <rosenfeld@grumpf.hope-2000.org>
Author: Yuri Pankov <yuri.pankov@nexenta.com>
https://www.illumos.org/issues/8064
It's currently nearly impossible to trace what process places a hold on
a vnode, as the only ways holds are place is via the `VN_HOLD()` and
`VN_HOLD_CALLER()` macros, which inline the bumping of `v_count`. Adding
static DTrace probes to these macros would enable tracing of where
specific vnode references come from.
For completeness and symmetry, a similar static probe should be added to
`vn_rele()` and `vn_rele_dnlc()`.
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Sebastien Roy <seb@delphix.com>
https://www.illumos.org/issues/8063
A standard practice in ZFS is to keep track of "per-txg" state. Any of
the 3 active TXG's (open, quiescing, syncing) can have different values
for this state. We should assert that we do not attempt to modify other
(inactive) TXG's.
Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
https://www.illumos.org/issues/8063
A standard practice in ZFS is to keep track of "per-txg" state. Any of
the 3 active TXG's (open, quiescing, syncing) can have different values
for this state. We should assert that we do not attempt to modify other
(inactive) TXG's.
Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
https://www.illumos.org/issues/7786
Currently, vdev_online() will only post sysevent if previous state was
"offline". It should also post the event when the state changes from "removed"
or "faulted" to "healthy" or "degraded".
This will fix the following scenario:
- pull disk from slot A
- check that hotspare has taken its place (if available)
- insert disk into slot B
- check that hotspare moved back to "avail" state (if spare was used)
The problem here is that we don't get any ESC_ZFS_VDEV_* notification and fail
to update the vdev FRU.
Reviewed by: Matthew Ahrens mahrens@delphix.com
Reviewed by: George Wilson george.wilson@delphix.com
Approved by: Albert Lee <trisk@forkgnu.org>
Author: Yuri Pankov <yuri.pankov@nexenta.com>
https://www.illumos.org/issues/7786
Currently, vdev_online() will only post sysevent if previous state was
"offline". It should also post the event when the state changes from "removed"
or "faulted" to "healthy" or "degraded".
This will fix the following scenario:
- pull disk from slot A
- check that hotspare has taken its place (if available)
- insert disk into slot B
- check that hotspare moved back to "avail" state (if spare was used)
The problem here is that we don't get any ESC_ZFS_VDEV_* notification and fail
to update the vdev FRU.
Reviewed by: Matthew Ahrens mahrens@delphix.com
Reviewed by: George Wilson george.wilson@delphix.com
Approved by: Albert Lee <trisk@forkgnu.org>
Author: Yuri Pankov <yuri.pankov@nexenta.com>
https://www.illumos.org/issues/8025
dbuf_read() creates a zio_root() to track and wait for all the zio's
that may happen as part of this call. However, if the blkptr_t for
this buffer is NULL or a hole, we will not create any more zio's, so
this zio_root() is unnecessary. This is always the case when calling
dbuf_read() on a bonus buffer, because it has no blkptr (it's part of
the containing dnode). For workloads that read a lot of bonus buffers
(e.g. file creation and removal), creating and destroying these
unnecessary zio's can decrease performance by around 3%.
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
https://www.illumos.org/issues/8025
dbuf_read() creates a zio_root() to track and wait for all the zio's
that may happen as part of this call. However, if the blkptr_t for
this buffer is NULL or a hole, we will not create any more zio's, so
this zio_root() is unnecessary. This is always the case when calling
dbuf_read() on a bonus buffer, because it has no blkptr (it's part of
the containing dnode). For workloads that read a lot of bonus buffers
(e.g. file creation and removal), creating and destroying these
unnecessary zio's can decrease performance by around 3%.
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
FreeBSD note: only a ZFS part of the change is merged, changes to the VM
subsystem are not ported (obviously). Also, now that FreeBSD has
vmem(9) we don't have to ifdef-out the code that uses it.
Andriy Gapon [Fri, 26 May 2017 11:05:56 +0000 (11:05 +0000)]
arc_init: make code closer to upstream by introducing 'allmem' variable
All the differences in calculations are kept.
A comment about arc_max being 1/2 of all memory is fixed to reflect the
actual code that uses 5/8 as a factor.
https://www.illumos.org/issues/7256
error = dmu_sync(zio, lr->lr_common.lrc_txg,
zfs_get_done, zgd);
ASSERT(error || lr->lr_length <= zp->z_blksz);
It's possible, although extremely rare, that the zfs_get_done() callback is
executed before dmu_sync() returns.
In that case the znode's range lock is dropped and the znode is unreferenced.
Thus, the assertion can access some invalid or wrong data via the zp pointer.
size variable caches the correct value of z_blksz and can be safely used here.
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Andriy Gapon <andriy.gapon@clusterhq.com>
https://www.illumos.org/issues/8061
sa_find_idx_tab() is declared as taking and returning "void *" parameters.
These can be declared to be the specific types.
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Chris Williamson <chris.williamson@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Pedro F. Giffuni [Fri, 26 May 2017 01:14:58 +0000 (01:14 +0000)]
fts_open: move bogus initialization further below, before it is used.
Move an unneeded initialization, introduced in r54770 to quiet down GCC,
to a place nearer to its first use. This has no practical effect, it just
keeps the garbage better sorted.
Hinted by: OpenBSD (CVS rev. 1.56, without obfuscations)
Ed Maste [Fri, 26 May 2017 00:51:05 +0000 (00:51 +0000)]
libc: rm stale generated files which are no longer syscalls
This is an attempt to help -DNO_CLEAN builds after r302092 (which
removed the pipe libc syscall wrapper) and r318736 (which removed
getdents, lstat, mknod, and stat).
Dependencies cannot cope with certain source tree changes,
particularly with respect to removing source files and replacing
generated files. Handle these cases from _worldtmp in an ad-hoc
fashion.
Reviewed by: bdrewery, cem
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10876
Dimitry Andric [Thu, 25 May 2017 23:14:51 +0000 (23:14 +0000)]
Pull in r303257 from upstream llvm trunk (by Krzysztof Parzyszek)
[PPC] Properly update register save area offsets
The variables MinGPR/MinG8R were not updated properly when resetting the
offsets, which in the included testcase lead to saving the CR register
in the same location as R30.
In FreeBSD we only use mdoc(7) format. A template is available as mdoc.template
The usage of man(7) format is discouraged and this file was driving people into
the front direction as a template to use.