Alexander Motin [Sun, 18 Oct 2015 11:21:08 +0000 (11:21 +0000)]
MFV r289493: 5745 zfs set allows only one dataset property to be set at a time
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Reviewed by: Richard PALO <richard@NetBSD.org>
Reviewed by: Steven Hartland <killing@multiplay.co.uk>
Approved by: Rich Lowe <richlowe@richlowe.net>
Author: Chris Williamson <chris.williamson@delphix.com>
Only marker is guaranteed to be present on the queue after the relock
in vm_pageout_fallback_object_lock() and vm_pageout_page_lock(). The
check for the m->queue == queue assumes that the page does belong to a
queue.
Modify the 'unchanged' calculation bu dereferencing the marker tailq
pointers, which is known to belong to the queue. Since for a page m
linked to the queue, m->queue must be equal to the queue index, assert
this instead of checking.
In collaboration with: alc
Sponsored by: The FreeBSD Foundation (kib)
MFC after: 2 weeks
Alexander Motin [Sun, 18 Oct 2015 08:12:05 +0000 (08:12 +0000)]
5745 zfs set allows only one dataset property to be set at a time
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Reviewed by: Richard PALO <richard@NetBSD.org>
Reviewed by: Steven Hartland <killing@multiplay.co.uk>
Approved by: Rich Lowe <richlowe@richlowe.net>
Author: Chris Williamson <chris.williamson@delphix.com>
Enji Cooper [Sun, 18 Oct 2015 04:07:40 +0000 (04:07 +0000)]
Only enable -fstack-protector-strong on gcc 4.9+ and default to -fstack-protector
when -fstack-protector-strong is not available, like it was implicitly before
r288669
As noted by antoine@, devel/gcc (which is 4.8.5) lacks -fstack-protector-strong
support, whereas 4.8.4i (devel/gcc48) has the support.
Until a version is available which has -fstack-protector-strong support, be
conservative and only enable support with 4.9+.
Ian Lepore [Sun, 18 Oct 2015 01:03:43 +0000 (01:03 +0000)]
Fix a strange macro re-definition compile error. If the VM_MAXUSER_ADDRESS
value is defined as a config option the definition is emitted into
opt_global.h which is force-included into everything. In addition, the
symbol is emitted by the genassym mechanism, but that by its nature reduces
the value to a 0xnnnnnnnn number. When compiling a .S file you end up
with two different definitions of the macro (they evaluate to the same
number, but the text is different, upsetting the compiler).
Nothing has changed about this code for a while but the compile error is
new, so this must be fallout from the clang 3.7 update or something.
Adrian Chadd [Sun, 18 Oct 2015 00:59:28 +0000 (00:59 +0000)]
if_arge: fix up TX workaround; add TX/RX requirements for busdma; add stats
The early ethernet MACs (I think AR71xx and AR913x) require that both
TX and RX require 4-byte alignment for all packets.
The later MACs have started relaxing the requirements.
For now, the 1-byte TX and 1-byte RX alignment requirements are only for
the QCA955x SoCs. I'll add in the relaxed requirements as I review the
datasheets and do testing.
* Add a hardware flags field and 1-byte / 4-byte TX/RX alignment.
* .. defaulting to 4-byte TX and 4-byte RX alignment.
* Only enforce the TX alignment fixup if the hardware requires a 4-byte
TX alignment. This avoids a call to m_defrag().
* Add counters for various situations for further debugging.
* Set the 1-byte and 4-byte busdma alignment requirement when
the tag is created.
This improves the straight bridging performance from 130mbit/sec
to 180mbit/sec, purely by removing the need for TX path bounce buffers.
The main performance issue is the RX alignment requirement and any RX
bounce buffering that's occuring. (In a local test, removing the RX
fixup path and just aligning buffers raises the performance to above
400mbit/sec.
In theory it's a no-op for SoCs before the QCA955x.
Tested:
* QCA9558 SoC in AP135 board, using software bridging between arge0/arge1.
Andrew Turner [Sat, 17 Oct 2015 19:52:17 +0000 (19:52 +0000)]
Replace build_section_pagetable with build_l1_block_pagetable as it takes
an extra argument to specify the number of 1GiB pages to map. This should
be a nop as we are only mapping a single page, but when we move to use an
extra level of page tables we will be able to map a second block, e.g. if
the kernel was loaded over a 1GiB boundary.
Enji Cooper [Sat, 17 Oct 2015 19:48:17 +0000 (19:48 +0000)]
Only use -fstack-protector-strong with supported compilers
This includes clang 3.5.0+, gcc 4.2.1, gcc 4.8.4+
This allows me to do subdirectory makes again after setting
MAKESYSPATH on 10.2-RELEASE as it comes with clang 3.4.1.
As a sidenote: this isn't technically correct for all vintages
of gcc 4.2.1, but will be correct when gcc is rebuilt/reinstalled
after r286074, so this version check should be good enough.
Bryan Drewery [Sat, 17 Oct 2015 18:22:18 +0000 (18:22 +0000)]
Fix wrong PATH being set for world 'includes' stage after r289438.
The 'includes' target is currently a pseudo target in bsd.subdir.mk that
does 'cd ${.CURDIR} && ${MAKE} buildincludes && ${MAKE} installincludes',
versus all over targets that just recurse.
In Makefile.inc1 the older duplicated bsd.subdir.mk logic for calling
'includes' was being executed in each subdir directly, meaning 'cd lib && make
includes' became 'cd lib && make buildincludes && make installincludes'. Now
that the bsd.subdir.mk logic is used it is calling 'make buildincludes && make
installincludes' from the top-level which pulls in the PATH=<default path>
from /Makefile.
The sub-make logic for 'includes' in bsd.subdir.mk was attempted to be removed
in r289282 but turned out to be wrong. I have a working version now but
it is not yet ready for commit. So for now in Makefile.inc1 split out
'includes' to 'buildincludes' and 'installincludes' which will avoid the
problem.
Bryan Drewery [Sat, 17 Oct 2015 16:42:54 +0000 (16:42 +0000)]
Rework the 'make -n -n' feature such that '-n' recurses and '-N' does not.
Bmake has a documented feature of '-N' to skip executing commands which is
specifically intended for debugging top-level builds and not recursing into
sub-directories. This matches the older 'make -n' behavior we added which made
'-n -n' the recursing target and '-n' a non-recursing target.
Removing the '-n -n' feature allows the build to work as documented in
the bmake manpage with '-n' and '-N'. The older '-n -n' feature was also
not documented anywhere that I could see.
Note that the ${_+_} var is still needed as currently bmake incorrectly
executes '+' commands when '-N' is specified.
The '-n' and '-n -n' features were broken for several reasons prior to this.
r251748 made '_+_' never expand with '-n -n' which resulted in many
sub-directories not being visited until fixed 2 years later in r288391, and
many targets were given .MAKE over the past few years which resulted in
non-sub-make commands, such as rm and ln and mtree, to be executed.
This should also allow removing some indirection hacks in bsd.subdir.mk and
other cases of .USE that have a .MAKE by using '+'.
Enji Cooper [Sat, 17 Oct 2015 09:07:53 +0000 (09:07 +0000)]
Set dev->fd to -1 when calling cam_close_spec_device with a valid dev->fd
descriptor to avoid trashing valid file descriptors that access dev->fd at a
later point in time
Enji Cooper [Sat, 17 Oct 2015 08:39:37 +0000 (08:39 +0000)]
Integrate tools/regression/acltools into the FreeBSD test suite as tests/sys/acl
- Make the requirements more complete for the testcases
- Detect prerequisites so the tests won't fail (zfs.ko is loaded, zpool(1)
is available, ACL support is enabled with UFS, etc).
- Work with temporary files/directories/mountpoints that work with atf/kyua
- Limit the testcases to work on temporary filesystems to reduce tainting the
test host
Bryan Drewery [Sat, 17 Oct 2015 05:57:29 +0000 (05:57 +0000)]
For 'buildenvvars' show any .exported variables as well to cover recent
exporting of OSRELDATE and VERSION. These already do export to 'buildenv'
fine.
Bryan Drewery [Sat, 17 Oct 2015 05:55:45 +0000 (05:55 +0000)]
Always export VERSION to the environment to avoid looking it up again in
sub-makes.
Some of the world phases that used plain '${MAKE} -f Makefile.inc1' were not
passing this variable along which caused them to look it up again. By
using bmake's .export we can remove it from all of the other environment
lines.
Add a comment about the usage for VERSION for ctfmerge.
Enji Cooper [Sat, 17 Oct 2015 04:32:21 +0000 (04:32 +0000)]
Integrate tools/test/posixshm and tools/regression/posixshm into the FreeBSD
test suite as tests/sys/posixshm
Some other highlights:
- Convert the testcases over to ATF
- Don't use hardcoded paths to /tmp (which violate the ATF/kyua samdbox); use
mkstemp to generate temporary paths for non-SHM_ANON shm objects.
Bryan Drewery [Sat, 17 Oct 2015 03:51:50 +0000 (03:51 +0000)]
Rework the world subdir build targets to use the standard SUBDIR_PARALLEL mechanism.
Back in r30113, the 'par-*' targets were added to parallelize portions of
the build in a very similar fashion as the SUBDIR_PARALLEL feature used in
r263778. Calling a target without 'par-' (for 'parallel') resulted in the
standard bsd.subdir.mk handling without parallelization. Given we have
SUBDIR_PARALLEL now there is no reason to duplicate the handling here.
In build logs this will result in the ${dir}.${target}__D targets now showing
as the normal ${target}_subdir_${dir} targets.
I audited all of the uses of Makefile.inc1 and Makefile's targets that use
bsd.subdir.mk and found that all but 'all' and 'install' were fine to use
as always parallel.
- For 'install' (from installworld -j) the ordering of lib/ and libexec/
before the rest of the system (described in r289433), and etc/ being last
(described in r289435), is all that matters. So now a .WAIT is added in
the proper places when invoking any 'install*' target. A parallel
installworld does work and took 46% of the time a non-parallel
install would take on my system with -j15 to ZFS.
- For 'all' I left the default handling for this to not run in parallel. A
'par-all' target is still used by the 'everything' stage of buildworld
to continue building in parallel as it already has been. This works
because most of the dependencies are handled by the early bootstrap
phases as well as 'libraries' and 'includes' phases. This lets
all of the SUBDIR build in parallel fine, such as bin/ and lib/. This
will not work if the user invokes 'all' though as we have dependencies
spread all over the system with no way to depend between them (except
for the dirdeps feature in the META_MODE build). Calling 'make all'
from the top-level is still useful at least when using SUBDIR_OVERRIDE.
Bryan Drewery [Fri, 16 Oct 2015 23:53:37 +0000 (23:53 +0000)]
Fix adding manpages installed by LOCAL_DIRS to whatis file.
The ordering of 'etc' in the install has a long history dating back to the
first time it was realized it needed to be "last" in r4486. That commit
still left it before LOCAL_DIRS though. By having it before LOCAL_DIRS
any manpages they install were not being added to the whatis database in the
install image. They would likely show up in the file after a periodic
rebuild of the file though.
Currently the whatis file is built by an 'afterinstall' hook in etc/Makefile
that calls share/man's 'makedb' target.
Bryan Drewery [Fri, 16 Oct 2015 21:09:15 +0000 (21:09 +0000)]
Correct a bitrotted comment about installworld order requirements.
The case of make(1) using a new /bin/sh issue was fixed in r173219 when ITOOLS
was introduced.
There are still issues with mid-install errors leaving a system unusable that
are currently non-trivial to solve. The safest ordering requires installing
rtld, libc and libthr (in that order) before anything else. We don't do that
now though. Much improvement is needed here still.
Discussed with: kip and kan (rtld/library ordering)
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
This is only a partial merge of respective ZFS infrastructure changes.
At this moment FreeBSD kernel has no those crypto algorithms, so the
parts of the code to enable them are commented out. When they are
implemented, it will be trivial to plug them in.
Fix two bugs in HTTPS tunnelling:
- If the proxy returns a non-200 result, set the error code accordingly
so the caller / user gets a somewhat meaningful error message.
- Consume and discard any HTTP response header following the result line.
Ed Schouten [Fri, 16 Oct 2015 10:26:15 +0000 (10:26 +0000)]
Use the right variable name.
MACHINE_CPUARCH expands to aarch64 for arm64, whereas MACHINE always
corresponds to the directory name under sys/ that contains the sources
for that architecture.
Bryan Drewery [Fri, 16 Oct 2015 05:54:41 +0000 (05:54 +0000)]
Avoid warning race with creating 'ldscripts' directory during build.
In r204548 the 'rm -f ldscripts' was added likely due to reading the
conditional as 'else it is a file'. That seems unlikely though and
the more likely case is just that the directory hasn't been created yet.
Because this races with ,ssother scripts, use 'mkdir -p' which is a minimal
modification vs upstream to avoid the warning of 'File exists' if another
script creates it first. This could replace the 'test -d' as well but
then it's more unneeded change to the upstream script.
Warner Losh [Fri, 16 Oct 2015 03:06:02 +0000 (03:06 +0000)]
Do not relocate extents to make them contiguous if the underlying drive can do
deletions. Ability to do deletions is a strong indication that this
optimization will not help performance. It will only generate extra write
traffic. These devices are typically flash based and have a limited number of
write cycles. In addition, making the file contiguous in LBA space doesn't
improve the access times from flash devices because they have no seek time.
Bryan Drewery [Fri, 16 Oct 2015 00:38:05 +0000 (00:38 +0000)]
Rename the /usr/share/doc/legal files to driver.LICENSE to work around
bug of installing 'realtek' and 'intel_iwn' as files rather then as
a 'LICENSE' file in their directories.
Also add obsolete entries for the older names and names that existed in head
for a period of time.
Conrad Meyer [Thu, 15 Oct 2015 23:45:43 +0000 (23:45 +0000)]
NTB: Add variable number MW, DB CB support code
This is a follow-up to r289208: "Xeon Errata Workaround."
Add logic to support a variable number of memory windows and doorbell
callbacks. This was added to the Linux driver in the "Xeon Errata
Workaround" commit, but I skipped it because it didn't look neccessary
at the time. It is needed for future Haswell split-BAR support, so
bring it in now.
A new tunable was added for if_ntb, 'hw.ntb.max_num_clients'. By
default, it is set to zero -- infer the number of clients from the
number of memory windows available from the hardware. Any other
positive value can specify a different number of clients, limited by the
number of doorbell callbacks available (4 under MSI-X, or 15 (Xeon) or
34 (SoC) under legacy INTx).
Bryan Drewery [Thu, 15 Oct 2015 22:49:56 +0000 (22:49 +0000)]
Make installing to a non-existent directory an error.
Before this, if a file was installed to DESTDIR/some/dir and that directory
was missing due to not having ran 'make distrib-dirs' yet, the file would
be installed as 'some/dir'. For something like bsd.incs.mk with INCLUDEDIR
being a sub-directory of /usr/include, this could result in all of the headers
being installed to a file rather than getting a directory of them.
Now it will error that the file/directory does not exist rather than hide
the issue.
Another option being discussed is to implement GNU's install -D flag which
would auto create any missing directories.
This is a mitigation of the problem. The proper order to the build is to
run 'make distrib-dirs' first, but that can be forgotten if building from
a sub-directory after updating the source code to the latest revision.
Ed Schouten [Thu, 15 Oct 2015 17:50:28 +0000 (17:50 +0000)]
Properly set the return value for casueword to 0 upon success.
While trying to get multithreading working for CloudABI on aarch64, I
noticed that compare-and-exchange operations in kernelspace would always
fail. It turns out that we don't properly set the return value to 0 when
the compare and exchange succeeds.
Approved by: andrew
Differential Revision: https://reviews.freebsd.org/D3899
ARM userspace accessors, e.g. {s,f}uword(9), copy{in,out}(9),
casuword(9) and others, use LDRT and STRT instructions to access
memory with the privileges of userspace. If the *RT instruction
faults on the kernel address, then additional checks must be done to
not confuse the VM system with invalid kernel-mode faults.
Put ARM on line with other FreeBSD architectures and disallow usermode
buffers which intersect with the kernel address space in advance,
before any accesses are performed. In other words, vm_fault(9) is no
longer called when e.g. suword(9) stores to invalid (i.e. not
userspace) address.
Also, switch ARM to use fueword(9) and casueword(9).
Note: there is a pending patch in D3617, which adds the special
processing for faults from LDRT and STRT. The addition of the
processing is useful for potential other uses of the instructions and
for completeness, but standard userspace accessors are better served
by not allowing such faults beforehand.
Reviewed by: andrew
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3816
MFC after: 2 weeks
Ed Maste [Thu, 15 Oct 2015 16:19:00 +0000 (16:19 +0000)]
newfs_msdos: rework option parsing to match NetBSD
NetBSD split newfs_msdos in two so that they can reuse the file system
creation part in makefs. This change is a step on the path of bringing
that support to FreeBSD.
Reviewed by: kib, pfg
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3905
Alexander Motin [Thu, 15 Oct 2015 08:47:32 +0000 (08:47 +0000)]
MFV r289312: 2605 want to resume interrupted zfs send
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Richard Elling <Richard.Elling@RichardElling.com>
Reviewed by: Xin Li <delphij@freebsd.org>
Reviewed by: Arne Jansen <sensille@gmx.net>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>
For more info, see:
- slides http://www.slideshare.net/MatthewAhrens/openzfs-send-and-receive
- video https://www.youtube.com/watch?v=iY44jPMvxog
- manpage changes (for zfs resume -s and zfs send -t)
- upcoming talk at the OpenZFS Developer Summit
The TL;DR is:
Use "zfs receive -s" to save the partially received state on failure.
On failure, get the receive token with "zfs get receive_resume_token <fs>"
Resume the send with "zfs send -t <token_value>"
Adrian Chadd [Thu, 15 Oct 2015 04:22:56 +0000 (04:22 +0000)]
Add support for the BCM57765 card reader.
This patch adds support for the BCM57765[2] card reader function included in
Broadcom's BCM57766 ethernet/sd3.0 controller. This controller is commonly
found in laptops and Apple hardware (MBP, iMac, etc).
The BCM57765 chipset is almost fully compatible with the SD3.0 spec, but
does not support deriving a frequency below 781KHz from its default base
clock via the standard SD3.0-configured 10-bit clock divisor.
If such a divisor is set, card identification (which requires a 400KHz
clock frequency) will time out[1].
As a work-around, I've made use of an undocumented device-specific clock
control register to switch the controller to a 63MHz clock source when
targeting clock speeds below 781KHz; the clock source is likewise switched
back to the 200MHz clock when targeting speeds greater than 781KHz.
Additionally, this patch fixes a small sdhci_pci bug; the
sdhci_pci_softc->quirks flag was not copied to the sdhci_slot, resulting in
`quirk` behavior not being applied by sdhci.c.
[1] A number of Linux/FreeBSD users have noted that bringing up the chipsets'
associated ethernet interface will allow SD cards to enumerate (slowly).
This is a controller implementation side-effect triggered by the ethernet
driver's reading of the hardware statistics registers.
[2] This may also fix card detection when using the BCM57785 chipset, but I
don't have access to the BCM57785 chipset and can't verify.
I actually snagged some BCM57785 hardware recently (2012 Retina MacBook Pro)
and can confirm that this also fixes card enumeration with the BCM57785
chipset; with the patch, I can boot off of the internal sdcard reader.
Bjoern A. Zeeb [Thu, 15 Oct 2015 01:51:10 +0000 (01:51 +0000)]
2nd try, after r289319:
HWPMC depends on pmu.c even if device pmu is not specified.
Would be great if we could just automatically enabled "device pmu"
if we try to compile in HWPMC.
Also several old kernel cnfigurations seem to have HWPMC enabled but are
pre-FDT and thus fail. So make pmu.c depend on fdt in case of hwpmc as
well.
Conrad Meyer [Wed, 14 Oct 2015 23:48:16 +0000 (23:48 +0000)]
NTB: MFV 1db97f25: Pull out platform detection logic
Pull out read of PPD and platform detection logic to new functions,
ntb_detect_xeon(), ntb_detect_soc(). No functional change -- mostly
this is just shuffling the code to more closely match the Linux driver.
Linux commit log:
To simplify some of the platform detection code. Move the platform
detection to a function to be called earlier.
Authored by: Dave Jiang
Obtained from: Linux (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division
Conrad Meyer [Wed, 14 Oct 2015 23:48:03 +0000 (23:48 +0000)]
NTB: Abstract doorbell register access
The doorbell registers (and associated mask) are 16-bit on Xeon but
64-bit on SoC. Abstract IO access to doorbell registers with
'db_ioread' and 'db_iowrite' (names and idea borrowed from the dual
BSD/GPL Linux driver).