andrew [Sat, 17 Oct 2015 19:52:17 +0000 (19:52 +0000)]
Replace build_section_pagetable with build_l1_block_pagetable as it takes
an extra argument to specify the number of 1GiB pages to map. This should
be a nop as we are only mapping a single page, but when we move to use an
extra level of page tables we will be able to map a second block, e.g. if
the kernel was loaded over a 1GiB boundary.
ngie [Sat, 17 Oct 2015 19:48:17 +0000 (19:48 +0000)]
Only use -fstack-protector-strong with supported compilers
This includes clang 3.5.0+, gcc 4.2.1, gcc 4.8.4+
This allows me to do subdirectory makes again after setting
MAKESYSPATH on 10.2-RELEASE as it comes with clang 3.4.1.
As a sidenote: this isn't technically correct for all vintages
of gcc 4.2.1, but will be correct when gcc is rebuilt/reinstalled
after r286074, so this version check should be good enough.
bdrewery [Sat, 17 Oct 2015 18:22:18 +0000 (18:22 +0000)]
Fix wrong PATH being set for world 'includes' stage after r289438.
The 'includes' target is currently a pseudo target in bsd.subdir.mk that
does 'cd ${.CURDIR} && ${MAKE} buildincludes && ${MAKE} installincludes',
versus all over targets that just recurse.
In Makefile.inc1 the older duplicated bsd.subdir.mk logic for calling
'includes' was being executed in each subdir directly, meaning 'cd lib && make
includes' became 'cd lib && make buildincludes && make installincludes'. Now
that the bsd.subdir.mk logic is used it is calling 'make buildincludes && make
installincludes' from the top-level which pulls in the PATH=<default path>
from /Makefile.
The sub-make logic for 'includes' in bsd.subdir.mk was attempted to be removed
in r289282 but turned out to be wrong. I have a working version now but
it is not yet ready for commit. So for now in Makefile.inc1 split out
'includes' to 'buildincludes' and 'installincludes' which will avoid the
problem.
bdrewery [Sat, 17 Oct 2015 16:42:54 +0000 (16:42 +0000)]
Rework the 'make -n -n' feature such that '-n' recurses and '-N' does not.
Bmake has a documented feature of '-N' to skip executing commands which is
specifically intended for debugging top-level builds and not recursing into
sub-directories. This matches the older 'make -n' behavior we added which made
'-n -n' the recursing target and '-n' a non-recursing target.
Removing the '-n -n' feature allows the build to work as documented in
the bmake manpage with '-n' and '-N'. The older '-n -n' feature was also
not documented anywhere that I could see.
Note that the ${_+_} var is still needed as currently bmake incorrectly
executes '+' commands when '-N' is specified.
The '-n' and '-n -n' features were broken for several reasons prior to this.
r251748 made '_+_' never expand with '-n -n' which resulted in many
sub-directories not being visited until fixed 2 years later in r288391, and
many targets were given .MAKE over the past few years which resulted in
non-sub-make commands, such as rm and ln and mtree, to be executed.
This should also allow removing some indirection hacks in bsd.subdir.mk and
other cases of .USE that have a .MAKE by using '+'.
ngie [Sat, 17 Oct 2015 09:07:53 +0000 (09:07 +0000)]
Set dev->fd to -1 when calling cam_close_spec_device with a valid dev->fd
descriptor to avoid trashing valid file descriptors that access dev->fd at a
later point in time
ngie [Sat, 17 Oct 2015 08:39:37 +0000 (08:39 +0000)]
Integrate tools/regression/acltools into the FreeBSD test suite as tests/sys/acl
- Make the requirements more complete for the testcases
- Detect prerequisites so the tests won't fail (zfs.ko is loaded, zpool(1)
is available, ACL support is enabled with UFS, etc).
- Work with temporary files/directories/mountpoints that work with atf/kyua
- Limit the testcases to work on temporary filesystems to reduce tainting the
test host
bdrewery [Sat, 17 Oct 2015 05:57:29 +0000 (05:57 +0000)]
For 'buildenvvars' show any .exported variables as well to cover recent
exporting of OSRELDATE and VERSION. These already do export to 'buildenv'
fine.
bdrewery [Sat, 17 Oct 2015 05:55:45 +0000 (05:55 +0000)]
Always export VERSION to the environment to avoid looking it up again in
sub-makes.
Some of the world phases that used plain '${MAKE} -f Makefile.inc1' were not
passing this variable along which caused them to look it up again. By
using bmake's .export we can remove it from all of the other environment
lines.
Add a comment about the usage for VERSION for ctfmerge.
ngie [Sat, 17 Oct 2015 04:32:21 +0000 (04:32 +0000)]
Integrate tools/test/posixshm and tools/regression/posixshm into the FreeBSD
test suite as tests/sys/posixshm
Some other highlights:
- Convert the testcases over to ATF
- Don't use hardcoded paths to /tmp (which violate the ATF/kyua samdbox); use
mkstemp to generate temporary paths for non-SHM_ANON shm objects.
bdrewery [Sat, 17 Oct 2015 03:51:50 +0000 (03:51 +0000)]
Rework the world subdir build targets to use the standard SUBDIR_PARALLEL mechanism.
Back in r30113, the 'par-*' targets were added to parallelize portions of
the build in a very similar fashion as the SUBDIR_PARALLEL feature used in
r263778. Calling a target without 'par-' (for 'parallel') resulted in the
standard bsd.subdir.mk handling without parallelization. Given we have
SUBDIR_PARALLEL now there is no reason to duplicate the handling here.
In build logs this will result in the ${dir}.${target}__D targets now showing
as the normal ${target}_subdir_${dir} targets.
I audited all of the uses of Makefile.inc1 and Makefile's targets that use
bsd.subdir.mk and found that all but 'all' and 'install' were fine to use
as always parallel.
- For 'install' (from installworld -j) the ordering of lib/ and libexec/
before the rest of the system (described in r289433), and etc/ being last
(described in r289435), is all that matters. So now a .WAIT is added in
the proper places when invoking any 'install*' target. A parallel
installworld does work and took 46% of the time a non-parallel
install would take on my system with -j15 to ZFS.
- For 'all' I left the default handling for this to not run in parallel. A
'par-all' target is still used by the 'everything' stage of buildworld
to continue building in parallel as it already has been. This works
because most of the dependencies are handled by the early bootstrap
phases as well as 'libraries' and 'includes' phases. This lets
all of the SUBDIR build in parallel fine, such as bin/ and lib/. This
will not work if the user invokes 'all' though as we have dependencies
spread all over the system with no way to depend between them (except
for the dirdeps feature in the META_MODE build). Calling 'make all'
from the top-level is still useful at least when using SUBDIR_OVERRIDE.
bdrewery [Fri, 16 Oct 2015 23:53:37 +0000 (23:53 +0000)]
Fix adding manpages installed by LOCAL_DIRS to whatis file.
The ordering of 'etc' in the install has a long history dating back to the
first time it was realized it needed to be "last" in r4486. That commit
still left it before LOCAL_DIRS though. By having it before LOCAL_DIRS
any manpages they install were not being added to the whatis database in the
install image. They would likely show up in the file after a periodic
rebuild of the file though.
Currently the whatis file is built by an 'afterinstall' hook in etc/Makefile
that calls share/man's 'makedb' target.
bdrewery [Fri, 16 Oct 2015 21:09:15 +0000 (21:09 +0000)]
Correct a bitrotted comment about installworld order requirements.
The case of make(1) using a new /bin/sh issue was fixed in r173219 when ITOOLS
was introduced.
There are still issues with mid-install errors leaving a system unusable that
are currently non-trivial to solve. The safest ordering requires installing
rtld, libc and libthr (in that order) before anything else. We don't do that
now though. Much improvement is needed here still.
Discussed with: kip and kan (rtld/library ordering)
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
This is only a partial merge of respective ZFS infrastructure changes.
At this moment FreeBSD kernel has no those crypto algorithms, so the
parts of the code to enable them are commented out. When they are
implemented, it will be trivial to plug them in.
des [Fri, 16 Oct 2015 12:21:44 +0000 (12:21 +0000)]
Fix two bugs in HTTPS tunnelling:
- If the proxy returns a non-200 result, set the error code accordingly
so the caller / user gets a somewhat meaningful error message.
- Consume and discard any HTTP response header following the result line.
ed [Fri, 16 Oct 2015 10:26:15 +0000 (10:26 +0000)]
Use the right variable name.
MACHINE_CPUARCH expands to aarch64 for arm64, whereas MACHINE always
corresponds to the directory name under sys/ that contains the sources
for that architecture.
bdrewery [Fri, 16 Oct 2015 05:54:41 +0000 (05:54 +0000)]
Avoid warning race with creating 'ldscripts' directory during build.
In r204548 the 'rm -f ldscripts' was added likely due to reading the
conditional as 'else it is a file'. That seems unlikely though and
the more likely case is just that the directory hasn't been created yet.
Because this races with ,ssother scripts, use 'mkdir -p' which is a minimal
modification vs upstream to avoid the warning of 'File exists' if another
script creates it first. This could replace the 'test -d' as well but
then it's more unneeded change to the upstream script.
imp [Fri, 16 Oct 2015 03:06:02 +0000 (03:06 +0000)]
Do not relocate extents to make them contiguous if the underlying drive can do
deletions. Ability to do deletions is a strong indication that this
optimization will not help performance. It will only generate extra write
traffic. These devices are typically flash based and have a limited number of
write cycles. In addition, making the file contiguous in LBA space doesn't
improve the access times from flash devices because they have no seek time.
bdrewery [Fri, 16 Oct 2015 00:38:05 +0000 (00:38 +0000)]
Rename the /usr/share/doc/legal files to driver.LICENSE to work around
bug of installing 'realtek' and 'intel_iwn' as files rather then as
a 'LICENSE' file in their directories.
Also add obsolete entries for the older names and names that existed in head
for a period of time.
cem [Thu, 15 Oct 2015 23:45:43 +0000 (23:45 +0000)]
NTB: Add variable number MW, DB CB support code
This is a follow-up to r289208: "Xeon Errata Workaround."
Add logic to support a variable number of memory windows and doorbell
callbacks. This was added to the Linux driver in the "Xeon Errata
Workaround" commit, but I skipped it because it didn't look neccessary
at the time. It is needed for future Haswell split-BAR support, so
bring it in now.
A new tunable was added for if_ntb, 'hw.ntb.max_num_clients'. By
default, it is set to zero -- infer the number of clients from the
number of memory windows available from the hardware. Any other
positive value can specify a different number of clients, limited by the
number of doorbell callbacks available (4 under MSI-X, or 15 (Xeon) or
34 (SoC) under legacy INTx).
bdrewery [Thu, 15 Oct 2015 22:49:56 +0000 (22:49 +0000)]
Make installing to a non-existent directory an error.
Before this, if a file was installed to DESTDIR/some/dir and that directory
was missing due to not having ran 'make distrib-dirs' yet, the file would
be installed as 'some/dir'. For something like bsd.incs.mk with INCLUDEDIR
being a sub-directory of /usr/include, this could result in all of the headers
being installed to a file rather than getting a directory of them.
Now it will error that the file/directory does not exist rather than hide
the issue.
Another option being discussed is to implement GNU's install -D flag which
would auto create any missing directories.
This is a mitigation of the problem. The proper order to the build is to
run 'make distrib-dirs' first, but that can be forgotten if building from
a sub-directory after updating the source code to the latest revision.
ed [Thu, 15 Oct 2015 17:50:28 +0000 (17:50 +0000)]
Properly set the return value for casueword to 0 upon success.
While trying to get multithreading working for CloudABI on aarch64, I
noticed that compare-and-exchange operations in kernelspace would always
fail. It turns out that we don't properly set the return value to 0 when
the compare and exchange succeeds.
Approved by: andrew
Differential Revision: https://reviews.freebsd.org/D3899
kib [Thu, 15 Oct 2015 17:40:39 +0000 (17:40 +0000)]
ARM userspace accessors, e.g. {s,f}uword(9), copy{in,out}(9),
casuword(9) and others, use LDRT and STRT instructions to access
memory with the privileges of userspace. If the *RT instruction
faults on the kernel address, then additional checks must be done to
not confuse the VM system with invalid kernel-mode faults.
Put ARM on line with other FreeBSD architectures and disallow usermode
buffers which intersect with the kernel address space in advance,
before any accesses are performed. In other words, vm_fault(9) is no
longer called when e.g. suword(9) stores to invalid (i.e. not
userspace) address.
Also, switch ARM to use fueword(9) and casueword(9).
Note: there is a pending patch in D3617, which adds the special
processing for faults from LDRT and STRT. The addition of the
processing is useful for potential other uses of the instructions and
for completeness, but standard userspace accessors are better served
by not allowing such faults beforehand.
Reviewed by: andrew
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3816
MFC after: 2 weeks
emaste [Thu, 15 Oct 2015 16:19:00 +0000 (16:19 +0000)]
newfs_msdos: rework option parsing to match NetBSD
NetBSD split newfs_msdos in two so that they can reuse the file system
creation part in makefs. This change is a step on the path of bringing
that support to FreeBSD.
Reviewed by: kib, pfg
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3905
mav [Thu, 15 Oct 2015 08:47:32 +0000 (08:47 +0000)]
MFV r289312: 2605 want to resume interrupted zfs send
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Richard Elling <Richard.Elling@RichardElling.com>
Reviewed by: Xin Li <delphij@freebsd.org>
Reviewed by: Arne Jansen <sensille@gmx.net>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>
For more info, see:
- slides http://www.slideshare.net/MatthewAhrens/openzfs-send-and-receive
- video https://www.youtube.com/watch?v=iY44jPMvxog
- manpage changes (for zfs resume -s and zfs send -t)
- upcoming talk at the OpenZFS Developer Summit
The TL;DR is:
Use "zfs receive -s" to save the partially received state on failure.
On failure, get the receive token with "zfs get receive_resume_token <fs>"
Resume the send with "zfs send -t <token_value>"
adrian [Thu, 15 Oct 2015 04:22:56 +0000 (04:22 +0000)]
Add support for the BCM57765 card reader.
This patch adds support for the BCM57765[2] card reader function included in
Broadcom's BCM57766 ethernet/sd3.0 controller. This controller is commonly
found in laptops and Apple hardware (MBP, iMac, etc).
The BCM57765 chipset is almost fully compatible with the SD3.0 spec, but
does not support deriving a frequency below 781KHz from its default base
clock via the standard SD3.0-configured 10-bit clock divisor.
If such a divisor is set, card identification (which requires a 400KHz
clock frequency) will time out[1].
As a work-around, I've made use of an undocumented device-specific clock
control register to switch the controller to a 63MHz clock source when
targeting clock speeds below 781KHz; the clock source is likewise switched
back to the 200MHz clock when targeting speeds greater than 781KHz.
Additionally, this patch fixes a small sdhci_pci bug; the
sdhci_pci_softc->quirks flag was not copied to the sdhci_slot, resulting in
`quirk` behavior not being applied by sdhci.c.
[1] A number of Linux/FreeBSD users have noted that bringing up the chipsets'
associated ethernet interface will allow SD cards to enumerate (slowly).
This is a controller implementation side-effect triggered by the ethernet
driver's reading of the hardware statistics registers.
[2] This may also fix card detection when using the BCM57785 chipset, but I
don't have access to the BCM57785 chipset and can't verify.
I actually snagged some BCM57785 hardware recently (2012 Retina MacBook Pro)
and can confirm that this also fixes card enumeration with the BCM57785
chipset; with the patch, I can boot off of the internal sdcard reader.
bz [Thu, 15 Oct 2015 01:51:10 +0000 (01:51 +0000)]
2nd try, after r289319:
HWPMC depends on pmu.c even if device pmu is not specified.
Would be great if we could just automatically enabled "device pmu"
if we try to compile in HWPMC.
Also several old kernel cnfigurations seem to have HWPMC enabled but are
pre-FDT and thus fail. So make pmu.c depend on fdt in case of hwpmc as
well.
cem [Wed, 14 Oct 2015 23:48:16 +0000 (23:48 +0000)]
NTB: MFV 1db97f25: Pull out platform detection logic
Pull out read of PPD and platform detection logic to new functions,
ntb_detect_xeon(), ntb_detect_soc(). No functional change -- mostly
this is just shuffling the code to more closely match the Linux driver.
Linux commit log:
To simplify some of the platform detection code. Move the platform
detection to a function to be called earlier.
Authored by: Dave Jiang
Obtained from: Linux (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division
cem [Wed, 14 Oct 2015 23:48:03 +0000 (23:48 +0000)]
NTB: Abstract doorbell register access
The doorbell registers (and associated mask) are 16-bit on Xeon but
64-bit on SoC. Abstract IO access to doorbell registers with
'db_ioread' and 'db_iowrite' (names and idea borrowed from the dual
BSD/GPL Linux driver).
cem [Wed, 14 Oct 2015 23:47:52 +0000 (23:47 +0000)]
if_ntb: MFV 3cc5ba19: Add alignment check to meet hardware requirement
Original Linux commit log:
The NTB translate register must have the value to be BAR size aligned.
This alignment check make sure that the DMA memory allocated has the
proper alignment. Another requirement for NTB to function properly with
memory window BAR size greater or equal to 4M is to use the CMA feature
in 3.16 kernel with the appropriate CONFIG_CMA_ALIGNMENT and
CONFIG_CMA_SIZE_MBYTES set.
Authored by: Dave Jiang
Obtained from: Linux (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division
cem [Wed, 14 Oct 2015 23:47:35 +0000 (23:47 +0000)]
NTB: MFV a1413cfb: correct the spread of queues over mw's
The detection of an uneven number of queues on the given memory windows
was not correct. The mw_num is zero based and the mod should be
division to spread them evenly over the mw's.
Authored by: Jon Mason
Obtained from: Linux (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division
cem [Wed, 14 Oct 2015 23:47:23 +0000 (23:47 +0000)]
NTB: Remap MSI-X messages over available slots
Remap MSI-X messages over available slots rather than falling back to
legacy INTx when fewer MSI-X slots are available than were requested.
N.B. the Linux driver does *not* do this.
To aid in testing, a tunable 'hw.ntb.force_remap_mode' has been added.
It defaults to off (0). When the tunable is enabled and sufficient
slots were available, the driver restricts the number of slots by one
and remaps the MSI-X messages over the remaining slots.
In case this is actually not okay (as I don't yet have access to this
hardware to test), a tunable 'hw.ntb.prefer_intx_to_remap' has been
added. It defaults to off (0). When the tunable is enabled and fewer
slots are available than requested, fall back to legacy INTx mode rather
than attempting to remap MSI-X messages.
cem [Wed, 14 Oct 2015 23:46:15 +0000 (23:46 +0000)]
NTB: MFV 53a788a7: Split ntb_setup_interrupts() into SOC, Xeon, and legacy routines
The names don't line up 100% with Linux. Our routines are named
ntb_setup_interrupts, ntb_setup_xeon_msix, ntb_setup_soc_msix, and
ntb_setup_legacy_interrupt. Linux SNB = FreeBSD Xeon; Linux BWD =
FreeBSD SOC. Original Linux commit log:
This is an cleanup effort to make ntb_setup_msix() more readable - use
ntb_setup_bwd_msix() to init MSI-Xs on BWD hardware and
ntb_setup_snb_msix() - on SNB hardware.
Function ntb_setup_snb_msix() also initializes MSI-Xs the way it should
has been done - looping pci_enable_msix() until success or failure.
Authored by: Alexander Gordeev
Obtained from: Linux (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division
gjb [Wed, 14 Oct 2015 22:33:11 +0000 (22:33 +0000)]
Deprecate MD5 checksum generation in favor of SHA512.
This was discussed during the 10.2-RELEASE cycle, however
since we were nearing the end of the cycle, we decided to
defer this change until after 10.2-RELEASE.
Reminded by: so (delphij), jmg
MFC after: 5 days
Sponsored by: The FreeBSD Foundation
bdrewery [Wed, 14 Oct 2015 20:30:32 +0000 (20:30 +0000)]
Recurse on 'buildconfig' and 'installconfig'. Remove the 'config' pseudo target.
The 'config' target isn't really needed right now so just remove it to avoid
any clashes with config(8) building. It's also likely misspelled and should
be 'configs' if we decide to add it back. This was just a convenience
target recently added.
projects / FreeBSD / FreeBSD.git / log