Replace dummy implementation of RCU in the LinuxKPI with one based on
the in-kernel concurrency kit's ck_epoch API. Factor RCU hlist_xxx()
functions into own rculist.h header file.
Andriy Gapon [Tue, 21 Feb 2017 17:47:08 +0000 (17:47 +0000)]
reimplement zfsctl (.zfs) support
The current code is written on top of GFS, a library with the generic
support for writing filesystems, which was ported from illumos.
Because of significant differences between illumos VFS and FreeBSD
VFS models, both the GFS and zfsctl code were heavily modified to
work on FreeBSD. Nonetheless, they still contain quite a few ugly
hacks and bugs.
This is a reimplementation of the zfsctl code where the VFS-specific
bits are written from scratch and only the code that interacts with
the rest of ZFS is reused.
Some highlights.
We use two types of nodes, static and on-demand. The static nodes
are used for permanent directories like .zfs, .zfs/snapshot, etc. The
on-demand nodes are used for ephemeral directories that act as snapshot
mount points.
Initially only static nodes are created. Their vnodes are instantiated
when they are looked up. The on-demand nodes and vnodes are instantiated
as needed and the nodes are destroyed as soon as the corresponding
vnodes are reclaimed.
We also try very hard to ensure that uncovered snapshot vnodes do not
linger. They are supposed to become inactive as soon as they are
uncovered and we try to recycle them immediately.
When a filesystem is unmounted all snapshots under .zfs are unmounted
first, then all vnodes are flushed and finally the static .zfs nodes
are destroyed.
There are some changes outside of zfsctl code too.
z_ctldir is never used directly (as it is an opaque pointer),
zfsctl_root() has to be used instead. The function returns a locked
vnode now, so it accepts a lock flags parameter. The function can
also fail now, e.g. during force unmounting, whereas previously it
was infallible.
zfsctl_root_lookup() is retired, instead of it VOP_LOOKUP() on the .zfs
vnode (obtained with zfsctl_root) is used.
Some ideas are picked from an independent work by will.
Get rid of foo_sys() in linuxulator code. It was commented out, and it
would be useless anyway - there is no point in pretending to have block
devices; our "block" devices are in fact character ones, and can only
be accessed as such.
Alexander Motin [Tue, 21 Feb 2017 14:31:58 +0000 (14:31 +0000)]
Remove duplicate INOT allocation.
For some reason isp_handle_platform_notify_fc() allocated INOT just
before calling isp_handle_platform_target_tmf(), which also allocates
INOT. It seems to be a braino introduced in r196008.
1) Add better spinlock debug names when WITNESS_ALL is defined.
2) Make sure that the calling thread gets bound to the current CPU
while a spinlock is locked. Some Linux kernel code depends on that the
CPU ID doesn't change while a spinlock is locked.
3) Add support for using LinuxKPI spinlocks during a panic().
Tasklets are implemented using a taskqueue and a small statemachine on
top. The additional statemachine is required to ensure all LinuxKPI
tasklets get serialized. FreeBSD taskqueues do not guarantee
serialisation of its tasks, except when there is only one worker
thread configured.
Make the LinuxKPI task struct persistent accross system calls.
A set of helper functions have been added to manage the life of the
LinuxKPI task struct. When an external system call or task is invoked,
a check is made to create the task struct by demand. A thread
destructor callback is registered to free the task struct when a
thread exits to avoid memory leaks.
This change lays the ground for emulating the Linux kernel more
closely which is a dependency by the code using the LinuxKPI APIs.
Add new dedicated td_lkpi_task field has been added to struct thread
instead of abusing td_retval[1].
Fix some header file inclusions to make LINT kernel build properly
after this change.
Bump the __FreeBSD_version to force a rebuild of all kernel modules.
Bartek Rutkowski [Tue, 21 Feb 2017 09:37:33 +0000 (09:37 +0000)]
Enable bsdinstall hardening options by default.
As discussed previously, in order to introduce new OS hardening
defaults, we've added them to bsdinstall in 'off by default' mode.
It has been there for a while, so the next step is to change them
to 'on by defaul' mode, so that in future we could simply enable
them in base OS.
Reviewed by: brd
Approved by: adrian
Differential Revision: https://reviews.freebsd.org/D9641
Alexander Motin [Tue, 21 Feb 2017 06:10:11 +0000 (06:10 +0000)]
Do not blindly free completed ATIOs/INOTs on invalidation.
When LUN is disabled, SIM starts returning queued ATIOs/INOTs. But at the
same time there can be some ATIOs/INOTs still carrying real new requests.
If we free those, SIM may leak some resources, forever expecting for any
response from us. So try to be careful, separating ATIOs/INOTs carrying
requests which still must be processed, from ATIOs/INOTs completed with
errors which can be freed.
John Baldwin [Mon, 20 Feb 2017 20:37:25 +0000 (20:37 +0000)]
Consolidate statements to initialize files.
Previously, the first lines of various generated files from system call
tables were generated in two sections. Some of the initialization was
done in BEGIN, and the rest was done when the first line was encountered.
The main reason for this split before r313564 was that most of the
initialization done in the second section depended on the $FreeBSD$ tag
extracted from the system call table. Now that the $FreeBSD$ tag is no
longer used, consolidate all of the file initialization in the BEGIN
section.
This change was tested by confirming that the content of generated files
did not change.
Eric Badger [Mon, 20 Feb 2017 15:53:16 +0000 (15:53 +0000)]
Defer ptracestop() signals that cannot be delivered immediately
When a thread is stopped in ptracestop(), the ptrace(2) user may request
a signal be delivered upon resumption of the thread. Heretofore, those signals
were discarded unless ptracestop()'s caller was issignal(). Fix this by
modifying ptracestop() to queue up signals requested by the ptrace user that
will be delivered when possible. Take special care when the signal is SIGKILL
(usually generated from a PT_KILL request); no new stop events should be
triggered after a PT_KILL.
Add a number of tests for the new functionality. Several tests were authored
by jhb.
Adrian Chadd [Mon, 20 Feb 2017 02:08:08 +0000 (02:08 +0000)]
[net80211] RX parameter shuffle in net80211 in preparation for 4x4 NICs and 160MHz channels.
* Migrate the rx_params stuff out from ieee80211_freebsd.h where it doesn't belong -
this isn't freebsd specific anymore.
* Don't use a hard-coded number of chains in the ioctl header; now we can shuffle
MAX_CHAINS around so it can be used in the right spot.
* Extend the signal/noisefloor levels in the mimo stats struct to userland to include
the signal and noisefloor levels for each 20MHz slice of a 160MHz channel.
* Bump the number of EVM pilots in preparation for 4x4 and 160MHz channels.
Tested:
* ath(4), STA mode
* iwn(4), STA mode
* local ath10k port, STA mode
TODO:
* 11ax chips will come with 5GHz 8x8 hardware for lots of MU-MIMO - I'll re-bump it
at that point.
Note:
* This breaks the driver and ifconfig ABI; please recompile the kernel,
ifconfig and wpa_supplicant/hostapd.
When using libfetch in an application that drops privileges when fetching
like pkg(8) then user complain because the application does not read anymore
${HOME}/.netrc. Now a caller can prepare a fd to the said file and manually
assign it to the structure.
It is also a first step to allow to capsicumize libfetch applications
Reviewed by: allanjude, des
Approved by: des
Differential Revision: https://reviews.freebsd.org/D9678
Enji Cooper [Sun, 19 Feb 2017 22:00:11 +0000 (22:00 +0000)]
A forced commit to note other portion of the Makefile change accidentally
committed in r313972
The code committed in r313962 implicitly relies on python 2.x to generate
testvect.h . There are a handful of issues with this approach:
- python is not an explicit build dependency for FreeBSD
- python 2.x is deprecated and will be removed sometime in the future
(potentially before 11.x's EOL), and the script does not function with
python 3.5 (it uses deprecated idioms and incompatible function calls).
- python(1) (by default) lives in /usr/local/bin (${LOCALBASE}/bin) and
gentestvect.py is a dependency of testvect.h (prior to r313972) which
means that if the mtime of the generator script was newer than the
mtime of the test vector, it could cause a spurious build failure in
build time or at install time.
Right now the noexec mount option disallows image activators to try
execve the files on the mount point. Also, after r127187, noexec
also limits max_prot map entries permissions for mappings of files
from such mounts, but not the actual mapping permissions.
As result, the API behaviour is inconsistent. The files from noexec
mount can be mapped with PROT_EXEC, but if mprotect(2) drops execution
permission, it cannot be re-enabled later. Make this consistent
logically and aligned with behaviour of other systems, by disallowing
PROT_EXEC for mmap(2).
Note that this change only ensures aligned results from mmap(2) and
mprotect(2), it does not prevent actual code execution from files
coming from noexec mount. Such files can always be read into
anonymous executable memory and executed from there.
Reported by: shamaz.mazum@gmail.com
PR: 217062
Reviewed by: alc
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Allan Jude [Sun, 19 Feb 2017 19:30:31 +0000 (19:30 +0000)]
improve PBKDF2 performance
The PBKDF2 in sys/geom/eli/pkcs5v2.c is around half the speed it could be
GELI's PBKDF2 uses a simple benchmark to determine a number of iterations
that will takes approximately 2 seconds. The security provided is actually
half what is expected, because an attacker could use the optimized
algorithm to brute force the key in half the expected time.
With this change, all newly generated GELI keys will be approximately 2x
as strong. Previously generated keys will talk half as long to calculate,
resulting in faster mounting of encrypted volumes. Users may choose to
rekey, to generate a new key with the larger default number of iterations
using the geli(8) setkey command.
Security of existing data is not compromised, as ~1 second per brute force
attempt is still a very high threshold.
Add USB Mass Storage CTL frontend. This makes it possible
for USB OTG-capable hardware to implement device side of USB
Mass Storage, ie pretend it's a flash drive. It's configured
in the same way as other CTL frontends, using ctladm(8)
or ctld(8). Differently from usfs(4), all the configuration
can be done without rebuilding the kernel.
Testing and review is welcome. Right now I'm still moving,
and I don't have access to my test environment, so I'm somewhat
reluctant to making larger changes to this code; on the other
hand I don't want to let it sit on Phab until my testing setup
is back, because I want to get it into 11.1-RELEASE.
Ed Maste [Sun, 19 Feb 2017 17:40:24 +0000 (17:40 +0000)]
bsdgrep: document ignored option -u
MSDOS and Windows GNU grep uses -u to mean "print byte offsets as if
running on an UNIX system." The option has no effect on systems that
do not use CRLF line endings.
Ed Maste [Sun, 19 Feb 2017 17:23:27 +0000 (17:23 +0000)]
bsdgrep: fix EOF handling with --mmap
Rework part of the loop in grep_fgetln to return the rest of the line
and ensure that we still advance the buffer by the length of the rest
of the line.
There are some Linux binaries that expect the system to obey the "addr"
parameter to mmap(2), even if MAP_FIXED is not explicitly specified.
Android ART is one example. Implement bug compatibility for this case
in linuxulator.
Mateusz Guzik [Sun, 19 Feb 2017 16:28:46 +0000 (16:28 +0000)]
locks: make trylock routines check for 'unowned' value
Since fcmpset can fail without lock contention e.g. on arm, it was possible
to get spurious failures when the caller was expecting the primitive to succeed.
Make sure the thread constructor and destructor eventhandlers are
called for all threads belonging to a procedure. Currently the first
thread in a procedure is kept around as an optimisation step and is
never freed. Because the first thread in a procedure is never freed
nor allocated, its destructor and constructor callbacks are never
called which means per thread structures allocated by dtrace and the
Linux emulation layers for example, might be present for threads which
don't need these structures.
This patch adds a thread construction and destruction call for the
first thread in a procedure.
Enji Cooper [Sat, 18 Feb 2017 21:47:32 +0000 (21:47 +0000)]
Include ${LOCALBASE}/bin in $PATH when running "make checkworld"
Some of the tests in devel/atf // devel/kyua rely on the tools being in $PATH,
which means that the tests fail when run via "make checkworld" because $PATH
is restricted to exclude directory elements like "${LOCALBASE}/bin".
Ian Lepore [Sat, 18 Feb 2017 18:31:42 +0000 (18:31 +0000)]
Add dtb files for Boundary Devices Nitrogen 6 family boards. FreeBSD runs
on a nitrogen6x quad 2gb, and is expected to run on all the imx6 boards in
the nitrogen6 family.
Alexander Motin [Sat, 18 Feb 2017 06:03:16 +0000 (06:03 +0000)]
Change XCOPY memory allocations.
Before this change XCOPY code could allocate memory in chunks up to 16-32MB
(VMware does XCOPY in 4MB chunks by default), that could be difficult for
VM subsystem to do due to KVA fragmentation, that sometimes created huge
allocation delays, blocking any I/O for respective LU for that time.
This change limits allocations down to TPC_MAX_IO_SIZE, which is 1MB now.
1MB is also not a cookie, but ZFS also can do that for large blocks, so
it should be less dramatic. As drawback this increases CPU overhead, but
it still look acceptable comparing to time consumed by ZFS read/write.
projects / FreeBSD / FreeBSD.git / log