libc: Use O_CLOEXEC for various internal file descriptors.
This fixes a race condition where another thread may fork() before CLOEXEC
is set, unintentionally passing the descriptor to the child process.
This commit only adds O_CLOEXEC flags to open() or openat() calls where no
fcntl(fd, F_SETFD, FD_CLOEXEC) follows. The separate fcntl() call still
leaves a race window so it should be fixed later.
Allow deferred word-splitting via f_sysrc_get() by allowing $IFS in the
"clean-room" environment used to query rc.conf(5) parameters.
This brings bsdconfig(8)'s sysrc.subr in-line with both the sysrc(8) manual
[provided by sysutils/sysrc] and sysrc(8)'s own sysrc.subr (now identical to
bsdconfig(8)'s sysrc.subr as of this patch).
Finally, this will allow a clean import of sysutils/sysrc (sans sysrc.subr,
already provided here).
Reviewed by: jilles
Approved by: adrian (co-mentor)
Simplify and somewhat redesign interaction between pf_purge_thread() and
pf_purge_expired_states().
Now pf purging daemon stores the current hash table index on stack
in pf_purge_thread(), and supplies it to next iteration of
pf_purge_expired_states(). The latter returns new index back.
The important change is that whenever pf_purge_expired_states() wraps
around the array it returns immediately. This makes our knowledge about
status of states expiry run more consistent. Prior to this change it
could happen that n-th run stopped on i-th entry, and returned (1) as
full run complete, then next (n+1) full run stopped on j-th entry, where
j < i, and that broke the mark-and-sweep algorythm that saves references
rules. A referenced rule was freed, and this later lead to a crash.
The drbr(9) API appeared to be so unclear, that most drivers in
tree used it incorrectly, which lead to inaccurate overrated
if_obytes accounting. The drbr(9) used to update ifnet stats on
drbr_enqueue(), which is not accurate since enqueuing doesn't
imply successful processing by driver. Dequeuing neither mean
that. Most drivers also called drbr_stats_update() which did
accounting again, leading to doubled if_obytes statistics. And
in case of severe transmitting, when a packet could be several
times enqueued and dequeued it could have been accounted several
times.
o Thus, make drbr(9) API thinner. Now drbr(9) merely chooses between
ALTQ queueing or buf_ring(9) queueing.
- It doesn't touch the buf_ring stats any more.
- It doesn't touch ifnet stats anymore.
- drbr_stats_update() no longer exists.
o buf_ring(9) handles its stats itself:
- It handles br_drops itself.
- br_prod_bytes stats are dropped. Rationale: no one ever
reads them but update of a common counter on every packet
negatively affects performance due to excessive cache
invalidation.
- buf_ring_enqueue_bytes() reduced to buf_ring_enqueue(), since
we no longer account bytes.
o Drivers handle their stats theirselves: if_obytes, if_omcasts.
o mlx4(4), igb(4), em(4), vxge(4), oce(4) and ixv(4) no longer
use drbr_stats_update(), and update ifnet stats theirselves.
o bxe(4) was the most correct driver, it didn't call
drbr_stats_update(), thus it was the only driver accurate under
moderate load. Now it also maintains stats itself.
o ixgbe(4) had already taken stats from hardware, so just
- drop software stats updating.
- take multicast packet count from hardware as well.
o mxge(4) just no longer needs NO_SLOW_STATS define.
o cxgb(4), cxgbe(4) need no change, since they obtain stats
from hardware.
Change queue overflow checks from DIAGNOSTIC+panic() to KASSERT() to make
them enabled on HEAD by default. It is probably better to do single compare
then hunt for unexpected memory corruption.
- Re-shuffle the <machine/pc/bios.h> headers to move all kernel-specific
bits under #ifdef _KERNEL but leave definitions for various structures
defined by standards ($PIR table, SMAP entries, etc.) available to
userland.
- Consolidate duplicate SMBIOS table structure definitions in ipmi(4)
and smbios(4) in <machine/pc/bios.h> and make them available to
userland.
Fix the mis-handling of the VV_TEXT on the nullfs vnodes.
If you have a binary on a filesystem which is also mounted over by
nullfs, you could execute the binary from the lower filesystem, or
from the nullfs mount. When executed from lower filesystem, the lower
vnode gets VV_TEXT flag set, and the file cannot be modified while the
binary is active. But, if executed as the nullfs alias, only the
nullfs vnode gets VV_TEXT set, and you still can open the lower vnode
for write.
Add a set of VOPs for the VV_TEXT query, set and clear operations,
which are correctly bypassed to lower vnode.
Make the loader a bit smarter, when it tries to open disk and the slice
number is not exactly specified. When the disk has MBR, also try to read
BSD label after ptable_getpart() call. When the disk has GPT, also set
d_partition to 255. Mostly, this is how it worked before.
Remove the topology lock from disk_gone(), it might be called with regular
mutexes held and the topology lock is an sx lock.
The topology lock was there to protect traversing through the list of providers
of disk's geom, but it seems that disk's geom has always exactly one provider.
Change the code to call g_wither_provider() for this one provider, which is
safe to do without holding the topology lock and assert that there is indeed
only one provider.
libc/fts: Use O_CLOEXEC for internal file descriptors.
Because fts keeps internal file descriptors open across calls, making such
descriptors close-on-exec helps not only multi-threaded applications but
also single-threaded applications.
In particular, this prevents passing a temporary file descriptor for saving
the current directory to processes created via find -exec.
Ensure that all cases that enqueue a netgraph item for delivery by a
ngthread properly set the item's depth to 1. In particular, prior to this
change if ng_snd_item failed to acquire a lock on a node, the item's depth
would not be set at all. This fix ensures that the error code from rcvmsg/
rcvdata is properly passed back to the apply callback. For example, this
fixes a bug where an error from rcvmsg/rcvdata would not previously
propagate back to a libnetgraph consumer when the message was queued.
The attempt to merge changes from the linux libtirpc caused
rpc.lockd to exit after startup under unclear conditions.
After many hours of selective experiments and inconsistent results
the conclusion is that it's better to just revert everything and
restart in a future time with a much smaller subset of the
changes.
____
MFC after: 3 days
Reported by: David Wolfskill
Tested by: David Wolfskill
The attempt to merge changes from the linux libtirpc caused
rpc.lockd to exit after startup under unclear conditions.
After many hours of selective experiments and inconsistent results
the conclusion is that it's better to just revert everything and
restart in a future time with a much smaller subset of the
changes.
____
MFC after: 3 days
Reported by: David Wolfskill
Tested by: David Wolfskill
Passing an invalid pointer results in undefined behaviour.
The wrappers in libthr access some of the data pointed to by the arguments
in userland, so that an invalid pointer will cause a signal and not an
[EFAULT] error return.
Furthermore, if the [EFAULT] error occurs when the kernel is writing, it is
not a proper error in the sense that the call still commits (changing the
signal disposition or accepting the signal).
Make sure we record NAK tokens in the TD structure for IN direction.
Improve host channel disabling. Wait two times 125us for channel to be
disabled. The DWC OTG doesn't like when channels are re-used too early.
Kernel and modules have "set_vnet" linker set, where virtualized
global variables are placed. When a module is loaded by link_elf
linker its variables from "set_vnet" linker set are copied to the
kernel "set_vnet" ("modspace") and all references to these variables
inside the module are relocated accordingly.
The issue is when a module is loaded that has references to global
variables from another, previously loaded module: these references are
not relocated so an invalid address is used when the module tries to
access the variable. The example is V_layer3_chain, defined in ipfw
module and accessed from ipfw_nat.
The same issue is with DPCPU variables, which use "set_pcpu" linker
set.
Fix this making the link_elf linker on a module load recognize
"external" DPCPU/VNET variables defined in the previously loaded
modules and relocate them accordingly. For this set_pcpu_list and
set_vnet_list are used, where the addresses of modules' "set_pcpu" and
"set_vnet" linker sets are stored.
Note, archs that use link_elf_obj (amd64) were not affected by this
issue.
Fix bug in TCP_KEEPCNT setting, which slipped in in the last round
of reviewing of r231025.
Unlike other options from this family TCP_KEEPCNT doesn't specify
time interval, but a count, thus parameter supplied doesn't need
to be multiplied by hz.
adrian [Thu, 27 Sep 2012 06:05:54 +0000 (06:05 +0000)]
Track the last ANI TX/RX sample correctly.
This doesn't specifically fix the issue(s) i'm seeing in this 2GHz
environment (where setting/increasing spur immunity causes OFDM restart
errors to skyrocket through the roof; but leaving it at 0 would leave
the environment cleaner..)
Pointy-hat-to: me, for committing this broken code in the first place.
Implementing pmap_kextract(va) as pmap_extract(kernel_pmap, va) is
problematic because some callers to pmap_kextract() expect its
implementation to be lock-less. In particular, uma_dbg_alloc() implicitly
requires this. Otherwise, lock-order reversals occur between pmap locks and
UMA zone locks. So, this change introduces a lock-less implementation of
pmap_kextract().
Disable recursion on the pvh global lock in the new armv6 pmap. While
recursion on this locks occurs in the old arm pmap, it thankfully doesn't
occur in the armv6 pmap.
Add 32-bit ABI compat shims. Those are necessary for i386 binary-only
tools like sysutils/hpacucli (HP P4xx RAID controller management
suite) working on amd64 systems.
- Make C11 atomic macros usable in expressions:
- Replace do-while statements with void expressions.
- Wrap __asm statements in statement expressions.
- Make the macros function-like:
- Evaluate all arguments exactly once.
- Make sure there's a sequence point between evaluation of the arguments
and the function body. Arguments should be evaluated before any memory
barriers.
- Fix use of __atomic_is_lock_free built-in. It requires the address of
an atomic variable as second argument. Use this built-in on clang as
well because clang's __c11_atomic_is_lock_free only takes the size of the
variable into account.
- In atomic_exchange_explicit put the barrier before instead of after the
__sync_lock_test_and_set call.
Merge similar fixes from 223198 from igb to ixgbe:
- Use a dedicated task to handle deferred transmits from the if_transmit
method instead of reusing the existing per-queue interrupt task.
Reusing the per-queue interrupt task could result in both an interrupt
thread and the taskqueue thread trying to handle received packets on a
single queue resulting in out-of-order packet processing and lock
contention.
- Don't define ixgbe_start() at all where if_transmit is used.
- Allow cross-architecture builds with 'generate-release.sh', which
is set by specifying '-a <arch>'. (Only supported for i386 on amd64
and powerpc on powerpc64 currently).
- Change how textproc/docproj is installed:
o Attempt to install from pkg(8);
o Fall back to pkg_add(1) if pkg(8) installation is not
successful;
o Fall back to installing from ports as last resort.
- Ensure the script is run by root[1].
- Get OSVERSION from param.h[1].
Add a COMPILER_FEATURES variable that is set in bsd.compiler.mk.
When the compiler is clang add c++11 to COMPILER_FEATURES.
When the compiler supports c++11, default to building libc++. This will
not impact the compliation of programs, but will make it easier for
users with clang compiled systems to test libc++ by passing -stdlib=libc++
to their c++ compliations.
mm [Wed, 26 Sep 2012 09:37:58 +0000 (09:37 +0000)]
Merge recent vendor changes in ZFS.
Illumos issued covered:
2811 missing implementation: zfs send -r
3139 zdb dies when it tries to determine path of unlinked file
3189 kernel panic in ZFS test suite during hotspare_onoffline_004_neg
3208 moving zpool cross-endian results in incorrect user/group accounting
adrian [Wed, 26 Sep 2012 03:45:42 +0000 (03:45 +0000)]
Map the non-QoS TID to the voice queue, in order to ensure important
things like EAPOL frames make it out.
After a whole bunch of hacking/testing, I discovered that they weren't
being early-dropped by the stack (but I should look at ensuring that
later..) but were even making to the hardware transmit queue.
They were mostly even being received by the remote end. However, the
remote end was completely ignoring them.
This didn't happen under 150-170MBit TCP tests as I'm guessing the TX
queue stayed very busy and the STA didn't do any scanning. However, when
doing 100Mbit/s of TCP traffic, the STA would do background scanning -
which involves it coming in and out of powersave mode with the AP.
Now, this is a total and utter hack around the real problems, which are:
* I need to implement proper power save handling and integrate it into
the filtered frames support, so the driver/stack doesn't send frames
whilst the station is actually in sleep;
* .. but frames were actually making it to the STA (macbook pro) and
the AP did receive an ACK; but a tcpdump on the receiving side showed
the EAPOL frame never made it. So the stack was dropping it for
some reason;
* Importantly - the EAPOL frames are currently going into the non-QoS
TID, which maps to the BE queue and is susceptible to that queue being
busy doing other things, but;
* There's other traffic going on in the non-QoS TID from other contexts
when scanning is going on and it's possible there's some races causing
sequence number/IV issues, but;
* Importantly importantlly, I think the interaction with TID 16 multicast
traffic in power save mode is causing issues - since I -believe- the
sequence number space being used by the EAPOL frames on TID 16 overlaps
with the multicast frames that have sequence numbers allocated and
are then stuffed on the cabq. Since with EAPOL frames being in TID 16
and queued to the BE queue, it's going to be waiting to be serviced
with all of the aggregate traffic going on - and if the CABQ gets
emptied beforehand, those TID 16 multicast frames with sequence numbers
will go out beforehand.
Now, there's quite likely a bunch of "stuff happening slightly out of
sequence" going on due to the nature of the TX path (read: lots of
overlapping and concurrent ath_start() and ath_raw_xmit() calls going
on, sigh) but I thought I had caught them all and stuffed each TID TX
behind a lock (that lasted as long as it needed to in order to get
the frame onto the relevant destination queue - thus keeping things
in order.)
Unfortunately the last problem is the big one and I'm going to stare at
it some more. If it _is_
So this is a work around for now to ensure that EAPOL frames actually
make it out before any other stuff in the non-QoS TID and HOPEFULLY
before the CABQ gets active.
I'm now going to spend a little time in the TX path figuring out exactly
why the sender is rejecting things. There's two (well, three if you count
EAPOL contents invalid) possibilities:
* The sequence number is out of order (ie, something else like the multicast
traffic on CABQ) is going out first on TID 16;
* The CCMP IV is out of order (similar to above - but less likely, as the
TX key for multicast traffic is different to unicast traffic);
* EAPOL contents strangely invalid.
Fix freebsd32_kmq_timedreceive() and freebsd32_kmq_timedsend() to use
getmq_read() and getmq_write() respectively, just like sys_kmq_timedreceive()
and sys_kmq_timedsend().
Sponsored by: FreeBSD Foundation
MFC after: 2 weeks
Avoid INVARIANTS panic destroying an in-use tap(4)
The requirement (implied by the KASSERT in tap_destroy) that the tap is
closed isn't valid; destroy_dev will block in devdrn while other threads
are in d_* functions.
Note: if_tun had the same issue, addressed in SVN revisions r186391,
r186483 and r186497. The use of the condvar there appears to be
redundant with the functionality provided by destroy_dev.
We cannot open file for reading and executing (O_RDONLY | O_EXEC).
Well, in theory we can pass those two flags, because O_RDONLY is 0,
but we won't be able to read from a descriptor opened with O_EXEC.
Update the comment.
Sponsored by: FreeBSD Foundation
MFC after: 2 weeks
O_EXEC flag is not part of the O_ACCMODE mask, check it separately.
If O_EXEC is provided don't require CAP_READ/CAP_WRITE, as O_EXEC
is mutually exclusive to O_RDONLY/O_WRONLY/O_RDWR.
Without this change CAP_FEXECVE capability right is not enforced.
Sponsored by: FreeBSD Foundation
MFC after: 3 days
adrian [Tue, 25 Sep 2012 20:41:43 +0000 (20:41 +0000)]
Oops - don't do the clrdmask check in ath_tx_xmit_normal() - the wrong
lock may be held.
Kim reported that the TID lock wasn't held when ath_tx_update_clrdmask()
was called. Well, the underlying hardware TXQ for that TID.
I'm betting it's the cabq stuff. ath_tx_xmit_normal() can be called
for both real and software cabq. For software cabq, the real destination
txq is different to the txq. So, the lock check will fail.
Change the module name for the I/O provider to "kernel" from
"genunix" This will requires us to modify externally created
DTrace scripts but makes logical sense for FreeBSD.
Some aac(4) adapters will always report that a direct access device is
offline in response to a INQUIRY command that does not retreive vital
product data(I personally have observed the behaviour on an Adaptec 2405
and a 5805). Force the peripheral qualifier to "connected" so that upper
layers correctly recognize that a disk is present.
This bug was uncovered by r216236. Prior to that fix, aac(4) was
accidentally clearing the peripheral qualifier for all inquiry commands.
This fixes an issue where passthrough devices were not created for
disks behind aac(4) controllers suffering from the bug. I have
verified that if a disk is not present that we still properly detect
that and not create the passthrough device.
Add optional entropy harvesting for software interrupts in swi_sched()
as controlled by kern.random.sys.harvest.swi. SWI harvesting feeds into
the interrupt FIFO and each event is estimated as providing a single bit of
entropy.
Attila Bogar reported a bug in mountd when multiple export
entries with different security flavors are in the exports(5)
file. For that case, mountd replies with the security flavors
of the last entry and not the correct one for the client host.
This patch fixes that by storing separate copies of the flavors
for each host/net case, plus a default one for the case where
no hosts/nets are specified on an entry in the exports(5) file.
Unlike the patch in the PR, it replies with the security flavors
for the entry instead of merging the security flavors for all
the entries and replying with that.
Use CAM_SEL_TIMEOUT and CAM_DEV_NOT_THERE to report missing targets or
LUNs respectively. This removes a huge number of error messages
from CAM during bus scans.
Copied almost verbatim from mav's commit r237460.
Submitted by: Mike Tancsa <mike at sentex dot net>
MFC after: 3 days
Specify MTX_RECURSE for the controller's io_lock. Without it, tws(4)
immediately panics on boot with INVARIANTS enabled. The driver already
clearly expects to be able to recurse on this mutex - the main I/O
is always recursing on this lock.
Reported and tested by: Mike Tancsa <mike at sentex dot net>
MFC after: 1 week
adrian [Mon, 24 Sep 2012 19:48:41 +0000 (19:48 +0000)]
Debugging output fixes:
* use the correct frame status - although the completion descriptor is
the _last_ in the frame/aggregate, the status is currently stored in
the _first_ buffer.
* Print out ath_buf specific fields once, not per descriptor in an ath_buf.
Make sure moused is exiting as quick as possible after that the
umsX character device returns a read error. Update devd.conf rules
to use "DEVFS" events which are generated after that the umsX
character device node has been created/destroyed, and then there
should be no need for moused to wait up to 10 seconds for umsX to
be ready. Opening umsX should not fail except if the kernel is low
on memory. In that case the user can replug the USB mouse or use
"usbconfig" to reset the device. In case of USB mouse devices,
moused should neither retry to open its character device,
once the first read error has happened. This is an indication
of device detach.
adrian [Mon, 24 Sep 2012 06:42:20 +0000 (06:42 +0000)]
Prepare for software retransmission of non-aggregate frames but ensure
it's disabled.
The previous commit to enable CLRDMASK setting didn't do it at all
correctly for non-aggregate sessions - so the CLRDMASK bit would be
cleared and never re-set.
* move ath_tx_update_clrdmask() to be called by functions that setup
descriptors and queue frames to the hardware, rather than scattered
everywhere.
* Force CLRDMASK to be set on all non-aggregate session frames being
transmitted.
* Use ath_tx_normal_comp() now on non-aggregate sessoin frames
that are queued via ath_tx_xmit_normal(). That way the TID hwq is
updated and they can trigger (eventual) filter frame queue resets
and software retransmits.
There's still a bit more work to do in this area to reverse the silly
short-sightedness on my part, however it's likely going to be better
to fix this now than just reverting the patch.
Thanks to people on the freebsd-wireless@ mailing list for promptly
pointing this out.
adrian [Mon, 24 Sep 2012 06:00:51 +0000 (06:00 +0000)]
In (eventual) preparation for supporting disabling the whole 11n/software
retry path - add some code to make it obvious (to me!) how to disable
the software tx path.
projects / FreeBSD / FreeBSD.git / log