mckusick [Tue, 16 Aug 2016 21:02:30 +0000 (21:02 +0000)]
Bug 211013 reports that a write error to a UFS filesystem running
with softupdates panics the kernel. The problem that has been pointed
out is that when there is a transient write error on certain metadata
blocks, specifically directory blocks (PAGEDEP), inode blocks
(INODEDEP), indirect pointer blocks (INDIRDEPS), and cylinder group
(BMSAFEMAP, but only when journaling is enabled), we get a panic
in one of the routines called by softdep_disk_io_initiation that
the I/O is "already started" when we retry the write.
These dependency types potentially need to do roll-backs when called
by softdep_disk_io_initiation before doing a write and then a
roll-forward when called by softdep_disk_write_complete after the
I/O completes. The panic happens when there is a transient error.
At the top of softdep_disk_write_complete we check to see if the
write had an error and if an error occurred we just return. This
return is correct most of the time because the main role of the routines
called by softdep_disk_write_complete is to process the now-completed
dependencies so that the next I/O steps can happen.
But for the four types listed above, they do not get to do their
rollback operations. This causes the panic when softdep_disk_io_initiation
gets called on the second attempt to do the write and the roll-back
routines find that the roll-backs have already been done. As an
aside I note that there is also the problem that the buffer will
have been unlocked and thus made visible to the filesystem and to
user applications with the roll-backs in place.
The way to resolve the problem is to add a flag to the routines called
by softdep_disk_write_complete for the four dependency types noted
that indicates whether the write was successful (WRITESUCCEEDED).
If the write does not succeed, they do just the roll-backs and then
return. If the write was successful they also do their usual
processing of the now-completed dependencies.
The fix was tested by selectively injecting write errors for buffers
holding dependencies of each of the four types noted above and then
verifying that the kernel no longer paniced and that following the
successful retry of the write that the filesystem could be unmounted
and successfully checked cleanly.
kib [Tue, 16 Aug 2016 17:30:58 +0000 (17:30 +0000)]
In UFS_BALLOC(), invalidate pages of indirect buffers on failed block
allocation unwinding.
Dandling buffers are released on UFS_BALLOC() failure to ensure that
later attempt to allocate blocks in close range do not find the blocks
with invalid content, since possible partial block allocations are
unwound. As such, it is not enough to just release the buffers, the
pages must also invalidated and removed from the vnode vm_object
queue. Otherwise the pages might be found later and used to
reconstruct indirect buffers when doing allocations at offset close to
the failure point, and their stale content compromise the filesystem
integrity.
Note that just marking the buffer as B_INVAL is not enough, B_NOCACHE
is required. To be sure, clear the B_CACHE flag as well. This
complements the r174973, which started releasing buffers.
Reported and tested by: pho
Reviewed by: mckusick
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
kib [Tue, 16 Aug 2016 17:18:38 +0000 (17:18 +0000)]
On unwind after failed block allocation in ffs_balloc_ufs{1,2}, assert
that recorded allocated blocks numbers match the physical block
numbers of dandling buffers which are released.
When finally freeing the blocks during unwind, assert that dandling
buffers where not re-allocated. They shouldn't, because the vnode lock
is owned exclusive.
Reviewed by: mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
mckusick [Tue, 16 Aug 2016 17:07:48 +0000 (17:07 +0000)]
Add two new macros, SLIST_CONCAT and LIST_CONCAT. Note in both the
queue.h header file and in the queue.3 manual page that they are O(n)
so should be used only in low-usage paths with short lists (otherwise
an STAILQ or TAILQ should be used).
kib [Tue, 16 Aug 2016 16:50:48 +0000 (16:50 +0000)]
When block allocation fails in UFS_BALLOC(), and the volume does not
have SU enabled, there is no point in calling softdep_request_cleanup().
The call cannot produce free blocks, but we unecessarily lock ufsmount
and do inode block write. Usual point of not doing optimizations for
the corner case of the full volume is not applicable there, the work
is easily avoidable, and the addition CPU and disk io load do not lead
to succeeding retry.
Reviewed by: mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
kib [Tue, 16 Aug 2016 16:49:56 +0000 (16:49 +0000)]
In ffs_balloc_ufs{1,2} routines, assert that unwind records do not
overflow local arrays. This is not immediately obvious from the
static code inspection, due to retry logic.
Reviewed by: mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
rrs [Tue, 16 Aug 2016 15:11:46 +0000 (15:11 +0000)]
Here we update the modular tcp to be able to switch to an
alternate TCP stack in other then the closed state (pre-listen/connect).
The idea is that *if* that is supported by the alternate stack, it
is asked if its ok to switch. If it approves the "handoff" then we
allow the switch to happen. Also the fini() function now gets a flag
to tell if you are switching away *or* the tcb is destroyed. The
init() call into the alternate stack is moved to the end so the
tcb is more fully formed before the init transpires.
Sponsored by: Netflix Inc.
Differential Revision: D6790
Add syntactic sugar to dtrace: "if" and "else" statements. The sugar is
baked down to standard dtrace features by adding additional clauses with
the appropriate predicates.
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Bryan Cantrill <bryan@joyent.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Matthew Ahrens <mahrens@delphix.com>
markj [Tue, 16 Aug 2016 02:25:19 +0000 (02:25 +0000)]
MFV r301526:
7035 string-related subroutines should validate input earlier
Reviewed by: Alex Wilson <alex.wilson@joyent.com>
Reviewed by: Bryan Cantrill <bryan@joyent.com>
Approved by: Matthew Ahrens <mahrens@delphix.com>
Author: Patrick Mooney <pmooney@pfmooney.com>
markj [Tue, 16 Aug 2016 02:20:02 +0000 (02:20 +0000)]
MFV r301525:
7033 ustack helper should fault on bad return values
Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
Reviewed by: Bryan Cantrill <bryan@joyent.com>
Approved by: Matthew Ahrens <mahrens@delphix.com>
Author: Alex Wilson <alex.wilson@joyent.com>
markj [Tue, 16 Aug 2016 02:18:34 +0000 (02:18 +0000)]
MFV r301524:
7034 negative record sizes should be rejected
Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
Reviewed by: Bryan Cantrill <bryan@joyent.com>
Approved by: Matthew Ahrens <mahrens@delphix.com>
Author: Alex Wilson <alex.wilson@joyent.com>
Reviewed by: Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Gary Mills <gary_mills@fastmail.fm>
jhb [Mon, 15 Aug 2016 20:38:02 +0000 (20:38 +0000)]
Remove the mcd(4) driver for Mitsumi CD-ROM players.
This is a driver for a pre-ATAPI ISA CD-ROM adapter. As noted in
the manpage, this driver is only useful as a backend to cdcontrol to
play audio CDs since it doesn't use DMA, so its data performance is
"abysmal" (and that was true in the mid 90's).
badger [Mon, 15 Aug 2016 20:09:09 +0000 (20:09 +0000)]
sem_post(): wake up the sleeper only after adjusting has_waiters
If the caller of sem_post() wakes up a thread sleeping via sem_wait()
before it clears the has_waiters flag, the caller of sem_wait() has no way of
knowing when it is safe to destroy the semaphore and reuse the memory. This is
because the caller of sem_post() may be interrupted between the wake step and
the clearing of has_waiters. It will then write into the has_waiters flag in
userspace after being preempted for some unknown amount of time.
bde [Mon, 15 Aug 2016 19:37:18 +0000 (19:37 +0000)]
Restructure the grabbing functions into mere wrappers of new open and
close functions. Scattered calls to sc_cnputc() and sc_cngetc() were
broken by turning the semi-reentrant inline context-switching code in
these functions into the grabbing functions. cncheckc() calls for
panic dumps are the main broken case. The grabbing functions have
special behaviour (mainly screen switching in sc_cngrab()) which makes
them unsuitable as replacements for the inline code.
kib [Mon, 15 Aug 2016 19:17:00 +0000 (19:17 +0000)]
Implement VOP_FDATASYNC() for msdosfs.
Standard VOP_FSYNC() implementation just syncs data buffers, and due
to this, is the correct and efficient implementation for msdosfs or
any other filesystem which uses bufer cache trivially. Provide
globally visible wrapper vop_stdfdatasync_buf() for future consumption
by other filesystems.
Reviewed by: mckusick
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D7471
kib [Mon, 15 Aug 2016 19:08:51 +0000 (19:08 +0000)]
Add an implementation of fdatasync(2).
The syscall is a trivial wrapper around new VOP_FDATASYNC(), sharing
code with fsync(2). For all filesystems, this commit provides the
implementation which delegates the work of VOP_FDATASYNC() to
VOP_FSYNC(). This is functionally correct but not efficient.
This is not yet POSIX-compliant implementation, because it does not
ensure that queued AIO requests are completed before returning.
ed [Mon, 15 Aug 2016 19:05:41 +0000 (19:05 +0000)]
Disable tests for non-standard behaviour of dirname(3)/basename(3).
The NetBSD ATF tests explicitly check that these functions do not modify
their input. These tests are NetBSD-specific. They test for something
that is not part of POSIX.
bde [Mon, 15 Aug 2016 18:02:37 +0000 (18:02 +0000)]
Fix restoring the kbd_mode part of the keyboard state in grab/ungrab.
Simply change the mode to K_XLATE using a local variable and use the
grab level as a flag to tell screen switches not to change it again,
so that we don't need to switch scp->kbd_mode. We did the latter,
but didn't have the complications to update the keyboard mode switch
for every screen switch. sc->kbd_mode remains at its user setting
for all scp's and ungrabbing restores to it.
jhb [Mon, 15 Aug 2016 17:42:54 +0000 (17:42 +0000)]
Add support for register dumps on VF devices.
- Add handling of VF register sets to t4_get_regs_len() and t4_get_regs().
- While here, use t4_get_regs_len() in the ioctl handler for regdump
instead of inlining it.
jhb [Mon, 15 Aug 2016 17:41:34 +0000 (17:41 +0000)]
Update mailbox writes to work with VF devices.
- Use alternate register locations for the data and control registers for
VFs.
- Do a dummy read to force the writes to the mailbox data registers to
post before the write to the control register on VFs.
- Do not check the PCI-e firmware register for errors on VFs.
jhb [Mon, 15 Aug 2016 17:40:05 +0000 (17:40 +0000)]
Make SGE parameter handling more VF-friendly.
Add fields to hold the SGE control register and free list buffer sizes to
the sge_params structure. Populate these new fields in
t4_init_sge_params() for PF devices and change t4_read_chip_settings() to
pull these values out of the params structure instead of reading
registers directly. This will permit t4_read_chip_settings() to be reused
for VF devices which cannot read SGE registers directly.
While here, move the call to t4_init_sge_params() to
get_params__post_init(). The VF driver will populate the SGE parameters
structure via a different method before calling t4_read_chip_settings().
bde [Mon, 15 Aug 2016 17:11:05 +0000 (17:11 +0000)]
[Oops, the previous commit was missing the update to syscons.h.]
Like scr_lock, the grab count needs to be per-physical-device to work.
This bug corrupted the grab count on both vtys if the ungrabbed vty is
different from the console, and failed to restore the keyboard state
on the ungrabbed vty, but not restoring it usually left the keyboard
mode part of the keyboard state uncorrupted at 1 (K_XLATE), while
after this fix the keyboard mode part is usually corrupted to 0 (K_RAW).
While here, rename the grab count from grabbed to grab_level.
bde [Mon, 15 Aug 2016 17:08:25 +0000 (17:08 +0000)]
Like scr_lock, the grab count needs to be per-physical-device to work.
This bug corrupted the grab count on both vtys if the ungrabbed vty is
different from the console, and failed to restore the keyboard state
on the ungrabbed vty, but not restoring the latter usually left the
keyboard mode part of it uncorrupted at 1 (K_XLATE), while after this
fix the keyboard mode part is usually corrupted to 0 (K_RAW).
While here, rename the grab count from 'grabbed' to grab_level.
bde [Mon, 15 Aug 2016 15:34:53 +0000 (15:34 +0000)]
Disable some more unsafe things in (low level) console mode:
- never call up to the tty layer to restart output for keyboard input in
console mode. This was already disallowed in kdb mode. Other cases
are rarely reached.
- disable the reboot, halt and powerdown keys in console mode. The suspend,
standby and panic keys are still allowed, and aren't even conditonal
on excessive configuration options. Some of these actions are still
available in ddb mode as ddb commands which are equally unsafe. Some
are useful at input prompts and should be restored when the locking is
fixed.
- disallow bells in kdb mode (should be in console mode, but the flag for
that is not available). Visual bell gives very alarming behaviour by
trying to use callouts which don't work in kdb mode. Audio bell uses
timeouts and hardware resources with mutexes that can deadlock in
reasonable use of ddb.
Screen switches in kdb mode are not very safe, but they are important
functionality and there is a lot of code to make them sort of work.
asomers [Mon, 15 Aug 2016 14:58:25 +0000 (14:58 +0000)]
Decrease the anti-congestion sleep in 480.leapfile-ntpd to 1 hour
24 hours is too long. Periodic scripts are executed serially, so when
combined with the sleep in 410.pkg-audit periodic could actually take more
than 24 hours and block the next invocation.
bde [Mon, 15 Aug 2016 14:28:16 +0000 (14:28 +0000)]
Change all uses of 'debugger' to kdb_active and remove this variable. This
restores avoidance of doing dangerous things like calling wakeup() and
callouts while in ddb.
Initialization of 'debugger' was broken by removing the cndbctl() console
method that was used mainly in this driver to initialize 'debugger' and
switch to the console screen on entry to ddb. The screen switch was
restored using the cngrab() method, but cngrab() is more general so it
should not initialize 'debugger' and never did. 'debugger' was just
an over-engineered alias for kdb_active anyway. It existed because
kdb_active (when it was named ddb_active) was considered as a private
kdb variable, and there are ordering problems initializing the variables
atomically with the state that they represent, but an extra variable and
method to set it increased these problems.
The bug caused LORs, but WITNESS is normally misconfigured with
WITNESS_SKIPSIN so it doesn't check the spinlocks used by wakeup() and
callouts.
bde [Mon, 15 Aug 2016 12:56:45 +0000 (12:56 +0000)]
Quick fix for locking fixes in r172250. The lock added there was per-
virtual-device, but needs to be per-physical-device so that it protects
shared data. Usually, scp->sc->write_in_progress got corrupted first
and further corruption was limited when this variable was left at nonzero
with no write in progress.
Attempt to fix missing lock destruction in r162285. Put it with the
lock destruction for r172250 after moving the latter. Both might be
unreachable.
To demonstrate the bug, find a buggy syscall or sysctl that calls
printf(9) and run this often. Run hd /dev/zero >/dev/ttyvN for any
N != 0. The console spam goes to ttyv0 and the non-console spam goes
to ttyvN, so the lock provided no protection (but it helped for
N == 0).
tuexen [Mon, 15 Aug 2016 10:16:08 +0000 (10:16 +0000)]
Ensure that sctp_it_ctl.cur_it does not point to a free object (during
a small time window).
Thanks to Byron Campen for reporting the issue and
suggesting a fix.
adrian [Sun, 14 Aug 2016 19:04:37 +0000 (19:04 +0000)]
[mips] convert over =v to =r for the inline assembly.
Later gcc and clang have deprecated =v (which maps to a specific temp
register) and instead we should just use =r to have the assembler
(hopefully!) save/restore things appropriately after choosing
a register.
Tested:
* AR9344 SoC, with userreg support
* AR9331 SoC, with no userreg support
bde [Sun, 14 Aug 2016 16:51:25 +0000 (16:51 +0000)]
Fix the variables $esp, $ds, $es, $fs, $gs and $ss in vm86 mode.
Fix PC_REGS() so that printing of instructions works in some useful
cases. ddb only understands a single flat address space, but this
macro allows mapping $cs:$eip into vm86's flat address space well
enough for the MI parts of ddb. This doesn't work for the MD parts
that do stack traces, and there are no similar macros for data addresses.
PC_REGS() has to use the trapframe pointer instead of the pcb for this.
For other CPUs, the trapframe pointer is not available except by tracing
back to it. But tracing back through vm86 trapframes is broken even
starting with one.
bde [Sun, 14 Aug 2016 15:52:00 +0000 (15:52 +0000)]
Print the tid of curthread in "show pcpu" in ddb.
It was remarkably hard to trace all current threads. "show pcpu" only
showed the pid, and there was nothing (?) better than searching ps output
to find the tids on CPUs. This change simplifies the search, but you
still have to trace the tid for each CPU manually.
bde [Sun, 14 Aug 2016 15:26:40 +0000 (15:26 +0000)]
In ddb ps, left justify the non-numeric fields 'state' and 'wmesg' and
the fixed-width numeric field 'wchan', as in ps(1). They were sort
of centered, although the template shows 'state' as right-justified.
The `wmesg' field very rarely has a prefix of '*' (for lock names)
that is still to the left of the header, and the width of this field
is reduced from 8 to 7 (more than 6 is an error).
The 'wmesg' and 'wchan' fields are still misnamed and poorly handled.
They are named sort of backwards relative to ps(1):
- wmesg in ddb = mwchan in ps
- wmesg in ddb = wchan in ps (if it is a wait channel name, not a lock name)
- wchan in ddb = nwchan in ps
ddb ps wastes lots of space for the unimportant 'wchan' field (20
columns altogether on 64-bit arches). ps(1) documents using a
compressed format, but the compression only omits leading nybbles of
0 so it has neveqr worked on arches that put the kernel in the top half
of the address space. It just avoids wasting space for an 0x prefix.
bde [Sun, 14 Aug 2016 13:23:02 +0000 (13:23 +0000)]
Don't print an extra newline after the instruction when printing for
single stepping of multiple instructions (e.g., s/p,<count> and n/p).
db_print_loc_and_inst() already prints a newline on all arches although
it probably shouldn't.
Especially on SMP systems, single stepping tends to deadlock or panic
too quickly to be useful for anything except finding bugs in itself,
but with printing "itself" includes console drivers so it is useful
for generating stress tests for console drivers.
mm [Sun, 14 Aug 2016 09:26:10 +0000 (09:26 +0000)]
MFV r304060:
Sync libarchive with vendor including three security fixes
Vendor issues fixed:
Issue #744: Very long pathnames evade symlink checks
Issue #748: libarchive can compress, but cannot decompress zip some files
PR #750: ustar: fix out of bounds read on empty string ("") filename
PR #755: fix use of acl_get_flagset_np() on FreeBSD
jhibbits [Sun, 14 Aug 2016 04:11:36 +0000 (04:11 +0000)]
Return 0 instead of an error code on failure to find dcsr.
mpc85xx_map_dcsr() returns a vm_offset_t, not an error code.
mpc85xx_fix_errata() will gracefully exit if mpc85xx_map_dcsr() returns 0, as
that indicates an error (NULL pointer).
jhibbits [Sun, 14 Aug 2016 03:49:37 +0000 (03:49 +0000)]
Only flush bp_kernload from the dcache, no need to sync the icache on the boot CPU.
__syncicache() only syncs the icache on the current CPU, it doesn't touch the
cache on any other core. Replace the call with cpu_flush_dcache() instead.
Since bp_kernload is not touched again by the boot CPU in this code path, dcbf
is no less efficient than the dcbst from __syncicache() by invalidating the
cache line.
Issue #744: Very long pathnames evade symlink checks
Issue #748: libarchive can compress, but cannot decompress zip some files
PR #750: ustar: fix out of bounds read on empty string ("") filename
PR #755: fix use of acl_get_flagset_np() on FreeBSD
Add syntactic sugar to dtrace: "if" and "else" statements. The sugar is
baked down to standard dtrace features by adding additional clauses with
the appropriate predicates.
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Bryan Cantrill <bryan@joyent.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Gary Mills <gary_mills@fastmail.fm>
jhibbits [Sat, 13 Aug 2016 18:46:49 +0000 (18:46 +0000)]
Add a kdb show command to print arbitrary SPRs on PowerPC
Summary:
There is often a need at the debugger to print arbitrary special
purpose registers (SPRs) on PowerPC. Using a rewritable asm stub, print any SPR
provided on the command line.
Note, as there is no checking in this, attempting to print a nonexistent SPR
may cause a Program exception (illegal instruction, or boundedly undefined).
Note also that this relies on the kernel text pages being writable. If in the
future this is made not the case, this will need to be reworked.
Test Plan:
Printing the Processor Version Register (PVR, SPR 287):
projects / FreeBSD / FreeBSD.git / log