MFC r305121,r305231:

MFC r305121,r305231:

Add some locking to sc_cngetc().

Keyboard input needs Giant locking, and that is not possible to do
correctly here.  Use mtx_trylock() and proceed unlocked as before if
we can't acquire Giant (non-recursively), except in kdb mode don't
even try to acquire Giant.  Everything here is a hack, but it often
works.  Even if mtx_trylock() succeeds, this might be a LOR.

Keyboard input also needs screen locking, to handle screen updates
and switches.  Add this, using the same simplistic screen locking
as for sc_cnputc().

Giant must be acquired before the screen lock, and the screen lock
must be dropped when calling the keyboard driver (else it would get a
harmless LOR if it tries to acquire Giant).  It was intended that sc
cn open/close hide the locking calls, and they do for i/o functions
functions except for this complication.

Non-console keyboard input is still only Giant-locked, with screen
locking in some called functions.  This is correct for the keyboard
parts only.

When Giant cannot be acquired properly, atkbd and kbdmux tend to race
and work (they assume that the caller acquired Giant properly and don't
try to acquire it again or check that it has been acquired, and the
races rarely matter), while ukbd tends to deadlock or panic (since it
does the opposite, and has other usb threads to deadlock with).

The keyboard (Giant) locking here does very little, but the screen
locking completes screen locking for console mode except for not
detecting or handling deadlock.

The log message for the previous commit didn't mention the most the
important detail that sc_cngetc() now opens and closes the keyboard
on every call again.  This was moved from sc_cngetc() to scn_cngrab/
ungrab() in r228644, but the change wasn't quite complete.  After
fixes for nesting in kbdd_poll() in ukbd and kbdmux, these opens
and closes should have no significant effect if done while grabbed.
They fix unusual cases when cngetc() is called while not grabbed.

This commit is the main fix for screen locking in sc_cnputc():
detect deadlock or likely-deadlock and handle it by buffering the
output atomically and printing it later if the deadlock condition
clears (and sc_cnputc() is called).

The most common deadlock is when the screen lock is held by ourself.
Then it would be safe to acquire the lock recursively if the console
driver is calling printf() in a safe context, but we don't know when
that is.  It is not safe to ignore the lock even in kdb or panic mode.
But ignore it in panic mode.  The only other known case of deadlock
is when another thread holds the lock but is running on a stopped CPU.
Detect that case approximately by using trylock and retrying for 1000
usec.  On a 4 GHz CPU, 100 usec is almost long enough -- screen switches
take slightly longer than that.  Not retrying at all is good enough
except for stress tests, and planned future versions will extend the
timeout so that the stress tests work better.

To see the behaviour when deadlock is detected, single step through
sctty_outwakeup() (or sc_puts() to start with deadlock).  Another
(serial) console is needed to the buffered-only output, but the
keyboard works in this context to continue or step out of the
deadlocked region.  The buffer is not large enough to hold all the
output for this.