eadler [Mon, 19 Mar 2018 03:22:43 +0000 (03:22 +0000)]
MFC r320210:
join(1): Fix field ordering for -v output
Per POSIX, join(1) (in modes other than -o) is a concatenation of selected
character fields. The joined field is first, followed by fields in the
order they occurred in the input files.
Our join(1) utility previously handled this correctly for lines with a match
in the other file. But it failed to order output fields correctly for
unmatched lines, printed in -a and -v modes.
A simple test case is:
$ touch a
$ echo "2 1" > b
$ join -v2 -2 2 a b
1 2
eadler [Sun, 18 Mar 2018 22:24:29 +0000 (22:24 +0000)]
MFC r319274:
- Add a simple example to uname(1) manual page to show how the hardware
platform (returned by -m) can be different from the machine's processor
architecture (-p)
- Document that make(1) sets universal MACHINE and MACHINE_ARCH variables
based on these values
indent(1): avoid calling write(2) with a negative second parameter.
indent(1): Avoid out of bound access of array codebuf.
indent(1): Avoid potential use-after-free.
indent(1): Fix breakage caused by single comment following "else".
indent(1) simply wasn't taught that "else" may be followed by a comment
without any opening brace anywhere on the line, so it was very confused
in such cases.
indent(1): fix struct termination detection.
indent(1): fix struct termination detection.
indent(1): Removed whitespace shouldn't be considered in column calculations.
indent(1): Support "f" and "F" floating constant suffixes.
indent(1): Use NULL instead of zero for pointers.
indent(1): Attempt to preserve some consistent style.
indent(1): Yet more style issues.
indent(1): Consistently indent declarations.
indent(1): Bail out if there's no more space on the parser stack.
indent(1): Remove dead code relating to unix-style comments.
indent(1): Simplify pr_comment().
indent(1): Fix wrapping of some lines in comments.
indent(1): Untangle the connection between pr_comment.c and io.c.
indent(1): Don't newline on cpp lines like #endif unless -bacc is on.
indent(1): replace function call to bzero with memset.
indent(1): Rearrange option parsing code to squelch clang's static analyzer.
indent(1): Use a dash in the license headers.
indent(1): accept offsetof(3) as a keyword.
indent(1): add some comments to quiet down Coverity.
indent(1): remove dead assignments.
indent(1): have the memset invocation somewhat more canonical.
indent(1): Capsicumify
indent(1): minor off-by-one error.
indent(1): fix regression introduced in r303596.
indent(1): Avoid out of bound access of array in_buffer
indent(1): Don't ignore newlines after comments that follow braces.
indent(1): Don't unnecessarily add a blank before a comment ends.
indent(1): Do not define opchar unless it will be used.
indent(1): Optimize parser stack usage.
indent(1): Remove an extra newline added in a previous commit.
indent(1): Avoid out-of-bound accesses of arrays.
indent(1): Avoid out-of-bound accesses of array ps.p_stack.
indent(1): Avoid out of bounds access of array ps.paren_indents
indent(1): add a piece missed in r311138.
indent(1): Support binary integer literals.
indent(1): don't produce unnecessary blank lines.
indent(1): rename the profile file.
indent(1): better alignment of comments on code.
imp [Sun, 18 Mar 2018 19:05:06 +0000 (19:05 +0000)]
Direct commit to stable
Remove libstand32 here. pc98 is a 32-bit platform, so it shouldn't compile
the extra 32-bit copy of libsa. The copy built in libstand is already 32-bit.
Add a comment saying we need an empty Makefile.pc98 since otherwise it would
pull in Makefile.i386 and there is no EFI on pc98, and the machines are too
old to have ZFS or GELI be a viable option (and besides, those don't compile).
Note: We also need r331140 to be MFC'd for pc98 build to work in all cases.
kp [Sun, 18 Mar 2018 11:25:39 +0000 (11:25 +0000)]
MFC r329950:
pf: Cope with overly large net.pf.states_hashsize
If the user configures a states_hashsize or source_nodes_hashsize value we may
not have enough memory to allocate this. This used to lock up pf, because these
allocations used M_WAITOK.
Cope with this by attempting the allocation with M_NOWAIT and falling back to
the default sizes (with M_WAITOK) if these fail.
PR: 209475
Submitted by: Fehmi Noyan Isi <fnoyanisi AT yahoo.com>
eadler [Sun, 18 Mar 2018 02:59:14 +0000 (02:59 +0000)]
MFC r330843:
efirtc: Pass a dummy tmcap pointer to efi_get_time_locked
As noted in the comment, UEFI spec claims the capabilities pointer is
optional, but some implementations will choke and attempt to dereference it
without checking. This specific problem was found on a Lenovo Thinkpad X220
that would panic in efirtc_identify.
dim [Sat, 17 Mar 2018 19:04:36 +0000 (19:04 +0000)]
Revert r330471 (MFC of r311861), since it results in compile errors
like:
sys/net80211/ieee80211_tdma.c:179: error: 'IEEE80211_MODE_VHT_2GHZ' undeclared (first use in this function)
sys/net80211/ieee80211_tdma.c:180: error: 'IEEE80211_MODE_VHT_5GHZ' undeclared (first use in this function)
The IEEE80211_MODE_VHT_2GHZ and IEEE80211_MODE_VHT_5GHZ enum values are
defined in r310147, but that commit cannot be MFCd as-is, because it
likely breaks ABI.
dim [Sat, 17 Mar 2018 18:55:46 +0000 (18:55 +0000)]
Repair obvious mismerge in r330897, resulting in misleading gcc error
messages like "sys/net/if_fddisubr.c:1: error: expected '=', ',', ';',
'asm' or '__attribute__' before '-' token".
eadler [Sat, 17 Mar 2018 01:27:54 +0000 (01:27 +0000)]
MFC r326959:
arc lint: ignore /tests/ in chmod
shell scripts in scripts don't need
to be chmod +x to work. In fact most are not.
Of the tests I found from a simple search:
65 are chmod +x
84 are chmod -x
patch(1): replace strnlen() with a simpler strlen().
patch(1): add support for git generated diffs.
patch: rejname[] is also -r option buffer, and should be PATH_MAX.
patch: further cleanup to git-style diffs.
The Genesys chip is failing when issueing READ_CAP(16) command.
Force a quirk to disable it and use READ_CAP(10) instead.
Also, depending on used firmware, GL3224 can be recognized
either as 'storage device' or 'mass storage class' -
enable both variants in scsi_quirk_table.
marius [Thu, 15 Mar 2018 23:02:49 +0000 (23:02 +0000)]
MFC: r327929
Use the correct revision specifier (EXT_CSD revision rather than
system specification version) for deciding whether the EXT_CSD
register includes the EXT_CSD_GEN_CMD6_TIME field.
marius [Thu, 15 Mar 2018 23:01:00 +0000 (23:01 +0000)]
MFC: r327355, r327926
- Don't allow userland to switch partitions; it's next to impossible
to recover from that, especially when something goes wrong.
- When userland changes EXT_CSD, update the kernel copy before using
relevant EXT_CSD bits in mmcsd_switch_part().
marius [Thu, 15 Mar 2018 22:58:31 +0000 (22:58 +0000)]
MFC: r327339, r327924
- There is no need to keep the tuning error and re-tuning interrupts
enabled (though, no interrupt generation enabled for them) all the
time as soon as (re-)tuning is supported; only enable them and let
them generate interrupts when actually using (re-)tuning.
- Also disable all interrupts except SDHCI_INT_DATA_AVAIL ones while
executing tuning and not just their signaling.
- Set the tuning error and re-tuning interrupt enable bits based on
the SDHCI_TUNING_ENABLED rather than the SDHCI_TUNING_SUPPORTED flag,
i. e. only when (re-)tuning is actually used. Currently, this change
makes no net difference, though.
kevans [Thu, 15 Mar 2018 19:31:39 +0000 (19:31 +0000)]
MFC r322278,324177: EFIRT Improvements
r322278 (imp): Fail to open efirt device when no EFI on system.
libefivar expects opening /dev/efi to indicate if the we can make efi
runtime calls. With a null routine, it was always succeeding leading
efi_variables_supported() to return the wrong value. Only succeed if
we have an efi_runtime table. Also, while I'm hear, out of an
abundance of caution, add a likely redundant check to make sure
efi_systbl is not NULL before dereferencing it. I know it can't be
NULL if efi_cfgtbl is non-NULL, but the compiler doesn't.
r324177 (andrew):
To prepare for adding EFI runtime services support on arm64 move the
machine independent parts of the existing code to a new file that can be
shared between amd64 and arm64.
Care has been taken to ensure that the MFC of r324177 did not clobber
cherry-picked MFC's.
emaste [Thu, 15 Mar 2018 12:56:22 +0000 (12:56 +0000)]
MFC r329370, r330239: Rationalize license text on Linuxolator files
Many licenses on Linuxolator files contained small variations from the
standard FreeBSD license text. To avoid license proliferation switch to
the standard 2-clause FreeBSD license for those files where I have
permission from each of the listed copyright holders.
Approved by: dchagin, kan, marcel, rdivacky, sos
Sponsored by: The FreeBSD Foundation
eadler [Thu, 15 Mar 2018 09:43:38 +0000 (09:43 +0000)]
MFC r305139:
df(1): Allow duplicate -l flags gracefully
Rather than producing a misleading error message when duplicate -l flags are
provided to df(1), simply ignore extra flags and proceed as if only one was
specified. This seems most reasonable given the usage for -l:
-l Only display information about locally-mounted file systems.
- add ñ, ', and _
- remove lint support
- add missing header for ioctl
- fix two typod
- Use `-r` for "reverse" mode and to match DragonFlyBSD.
- Move defines around to clear up logic
- use `errx` instead of `fprintf` and `exit`
- Use copyright comment header
- Make it easier to compile on !FreeBSD
- Diff reduction against DragonFlyBSD
- bump Dd
- use 'r' instead of 'D' from the original submission
eadler [Thu, 15 Mar 2018 09:04:23 +0000 (09:04 +0000)]
MFC r303036,r303038,r306822,r307923,:
makefs: reorder 'usage' alphabetically
From NetBSD, Mon Aug 15 14:45:01 2011 +0000 (wiz)
Re-order `usage' alphabetically;
rename option arguments in the manpage's `SYNOPSIS' section to
match those from `usage' (not the other way around; the `usage'-line
(and other parts of makefs.c) contain the correct names);
minor punctuation improvements.
makefs: sync NetBSD IDs with upstream for changes that we already have
May 22 21:51:39 2011 +0000 (christos):
From Nathan Whitehorn (nwhitehorn at freebsd dot org):
Add code to generate bootable ISOs on Powermac and CHRP systems.
Synthesize some partition maps (APM and MBR, respectively) pointing
to (a) the whole disk, and (b) relevant El Torito boot images that
have been added by other code. These partition maps are a little
bit funny looking, but they seem to work. FreeBSD has been using
this successfully in their release generation on powerpc, as well
as generating all non-SPARC install media. SPARC support could
probably be added as an extension of this patch.
makefs.8 1.33
Tue Aug 23 17:09:11 2011 +0000 (christos):
PR/45285: Martin Matuska: makefs does not properly convert ISO level 1 and 2
filenames (buffer overflow)
makefs does not properly verify the maximum filename length in the
special "." case for both ISO level 1 and ISO level 2 filename
conversion. This creates broken images or causes a buffer overflow
(ISO level 2).
ISO level 1:
If a filename contains only dots or up to 8 characters followed by
dots the 8+3 limit check doesn't work.
ISO level 2:
If a filename contains a dot in the first 30 characters and a dot
on the 30th character, the length limit check doesn't work and the
buffer is overflowed.
Allow building makefs(8) from another Makefile (such as one in
a seperate directory hierarchy used to build tools). This boils
down to replacing the use of ${.CURDIR} with either ${SRCDIR}
or ${SRCTOP}. SRCDIR is defined as the directory in which the
Makefile lives that bmake(1) is currently reading. Use SRCTOP
when reaching outside of makefs's directory.
eadler [Thu, 15 Mar 2018 02:20:06 +0000 (02:20 +0000)]
MFC r302525,r302526:
Do allow auditing of read(2) and write(2) system calls, by assigning
those system calls audit event identifiers AUE_READ and AUE_WRITE.
While auditing file-descriptor I/O is not required by the Common
Criteria, in practice this proves useful for both live and forensic
analysis.
NB: freebsd32 already assigns AUE_READ and AUE_WRITE to read(2) and
write(2).
In process-descriptor close(2) and fstat(2), audit target process
information. pgkill(2) already audits target process ID.
marius [Wed, 14 Mar 2018 23:59:50 +0000 (23:59 +0000)]
MFC: 327314
With the advent of interrupt remapping, Intel has repurposed bit 11
(now: Interrupt_Index[15]) and assigned the previously reserved bits
55:48 (Interrupt_Index[14:0] goes into 63:49 while Destination Field
used 63:56 and bit 48 now is Interrupt_Format) in the IO redirection
tables (see the VT-d specification, "5.1.5.1 I/OxAPIC Programming").
Thus, when not using interrupt remapping, ensure that all previously
reserved bits in the high part of the RTEs are zero instead of doing
a read-modify-write for their Destination Field bits only.
Otherwise, on machines based on Apollo Lake and its derivatives such
as Denverton, typically some of the previously preserved bits remain
set after boot when not employing interrupt remapping. The result is
that INTx interrupts are not getting delivered.
Note: With an AMD IOMMU, interrupt remapping apparently bypasses the
IO APIC altogether.
eadler [Wed, 14 Mar 2018 08:45:19 +0000 (08:45 +0000)]
MFC r302509:
Fix heap overflow in bhnd(4) SPROM parsing.
The bus_region_* APIs accept the number of data items to be read, while
the code was passing the total number of bytes, resulting in an overflow
of the SPROM parser's buffer.
Fix removal of the keyboard cursor image in text mode, especially
in the vga renderer. Removal used stale attributes and didn't try to
merge with the current attribute for cut marking, so special rendering
of cut marking was lost in many cases. The gfb renderer is too broken
to support special rendering of cut marking at all, so this change is
supposed to be just a style fix for it. Remove all traces of the
saveunder method which was used to implement this bug.
Fix drawing of the cursor image in text mode, only in the vga
renderer. This used a stale attribute from the frame buffer instead
of from the saveunder, but did merge with the current attribute for
cut marking so it caused less obvious bugs (subtle misrendering for
the character under the cursor).
The saveunder method may be good in simpler drivers, but in syscons
the 'under' is already saved in a better way in the vtb. Just redraw
it from there, with visible complications for cut marking and
invisible complications for mouse cursors. Almost all drawing
requests are passed a flag 'flip' which currently means to flip to
reverse video for characters in the cut marking region, but should
mean that the the characters are in the cut marking regions so should
be rendered specially, preferably using something better than reverse
video. The gfb renderer always ignores this flag. The vga renderer
ignored it for removal of the text cursor -- the saveunder gave the
stale rendering at the time the cursor was drawn. Mouse cursors need
even more complicated methods. They are handled by drawing them last
and removing them first. Removing them usually redraws many other
characters with the correct cut marking (but transiently loses the
keyboard cursor, which is redrawn soon). This tended to hide the
saveunder bug for forward motions of the keyboard cursor. But slow
backward motions of the keyboard cursor always lost the cut marking,
and fast backwards motions lost in for about 4 in every 5 characters,
depending on races with the scrn_update() timeout handler. This is
because the forward motions are usually into the region redrawn for
the mouse cursor, while backwards motions rarely are.
Text cursor drawing in the vga renderer used also used a
possibly-stale copy of the character and its attribute. The vga
render has the "optimization" of sometimes reading characters from the
screen instead of from the vtb (this was not so good even in 1990 when
main memory was only a few times faster than video RAM). Due to care
in update orders, the character is never stale, but its attribute
might be (just the cut marking part, again due to care in order).
gfb doesn't have the scp->scr pointer used for the "optimization", and
vga only uses this pointer for text mode. So most cases have to
refresh from the vtb, and we can be sure that the ordering of vtb
updates and drawing is as required for this to work.
Quick fix for removal of the mouse cursor in vga direct graphics modes
(that is, in all supported 8, 15, 16 and 24-color modes). Moving the
mouse cursor while holding down a button (giving cut marking) left a
trail of garbage from misremoved mouse cursors (usually colored
rectangles and not cursor shapes). Cases with a button not held down
worked better and may even have worked.
No renderer support for removing (software) mouse cursors is needed
(and many renderers don't have any), since sc_remove_mouse_image()
marks for update the region containing the image and usually much
more. The mouse cursor can be (partially) over as many as 4 character
cells, and removing it in only the 1-4 cells occupied by it would be
best for efficiency and for avoiding flicker. However,
sc_remove_mouse_image() can only mark a single linear region and
usually marks a full row of cells and 1 more to be sure to cover the
4 cells. It always does this, so using the special rendering method
just wastes even more time and gives even more flicker. The special
methods will be removed soon.
The general method always works. vga_pxlmouse_direct() appeared to
defer to it by returning immediately if !on. However,
vga_pxlmouse_direct() actually did foot-shooting using a disguised
saveunder method. Normal order near a mouse move is:
(1) remove the mouse cursor in the renderer (optional)
(2) remove the mouse cursor again and refresh the screen over the
mouse cursor and much more from the vtb. When the mouse has
actually moved and a button is down, many attributes in this
region are changed to be up to date with the new cut marking
(3) draw the keyboard cursor again if it was clobbered by the update
(4) draw the mouse cursor image in its new position.
The bug was to remove the mouse cursor again in step (4), before the
drawing it again in (4), using a saveunder that was valid in step (1)
at best. The quick fix is to use the saveunder in step (1) and not
in step (4). Using it in step (4) also used it before it was
initialized, initially and after mode and screen switches.
Special rendering methods for removing mouse cursors cannot be removed
like I hoped, since they are needed for removing parts over the border.
Continue fixing bugs in them.
In the vga planar mode renderer, remove removal of the part of the
image over the text window. This was hard-coded for nearly 8x16 fonts
and in practice didn't remove enough for 8x8 fonts. This used the
wrong attribute over cutmarked regions. The caller refreshes with the
correct attribute later, so the attribute bug only caused flicker.
The caller uses the same hard-coding, so the refreshes fix up all the
spots with the wrong attribute, but keep missing the missed spots.
This still gives trails of bits of cursors for cursor motions in the
affected configurations (mainly depth 4 modes with 8x8) fonts. 8x14
fonts barely escape the problem since although the cursor is drawn
as 16x16, its active part is only 9x13 and the active part fits in
the hard-coded 2x2 character cell window for 8x14 fonts. 8x8 fonts
need a 2x3 window.
In the fb non-sparc64 renderer, the buggy image removal was buggier
and was already avoided by returning before it. Remove it completely
and fix nearby style bugs. It was essentially the same as for the vga
planar mode renderer (obfuscated by swapping x and y). This was buggier
since fb should handle more types of hardware so the hard-coding is
wronger.
The remaining fb image removal is also buggier. It never supported
software cursors drawn into the border, and the hardware cursor is
probably broken by other bugs to be fixed soon.
Fix clobbering of the default attribute and the screen position in
scteken_init(). Move the internals of scteken_sync() into a local
function to help do this.
scteken_init() reset or adjusted the default attribute and screen
position at least 3 and 5 times, respectively. Warm init shouldn't
do any more than reset the "input" state.
(scterm-sc.c (which still works after minor editing), only resets
the escape state and the saved cursor position, and then does a
nearly-null sync of the current color.)
This mainly broke mode changes, and was most noticeable when the
background color is not teken's default (usually black). Then the
screen gets cleared in the wrong color. vidcontrol restores the
default normal attribute and tries to restore the default reverse
attribute. vidcontrol doesn't clear the screen again after restoring
the attribute(s), and it is too late to do it there without flicker.
Now the default normal attribute is restored before the change affects
the rendering.
When the foreground color is not teken's default, clearing with the
wrong attributes gave strange cursor colors for some cursor types.
The default reverse attribute is not restored since it is unsupported.
2/3 of the clobbering was from 2 resetting window resizing calls. The
second one is needed to restore the size, but must not reset. Window
resizing also sanitizes the cursor position, and after the main reset
resets the window size, the cursor row would often be adjusted from
24 to 23 if it were not already reset to 0. scteken_sync() is good
for restoring the window size and the cursor position in the correct
order, but was unusable at init time since scp->ts is not always
initialized then. Adjust to use its internals.
I didn't notice any problems from the cursor reset. The cursor should
be reset, and a previous fix was to reset it consistently a little
later.
Doing nothing for warm init works almost as well, if not better. It
is not very useful to reset the escape state for mode changes, since
the reset is especially likely to be null then. The escape state is
most likely to be non-initial and corrupted by its most normal uses
-- sloppy non-atomic output where a context switch or just mixing
stdout with stderr splits up escape sequences.
Fix removal of mouse image by the vga planar renderer in the right border
in unusual cases. Optimize and significantly clean up removal in this
renderer. Optimize removal in the vga direct renderer.
Removal only needs to be done in the border (the part with pixels) in
both cases. The planar renderer used the condition scp->xoff > 0 to
test whether a right border exists. This actually tests for a left
border, and when the total horizontal border is 8 pixels, rounding gives
only a right border. This was the unusual broken case. An example
is easy to configure using something like "vidcontrol -f 8x16 iso-8x16
-g 79x25 MODE_27".
Optimize the planar case a little by only removing 9x13 active pixels
out of 16x16. Optimize it a lot by not doing anything if there is no
overlap with the border. Don't unroll the main loop or hard-code so
many assumptions about font sizes in it. On my Haswell system, graphics
memory and i/o accesses takes about 520 cycles each so optimizations from
unrolling are in the noise.
Optimize the direct case to not do anything if there is no overlap with
the border. Do a sanity check on the saveunder's coordinates. This
requires a previous change to pass non-rounded coordinates.
Improve drawing of the vga planar mode mouse image a little. Unobfuscate
the method a lot.
Reduce the AND mask to the complement of the cursor's frame, so that area
inside the frame is not drawn first in black and then in lightwhite. The
AND-OR method is only directly suitable for the text mouse image, since
it doesn't go to the hardware there. Planar mode Mouse cursor drawing
takes 10-20 usec on my Haswell system (approx. 100 graphics accesses
at 130 nsec each), so the transient was not visible.
The method used the fancy read mode 1 and its color compare and color
don't care registers with value 0 in them so that all colors matched.
All that this did was make byte reads of frame buffer memory return 0xff,
so that the x86 case could obfuscate read+write as "and". The read must
be done for its side effect on the graphics controller but is not used,
except it must return 0xff to avoid affecting the write when the write
is obfuscated as a read-modify-write "and". Perhaps that was a good
optimization for 8088 CPUs where each extra instruction byte took as
long as a byte memory access.
Just use read+write after removing the fancy read mode. Remove x86
ifdefs that did the "and". After removing the "and" in the non-x86
part of the ifdefs, fix 4 of 6 cases where the shift was wrong.
Further unobfuscate the method of drawing the mouse cursor in vga planar
mode.
Don't manually unroll the 2 inner loops. On Haswell, doing so gave a
speedup of about 0.5% (about 4 cycles per iteration out of 1400), but
hard-coded a limit of width 9 and made better better optimizations
harder to see. gcc-4.2.1 -O does the unrolling anyway, unless tricked
with a volatile hack. gcc's unrolling is not very good and gives a
a speedup of about half as much (about 2 cycles per iteration). (All
timing on i386.)
Manual unrolling was only feasible because the inner loop only iterates
once or twice. Usually twice, but a dynamic check is needed to decide,
and was not moved from the second-innermost loop manually or by gcc.
This commit basically adds another dynamic check in the inner loop.
Cursor widths of 10-17 require 3 iterations in the inner loop and this
is not so easy to unroll -- even gcc stops at 2.
Optimize drawing of the mouse cursor in vga planar mode almost as
much as possible, by avoiding null ANDs and ORs to the frame buffer.
Mouse cursors are fairly sparse, especially for their frame. Pixels
are written in groups of 8 in planar mode and the per-group sparseness
is not as large, but it still averages about 40% with the current
9x13 mouse cursor. The average drawing time is reduced by about this
amount (from 22 usec constant to 12.5 usec average on Haswell).
This optimization is relatively larger with larger cursors. Width 10
requires 6 frame buffer accesses per line instead of 4 if not done
sparsely, but rarely more than 4 if done sparsely.
Adjust shifting so that cursor widths up to 17 (was 9) work in vga planar
mode.
Direct mode always supported widths up to 32, except for its hard-coded
16s matching the pixmap size. Text mode is still limited to 9 its 2x2
character cell method and missing adjustments for the gap between
characters, if any.
Cursor heights can be almost anything in graphics modes.
eadler [Wed, 14 Mar 2018 07:58:11 +0000 (07:58 +0000)]
MFC r316136:
The switch to kernel terminal context needs to update more than the cursor
position. Especially the screen size, and potentially everything except
the input state and attributes. Do this by changing the cursor position
setting method to a general syncing method.
Use proper constructors instead of copying to create kernel terminal
contexts. We really want clones and not new instances, but there is
no method for cloning and there is nothing in the active instance that
needs to be cloned exactly.
Add proper destructors for kernel terminal contexts. I doubt that the
destructor code has every been reached, but if it was then it leaked the
memory of the clones.
Remove freeing of statically allocated memory for the non-kernel terminal
context for the same terminal as the kernel. This is in the nearly
unreachable code. This used to not happen because delicate context
swapping made the user context use the dynamic memory and kernel
context the static memory. I didn't restore this swapping since it
would have been unnatural to have all kernel contexts except 1 dynamic.
The constructor for terminal context has bad layering for reasons
related to the bug. It has to return static memory early before
malloc() works. Callers also can't allocate memory until after the
first constructor selects an emulator and tells upper layers the size
of its context. After that, the cloning hack required the cloning
code to allocate the memory, but for all other constructors it would
be better for the terminal layer to allocate and deallocate the
memory in all cases.
Zero the memory when allocating terminal contexts dynamically.
eadler [Wed, 14 Mar 2018 07:55:29 +0000 (07:55 +0000)]
MFC r315984:
Restore switching to a separate kernel terminal "input" state and extend
it to a separate state for each CPU.
Terminal "input" is user or kernel output. Its state includes the current
parser state for escape sequences and multi-byte characters, and some
results of previous parsing (mainly attributes), and in teken the cursor
position, but not completed output. This state must be switched for kernel
output since the kernel can preempt anything, including itself, and this
must not affect the preempted state more than necessary. Since vty0 is
shared, it is necessary to affect the frame buffer and cursor position and
history, but escape sequences must not be affected and attributes for
further output must not be affected.
This used to work. The syscons terminal state contained mainly the parser
state for escape sequences and attributes, but not the cursor position,
and was switched. This was first broken by SMP and/or preemptive kernels.
Then there should really be a separate state for each thread, and one more
for ddb, or locking to prevent preemption. Serialization of printf() helps.
But it is arcane that full syscons escape sequences mostly work in kernel
printf(), and I have never seen them used except by me to test this fix.
They worked perfectly except for the races, since "input" from the kernel
was not special in any way.
This was broken to use teken. The general switch was removed, and the
kernel normal attribute was switched specially. The kernel reverse
attribute (config option SC_CONS_REVERSE_ATTR) became unused, and is
still unusable because teken doesn't support default reverse attributes
(it used to only be used via the ANSI escape sequence to set reverse
video).
The only new difficulty for using teken seems to be that the cursor
position is in the "input" state, so it must be updated in the active
input state for each half of the switch. Do this to complete the
restoration.
The per-CPU state is mainly to make per-CPU coloring work cleanly, at
a cost of some space. Each CPU gets its own full set of attribute
(not just the current attribute) maintained in the usual way. This
also reduces races from unserialized printf()s. However, this gives
races for serialized printf()s that otherwise have none. Nothing
prevents the CPU doing the a printf() changing in the middle of an
escape sequence.
eadler [Wed, 14 Mar 2018 07:47:26 +0000 (07:47 +0000)]
MFC r315418,r315480,r316019:
Add teken_256to16() to convert xterm-256 256-color codes to xterm 16-color
codes. This will be used to fix bright colors.
Improve teken_256to8(). Use a lookup table instead of calculations. The
calculations were inaccurate since they used indexes into the xterm-256
6x6x6 color map instead of actual xterm colors. Also, change the threshold
for converting to a primary color: require the primary's component to be
2 or more higher instead of just higher. This affects about 1/5 of the
table entries and gives uniformly distributed colors in the 6x6x6 submap
except for greys (35 entries each for red, green, blue, cyan, brown and
magenta, instead of approx. only 15 each for the mixed colors). Even
more mixed colors would be better for matching colors, but uniform
distribution is best for preserving contrast.
For teken_256to16(), bright colors are just the ones with luminosity >=
60%. These are actually light colors (more white instead of more
saturation), while xterm bright colors except for white itself are
actually saturated with no white, so have luminosity only 50%.
These functions are layering violations. teken cannot do correct
conversions since it shouldn't know the color maps of anything except
xterm. Translating through xterm-16 colors loses information. This
gives bugs like xterm-256 near-brown -> xterm-16 red -> VGA red.
Fix bright colors for syscons, and make them work for the first time
for vt. Restore syscons' rendering of background (bg) brightness as
foreground (fg) blinking and vice versa, and add rendering of blinking
as background brightness to vt.
Bright/saturated is conflated with light/white in the implementation
and in this description.
Bright colors were broken in all cases, but appeared to work in the
only case shown by "vidcontrol show". A boldness hack was applied
only in 1 layering-violation place (for some syscons sequences) where
it made some cases seem to work but was undone by clearing bold using
ANSI sequences, and more seriously was not undone when setting
ANSI/xterm dark colors so left them bright. Move this hack to drivers.
The boldness hack is only for fg brightness. Restore/add a similar hack
for bg brightness rendered as fg blinking and vice versa. This works
even better for vt, since vt changes the default text mode to give the
more useful bg brightness instead of fg blinking.
The brightness bit in colors was unnecessarily removed by the boldness
hack. In other cases, it was lost later by teken_256to8(). Use
teken_256to16() to not lose it. teken_256to8() was intended to be
used for bg colors to allow finer or bg-specific control for the more
difficult reduction to 8; however, since 16 bg colors actually work
on VGA except in syscons text mode and the conversion isn't subtle
enough to significantly in that mode, teken_256to8() is not used now.
There are still bugs, especially in vidcontrol, if bright/blinking
background colors are set.
Restore XOR logic for bold/bright fg in syscons (don't change OR
logic for vt). Remove broken ifdef on FG_UNDERLINE and its wrong
or missing bit and restore the correct hard-coded bit. FG_UNDERLINE
is only for mono mode which is not really supported.
Restore XOR logic for blinking/bright bg in syscons (in vt, add
OR logic and render as bright bg). Remove related broken ifdef
on BG_BLINKING and its missing bit and restore the correct
hard-coded bit. The same bit means blinking or bright bg depending
on the mode, and we want to ignore the difference everywhere.
Simplify conversions of attributes in syscons. Don't pretend to
support bold fonts. Don't support unusual encodings of brightness.
It is as good as possible to map 16 VGA colors to 16 xterm-16
colors. E.g., VGA brown -> xterm-16 Olive will be converted back
to VGA brown, so we don't need to convert to xterm-256 Brown. Teken
cons25 compatibility code already does the same, and duplicates some
small tables. This is mostly for the sc -> te direction. The other
direction uses teken_256to16() which is too generic.
Oops, my fix for bright colors broke bright black some more (in cases
that used to work via the bold hack).
Fix the table entry for bright black. Fix spelling of plain black in
nearby table entries (use the macro for black everywhere everywhere).
Fix the currently-unused non-bright color table to not have bright
colors in entries 9-15.
Improve nearby comments. Start converting to the xterm terminology
and default rendering of "bright" instead of "light" for bright
colors.
Syscons wasn't affected by the bug since I optimized it a little by
converting colors 0-15 directly. This also fixes the layering of
the conversion for these colors.
Apply the same optimization to vt (actually the layer above it). This
also moves the conversion 1 closer to the correct layer for colors
0-15.
The optimization of just avoiding 2 calls to a trivial function is worth
about 10% for simple output to the virtual buffer with occasional
rendering. The optimization is so large because the 2 calls are done
on every character, so although there are too many other calls and
other instructions per character, there are only about 10 times as
many. Old versions of syscons were about 10 times faster for simple
output, by using a fast path with about 12 instructions per character.
Rendering to even slow hardware takes relatively little time provided
it is rarely actually done.
eadler [Wed, 14 Mar 2018 07:39:28 +0000 (07:39 +0000)]
MFC r315003,r315065,r315066:
Rename scteken_revattr() to scteken_sc_to_te_attr(). scteken_revattr()
looked like it might handle reverse attributes, but it actually handles
conversion of attributes in the direction indicated by the new name.
Reverse attributes are just broken.
Rename scteken_attr() to scteken_te_to_sc_attr(). scteken_attr() looked
like it might give teken attributes, but it actually gives sc attributes.
Change scteken_te_to_sc_attr() to return int instead of unsigned int.
u_char would be enough, and it promotes to int, and syscons uses int
or u_short for its attributes everywhere else (u_short holds a shifted
form and it promotes to int too).
Add a scteken_set_cursor() (sc to teken) method and use it to fix
some cases of initialization and resetting of the teken cursor position.
(This bad name is consistent with others, but it is too easy to confuse
with scteken_cursor() which goes in the opposite direction.)
The following cases were broken:
- for booting without a syscons console, the teken and sc positions for
ttyv0 were (0, 0), but are supposed to be somewhere in the middle of
the screen (after carefully preserved BIOS and loader messages) (at
least if there is no mode switch that loses the messages).
- after mode switches, the screen is cleared and the cursor is supposed to
be moved to (0, 0), but it was only moved there for sc.
The following case was hacked to work:
- for booting with a syscons console, it was arranged that scteken_init()
for the console could see a nonzero cursor position and adjust, although
this broke the sc seeing it in the non-console case above.
Fix the attribute for scteken_clear() (change it back from the user
user default normal attribute to the current attribute).
This change only fixes a logic error. scterm_clear() used to be
used for terminal reset, but teken uses a general fill function for
that, leaving scterm_clear() only used for initialization and mode
change, when using the user default attribute is correct. It is not
really a terminal function, but needs to sync its changes with the
terminal layer. Syncing of the attribute is currently broken for
terminal reset, but works for initialization and mode change.
eadler [Wed, 14 Mar 2018 07:32:48 +0000 (07:32 +0000)]
MFC r315000:
Start fixing some bugs in attribute handling.
This change just does cleanups missed in r56043 17 years ago. The
default attributes were still stored in structs for the purpose of
changing them and passing around pointers to the defaults, but r56043
added another layer that made the defaults invariant and only used for
initialization and reset. Just use the defaults directly. This was
already done for the kernel defaults. The defaults for reverse
attributes aren't actually used, but are ignored in layers that no
longer support them.
eadler [Wed, 14 Mar 2018 07:30:58 +0000 (07:30 +0000)]
MFC r314641,r314646,r314997,r315390:
Colorize syscons kernel console output according to a table indexed
by the CPU number.
This was originally for debugging near-deadlock conditions where
multiple CPUs either deadlock or scramble each other's output trying
to report the problem, but I found it interesting and sometimes
useful for ordinary kernel messages. Ordinary kernel messages
shouldn't be interleaved, but if they are then the colorization
makes them readable even if the interleaving is for every character
(provided the CPU printing each message doesn't change).
The default colors are 8-15 starting at 15 (bright white on black)
for CPU 0 and repeating every 8 CPUs. This works best with 8 CPUs.
Non-bright colors and nonzero background colors need special
configuration to avoid unreadable and ugly combinations so are not
configured by default. The next bright color after 15 is 8 (bright
black = dark gray) is not very readable but is the only other color
used with 2 CPUs. After that the next bright color is 9 (bright
blue) which is not much brighter than bright black, but is used with
3+ CPUs. Other bright colors are brighter.
Colorization is configured by default so that it gets tested. It can
only be turned off by configuring SC_KERNEL_CONS_ATTR to anything other
than FG_WHITE. After booting, all colors can be changed using the
syscons.kattr sysctl. This is a SYSCTL_OPAQUE, and no utility is
provided to change it (sysctl only displays it).
The default colors work in all VGA modes that I could test. In 2-color
graphics modes, all 8 bright colors are displayed as bright white, so
the colorization has no effect, but anything with a nonzero background
gives white on white unless the foreground is zero. I don't have an
mono or VGA grayscale hardware to test on. Support for mono mode seems
to have never worked right in syscons (I think bright white gives white
underline with either bold or bright), but VGA grayscale should work
better than 2-color graphics.
Implement ec_putc() (emergency kernel [syscons] console putc()) and use
it in emergency in sc_cnputc().
Locking fixes in sc_cnputc() previously turned off normal output in
near-deadlock conditions and added deferred output which might never
be completed. Emergency output goes to the frame buffer using
sufficiently atomic non-blocking writes if the console is in text
mode (in graphics mode, nothing is done, modulo races setting the
graphics mode bit). Screen updates overwrite the emergency output
if the emergency condition clears enough to reach them.
ec_putc() also works for "early" console output in normal x86 text
mode as soon as this mode is initialized (if ever). This uses a
hard-coded x86 frame buffer address before cninit() and a hopefully
MI address after cninit(). But non-x86 is more likely to not support
text mode, when ec_putc() will be null. ec_putc() has no dependencies
of syscons before cninit(), and only has them later to track syscons'
mode changes. This commit doesn't attach ec_putc() for early use.
To test emergency use, put a breakpoint in central syscons output code
like sc_puts() and do some user output. The system used to race or
deadlock in ddb output soon after entry to ddb. The locking fixes
deferred the output until after leaving ddb, so ddb was unusable and
you had to try typing c[ontinue] blindly until it exited, or better use
a serial console in parallel. Now the output goes to a window in the
middle 2/3 of the screen. Scrolling is circular and there is no cursor,
but otherwise ec_putc() provides full dumb terminal functionality and
very fast output that hides artificates from dumb overwrites.
eadler [Wed, 14 Mar 2018 07:16:29 +0000 (07:16 +0000)]
MFC r305121,r305231:
Add some locking to sc_cngetc().
Keyboard input needs Giant locking, and that is not possible to do
correctly here. Use mtx_trylock() and proceed unlocked as before if
we can't acquire Giant (non-recursively), except in kdb mode don't
even try to acquire Giant. Everything here is a hack, but it often
works. Even if mtx_trylock() succeeds, this might be a LOR.
Keyboard input also needs screen locking, to handle screen updates
and switches. Add this, using the same simplistic screen locking
as for sc_cnputc().
Giant must be acquired before the screen lock, and the screen lock
must be dropped when calling the keyboard driver (else it would get a
harmless LOR if it tries to acquire Giant). It was intended that sc
cn open/close hide the locking calls, and they do for i/o functions
functions except for this complication.
Non-console keyboard input is still only Giant-locked, with screen
locking in some called functions. This is correct for the keyboard
parts only.
When Giant cannot be acquired properly, atkbd and kbdmux tend to race
and work (they assume that the caller acquired Giant properly and don't
try to acquire it again or check that it has been acquired, and the
races rarely matter), while ukbd tends to deadlock or panic (since it
does the opposite, and has other usb threads to deadlock with).
The keyboard (Giant) locking here does very little, but the screen
locking completes screen locking for console mode except for not
detecting or handling deadlock.
The log message for the previous commit didn't mention the most the
important detail that sc_cngetc() now opens and closes the keyboard
on every call again. This was moved from sc_cngetc() to scn_cngrab/
ungrab() in r228644, but the change wasn't quite complete. After
fixes for nesting in kbdd_poll() in ukbd and kbdmux, these opens
and closes should have no significant effect if done while grabbed.
They fix unusual cases when cngetc() is called while not grabbed.
This commit is the main fix for screen locking in sc_cnputc():
detect deadlock or likely-deadlock and handle it by buffering the
output atomically and printing it later if the deadlock condition
clears (and sc_cnputc() is called).
The most common deadlock is when the screen lock is held by ourself.
Then it would be safe to acquire the lock recursively if the console
driver is calling printf() in a safe context, but we don't know when
that is. It is not safe to ignore the lock even in kdb or panic mode.
But ignore it in panic mode. The only other known case of deadlock
is when another thread holds the lock but is running on a stopped CPU.
Detect that case approximately by using trylock and retrying for 1000
usec. On a 4 GHz CPU, 100 usec is almost long enough -- screen switches
take slightly longer than that. Not retrying at all is good enough
except for stress tests, and planned future versions will extend the
timeout so that the stress tests work better.
To see the behaviour when deadlock is detected, single step through
sctty_outwakeup() (or sc_puts() to start with deadlock). Another
(serial) console is needed to the buffered-only output, but the
keyboard works in this context to continue or step out of the
deadlocked region. The buffer is not large enough to hold all the
output for this.
eadler [Wed, 14 Mar 2018 07:11:33 +0000 (07:11 +0000)]
MFC r305060:
Fix keyboard polling "on/off" to support recursion. vt depends on
this, and sc will soon depend on it again.
The on/off request is passed without modification to lower layers,
so the bug was smaller in this layer than in in lower layers (the
sequence on;on;off left polling off when it should be on, but the
sequence on;on;off;on;off... doesn't allow the interrupt handler
to eat the input after an "off" that should't turn off polled mode,
provided lower layers don't have the bug, since this layer is virtual.
The bug was small in lower layers too. Normally everything is Giant
locked for keyboards, and this locks out the interrupt handler in
on;on;off;on;off... sequences. However, PR 211884 says that fixing
this bug in ukbd in r303765 apparently causes the eating-by-interrupt
behaviour that the fix is to prevent.
eadler [Wed, 14 Mar 2018 07:08:46 +0000 (07:08 +0000)]
MFC r305059:
Start adding locking to sc_cngetc().
Restore an splx() lost in r228644. We aren't nearly ready to remove
spl's. They give hints about missing locking. This lost one was
misplaced. Dropping it early for convenience gave race windows for
accesses to the fkey buffer. Giant locking accidentally fixed this
for non-console cases.
Put the spl's around the whole function. Since there are many returns
that would need splx() just before them for a direct fix, split the
function into a wrapper that does the spl's and a "locked" function
that does the work.
Return earlier when no keyboard is attached to match the ordering in a
planned version. This breaks the dubious feature of returning keys
from the fkey buffer after the keyboard has gone away. Losing the keys
wouldn't matter, but we keep them too long now.
projects / FreeBSD / FreeBSD.git / log