pstef [Mon, 4 Jun 2018 21:05:56 +0000 (21:05 +0000)]
indent(1): add --version option
There exist multi-platform programs that check indent's version in order to
know what they can expect from it. GNU indent provides that via --version,
so implement the same option here.
markj [Mon, 4 Jun 2018 19:35:15 +0000 (19:35 +0000)]
Reimplement brk() and sbrk() to avoid the use of _end.
Previously, libc.so would initialize its notion of the break address
using _end, a special symbol emitted by the static linker following
the bss section. Compatibility issues between lld and ld.bfd could
cause the wrong definition of _end (libc.so's definition rather than
that of the executable) to be used, breaking the brk()/sbrk()
interface.
Avoid this problem and future interoperability issues by simply not
relying on _end. Instead, modify the break() system call to return
the kernel's view of the current break address, and have libc
initialize its state using an extra syscall upon the first use of the
interface. As a side effect, this appears to fix brk()/sbrk() usage
in executables run with rtld direct exec, since the kernel and libc.so
no longer maintain separate views of the process' break address.
cem [Mon, 4 Jun 2018 18:51:06 +0000 (18:51 +0000)]
Correctly handle the padding for IPv6-AH, as specified by RFC4302
The RFC specifies that under IPv6 the complete AH header must be 64 bit
aligned, and under IPv4, 32 bit aligned. Prior to this change, we (along
with other BSDs and MacOS) had violated this requirement.
This makes it possible to set up IPv6-AH between Linux and BSD, and also
probably between Windows and BSD.
PR: 222684
Reported and tested by: Jason Mader <jasonmader AT gmail.com>
Obtained from: NetBSD xform_ah.c 1.105
(b939fe2483972eb43d71bf990cfb7f26dece7839 NetBSD/src on GH)
by Maxime Villard
MFC after: 35.2731 hours
Relnotes: probably (breaks ipv6 compat with older FreeBSD/NetBSD/MacOS)
Sponsored by: Dell EMC Isilon
cem [Mon, 4 Jun 2018 18:47:14 +0000 (18:47 +0000)]
str(r)chr: Replace union abuse with __DECONST
Writing one union member and reading another is technically illegal C,
although we do it in many places in the tree. Use the __DECONST macro
instead, which is (technically) a valid C construct.
Trivial style(9) cleanups to touched lines while here.
alc [Mon, 4 Jun 2018 16:28:06 +0000 (16:28 +0000)]
Use a single, consistent approach to returning success versus failure in
vm_map_madvise(). Previously, vm_map_madvise() used a traditional Unix-
style "return (0);" to indicate success in the common case, but Mach-
style return values in the edge cases. Since KERN_SUCCESS equals zero,
the only problem with this inconsistency was stylistic. vm_map_madvise()
has exactly two callers in the entire source tree, and only one of them
cares about the return value. That caller, kern_madvise(), can be
simplified if vm_map_madvise() consistently uses Unix-style return
values.
Since vm_map_madvise() uses the variable modify_map as a Boolean, make it
one.
Eliminate a redundant error check from kern_madvise(). Add a comment
explaining where the check is performed.
Explicitly note that exec_release_args_kva() doesn't care about
vm_map_madvise()'s return value. Since MADV_FREE is passed as the
behavior, the return value will always be zero.
Fix build: ignore a GCC 7.2.0 warning which says that third argument of
memset(3) should contain the number of elements multiplied by the element
size.
jhibbits [Mon, 4 Jun 2018 14:42:13 +0000 (14:42 +0000)]
Set kernelname in bootconfig to the kernel file
Summary:
The kernel reads 'kernelname' to set the kern.bootfile sysctl. By setting this,
'make installkernel' will backup the running kernel as appropriate.
mmacy [Mon, 4 Jun 2018 02:05:48 +0000 (02:05 +0000)]
hwpmc: ABI fixes
- increase pmc cpuid field from 8 to 12 bits
- add cpuid version string to initialize entry in the log
so that filter can identify which counter index an
event name maps to
- GC unused config flags
- make fixed counter assignment more robust as well as the
changes needed to be properly identified for filter
mmacy [Mon, 4 Jun 2018 01:10:23 +0000 (01:10 +0000)]
hwpmc: support sampling both kernel and user stacks when interrupted in kernel
This adds the -U options to pmcstat which will attribute in-kernel samples
back to the user stack that invoked the system call. It is not the default,
because when looking at kernel profiles it is generally more desirable to
merge all instances of a given system call together.
Although heavily revised, this change is directly derived from D7350 by
Jonathan T. Looney.
eadler [Sun, 3 Jun 2018 23:40:54 +0000 (23:40 +0000)]
top(1): another pass of cleanup
- avoid the need to call a function to get size of known array. I'll
likely re-arrange some of the indirect in a later to avoid the magic
constants.
- use correct type
- add const
- replace caddr_t with void*. This corrects an alignment warning.
- remove duplicated include from immediately prior commit
Under base clang we're now down to:
- 3 warning in top.c, 1 warning in mahcine.c, 4 warning in display.c,
- 1 warning in utils.c
Tested with base clang, gcc7, gcc9, base gcc (mips)
eadler [Sun, 3 Jun 2018 22:42:54 +0000 (22:42 +0000)]
top(1): top warnings and cleanup
- Add const where helpful
- add missing 'static' for file-local functions
- use nitems where possible
- convert manual abort() to assert
- use strndup instead of homegrown version
pstef [Sun, 3 Jun 2018 21:40:38 +0000 (21:40 +0000)]
indent(1): new option -lpl
With -lpl, code surrounded by parentheses in continuation lines is lined up
even if it would extend past the right margin.
With -nlpl (the default), such a line that would extend past the right
margin is moved left to keep it within the margin, if that does not require
placing it to the left of the prevailing indentation level.
These switches have no effect if -nlp is selected.
pstef [Sun, 3 Jun 2018 20:59:59 +0000 (20:59 +0000)]
indent(1): new option -lpl (always line up to parenthesis)
With -lp, if a line has an opening paren which is not closed on that line,
then continuation lines will be lined up to start at the character position
just after the opening paren.
rmacklem [Sun, 3 Jun 2018 19:46:44 +0000 (19:46 +0000)]
Fix a gcc8 warning about a write only variable.
gcc8 warns that "verf" was set but not used. This was because the code
that uses it is disabled via a "#if 0".
This patch adds a "#if 0" to the variable's declaration and assignment
to get rid of the warning.
This way the code could be re-enabled without difficulty.
pstef [Sun, 3 Jun 2018 18:19:41 +0000 (18:19 +0000)]
indent(1): improve CHECK_SIZE_ macros
Rewrite the macros so that they take a parameter. Consumers use it to signal
how much room in the buffer they need; this lets them do that once when
required space is known instead of doing the check once every loop step.
Also take the parameter value into consideration when resizing the buffer;
the requested space may be larger than the constant 400 bytes that the
previous version used - now it's the sum of those two values.
On the consumer side, don't copy strings byte by byte - use memcpy().
Deduplicate code that copied base 2, base 8 and base 16 literals.
Don't advance the e_token pointer once the token has been copied into
s_token. This allows easy calculation of the token's length.
pstef [Sun, 3 Jun 2018 17:55:50 +0000 (17:55 +0000)]
indent(1): remove troff output support
The troff output in indent was invented at Sun and the online documentation
for some post-SunOS operating system includes this:
The usual way to get a troffed listing is with the command
indent -troff program.c | troff -mindent
The indent manual page in FreeBSD 1.0 already lacks that information and
troff -mindent complains about not being able to find the macro file.
It seems that the file did exist on SunOS and was supposed to be imported
into 4.3BSD together with the feature, but that has never happened.
Removal of troff output support simplifies a lot of indent's code.
pstef [Sun, 3 Jun 2018 16:27:40 +0000 (16:27 +0000)]
indent(1): remove is_procname.
It was a shorthand for checking if ps.procname is a non-empty string; the
same can be done with ps.procname[0] which avoids the need for updating
is_procname after every call to lexi().
pstef [Sun, 3 Jun 2018 15:28:55 +0000 (15:28 +0000)]
indent(1): improve handling of boxed comments indentation
The trick is to copy everything from the start of the line into the buffer
that stores newlines and comments until indent finds a brace or an else.
pr_comment() will use that information to calculate the original indentation
of the boxed comment.
This requires storing two pieces of information: the real start of the
buffer (sc_buf) and the start of the comment (save_com).
pstef [Sun, 3 Jun 2018 14:13:11 +0000 (14:13 +0000)]
indent(1): improve predictability of lexi()
lexi() reads the input stream and categorizes the next token. indent will
sometimes buffer up a sequence of tokens in order rearrange them. That is
needed for properly cuddling else or placing braces correctly according to
the chosen style (KNF vs Allman) when comments are around. The loop that
buffers tokens up uses lexi() to decide if it's time to stop buffering. Then
the temporary buffer is used to feed lexi() the same tokens again, this time
for normal processing.
The problem is that lexi() apart from recognizing the token, can change
a lot of information about the current state, for example ps.last_nl,
ps.keyword, buf_ptr. It also abandons leading whitespace, which is needed
mainly for comment-related considerations. So the call to lexi() while
tokens are buffered up and categorized can change the state before they're
read again for normal processing which may easily result in changing
interpretation of the current state and lead to incorrect output.
To work around the problems:
1) copy the whitespace into the save_com buffer so that it will be read
again when processed
2) trick lexi() into modifying a temporary copy of the parser state instead
of the original.
pstef [Sun, 3 Jun 2018 14:03:20 +0000 (14:03 +0000)]
indent(1): improve handling of comments and newlines between "if (...)" or
"while (...)" and "else" or "{"
* Don't flush newlines - there can be multiple of them and they can happen
before a token that isn't else or {. Instead, always store them in save_com.
* Don't dump the buffer's contents on newline assuming that there is only
one comment before else or {.
* Avoid producing surplus newlines, especially before else when -ce is on.
* When -bl is on, don't treat { as a comment (was implemented by falling
through "case lbrace:" to "case comment:").
This commit fixes the above, but exposes another bug and thus breaks several
other tests. Another commit will make them pass again.
eadler [Sun, 3 Jun 2018 05:07:39 +0000 (05:07 +0000)]
top(1): use greater warnings
One of the downsides of using numeric WARNS is that if we only have a
single type of issue we get no protection from other changes. For
example, we got no warning for missing variable declaration, due to
the issues with "const".
For this utility, explicitly list out the warnings which are failing.
They should still be fixed, so only reduce them to warning instead of
error.
Tested with: clang base (amd64, i386), gcc6, gcc7, gcc9, gcc base (mips)
jhibbits [Sun, 3 Jun 2018 03:53:11 +0000 (03:53 +0000)]
Revert r326083, it doesn't behave as expected.
Even though there do appear to be more artificial frames, with 12, stack
traces no longer list at all. Revert until a better, more stable value can
be determined.
mjg [Sat, 2 Jun 2018 22:20:09 +0000 (22:20 +0000)]
malloc: try to use builtins for zeroing at the callsite
Plenty of allocation sites pass M_ZERO and sizes which are small and known
at compilation time. Handling them internally in malloc loses this information
and results in avoidable calls to memset.
Instead, let the compiler take the advantage of it whenever possible.
eadler [Sat, 2 Jun 2018 21:40:45 +0000 (21:40 +0000)]
top(1): cleanup memory allocation and warnings
- Prefer calloc over malloc. This is more predicable and we're not in a
performance sensitive context. [1]
- Remove bogus comment (obsolete from prior commit). [2]
- Remove void casts and type casts of NULL
- Remove redundant declaration of 'quit'
- Add additional const
mjg [Sat, 2 Jun 2018 20:14:43 +0000 (20:14 +0000)]
amd64: add a mild depessimization to rep mov/stos users
Currently all the primitives are waiting for a rewrite, tidy them up in the
meantime.
Vast majority of cases pass sizes which are multiple of 8. Which means the
following rep stosb/movb has nothing to do. Turns out testing first if there
is anything to do is a big win across the board (cpus with and without ERMS,
Intel and AMD) while not pessimizing the case where there is work to do.
jhibbits [Sat, 2 Jun 2018 19:17:11 +0000 (19:17 +0000)]
Added ptrace support for reading/writing powerpc VSX registers
Summary:
Added ptrace support for getting/setting the remaining part of the VSX registers
(the part that's not already covered by FPR or VR registers).
This is necessary to add support for VSX registers in debuggers.
Submitted by: Luis Pires
Differential Revision: https://reviews.freebsd.org/D15458
mjg [Sat, 2 Jun 2018 18:03:35 +0000 (18:03 +0000)]
Use __builtin for various mem* and b* (e.g. bzero) routines.
Some of the routines were using artificially limited builtin already,
drop the explicit limit.
The use of builtins allows quite often allows the compiler to elide the call
or most zeroing to begin with. For instance, if the target object is 32 bytes
in size and gets zeroed + has 16 bytes initialized, the compiler can just
add code to zero out the rest.
Note not all the primites have asm variants and some of the existing ones
are not optimized. Maintaines are strongly encourage to take a look
(regardless of this change).
mjg [Sat, 2 Jun 2018 17:57:09 +0000 (17:57 +0000)]
libkern: tidy up memset
1. Remove special-casing of 0 as it just results in an extra function call.
This is clearly pessimal.
2. Drop the inline stuff. For the most part it is much better served with
__builtin_memset (coming later).
3. Move the declaration to systm.h to match other funcs.
Archs are encouraged to implement the variant for their own platform so that
this implementation can be dropped.
tuexen [Sat, 2 Jun 2018 16:28:10 +0000 (16:28 +0000)]
Don't overflow a buffer if we receive an INIT or INIT-ACK chunk
without a RANDOM parameter but with a CHUNKS or HMAC-ALGO parameter.
Please note that sending this combination violates the specification.
Thnanks to Ronald E. Crane for reporting the issue for the userland
stack.
bde [Sat, 2 Jun 2018 14:07:27 +0000 (14:07 +0000)]
Improve defaults for per-CPU kernel console colors, especially with 2
or 4 CPUs. Add a compile-time option SC_KERNEL_CONS_ATTRS to control the
defaults.
Default to color numbers in reverse order to CPU numbers (instead of
in the same order with white first and wrapping to dark grey), so that
the brightest bright colors are used first. Don't use dark grey at all;
replace it by dark green.
Syscons has too many compile-time options, but this one is needed in
in case the defaults give something like white on white, or the user
really hates this feature and can't wait to turn it off in rc.
bde [Sat, 2 Jun 2018 10:36:30 +0000 (10:36 +0000)]
Use per-CPU attributes earlier.
The per-CPU ts is not initialized early, so the global kernel ts is used
early, but it ony has 1 (normal) attribute. Switch this to the per-CPU
attribute.
The difference is most visible with EARLY_AP_STARTUP.
Change to using the curcpu macro instead of PCPU_GET(cpuid) in 2 places for
the above and in 1 other place in my old code in syscons. The function-like
spelling is perhaps better for indicating that curcpu is volatile (unlike
curthread), but for CPU attributes volatility is a feature.
bde [Sat, 2 Jun 2018 08:38:59 +0000 (08:38 +0000)]
Fix low-level locking during panics.
The SCHEDULER_STOPPED() hack breaks locking generally, and
mtx_trylock_*() especially. When mtx_trylock_*() returns nonzero,
naive code version here trusts it to have worked. But when
SCHEDULER_STOPPED() is true, mtx_trylock_*() returns 1 without doing
anything. Then mtx_unlock_*() crashes especially badly attempting to
unlock iff the error is detected, since mutex unlocking functions don't
check SCHEDULER_STOPPED().
syscons already didn't trust mtx_trylock_spin(), but it was missing the
logic to turn on sp->kdb_locked when turning off sp->mtx_locked during
panics. It also used panicstr instead of SCHEDULER_LOCKED because I
thought that panicstr was more fragile. They only differ for a window
of lines in panic(), and in broken cases where stop_cpus_hard() in panic()
didn't work.
bde [Sat, 2 Jun 2018 06:40:15 +0000 (06:40 +0000)]
Finish COMPAT_AOUT support for amd64. It wasn't in any amd64 or MI
file in /sys/conf, so was unavailable in configurations that don't use
modules, and was not testable or notable in NOTES. Its normal
configuration (not using a module) is still silently deprecated in
aout(4) by not mentioning it there.
Update i386 NOTES for COMPAT_AOUT. It is not i386-only, or even very MD.
Sort its entry better.
Finish gzip configuration (but not support) for amd64. gzip is really
gzipped aout. It is currently broken even for i386 (a call to vm fails).
amd64 has always attempted to configure and test it, but it depends on
COMPAT_AOUT (as noted). The bug that it depends on unconfigured files
was not detected since it is configured as a device. All other optional
image activators are configured properly using an option.
bde [Sat, 2 Jun 2018 05:48:44 +0000 (05:48 +0000)]
Fix high resolution kernel profiling just enough to not crash at boot
time, especially for SMP. If configured, it turns itself on at boot
time for calibration, so is fragile even if never otherwise used.
Both types of kernel profiling were supposed to use a global spinlock
in the SMP case. If hi-res profiling is configured (but not necessarily
used), this was supposed to be optimized by only using it when
necessary, and slightly more efficiently, in asm. But it was not done
at all for mcount entry where it is necessary. This caused crashes
in the SMP case when either type of profiling was enabled. For mcount
exit, it only caused wrong times. The times were wrongest with an
i8254 timer since using that requires exclusive access to the hardware.
The i8254 timer was too slow to use here 20 years ago and is much less
usable now, but it is the default for the SMP case since TSCs weren't
invariant when SMP was new. Do the locking in all hi-res SMP cases for
simplicity.
Calibration uses special asms, and the clobber lists in these were sort
of inverted. They contained the arg and return registers which are not
clobbered, but on amd64 they didn't contain the residue of the call-used
registers which may be clobbered (%r10 and %r11). This usually caused
hangs at boot time. This usually affected even the UP case.
bde [Sat, 2 Jun 2018 04:25:09 +0000 (04:25 +0000)]
Fix recent breakages of kernel profiling, mostly on i386 (high resolution
kernel profiling remains broken).
memmove() was broken using ALTENTRY(). ALTENTRY() is only different from
ENTRY() in the profiling case, and its use in that case was sort of
backwards. The backwardness magically turned memmove() into memcpy()
instead of completely breaking it. Only the high resolution parts of
profiling itself were broken. Use ordinary ENTRY() for memmove().
Turn bcopy() into a tail call to memmove() to reduce complications.
This gives slightly different pessimizations and profiling lossage.
The pessimizations are minimized by not using a frame pointer() for
bcopy().
Calls to profiling functions from exception trampolines were not
relocated. This caused crashes on the first exception. Fix this using
function pointers.
Addresses of exception handlers in trampolines were not relocated. This
caused unknown offsets in the profiling data. Relocate by abusing
setidt_disp as for pmc although this is slower than necessary and
requires namespace pollution. pmc seems to be missing some relocations.
Stack traces and lots of other things in debuggers need similar relocations.
Most user addresses were misclassified as unknown kernel addresses and
then ignored. Treat all unknown addresses as user. Now only user
addresses in the kernel text range are significantly misclassified (as
known kernel addresses).
The ibrs functions didn't preserve enough registers. This is the only
recent breakage on amd64. Although these functions are written in
asm, in the profiling case they call profiling functions which are
mostly for the C ABI, so they only have to save call-used registers.
They also have to save arg and return registers in some cases and
actually save them in all cases to reduce complications. They end up
saving all registers except %ecx on i386 and %r10 and %r11 on amd64.
Saving these is only needed for 1 caller on each of amd64 and i386.
Save them there. This is slightly simpler.
Remove saving %ecx in handle_ibrs_exit on i386. Both handle_ibrs_entry
and handle_ibrs_exit use %ecx, but only the latter needed to or did
save it. But saving it there doesn't work for the profiling case.
amd64 has more automatic saving of the most common scratch registers
%rax, %rcx and %rdx (its complications for %r10 are from unusual use
of %r10 by SYSCALL). Thus profiling of handle_ibrs_exit_rs() was not
broken, and I didn't simplify the saving by moving the saving of these
registers from it to the caller.
rmacklem [Sat, 2 Jun 2018 00:11:26 +0000 (00:11 +0000)]
Fix the default number of threads for Flex File layout pNFS client I/O.
The intent was that the default would be based on number of CPUs, but the
code disabled using taskqueue() by default.
This code is only executed when mounting a NFSv4.1 server that supports the
Flexible File layout for pNFS and, since such servers are rare, this change
shouldn't result in a POLA violation.
(The FreeBSD pNFS server is still a project and the only other one that
uses Flexible File layout is being developed by Primary Data and I don't
know if they have even shipped any to customers yet.)
Found while testing the pNFS server.
eadler [Sat, 2 Jun 2018 00:02:15 +0000 (00:02 +0000)]
top(1): ansify, style(9). and nits
- Prefer using ansi prototypes rather than C prototypes
- Keep type on separate line from name of function
- Try to keep things const where possible. This will help get to WARNS=6
- switch to "bool" where it makes sense
markj [Sat, 2 Jun 2018 00:01:07 +0000 (00:01 +0000)]
Remove the "pass" variable from the page daemon control loop.
It serves little purpose after r308474 and r329882. As a side
effect, the removal fixes a bug in r329882 which caused the
page daemon to periodically invoke lowmem handlers even in the
absence of memory pressure.
Reviewed by: jeff
Differential Revision: https://reviews.freebsd.org/D15491