mmacy [Mon, 4 Jun 2018 02:05:48 +0000 (02:05 +0000)]
hwpmc: ABI fixes
- increase pmc cpuid field from 8 to 12 bits
- add cpuid version string to initialize entry in the log
so that filter can identify which counter index an
event name maps to
- GC unused config flags
- make fixed counter assignment more robust as well as the
changes needed to be properly identified for filter
mmacy [Mon, 4 Jun 2018 01:10:23 +0000 (01:10 +0000)]
hwpmc: support sampling both kernel and user stacks when interrupted in kernel
This adds the -U options to pmcstat which will attribute in-kernel samples
back to the user stack that invoked the system call. It is not the default,
because when looking at kernel profiles it is generally more desirable to
merge all instances of a given system call together.
Although heavily revised, this change is directly derived from D7350 by
Jonathan T. Looney.
eadler [Sun, 3 Jun 2018 23:40:54 +0000 (23:40 +0000)]
top(1): another pass of cleanup
- avoid the need to call a function to get size of known array. I'll
likely re-arrange some of the indirect in a later to avoid the magic
constants.
- use correct type
- add const
- replace caddr_t with void*. This corrects an alignment warning.
- remove duplicated include from immediately prior commit
Under base clang we're now down to:
- 3 warning in top.c, 1 warning in mahcine.c, 4 warning in display.c,
- 1 warning in utils.c
Tested with base clang, gcc7, gcc9, base gcc (mips)
eadler [Sun, 3 Jun 2018 22:42:54 +0000 (22:42 +0000)]
top(1): top warnings and cleanup
- Add const where helpful
- add missing 'static' for file-local functions
- use nitems where possible
- convert manual abort() to assert
- use strndup instead of homegrown version
pstef [Sun, 3 Jun 2018 21:40:38 +0000 (21:40 +0000)]
indent(1): new option -lpl
With -lpl, code surrounded by parentheses in continuation lines is lined up
even if it would extend past the right margin.
With -nlpl (the default), such a line that would extend past the right
margin is moved left to keep it within the margin, if that does not require
placing it to the left of the prevailing indentation level.
These switches have no effect if -nlp is selected.
pstef [Sun, 3 Jun 2018 20:59:59 +0000 (20:59 +0000)]
indent(1): new option -lpl (always line up to parenthesis)
With -lp, if a line has an opening paren which is not closed on that line,
then continuation lines will be lined up to start at the character position
just after the opening paren.
rmacklem [Sun, 3 Jun 2018 19:46:44 +0000 (19:46 +0000)]
Fix a gcc8 warning about a write only variable.
gcc8 warns that "verf" was set but not used. This was because the code
that uses it is disabled via a "#if 0".
This patch adds a "#if 0" to the variable's declaration and assignment
to get rid of the warning.
This way the code could be re-enabled without difficulty.
pstef [Sun, 3 Jun 2018 18:19:41 +0000 (18:19 +0000)]
indent(1): improve CHECK_SIZE_ macros
Rewrite the macros so that they take a parameter. Consumers use it to signal
how much room in the buffer they need; this lets them do that once when
required space is known instead of doing the check once every loop step.
Also take the parameter value into consideration when resizing the buffer;
the requested space may be larger than the constant 400 bytes that the
previous version used - now it's the sum of those two values.
On the consumer side, don't copy strings byte by byte - use memcpy().
Deduplicate code that copied base 2, base 8 and base 16 literals.
Don't advance the e_token pointer once the token has been copied into
s_token. This allows easy calculation of the token's length.
pstef [Sun, 3 Jun 2018 17:55:50 +0000 (17:55 +0000)]
indent(1): remove troff output support
The troff output in indent was invented at Sun and the online documentation
for some post-SunOS operating system includes this:
The usual way to get a troffed listing is with the command
indent -troff program.c | troff -mindent
The indent manual page in FreeBSD 1.0 already lacks that information and
troff -mindent complains about not being able to find the macro file.
It seems that the file did exist on SunOS and was supposed to be imported
into 4.3BSD together with the feature, but that has never happened.
Removal of troff output support simplifies a lot of indent's code.
pstef [Sun, 3 Jun 2018 16:27:40 +0000 (16:27 +0000)]
indent(1): remove is_procname.
It was a shorthand for checking if ps.procname is a non-empty string; the
same can be done with ps.procname[0] which avoids the need for updating
is_procname after every call to lexi().
pstef [Sun, 3 Jun 2018 15:28:55 +0000 (15:28 +0000)]
indent(1): improve handling of boxed comments indentation
The trick is to copy everything from the start of the line into the buffer
that stores newlines and comments until indent finds a brace or an else.
pr_comment() will use that information to calculate the original indentation
of the boxed comment.
This requires storing two pieces of information: the real start of the
buffer (sc_buf) and the start of the comment (save_com).
pstef [Sun, 3 Jun 2018 14:13:11 +0000 (14:13 +0000)]
indent(1): improve predictability of lexi()
lexi() reads the input stream and categorizes the next token. indent will
sometimes buffer up a sequence of tokens in order rearrange them. That is
needed for properly cuddling else or placing braces correctly according to
the chosen style (KNF vs Allman) when comments are around. The loop that
buffers tokens up uses lexi() to decide if it's time to stop buffering. Then
the temporary buffer is used to feed lexi() the same tokens again, this time
for normal processing.
The problem is that lexi() apart from recognizing the token, can change
a lot of information about the current state, for example ps.last_nl,
ps.keyword, buf_ptr. It also abandons leading whitespace, which is needed
mainly for comment-related considerations. So the call to lexi() while
tokens are buffered up and categorized can change the state before they're
read again for normal processing which may easily result in changing
interpretation of the current state and lead to incorrect output.
To work around the problems:
1) copy the whitespace into the save_com buffer so that it will be read
again when processed
2) trick lexi() into modifying a temporary copy of the parser state instead
of the original.
pstef [Sun, 3 Jun 2018 14:03:20 +0000 (14:03 +0000)]
indent(1): improve handling of comments and newlines between "if (...)" or
"while (...)" and "else" or "{"
* Don't flush newlines - there can be multiple of them and they can happen
before a token that isn't else or {. Instead, always store them in save_com.
* Don't dump the buffer's contents on newline assuming that there is only
one comment before else or {.
* Avoid producing surplus newlines, especially before else when -ce is on.
* When -bl is on, don't treat { as a comment (was implemented by falling
through "case lbrace:" to "case comment:").
This commit fixes the above, but exposes another bug and thus breaks several
other tests. Another commit will make them pass again.
eadler [Sun, 3 Jun 2018 05:07:39 +0000 (05:07 +0000)]
top(1): use greater warnings
One of the downsides of using numeric WARNS is that if we only have a
single type of issue we get no protection from other changes. For
example, we got no warning for missing variable declaration, due to
the issues with "const".
For this utility, explicitly list out the warnings which are failing.
They should still be fixed, so only reduce them to warning instead of
error.
Tested with: clang base (amd64, i386), gcc6, gcc7, gcc9, gcc base (mips)
jhibbits [Sun, 3 Jun 2018 03:53:11 +0000 (03:53 +0000)]
Revert r326083, it doesn't behave as expected.
Even though there do appear to be more artificial frames, with 12, stack
traces no longer list at all. Revert until a better, more stable value can
be determined.
mjg [Sat, 2 Jun 2018 22:20:09 +0000 (22:20 +0000)]
malloc: try to use builtins for zeroing at the callsite
Plenty of allocation sites pass M_ZERO and sizes which are small and known
at compilation time. Handling them internally in malloc loses this information
and results in avoidable calls to memset.
Instead, let the compiler take the advantage of it whenever possible.
eadler [Sat, 2 Jun 2018 21:40:45 +0000 (21:40 +0000)]
top(1): cleanup memory allocation and warnings
- Prefer calloc over malloc. This is more predicable and we're not in a
performance sensitive context. [1]
- Remove bogus comment (obsolete from prior commit). [2]
- Remove void casts and type casts of NULL
- Remove redundant declaration of 'quit'
- Add additional const
mjg [Sat, 2 Jun 2018 20:14:43 +0000 (20:14 +0000)]
amd64: add a mild depessimization to rep mov/stos users
Currently all the primitives are waiting for a rewrite, tidy them up in the
meantime.
Vast majority of cases pass sizes which are multiple of 8. Which means the
following rep stosb/movb has nothing to do. Turns out testing first if there
is anything to do is a big win across the board (cpus with and without ERMS,
Intel and AMD) while not pessimizing the case where there is work to do.
jhibbits [Sat, 2 Jun 2018 19:17:11 +0000 (19:17 +0000)]
Added ptrace support for reading/writing powerpc VSX registers
Summary:
Added ptrace support for getting/setting the remaining part of the VSX registers
(the part that's not already covered by FPR or VR registers).
This is necessary to add support for VSX registers in debuggers.
Submitted by: Luis Pires
Differential Revision: https://reviews.freebsd.org/D15458
mjg [Sat, 2 Jun 2018 18:03:35 +0000 (18:03 +0000)]
Use __builtin for various mem* and b* (e.g. bzero) routines.
Some of the routines were using artificially limited builtin already,
drop the explicit limit.
The use of builtins allows quite often allows the compiler to elide the call
or most zeroing to begin with. For instance, if the target object is 32 bytes
in size and gets zeroed + has 16 bytes initialized, the compiler can just
add code to zero out the rest.
Note not all the primites have asm variants and some of the existing ones
are not optimized. Maintaines are strongly encourage to take a look
(regardless of this change).
mjg [Sat, 2 Jun 2018 17:57:09 +0000 (17:57 +0000)]
libkern: tidy up memset
1. Remove special-casing of 0 as it just results in an extra function call.
This is clearly pessimal.
2. Drop the inline stuff. For the most part it is much better served with
__builtin_memset (coming later).
3. Move the declaration to systm.h to match other funcs.
Archs are encouraged to implement the variant for their own platform so that
this implementation can be dropped.
tuexen [Sat, 2 Jun 2018 16:28:10 +0000 (16:28 +0000)]
Don't overflow a buffer if we receive an INIT or INIT-ACK chunk
without a RANDOM parameter but with a CHUNKS or HMAC-ALGO parameter.
Please note that sending this combination violates the specification.
Thnanks to Ronald E. Crane for reporting the issue for the userland
stack.
bde [Sat, 2 Jun 2018 14:07:27 +0000 (14:07 +0000)]
Improve defaults for per-CPU kernel console colors, especially with 2
or 4 CPUs. Add a compile-time option SC_KERNEL_CONS_ATTRS to control the
defaults.
Default to color numbers in reverse order to CPU numbers (instead of
in the same order with white first and wrapping to dark grey), so that
the brightest bright colors are used first. Don't use dark grey at all;
replace it by dark green.
Syscons has too many compile-time options, but this one is needed in
in case the defaults give something like white on white, or the user
really hates this feature and can't wait to turn it off in rc.
bde [Sat, 2 Jun 2018 10:36:30 +0000 (10:36 +0000)]
Use per-CPU attributes earlier.
The per-CPU ts is not initialized early, so the global kernel ts is used
early, but it ony has 1 (normal) attribute. Switch this to the per-CPU
attribute.
The difference is most visible with EARLY_AP_STARTUP.
Change to using the curcpu macro instead of PCPU_GET(cpuid) in 2 places for
the above and in 1 other place in my old code in syscons. The function-like
spelling is perhaps better for indicating that curcpu is volatile (unlike
curthread), but for CPU attributes volatility is a feature.
bde [Sat, 2 Jun 2018 08:38:59 +0000 (08:38 +0000)]
Fix low-level locking during panics.
The SCHEDULER_STOPPED() hack breaks locking generally, and
mtx_trylock_*() especially. When mtx_trylock_*() returns nonzero,
naive code version here trusts it to have worked. But when
SCHEDULER_STOPPED() is true, mtx_trylock_*() returns 1 without doing
anything. Then mtx_unlock_*() crashes especially badly attempting to
unlock iff the error is detected, since mutex unlocking functions don't
check SCHEDULER_STOPPED().
syscons already didn't trust mtx_trylock_spin(), but it was missing the
logic to turn on sp->kdb_locked when turning off sp->mtx_locked during
panics. It also used panicstr instead of SCHEDULER_LOCKED because I
thought that panicstr was more fragile. They only differ for a window
of lines in panic(), and in broken cases where stop_cpus_hard() in panic()
didn't work.
bde [Sat, 2 Jun 2018 06:40:15 +0000 (06:40 +0000)]
Finish COMPAT_AOUT support for amd64. It wasn't in any amd64 or MI
file in /sys/conf, so was unavailable in configurations that don't use
modules, and was not testable or notable in NOTES. Its normal
configuration (not using a module) is still silently deprecated in
aout(4) by not mentioning it there.
Update i386 NOTES for COMPAT_AOUT. It is not i386-only, or even very MD.
Sort its entry better.
Finish gzip configuration (but not support) for amd64. gzip is really
gzipped aout. It is currently broken even for i386 (a call to vm fails).
amd64 has always attempted to configure and test it, but it depends on
COMPAT_AOUT (as noted). The bug that it depends on unconfigured files
was not detected since it is configured as a device. All other optional
image activators are configured properly using an option.
bde [Sat, 2 Jun 2018 05:48:44 +0000 (05:48 +0000)]
Fix high resolution kernel profiling just enough to not crash at boot
time, especially for SMP. If configured, it turns itself on at boot
time for calibration, so is fragile even if never otherwise used.
Both types of kernel profiling were supposed to use a global spinlock
in the SMP case. If hi-res profiling is configured (but not necessarily
used), this was supposed to be optimized by only using it when
necessary, and slightly more efficiently, in asm. But it was not done
at all for mcount entry where it is necessary. This caused crashes
in the SMP case when either type of profiling was enabled. For mcount
exit, it only caused wrong times. The times were wrongest with an
i8254 timer since using that requires exclusive access to the hardware.
The i8254 timer was too slow to use here 20 years ago and is much less
usable now, but it is the default for the SMP case since TSCs weren't
invariant when SMP was new. Do the locking in all hi-res SMP cases for
simplicity.
Calibration uses special asms, and the clobber lists in these were sort
of inverted. They contained the arg and return registers which are not
clobbered, but on amd64 they didn't contain the residue of the call-used
registers which may be clobbered (%r10 and %r11). This usually caused
hangs at boot time. This usually affected even the UP case.
bde [Sat, 2 Jun 2018 04:25:09 +0000 (04:25 +0000)]
Fix recent breakages of kernel profiling, mostly on i386 (high resolution
kernel profiling remains broken).
memmove() was broken using ALTENTRY(). ALTENTRY() is only different from
ENTRY() in the profiling case, and its use in that case was sort of
backwards. The backwardness magically turned memmove() into memcpy()
instead of completely breaking it. Only the high resolution parts of
profiling itself were broken. Use ordinary ENTRY() for memmove().
Turn bcopy() into a tail call to memmove() to reduce complications.
This gives slightly different pessimizations and profiling lossage.
The pessimizations are minimized by not using a frame pointer() for
bcopy().
Calls to profiling functions from exception trampolines were not
relocated. This caused crashes on the first exception. Fix this using
function pointers.
Addresses of exception handlers in trampolines were not relocated. This
caused unknown offsets in the profiling data. Relocate by abusing
setidt_disp as for pmc although this is slower than necessary and
requires namespace pollution. pmc seems to be missing some relocations.
Stack traces and lots of other things in debuggers need similar relocations.
Most user addresses were misclassified as unknown kernel addresses and
then ignored. Treat all unknown addresses as user. Now only user
addresses in the kernel text range are significantly misclassified (as
known kernel addresses).
The ibrs functions didn't preserve enough registers. This is the only
recent breakage on amd64. Although these functions are written in
asm, in the profiling case they call profiling functions which are
mostly for the C ABI, so they only have to save call-used registers.
They also have to save arg and return registers in some cases and
actually save them in all cases to reduce complications. They end up
saving all registers except %ecx on i386 and %r10 and %r11 on amd64.
Saving these is only needed for 1 caller on each of amd64 and i386.
Save them there. This is slightly simpler.
Remove saving %ecx in handle_ibrs_exit on i386. Both handle_ibrs_entry
and handle_ibrs_exit use %ecx, but only the latter needed to or did
save it. But saving it there doesn't work for the profiling case.
amd64 has more automatic saving of the most common scratch registers
%rax, %rcx and %rdx (its complications for %r10 are from unusual use
of %r10 by SYSCALL). Thus profiling of handle_ibrs_exit_rs() was not
broken, and I didn't simplify the saving by moving the saving of these
registers from it to the caller.
rmacklem [Sat, 2 Jun 2018 00:11:26 +0000 (00:11 +0000)]
Fix the default number of threads for Flex File layout pNFS client I/O.
The intent was that the default would be based on number of CPUs, but the
code disabled using taskqueue() by default.
This code is only executed when mounting a NFSv4.1 server that supports the
Flexible File layout for pNFS and, since such servers are rare, this change
shouldn't result in a POLA violation.
(The FreeBSD pNFS server is still a project and the only other one that
uses Flexible File layout is being developed by Primary Data and I don't
know if they have even shipped any to customers yet.)
Found while testing the pNFS server.
eadler [Sat, 2 Jun 2018 00:02:15 +0000 (00:02 +0000)]
top(1): ansify, style(9). and nits
- Prefer using ansi prototypes rather than C prototypes
- Keep type on separate line from name of function
- Try to keep things const where possible. This will help get to WARNS=6
- switch to "bool" where it makes sense
markj [Sat, 2 Jun 2018 00:01:07 +0000 (00:01 +0000)]
Remove the "pass" variable from the page daemon control loop.
It serves little purpose after r308474 and r329882. As a side
effect, the removal fixes a bug in r329882 which caused the
page daemon to periodically invoke lowmem handlers even in the
absence of memory pressure.
Reviewed by: jeff
Differential Revision: https://reviews.freebsd.org/D15491
markj [Fri, 1 Jun 2018 23:49:32 +0000 (23:49 +0000)]
Avoid completing I/O when dumping core after a panic.
Filesystem or pager completion callbacks are generally non-functional
after a panic and may trigger deadlocks if invoked in this context
(e.g., by attempting to destroying a buffer mapping). To avoid this
situation, short-circuit I/O completion in biodone().
cem [Fri, 1 Jun 2018 22:34:59 +0000 (22:34 +0000)]
touch.1: Update to conform to POSIX 2004
POSIX borrowed the "double leap second" bug from C89. Double leap seconds can
never happen. This mistake was present in at least POSIX 1997 and fixed by
POSIX 2004. I can't find a copy of 2001 online to determine if the bug was
present in that revision.
While here, remove duplicate language between -d and -t. A few other minor
enhancements and an igor (lint) bugfix.
alc [Fri, 1 Jun 2018 21:37:42 +0000 (21:37 +0000)]
Only a small subset of mmap(2)'s flags should be used in combination with
the flag MAP_GUARD. Rather than enumerating the flags that are not
allowed, enumerate the flags that are allowed. The list of allowed flags
is much shorter and less likely to change. (As an aside, one of the
previously enumerated flags, MAP_PREFAULT, was not even a legal flag for
mmap(2). However, because of an earlier check within kern_mmap(), this
misuse of MAP_PREFAULT was harmless.)
asomers [Fri, 1 Jun 2018 21:24:10 +0000 (21:24 +0000)]
audit(4): add tests for the fd audit class
The only syscalls in this class are rmdir, unlink, unlinkat, rename, and
renameat. Also, set is_exclusive for all audit(4) tests, because they can
start and stop auditd.
pstef [Fri, 1 Jun 2018 19:56:41 +0000 (19:56 +0000)]
indent(1): restore working -pcs
My previous indent(1) commit accidentally broke the -pcs option (which adds
space between function name and opening parenthesis in function calls) by
copying all but one of a few conditions in an if clause. Reinstate the
condition.
Add a regression test to lower the chances of breaking it again.
Correct a comment with description of what the option does.
rmacklem [Fri, 1 Jun 2018 19:47:41 +0000 (19:47 +0000)]
Add the BindConnectiontoSession operation to the NFSv4.1 server.
Under some fairly unusual circumstances, the Linux NFSv4.1 client is
doing a BindConnectiontoSession operation for TCP connections.
It is also used by the ESXi6.5 NFSv4.1 client.
This patch adds this operation to the NFSv4.1 server.
jtl [Fri, 1 Jun 2018 16:47:39 +0000 (16:47 +0000)]
Update the sysctl(9) manpage to indicate that <sys/param.h> is required
instead of <sys/types.h>. (<sys/sysctl.h> includes NULL, which is defined
with <sys/param.h> and not <sys/types.h>.)