grep(1) changes:
- Pet mandoc & igor.
- Stylize the text more with macros when appropriate.
- Stylize equal signs in long options (e.g., "--color=auto") with
the "Cm" macro as suggested by mdoc(7).
- Add missing arguments to --exlude, --exclude-dir, --include and
--include-dir.
- Remove a duplicate entry for the --context flag.
- Use a list in the EXAMPLES sections to make it easier to tell
which paragraphs belong to which example.
- Cross reference zgrep(1).
zgrep(1) changes:
- Fix Nd.
- Split synopsis into paragraphs for readability.
- Cross reference bzip(1), grep(1) and xz(1).
Reviewed by: bcr
Approved by: mat (mentor)
Differential Revision: https://reviews.freebsd.org/D16779
- Move some information out of the SYNOPSIS section because it is formated
in a broken way by mandoc(1) otherwise.
- Improve the formatting of the list of provided ioctls.
- Use "Fa" for struct fields.
- Pet mandoc and igor.
- Fix typos.
Reviewed by: kevans
Approved by: mat (mentor)
Differential Revision: https://reviews.freebsd.org/D16765
Kyle Evans [Mon, 20 Aug 2018 22:08:03 +0000 (22:08 +0000)]
config(8): Allow escape-quoted empty strings
For use with things like BOOT_TAG=\"\" -- there are valid reasons to allow
empty strings, especially as these are usually being passed through as
options. The same argument could perhaps be made for the unquoted
variant in things like MODULES_OVERRIDE="", but it's not immediately clear
that this is an issue so I've left it untouched.
Kirk McKusick [Mon, 20 Aug 2018 20:44:11 +0000 (20:44 +0000)]
Fix incorrect output when printing block lists for files small enough
to fit in only direct blocks whose size is exactly a multiple of the
filesystem block size.
Reported by: Peter Holm
Tested by: Peter Holm
Sponsored by: Netflix
Warner Losh [Mon, 20 Aug 2018 19:39:49 +0000 (19:39 +0000)]
mergemaster: better defaults for SOURCEDIR
If we can't find a Makefile.inc1 in the specified / default SOURCEDIR, and
there's a Makefile.inc1 in the current directory, offer the user the choice
of using . for SOURCEDIR.
If an exception or NMI occurs before CPU switched to a pmap different
from vmspace0, PCPU kcr3 is left zero for pti config, which causes
triple-fault in the handler.
Sponsored by: The FreeBSD Foundation
MFC after: 3 days
Added ACPI Device HID AMDI0010 for the designware I2C controllers in
future AMD platforms. Also, when verifying component version check for
minimal value instead of exact match.
In r324732 sysinstall was replaced by bsdinstall.
However, for post-install configuration, bsdinstall
is not of much use. Point the user to bsdconfig instead.
Warner Losh [Mon, 20 Aug 2018 16:44:09 +0000 (16:44 +0000)]
Serial console menus for lua.
Remove a bunch of special cases for UEFI and serial consoles. We do
want to do curses and menu things here. This makes us match what we do
in FORTH, with the possible exception of boxes around menus.
Randall Stewart [Mon, 20 Aug 2018 12:43:18 +0000 (12:43 +0000)]
This change represents a substantial restructure of the way we
reassembly inbound tcp segments. The old algorithm just blindly
dropped in segments without coalescing. This meant that every
segment could take up greater and greater room on the linked list
of segments. This of course is now subject to a tighter limit (100)
of segments which in a high BDP situation will cause us to be a
lot more in-efficent as we drop segments beyond 100 entries that
we receive. What this restructure does is cause the reassembly
buffer to coalesce segments putting an emphasis on the two
common cases (which avoid walking the list of segments) i.e.
where we add to the back of the queue of segments and where we
add to the front. We also have the reassembly buffer supporting
a couple of debug options (black box logging as well as counters
for code coverage). These are compiled out by default but can
be added by uncommenting the defines.
Sponsored by: Netflix Inc.
Differential Revision: https://reviews.freebsd.org/D16626
Alex Richardson [Mon, 20 Aug 2018 10:39:53 +0000 (10:39 +0000)]
Don't create directories in ${WORLDTMP}/legacy with mtree
This has two advantages:
1) We no longer create lots of empty directories that are not needed
2) This is a requirement for building on non-FreeBSD hosts since mtree will
only exist after the bootstrap-tools phase there.
Alex Richardson [Mon, 20 Aug 2018 10:39:42 +0000 (10:39 +0000)]
Avoid depending on system headers from the source tree during bootstrap
This can cause surprising errors if the build tools is built against
headers that don't match the host system. It is also required in order
to allow building on non-FreeBSD systems where the headers in
/usr/include/sys are usually completely incompatible with those in the
source tree.
I added an error to Makefile.boot if this is done and found this was
only the case in libnv. With this error in the Makefile ABI breakages
such as r336019 should no longer be possible.
Alex Richardson [Mon, 20 Aug 2018 10:39:37 +0000 (10:39 +0000)]
Make mkioctls script work on Linux and MacOS
Using find -s will not work with the Linux or MacOS find command. We pipe
to sort instead since the only real requirement here is that the order
stays the same. While I am touching this file I also fixed a `==` construct
which is not supported by POSIX sh but appears to work on FreeBSD.
Marcelo Araujo [Mon, 20 Aug 2018 04:56:37 +0000 (04:56 +0000)]
NVMe spec version 1.3c says that "serial number" field must be 7-bit ASCII,
with unused bytes padded by space characters. Same for firmware number and
namespace number.
Kyle Evans [Mon, 20 Aug 2018 02:40:10 +0000 (02:40 +0000)]
lualoader: Install all manpages
Now that a complete set is written, save for one describing loader.lua,
install all of them. This was not previously done as they were written to
hopefully avoid confusion as bits and pieces of the overall system were
undocumented.
Xin LI [Mon, 20 Aug 2018 02:17:55 +0000 (02:17 +0000)]
In r331279 the code used ENOSYS to check the existence of getrandom(2).
This will only work if the caller already handles SIGSYS, which is not
always the case.
Address this by checking osreldate instead. Note that because there
was not __FreeBSD_version bump when the system call was added, use 1200061 (r332100) which is the first bump after the introduction of
the system call.
PR: 230762
Reported by: Jenkins via Mark Millard
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D16807
John-Mark Gurney [Sun, 19 Aug 2018 21:37:51 +0000 (21:37 +0000)]
use sbuf so that lines are printed together... As aarch64 often
has SMP enabled, lines can get intermixed with other console output
making these lines hard to read...
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D16689
Matt Macy [Sun, 19 Aug 2018 21:10:21 +0000 (21:10 +0000)]
add snps IP uart support / genaralize UART
This is an amalgam of a patch by Doug Ambrisko to
generalize uart_acpi_find_device, imp moving the
ACPI table to uart_dev_ns8250.c and advice by jhb
to work around a bug in the EPYC 3151 BIOS
(the BIOS incorrectly marks the serial ports as
disabled)
Emmanuel Vadot [Sun, 19 Aug 2018 18:55:33 +0000 (18:55 +0000)]
arm64: allwinner: Add aw_syscon driver to GENERIC
Recent DTS use the syscon for the emac controller.
We support this but since U-Boot is still using old DTS it was never
needed for us to add this support, but this is a problem when using upstream
recent DTS and will be when U-Boot will catch up.
While here add a new compatible to the aw_syscon driver as Linux changed it ...
Update L1TF workaround to sustain L1D pollution from NMI.
Current mitigation for L1TF in bhyve flushes L1D either by an explicit
WRMSR command, or by software reading enough uninteresting data to
fully populate all lines of L1D. If NMI occurs after either of
methods is completed, but before VM entry, L1D becomes polluted with
the cache lines touched by NMI handlers. There is no interesting data
which NMI accesses, but something sensitive might be co-located on the
same cache line, and then L1TF exposes that to a rogue guest.
Use VM entry MSR load list to ensure atomicity of L1D cache and VM
entry if updated microcode was loaded. If only software flush method
is available, try to help the bhyve sw flusher by also flushing L1D on
NMI exit to kernel mode.
Suggested by and discussed with: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D16790
Kyle Evans [Sun, 19 Aug 2018 18:43:10 +0000 (18:43 +0000)]
lualoader: Add drawer-exported variables for default logodefs
Uncovered while writing the documentation from this, we previously
explicitly fell back to orb or orbbw if an invalid or incompatible logodef
was selected -- in contrast to branddefs, which have an exported variable
that one can whip up a quick local.lua to override in a safe manner that
works regardless of whether or not loader.conf(5) successfully loads.
Kyle Evans [Sun, 19 Aug 2018 18:37:33 +0000 (18:37 +0000)]
lualoader: Hide the rest of the private interfaces
These are less controversial than the others, thus done in a separate
commit. These are all used internally and ways to override are provided via
soon-to-be-documented API or loader.conf(5) variables.
Kyle Evans [Sun, 19 Aug 2018 18:22:01 +0000 (18:22 +0000)]
lualoader: Hide most of the internal drawing functions
Ideally, all of the functionality to revamp the loader screen has associated
APIs that are flexible enough that third-party scripts wouldn't need to
override these.
Warner Losh [Sun, 19 Aug 2018 18:18:19 +0000 (18:18 +0000)]
Turn back the clock just a little: make userboot.so always be 4th
Turns out there was a hidden dependency we hasn't counted upon. The
host load /boot/userboot.so to boot the VMs it runs. This means that
the change to lua meant suddently that nobody could run their older
VMs because LUA wasn't in 10.0, last month's HardenedBSD, 11.2 or
whatever. Even more than for the /boot/loader* binaries, we need a
good coexistance strategy for this. While that's being designed and
implemented, drop back to always 4th for userboot.so. This will fail
safe in all but the most extreme environments (but lua-only hacks
to .lua files won't be processes in VMs until we fix it).
John Baldwin [Sun, 19 Aug 2018 17:36:50 +0000 (17:36 +0000)]
Fix the MPTable probe code after the 4:4 changes on i386.
The MPTable probe code was using PMAP_MAP_LOW as the PA -> VA offset
when searching for the table signature but still using KERNBASE once
it had found the table. As a result, the mpfps table pointed into a
random part of the kernel text instead of the actual MP Table.
Rather than adding more #ifdef's, use BIOS_PADDRTOVADDR from
<machine/pc/bios.h> which already uses PMAP_MAP_LOW on i386 and KERNBASE
on amd64.
Kirk McKusick [Sun, 19 Aug 2018 17:19:20 +0000 (17:19 +0000)]
For traditional disks, the filesystem attempts to allocate the
blocks of a file as contiguously as possible. Since the filesystem
does not know how large a file will grow when it is first being
written, it initially places the file in a set of blocks in which
it currently fits. As it grows, it is relocated to areas with
larger contiguous blocks. In this way it saves its large contiguous
sets of blocks for the files that need them and thus avoids
unnecessaily fragmenting its disk space.
We used to skip reallocating the blocks of a file into a contiguous
sequence if the underlying flash device requested BIO_DELETE
notifications, because devices that benefit from BIO_DELETE also
benefit from not moving the data. However, in the algorithm described
above that reallocates the blocks, the destination for the data is
usually moved before the data is written to the initially allocated
location. So we rarely suffer the penalty of extra writes. With
the addition of the consolodation of contiguous blocks into single
BIO_DELETE operations, having fewer but larger contiguous blocks
reduces the number of (slow and expensive) BIO_DELETE operations.
So when doing BIO_DELETE consolodation, we do block reallocation.
Reviewed by: kib
Tested by: Peter Holm
Sponsored by: Netflix
Kirk McKusick [Sun, 19 Aug 2018 16:56:42 +0000 (16:56 +0000)]
Add consolodation of TRIM / BIO_DELETE commands to the UFS/FFS filesystem.
When deleting files on filesystems that are stored on flash-memory
(solid-state) disk drives, the filesystem notifies the underlying
disk of the blocks that it is no longer using. The notification
allows the drive to avoid saving these blocks when it needs to
flash (zero out) one of its flash pages. These notifications of
no-longer-being-used blocks are referred to as TRIM notifications.
In FreeBSD these TRIM notifications are sent from the filesystem
to the drive using the BIO_DELETE command.
Until now, the filesystem would send a separate message to the drive
for each block of the file that was deleted. Each Gigabyte of file
size resulted in over 3000 TRIM messages being sent to the drive.
This burst of messages can overwhelm the drive's task queue causing
multiple second delays for read and write requests.
This implementation collects runs of contiguous blocks in the file
and then consolodates them into a single BIO_DELETE command to the
drive. The BIO_DELETE command describes the run of blocks as a
single large block being deleted. Each Gigabyte of file size can
result in as few as two BIO_DELETE commands and is typically less
than ten. Though these larger BIO_DELETE commands take longer to
run, they do not clog the drive task queue, so read and write
commands can intersperse effectively with them.
Though this new feature has been throughly reviewed and tested, it
is being added disabled by default so as to minimize the possibility
of disrupting the upcoming 12.0 release. It can be enabled by running
``sysctl vfs.ffs.dotrimcons=1''. Users are encouraged to test it.
If no problems arise, we will consider requesting that it be enabled
by default for 12.0.
Reviewed by: kib
Tested by: Peter Holm
Sponsored by: Netflix
John Baldwin [Sun, 19 Aug 2018 16:14:59 +0000 (16:14 +0000)]
Remove some vestiges of IPI_LAZYPMAP on i386.
The support for lazy pmap invalidations on i386 was removed in r281707.
This removes the constant for the IPI and stops accounting for it when
sizing the interrupt count arrays.
Michael Tuexen [Sun, 19 Aug 2018 14:56:10 +0000 (14:56 +0000)]
Don't expose the uptime via the TCP timestamps.
The TCP client side or the TCP server side when not using SYN-cookies
used the uptime as the TCP timestamp value. This patch uses in all
cases an offset, which is the result of a keyed hash function taking
the source and destination addresses and port numbers into account.
The keyed hash function is the same a used for the initial TSN.
Kyle Evans [Sun, 19 Aug 2018 14:26:33 +0000 (14:26 +0000)]
stand: Flip the default interpreter to Lua
After years in the making, lualoader is ready to make its debut. Both
flavors of loader are still built by default, and may be installed as
/boot/loader or /boot/loader.efi as appropriate either by manually creating
hard links or using LOADER_DEFAULT_INTERP as documented in build(7).
Cy Schubert [Sun, 19 Aug 2018 13:45:03 +0000 (13:45 +0000)]
The bucket index is subtracted by one at lines 2304 and 2314. When 0 it
becomes -1, except these are unsigned integers, so they become very large
numbers. Thus are always larger than the maximum bucket; the hash table
insertion fails causing NAT to fail.
This commit ensures that if the index is already zero it is not reduced
prior to insertion into the hash table.
Cy Schubert [Sun, 19 Aug 2018 13:44:59 +0000 (13:44 +0000)]
Add handy DTrace probes useful in diagnosing NAT issues. DTrace probes
are situated next to error counters and/or in one instance prior to the
-1 return from various functions. This was useful in diagnosis of
PR/208566 and will be handy in the future diagnosing NAT failures.
Cy Schubert [Sun, 19 Aug 2018 13:44:56 +0000 (13:44 +0000)]
Expose np (nat_t - an entry in the nat table structure) in the DTrace
probe when nat fails (label badnat). This is useful in diagnosing
failed NAT issues and was used in PR/208566.
Kyle Evans [Sun, 19 Aug 2018 04:15:38 +0000 (04:15 +0000)]
diff(1): Refactor -B a little bit
Instead of doing a second pass to skip empty lines if we've specified -I, go
ahead and check both at once. Ignore critera has been split out into its own
function to try and keep the logic cleaner.
Kyle Evans [Sun, 19 Aug 2018 03:57:20 +0000 (03:57 +0000)]
diff(1): Implement -B/--ignore-blank-lines
As noted by cem in r338035, coccinelle invokes diff(1) with the -B flag.
This was not previously implemented here, so one was forced to create a link
for GNU diff to /usr/local/bin/diff
Implement the -B flag and add some primitive tests for it. It is implemented
in the same fashion that -I is implemented; each chunk's lines are scanned,
and if a non-blank line is encountered then the chunk will be output.
Otherwise, it's skipped.
Conrad Meyer [Sun, 19 Aug 2018 00:22:21 +0000 (00:22 +0000)]
Remove unused and easy to misuse PNP macro parameter
Inspired by r338025, just remove the element size parameter to the
MODULE_PNP_INFO macro entirely. The 'table' parameter is now required to
have correct pointer (or array) type. Since all invocations of the macro
already had this property and the emitted PNP data continues to include the
element size, there is no functional change.
(Note that coccinelle invokes diff(1) via a PATH search and expects diff to
tolerate the -B flag, which BSD diff does not. So I had to link gdiff into
PATH as diff to use spatch.)
Kirk McKusick [Sat, 18 Aug 2018 22:21:59 +0000 (22:21 +0000)]
Replace the TRIM consolodation framework originally added in -r337396
driven by problems found with the algorithms being tested for TRIM
consolodation.
Kyle Evans [Sat, 18 Aug 2018 20:55:20 +0000 (20:55 +0000)]
ls(1): Support other aliases for --color arguments used by GNU ls(1)
These aliases are supported and documented in the man page. For now, they
will not be mentioned in the error when an invalid argument is encountered,
instead keeping that list to the shorter 'preferred' names of each argument.
Dimitry Andric [Sat, 18 Aug 2018 20:41:43 +0000 (20:41 +0000)]
Use the size of one bge_devs element for the MODULE_PNP_INFO macro,
instead of the size of the whole bge_devs array.
This should stop kldxref searching beyond the end of .rodata when it
processes relocations, and emitting "unhandled relocation type" errors,
at least on i386.
This is very primitive code to inspect the PCI error state and AER
error state, dump the log and clear errors, from ddb.
pci_print_faulted_dev() is made external to allow calling it from
other places. It was called from NMI handler but this chunk is not
included.
Also there is a tunable-controlled code to clear AER on device attach,
disabled by default.
All this code was useful to me when I debugged ACPI_DMAR failures (not
faults) long time ago.
Reviewed by: cem, imp (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D7813
John Baldwin [Sat, 18 Aug 2018 20:32:08 +0000 (20:32 +0000)]
Make 'device crypto' lines more consistent.
- In configurations with a pseudo devices section, move 'device crypto'
into that section.
- Use a consistent comment. Note that other things common in kernel
configs such as GELI also require 'device crypto', not just IPSEC.
John Baldwin [Sat, 18 Aug 2018 20:28:25 +0000 (20:28 +0000)]
Fix casts between 64-bit physical addresses and pointers in EFI.
Compiling FreeBSD/i386 with modern GCC triggers warnings for various
places that convert 64-bit EFI_ADDRs to pointers and vice versa.
- Cast pointers to uintptr_t rather than to uint64_t when assigning
to a 64-bit integer.
- Cast 64-bit integers to uintptr_t before a cast to a pointer.
Kyle Evans [Sat, 18 Aug 2018 19:45:56 +0000 (19:45 +0000)]
res_find: Fix fallback logic
The fallback logic was broken if hints were found in multiple environments.
If we found a hint in either the loader environment or the static
environment, fallback would be incremented excessively when we returned to
the environment-selection bits. These checks should have also been guarded
by the fbacklvl checks. As a result, fbacklvl could quickly get to a point
where we skip either the static environment and/or the static hints
depending on which environments contained valid hints.
The impact of this bug is minimal, mostly affecting mips boards that use
static hints and may have hints in either the loader environment or the
static environment.
There may be better ways to express the searchable environments and
describing their characteristics (immutable, already searched, etc.) but
this may be revisited after 12 branches.
Reported by: Dan Nelson <dnelson_1901@yahoo.com>
Triaged by: Dan Nelson <dnelson_1901@yahoo.com>
MFC after: 3 days
Rick Macklem [Sat, 18 Aug 2018 19:14:06 +0000 (19:14 +0000)]
Fix LORs between vn_start_write() and vn_lock() in nfsrv_copymr().
When coding the pNFS server, I added vn_start_write() calls in nfsrv_copymr()
done while the vnodes were locked, not realizing I had introduced LORs and
possible deadlock when an exported file system on the MDS is suspended.
This patch fixes the LORs by moving the vn_start_write() calls up to before
where the vnodes are locked. For "tvp", the vn_start_write() probaby isn't
necessary, because NFS mounts can't be suspended. However, I think doing
so is harmless.
Thanks go to kib@ for letting me know that I had introduced these LORs.
This patch only affects the behaviour of the pNFS server when pnfsdscopymr(8)
is used to recover a mirrored DS.
Alan Cox [Sat, 18 Aug 2018 18:33:50 +0000 (18:33 +0000)]
Eliminate the arena parameter to kmem_malloc_domain(). It is redundant.
The domain and flags parameters suffice. In fact, the related functions
kmem_alloc_{attr,contig}_domain() don't have an arena parameter.
Eugene Grosbein [Sat, 18 Aug 2018 10:58:44 +0000 (10:58 +0000)]
bsnmpd(8): fix and optimize interface description processing
* correctly prepare a buffer to obtain interface description from a kernel and
truncate long description instead of dropping it altogether and
spamming logs;
* skip calling strlen() for each description and each SNMP request
for MIB-II/ifXTable's ifAlias.
* teach bsnmpd to allocate memory dynamically for interface descriptions
to decrease memory usage for common case and not to break
if long description occurs;
PR: 217763
Reviewed by: harti and others
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D16459
Kyle Evans [Sat, 18 Aug 2018 03:20:59 +0000 (03:20 +0000)]
libbe(3): Move build goop back out of cddl/
Some background: in the GSoC project, libbe/Makefile lived in lib/libbe. I
created projects/bectl branch, maintained the above for all of five
minutes before I misread Makefile.inc1 and decided that it couldn't possibly
build outside of cddl/, so I kicked the Makefile out into the cddl/ build
and all was good. The misreading was of the bit where .WAIT is added to
SUBDIR after lib, libexec but prior to building bin and cddl *only during
the install targets*, which is the critical part.
Fast forward- buildworld was still broken in my branch unbeknownst to me
because I didn't nuke my OBJDIR. Combing through Makefile.inc1 eventually
revealed the necessary magic to make sure that libbe's dependencies are
specified well enough, and it becomes clear what needs done to make a
non-cddl/ build work. This is an interesting prospect, because the build
split is kind of annoying to work with.
IGNORE_PRAGMA is added to avoid dropping WARNS by one more. This was
previously pulled in via cddl/Makefile.inc.
Kyle Evans [Sat, 18 Aug 2018 01:12:44 +0000 (01:12 +0000)]
bectl(8): Allow running a custom command in the 'jail' subcommand
Instead of always running /bin/sh, allow the user to specify the command
to run. The jail is not removed when the command finishes. Meaning,
`bectl unjail` will still need to be run.
projects / FreeBSD / FreeBSD.git / log