gjb [Mon, 26 Feb 2018 19:26:59 +0000 (19:26 +0000)]
Bump the size of virtual machine disk images from 20G to 30G,
providing more space for a local buildworld to succeed without
attaching separate disks for /usr/src and /usr/obj.
jhb [Mon, 26 Feb 2018 19:19:05 +0000 (19:19 +0000)]
Add a new variant of the GLA2GPA ioctl for use by the debug server.
Unlike the existing GLA2GPA ioctl, GLA2GPA_NOFAULT does not modify
the guest. In particular, it does not inject any faults or modify
PTEs in the guest when performing an address space translation.
This is used by bhyve's debug server to read and write memory for
the remote debugger.
dab [Mon, 26 Feb 2018 18:23:36 +0000 (18:23 +0000)]
iconv uses strlen directly on user supplied memory
`iconv_sysctl_add` from `sys/libkern/iconv.c` incorrectly limits the
size of user strings, such that several out of bounds reads could have
been possible.
static int
iconv_sysctl_add(SYSCTL_HANDLER_ARGS)
{
struct iconv_converter_class *dcp;
struct iconv_cspair *csp;
struct iconv_add_in din;
struct iconv_add_out dout;
int error;
error = SYSCTL_IN(req, &din, sizeof(din));
if (error)
return error;
if (din.ia_version != ICONV_ADD_VER)
return EINVAL;
if (din.ia_datalen > ICONV_CSMAXDATALEN)
return EINVAL;
if (strlen(din.ia_from) >= ICONV_CSNMAXLEN)
return EINVAL;
if (strlen(din.ia_to) >= ICONV_CSNMAXLEN)
return EINVAL;
if (strlen(din.ia_converter) >= ICONV_CNVNMAXLEN)
return EINVAL;
...
Since the `din` struct is directly copied from userland, there is no
guarantee that the strings supplied will be NULL terminated. The
`strlen` calls could continue reading past the designated buffer
sizes.
Declaration of `struct iconv_add_in` is found in `sys/sys/iconv.h`:
struct iconv_add_in {
int ia_version;
char ia_converter[ICONV_CNVNMAXLEN];
char ia_to[ICONV_CSNMAXLEN];
char ia_from[ICONV_CSNMAXLEN];
int ia_datalen;
const void *ia_data;
};
Our strings are followed by the `ia_datalen` member, which is checked
before the `strlen` calls:
if (din.ia_datalen > ICONV_CSMAXDATALEN)
Since `ICONV_CSMAXDATALEN` has value `0x41000` (and is `unsigned`),
this ensures that `din.ia_datalen` contains at least 1 byte of 0, so
it is not possible to trigger a read out of bounds of the `struct`
however, this code is fragile and could introduce subtle bugs in the
future if the `struct` is ever modified.
kevans [Mon, 26 Feb 2018 18:01:35 +0000 (18:01 +0000)]
libsa: Add MAXWAIT to net for establishing max total timeout
Current timeout behavior is to progress in timeout values from MINTMO to
MAXTMO in MINTMO steps before finally timing out. This results in a fairly
long time before operations finally timeout, which may not be ideal for some
use-cases.
Add MAXWAIT that may be configured along with MINTMO/MAXTMO. If we attempt
to start our send/recv cycle over again but MAXWAIT > 0 and MAXWAIT seconds
have already passed, then go ahead and timeout.
This is intended for those that just want to say "timeout after 180 seconds"
rather than calculate and tweak MINTMO/MAXTMO to get their desired timeout.
The default is 0, or "progress from MINTMO to MAXTMO with no exception."
This has been modified since review to allow for it to be defined via CFLAGS
and doing appropriate error checking. Future work may add some Makefile foo
to respect LOADER_NET_MAXWAIT if it's specified in the environment and pass
it in as MAXWAIT accordingly.
kevans [Mon, 26 Feb 2018 15:37:32 +0000 (15:37 +0000)]
lualoader: Re-work menu skipping bits
This is motivated by a want to reduce heap usage if the menu is being
skipped. Currently, the menu module must be loaded regardless of whether
it's being skipped or not, which adds a cool ~50-100KB worth of memory
usage.
Move the menu skip logic out to core (and remove a debug print), then check
in loader.lua if we should be skipping the menu and avoid loading the menu
module entirely if so. This keeps our memory usage below ~115KB for a boot
with the menu stripped.
Also worth noting: with this change, we no longer explicitly invoke autoboot
if we're skipping the menu. Instead, we let the standard loader behavior
apply: try to autoboot if we need to, then drop to a loader prompt if not or
if the autoboot sequence is interrupted. The only thing we still handle
before dropping to the loader autoboot sequence is loadelf(), so that we can
still apply any of our kernel loading behavior.
kevans [Mon, 26 Feb 2018 14:00:23 +0000 (14:00 +0000)]
ofw_fdt: Simplify parts with new libfdt methods
libfdt now provides methods to iterate through subnodes and properties in a
convenient fashion.
Replace our ofw_fdt_{peer,child} searches with calls to their corresponding
libfdt methods. Rework ofw_fdt_nextprop to use the
fdt_for_each_property_offset macro, making it even more obvious what it's
doing.
cognet [Mon, 26 Feb 2018 13:12:51 +0000 (13:12 +0000)]
In do_ast, make sure the interrupts are enabled before calling ast().
We can reach that point with IRQs disabled, and calling ast() with IRQs \7f
disabled can lead to a deadlock.
This should fix the freezes on arm64 under load.
kevans [Mon, 26 Feb 2018 04:55:08 +0000 (04:55 +0000)]
style.lua(9): Add some additional notes about naming and commas
camelCase tends to be preferred for function identifiers, while
internal_underscores are preferred for variable identifiers. This convention
makes it a little bit easier to eyeball whether variable/function usage is
correct.
The optional commas for final table values are preferred to reduce chances
for error.
kevans [Mon, 26 Feb 2018 04:33:05 +0000 (04:33 +0000)]
Add MAINTAINERS note for lualoader (stand/lua, specifically)
While it's a work in progress, at least, I would like a chance to review any
lua that goes into the tree for lualoader. I am also willing to help others
get started writing features or fixing any bugs wandered across.
kevans [Mon, 26 Feb 2018 04:12:54 +0000 (04:12 +0000)]
lualoader: screen argument fixes
screen was also guilty of not-so-great argument names, but it was also
guilty of handling color sequences on its own. Change those bits to using
the color module instead.
As a side note, between color and screen, I'm not 100% sure that returning
the color_value is the right thing to do if we won't generate the escape
sequences. This should be re-evaluated at a later time, and they should
likely return nil instead.
kevans [Mon, 26 Feb 2018 03:46:17 +0000 (03:46 +0000)]
lualoader: A little more general menu cleanup
Instead of a single-letter parameter ('m'), use something a little more
descriptive and meaningful: 'menudef' ("menu definition") -- these functions
expect to be passed a menudef, so call it what it is.
While here, throw an assertion in that we have a handler for the selected
menu item. This is more of a debugging aide so that it's more obvious when
one is testing a menudef that they've added an entry item that we don't
handle.
This is an improvement over the past behavior of ignoring the unknown menu
entry.
imp [Mon, 26 Feb 2018 03:16:53 +0000 (03:16 +0000)]
Take a meat cleaver to defaults/loader.conf
Remove almost all of the _load=XXX options (kept only those relevant
to splash screens, since there were other settings).
Remove the excessively cutesy comment blocks.
Remove excessive comments and replace with similar content
Remove gratuitous blank lines (while leaving some)
We have too many modules to list them all here. There's no purpose in
doing so and it's a giant hassle to maintain. In addition the extra
~500 lines slow this down on small platforms. It slowed it down
so much small platforms forked, which caused other issues...
This is a compromise between those two extremes.
imp [Mon, 26 Feb 2018 03:16:47 +0000 (03:16 +0000)]
Go back to one loader.conf
We really only need one loader.conf. The other loader.conf was created
because the current one took forever to parse in FORTH. That will be
fixed in the next commit.
imp [Mon, 26 Feb 2018 03:16:04 +0000 (03:16 +0000)]
Add NO_OBJ to those directories that don't make anything.
For directories that don't many anything, add NO_OBJ=t just before we
include bsd.init.mk. This prevents them from creating an OBJ
directory. In addition, prevent defs.mk from creating the machine
related links in these cases. They aren't needed and break, at least
on stable, the read-only src tree build.
pkelsey [Mon, 26 Feb 2018 03:03:41 +0000 (03:03 +0000)]
Greatly reduce the number of #ifdefs supporting the TCP_RFC7413 kernel option.
The conditional compilation support is now centralized in
tcp_fastopen.h and tcp_var.h. This doesn't provide the minimum
theoretical code/data footprint when TCP_RFC7413 is disabled, but
nearly all the TFO code should wind up being removed by the optimizer,
the additional footprint in the syncache entries is a single pointer,
and the additional overhead in the tcpcb is at the end of the
structure.
This enables the TCP_RFC7413 kernel option by default in amd64 and
arm64 GENERIC.
pkelsey [Mon, 26 Feb 2018 02:53:22 +0000 (02:53 +0000)]
This is an implementation of the client side of TCP Fast Open (TFO)
[RFC7413]. It also includes a pre-shared key mode of operation in
which the server requires the client to be in possession of a shared
secret in order to successfully open TFO connections with that server.
The names of some existing fastopen sysctls have changed (e.g.,
net.inet.tcp.fastopen.enabled -> net.inet.tcp.fastopen.server_enable).
pkelsey [Mon, 26 Feb 2018 02:43:26 +0000 (02:43 +0000)]
Fix harmless locking bug in tfp_fastopen_check_cookie().
The keylist lock was not being acquired early enough. The only side
effect of this bug is that the effective add time of a new key could
be slightly later than it would have been otherwise, as seen by a TFO
client.
trasz [Sun, 25 Feb 2018 20:15:06 +0000 (20:15 +0000)]
Prevent getty(8) from looping indefinitely if the device node doesn't
exist. This behaviour makes no sense for eg USB serial adapters, or
USB device-side serial templates.
This mostly reverts to pre-r135941 behaviour.
Reviewed by: imp@
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D14198
andrew [Sun, 25 Feb 2018 19:43:00 +0000 (19:43 +0000)]
Teach the Arm pl011 driver to attach to a SBSA uart. This is defined in
the Server Base System Architecture to be a subset of the pl011 r1p5. As
we don't use the removed features it is safe to just attach to the existing
driver as is.
ian [Sun, 25 Feb 2018 18:26:50 +0000 (18:26 +0000)]
Instead of building ofw_iicbus as a separate module, just compile it in to
the iicbus module for FDT-based systems.
The primary motivation for this is that host controller drivers which
declare DRIVER_MODULE(ofw_iicbus, thisdriver, etc, etc) now only need a
single MODULE_DEPEND(thisdriver, ofw_iicbus) for runtime linking to resolve
all the symbols. With ofw_iicbus and iicbus in separate modules, drivers
would need to declare a MODULE_DEPEND() on both, because symbol lookup is
non-recursive through the dependency chain. Requiring a driver to have
MODULE_DEPENDS() on both amounts to requiring the drivers to understand the
kobj inheritence details of how ofw_iicbus is implemented, which seems like
something they shouldn't have to know (and could even change some day).
Also, this is somewhat analogous to how the drivers get built when compiled
into the kernel. You don't have to ask for ofw_iicbus separately, it just
gets built in along with iicbus when option FDT is in effect.
kevans [Sun, 25 Feb 2018 17:02:50 +0000 (17:02 +0000)]
lualoader: Track the menu currently drawn, instead of validity
This cleans up the odd approach to menu drawing. Instead of tracking
validity, we track the menu that was drawn on the screen. Whenever we draw a
menu, we'll set this to that menu.
Anything that invalidates the screen should go ahead and trigger an explicit
redraw, rather than finding a wy to set screen_invalid.
The currently drawn menu is then reset in menu.run as we exit the menu
system, so that dropping to the loader prompt or leaving menu.run() will
just behave as expected without doing redundant work every time we leave a
menu.
kevans [Sun, 25 Feb 2018 16:29:02 +0000 (16:29 +0000)]
lualoader: Invalidate the screen from menu perspective upon mnu exit
In the common case, this will effectively do nothing as the menu will get
redrawn as we leave submenus regardless of whether the screen has been
marked invalid or not
However, upon escape to the loader prompt, one could do either of the
following to re-enter the menu system:
-- Method 1
require('menu').run()
-- Method 2
require('menu').process(menu.default)
With method 1, the menu will get redrawn anyways as we do this before
autoboot checking upon entry. With method 2, however, the menu will not be
redrawn without this invalidation.
Both methods are acceptable for re-entering the menu system, although the
latter method in the local module for processing new and interesting menus
is more expected.
mjg [Sun, 25 Feb 2018 15:16:58 +0000 (15:16 +0000)]
Don't generate data in sysctl_out_proc unless we intend to copy out.
The first call is used to gauge how much spaces is needed. Just computing
the size instead of generating the output allows to not take the proctree
lock.
kp [Sun, 25 Feb 2018 08:56:44 +0000 (08:56 +0000)]
pf: Cope with overly large net.pf.states_hashsize
If the user configures a states_hashsize or source_nodes_hashsize value we may
not have enough memory to allocate this. This used to lock up pf, because these
allocations used M_WAITOK.
Cope with this by attempting the allocation with M_NOWAIT and falling back to
the default sizes (with M_WAITOK) if these fail.
kevans [Sun, 25 Feb 2018 05:14:06 +0000 (05:14 +0000)]
lualoader: Explain deviation from naming guidelines
cli_execute is likely the only exception that we should make, due to it
being a global. We don't really need other globals, so this won't really end
up an epidemic.
kevans [Sun, 25 Feb 2018 05:00:54 +0000 (05:00 +0000)]
lualoader: Pull autoboot handling out into menu.run()
There's no reason for autoboot handling to be mixed in with menu processing.
It is a distinct process that should only be done once when entering the
menu system.
menu.process has been modified to take an initial keypress to process and to
only draw the screen initially if it's been invalidated. The keypress is
kind of a kludge, although it could be argued to be a potentially useful
kludge if there are other processes that may need to feed a keypress into
the menu system.
kevans [Sun, 25 Feb 2018 04:44:45 +0000 (04:44 +0000)]
lualoader: Pull menu redrawing specifics out of menu.process
In general, every menu redraw is going to require a screen clear and cursor
reset. Each redraw also has the potential to invalidate the alias table, so
we move the alias table being used out into a module variable. This allows
third party consumers to also inspect or update the alias table if they need
to.
While here, stop searching the alias table once we've found a match.
kevans [Sun, 25 Feb 2018 04:11:08 +0000 (04:11 +0000)]
lualoader: Clean up menu handling a little bit
This is driven by an urge to separate out the bits that really only need to
happen when the menu system starts up. Key points:
- menu.process now does the bulk of menu handling. It retains autoboot
handling for dubious reasons, and it no longer accepts a 'nil' menu to
process as 'the default'. Its return value is insignificant.
- The MENU_SUBMENU handler now returns nothing. If menu.process has exited,
then we continue processing menu items on the parent menu as expected.
- menu.run is now the entry point of the menu system. It checks whether the
menu should be skipped, processes the default menu, then returns.
markj [Sat, 24 Feb 2018 20:47:22 +0000 (20:47 +0000)]
Restore the pre-r329882 inactive page shortage computation.
With r329882, in the absence of a free page shortage we would only take
len(PQ_INACTIVE)+len(PQ_LAUNDRY) into account when deciding whether to
aggressively scan PQ_ACTIVE. Previously we would also include the
number of free pages in this computation, ensuring that we wouldn't scan
PQ_ACTIVE with plenty of free memory available. The change in behaviour
was most noticeable immediately after booting, when PQ_INACTIVE and
PQ_LAUNDRY are nearly empty.
kevans [Sat, 24 Feb 2018 20:07:39 +0000 (20:07 +0000)]
lualoader: throw out nextboot's usage of standard config processing
It should use the common parser, but it should not be processed like a
standard file. Rewite check_nextboot to read the file in, check whether it
should continue, then parse as needed.
This allows us to throw the recently introduced check_and_halt callback
swiftly out the window.
kevans [Sat, 24 Feb 2018 20:00:31 +0000 (20:00 +0000)]
lualoader: Strip config.parse of its I/O privileges
config.parse is now purely a parser, rather than a whole proccessor. The
standard process for loading a config file has been split out into
config.processFile.
This clears the way for having nextboot read its own config file and decide
there whether it should parse the rest of the file.
kevans [Sat, 24 Feb 2018 19:51:18 +0000 (19:51 +0000)]
lualoader: Split config file I/O out into a separate function
This is step 1 towards revoking config.parse of it I/O privileges. Ideally,
all reading would be done before config.parse and config.parse would just
take text and parse it rather than being charged with the entire process.
dteske [Sat, 24 Feb 2018 17:13:15 +0000 (17:13 +0000)]
Updates and enhancements to io.d to aid DTrace scripting
+ Add dev_type do devinfo_t
+ Add b_cmd to bufinfo_t
+ Add constants for BIO_* and DEVSTAT_TYPE_*
+ Add inline for converting BIO_* int to string
+ Add inline for converting DEVSTAT_TYPE_* int to string
+ Add mask for dev_type & DEVSTAT_TYPE_MASK to string
+ Add mask for dev_type & DEVSTAT_TYPE_IF_MASK to string
Reviewed by: markj
Sponsored by: Smule, Inc.
Differential Revision: https://reviews.freebsd.org/D14396
andrew [Sat, 24 Feb 2018 10:33:31 +0000 (10:33 +0000)]
Correctly set the 16kB page size field in the ITS BASER register. Some
new arm64 hardware, e.g. ThunderX2, seems to use this page size so was
failing to attach as the register value read back was incorrect.
kib [Sat, 24 Feb 2018 10:26:26 +0000 (10:26 +0000)]
Hide all vm/vm_pageout.h content under #ifdef _KERNEL.
There are no parts useful for usermode applications in
vm/vm_pageout.h. Even for the specific applications like fstat and
lsof.
In my opinion, this protection is redundant and instead userspace
should not include the header at all. Since there are apparently
broken third party codebases, give them a bit of slack by providing
transitional period.
Reported by: julian
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
hselasky [Sat, 24 Feb 2018 08:44:51 +0000 (08:44 +0000)]
Fix race when detach is called right after attach in if_axge, that the
network device pointer might be NULL. Wait for any pending tasks to
complete before calling axge_stop().
kevans [Sat, 24 Feb 2018 03:47:04 +0000 (03:47 +0000)]
lualoader: Add comment on trailing space, don't operate on nil
Functionally, the latter error wouldn't necessarily hurt anything. io.write
will just error out as it's not passed a valid file handle. Still, we can do
better than that.
kevans [Sat, 24 Feb 2018 03:43:10 +0000 (03:43 +0000)]
lualoader: Correct test and name
The functionality was correct, but our style guidelines tend to request that
we shy away from using boolean operations in place of explicit comparisons
to nil.
kevans [Sat, 24 Feb 2018 03:35:35 +0000 (03:35 +0000)]
lualoader: Add nextboot support
config.parse now takes an extra callback that is invoked on the full text of
the config file. This callback dictates where we should actually try to
parse this file or not.
For nextboot, we use this to halt parsing if we see 'nextboot_enable="NO"'.
If we don't, parse it and write 'nextboot_enable="NO" ' to it. The same
caveat as with forth still applies- writing is only supported by UFS.
jhibbits [Sat, 24 Feb 2018 01:46:56 +0000 (01:46 +0000)]
Remove platform_cpu_idle() and platform_cpu_idle_wakeup() interfaces
These interfaces were put in place to let QorIQ SoCs dictate CPU idling
semantics, in order to support capabilities such as NAP mode and deep sleep.
However, this never stabilized, and the idling support reverted back to
CPU-level rather than SoC level. Move this code back to cpu.c instead. If
at a later date the lower power modes do come to fruition, it should be done
by overriding the cpu_idle_hook instead of this platform hook.
jkim [Sat, 24 Feb 2018 01:24:57 +0000 (01:24 +0000)]
Partially revert r197863 to reduce diff against i386.
When I wrote the patch, I wanted to remove SYSINIT() usage from amd64 code.
There is no reason to keep the divergence any more because iwasaki merged
most amd64 suspend/resume code to i386 with r235622. Note this also fixed
an enge case reported by royger. [1]
jeff [Fri, 23 Feb 2018 22:51:51 +0000 (22:51 +0000)]
Add a generic Proportional Integral Derivative (PID) controller algorithm and
use it to regulate page daemon output.
This provides much smoother and more responsive page daemon output, anticipating
demand and avoiding pageout stalls by increasing the number of pages to match
the workload. This is a reimplementation of work done by myself and mlaier at
Isilon.
cem [Fri, 23 Feb 2018 20:15:19 +0000 (20:15 +0000)]
Remove unused error return from API that cannot fail
No implementation of fpu_kern_enter() can fail, and it was causing needless
error checking boilerplate and confusion. Change the return code to void to
match reality.
(This trivial change took nine days to land because of the commit hook on
sys/dev/random. Please consider removing the hook or otherwise lowering the
bar -- secteam never seems to have free time to review patches.)