ian [Sun, 29 Oct 2017 20:50:03 +0000 (20:50 +0000)]
Improve the performance of the hpet timer in bhyve guests by making the
timer frequency a power of two. This changes the frequency from 10 to
16.7 MHz (2 ^ 24 HZ). Using a power of two avoids roundoff errors when
doing arithmetic in sbintime_t units.
Testing shows this can fix erratic ntpd behavior in guests using the
hpet timer (which is the default for multicore guests).
eadler [Sun, 29 Oct 2017 08:25:31 +0000 (08:25 +0000)]
Update UPDATING
- Remove FreeBSD 4.x of building the kernel.
While it might technically work, it is better to
document the 'correct' way than how to shoot oneself
in the foot
imp [Sun, 29 Oct 2017 05:26:59 +0000 (05:26 +0000)]
Define new EFI variables
Define EFISRC, EFIINC and EFIINCMD. Use them, as well as using other
symbols defined in defs.mk. Prefer <bsd.init.mk> to ../../Makefile.inc
or <src.opts.mk>.
bdrewery [Sun, 29 Oct 2017 01:21:42 +0000 (01:21 +0000)]
xdev: Add missing 'make obj' treewalk for 'make includes'.
It is possible that building headers requires an OBJDIR.
The other phases of xdev have their own 'make obj' calls
where needed, such as inside 'make libraries' itself.
bdrewery [Sun, 29 Oct 2017 01:21:39 +0000 (01:21 +0000)]
lib32/libsoft: Store object files in a more consistent structure.
Having objects in world32 and a sysroot in lib32 was confusing and
inconsistent with the normal build. Now objects are stored in
obj-lib32 (or obj-libsoft) and the sysroot (analagous to WORLDTMP)
is stored in obj-lib32/tmp.
bdrewery [Sun, 29 Oct 2017 01:21:29 +0000 (01:21 +0000)]
Change native-xtools to not install by default; add a native-xtools-install.
Without this the user has to mess with 'make -f Makefile.inc1 ...' to figure
out where the files are installed in the OBJDIR and then they need to copy them
to where they really wanted them. Using DESTDIR may be problematic after
r325001 as well.
The files will be installed to DESTDIR/NXTP where NXTP defaults to /nxb-bin.
bdrewery [Sun, 29 Oct 2017 01:21:19 +0000 (01:21 +0000)]
Fix xdev TARGET/TARGET_ARCH assertion and expand to native-xtools.
The top of Makefile.inc1 requires TARGET/TARGET_ARCH be defined. Just
building 'make xdev' would already set them, so this error was never
triggered. Moving it to Makefile fixes the problem.
bdrewery [Sun, 29 Oct 2017 01:21:09 +0000 (01:21 +0000)]
Rework r254951 to not reach into private OBJDIR areas.
The original change was dealing with the build wanting to run a newer
install(1) that was not yet installed. The solution to look into the private
legacy directory of the existing build conflicts with 2 upcoming features: a
changed OBJDIR format, and splitting the host tools into arch-dependent and
arch-independent directories. Rather than hardcoding and changing the paths in
this script, just let kernel-toolchain do the work, while disabling much of the
meat. With -j15 this finishes in 25 seconds for me and 117 seconds with -j1.
All that is really needed is bootstrap-tools, but the system is not currently
written in a way that all previous dependent steps will have ran. The previous
steps, such as _worldtmp, are being reworked and renamed and so cannot be
relied upon to be right.
Only remove them if the option is enabled and also handle libsoft
by using the proper LIBCOMPAT_OBJTREE. LIBCOMPAT:D will expand
the text after it as a proper glob to the command line if LIBCOMPAT
is defined.
bdrewery [Sun, 29 Oct 2017 01:20:53 +0000 (01:20 +0000)]
Move FAST_DEPEND cleanup hack into its own target and call from _cleanobj.
The _worldtmp target is for setting up WORLDTMP. Nothing between _worldtmp
and _cleanobj will read these files. Move to its own target since it is
so large.
bdrewery [Sun, 29 Oct 2017 01:20:47 +0000 (01:20 +0000)]
Disallow using a colon ':' in source checkout paths.
This leaks into the PATH handling for WORLDTMP and breaks
finding cross-tools. The PATH handling could be fixed to
properly quote but is not worth the effort.
Also allow this sanity check to always run even with 'make -n'.
pfg [Sat, 28 Oct 2017 20:09:34 +0000 (20:09 +0000)]
Fix out-of-bounds read in libc/regex.
The bug is an out-of-bounds read detected with address sanitizer that
happens when 'sp' in p_b_coll_elems() includes NUL byte[s], e.g. if it's
equal to "GS\x00". In that case len will be equal to 4, and the
strncmp(cp->name, sp, len) call will succeed when cp->name is "GS" but the
cp->name[len] == '\0' comparison will cause the read to go out-of-bounds.
Checking the length using strlen() instead eliminates the issue.
The bug was found in LLVM with oss-fuzz:
https://reviews.llvm.org/D39380
MFC after: 1 week
Obtained from: Vlad Tsyrklevich through posting on openbsd-tech
ian [Sat, 28 Oct 2017 20:03:29 +0000 (20:03 +0000)]
Split the hardware type enum and the hw feature flags bits into separate
fields in the softc; they're ORed together in the ofw_compat_data.
I already caught myself doing 'sc->fectype == <enum val>' without masking
out the feature bits in one place, and that's sure to happen again.
Glomming them together is convenient for storing them in the ofw_compat_data
array, but there's no reason to keep them together in the softc.
ian [Sat, 28 Oct 2017 19:34:08 +0000 (19:34 +0000)]
Use the 16-bit receive shift feature in ffec hardware that supports it.
When available, enabling this feature causes the hardware to write data
to the receive buffer starting at a 16-bit offset from the start address.
This eliminates the need to copy the data after receiving to re-align
the protocol headers to a 32-bit boundary.
oshogbo [Sat, 28 Oct 2017 19:23:57 +0000 (19:23 +0000)]
Introduce caspermocks.
The idea behinds mocks is that we don't need to ifdef a lot of code in
tools itself but those defines are hidden in the casper library.
Right now the mocks are implemented as define/inlines functions.
There was a very long discussion how this should be implemented.
This approach has some advantages like we don't need to link to any additional
libraries. Unfortunately there are also some disadvantages for example it is
easy to get library out of sync between two versions of functions or that we
need extra define to compile program with casper support.
This isn't an ideal solution but it's good enough for now and should simplify
capsicumizing programs. This also doesn't close us any other ways to do those
mocks and this should evolve in time.
ian [Sat, 28 Oct 2017 19:08:06 +0000 (19:08 +0000)]
Support up to 3 IRQs in the ffec driver.
Newer hardware splits the interrupts onto 3 different irq lines, but the
docs barely mention that there are multiple interrupts, and do not detail
how they're split up. The code now supports 1-3 irqs, and uses the same
interrupt service routine to handle all of them.
I modified the submitted changes to use bus_alloc_resources() instead of
using loops to allocate each irq separately. Thus, blame any bugs on me (I
can't actually test on imx7 hardware).
ian [Sat, 28 Oct 2017 18:56:27 +0000 (18:56 +0000)]
Add a #define RESOURCE_SPEC_END. Similar to DEVMETHOD_END and
KOBJMETHOD_END, this is to serve as the end marker in an array of
resource_spec structures.
ian [Sat, 28 Oct 2017 17:30:49 +0000 (17:30 +0000)]
Avoid AXI bus issues due to a MAC reset on imx6sx and imx7.
When the FEC is connected to the AXI bus (indicated by AVB flag), a
MAC reset while a bus transaction is pending can hang the bus.
Instead of resetting, turn off the ENABLE bit, which allows the
hardware to complete any in-progress transfers (appending a bad CRC
to any partial packet) and release the AXI bus. This could probably
be done unconditionally for all hardware variants, but that hasn't
been tested.
ian [Sat, 28 Oct 2017 17:06:13 +0000 (17:06 +0000)]
Add FECFLAG_AVB variant flag to support new features on imx7.
This flag is analogous to the Linux driver FEC_QUIRK_HAS_AVB. It
indicates an FEC with support for Audio Video Bridging (AVB). This
indicator is used for various other parts in the Linux driver
(drivers/net/ethernet/freescale/fec_main.c).
Use it to customize the receive/transmit buffer alignment. The receive
buffer alignment increased to 64-bytes on the i.MX 6SoloX and i.MX
7Dual. There are no hard alignment restrictions for transmit buffers on
these chips.
Fix the ffec_softc::fectype type to provide enough storage for the
feature flags.
markj [Sat, 28 Oct 2017 02:48:37 +0000 (02:48 +0000)]
Remove workqueue items after updating the workqueue tail pointer.
When QUEUE_MACRO_DEBUG_TRASH is configured, the queue linkage fields
are trashed upon removal of the item, so be sure to only read them before
removing the item.
gonzo [Fri, 27 Oct 2017 21:22:38 +0000 (21:22 +0000)]
Fix MAC address detection regression introduced by r324184
To accomodate all variaties of Pi DTS files floating around
we look for MAC address property either in DTS node for
USB ethernet (if it exists) or at predefined path
".../usb/hub/ethernet".
After r324184 smsc_fdt_find_eth_node started to return node
with compatibility string "usb424,ec00" as an eth node.
In imported GNU dts files this node still does not have
MAC address related property, and therefor following check for
"mac-address" and "local-mac-address" fails.
To make this logic more robust do not just search for the node
but also make sure it has required property, so if node with
accepted compatibility string exists but doesn't have the
property we fall back to looking for hardoded path mentioned above.
tuexen [Fri, 27 Oct 2017 19:27:05 +0000 (19:27 +0000)]
Fix parsing error when processing cmsg in SCTP send calls. Thei bug is
related to a signed/unsigned mismatch.
This should most likely fix the issue in sctp_sosend reported by
Dmitry Vyukov on the freebsd-hackers mailing list and found by
running syzkaller.
markj [Fri, 27 Oct 2017 17:05:14 +0000 (17:05 +0000)]
Fix a lock leak in g_mirror_destroy().
g_mirror_destroy() is supposed to unlock the softc before indicating
success, but it wasn't doing so if the caller raced with another
thread destroying the mirror.
emaste [Fri, 27 Oct 2017 16:23:45 +0000 (16:23 +0000)]
libdtrace: replace "DOODAD" with more descriptive string
Previously some unimplemented libdtrace routines printed the function,
file and line number, followed by "DOODAD." That is not particularly
informative, so replace it with a message reporting the actual issue.
jhb [Fri, 27 Oct 2017 14:57:14 +0000 (14:57 +0000)]
Rework pass through changes in r305485 to be safer.
Specifically, devices that do not support PCI-e FLR and were not
gracefully shutdown by the guest OS could continue to issue DMA
requests after the VM was terminated. The changes in r305485 meant
that those DMA requests were completed against the host's memory which
could result in random memory corruption. Instead, leave ppt devices
that are not attached to a VM disabled in the IOMMU and only restore
the devices to the host domain if the ppt(4) driver is detached from a
device.
As an added safety belt, disable busmastering for a pass-through device
when before adding it to the host domain during ppt(4) detach.
https://www.illumos.org/issues/640
du(1), df(1m), ls(1), and swap(1m) all include a copy (it appears literally
copied) of the 'number_to_scaled_string' function in their source. This should
be moved to a shared library and all 4 commands should use this instead.
FreeBSD note: of all libcmdutils functionality ZFS (and other illumos
contrib code) currently uses only nicenum() function (which is similar
to humanize_number but has some formatting differences). For this
reason I decided to not port the whole library. As a result, nicenum.c
from libcmdutils is compiled into libzfs and libzpool. This is a bit
ugly, but works. If one day we are forced to create libillumos, then
the file should be moved to that library.
Reviewed by: Sebastian Wiedenroth <wiedi@frubar.net>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Yuri Pankov <yuripv@gmx.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Jason King <jason.brian.king@gmail.com>
jhb [Fri, 27 Oct 2017 03:16:19 +0000 (03:16 +0000)]
Discard the correct thread event reported for a ptrace stop.
When multiple threads wish to report a tracing event to a debugger,
both threads call ptracestop() and one thread will win the race to be
the reporting thread (p->p_xthread). The debugger uses PT_LWPINFO
with the process ID to determine which thread / LWP is reporting an
event and the details of that event. This event is cleared as a side
effect of the subsequent ptrace event that resumed the process
(PT_CONTINUE, PT_STEP, etc.). However, ptrace() was clearing the
event identified by the LWP ID passed to the resume request even if
that wasn't the 'p_xthread'. This could result in clearing an event
that had not yet been observed by the debugger and leaving the
existing event for 'p_thread' pending so that it was reported a second
time.
Specifically, if the debugger stopped due to a software breakpoint in
one thread, but then switched to another thread that was used to
resume (e.g. if the user switched to a different thread and issued a
step), the resume request (PT_STEP) cleared a pending event (if any)
for the thread being stepped. However, the process immediately
stopped and the first thread reported it's breakpoint event a second
time. The debugger decremented the PC for "both" breakpoint events
which resulted in the PC now pointing into the middle of an
instruction (on x86) and a SIGILL fault when the process was resumed a
second time.
To fix, always clear the pending event for 'p_xthread' when resuming a
process. ptrace() still honors the requested LWP ID when enabling
single-stepping (PT_STEP) or setting a different PC (PT_CONTINUE).
imp [Thu, 26 Oct 2017 22:53:49 +0000 (22:53 +0000)]
Always send STANDBY IMMEDIATE when shutting down
To save SMART data and for a drive to understand that it's been nicely
shutdown, we need to send a STANDBY IMMEDIATE. Modify adaspindown to
use a local CCB on the stack. When we're panicing, used
xpt_polled_action rather than cam_periph_runccb so that we can SEND
IMMEDIATE after we've shutdown the scheduler.
imp [Thu, 26 Oct 2017 22:52:51 +0000 (22:52 +0000)]
Various IPMI watchdog timer improvements
o Make hw.ipmi.on a tuneable
o Changes to keep shutdown from hanging indefinitately after the wd
would normally have been disabled.
o Add support for setting pretimeout (which fires an interrupt
some time before the actual watchdog expires)
o Allow refinement of the actions to take when the watchdog expires
o Allow special startup timeout to keep us from hanging in boot
before watchdogd is started, but after we've loaded the kernel.
kp [Thu, 26 Oct 2017 20:53:56 +0000 (20:53 +0000)]
pf tests: destroy jails before destroying interfaces
When cleaning up we must destroy the jails before we destroy the interfaces.
Otherwise we might try to destroy interfaces that belong to a jail, which won't
work and fail to completely clean up.
asomers [Thu, 26 Oct 2017 19:45:15 +0000 (19:45 +0000)]
Fix aio_suspend in 32-bit emulation
An off-by-one error has been present since the system call was first present
in 185878. It additionally became a memory corruption bug after change
324941. The failure is actually revealed by our existing AIO tests.
However, apparently nobody's been running those in 32-bit emulation mode.
Reported by: Coverity, cem
CID: 1382114
MFC after: 18 days
X-MFC-With: 324941
Sponsored by: Spectra Logic Corp
jilles [Thu, 26 Oct 2017 18:32:04 +0000 (18:32 +0000)]
libnv: Fix strict-aliasing violation with cookie
In rS323851, some casts were adjusted in calls to nvlist_next() and
nvlist_get_pararr() in order to make scan-build happy. I think these changes
just confused scan-build into not reporting the strict-aliasing violation.
For example, nvlist_xdescriptors() is causing nvlist_next() to write to its
local variable nvp of type nvpair_t * using the lvalue *cookiep of type
void *, which is not allowed. Given the APIs of nvlist_next(),
nvlist_get_parent() and nvlist_get_pararr(), one possible fix is to create a
local void *cookie in nvlist_xdescriptors() and other places, and to convert
the value to nvpair_t * when necessary. This patch implements that fix.
https://www.illumos.org/issues/640
du(1), df(1m), ls(1), and swap(1m) all include a copy (it appears literally
copied) of the 'number_to_scaled_string' function in their source. This should
be moved to a shared library and all 4 commands should use this instead.
Reviewed by: Sebastian Wiedenroth <wiedi@frubar.net>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Yuri Pankov <yuripv@gmx.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Jason King <jason.brian.king@gmail.com>
asomers [Thu, 26 Oct 2017 15:28:18 +0000 (15:28 +0000)]
zfsd should be able to online an L2ARC that disappears and returns
Previously, this didn't work because L2ARC devices' labels don't contain
pool GUIDs. Modify zfsd so that the pool GUID won't be required:
lib/libdevdctl/guid.h
Change INVALID_GUID from a uint64_t constant to a function that
returns an invalid Guid object. Remove the void constructor.
Nothing uses it, and it violates RAII.
cddl/usr.sbin/zfsd/case_file.h
cddl/usr.sbin/zfsd/case_file.cc
Allow CaseFile::Find to match a CaseFile based on Vdev GUID alone.
In CaseFile::ReEvaluate, attempt to online devices even if the newly
arrived device has no pool GUID.
cddl/usr.sbin/zfsd/vdev_iterator.cc
Iterate through a pool's cache devices as well as its regular
devices.
des [Thu, 26 Oct 2017 13:23:13 +0000 (13:23 +0000)]
If the user-provided password exceeds the maximum password length, don't
bother passing it to crypt(). It won't succeed and may allow an attacker
to confirm that the user exists.
truckman [Thu, 26 Oct 2017 10:11:35 +0000 (10:11 +0000)]
Fix Dummynet AQM packet marking function ecn_mark() and fq_codel /
fq_pie schedulers packet classification functions in layer2 (bridge mode).
Dummynet AQM packet marking function ecn_mark() and fq_codel/fq_pie
schedulers packet classification functions (fq_codel_classify_flow()
and fq_pie_classify_flow()) assume mbuf is pointing at L3 (IP)
packet. However, this assumption is incorrect if ipfw/dummynet is
used to manage layer2 traffic (bridge mode) since mbuf will point
at L2 frame. This patch solves this problem by identifying the
source of the frame/packet (L2 or L3) and adding ETHER_HDR_LEN
offset when converting an mbuf pointer to ip pointer if the traffic
is from layer2. More specifically, in dummynet packet tagging
function, tag_mbuf(), iphdr_off is set to ETHER_HDR_LEN if the
traffic is from layer2 and set to zero otherwise. Whenever an access
to IP header is required, mtodo(m, dn_tag_get(m)->iphdr_off) is
used instead of mtod(m, struct ip *) to correctly convert mbuf
pointer to ip pointer in both L2 and L3 traffic.
bdrewery [Wed, 25 Oct 2017 21:46:36 +0000 (21:46 +0000)]
Fix native-xtools build to use a proper sysroot.
This takes longer but should reliably produce working binaries.
The old version linked against system libraries and headers
which would be a problem if building a native-xtools against
a newer source than the host system. With a proper sysroot made
first this is not a problem.
This also allows:
- NXBDIRS to be built in parallel
- NXBDIRS to be installed to NXBDESTDIR in parallel
- SYSTEM_COMPILER logic to work again. Note that because this change
is adding a sysroot phase the compiler may be built up to twice now.
The first is the "cross-compiler" even though it is for the native
architecture, but it is still built to ensure the latest compiler
is used to generate the binaries, unless SYSTEM_COMPILER allows
/usr/bin/cc to be used. Then the target compiler is built
which is actually a cross-compiler since it runs on native host
but generates TARGET.TARGET_ARCH binaries.
Note this also changes the path used for the OBJDIR. It used to use
/usr/obj/target.target_arch/nxb/<srcdir> for objects and
/usr/obj/target.target_arch/nxb-bin for installed files, but now uses
/usr/obj/nxb/target.target_arch/<srcdir> for objects and
/usr/obj/nxb/target.target_arch/<srcdir>/nxb-bin for installed files.
- NXBDESTDIR can be specified for where to install or queried with
`make -f Makefile.inc1 TARGET=... TARGET_ARCH=... -V NXBDESTDIR`
This could potentially be improved to reuse an existing sysroot. The
problem is with building the SUBDIR_OVERRIDE list it needs to use a
different OBJDIR since it is building all statically. We don't want to
pollute the existing 'buildworld' OBJDIR and cause confusion on the next
build. Using a separate OBJDIR for the 'everything' phase mostly works
except for some things like linking in INTERNALLIBS that exist in the
other OBJDIR.
bdrewery [Wed, 25 Oct 2017 21:46:33 +0000 (21:46 +0000)]
native-xtools: Override proper NXBDESTDIR.
The new native-xtools uses 'make toolchain' so overriding DESTDIR
as a make argument may interfere with WORLDTMP handling.
The target also does a 'mkdir -p ${NXBDESTDIR}/usr', so we should
be modifying that rather than DESTDIR.
Note this causes the native-xtools binaries to be installed in
NANO_WORLDDIR/usr NANO_WORLDDIR/bin rather than NANO_WORLDDIR/nxb-bin/usr
and NANO_WORLDDIR/nxb-bin/bin. This was the case before this change
as well.
kp [Wed, 25 Oct 2017 19:21:48 +0000 (19:21 +0000)]
Evaluate packet size after the firewall had its chance in the ip6 fast path
Defer the packet size check until after the firewall has had a look at it. This
means that the firewall now has the opportunity to (re-)fragment an oversized
packet.
This mirrors what the slow path does.
asomers [Wed, 25 Oct 2017 16:01:19 +0000 (16:01 +0000)]
Fix zpool_read_all_labels when vfs.aio.enable_unsafe=0
Previously, zpool_read_all_labels was trying to do 256KB reads, which are
greater than the default MAXPHYS and therefore must go through the slow,
unsafe AIO path. Shrink these reads to 112KB so they can use the safe, fast
AIO path instead.
projects / FreeBSD / FreeBSD.git / log