Fix ahci(4) driver attach to controller with 32 ports.
Incorrect sign expansion in variables that supposed to be a bit fields
caused infinite loop. Fixing this allows system properly detect maximal
possible 32 devices configured on AHCI HBA of BHyVe. That case did not
happen in a wild before due to lack of hardware AHCI HBAs with 32 ports.
fcntl(2): Document interrupt/restart for file locks.
Since r302216, thread suspension causes advisory file locks to restart
(instead of continuing to wait) and for a long time SA_RESTART has
affected advisory file locks. These are both not compliant to POSIX.1.
To clarify that restarting means something, add a paragraph about fair
queuing. Note that the network lock manager does not implement fair
queuing.
Reviewed by: kib (previous version)
Approved by: re (gjb)
Change the type of the map entry's next_read field from a vm_pindex_t to a
vm_offset_t. (This field is used to detect sequential access to the virtual
address range represented by the map entry.) There are three reasons to
make this change. First, a vm_offset_t is smaller on 32-bit architectures.
Consequently, a struct vm_map_entry is now smaller on 32-bit architectures.
Second, a vm_offset_t can be written atomically, whereas it may not be
possible to write a vm_pindex_t atomically on a 32-bit architecture. Third,
using a vm_pindex_t makes the next_read field dependent on which object in
the shadow chain is being read from.
Sometimes the software loses the race when appending more descriptors to
the tx ring and the tx queue stops.
This commit detects this condition and restart the tx queue whenever it stall.
Tested by: sobomax@, Keith White <kwhite@site.uottawa.ca>,
Paul Mather <paul@gromit.dlib.vt.edu>
Sponsored by: Rubicon Communications (Netgate)
Approved by: re (kib)
Autotune the number of pages set aside for UMA startup based on the number
of CPUs present. On amd64 this unbreaks the boot for systems with 92 or
more CPUs; the limit will vary on other systems depending on the size of
their uma_zone and uma_cache structures.
The major consumer of pages during UMA startup is the 19 zone structures
which are set up before UMA has bootstrapped itself sufficiently to use
the rest of the available memory: UMA Slabs, UMA Hash, 4 / 6 / 8 / 12 /
16 / 32 / 64 / 128 / 256 Bucket, vmem btag, VM OBJECT, RADIX NODE, MAP,
KMAP ENTRY, MAP ENTRY, VMSPACE, and fakepg. If the zone structures occupy
more than one page, they will not share pages and the number of pages
currently needed for startup is 19 * pages_per_zone + N, where N is the
number of pages used for allocating other structures; on amd64 N = 3 at
present (2 pages are allocated for UMA Kegs, and one page for UMA Hash).
This patch adds a new definition UMA_BOOT_PAGES_ZONES, currently set to 32,
and if a zone structure does not fit into a single page sets boot_pages to
UMA_BOOT_PAGES_ZONES * pages_per_zone instead of UMA_BOOT_PAGES (which
remains at 64). Consequently this patch has no effect on systems where the
zone structure fits into 2 or fewer pages (on amd64, 59 or fewer CPUs), but
increases boot_pages sufficiently on systems where the large number of CPUs
makes this structure larger. It seems safe to assume that systems with 60+
CPUs can afford to set aside an additional 128kB of memory per 32 CPUs.
The vm.boot_pages tunable continues to override this computation, but is
unlikely to be necessary in the future.
Tested on: EC2 x1.32xlarge
Relnotes: FreeBSD can now boot on 92+ CPU systems without requiring
vm.boot_pages to be manually adjusted.
Reviewed by: jeff, alc, adrian
Approved by: re (kib)
Add new unmount(2) flag, MNT_NONBUSY, to check whether there are
any open vnodes before proceeding. Make autounmound(8) use this flag.
Without it, even an unsuccessfull unmount causes filesystem flush,
which interferes with normal operation.
Due to ARC initial configuration not being done and kmem information
not being available we need to blindly set zfs_arc_max and zfs_arc_min
when configured via the tunable.
This fixes vfs.zfs.arc_(min|max) configuration via loader.conf broken by
r302265.
andrew [Wed, 6 Jul 2016 16:20:10 +0000 (16:20 +0000)]
Remove the old pre-INTRNG arm64 interrupt framework. GENERIC was switched
to INTRNG in r301565 with the old code no longer being built by default with
no reports of issues on any supported hardware.
Approved by: re (gjb)
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
The TCPPCAP debugging feature caches recently-used mbufs for use in
debugging TCP connections. This commit provides a mechanism to free those
mbufs when the system is under memory pressure.
Because this will result in lost debugging information, the behavior is
controllable by a sysctl. The default setting is to free the mbufs.
- Replace all CTASSERT macro instances with static_assert's.
- Remove the WRAPPED_CTASSERT macro; it's now an unnecessary obfuscation.
- Localize all static_assert's to the structures being tested.
- Sort some headers per-style(9).
Replace a number of conflations of mp_ncpus and mp_maxid with either
mp_maxid or CPU_FOREACH() as appropriate. This fixes a number of places in
the kernel that assumed CPU IDs are dense in [0, mp_ncpus) and would try,
for example, to run tasks on CPUs that did not exist or to allocate too
few buffers on systems with sparse CPU IDs in which there are holes in the
range and mp_maxid > mp_ncpus. Such circumstances generally occur on
systems with SMT, but on which SMT is disabled. This patch restores system
operation at least on POWER8 systems configured in this way.
There are a number of other places in the kernel with potential problems
in these situations, but where sparse CPU IDs are not currently known
to occur, mostly in the ARM machine-dependent code. These will be fixed
in a follow-up commit after the stable/11 branch.
PR: kern/210106
Reviewed by: jhb
Approved by: re (glebius)
Only set the ipfilter running state to 'not running' if we are
doing the teardown. ipf_destroy_all() may free ipfmain in case
of ipf_dynamic_softc being true, thus we are avoiding a possible
memory modified after free as well.
Reported by: Coverity
Coverity CID: 1357320
Approved by: re (hrs)
MFC after: 10 days
Fix gcc build errors with SSE 4.2 detection and gcc warnings
- Remove -Wunused-but-set-variable's (`len`, etc).
- Replace clang-specific tests in sse42_supported(..) with generic,
FreeBSD-supported CPU feature tests, using macros and functions
from machine/cpufunc.h and machine/specialreg.h . The previous method
for determining SSE4.2 availability was only compatible with clang.
- Sort #includes per style(9).
- Put parentheses around bitwise OR'ed values in the `FIELD_COPY(..)` and
`FIELD_REPLACE(..)` macros to mute warning from gcc 4.2.1.
- Remove -Wunused-but-set-variable's (`setup_addr`, `status_addr`).
Add `WRAPPED_CTASSERT` macro by annotating CTASSERTs with __unused
to deal with -Wunused-local-typedefs warnings from gcc 4.8+.
All other compilers (clang, etc) use CTASSERT as-is. A more generic
solution for this issue will be proposed after ^/stable/11 is forked.
Consolidate all CTASSERTs under one block instead of inlining them in
functions.
Remove incorrect attributes from posix_memalign(3) declaration.
Both __alloc_align and __alloc_size can't be used when the function
returns a pointer to memory. This fixes breakage when building with
clang 3.4:
In file included from /usr/src/svn/usr.sbin/bhyve/atkbdc.c:40:
/usr/include/stdlib.h:176:6: error: '__alloc_size__' attribute only
applies to functions that return a pointer [-Werror,-Wignored-attributes]
cem [Tue, 5 Jul 2016 20:53:32 +0000 (20:53 +0000)]
ioat(4): Block asynchronous work during HW reset
Fix the race between ioat_reset_hw and ioat_process_events.
HW reset isn't protected by a lock because it can sleep for a long time
(40.1 ms). This resulted in a race where we would process bogus parts
of the descriptor ring as if it had completed. This looked like
duplicate completions on old events, if your ring had looped at least
once.
Block callout and interrupt work while reset runs so the completion end
of things does not observe indeterminate state and process invalid parts
of the ring.
Start the channel with a manually implemented ioat_null() to keep other
submitters quiesced while we wait for the channel to start (100 us).
r295605 may have made the race between ioat_reset_hw and
ioat_process_events wider, but I believe it already existed before that
revision. ioat_process_events can be invoked by two asynchronous
sources: callout (softclock) and device interrupt. Those could race
each other, to the same effect.
The paradigm of a callout is that it has three consequent states:
not scheduled -> scheduled -> running -> not scheduled. The API and the
manual page assume that, some comments in the code assume that, and looks
like some contributors to the code also did. The problem is that this
paradigm isn't true. A callout can be scheduled and running at the same
time, which makes API description ambigouous. In such case callout_stop()
family of functions/macros should return 1 and 0 at the same time, since it
successfully unscheduled future callout but the current one is running.
Before this change we returned 1 in such a case, with an exception that
if running callout was migrating we returned 0, unless CS_MIGRBLOCK was
specified.
With this change, we now return 0 in case if future callout was unscheduled,
but another one is still in action, indicating to API users that resources
are not yet safe to be freed.
However, the sleepqueue code relies on getting 1 return code in that case,
and there already was CS_MIGRBLOCK flag, that covered one of the edge cases.
In the new return path we will also use this flag, to keep sleepqueue safe.
Since the flag CS_MIGRBLOCK doesn't block migration and now isn't limited to
migration edge case, rename it to CS_EXECUTING.
This change fixes panics on a high loaded TCP server.
Compile in the kassert_panic() function with INVARIANT_SUPPORT
option, not INVARIANTS. The function is required if we want
to load in a module that is compiled with INVARIANTS.
Ensure that spinlock sections are balanced even after a panic.
vpanic() uses spinlock_enter() to disable interrupts before dumping core.
However, when the scheduler is stopped and INVARIANTS is not configured,
thread_lock() does not acquire a spinlock section, while thread_unlock()
releases one. This can result in interrupts staying enabled while the
kernel dumps core, complicating post-mortem analysis of the crash.
Call audit hooks to capture vnode attributes for three file-descriptor
method implementations: fstat(2), close(2), and poll(2). This change
synchronises auditing here with similar auditing for VFS-specific system
calls such as stat(2) that audit more complete vnode information.
Clarify the vnode_destroy_vobject() logic handling for already terminated
objects.
Assert that there is no new waiters for the already terminated objects.
Old waiters should have been notified by the termination calling
vnode_pager_dealloc() (old/new are with regard of the lock acquisition
interval).
Only clear the vp->v_object for the case of already terminated object,
since other branches call vnode_pager_dealloc(), which should clear
the pointer. Assert this.
Tested by: pho
Reviewed by: alc
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Approved by: re (gjb)
Fix a regression introduced on revision r271909, when using argument -g
or several hops we have segmentation fault because we overwrite the same
structure to store information for host and gateway.
Unbreak the LBC driver, broken with the large RMan and 36-bit physical address changes.
Remove the use of fdt_data_to_res(), and instead construct the resources
manually. Additionally, avoid the 32-bit size limitation of fdt_data_get(), by
building physical addresses manually from the lbc ranges property.
cxgbe(4): Changes to the CPL-handler registration mechanism and code
related to "shared" CPLs.
a) Combine t4_set_tcb_field and t4_set_tcb_field_rpl into a single
function. Allow callers to direct the response to any iq. Tidy up
set_ulp_mode_iscsi while there to use names from t4_tcb.h instead of
magic constants.
b) Remove all CPL handler tables from struct adapter. This reduces its
size by around 2KB. All handlers are now registered at MOD_LOAD instead
of attach or some kind of initialization/activation. The registration
functions do not need an adapter parameter any more.
c) Add per-iq handlers to deal with CPLs whose destination cannot be
determined solely from the opcode. There are 2 such CPLs in use right
now: SET_TCB_RPL and L2T_WRITE_RPL. The base driver continues to send
filter and L2T_WRITEs over the mgmtq and solicits the reply on fwq.
t4_tom (including the DDP code) now uses the port's ctrlq to send
L2T_WRITEs and SET_TCB_FIELDs and solicits the reply on an ofld_rxq.
fwq and ofld_rxq have different handlers that know what kind of tid to
expect in the reply. Update t4_write_l2e and callers to to support any
wrq/iq combination.
Fix a race condition between the main thread in aqm_pie_cleanup() and the
callout thread that can cause a kernel panic. Always do the final cleanup
in the callout thread by passing a separate callout function for that task
to callout_reset_sbt().
Protect the ref_count decrement in the callout with DN_BH_WLOCK(). All
other ref_count manipulation is protected with this lock.
There is still a tiny window between ref_count reaching zero and the end
of the callout function where it is unsafe to unload the module. Fixing
this would require the use of callout_drain(), but this can't be done
because dummynet holds a mutex and callout_drain() might sleep.
Remove the callout_pending(), callout_active(), and callout_deactivate()
calls from calculate_drop_prob(). They are not needed because this callout
uses callout_init_mtx().
Submitted by: Rasool Al-Saadi <ralsaadi@swin.edu.au>
Approved by: re (gjb)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D6928
Import bhyve_graphics into CURRENT. Thanks to all who tested
this on the branch.
Original commit message:
Initial bhyve native graphics support.
This adds emulations for a raw framebuffer device, PS2 keyboard/mouse,
XHCI USB controller and a USB tablet.
A simple VNC server is provided for keyboard/mouse input, and graphics
output.
A VGA emulation is included, but is currently disconnected until an
additional bhyve change to block out VGA memory is committed.
Credits:
- raw framebuffer, VNC server, XHCI controller, USB bus/device emulation
and UEFI f/w support by Leon Dang
- VGA, console/g, initial VNC server by tychon@
- PS2 keyboard/mouse jointly done by tychon@ and Leon Dang
- hypervisor framebuffer mem support by neel@
Tested by: Michael Dexter, in a number of revisions of this code.
With the appropriate UEFI image, FreeBSD, Windows and Linux guests can
installed and run in graphics mode using the UEFI/GOP framebuffer.
For all locales with variants:
- if no ambiguity on the locale (only one variant) just use the regular name
- if ambiguity, pick one as default and append @<variant> to the others
respecting POSIX
As a result:
- All the 3 components locales added recently are renamed to the usual 2
components version for all but sr_RS.UTF-8
- Set sr_RS.UTF-8 to the cyrillic variant
- Add sr_RS.UTF-8@latin
- Remove the symlinks aliases they were created to represent the 2 components
version as aliasas and are now useless
- Update the OptionalObsoleteFiles.inc and ObsoleteFiles.inc to reflect those
changes
Provide helper macros to detect 'non-silent SBDRY' state and to
calculate appropriate return value for stops. Simplify the code by
using them.
Fix typo in sig_suspend_threads(). The thread which sleep must be
aborted is td2. (*)
In issignal(), when handling stopping signal for thread in
TD_SBDRY_INTR state, do not stop, this is wrong and fires assert.
This is yet another place where execution should be forced out of
SBDRY-protected region. For such case, return -1 from issignal() and
translate it to corresponding error code in sleepq_catch_signals().
Assert that other consumers of cursig() are not affected by the new
return value. (*)
Micro-optimize, mostly VFS and VOP methods, by avoiding calling the
functions when SIGDEFERSTOP_NOP non-change is requested. (**)
Reported and tested by: pho (*)
Requested by: bde (**)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Approved by: re (gjb)
Output the diffs to standard error when comparing the expected vs the
obtained output from lastcomm instead of just printing out a summary, e.g.
"they differed".
This will make failures with results more apparent when running kyua debug,
kyua report-html, etc.
Differential Revision: https://reviews.freebsd.org/D7079 (as part of a larger diff)
MFC after: 1 week
Approved by: re (gjb)
Reviewed by: cem
Sponsored by: EMC / Isilon Storage Division
Fix race for incrementally rebuilding VERSION_MAP.
The dependency is needed in PROG_FULL since only the build of PROG_FULL
is using the LDFLAGS and depending on VERSION_MAP. This was not a problem
with MK_DEBUG_FILES==no since it only builds PROG.
This should probably be using bsd.lib.mk instead [1]
Remove racy assert. The thread which changes vnode usecount from 0 to 1
does it under the vnode interlock, but the interlock is not owned by the
asserting thread. As result, we might read increased use counter but also
still see VI_OWEINACT.
In collaboration with: nwhitehorn
Hardware donated by: IBM LTC
Sponsored by: The FreeBSD Foundation (kib)
Approved by: re (gjb)
Add more machines to the bsdinstall workaround blacklist
Apply the LenovoFix (fake partition in pMBR is 2nd rather than 1st)
and GPT Active fix (mark the fake pMBR partition active) to respective
hardware that is confirmed to require such to boot correct.
Submitted by: Sam Fourman Jr. <sfourman@gmail.com> (ThinkPad W520)
Submitted by: Matthias Apitz <guru@unixarea.de> (Latitude E6330)
Approved by: re (gjb)
Sponsored by: ScaleEngine Inc.
cxgbe(4): Avoid a NULL dereference while dumping the L2 table. Entries
used by switching filters that rewrite L2 information do not have any
associated ifnet.
Clean up some FDT-related code in the PowerPC bootloader, improving error
checking and robustness. Prevents errors and crashes in FDT commands on
PowerMac G5 systems.
When a process knote was attached to the process which is already exiting,
the knote is activated immediately. If the exit1() later activates
knotes, such knote is attempted to be activated second time. Detect
the condition by zeroed kn_ptr.p_proc pointer, and avoid excessive
activation.
Before r302235, such knotes were removed from the knlist immediately
upon activation.
Reported by: truckman
Sponsored by: The FreeBSD Foundation
Approved by: re (gjb)
Fix detection of USB device disconnects in USB host mode when the USB
device is connected directly to the USB port of the DWC OTG, in this
case a RPI-zero.
PR: 210695
Approved by: re (gjb)
MFC after: 1 week
bz [Thu, 30 Jun 2016 19:32:45 +0000 (19:32 +0000)]
In case of the global eventhandler make sure the current VNET
is still operational before doing any work; otherwise we might
run into, e.g., destroyed locks.
PR: 210724
Reported by: olevole olevole.ru
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Obtained from: projects/vnet
Approved by: re (gjb)
bz [Thu, 30 Jun 2016 15:01:07 +0000 (15:01 +0000)]
Virtualise ipfilter.
Split initializzation an teardown into module (global state) and VNET
(per virtual network stack) parts. Virtualise global state, which is
not "const".
Cleanup eventhandlers, so that we can make use of the passed in argument
to get the vnet state from the ifp; disable the "cloner" event as it is
too early, has no state, and can fire before initialisation (see comment
in the source).
Handle the dynamic sysctls specially. The problem is that "ipmain"
is the virtualized struct, but the fields used for the sysctls are
hanging off memory allocated and attached to the virtualized "ipmain"
thus standard VNET macros and sysctl handling do not work.
We still say it is VNET sysctls to get the proper protection checks
in the VIMAGE case; to solve the problem of accessing the right bit
of memory hanging of each per-VNET ipmain, we use a dedicated handler
function wrapping around sysctl_ipf_int() undoing the base calculation
from kern_sysctl.c and then adding the passed-in offset into the right
struct depending on handler. A bit of a mess exposing VNET-internals
this way but the only way to keep the code without having to massively
restructure ipf internals.
Approved by: re (hrs)
Sponsored by: The FreeBSD Foundation
Obtained from: projects/vnet
MFC after: 2 weeks
Reviewed by: cy
Differential Revision: https://reviews.freebsd.org/D7000
mav [Thu, 30 Jun 2016 14:55:49 +0000 (14:55 +0000)]
Revert r299454 and r299448.
Those changes were found confusing FreeBSD libc ACL code, that doesn't
differentiate ACL for directories and files, and report ACLs for all
directories created after those patches as non-trivial. On the other
side these changes were considered wrong from POSIX and NFSv4 points of
view. Until further investigation done upstream, revert those changes
locally in preparation for FreeBSD 11.0 release.
mm [Thu, 30 Jun 2016 08:51:50 +0000 (08:51 +0000)]
MFV r302264:
Sync libarchive with vendor, bugfixes for tests:
- fix tests on filesystems without birthtime support, e.g. UFS1 (1)
- vendor issue #729: avoid use of C99 for-scope declarations in
test_write_format_gnutar_filenames.c
tuexen [Thu, 30 Jun 2016 06:06:35 +0000 (06:06 +0000)]
This patch fixes two bugs related to the setting of the I-Bit
for SCTP DATA and I-DATA chunks.
* For fragmented user messages, set the I-Bit only on the last
fragment.
* When using explicit EOR mode, set the I-Bit on the last
fragment, whenever SCTP_SACK_IMMEDIATELY was set in snd_flags
for any of the send() calls.
wma [Thu, 30 Jun 2016 05:18:37 +0000 (05:18 +0000)]
ARM, ARM64: Workaround for buf_ring reordering
This patch offers a workaround to buf_ring reordering
visible on armv7 and armv8. This is supposed to be
removed once new buf_ring implementation is integrated
into the tree.
bz [Thu, 30 Jun 2016 01:33:14 +0000 (01:33 +0000)]
Move the ipfw_log_bpf() calls from global module initialisation to
per-VNET initialisation and virtualise the interface cloning to
allow a dedicated ipfw log interface per VNET.
Approved by: re (gjb)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
cperciva [Wed, 29 Jun 2016 23:56:24 +0000 (23:56 +0000)]
Enable indirect segment I/Os by default in the Xen blkfront driver when
running on EC2. Due to improvements in EC2, the performance penalty which
was present on some EC2 instances no longer exists, and enabling this
feature now consistently yields ~20% higher throughput with equal or lower
latency.
Reverts: r286063
Approved by: re (gjb)
MFC after: 2 weeks
Relnotes: Improved disk throughput on EC2
gonzo [Wed, 29 Jun 2016 23:33:44 +0000 (23:33 +0000)]
Fixed FreeBSD/mips MALTA support for QEMU
Recource management functions in GT PCI controller driver
treated memory/IO resources as KSEG1 addresses, later during
activation these values would be increased by KSEG1 base again
rendering the address invalid and causing "bus error" trap.
Actual logic was converted to use real physical addresses,
so mapping takes place only during activation.
Submitted by: Aleksandr Rybalko <ray@FreeBSD.org>
Approved by: re (gjb)
bdrewery [Wed, 29 Jun 2016 22:54:01 +0000 (22:54 +0000)]
Flush every line when using meta mode and no -j or with -B.
Otherwise the output is buffered and it appears that make is stuck on something
long-running. This problem is not present with -j as it uses different
code that was already flushing.
sobomax [Wed, 29 Jun 2016 18:19:05 +0000 (18:19 +0000)]
1.Improve handling around last compressed block of the file, which is
necessary because CLOOP format lacks explicit EOF or length, so that
in the presence of padding or when the CLOOP is put onto a larger
partition upper level provider size may be larger. Bound amount
of extra data that we might touch to the max length of the compressed
block and detect zero-padding in the last cluster, which when
sector is all-zero might cause us to emit bogus I/O error after
decompression of that fails. To not make code any more complicated
that it needs to be deal with it in lazy-manner, i.e. when we
first access that specific cluster.
This change also fixes stupid mistake in the LZMA code, inherited
from geom_lzma, which does not share length of the output buffer
buffer with the decompression routine, so that in the presence
of corrupted or purposedly tailored data may easily cause heap
overflow and kernel memory corruption.
Beef up validation of the CLOOP TOC by checking that lengths of
all but the last compressed clusters match upper limit set by
the decompressor and improve some error diagnostic output while
I am here.
2.Add kern.geom.uzip.attach_to tunable to artifically limit
attaching uzip to certain devices in the dev tree only.
For example the following only makes us attaching to the
GPT labels:
kern.geom.uzip.attach_to="gpt/*"
3.Add kern.geom.uzip.noattach_to, which does opposite to the (2)
above, i.e. prevents geom_uzip from tasting / attaching to
providers matching some pattern. By default we don't attach
to our own kind, i.e. kern.geom.uzip.noattach_to="*.uzip".
It saves us quite some CPU cycles, esp on low-end embedded
systems.
Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D7013
projects / FreeBSD / FreeBSD.git / log