Split raw reading/programming into smaller chunks to avoid allocating too
big chunk of kernel memory. Validate size of data. Add error handling to
avoid calling copyout() when data has not been read correctly.
Andriy Gapon [Wed, 20 Nov 2013 10:41:10 +0000 (10:41 +0000)]
4101 metaslab_debug should allow for fine-grained control
4102 space_maps should store more information about themselves
4103 space map object blocksize should be increased
4104 ::spa_space no longer works
4105 removing a mirrored log device results in a leaked object
4106 asynchronously load metaslab
Vm map code performs clipping when map entry covers region which is
larger than the operational region. If the op region size is zero,
clipping would create a zero-sized map entry. The result is that vm
map splay starts behaving inconsistently, sometimes returning
zero-sized entry, sometimes the next (or previous) entry.
One step further, it could result in e.g. vm_map_wire() setting
MAP_ENTRY_IN_TRANSITION on the zero-sized entry, but failing to clear
it in the done part. The vm_map_delete() than hangs forever waiting
for the flag removal.
Verify for zero-length requests and act as if it is always successfull
without performing any action on the address space.
Revert back to use int for the page counts. In vn_io_fault(), the i/o
is chunked to pieces limited by integer io_hold_cnt tunable, while
vm_fault_quick_hold_pages() takes integer max_count as the upper bound.
Rearrange the checks to correctly handle overflowing address arithmetic.
Justin Hibbits [Wed, 20 Nov 2013 01:42:29 +0000 (01:42 +0000)]
Use 'int' to store the return value of getopt(), rather than char.
On some architectures (powerpc), char is unsigned by default, which means
comparisons against -1 always fail, so the programs get stuck in an
infinite loop.
Zbigniew Bodek [Tue, 19 Nov 2013 23:37:50 +0000 (23:37 +0000)]
Apply access flags for managed and unmanaged pages properly on ARMv6/v7
When entering a mapping via pmap_enter() unmanaged pages ought to be
naturally excluded from the "modified" and "referenced" emulation.
RW permission should be granted implicitly when requested,
otherwise unmanaged page will not recover from the permission fault
since there will be no PV entry to indicate that the page can be written.
In addition, only managed pages that participate in "modified"
emulation need to be marked as "dirty" and "writeable" when entered
with RW permissions. Likewise with "referenced" flag for managed pages.
Unmanaged ones however should not be marked as such.
Zbigniew Bodek [Tue, 19 Nov 2013 23:31:39 +0000 (23:31 +0000)]
Avoid clearing EXEC permission bit when setting the page RW on ARMv6/v7
When emulating modified bit the executable attribute was cleared by
mistake when calling pmap_set_prot(). This was not a problem before
changes to ref/mod emulation since all the pages were created RW basing
on the "prot" argument in pmap_enter(). Now however not all pages are RW
and the RW permission can be cleared in the process.
Add "resize" verb to gmirror(8) and such functionality to geom_mirror(4).
Now it is easy to expand the size of the mirror when all its components
are replaced. Also add g_resize method to geom_mirror class. It will write
updated metadata to new last sector, when parent provider is resized.
Ian Lepore [Tue, 19 Nov 2013 22:14:35 +0000 (22:14 +0000)]
Bugfixes... the host capabilties from FDT data are stored in host.caps, not
host.host_ocr, examine the correct field when setting up the hardware. Also,
the offset for the capabilties register should be 0x140, not 0x240.
Submitted by: Ilya Bakulin <ilya@bakulin.de>
Pointy hat to: me
Andriy Gapon [Tue, 19 Nov 2013 18:43:47 +0000 (18:43 +0000)]
zfs page_busy: fix the boundaries of the cleared range
This is a fix for a regression introduced in r246293.
vm_page_clear_dirty expects the range to have DEV_BSIZE aligned boundaries,
otherwise it extends them. Thus it can happen that the whole page is
marked clean while actually having some small dirty region(s).
This commit makes the range properly aligned and ensures that only
the clean data is marked as such.
It would interesting to evaluate how much benefit clearing with DEV_BSIZE
granularity produces. Perhaps instead we should clear the whole page
when it is completely overwritten and don't bother clearing any bits
if only a portion a page is written.
Reported by: George Hartzell <hartzell@alerce.com>,
Richard Todd <rmtodd@servalan.servalan.com>
Tested by: George Hartzell <hartzell@alerce.com>,
Reviewed by: kib
MFC after: 5 days
Andriy Gapon [Tue, 19 Nov 2013 18:35:38 +0000 (18:35 +0000)]
fsx: add an option to randomly call msync(MS_INVALIDATE)
This call should be a sufficiently close approximation of what happens
when a filesystem is unmounted and remounted. To be more specific, it
should test that the data that was in the page cache is the same data
that ends up on a stable storage or in a filesystem's internal cache,
if any.
This will catch the cases where a page with modified data is marked as
a clean page for whatever reason.
While there, make logging of the special events (open+close before
plus invalidation now) more generic and slightly better than the previous
hack.
Andriy Gapon [Tue, 19 Nov 2013 18:35:01 +0000 (18:35 +0000)]
fsx: new option to disable msync(MS_SYNC) after each write via mmaped region
This option should be useful for testing if a filesystem uses the
unified buffer / page cache.
Or, if filesystem's emulation of the unified cache works as expected.
This should be the case for e.g. ZFS.
Dimitry Andric [Tue, 19 Nov 2013 17:53:19 +0000 (17:53 +0000)]
Pull in r191896 from upstream llvm trunk:
CaptureTracking: Plug a loophole in the "too many uses" heuristic.
The heuristic was added to avoid spending too much compile time in a
specially crafted test case (PR17461, PR16474) with many uses on a
select or bitcast instruction can still trigger the slow case. Add a
check for that case.
This only affects compile time, don't have a good way to test it.
This fixes the excessive compile time spent on a specific file of the
graphics/rawtherapee port.
Bryan Drewery [Tue, 19 Nov 2013 15:35:26 +0000 (15:35 +0000)]
Support SNI in libfetch
SNI is Server Name Indentification which is a protocol for TLS that
indicates the host that is being connected to at the start of the
handshake. It allows to use Virtual Hosts on HTTPS.
Alexander Motin [Tue, 19 Nov 2013 11:19:07 +0000 (11:19 +0000)]
Reenable vfs.zfs.zio.use_uma for amd64, disabled at r209261.
On machines with seveal CPUs and enough RAM this can easily twice improve
ZFS performance or twice reduce CPU usage. It was disabled three years
ago due to memory and KVA exhaustion reports, but our VM subsystem got
improved a lot since that time, hopefully enough to make another try.
Alexander Motin [Tue, 19 Nov 2013 10:51:46 +0000 (10:51 +0000)]
Implement mechanism to safely but slowly purge UMA per-CPU caches.
This is a last resort for very low memory condition in case other measures
to free memory were ineffective. Sequentially cycle through all CPUs and
extract per-CPU cache buckets into zone cache from where they can be freed.
Alexander Motin [Tue, 19 Nov 2013 10:17:10 +0000 (10:17 +0000)]
Grow UMA zone bucket size also on lock congestion during item free.
Lock congestion is the same, whether it happens on alloc or free, so
handle it equally. Now that we have back pressure, there is no problem
to grow buckets a bit faster. Any way growth is much slower then in 9.x.
Alexander Motin [Tue, 19 Nov 2013 10:10:44 +0000 (10:10 +0000)]
Add two new UMA bucket zones to store 3 and 9 items per bucket.
These new buckets make bucket size self-tuning more soft and precise.
Without them there are buckets for 1, 5, 13, 29, ... items. While at
bigger sizes difference about 2x is fine, at smallest ones it is 5x and
2.6x respectively. New buckets make that line look like 1, 3, 5, 9, 13,
29, reducing jumps between steps, making algorithm work softer, allocating
and freeing memory in better fitting chunks. Otherwise there is quite a
big gap between allocating 128K and 5x128K of RAM at once.
Alexander Motin [Tue, 19 Nov 2013 10:05:53 +0000 (10:05 +0000)]
Implement soft pressure on UMA cache bucket sizes.
Every time system detects low memory condition decrease bucket sizes for
each zone by one item. As result, higher memory pressure will push to
smaller bucket sizes and so smaller per-CPU caches and so more efficient
memory use.
Before this change there was no force to oppose buckets growth as result
of practically inevitable zone lock conflicts, and after some run time
per-CPU caches could consume enough RAM to kill the system.
Mark Johnston [Tue, 19 Nov 2013 00:37:53 +0000 (00:37 +0000)]
Import the axge(4) driver for the ASIX AX88178A and AX88179 USB Ethernet
adapters. Both devices support Gigabit Ethernet and USB 2.0, and the AX88179
supports USB 3.0. The driver was written by kevlo@ and lwhsu@, with a few
bug fixes from me.
Allow ethernet drivers to pass in packets connected via the nextpkt pointer.
Handling packets in this way allows drivers to amortize work during packet reception.
Tijl Coosemans [Mon, 18 Nov 2013 18:14:23 +0000 (18:14 +0000)]
Bug fixes in iconv(3) UTF-7 support.
- Add ' to the list of directly encoded characters and * to the list of
optionally directly encoded characters as per RFC 2152.
- In _citrus_UTF7_mbtoutf16 on end of input when the next output character
has only been partially decoded, save a copy of the buffer of input
characters (not just its length). On the next call with more input
characters this buffer is reprocessed together with the new input to
form a fully decoded output character.
- At the end of a base64 encoded sequence fully discard '-' (BASE64_OUT)
by decrementing psenc->chlen and i. This is needed to make room in
psenc->ch (input buffer) in case the next input character starts a new
base64 encoded sequence. And also, if this is the end of input and no
output character can be returned, this brings the encoder in the initial
state as indicated by _citrus_UTF7_stdenc_get_state_desc_generic which
is used by the caller to distinguish between no output and partial
output.
- In _citrus_UTF7_mbrtowc_priv pass the s parameter (input pointer)
directly to _citrus_UTF7_mbtoutf16 instead of a copy (s0). This way s
is updated correctly in case of errors.
- In _citrus_UTF7_mbrtowc_priv when called with psenc->surrogate set
(previous call did not have enough input), retrieve the previously
decoded UTF-16 character from (psenc->cache >> psenc->bits) instead of
(psenc->cache >> 2).
Alan Somers [Mon, 18 Nov 2013 16:51:56 +0000 (16:51 +0000)]
opensolaris/uts/common/dtrace/fasttrap.c
Fix several problems that can cause panics on kldload and kldunload.
* kproc_create(fasttrap_pid_cleanup_cb, ...) gets called before
fasttrap_provs.fth_table gets allocated. This can lead to a panic
on module load, because fasttrap_pid_cleanup_cb references
fasttrap_provs.fth_table. Move kproc_create down after the point
that fasttrap_provs.fth_table gets allocated, and modify the error
handling accordingly.
* dtrace_fasttrap_{fork,exec,exit} weren't getting NULLed until
after fasttrap_provs.fth_table got freed. That caused panics on
module unload because fasttrap_exec_exit calls
fasttrap_provider_retire, which references
fasttrap_provs.fth_table. NULL those function pointers earlier.
* There wasn't any code to destroy the
fasttrap_{tpoints,provs,procs}.fth_table mutexes on module unload,
leading to a resource leak when WITNESS is enabled. Destroy those
mutexes during fasttrap_unload().
Glen Barber [Mon, 18 Nov 2013 16:25:56 +0000 (16:25 +0000)]
Add the 'dvd1.iso' target. This mimics the 'release.iso' target,
with the additional step of fetching packages for inclusion on the
dvd image.
The 'pkg-stage' target is used to run 'scripts/pkg-stage.sh' if
the '${TARGET}/pkg-stage.conf' configuration file exists (currently
only amd64 and i386).
Allow dvd1.iso to be skipped if NODVD=1.
MFC after: 3 days
X-MFC-With: r258305, r258307, r258308, r258309
Sponsored by: The FreeBSD Foundation
Glen Barber [Mon, 18 Nov 2013 16:04:04 +0000 (16:04 +0000)]
Unconditionally copy the build host /etc/resolv.conf into
the chroot directory, since hostname resolution may be
needed in the case of building a dvd image (with packages)
and also setting 'NOPORTS=1'.
MFC after: 3 days
X-MFC-With: r258305, r258307
Sponsored by: The FreeBSD Foundation
Glen Barber [Mon, 18 Nov 2013 15:22:55 +0000 (15:22 +0000)]
Add a script and configuration files to fetch pre-built packages
from pkg.FreeBSD.org for inclusion on release medium (dvd1.iso).
The script sources ${.CURDIR}/${TARGET}/pkg-stage.conf, which sets
several environment variables, such as the pkg(8) ABI, PACKAGESITE,
PKG_DBDIR, and PKG_CACHEDIR. PKG_CACHEDIR is set to the directory
on the release medium used by bsdconfig(8) (/packages/${ABI}). ABI
is determined by output of 'make -C /usr/src/release -V REVISION'.
See pkg.conf(5) for descripton on other variables set here.
The list of packages to include are set within the configuration
file.
The script and configuration files are intended to be run by the
'make dvd' target within the release directory, and assume the
release is built within a chroot environment (such as by using
release.sh).
Julio Merino [Mon, 18 Nov 2013 12:57:46 +0000 (12:57 +0000)]
Add some sample test programs.
This change adds some sample test cases to share/examples/tests/
demonstrating the basic usage of the atf and plain interfaces.
These test programs are fully-functional and are installed as part
of the test suite, which guarantees that the sample code remains
correct. However, they currently mostly serve as a placeholder for
additional examples and may be incomplete (depending on how you
look at them). I will see what else can be useful while working on
documentation.
As a bonus, the addition of these tests exercise the *.test.mk files,
one of which (plain.test.mk) was not yet in use, and also demonstrates
that it's possible to mix different kinds of test programs into the
same test suite.
Julio Merino [Mon, 18 Nov 2013 12:44:05 +0000 (12:44 +0000)]
Remove registration of C++ test programs into PROGS.
C++ programs need to be added to PROGS_CXX, not PROGS, and the code was
actually doing both. Just keep the registration into PROGS_CXX to
prevent possible obscure build problems.
Mark Johnston [Mon, 18 Nov 2013 03:24:50 +0000 (03:24 +0000)]
The fasttrap ioctl used to create probes takes a variable-sized argument.
It was not being correctly copied into the kernel on FreeBSD, and as a
result, probes with multiple probe sites were not being created properly.
To fix this, change the ioctl definition so that the fasttrap ioctl handler
is responsible for copying in userland data.
Julio Merino [Sun, 17 Nov 2013 23:19:06 +0000 (23:19 +0000)]
Update notes for imports of atf.
This is because the atf vendor branch now includes a verbatim copy of
the distfile sources. As a result, the list of files to-be-removed from
the contrib/ directory is now more aggressive (and different) and the
upgrade notes now only describe stuff that is specific to the atf import
and is not documented in the Subversion Primer.
Julio Merino [Sun, 17 Nov 2013 23:12:55 +0000 (23:12 +0000)]
Fix the build of some ATF tests.
When building various programs from a single Makefile, program-specific
variables are of the form <VAR>.<PROG>, not <VAR>_<PROG>. Fix this
obvious typo to fix the build when WITH_TESTS=yes.
I am not sure how this ever worked before given that manual inspection
of bsd.progs.mk clearly shows that the expected character between the
two components is a dot and not an underscore... but I suspect the
changes in r258095 exposed this oddity.
Peter Wemm [Sun, 17 Nov 2013 22:52:17 +0000 (22:52 +0000)]
Attempt to move the POSIX iconv* symbols out of runtime linker space.
FreeBSD systems usually implemented this as a third party module and
our implementation hasn't played as nicely with the old way as it could
have.
To that end:
* Rename the iconv* symbols in libc.so.7 to have a __bsd_ prefix.
* Provide .symver compatability with existing 10.x+ binaries that
referenced the iconv symbols. All existing binaries should work.
* Like on Linux/glibc systems, add a libc_nonshared.a to the ldscript
at /usr/lib/libc.so.
* Move the "iconv*" wrapper symbols to libc_nonshared.a
This should solve the runtime ambiguity about which symbols resolve
to where. If you compile against the iconv in libc, your runtime
dependencies will be unambiguous.
Old 9.x libraries and binaries will always resolve against their
libiconv.so.3 like they did on 9.x. They won't resolve against libc.
Old 10.x binaries will be satisified by the .symver helpers.
This should allow ports to selectively compile against the libiconv
port if needed and it should behave without ambiguity now.
Jilles Tjoelker [Sun, 17 Nov 2013 22:31:23 +0000 (22:31 +0000)]
Fix siginfo_t.si_status for wait6/waitid/SIGCHLD.
Per POSIX, si_status should contain the value passed to exit() for
si_code==CLD_EXITED and the signal number for other si_code. This was
incorrect for CLD_EXITED and CLD_DUMPED.
This is still not fully POSIX-compliant (Austin group issue #594 says that
the full value passed to exit() shall be returned via si_status, not just
the low 8 bits) but is sufficient for a si_status-related test in libnih
(upstart, Debian/kFreeBSD).
Justin Hibbits [Sun, 17 Nov 2013 20:29:33 +0000 (20:29 +0000)]
Add a sysctl to allow disabling resetting the OF syscons.
On some machines (G5 with lots of RAM), entering OF sometimes causes the
machine to hang. Once the machine is booted, currently the only entry point
into OF is through resetting the framebuffer on mode switch on these machines.
Disabling this allows the machine to stay up at the expense of less usable
consoles after X is started.
Nathan Whitehorn [Sun, 17 Nov 2013 19:01:13 +0000 (19:01 +0000)]
Use #address-cells and #size-cells here too instead of guessing. There is
some comment I wrote about these values "lying" in the negative diff, which
referes to an earlier misunderstanding about which node to read them from.
This gets at least the PPC64 kernel booting in the mac99 system model in
QEMU after bypassing the MacIO ATA driver, which apparently still has
problems.
Devin Teske [Sun, 17 Nov 2013 18:12:17 +0000 (18:12 +0000)]
Add a try-include word (which acts the same as "include") and use it to
conditionally include (but ignore failures) /boot/loader.rc.local and
/boot/menu.rc.local -- to make customizing the menu easier.
Devin Teske [Sun, 17 Nov 2013 17:48:55 +0000 (17:48 +0000)]
Always shutdown the media when we're exiting the packages module (prevents
errors on re-entry for physical media). Also, while we're here, stop
ejecting the CDROM when we're done with it (but leave the functions for
later use so that we could perhaps -- from the installer standpoint -- use
it to eject the media after an install).
Devin Teske [Sun, 17 Nov 2013 17:42:15 +0000 (17:42 +0000)]
Improve debugging with f_eval_catch() introduced by SVN r257784 and also
fix a bug where "pkg update" was not getting the value of PACKAGESITE.
NB: PACKAGESITE needs to be explicitly exported in support of children.
Devin Teske [Sun, 17 Nov 2013 17:31:07 +0000 (17:31 +0000)]
Move function name declaration to top of function (where it is closest to
the value it needs to be), s/fname/funcname/g, and move function name usage
to within printf format string.
drm: Support DRM_CAP_TIMESTAMP_MONOTONIC capability
This fixes DPMS with KDE and radeonkms. Without this, the display would
freeze when the monitor is put into sleep state, and only resumes after
several dozens of minutes once the monitor is powered on again.
Gleb Smirnoff [Sun, 17 Nov 2013 15:14:07 +0000 (15:14 +0000)]
In r257692 I intentionally deleted code that handled P2P interfaces
with equal addresses on both sides. It appeared that OpenVPN uses
such configutations.
Nathan Whitehorn [Sun, 17 Nov 2013 15:12:03 +0000 (15:12 +0000)]
Unify handling of illegal instruction faults between AIM and Book-E. This
allows FPU emulation on AIM as well as providing support for the mfpvr
and lwsync instructions from userland on e500 cores. lwsync, in particular,
is required for many C++ programs to work correctly.
Nathan Whitehorn [Sun, 17 Nov 2013 14:44:22 +0000 (14:44 +0000)]
Split the function of the PCB_FPU flags into two: PCB_FPU now indicates that
the actual FPU is enabled, while PCB_FPREGS indicates that the FPU state
structure in the PCB is valid. This separation reflects the situation on
FPU-less systems in which the FP state is used by the emulator but we don't
actually want to try to turn on the non-existant FPU.
Use this flag to save and restore FP regs properly on both AIM and Book-E.
As a side effect, this sets up hard-FP and Altivec on Book-E CPUs with such
abilities except for a trap handler to call enable_fpu()/enable_altivec().
Peter Wemm [Sun, 17 Nov 2013 06:50:41 +0000 (06:50 +0000)]
Alias WCHAR_T to UCS-4-INTERNAL. WCHAR_T is the internal encoding
of a wchar_t type - which is 32 bit on FreeBSD. This matches observed
behavior on a libiconv machine.
Alexander Motin [Sun, 17 Nov 2013 05:38:54 +0000 (05:38 +0000)]
In addition to r258220 allow shrinking in "automatic" mode if there is
already valid metadata found at the new location. This should allow easy
transparent recovery if first resize was done by mistake.
While there, unify metadata write code and fix minor memory leak.
Nathan Whitehorn [Sun, 17 Nov 2013 05:03:15 +0000 (05:03 +0000)]
Make single precision floating point arithmetic actually work -- I think
it never did -- and fix an obvious missing line. Floating point emulation
on Book-E still needs some work but this gets it basically functional on
soft-FPU systems (hard FPU for Book-E is not yet implemented).
Nathan Whitehorn [Sun, 17 Nov 2013 02:05:20 +0000 (02:05 +0000)]
There is no reason Book-E needs to save XER and CTR on context switches.
They aren't Book-E specific registers to begin with and, even if they were,
are defined volatile by the ABI.
Nathan Whitehorn [Sun, 17 Nov 2013 02:03:36 +0000 (02:03 +0000)]
Move CCSR discovery into the platform module, while simultaneously making
it more flexible about how the CCSR range is found. With this change, the
stock MPC85XX will boot on a Routerboard 800.