mckusick [Wed, 28 Aug 2013 17:38:05 +0000 (17:38 +0000)]
A performance problem was reported in PR kern/181226:
I have 25TB Dell PERC 6 RAID5 array. When it becomes almost
full (10-20GB free), processes which write data to it start
eating 100% CPU and write speed drops below 1MB/sec (normally
to gives 400MB/sec). The revision at which it first became
apparent was http://svnweb.freebsd.org/changeset/base/249782.
The offending change reserved an area in each cylinder group to
store metadata. The new algorithm attempts to save this area for
metadata and allows its use for non-metadata only after all the
data areas have been exhausted. The size of the reserved area
defaults to half of minfree, so the filesystem reports full before
the data area can completely fill. However, in this report, the
filesystem has had minfree reduced to 1% thus forcing the metadata
area to be used for data. As the filesystem approached full, it
had only metadata areas left to allocate. The result was that
every block allocation had to scan summary data for 30,000 cylinder
groups before falling back to searching up to 30,000 metadata areas.
The fix is to give up on saving the metadata areas once the free
space reserve drops below 2%. The effect of this change is to use
the old algorithm of just accepting the first available block that
we find. Since most filesystems use the default 5% minfree, this
will have no effect on their operation. For those that want to push
to the limit, they will get their crappy block placements quickly.
gavin [Wed, 28 Aug 2013 15:12:15 +0000 (15:12 +0000)]
After writing a kernel core dump into /var/crash, call sync(8).
If we panic again shortly after boot (say, within 30 seconds), any core
dump we wrote out may be lost on reboot. In this situation, we really
want to keep that core file, as it may be the only way to have the issue
resolved. Call sync(8) after writing out the core file and running
crashinfo(8), in the hope that these will not be lost if we panic
again. sync(8) is only called in the case where there is a core dump
to be written out, so won't be called during normal boots.
Discovered by: Trying to debug an IPSEC panic
MFC after: 1 week
ivoras [Wed, 28 Aug 2013 10:06:20 +0000 (10:06 +0000)]
Take a very small step toward the Century of the Anchovy by increasing the
time dirhash entries stay in memory before being considered for eviction to
1 minute.
delphij [Wed, 28 Aug 2013 00:39:47 +0000 (00:39 +0000)]
Previously, both zfs_rename and zfs_link does a check on whether
the passed vnode belongs to the same mount point (v_vfsp or also
known as v_mount in FreeBSD). This check prevents the code from
proceeding further on vnodes that do not belong to ZFS, for
instance, on UFS or NULLFS.
The recent change (merged as r254585) on upstream changes the
check of v_vfsp to instead check the znode's z_zfsvfs. On Illumos
this would work because when the vnode comes from lofs, the
VOP_REALVP() would give the right vnode, this is not true on
FreeBSD where our VOP_REALVP is a no-op, and as such tdvp is
not guaranteed to be a ZFS vnode, and will later trigger a
failed assertion when verifying the vnode.
This changeset modifies our local shims (zfs_freebsd_rename and
zfs_freebsd_link) to check if v_mount matches before proceeding
further.
zeising [Tue, 27 Aug 2013 23:30:02 +0000 (23:30 +0000)]
Really regen after r254962.
This removes the WITH_BSDCONFIG description alltogether, since this option
is removed.
At the same time, fix the WITHOUT_LIBCPLUSPLUS option that had gotten
inverted.
jlh [Tue, 27 Aug 2013 21:20:28 +0000 (21:20 +0000)]
Make the period of each periodic security script configurable.
There are now six additional variables
weekly_status_security_enable
weekly_status_security_inline
weekly_status_security_output
monthly_status_security_enable
monthly_status_security_inline
monthly_status_security_output
alongside their existing daily counterparts. They all have the same
default values.
All other "daily_status_security_${scriptname}_${whatever}"
variables have been renamed to "security_status_${name}_${whatever}".
A compatibility shim has been introduced for the old variable names,
which we will be able to remove in 11.0-RELEASE.
"security_status_${name}_enable" is still a boolean but a new
"security_status_${name}_period" allows to define the period of
each script. The value is one of "daily" (the default for backward
compatibility), "weekly", "monthly" and "NO".
Note that when the security periodic scripts are run directly from
crontab(5) (as opposed to being called by daily or weekly periodic
scripts), they will run unless the test is explicitely disabled with a
"NO", either for in the "_enable" or the "_period" variable.
When the security output is not inlined, the mail subject has been
changed from "$host $arg run output" to "$host $arg $period run output".
For instance:
myfbsd security run output -> myfbsd security daily run output
I don't think this is considered as a stable API, but feel free to
correct me if I'm wrong.
Finally, I will rearrange periodic.conf(5) and default/periodic.conf
to put the security options in their own section. I left them in
place for this commit to make reviewing easier.
andre [Tue, 27 Aug 2013 20:52:02 +0000 (20:52 +0000)]
Pad m_hdr on 32bit architectures to to prevent alignment and padding
problems with the way MLEN, MHLEN, and struct mbuf are set up.
CTASSERT's are provided to detect such issues at compile time in the
future.
The #define MLEN and MHLEN calculation do not take actual compiler-
induced alignment and padding inside the complete struct mbuf into
account. Accordingly appropriate attention is required when changing
members of struct mbuf.
Ideally one would calculate MLEN as (MSIZE - sizeof(((struct mbuf *)0)->m_hdr)
but that doesn't work as the compiler refuses to operate on an as of
yet incomplete structure.
In particular ARM 32bit has more strict alignment requirements which
caused 4 bytes of padding between m_hdr and pkthdr in struct mbuf
because of the 64bit members in pkthdr. This wasn't picked up by MLEN
and MHLEN causing an overflow of the mbuf provided data storage by
overestimating its size.
I386 didn't show this problem because it handles unaligned access just
fine, albeit at a small performance penalty.
On 64bit architectures the struct mbuf layout is 64bit aligned in all
places.
Reported by: Thomas Skibo <ThomasSkibo-at-sbcglobal-dot-net>
Tested by: tuexen, ian, Thomas Skibo (extended patch)
Sponsored by: The FreeBSD Foundation
ken [Tue, 27 Aug 2013 19:47:03 +0000 (19:47 +0000)]
If a drive returns ASC/ASCQ 0x04,0x11 "Logical unit not ready,
notify (enable spinup) required", instead of doing the normal
retries, poll for a change in status.
We will poll every half second for a minute for the status to
change.
Hitachi drives (and likely other SAS drives) return that ASC/ASCQ
when they are waiting to spin up. What it means is that they are
waiting for the SAS expander to send them the SAS
NOTIFY (ENABLE SPINUP) primitive.
That primitive is the mechanism expanders/enclosures use to
sequence drive spinup to avoid overloading power supplies.
kargl [Tue, 27 Aug 2013 19:46:56 +0000 (19:46 +0000)]
* s_erf.c:
. Use integer literal constants instead of double literal constants.
* s_erff.c:
. Use integer literal constants instead of casting double literal
constants to float.
. Update the threshold values from those carried over from erf() to
values appropriate for float.
. New sets of polynomial coefficients for the rational approximations.
These coefficients have little, but positive, effect on the maximum
error in ULP in the four intervals, but do improve the overall
speed of execution.
. Remove redundant GET_FLOAT_WORD(ix,x) as hx already contained the
contents that is packed into ix.
. Update the mask that is used to zero-out lower-order bits in x in
the intervals [1.25, 2.857143] and [2.857143, 12]. In tests on
amd64, this change improves the maximum error in ULP from 6.27739
and 63.8095 to 3.16774 and 2.92095 on these intervals for erffc().
rdivacky [Tue, 27 Aug 2013 18:35:04 +0000 (18:35 +0000)]
Assemble linux32_locore.s and ia32_sigtramp.S with clang integrated assembler.
Support for .code32 and .code64 in llvm was implemented more than 2 years ago.
Tested by: Dan McGregor <dan.mcgregor at usask dot ca>
neel [Tue, 27 Aug 2013 16:50:48 +0000 (16:50 +0000)]
Allow single byte reads of the emulated MSI-X tables. This is not required
by the PCI specification but needed to dump MMIO space from "ddb" in the
guest.
will [Tue, 27 Aug 2013 15:50:26 +0000 (15:50 +0000)]
Make the PAM password strength checking module WARNS=2 safe.
lib/libpam/modules/pam_passwdqc/Makefile:
Bump WARNS to 2.
contrib/pam_modules/pam_passwdqc/pam_passwdqc.c:
Bump _XOPEN_SOURCE and _XOPEN_VERSION from 500 to 600
so that vsnprint() is declared.
Use the two new union types (pam_conv_item_t and
pam_text_item_t) to resolve strict aliasing violations
caused by casts to comply with the pam_get_item() API taking
a "const void **" for all item types. Warnings are
generated for casts that create "type puns" (pointers of
conflicting sized types that are set to access the same
memory location) since these pointers may be used in ways
that violate C's strict aliasing rules. Casts to a new
type must be performed through a union in order to be
compliant, and access must be performed through only one
of the union's data types during the lifetime of the union
instance. Handle strict-aliasing warnings through pointer
assignments, which drastically simplifies this change.
Correct a CLANG "printf-like function with more arguments
than format" error.
adrian [Tue, 27 Aug 2013 14:37:13 +0000 (14:37 +0000)]
Create a new function to complete 802.11 mbuf transmission.
The aim of this function is to eventually be the completion entry point
for all 802.11 encapsulated mbufs. All the wifi drivers end up doing
what is in this function so it's an easy win to turn it into a net80211
method and abstract out this code.
Ideally the drivers will all eventually be modified to queue up completed
mbufs and call this function with all the driver locks not held.
This will allow for some much more interesting software queue handling
in the future (like net80211 based A-MSDU, fast-frames, A-MPDU aggregation
and retransmission.)
will [Tue, 27 Aug 2013 04:42:42 +0000 (04:42 +0000)]
Fix 'make release' on older hosts: use buildworld legacy utilities.
Newer FreeBSD installs require an install(1) that supports the new flags.
This adds ${MAKEOBJDIRPREFIX}${.CURDIR}/tmp/legacy/{bin,usr/{bin,sbin}}
to the PATH while generating an mtree database for 'make release'.
Note that the problem only exists here because mm-mtree.sh generates
its own object tree to avoid mucking with the existing one, which
results in a PATH containing legacy utility dirs that are empty.
bryanv [Tue, 27 Aug 2013 04:05:18 +0000 (04:05 +0000)]
Couple minor if_vmx tweaks
- Use queue size fields from the Tx/Rx queues in various places
instead of (currently the same values) from the softc.
- Fix potential crash in detach if the attached failed to alloc
queue memory.
- Move the VMXNET3_MAX_RX_SEGS define to a better spot.
- Tweak frame size calculation w.r.t. ETHER_ALIGN. This could be
tweaked some more, or removed since it probably doesn't matter
much for x86 (and the x86 class of machines this driver will
be used on).
will [Tue, 27 Aug 2013 04:01:31 +0000 (04:01 +0000)]
Build all ZFS testing & debugging tools with -g.
These programs and everything using libzpool rely on the embedded asserts to
verify the correctness of operations. Given that, the core dumps would be
useless without debug symbols.
gonzo [Tue, 27 Aug 2013 01:40:13 +0000 (01:40 +0000)]
Fixes for compatibility with QEMU:
- Route PCI interrupt for NIC
- Make "no mapping" warning more user-friendly: add device name and mention
that it's IRQ mapping
- Do not overlap ICUs' IO window with PCI devices' IO windows by starting
IO rman at offset 0x100
kib [Tue, 27 Aug 2013 01:31:12 +0000 (01:31 +0000)]
When allocating a pbuf for the cluster write, do not sleep waiting
for the available pbuf when passed vnode is backing md(4). Other i/o
directed to the same md device might already hold pbufs, and then we
could deadlock since only our progress can free a pbuf needed for
wakeup.
will [Mon, 26 Aug 2013 23:48:21 +0000 (23:48 +0000)]
Add the ability to display the default FIB number for a process to the
ps(1) utility, e.g. "ps -O fib".
bin/ps/keyword.c:
Add the "fib" keyword and default its column name to "FIB".
bin/ps/ps.1:
Add "fib" as a supported keyword.
sys/compat/freebsd32/freebsd32.h:
sys/kern/kern_proc.c:
sys/sys/user.h:
Add the default fib number for a process (p->p_fibnum)
to the user land accessible process data of struct kinfo_proc.
Submitted by: Oliver Fromme <olli@fromme.com>, gibbs
dteske [Mon, 26 Aug 2013 23:37:11 +0000 (23:37 +0000)]
Building upon SVN r254237, disable automated activation of alternate layouts
and add support for default underride to $loader_version, acting as a way to
name a release. Release text is not displayed for the aforementioned feature
of alternate display layout (introduced in r254237); however, for all other
layouts (incl. default), the release name is displayed at lower-right.
See version.4th(8) for additional information and/or historical details.
NOTE: Also a minor edit to version.4th(8) while we're here.
joerg [Mon, 26 Aug 2013 21:15:50 +0000 (21:15 +0000)]
Reimplement the FDOPT_NOERROR feature that was kicked out in r134081.
It is needed for fdread(1) in order to be able to recover from CRC
errors in the data field of a floppy sector (by returning the sector
data that failed CRC, rather than inventing dummy data).
When closing the device, clear all transient device options.
jmg [Mon, 26 Aug 2013 18:47:10 +0000 (18:47 +0000)]
none of the drivers in the tree support CDIOCCAPABILITY or CDIOCPITCH..
remove the documentation so people won't get confused and think they
are supported...
raj [Mon, 26 Aug 2013 17:12:30 +0000 (17:12 +0000)]
Introduce superpages support for ARMv6/v7.
Promoting base pages to superpages can increase TLB coverage and allow for
efficient use of page table entries. This development provides FreeBSD/ARM
with superpages management mechanism roughly equivalent to what we have for
i386 and amd64 architectures.
1. Add mechanism for automatic promotion of 4KB page mappings to 1MB section
mappings (and demotion when not needed, respectively).
2. Managed and non-kernel mappings are now superpages-aware.
3. The functionality can be enabled by setting "vm.pmap.sp_enabled" tunable to
a non-zero value (either in loader.conf or by modifying "sp_enabled"
variable in pmap-v6.c file). By default, automatic promotion is currently
disabled.
Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Reviewed by: alc
Sponsored by: The FreeBSD Foundation, Semihalf
raj [Mon, 26 Aug 2013 16:23:54 +0000 (16:23 +0000)]
Provide settings for superpage reservation system on ARM.
This allows for enabling and configuring superpages reservation mechanism in
order to allocate and populate 256 4KB base pages (for the purpose of
promotion to a 1MB superpage).
Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Reviewed by: alc
Sponsored by: The FreeBSD Foundation, Semihalf
markj [Sun, 25 Aug 2013 21:54:41 +0000 (21:54 +0000)]
Implement the ip, tcp, and udp DTrace providers. The probe definitions use
dynamic translation so that their arguments match the definitions for
these providers in Solaris and illumos. Thus, existing scripts for these
providers should work unmodified on FreeBSD.
jilles [Sun, 25 Aug 2013 21:52:04 +0000 (21:52 +0000)]
Fix fcntl F_GETFL F_SETFL for files opened execute-only (O_EXEC).
The FFLAGS and OFLAGS now work correctly also for files opened with O_EXEC.
Except possibly fuse, the other users pass values without O_EXEC set. fuse
appears to assume O_EXEC is handled correctly.
Although F_SETFL may not be commonly used for execute-only file descriptors,
F_GETFL may be useful to find the access mode.
dumbbell [Sun, 25 Aug 2013 18:09:11 +0000 (18:09 +0000)]
vga_pci: Add API to map the Video BIOS
Here are two new functions to map and unmap the Video BIOS:
void * vga_pci_map_bios(device_t dev, size_t *size);
void vga_pci_unmap_bios(device_t dev, void *bios);
The BIOS is either taken from the shadow copy made by the System BIOS at
boot time if the given device was used for the default display (i386,
amd64 and ia64 only), or from the PCI expansion ROM.
Additionally, one can determine if a given device was the default
display at boot time using the following new function:
void vga_pci_unmap_bios(device_t dev, void *bios);
dumbbell [Sun, 25 Aug 2013 15:15:55 +0000 (15:15 +0000)]
drm/ttm: Fix unmap of buffer object
Add a new ttm_bo_release_mmap() function to unmap pages in a
vm_object_t. Pages are freed when the buffer object is later released.
This function is called in ttm_bo_unmap_virtual_locked(), replacing
Linux' unmap_mapping_range(). In particular this is called when a buffer
object is about to be moved, so that its mapping is invalidated.
However, we don't use this function in ttm_bo_vm_dtor(), because the
vm_object_t is already marked as OBJ_DEAD and the pages will be
unmapped.
dumbbell [Sun, 25 Aug 2013 15:00:48 +0000 (15:00 +0000)]
drm/ttm: Make ttm_bo_wait() call uninterruptible in page fault handler
This fixes a crash where a SIGLALRM, heavily used by X.Org, would
interrupt the wait, causing the page fault to fail and the "Xorg"
process to receive a SIGSEGV.
drm/ttm: fix fence locking in ttm_buffer_object_transfer
which leaves behind a might_sleep in atomic context, since the
fence_lock spinlock is held over a kmalloc(GFP_KERNEL) call. The fix
is to revert the above commit and only take the lock where we need it,
around the call to ->sync_obj_ref.
v2: Fixup things noticed by Maarten Lankhorst:
- Brown paper bag locking bug.
- No need for kzalloc if we clear the entire thing on the next line.
- check for bo->sync_obj (totally unlikely race, but still someone
else could have snuck in) and clear fbo->sync_obj if it's cleared
already.
Reported-by: Dave Airlie <airlied@gmail.com> Cc: Jerome Glisse <jglisse@redhat.com> Cc: Maarten Lankhorst <maarten.lankhorst@canonical.com> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> Signed-off-by: Dave Airlie <airlied@redhat.com>
Approved by: kib@
Author: Dave Airlie <airlied@gmail.com>
Date: Wed Jan 16 15:58:34 2013 +1000
ttm: on move memory failure don't leave a node dangling
if we have a move notify callback, when moving fails, we call move notify
the opposite way around, however this ends up with *mem containing the mm_node
from the bo, which means we double free it. This is a follow on to the previous
fix.
Author: Dave Airlie <airlied@gmail.com>
Date: Wed Jan 16 14:25:44 2013 +1000
ttm: don't destroy old mm_node on memcpy failure
When we are using memcpy to move objects around, and we fail to memcpy
due to lack of memory to populate or failure to finish the copy, we don't
want to destroy the mm_node that has been copied into old_copy.
While working on a new kms driver that uses memcpy, if I overallocated bo's
up to the memory limits, and eviction failed, then machine would oops soon
after due to having an active bo with an already freed drm_mm embedded in it,
freeing it a second time didn't end well.
drm/ttm: use ttm_bo_reserve_slowpath_nolru in ttm_eu_reserve_buffers, v2
This requires re-use of the seqno, which increases fairness slightly.
Instead of spinning with a new seqno every time we keep the current one,
but still drop all other reservations we hold. Only when we succeed,
we try to get back our other reservations again.
This should increase fairness slightly as well.
Changes since v1:
- Increase val_seq before calling ttm_bo_reserve_slowpath_nolru and
retrying to take all entries to prevent a race.
With the lru lock no longer required for protecting reservations we
can just do a ttm_bo_reserve_nolru on -EBUSY, and handle all errors
in a single path.
There should no longer be assumptions that reserve will always succeed
with the lru lock held, so we can safely break the whole atomic
reserve/lru thing. As a bonus this fixes most lockdep annotations for
reservations.
projects / FreeBSD / FreeBSD.git / log