markj [Thu, 7 Aug 2014 18:36:47 +0000 (18:36 +0000)]
MFC r265456, r265578:
Add a postinit debugger hook to rtld. This will be used by dtrace(1) to halt
the victim process before its entry point is called, at which point probes
and DOF data are registered with the kernel. The r_debug_state hook cannot
be used for this purpose, as it is called before the program's init routines
are invoked and in particular before DOF data is registered (via drti.o).
mckusick [Wed, 6 Aug 2014 23:33:16 +0000 (23:33 +0000)]
MFC of r269303:
When restoring a UFS dump onto a ZFS filesystem, an assertion in
restore was failing because ZFS was reporting a blocksize that was
not a multiple of 1024. Replace restore's failed assertion with
code that writes restored files in a blocksize that works for
restore (a multiple of 1024) despite being non-optimal for ZFS.
ache [Wed, 6 Aug 2014 10:33:43 +0000 (10:33 +0000)]
MFC: r268997
For "a"-mode files and rewind/fseek + fwrite combination return meaningful
value now, like Apple does, but avoid their __sflush physical write
performance degradation as much as possible.
marius [Tue, 5 Aug 2014 16:04:22 +0000 (16:04 +0000)]
MFC: r260457
The changes in r233781 attempted to make logging during a machine check
exception more readable. In practice they prevented all logging during
a machine check exception on at least some systems. Specifically, when
an uncorrected ECC error is detected in a DIMM on a Nehalem/Westmere
class machine, all CPUs receive a machine check exception, but only
CPUs on the same package as the memory controller for the erroring DIMM
log an error. The CPUs on the other package would complete the scan of
their machine check banks and panic before the first set of CPUs could
log an error. The end result was a clearer display during the panic
(no interleaved messages), but a crashdump without any useful info about
the error that occurred.
To handle this case, make all CPUs spin in the machine check handler
once they have completed their scan of their machine check banks until
at least one machine check error is logged. I tried using a DELAY()
instead so that the CPUs would not potentially hang forever, but that
was not reliable in testing.
While here, don't clear MCIP from MSR_MCG_STATUS before invoking panic.
Only clear it if the machine check handler does not panic and returns
to the interrupted thread.
Intel desktop Haswell CPUs may report benign corrected parity errors (see
HSD131 erratum in [1]) at a considerable rate. So filter these (default),
unless logging is enabled. Unfortunately, there really is no better way to
reasonably implement suppressing these errors than to just skipping them
in mca_log(). Given that they are reported for bank 0, they'd need to be
masked in MSR_MC0_CTL. However, P6 family processors require that register
to be set to either all 0s or all 1s, disabling way more than the one error
in question when using all 0s there. Alternatively, it could be masked for
the corresponding CMCI, but that still wouldn't keep the periodic scanner
from detecting these spurious errors. Apart from that, register contents of
MSR_MC0_CTL{,2} don't seem to be publicly documented, neither in the Intel
Architectures Developer's Manual nor in the Haswell datasheets.
Note that while HSD131 actually is only about C0-stepping as of revision
014 of the Intel desktop 4th generation processor family specification
update, these corrected errors also have been observed with D0-stepping
aka "Haswell Refresh".
mav [Tue, 5 Aug 2014 08:28:29 +0000 (08:28 +0000)]
MFC r269441:
Add missing comparisons to make list IDs in EXTENDED COPY per-initiator,
as they should be. Wrap it into a function to not duplicate the code.
markj [Tue, 5 Aug 2014 01:53:15 +0000 (01:53 +0000)]
MFC r267759, r267761
r267759:
Fix a couple of bugs on amd64 when fetching probe arguments beyond the
first five for probes entered through a UD fault (i.e. FBT probes).
Specifically, handle the fact that dtrace_invop_callsite must be
16 byte-aligned and thus may not immediately follow the call to
dtrace_invop() in dtrace_invop_start(). Also fetch register arguments and
the stack pointer through a struct trapframe instead of a struct reg.
r267761:
Fix some bugs when fetching probe arguments in i386. Firstly ensure that
the 4 byte-aligned dtrace_invop_callsite can be found and that it
immediately follows the call to dtrace_invop(). Secondly, fix some pointer
arithmetic to account for differences between struct i386_frame and illumos'
struct frame. Finally, ensure that dtrace_getarg() isn't inlined. It works
by following a fixed number of frame pointers to the probe site, so inlining
breaks it.
markj [Tue, 5 Aug 2014 00:25:46 +0000 (00:25 +0000)]
MFC r267706:
Allow creation of SDT probes from a module in which no providers are
defined. This ensures that the sdt:zfs:: probes appear despite the fact
the sdt provider is defined in the kernel rather than in zfs.ko.
markj [Mon, 4 Aug 2014 21:41:00 +0000 (21:41 +0000)]
MFC r256822:
When fetching function arguments out of a frame on amd64, explicitly select
the register based on the argument index rather than relying on the fields
in struct reg to be in the right order. This assumption is incorrect on
FreeBSD and generally led to bogus argument values for the sixth argument
of PID and USDT probes; the first five are passed directly to dtrace_probe()
via the fasttrap trap handler and so were correctly handled.
markj [Mon, 4 Aug 2014 15:36:22 +0000 (15:36 +0000)]
MFC r256571:
Add a function, memstr, which can be used to convert a buffer of
null-separated strings to a single string. This can be used to print the
full arguments of a process using execsnoop (from the DTrace toolkit) or
with the following one-liner:
Note that this relies on the process arguments being cached via the struct
proc, which means that it will not work for argvs longer than
kern.ps_arg_cache_limit. However, the following rather non-portable
script can be used to extract any argv at exec time:
dim [Mon, 4 Aug 2014 14:56:49 +0000 (14:56 +0000)]
MFC r269125:
In r232153, libarchive 3.0.3 was imported, replacing the archive_hash.h
header with archive_crypto_private.h, and its ARCHIVE_HASH_xxx macros
were renamed to ARCHIVE_CRYPTO_xxx.
Rename these macros in lib/libarchive/config_freebsd.h, to re-enable the
hashes for libarchive again. This affects the mtree format writer, and
the xar format reader and writer modules.
This also requires changes in the library order for statically linking
rescue, otherwise ld would complain about redefined symbols. Thanks to
jkim for pointing out the solution.
pfg [Mon, 4 Aug 2014 00:51:57 +0000 (00:51 +0000)]
MFC r268945:
Fix hdestroy() compliance issue.
The hcreate(3) implementation and related functions we inherited
from NetBSD used to free() the key value, something that is not
supported by the standard implementation.
This would cause a segmentation fault when attempting to run
the examples from the opengroup and linux manpages.
There is no need to bump the __FreeBSD_version as we have
always claimed XPG4.2 compliance but if some reference is
required, the bump for r269484 can be used.
peter [Sun, 3 Aug 2014 22:59:47 +0000 (22:59 +0000)]
Insta-MFC r269489: partial revert of r262867 which was MFC'ed as r263820.
Don't ignore sndbuf/rcvbuf limits for SOCK_DGRAM sockets. This appears
to be an edit error or patch fuzz mismatch.
pfg [Sun, 3 Aug 2014 18:39:11 +0000 (18:39 +0000)]
MFC r268066:
regex(3): Add support for \< and \> word delimiters
Solaris and other OSs have support for \< and \> as word
delimiters in utilities like sed(1). These are useful to
have for general compatiblity with Solaris but should be
avoided for portability with other systems, including the
traditional BSDs.
Bump __FreeBSD_version as this is likely to affect some
userland utilities.
pfg [Sun, 3 Aug 2014 18:31:52 +0000 (18:31 +0000)]
MFC r269124:
strftime() xlocale cleanups.
Replace fprintf_l with fputs when output is unformatted.
Use locale_t in _conv() since it was using sprintf (now sprintf_l)
Use locale_t on _yconv() since it calls _conv()
Obtained from: Apple Inc. (Libc 997.90.3)
CR: D482
Reviewed by: theraven
rmacklem [Sun, 3 Aug 2014 00:35:10 +0000 (00:35 +0000)]
MFC: r268273
The new NFSv3 server did not generate directory postop attributes for
the reply to ReaddirPlus when the server failed within the loop
that calls VFS_VGET(). This failure is most likely an error
return from VFS_VGET() caused by a bogus d_fileno that was
truncated to 32bits.
This patch fixes the server so that it will return directory postop
attributes for the failure. It does not fix the underlying issue caused
by d_fileno being uint32_t when a file system like ZFS generates
a fileno that is greater than 32bits.
marcel [Sat, 2 Aug 2014 22:25:24 +0000 (22:25 +0000)]
MFC 259910, 260023, 260028, 260600 & 260701:
o Fix "kptdir is itself virtual" error, caused by having the kptdir in PBVM.
o Allow building a cross libkvm for ia64.
o Add support for virtual cores (aka minidumps).
o We don't have to worry about page sizes when working on virtual cores.
o Handle truncation of the size returned by _kvm_kvatop().
hselasky [Sat, 2 Aug 2014 20:58:46 +0000 (20:58 +0000)]
Partial MFC of r267961, r267973, r267985, r267992, r267993 and r268005:
Backport some macro definitions to make backporting code from FreeBSD
current easier.
mav [Sat, 2 Aug 2014 06:56:00 +0000 (06:56 +0000)]
MFC r269123:
Implement separate I/O dispatch method for ZVOLs in "dev" mode.
Unlike disk devices ZVOLs process all requests synchronously. That makes
impossible sending multiple requests to them from single thread. From the
other side ZVOLs have real d_read/d_write methods, which unlike d_strategy
can handle uio scatter/gather and have no strict I/O size limitations.
So, if ZVOL in "dev" mode is detected, use of d_read/d_write methods instead
of d_strategy allows to avoid pointless splitting of large requests into
MAXPHYS (128K) sized chunks.
delphij [Sat, 2 Aug 2014 04:06:35 +0000 (04:06 +0000)]
MFC r268865: MFV r268852:
Reduce lock contention on the z_teardown_lock under heavily cached
read workload by splitting the single teardown rrw lock into
RRM_NUM_LOCKS (17) of them.
Read acquisitions are randomly distributed among these locks based
on curthread pointer. Write acquisitions are going to all the
locks, which for the usage of this type of lock should be rare.
Illumos issue:
5008 lock contention (rrw_exit) while running a read only load
delphij [Sat, 2 Aug 2014 04:01:44 +0000 (04:01 +0000)]
MFC r268859: MFV r268851:
When a sync task is waiting for a txg to complete, we should hurry it along
by increasing the number of outstanding async writes (i.e. make
vdev_queue_max_async_writes() return a larger number).
Illumos issue:
4753 increase number of outstanding async writes when sync task is waiting
delphij [Sat, 2 Aug 2014 03:59:35 +0000 (03:59 +0000)]
MFC r268858: MFV r268850:
Change the interaction between the DMU and ARC so that when the DMU is
shutting down an objset, we do not evict the data from the ARC. Instead
we simply coordinate the destruction of the DMU's data with the ARC.
The only case where we actually need to explicitly evict from the ARC is
when dbuf_rele_and_unlock() determines that the administrator has requested
that it not be kept in memory, via the primarycache/secondarycache properties.
In this case, we evict the data from the ARC by its blkptr_t, the same way
as when a block is freed we explicitly evict it from the ARC.
Illumos issue:
4631 zvol_get_stats triggering too many reads
delphij [Sat, 2 Aug 2014 03:56:06 +0000 (03:56 +0000)]
MFC r268855: MFV r268848:
Instead of asserting all zio's be properly aligned, only assert
on the logical ones.
Cap uberblocks at 8k, otherwise with ashift=17, there would be
only one uberblock.
This fixes a problem that zdb would trip assert on pools with
ashift >= 0xe (8k).
While there, also change the code so it only attempt to condense
space map unless the uncondensed size consumes greater than
zfs_metaslab_condense_block_threshold blocks.
Illumos issue:
4958 zdb trips assert on pools with ashift >= 0xe
rmacklem [Fri, 1 Aug 2014 21:10:41 +0000 (21:10 +0000)]
MFC: r268115
Merge the NFSv4.1 server code in projects/nfsv4.1-server over
into head. The code is not believed to have any effect
on the semantics of non-NFSv4.1 server behaviour.
It is a rather large merge, but I am hoping that there will
not be any regressions for the NFS server.
jhb [Fri, 1 Aug 2014 21:00:18 +0000 (21:00 +0000)]
MFC 256657,257423,264837,267559:
Sync vmrun.sh with HEAD:
- Add -e option to vmrun.sh passed to bhyveload(8) to set loader
environment variables.
- Stop passing unused -I option to bhyve(8).
- Reformat the usage to fit in 80 colums and other cleanups.
- Add -C option to specify the console device.
- Add -H option to pass a host path to bhyveload(8).
- Support for multiple disk and tap devices.
truckman [Fri, 1 Aug 2014 15:04:46 +0000 (15:04 +0000)]
MFC r268780
Nuke the never-used RF_TIMESHARE feature, reducing the complexity of the
code. The consensus on arch@ is that this feature might have been useful
in the distant past, but is now just unnecessary bloat.
The int_rman_activate_resource() and int_rman_deactivate_resource()
functions become trivial, so manually inline them.
The special deferred handling of RF_ACTIVE is no longer needed in
reserve_resource_bound(), so eliminate the associated code at the
end of the function.
These changes reduce the object file size by more than 500 bytes on i386.
Update the rman.9 man page to reflect the removal of the RF_TIMESHARE
feature.
Add a 'raw' parameter to the 'modinfo' subcommand. This is handy when
trying to figure out why a QSFP+/SFP+ connector or cable wasn't
identified correctly by cxgbe(4). Its output looks like this:
r268971:
Simplify r267600, there's no need to distinguish between allocated and
inlined mbufs.
r269032:
cxgbe(4): Keep track of the clusters that have to be freed by the
custom free routine (rxb_free) in the driver. Fail MOD_UNLOAD with
EBUSY if any such cluster has been handed up to the kernel but hasn't
been freed yet. This prevents a panic later when the cluster finally
needs to be freed but rxb_free is gone from the kernel.
MFC r264434:
DTrace's pid provider works by inserting breakpoint instructions at probe
sites and installing a hook at the kernel's trap handler. The fasttrap code
will emulate the overwritten instruction in some common cases, but otherwise
copies it out into some scratch space in the traced process' address space
and ensures that it's executed after returning from the trap.
In Solaris and illumos, this (per-thread) scratch space comes from some
reserved space in TLS, accessible via the fs segment register. This
approach is somewhat unappealing on FreeBSD since it would require some
modifications to rtld and jemalloc (for static TLS) to ensure that TLS is
executable, and would thus introduce dependencies on their implementation
details. I think it would also be impossible to safely trace static binaries
compiled without these modifications.
This change implements the functionality in a different way, by having
fasttrap map pages into the target process' address space on demand. Each
page is divided into 64-byte chunks for use by individual threads, and
fasttrap's process descriptor struct has been extended to keep track of
any scratch space allocated for the corresponding process.
With this change it's possible to trace all libc functions in a program,
e.g. with
MFC r268808:
Increase maximal number of SCSI ports in CTL from 32 to 128.
After I gave each iSCSI target its own port, the old limit appeared to be
not so big. This change almost proportionally increases per-LUN memory
use, but it is still three times better then it was before r268807.
MFC r268807:
Reduce per-LUN memory usage from 18MB to 1.8MB.
CTL never had use for CA support code since SPI has gone, and there is no
even frontends supporting that. But it still was reserving 256 bytes of
memory per LUN per every possible initiator on every possible port.
Wrap unused code with ifdef's in case somebody ever need it.
MFC r268767:
Add support for VMWare dialect of EXTENDED COPY command, aka VAAI Clone.
This allows to clone VMs and move them between LUNs inside one storage
host without generating extra network traffic to the initiator and back,
and without being limited by network bandwidth.
LUNs participating in copy operation should have UNIQUE NAA or EUI IDs set.
For LUNs without these IDs VMWare will use traditional copy operations.
Beware: the above LUN IDs explicitly set to values non-unique from the VM
cluster point of view may cause data corruption if wrong LUN is addressed!
MFC: r268726
Move the "retry:" label so that the calls to m_pullup() are
not done after the call to m_defrag(). This fixes a problem
where m_pullup() would prepend an mbuf to the list created
by m_defrag() making the chain greater than 32 again.
MFC r263329:
Only invoke fasttrap hooks for traps from user mode, and ensure that they're
called with interrupts enabled. Calling fasttrap_pid_probe() with interrupts
disabled can lead to deadlock if fasttrap writes to the process' address
space.
MFC r262669:
When our linker merges .SUNW_dof sections from multiple files, it simply
concatenates the DOF tables into one section. Previously, the USDT init
code in drti.o would only look at the first table in the DOF section; with
this change, it iterates over all the tables, passing each DOF table to
the kernel.
marius [Tue, 29 Jul 2014 13:11:37 +0000 (13:11 +0000)]
MFC: r269051
Copying pages via temporary mappings in the !DMAP case of pmap_copy_pages()
involves updating the corresponding page tables followed by accesses to the
pages in question. This sequence is subject to the situation exactly described
in the "AMD64 Architecture Programmer's Manual Volume 2: System Programming"
rev. 3.23, "7.3.1 Special Coherency Considerations" [1, p. 171 f.]. Therefore,
issuing the INVLPG right after modifying the PTE bits is crucial (see also
r269050, MFCed to stable/10 in r269235).
For the amd64 PMAP code, the order of instructions was already correct. The
above fact still is worth documenting, though.
marius [Tue, 29 Jul 2014 13:08:37 +0000 (13:08 +0000)]
MFC: r269050
- Copying and zeroing pages via temporary mappings involves updating the
corresponding page tables followed by accesses to the pages in question.
This sequence is subject to the situation exactly described in the "AMD64
Architecture Programmer's Manual Volume 2: System Programming" rev. 3.23,
"7.3.1 Special Coherency Considerations" [1, p. 171 f.]. Therefore, issuing
the INVLPG right after modifying the PTE bits is crucial.
For pmap_copy_page(), this has been broken in r124956 and later on carried
over to pmap_copy_pages() derived from the former, while all other places
in the i386 PMAP code use the correct order of instructions in this regard.
Fixing the latter breakage solves the problem of data corruption seen with
unmapped I/O enabled when running at least bare metal on AMD R-268D APUs.
However, this might also fix similar corruption reported for virtualized
environments.
- In pmap_copy_pages(), correctly set the cache bits on the source page being
copied. This change is thought to be a NOP for the real world, though. [2]
When doing an "extreme rewind" import ("zpool import -XF"), we attempt
to verify all data in the pool, essentially scrubbing the entire pool.
The problem is that spa_load_verify_cb() issues an unbounded number of
concurrent scrub i/os. This can lead to all of memory being used for
these zio's, wedging the system. Like normal scrub, we need to put a
cap on the number of outstanding i/os, and have the traverse thread
block when we reach this cap.
For this purpose the cap can be very large (10,000) to optimize the
elevator algorithm. Three kernel tunables have been added:
The latter two tunables controls whether metadata and/or user data
when doing extreme rewind.
Make 'zpool import -T' imply scrub.
Make zpool import -T <txg> accept hexadecimal values for the txg when
prefixed with 0x.
Skip txg's for which there is no uberblock when doing extreme rewind.
Skip reading all user data twice by skipping prefetches when doing
extreme rewinds as we do not access via the ARC.
Illumos issues:
4970 need controls on i/o issued by zpool import -XF
4971 zpool import -T should accept hex values
4972 zpool import -T implies extreme rewind, and thus a scrub
4973 spa_load_retry retries the same txg
4974 spa_load_verify() reads all data twice
MFC r268236,268264,268524,268646,268802,269021:
This brings VHD support to mkimg(1); both dynamic and fixed file formats.
Dynamic VHD and VMDK file images are now sparsely written, meaning that
"free" sectors do not occupy space.
MFC r268608:
The tmpfs_link() must not dereference the filesystem-specific data for
a vnode until it is verified that the vnode indeed belongs to tmpfs mount.
MFC r268606:
Generalize vn_get_ino() to allow filesystems to use custom vnode
producer. Convert inline copies of vn_get_ino() in msdosfs and cd9660
into the uses of vn_get_ino_gen().
projects / FreeBSD / stable / 10.git / log