https://www.illumos.org/issues/7843
get_clones_stat() could be very slow if a snapshot has many (thousands) clones.
Clone names are added to an nvlist that's created with NV_UNIQUE_NAME.
So, each time a new name is appended to the list, the whole list is searched
linearly to see if that name is not already in the list. That results in the
quadratic complexity.
That should be easy to fix as we know in advance that we should not get any
duplicate names, so we can drop NV_UNIQUE_NAME when creating the list.
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Andriy Gapon <avg@FreeBSD.org>
https://www.illumos.org/issues/7843
get_clones_stat() could be very slow if a snapshot has many (thousands) clones.
Clone names are added to an nvlist that's created with NV_UNIQUE_NAME.
So, each time a new name is appended to the list, the whole list is searched
linearly to see if that name is not already in the list. That results in the
quadratic complexity.
That should be easy to fix as we know in advance that we should not get any
duplicate names, so we can drop NV_UNIQUE_NAME when creating the list.
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Andriy Gapon <avg@FreeBSD.org>
Michal Meloun [Wed, 8 Mar 2017 11:40:27 +0000 (11:40 +0000)]
Unbreak ARMv6 world.
The new compiler_rt library imported with clang 4.0.0 have several fatal
issues (non-functional __udivsi3 for example) with ARM specific instrict
functions. As temporary workaround, until upstream solve these problems,
disable all thumb[1][2] related feature.
Alexander Motin [Wed, 8 Mar 2017 11:24:33 +0000 (11:24 +0000)]
Add initial support for UNMAP granularity.
Report UNMAP granularity as stripesize/-offset if we have no other values
to report there.
Add new quirk DA_Q_STRICT_UNMAP for cases when target is too critical to
misaligned UNMAP request, reporting errors instead of being suboptimal.
Setting this quirk makes da periph to forcefully align all UNMAP requests
to avoid those errors by the cost of some odd ranges not being UNMAP'ed.
This makes UNMAP usable within VMware 6.x VMs, just now 100% efficient.
Enji Cooper [Wed, 8 Mar 2017 05:31:54 +0000 (05:31 +0000)]
usr.bin/fortune: convert to OBJTOP/SRCTOP idioms
- Use OBJTOP/SRCTOP-relative paths when looking for include files and
strfile.
- Add FORTUNES_OBJ and FORTUNES_SRC to abbreviate usr.bin/fortune
pathing.
This is being done to simplify make output/idioms.
Warner Losh [Wed, 8 Mar 2017 02:47:59 +0000 (02:47 +0000)]
Copy needed include files from EDK2. This is a minimal set gleened
from the .depend files after the build:
cp -r ../vendor/edk2/MdePkg/Include sys/contrib/edk2
cd lib/libefivar
make
pushd `make -V .OBJDIR`
cat .depend*.o | grep sys/contrib | cut -d' ' -f 3 |
sort -u | sed -e 's=/full/path/sys/contrib/edk2/==' > /tmp/xxx
popd
cd ../../sys/contrib/edk2
rm -rf Include
for i in `cat /tmp/xxx`; do
svn cp svn+ssh://repo.freebsd.org/base/vendor/edk2/dist/MdePkg/$i $i
done
svn cp svn+ssh://repo.freebsd.org/base/vendor/edk2/dist/MdePkg/MdePkg.dec .
The original EDK2 repo is ~265MB, the MdePkg is ~23MB, all
MdePkg/Includes is ~7MB and this minimal set is ~1.3MB.
Warner Losh [Tue, 7 Mar 2017 23:06:41 +0000 (23:06 +0000)]
Avoid dereferencing unintialized elements in the error path.
Some drives sometimes have errors for things like setting the number
of queue entries in the submission queue. The error paths taken for
these drives ensure a panic dereferencing uninialized data.
Warner Losh [Tue, 7 Mar 2017 23:02:59 +0000 (23:02 +0000)]
cwd10 takes the low 32-bits and cwd11 takes the upper 32-bits of the
lba. Rather than do a cast to uint64_t, which clang warns might be
unaligned, do the stores 32-bits at a time.
Marius Strobl [Tue, 7 Mar 2017 22:42:44 +0000 (22:42 +0000)]
Add and use a MMC_DECLARE_BRIDGE macro for declaring mmc(4) bridges
as kernel drivers and their dependency onto mmc(4); this allows for
incrementing the mmc(4) module version but also for entire omission
of these bridge declarations for mmccam(4) in a single place, i. e.
in dev/mmc/bridge.h.
Justin Hibbits [Tue, 7 Mar 2017 22:11:57 +0000 (22:11 +0000)]
Fix booting with >4GB RAM on PowerMac G5 hardware
===
From Nathan Whitehorn:
Open Firmware runs in virtual mode on the Powermac G5. This runs inside the
kernel page table, which preserves all address translations made by OF before
the kernel starts; as a result, the kernel address space is a strict superset of
OF's.
Where this explodes is if OF uses an unmapped SLB entry. The SLB fault handler
runs in real mode and refers to the PCPU pointer in SPRG0, which blows up the
kernel. Having a value of SPRG0 that works for the kernel is less fatal than
preserving OF's value in this case.
===
The result of this is seemingly random panics from NULL dereferences, or hangs
immediately upon boot. By not restoring SPRG0 for Open Firmware entry the
kernel PCPU pointer is preserved and SLB faults are successful, resulting in a
stable kernel.
Warner Losh [Tue, 7 Mar 2017 21:47:54 +0000 (21:47 +0000)]
Make multi-namespace nvme drives more robust.
Fix assumptions about name spaces in NVME driver. First, it assumes
cdata.nn is the number of configured devices. However, it is the
number of supported name spaces. Second, it assumes that there will
never be more than 16 name spaces supported, but a certain drive I'm
testing reports 1024. It assumes that name spaces are a tightly packed
namespace, but the standard seems to indicate otherwise. Finally, it
assumes that an error would be generated when quearying an
unconfigured namespace. Instead, it succeeds but the identify data is
all zeros.
Fix these by limiting the number of name spaces we probe to 16. Remove
aborting when we find one in error. When the size of the name space is
zero, ignore it.
This is admittedly a bandaide. The long term fix will be to
participate in the enumeration and name space change protocols
definfed in the NVNe standard.
EDK2 is Intel's BSD Licensed UEFI implementation. We'll be bringing in
various routines from there rather than reimplementing them from
scratch for libefivar and the EFI boot loader.
The upstream repo has ^M ending on everything (sometimes multiple
times!), so the following script was run prior to import so changes we
have to do don't first include changing every line:
Also, only the MdePkg was brought in (it's 17MB, while the entire repo
is 250MB). It's almost completely certain nothing else will be used,
but if it is, it can be brough in along side MdePkg in the future.
Add support for constant pointer constructs to READ_ONCE() in the
LinuxKPI. When the type of the argument is constant the temporary
variable cannot be assigned after the barrier. Instead assign the
temporary variable by initialization.
Enji Cooper [Tue, 7 Mar 2017 17:53:53 +0000 (17:53 +0000)]
Add bsd.man.mk references for MAN under bsd.lib.mk and bsd.prog.mk
The latter set of manpages directly consume bsd.man.mk, so the bsd.man.mk
behavior should be the source of truth for underlying behavior, whereas
the other manpage fragment descriptions should document how they tweak
the variable behavior, if at all (bsd.prog.mk does tweak the default
value, as noted in its description)
Alexander Motin [Tue, 7 Mar 2017 17:41:08 +0000 (17:41 +0000)]
Add mechanism to unload CAM periph drivers.
For now it allows to unload CTL kernel module if there are no target-capable
SIMs in CAM. As next step full teardown of CAM targets can be implemented.
Dmitry Chagin [Tue, 7 Mar 2017 17:07:16 +0000 (17:07 +0000)]
Reduce code duplication between MD Linux code by moving SYSV IPC 64-bit
related struct definitions out into the MI path.
Invert the native ipc structs to the Linux ipc structs convesion logic.
Since 64-bit variant of ipc structs has more precision convert native ipc
structs to the 64-bit Linux ipc structs and then truncate 64-bit values
into the non 64-bit if needed. Unlike Linux, return EOVERFLOW if the
values do not fit.
Fix SYSV IPC for 64-bit Linuxulator which never sets IPC_64 bit.
Andriy Gapon [Tue, 7 Mar 2017 16:07:52 +0000 (16:07 +0000)]
firewire/sbp: try to improve locking, plus a few style nits
This change tries to fix the most obvious locking problems.
sbp_cam_scan_lun() is never called with the sbp lock held, so the lock
needs to be acquired internally (if it's needed at all).
Without this change a kernel with INVARIANTS panics when a firewire disk
is connected:
panic: mutex sbp not owned at /usr/src/sys/dev/firewire/sbp.c:967
KDB: stack backtrace:
db_trace_self_wrapper() at 0xffffffff80420bbb = db_trace_self_wrapper+0x2b/frame 0xfffffe0504df0930
kdb_backtrace() at 0xffffffff80670359 = kdb_backtrace+0x39/frame 0xfffffe0504df09e0
vpanic() at 0xffffffff8063986c = vpanic+0x14c/frame 0xfffffe0504df0a20
panic() at 0xffffffff806395b3 = panic+0x43/frame 0xfffffe0504df0a80
__mtx_assert() at 0xffffffff8061c40d = __mtx_assert+0xed/frame 0xfffffe0504df0ac0
sbp_cam_scan_lun() at 0xffffffff80474667 = sbp_cam_scan_lun+0x37/frame 0xfffffe0504df0af0
xpt_done_process() at 0xffffffff802aacfa = xpt_done_process+0x2da/frame 0xfffffe0504df0b30
xpt_done_td() at 0xffffffff802ac2e5 = xpt_done_td+0xd5/frame 0xfffffe0504df0b80
fork_exit() at 0xffffffff805ff72f = fork_exit+0xdf/frame 0xfffffe0504df0bf0
fork_trampoline() at 0xffffffff8082483e = fork_trampoline+0xe/frame
0xfffffe0504df0bf0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
Also, I tried to reduce the scope of the sbp lock to avoid holding it
while doing bus_dma allocations.
The code badly needs some re-engineering. SBP really should implement
a CAM transport, so that it avoids control flow inversion when re-scanning
the bus. Also, the sbp lock seems to be too coarse.
Additionally, the commit includes some changes not related to locking.
- sbp_cam_scan_lun: restore CAM_DEV_QFREEZE before re-queueing the ccb
because xpt_setup_ccb resets ccb_h.flags
- sbp_post_busreset: call xpt_release_simq only if it's actually frozen
- don't place private SIMQ_FREEZED flag (sic, "freezed") into sim->flags,
use sbp->flags for that
- some style fixes and control flow enhancements
Andriy Gapon [Tue, 7 Mar 2017 15:43:49 +0000 (15:43 +0000)]
qlxgbe: add GCC_MS_EXTENSIONS to CFLAGS to make old base GCC happy
The module uses unnamed structure and union fields and base GCC in
stable/10 doesn't like it.
I think that that is a C11 feature, so it is courteous of more modern
compilers to not complain about it when compiling in C99 mode.
Eric Badger [Tue, 7 Mar 2017 13:41:01 +0000 (13:41 +0000)]
don't stop in issignal() if P_SINGLE_EXIT is set
Suppose a traced process is stopped in ptracestop() due to receipt of a
SIGSTOP signal, and is awaiting orders from the tracing process on how
to handle the signal. Before sending any such orders, the tracing
process exits. This should kill the traced process. But suppose a second
thread handles the SIGKILL and proceeds to exit1(), calling
thread_single(). The first thread will now awaken and will have a chance
to check once more if it should go to sleep due to the SIGSTOP. It must
not sleep after P_SINGLE_EXIT has been set; this would prevent the
SIGKILL from taking effect, leaving a stopped orphan behind after the
tracing process dies.
When selecting brand based on old Elf branding, prefer the brand which
interpreter exactly matches the one requested by the activated image.
This change applies r295277, which did the same for note branding, to
the old brand selection, with the same reasoning of fixing compat32
interpreter substitution.
PR: 211837
Reported by: kenji@kens.fm
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Consistently use vm_ooffset_t type for the vm object offset in
elf_load_section.
The values passed currently as vm_offset_t are phdr.p_offset, which
have the native Elf word size. Since elf_load_section interprets them
as the file offset, use vm object offset type.
Noted and reviewed by: alc
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
This change makes the workqueue implementation behave more like in
Linux, both functionality wise and structure wise.
All workqueue code has been moved to linux_work.c
Add an atomic based statemachine to the work_struct to ensure proper
operation. Prior to this change struct_work was directly mapped to a
FreeBSD task. When a taskqueue has multiple threads the same task may
end up being executed on more than one worker thread simultaneously.
This might cause problems with code coming from Linux, which expects
serial behaviour, similar to Linux tasklets.
Move all global workqueue function names into the linux_xxx domain to
avoid symbol name clashes in the future.
Implement a few more workqueue related functions and macros.
Create two multithreaded taskqueues for the LinuxKPI during module
load, one for time-consuming callbacks and one for non-time consuming
callbacks.
xen/netfront: fix inbound packet flags for checksum offload
Currently netfront is setting the flags of inbound packets with the checksum
not present (offloaded) to (CSUM_IP_CHECKED | CSUM_IP_VALID | CSUM_DATA_VALID |
CSUM_PSEUDO_HDR). According to the mbuf(9) man page this is not the correct
combination of flags, it should instead be (CSUM_DATA_VALID |
CSUM_PSEUDO_HDR).
Reviewed by: Wei Liu <wei.liu2@citrix.com>
MFC after: 2 weeks
Sponsored by: Citrix Systems R&D
Differential revision: https://reviews.freebsd.org/D9831
xenstore: fix suspension when using the xenstore device
Lock the xenstore request mutex when suspending user-space processes, in order
to prevent any process from holding this lock when going into suspension, or
else the xenstore suspend process is going to deadlock.
When running on Xen, it's possible that a suspend request to the hypervisor
fails (return from HYPERVISOR_suspend different than 0). This means that the
suspend hasn't succeed, and the resume procedure needs to properly handle this
case.
First of all, when such situation happens there's no need to reset the vector
callback, hypercall page, shared info, event channels or grant table, because
it's state is preserved. Also, the PV drivers don't need to be reset to the
initial state, since the connection with the backed has not been interrupted.
Conrad Meyer [Tue, 7 Mar 2017 04:51:35 +0000 (04:51 +0000)]
ps(1): Fix -w + UNLIMITED handling
A follow-up fix for r314685.
Because the -w flag is parsed after ps(1) infers termwidth from COLUMNS and
stdout, and UNLIMITED happens to be the zero value, the single -w flag in
combination with a non-terminal stdout or COLUMNS=0 could result in output
truncated at 131 characters. (Despite the output being unlimited without
-w.)
Obviously, adding more -w shouldn't truncate output lines.
The committed patch is from bdrewery@, and I've reviewed and tested it.
Enji Cooper [Tue, 7 Mar 2017 04:33:17 +0000 (04:33 +0000)]
Don't rely on dependency in Makefile.inc1 for strfile; make datfiles depend on strfile
In most cases strfile is built as part of build-tools, but in the event that someone
cd'ed to the directory, tried to build from scratch, and had MK_GAMES=no previously,
the build would fail in .../datfiles , trying to find strfile .
Mark this directory tree "SUBDIR_PARALLEL" safe to help facilitate this, instead of
shuffling around the SUBDIR entries (all of the other Makefiles will build standalone).
Enji Cooper [Tue, 7 Mar 2017 04:13:22 +0000 (04:13 +0000)]
mergemaster: fix description of -p
-p only handles updating /etc/master.passwd and /etc/group . No more,
no less.
Also, mergemaster (and no other portions of the vanilla FreeBSD build
process) should be messing with __MAKECONF or SRCCONF as part of the
installworld or distribution process. Don't insinuate that mergemaster
does that as it's a false claim.
We've found a recurring problem where some userland process would be
stuck spinning at 100% cpu around sbcut_internal(). Inside
sbflush_internal(), sb_ccc reached to about 4GB and before passing it
to sbcut_internal(), we type-cast it from uint to int making it -ve.
The root cause of sockbuf growing this large is unknown. Correct fix
is also not clear but based on mailing list discussions, adding
KASSERTs to panic instead of looping endlessly.
Introduce the concept of IPsec security policies scope.
Currently are defined three scopes: global, ifnet, and pcb.
Generic security policies that IKE daemon can add via PF_KEY interface
or an administrator creates with setkey(8) utility have GLOBAL scope.
Such policies can be applied by the kernel to outgoing packets and checked
agains inbound packets after IPsec processing.
Security policies created by if_ipsec(4) interfaces have IFNET scope.
Such policies are applied to packets that are passed through if_ipsec(4)
interface.
And security policies created by application using setsockopt()
IP_IPSEC_POLICY option have PCB scope. Such policies are applied to
packets related to specific socket. Currently there is no way to list
PCB policies via setkey(8) utility.
Modify setkey(8) and libipsec(3) to be able distinguish the scope of
security policies in the `setkey -DP` listing. Add two optional flags:
'-t' to list only policies related to virtual *tunneling* interfaces,
i.e. policies with IFNET scope, and '-g' to list only policies with GLOBAL
scope. By default policies from all scopes are listed.
To implement this PF_KEY's sadb_x_policy structure was modified.
sadb_x_policy_reserved field is used to pass the policy scope from the
kernel to userland. SADB_SPDDUMP message extended to support filtering
by scope: sadb_msg_satype field is used to specify bit mask of requested
scopes.
For IFNET policies the sadb_x_policy_priority field of struct sadb_x_policy
is used to pass if_ipsec's interface if_index to the userland. For GLOBAL
policies sadb_x_policy_priority is used only to manage order of security
policies in the SPDB. For IFNET policies it is not used, so it can be used
to keep if_index.
After this change the output of `setkey -DP` now looks like:
# setkey -DPt
0.0.0.0/0[any] 0.0.0.0/0[any] any
in ipsec
esp/tunnel/87.250.242.144-87.250.242.145/unique:145
spid=7 seq=3 pid=58025 scope=ifnet ifname=ipsec0
refcnt=1
# setkey -DPg
::/0 ::/0 icmp6 135,0
out none
spid=5 seq=1 pid=872 scope=global
refcnt=1
Marius Strobl [Mon, 6 Mar 2017 23:47:59 +0000 (23:47 +0000)]
o Another round fixes for mmc(4), mmcsd(4) and sdhci(4) regarding
comments, marking unused parameters as such, style(9), whitespace,
etc.
o In the mmc(4) bridges and sdhci(4) (bus) front-ends:
- Remove redundant assignments of the default bus_generic_print_child
device method (I've whipped these out of the tree as part of r227843
once, but they keep coming back ...),
- use DEVMETHOD_END,
- use NULL instead of 0 for pointers.
o Trim/adjust includes.
Kristof Provost [Mon, 6 Mar 2017 23:41:23 +0000 (23:41 +0000)]
pf: Fix a crash in low-memory situations
If the call to pf_state_key_clone() in pf_get_translation() fails (i.e. there's
no more memory for it) it frees skp. This is wrong, because skp is a
pf_state_key **, so we need to free *skp, as is done later in the function.
Getting it wrong means we try to free a stack variable of the calling
pf_test_rule() function, and we panic.
Enji Cooper [Mon, 6 Mar 2017 22:18:14 +0000 (22:18 +0000)]
Remove *-api(3) manpages removed in ATF 0.21
I overlooked the fact that these manpages had been removed upstream
and replaced with their non *-api(3) equivalents. Follow upstream's
lead and remove the unused manpages.
Enji Cooper [Mon, 6 Mar 2017 22:08:47 +0000 (22:08 +0000)]
Fix issues noted by igor/manlint
- Fix typos [1]:
-- manged -> managed
-- specifiying -> specifying
- Escape '.' at start of lines using & to tell the roff processor
that the line isn't meant to be treated as a command [2].
Enji Cooper [Mon, 6 Mar 2017 21:43:47 +0000 (21:43 +0000)]
Fix igor/manlint issues with atf-c(3)
- Remove surrounding ellipses in i.e. section and add a comma before
and after the i.e. reference [1].
- Fix typo (specifiying -> specifying) [1].
- Escape '.' at start of lines using & to tell the roff processor
that the line isn't meant to be treated as a command [2].
Enji Cooper [Mon, 6 Mar 2017 21:39:57 +0000 (21:39 +0000)]
Fix issues with atf-check(1) found by igor and refer to atf-check(1)
- Fix typo (specifiying -> specifying)
- Add atf-check(1) to SEE ALSO section for completeness. It's mentioned
above, but by convention the SEE ALSO section should list all relevant
references to other tools and APIs.
Enji Cooper [Mon, 6 Mar 2017 21:35:33 +0000 (21:35 +0000)]
Fix manlint issues with atf-check(1)
- Use `.Bf Em`/`.Ef` instead of prefixing lines with `.Em`. The forms
are equivalent with traditional roff, but unnecessarily verbose. The
former form applies the .Em macro to the enclosed block.
- Move EXIT_STATUS section down so the section complies with section
ordering specified by mdoc(7) and enforced by manlint(1).
Enji Cooper [Mon, 6 Mar 2017 21:30:06 +0000 (21:30 +0000)]
Fix out-of-order sections in atf-sh(1)
- `.Op` must be used in the SYNOPSIS section, not the NAME section.
- Move ATF_SHELL environment variable description up to first
ENVIRONMENT section. Garbage collect the duplicate ENVIRONMENT
section.
Enji Cooper [Mon, 6 Mar 2017 21:24:55 +0000 (21:24 +0000)]
Fix atf-sh(3) manpage issues
- Fix spelling errors (specifiying -> specifying) [1]
- Escape '.' at start of lines using & to tell the roff processor
that the line isn't meant to be treated as a command [2].
Pull in r296992 from upstream llvm trunk (by Sanjoy Das):
[SCEV] Decrease the recursion threshold for CompareValueComplexity
Fixes PR32142.
r287232 accidentally increased the recursion threshold for
CompareValueComplexity from 2 to 32. This change reverses that
change by introducing a separate flag for CompareValueComplexity's
threshold.
The latter revision fixes the excessive compile times for skein_block.c.
Bryan Drewery [Mon, 6 Mar 2017 21:06:55 +0000 (21:06 +0000)]
Fix bootstrapping libmd on older systems after r314709.
This follows another fix to bootstrap libmd after r313404. The
MD5FileChunk prototype is needed to build libmd, but it is
only reliably in the src tree's sys/md5.h header. Rather than
polluting the legacy build with this header for the entire build,
just symlink it in here for now as is done in the elftoolchain
build. Libmd is already referencing other src tree headers by
its used of CFLAGS+= ${SRCTOP}/sys/crypto/sha2. This, and
other uses of CFLAGS+= ${SRCTOP}/sys..., may later change to
be in the legacy mechanism.
Alexander Motin [Mon, 6 Mar 2017 19:39:31 +0000 (19:39 +0000)]
Import mpr(4) driver P12 to P14 diff from vendor site.
This is mostly a version bump to stay in version number sync with firmware.
The only change there was cosmetic: Display degraded speed message upon
receiving Active Cable Exception Event with DEGRADED reason code.
Gleb Smirnoff [Mon, 6 Mar 2017 19:14:08 +0000 (19:14 +0000)]
In panic() print current timestamp, which matches timestamp in the dump
header. This will help to correlate console server logs with dump files,
no matter how precise is clock on a console server appliance, and how
buggy the appliance is.
projects / FreeBSD / FreeBSD.git / log