MFC r295618,r300100,r300531:

MFC r295618,r300100,r300531:

r295618 (by cem):

NTB: workaround for high traffic hardware hang

This patch comes from Dave Jiang's Linux tree, davejiang/ntb.  It hasn't
been accepted into Linus' tree, so I do not have an authoritative SHA1
to point at.  Original commit log:

=====================================================================
A hardware errata causes the NTB to hang when heavy bi-directional
traffic in addition to the usage of BAR0/1 (where the registers reside,
including the doorbell registers to trigger interrupts).

This workaround is only available on Haswell and Broadwell platform.
The workaround is to enable split BAR in the BIOS to allow the 64bit
BAR4 to be split into two 32bit BAR4 and BAR5. The BAR4 shall be pointed
to LAPIC region of the remote host. We will bypass the db mechanism and
directly trigger the MSIX interrupts. The offsets and vectors are
exchanged during transport scratch pad negotiation. The scratch pads are
now overloaded in order to allow the exchange of the information. This
gets around using the doorbell and prevents the lockup with additional
pcode changes in BIOS.

Signed-off-by:	Dave Jiang <dave.jiang@intel.com>
=====================================================================

Notable changes in the FreeBSD version of this patch:
* The MSIX BAR is configurable, like hw.ntb.b2b_mw_idx (msix_mw_idx).
  The Linux version of the patch only uses BAR4.
* MSIX negotiation aborts if the link goes down.

Obtained from:	Linux (Dual BSD/GPL driver)

r300100 (by cem):

ntb_hw(4): Add sysctls for administrative/test link config, state

dev.ntb_hw.0.admin_up=0/1: Like ifconfig UP/DOWN.
dev.ntb_hw.0.active=0/1:   Like ifconfig 'status'

r300531 (by cem):

ntb_hw(4): Only record the first three MSIX vectors

Don't overrun the msix_data array by reading the (unused) link state
interrupt information.

git-svn-id: svn://svn.freebsd.org/base/stable/10@301811 ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f