From f2ff94851b76485b6eb51255208a53847c39b044 Mon Sep 17 00:00:00 2001 From: Pyun YongHyeon Date: Mon, 21 Jan 2008 06:38:23 +0000 Subject: [PATCH] Overhaul sf(4) to make it run on all architectures and implement checksum offoload by downloading AIC-6915 firmware. Changes are o Header file cleanup. o Simplified probe logic. o s/u_int{8,16,32}_t/uint{8,16,32}_t/g o K&R -> ANSI C. o In register access function, added support both memory mapped and IO space register acccess. The function will dynamically detect which method would be choosed. o sf_setperf() was modified to support strict-alignment architectures. o Use SF_MII_DATAPORT instead of hardcoded value 0xffff. o Added link state/speed, duplex changes handling task q. The task q is also responsible for flow control settings. o Always hornor link up/down state reported by mii layers. The link state information is used in sf_start() to determine whether we got a valid link. o Added experimental flow-control setup. It was commented out but will be activated once we have flow-cotrol infrastructure in mii layer. o Simplify IFF_UP/IFCAP_POLLING and IFF_PROMISC handling logic. Rx filter always honors promiscuous mode. o Implemented suspend/resume methods. o Reorganized Rx filter routine so promiscuous mode changes doesn't require interface re-initialization. o Reimplemnted driver probe routine such that it looks for matching device from supported hardware list table. This change will help to add newer hardware revision to the driver. o Use ETHER_ADDR_LEN instead of hardcoded value. o Prefer memory space register mapping over I/O space as the hardware requires lots of register access to get various consumer/producer index. Failing to get memory space mapping, sf(4) falls back to I/O space mapping. Use of memory space register mapping requires somewhat large memory space(512K), though. o Switch to simpler bus_{read,write}_{1,2,4}. o Use PCIR_BAR macro to get BARs. o Program PCI cache line size if the cache line size was set to 0 and enable PCI MWI. o Add a new sysctl node 'dev.sf.N.stats' that shows various MAC counters for Rx/Tx statistics. o Add a sysctl node to configure interrupt moderation timer. The timer defers interrupts generation until time specified in timer control register is expired. The value in the timer register is in units of 102.4us. The allowable range for the timer is 0 - 31 (0 ~ 3.276ms). The default value is 1(102.4us). Users can change the timer value with dev.sf.N.int_mod sysctl(8) variable/loader(8) tunable. o bus_dma(9) conversion - Enable 64bit DMA addressing. - Enable 64bit descriptor format support. - Apply descriptor ring alignment requirements(256 bytes alignment). - Apply Rx buffer address alignment requirements(4 bytes alignment). - Apply 4GB boundary restrictions(Tx/Rx ring and its completion ring should live in the same 4GB address space.) - Set number of allowable number of DMA segments to 16. In fact, AIC-6915 doesn't have a limit for number of DMA segments but it would be waste of Tx descriptor resource if we allow more than 16. - Rx/Tx side bus_dmamap_load_mbuf_sg(9) support. - Added alignment fixup code for strict-alignment architectures. - Added endianness support code in Tx/Rx descriptor access. With these changes sf(4) should work on all platforms. o Don't set if_mtu in device attach, it's handled in ether_ifattach. o Use our own callout to drive watchdog timer. o Enable VLAN oversized frames and announce sf(4)'s VLAN capability to upper layer. o In sf_detach(), remove mtx_initialized KASSERT as it's not possible to get there without initialzing the mutex. Also mark that we're about to detaching so active bpf listeners do not panic the system. o To reduce PCI register access cycles, Rx completion ring is directly scanned instead of reading consumer/producer index registers. In theory, Tx completion ring also can be directly scanned. However the completion ring is composed of two types completion(1 for Tx done and 1 and DMA done). So reading producer index via register access would be more safer way to detect the ring wrap-around. o In sf_rxeof(), don't use m_devget(9) to align recevied frames. The alignment is required only for strict-alignment architectures and now the alignment is handled by sf_fixup_rx() if required. The removal of the copy operation in fast path should increase Rx performance a lot on non-strict-alignemnt architectures such as i386 and amd64. o In sf_newbuf(), don't set descriptor valid bit as sf(4) is programmed to run with normal mode. In normal mode, the valid bit have no meaning. The valid bit should be used only when the hardware uses polling(prefetch) mode. The end of descriptor queue bit could be used if needed, but sf(4) relys on auto-wrapping of hardware on 256 descriptor queue entries so both valid and descriptor end bit are not used anymore. o Don't disable generation of Tx DMA completion as said in datasheet and use the Tx DMA completion entry instead of relying on Tx done completion entry. Also added additional Tx completion entry type check in Tx completion handler. o Don't blindly reset watchdog timer in sf_txeof(). sf(4) now unarm the the watchdog only if there are no active Tx descriptors in Tx queue. o Don't manually update various counters in driver, instead, use built-in MAC statistic registers to update them. The statistic registers are updated in every second. o Modified Tx underrun handlers to increase the threshold value in units of 256 bytes. Previously it used to increase 16 bytes at a time which seems to take too long to stabalize whenever Tx underrun occurrs. o In interrupt handler, additional check for the interrupt is performed such that interrupts only for this device is allowed to process descriptor rings. Because reading SF_ISR register clears all interrtups, nuke writing to a SF_ISR register. o Tx underrun is abonormal condition and SF_ISR_ABNORMALINTR includes the interrupt. So there is no need to inspect the Tx underrun again in main interrupt loop. o Don't blindly reinitialize hardware for abnormal interrupt condition. sf(4) reintializes the hardware only when it encounters DMA error which requires an explicit hardware reinitialization. o Fix a long standing bug that incorrectly clears MAC statistic registers in sf_init_locked. o Added strict-alignment safe way of ethernet address reprogramming as IF_LLADDR may return unaligned address. o Move sf_reset() to sf_init_locked in order to always reset the hardware to a known state prior to configuring hardware. o Set default Rx DMA, Tx DMA paramters as shown in datasheet. o Enable PCI busmaster logic and autopadding for VLAN frames. o Rework sf_encap. - Previously sf(4) used to type 0 of Tx descriptor with padding enabled to store driver private data. Emebedding private data structures into descriptors is bad idea as the structure size would be different between 64bit and 32bit architectures. The type 0 descriptor allows fixed number of DMA segments in a descriptor format and provides relatively simple interface to manage multi-fragmented frames. However, it wastes lots of Tx descriptors as not all frames are fragmented as the number of allowable segments in a descriptor. - To overcome the limitation of type 0 descriptor, switch to type 2 descriptor which allows 64bit DMA addressing and can handle unliumited number of fragmented DMA segments. The drawback of type 2 descriptor is in its complexity in managing descriptors as driver should handle the end of Tx ring manually. - Manually set Tx desciptor queue end mark and record number of used descriptors to reclaim used descriptors in sf_txeof(). o Rework sf_start. - Honor link up/down state before attempting transmission. - Because sf(4) uses only one of two Tx queues, use low priority queue instead of high one. This will remove one shift operation in each Tx kick command. - Cache last produder index into softc such that subsequenet Tx operation doesn't need to access producer index register. o Rewrote sf_stats_update to include all available MAC statistic counters. o Employ AIC-6915 firmware from Adaptec and implement firmware download routine and TCP/UDP checksum offload. Partial checksum offload support was commented out due to the possibility of firmware bug in RxGFP. The firmware can strip VLAN tag in Rx path but the lack of firmware assistance of VLAN tag insertion in transmit side made it useless on FreeBSD. Unlike checksum offload, FreeBSD requires both Tx/Rx hardware VLAN assistance capability. The firmware may also detect wakeup frame and can wake system up from states other than D0. However, the lack of wakeup support form D3cold state keep me from adding WOL capability. Also detecting WOL frame requires firmware support but it's not yet known to me whether the firmware can process the WOL frame. o Changed *_ADDR_HIADDR to *_ADDR_HI to match other definitions of registers. o Added definitioan to interrupt moderation related constants. o Redefined SF_INTRS to include Tx DMA done and DMA errors. Removed Tx done as it's not needed anymore. o Added definition for Rx/Tx DMA high priority threshold. o Nuked unused marco SF_IDX_LO, SF_IDX_HI. o Added complete MAC statistic register definition. o Modified sf_stats structure to hold all MAC statistic regiters. o Nuke various driver private padding data in Tx/Rx descriptor definition. sf(4) no longer requires private padding. Also remove unused padding related definitions. This greatly simplifies descriptor manipulation on 64bit architectures. o Becase we no longer pad driver private data into descriptor, remove deprecated/not-applicable comments for padding. o Redefine Rx/Tx desciptor status. sf(4) doesn't use bit fileds anymore to support endianness. Tested by: bruffer (initial version) --- sys/dev/sf/if_sf.c | 2598 +++++++++++++++++++++++++++++------------ sys/dev/sf/if_sfreg.h | 571 ++++----- 2 files changed, 2163 insertions(+), 1006 deletions(-) diff --git a/sys/dev/sf/if_sf.c b/sys/dev/sf/if_sf.c index 37e26bd1f41..b1c35b7b38b 100644 --- a/sys/dev/sf/if_sf.c +++ b/sys/dev/sf/if_sf.c @@ -49,8 +49,8 @@ __FBSDID("$FreeBSD$"); * producer/consumer index mechanism and several different buffer * queue and completion queue descriptor types. Any one of a number * of different driver designs can be used, depending on system and - * OS requirements. This driver makes use of type0 transmit frame - * descriptors (since BSD fragments packets across an mbuf chain) + * OS requirements. This driver makes use of type2 transmit frame + * descriptors to take full advantage of fragmented packets buffers * and two RX buffer queues prioritized on size (one queue for small * frames that will fit into a single mbuf, another with full size * mbuf clusters for everything else). The producer/consumer indexes @@ -59,8 +59,9 @@ __FBSDID("$FreeBSD$"); * One downside to the Starfire has to do with alignment: buffer * queues must be aligned on 256-byte boundaries, and receive buffers * must be aligned on longword boundaries. The receive buffer alignment - * causes problems on the Alpha platform, where the packet payload - * should be longword aligned. There is no simple way around this. + * causes problems on the strict alignment architecture, where the + * packet payload should be longword aligned. There is no simple way + * around this. * * For receive filtering, the Starfire offers 16 perfect filter slots * and a 512-bit hash table. @@ -85,105 +86,122 @@ __FBSDID("$FreeBSD$"); #include #include -#include -#include -#include +#include +#include #include +#include +#include +#include #include #include +#include +#include +#include +#include #include #include #include #include #include #include - -#include - -#include /* for vtophys */ -#include /* for vtophys */ -#include -#include -#include -#include +#include #include #include -/* "device miibus" required. See GENERIC if you get errors here. */ -#include "miibus_if.h" - #include #include -#define SF_USEIOSPACE +#include #include +#include +#include + +/* "device miibus" required. See GENERIC if you get errors here. */ +#include "miibus_if.h" MODULE_DEPEND(sf, pci, 1, 1, 1); MODULE_DEPEND(sf, ether, 1, 1, 1); MODULE_DEPEND(sf, miibus, 1, 1, 1); +#undef SF_GFP_DEBUG +#define SF_CSUM_FEATURES (CSUM_TCP | CSUM_UDP) +/* Define this to activate partial TCP/UDP checksum offload. */ +#undef SF_PARTIAL_CSUM_SUPPORT + static struct sf_type sf_devs[] = { - { AD_VENDORID, AD_DEVICEID_STARFIRE, - "Adaptec AIC-6915 10/100BaseTX" }, - { 0, 0, NULL } + { AD_VENDORID, AD_DEVICEID_STARFIRE, "Adaptec AIC-6915 10/100BaseTX", + AD_SUBSYSID_62011_REV0, "Adaptec ANA-62011 (rev 0) 10/100BaseTX" }, + { AD_VENDORID, AD_DEVICEID_STARFIRE, "Adaptec AIC-6915 10/100BaseTX", + AD_SUBSYSID_62011_REV1, "Adaptec ANA-62011 (rev 1) 10/100BaseTX" }, + { AD_VENDORID, AD_DEVICEID_STARFIRE, "Adaptec AIC-6915 10/100BaseTX", + AD_SUBSYSID_62022, "Adaptec ANA-62022 10/100BaseTX" }, + { AD_VENDORID, AD_DEVICEID_STARFIRE, "Adaptec AIC-6915 10/100BaseTX", + AD_SUBSYSID_62044_REV0, "Adaptec ANA-62044 (rev 0) 10/100BaseTX" }, + { AD_VENDORID, AD_DEVICEID_STARFIRE, "Adaptec AIC-6915 10/100BaseTX", + AD_SUBSYSID_62044_REV1, "Adaptec ANA-62044 (rev 1) 10/100BaseTX" }, + { AD_VENDORID, AD_DEVICEID_STARFIRE, "Adaptec AIC-6915 10/100BaseTX", + AD_SUBSYSID_62020, "Adaptec ANA-62020 10/100BaseFX" }, + { AD_VENDORID, AD_DEVICEID_STARFIRE, "Adaptec AIC-6915 10/100BaseTX", + AD_SUBSYSID_69011, "Adaptec ANA-69011 10/100BaseTX" }, }; static int sf_probe(device_t); static int sf_attach(device_t); static int sf_detach(device_t); +static int sf_shutdown(device_t); +static int sf_suspend(device_t); +static int sf_resume(device_t); static void sf_intr(void *); -static void sf_stats_update(void *); +static void sf_tick(void *); +static void sf_stats_update(struct sf_softc *); +#ifndef __NO_STRICT_ALIGNMENT +static __inline void sf_fixup_rx(struct mbuf *); +#endif static void sf_rxeof(struct sf_softc *); static void sf_txeof(struct sf_softc *); -static int sf_encap(struct sf_softc *, struct sf_tx_bufdesc_type0 *, - struct mbuf *); +static int sf_encap(struct sf_softc *, struct mbuf **); static void sf_start(struct ifnet *); static void sf_start_locked(struct ifnet *); static int sf_ioctl(struct ifnet *, u_long, caddr_t); +static void sf_download_fw(struct sf_softc *); static void sf_init(void *); static void sf_init_locked(struct sf_softc *); static void sf_stop(struct sf_softc *); -static void sf_watchdog(struct ifnet *); -static int sf_shutdown(device_t); +static void sf_watchdog(struct sf_softc *); static int sf_ifmedia_upd(struct ifnet *); -static void sf_ifmedia_upd_locked(struct ifnet *); static void sf_ifmedia_sts(struct ifnet *, struct ifmediareq *); static void sf_reset(struct sf_softc *); +static int sf_dma_alloc(struct sf_softc *); +static void sf_dma_free(struct sf_softc *); static int sf_init_rx_ring(struct sf_softc *); static void sf_init_tx_ring(struct sf_softc *); -static int sf_newbuf(struct sf_softc *, struct sf_rx_bufdesc_type0 *, - struct mbuf *); -static void sf_setmulti(struct sf_softc *); -static int sf_setperf(struct sf_softc *, int, caddr_t); +static int sf_newbuf(struct sf_softc *, int); +static void sf_rxfilter(struct sf_softc *); +static int sf_setperf(struct sf_softc *, int, uint8_t *); static int sf_sethash(struct sf_softc *, caddr_t, int); #ifdef notdef -static int sf_setvlan(struct sf_softc *, int, u_int32_t); +static int sf_setvlan(struct sf_softc *, int, uint32_t); #endif -static u_int8_t sf_read_eeprom(struct sf_softc *, int); +static uint8_t sf_read_eeprom(struct sf_softc *, int); static int sf_miibus_readreg(device_t, int, int); static int sf_miibus_writereg(device_t, int, int, int); static void sf_miibus_statchg(device_t); +static void sf_link_task(void *, int); #ifdef DEVICE_POLLING static void sf_poll(struct ifnet *ifp, enum poll_cmd cmd, int count); -static void sf_poll_locked(struct ifnet *ifp, enum poll_cmd cmd, int count); #endif -static u_int32_t csr_read_4(struct sf_softc *, int); -static void csr_write_4(struct sf_softc *, int, u_int32_t); +static uint32_t csr_read_4(struct sf_softc *, int); +static void csr_write_4(struct sf_softc *, int, uint32_t); static void sf_txthresh_adjust(struct sf_softc *); - -#ifdef SF_USEIOSPACE -#define SF_RES SYS_RES_IOPORT -#define SF_RID SF_PCI_LOIO -#else -#define SF_RES SYS_RES_MEMORY -#define SF_RID SF_PCI_LOMEM -#endif +static int sf_sysctl_stats(SYSCTL_HANDLER_ARGS); +static int sysctl_int_range(SYSCTL_HANDLER_ARGS, int, int); +static int sysctl_hw_sf_int_mod(SYSCTL_HANDLER_ARGS); static device_method_t sf_methods[] = { /* Device interface */ @@ -191,6 +209,8 @@ static device_method_t sf_methods[] = { DEVMETHOD(device_attach, sf_attach), DEVMETHOD(device_detach, sf_detach), DEVMETHOD(device_shutdown, sf_shutdown), + DEVMETHOD(device_suspend, sf_suspend), + DEVMETHOD(device_resume, sf_resume), /* bus interface */ DEVMETHOD(bus_print_child, bus_generic_print_child), @@ -201,7 +221,7 @@ static device_method_t sf_methods[] = { DEVMETHOD(miibus_writereg, sf_miibus_writereg), DEVMETHOD(miibus_statchg, sf_miibus_statchg), - { 0, 0 } + { NULL, NULL } }; static driver_t sf_driver = { @@ -221,48 +241,42 @@ DRIVER_MODULE(miibus, sf, miibus_driver, miibus_devclass, 0, 0); #define SF_CLRBIT(sc, reg, x) \ csr_write_4(sc, reg, csr_read_4(sc, reg) & ~(x)) -static u_int32_t -csr_read_4(sc, reg) - struct sf_softc *sc; - int reg; +static uint32_t +csr_read_4(struct sf_softc *sc, int reg) { - u_int32_t val; + uint32_t val; -#ifdef SF_USEIOSPACE - CSR_WRITE_4(sc, SF_INDIRECTIO_ADDR, reg + SF_RMAP_INTREG_BASE); - val = CSR_READ_4(sc, SF_INDIRECTIO_DATA); -#else - val = CSR_READ_4(sc, (reg + SF_RMAP_INTREG_BASE)); -#endif + if (sc->sf_restype == SYS_RES_MEMORY) + val = CSR_READ_4(sc, (reg + SF_RMAP_INTREG_BASE)); + else { + CSR_WRITE_4(sc, SF_INDIRECTIO_ADDR, reg + SF_RMAP_INTREG_BASE); + val = CSR_READ_4(sc, SF_INDIRECTIO_DATA); + } - return(val); + return (val); } -static u_int8_t -sf_read_eeprom(sc, reg) - struct sf_softc *sc; - int reg; +static uint8_t +sf_read_eeprom(struct sf_softc *sc, int reg) { - u_int8_t val; + uint8_t val; val = (csr_read_4(sc, SF_EEADDR_BASE + (reg & 0xFFFFFFFC)) >> (8 * (reg & 3))) & 0xFF; - return(val); + return (val); } static void -csr_write_4(sc, reg, val) - struct sf_softc *sc; - int reg; - u_int32_t val; +csr_write_4(struct sf_softc *sc, int reg, uint32_t val) { -#ifdef SF_USEIOSPACE - CSR_WRITE_4(sc, SF_INDIRECTIO_ADDR, reg + SF_RMAP_INTREG_BASE); - CSR_WRITE_4(sc, SF_INDIRECTIO_DATA, val); -#else - CSR_WRITE_4(sc, (reg + SF_RMAP_INTREG_BASE), val); -#endif + + if (sc->sf_restype == SYS_RES_MEMORY) + CSR_WRITE_4(sc, (reg + SF_RMAP_INTREG_BASE), val); + else { + CSR_WRITE_4(sc, SF_INDIRECTIO_ADDR, reg + SF_RMAP_INTREG_BASE); + CSR_WRITE_4(sc, SF_INDIRECTIO_DATA, val); + } } /* @@ -271,29 +285,23 @@ csr_write_4(sc, reg, val) * some sanity tests. */ static int -sf_setperf(sc, idx, mac) - struct sf_softc *sc; - int idx; - caddr_t mac; +sf_setperf(struct sf_softc *sc, int idx, uint8_t *mac) { - u_int16_t *p; if (idx < 0 || idx > SF_RXFILT_PERFECT_CNT) - return(EINVAL); + return (EINVAL); if (mac == NULL) - return(EINVAL); - - p = (u_int16_t *)mac; + return (EINVAL); csr_write_4(sc, SF_RXFILT_PERFECT_BASE + - (idx * SF_RXFILT_PERFECT_SKIP), htons(p[2])); + (idx * SF_RXFILT_PERFECT_SKIP) + 0, mac[5] | (mac[4] << 8)); csr_write_4(sc, SF_RXFILT_PERFECT_BASE + - (idx * SF_RXFILT_PERFECT_SKIP) + 4, htons(p[1])); + (idx * SF_RXFILT_PERFECT_SKIP) + 4, mac[3] | (mac[2] << 8)); csr_write_4(sc, SF_RXFILT_PERFECT_BASE + - (idx * SF_RXFILT_PERFECT_SKIP) + 8, htons(p[0])); + (idx * SF_RXFILT_PERFECT_SKIP) + 8, mac[1] | (mac[0] << 8)); - return(0); + return (0); } /* @@ -302,15 +310,12 @@ sf_setperf(sc, idx, mac) * priority hash table instead of the filter hash table. */ static int -sf_sethash(sc, mac, prio) - struct sf_softc *sc; - caddr_t mac; - int prio; +sf_sethash(struct sf_softc *sc, caddr_t mac, int prio) { - u_int32_t h; + uint32_t h; if (mac == NULL) - return(EINVAL); + return (EINVAL); h = ether_crc32_be(mac, ETHER_ADDR_LEN) >> 23; @@ -322,7 +327,7 @@ sf_sethash(sc, mac, prio) (SF_RXFILT_HASH_SKIP * (h >> 4)), (1 << (h & 0xF))); } - return(0); + return (0); } #ifdef notdef @@ -330,51 +335,46 @@ sf_sethash(sc, mac, prio) * Set a VLAN tag in the receive filter. */ static int -sf_setvlan(sc, idx, vlan) - struct sf_softc *sc; - int idx; - u_int32_t vlan; +sf_setvlan(struct sf_softc *sc, int idx, uint32_t vlan) { + if (idx < 0 || idx >> SF_RXFILT_HASH_CNT) - return(EINVAL); + return (EINVAL); csr_write_4(sc, SF_RXFILT_HASH_BASE + (idx * SF_RXFILT_HASH_SKIP) + SF_RXFILT_HASH_VLANOFF, vlan); - return(0); + return (0); } #endif static int -sf_miibus_readreg(dev, phy, reg) - device_t dev; - int phy, reg; +sf_miibus_readreg(device_t dev, int phy, int reg) { struct sf_softc *sc; int i; - u_int32_t val = 0; + uint32_t val = 0; sc = device_get_softc(dev); for (i = 0; i < SF_TIMEOUT; i++) { val = csr_read_4(sc, SF_PHY_REG(phy, reg)); - if (val & SF_MII_DATAVALID) + if ((val & SF_MII_DATAVALID) != 0) break; } if (i == SF_TIMEOUT) - return(0); + return (0); - if ((val & 0x0000FFFF) == 0xFFFF) - return(0); + val &= SF_MII_DATAPORT; + if (val == 0xffff) + return (0); - return(val & 0x0000FFFF); + return (val); } static int -sf_miibus_writereg(dev, phy, reg, val) - device_t dev; - int phy, reg, val; +sf_miibus_writereg(device_t dev, int phy, int reg, int val) { struct sf_softc *sc; int i; @@ -386,122 +386,170 @@ sf_miibus_writereg(dev, phy, reg, val) for (i = 0; i < SF_TIMEOUT; i++) { busy = csr_read_4(sc, SF_PHY_REG(phy, reg)); - if (!(busy & SF_MII_BUSY)) + if ((busy & SF_MII_BUSY) == 0) break; } - return(0); + return (0); } static void -sf_miibus_statchg(dev) - device_t dev; +sf_miibus_statchg(device_t dev) { struct sf_softc *sc; - struct mii_data *mii; sc = device_get_softc(dev); + taskqueue_enqueue(taskqueue_swi, &sc->sf_link_task); +} + +static void +sf_link_task(void *arg, int pending) +{ + struct sf_softc *sc; + struct mii_data *mii; + struct ifnet *ifp; + uint32_t val; + + sc = (struct sf_softc *)arg; + + SF_LOCK(sc); + mii = device_get_softc(sc->sf_miibus); + ifp = sc->sf_ifp; + if (mii == NULL || ifp == NULL || + (ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) { + SF_UNLOCK(sc); + return; + } - if ((mii->mii_media_active & IFM_GMASK) == IFM_FDX) { - SF_SETBIT(sc, SF_MACCFG_1, SF_MACCFG1_FULLDUPLEX); + if (mii->mii_media_status & IFM_ACTIVE) { + if (IFM_SUBTYPE(mii->mii_media_active) != IFM_NONE) + sc->sf_link = 1; + } else + sc->sf_link = 0; + + val = csr_read_4(sc, SF_MACCFG_1); + val &= ~SF_MACCFG1_FULLDUPLEX; + val &= ~(SF_MACCFG1_RX_FLOWENB | SF_MACCFG1_TX_FLOWENB); + if ((IFM_OPTIONS(mii->mii_media_active) & IFM_FDX) != 0) { + val |= SF_MACCFG1_FULLDUPLEX; csr_write_4(sc, SF_BKTOBKIPG, SF_IPGT_FDX); - } else { - SF_CLRBIT(sc, SF_MACCFG_1, SF_MACCFG1_FULLDUPLEX); +#ifdef notyet + /* Configure flow-control bits. */ + if ((IFM_OPTIONS(sc->sc_mii->mii_media_active) & + IFM_ETH_RXPAUSE) != 0) + val |= SF_MACCFG1_RX_FLOWENB; + if ((IFM_OPTIONS(sc->sc_mii->mii_media_active) & + IFM_ETH_TXPAUSE) != 0) + val |= SF_MACCFG1_TX_FLOWENB; +#endif + } else csr_write_4(sc, SF_BKTOBKIPG, SF_IPGT_HDX); - } + + /* Make sure to reset MAC to take changes effect. */ + csr_write_4(sc, SF_MACCFG_1, val | SF_MACCFG1_SOFTRESET); + DELAY(1000); + csr_write_4(sc, SF_MACCFG_1, val); + + val = csr_read_4(sc, SF_TIMER_CTL); + if (IFM_SUBTYPE(mii->mii_media_active) == IFM_100_TX) + val |= SF_TIMER_TIMES_TEN; + else + val &= ~SF_TIMER_TIMES_TEN; + csr_write_4(sc, SF_TIMER_CTL, val); + + SF_UNLOCK(sc); } static void -sf_setmulti(sc) - struct sf_softc *sc; +sf_rxfilter(struct sf_softc *sc) { struct ifnet *ifp; int i; struct ifmultiaddr *ifma; - u_int8_t dummy[] = { 0, 0, 0, 0, 0, 0 }; + uint8_t dummy[ETHER_ADDR_LEN] = { 0, 0, 0, 0, 0, 0 }; + uint32_t rxfilt; ifp = sc->sf_ifp; /* First zot all the existing filters. */ for (i = 1; i < SF_RXFILT_PERFECT_CNT; i++) - sf_setperf(sc, i, (char *)&dummy); - for (i = SF_RXFILT_HASH_BASE; - i < (SF_RXFILT_HASH_MAX + 1); i += 4) + sf_setperf(sc, i, dummy); + for (i = SF_RXFILT_HASH_BASE; i < (SF_RXFILT_HASH_MAX + 1); + i += sizeof(uint32_t)) csr_write_4(sc, i, 0); - SF_CLRBIT(sc, SF_RXFILT, SF_RXFILT_ALLMULTI); - /* Now program new ones. */ - if (ifp->if_flags & IFF_ALLMULTI || ifp->if_flags & IFF_PROMISC) { - SF_SETBIT(sc, SF_RXFILT, SF_RXFILT_ALLMULTI); - } else { - i = 1; - IF_ADDR_LOCK(ifp); - TAILQ_FOREACH_REVERSE(ifma, &ifp->if_multiaddrs, ifmultihead, ifma_link) { - if (ifma->ifma_addr->sa_family != AF_LINK) - continue; - /* - * Program the first 15 multicast groups - * into the perfect filter. For all others, - * use the hash table. - */ - if (i < SF_RXFILT_PERFECT_CNT) { - sf_setperf(sc, i, - LLADDR((struct sockaddr_dl *)ifma->ifma_addr)); - i++; - continue; - } + rxfilt = csr_read_4(sc, SF_RXFILT); + rxfilt &= ~(SF_RXFILT_PROMISC | SF_RXFILT_ALLMULTI | SF_RXFILT_BROAD); + if ((ifp->if_flags & IFF_BROADCAST) != 0) + rxfilt |= SF_RXFILT_BROAD; + if ((ifp->if_flags & IFF_ALLMULTI) != 0 || + (ifp->if_flags & IFF_PROMISC) != 0) { + if ((ifp->if_flags & IFF_PROMISC) != 0) + rxfilt |= SF_RXFILT_PROMISC; + if ((ifp->if_flags & IFF_ALLMULTI) != 0) + rxfilt |= SF_RXFILT_ALLMULTI; + goto done; + } - sf_sethash(sc, - LLADDR((struct sockaddr_dl *)ifma->ifma_addr), 0); + /* Now program new ones. */ + i = 1; + IF_ADDR_LOCK(ifp); + TAILQ_FOREACH_REVERSE(ifma, &ifp->if_multiaddrs, ifmultihead, + ifma_link) { + if (ifma->ifma_addr->sa_family != AF_LINK) + continue; + /* + * Program the first 15 multicast groups + * into the perfect filter. For all others, + * use the hash table. + */ + if (i < SF_RXFILT_PERFECT_CNT) { + sf_setperf(sc, i, + LLADDR((struct sockaddr_dl *)ifma->ifma_addr)); + i++; + continue; } - IF_ADDR_UNLOCK(ifp); + + sf_sethash(sc, + LLADDR((struct sockaddr_dl *)ifma->ifma_addr), 0); } + IF_ADDR_UNLOCK(ifp); + +done: + csr_write_4(sc, SF_RXFILT, rxfilt); } /* * Set media options. */ static int -sf_ifmedia_upd(ifp) - struct ifnet *ifp; +sf_ifmedia_upd(struct ifnet *ifp) { struct sf_softc *sc; + struct mii_data *mii; + int error; sc = ifp->if_softc; SF_LOCK(sc); - sf_ifmedia_upd_locked(ifp); - SF_UNLOCK(sc); - - return(0); -} - -static void -sf_ifmedia_upd_locked(ifp) - struct ifnet *ifp; -{ - struct sf_softc *sc; - struct mii_data *mii; - sc = ifp->if_softc; mii = device_get_softc(sc->sf_miibus); - SF_LOCK_ASSERT(sc); - sc->sf_link = 0; if (mii->mii_instance) { struct mii_softc *miisc; LIST_FOREACH(miisc, &mii->mii_phys, mii_list) mii_phy_reset(miisc); } - mii_mediachg(mii); + error = mii_mediachg(mii); + SF_UNLOCK(sc); + + return (error); } /* * Report current media status. */ static void -sf_ifmedia_sts(ifp, ifmr) - struct ifnet *ifp; - struct ifmediareq *ifmr; +sf_ifmedia_sts(struct ifnet *ifp, struct ifmediareq *ifmr) { struct sf_softc *sc; struct mii_data *mii; @@ -517,44 +565,41 @@ sf_ifmedia_sts(ifp, ifmr) } static int -sf_ioctl(ifp, command, data) - struct ifnet *ifp; - u_long command; - caddr_t data; +sf_ioctl(struct ifnet *ifp, u_long command, caddr_t data) { - struct sf_softc *sc = ifp->if_softc; - struct ifreq *ifr = (struct ifreq *) data; + struct sf_softc *sc; + struct ifreq *ifr; struct mii_data *mii; - int error = 0; + int error, mask; - switch(command) { + sc = ifp->if_softc; + ifr = (struct ifreq *)data; + error = 0; + + switch (command) { case SIOCSIFFLAGS: SF_LOCK(sc); if (ifp->if_flags & IFF_UP) { - if (ifp->if_drv_flags & IFF_DRV_RUNNING && - ifp->if_flags & IFF_PROMISC && - !(sc->sf_if_flags & IFF_PROMISC)) { - SF_SETBIT(sc, SF_RXFILT, SF_RXFILT_PROMISC); - } else if (ifp->if_drv_flags & IFF_DRV_RUNNING && - !(ifp->if_flags & IFF_PROMISC) && - sc->sf_if_flags & IFF_PROMISC) { - SF_CLRBIT(sc, SF_RXFILT, SF_RXFILT_PROMISC); - } else if (!(ifp->if_drv_flags & IFF_DRV_RUNNING)) - sf_init_locked(sc); + if ((ifp->if_drv_flags & IFF_DRV_RUNNING) != 0) { + if ((ifp->if_flags ^ sc->sf_if_flags) & + (IFF_PROMISC | IFF_ALLMULTI)) + sf_rxfilter(sc); + } else { + if (sc->sf_detach == 0) + sf_init_locked(sc); + } } else { - if (ifp->if_drv_flags & IFF_DRV_RUNNING) + if ((ifp->if_drv_flags & IFF_DRV_RUNNING) != 0) sf_stop(sc); } sc->sf_if_flags = ifp->if_flags; SF_UNLOCK(sc); - error = 0; break; case SIOCADDMULTI: case SIOCDELMULTI: SF_LOCK(sc); - sf_setmulti(sc); + sf_rxfilter(sc); SF_UNLOCK(sc); - error = 0; break; case SIOCGIFMEDIA: case SIOCSIFMEDIA: @@ -562,45 +607,70 @@ sf_ioctl(ifp, command, data) error = ifmedia_ioctl(ifp, ifr, &mii->mii_media, command); break; case SIOCSIFCAP: + mask = ifr->ifr_reqcap ^ ifp->if_capenable; #ifdef DEVICE_POLLING - if (ifr->ifr_reqcap & IFCAP_POLLING && - !(ifp->if_capenable & IFCAP_POLLING)) { - error = ether_poll_register(sf_poll, ifp); - if (error) - return(error); - SF_LOCK(sc); - /* Disable interrupts */ - csr_write_4(sc, SF_IMR, 0x00000000); - ifp->if_capenable |= IFCAP_POLLING; - SF_UNLOCK(sc); - return (error); - - } - if (!(ifr->ifr_reqcap & IFCAP_POLLING) && - ifp->if_capenable & IFCAP_POLLING) { - error = ether_poll_deregister(ifp); - /* Enable interrupts. */ - SF_LOCK(sc); - csr_write_4(sc, SF_IMR, SF_INTRS); - ifp->if_capenable &= ~IFCAP_POLLING; - SF_UNLOCK(sc); - return (error); + if ((mask & IFCAP_POLLING) != 0) { + if ((ifr->ifr_reqcap & IFCAP_POLLING) != 0) { + error = ether_poll_register(sf_poll, ifp); + if (error != 0) + break; + SF_LOCK(sc); + /* Disable interrupts. */ + csr_write_4(sc, SF_IMR, 0); + ifp->if_capenable |= IFCAP_POLLING; + SF_UNLOCK(sc); + } else { + error = ether_poll_deregister(ifp); + /* Enable interrupts. */ + SF_LOCK(sc); + csr_write_4(sc, SF_IMR, SF_INTRS); + ifp->if_capenable &= ~IFCAP_POLLING; + SF_UNLOCK(sc); + } } #endif /* DEVICE_POLLING */ + if ((mask & IFCAP_TXCSUM) != 0) { + if ((IFCAP_TXCSUM & ifp->if_capabilities) != 0) { + SF_LOCK(sc); + ifp->if_capenable ^= IFCAP_TXCSUM; + if ((IFCAP_TXCSUM & ifp->if_capenable) != 0) { + ifp->if_hwassist |= SF_CSUM_FEATURES; + SF_SETBIT(sc, SF_GEN_ETH_CTL, + SF_ETHCTL_TXGFP_ENB); + } else { + ifp->if_hwassist &= ~SF_CSUM_FEATURES; + SF_CLRBIT(sc, SF_GEN_ETH_CTL, + SF_ETHCTL_TXGFP_ENB); + } + SF_UNLOCK(sc); + } + } + if ((mask & IFCAP_RXCSUM) != 0) { + if ((IFCAP_RXCSUM & ifp->if_capabilities) != 0) { + SF_LOCK(sc); + ifp->if_capenable ^= IFCAP_RXCSUM; + if ((IFCAP_RXCSUM & ifp->if_capenable) != 0) + SF_SETBIT(sc, SF_GEN_ETH_CTL, + SF_ETHCTL_RXGFP_ENB); + else + SF_CLRBIT(sc, SF_GEN_ETH_CTL, + SF_ETHCTL_RXGFP_ENB); + SF_UNLOCK(sc); + } + } break; default: error = ether_ioctl(ifp, command, data); break; } - return(error); + return (error); } static void -sf_reset(sc) - struct sf_softc *sc; +sf_reset(struct sf_softc *sc) { - register int i; + int i; csr_write_4(sc, SF_GEN_ETH_CTL, 0); SF_SETBIT(sc, SF_MACCFG_1, SF_MACCFG1_SOFTRESET); @@ -629,50 +699,35 @@ sf_reset(sc) * NIC has been found, if possible. */ static int -sf_probe(dev) - device_t dev; +sf_probe(device_t dev) { struct sf_type *t; + uint16_t vid; + uint16_t did; + uint16_t sdid; + int i; - t = sf_devs; + vid = pci_get_vendor(dev); + did = pci_get_device(dev); + sdid = pci_get_subdevice(dev); - while(t->sf_name != NULL) { - if ((pci_get_vendor(dev) == t->sf_vid) && - (pci_get_device(dev) == t->sf_did)) { - switch((pci_read_config(dev, - SF_PCI_SUBVEN_ID, 4) >> 16) & 0xFFFF) { - case AD_SUBSYSID_62011_REV0: - case AD_SUBSYSID_62011_REV1: - device_set_desc(dev, - "Adaptec ANA-62011 10/100BaseTX"); - return (BUS_PROBE_DEFAULT); - case AD_SUBSYSID_62022: - device_set_desc(dev, - "Adaptec ANA-62022 10/100BaseTX"); - return (BUS_PROBE_DEFAULT); - case AD_SUBSYSID_62044_REV0: - case AD_SUBSYSID_62044_REV1: - device_set_desc(dev, - "Adaptec ANA-62044 10/100BaseTX"); - return (BUS_PROBE_DEFAULT); - case AD_SUBSYSID_62020: - device_set_desc(dev, - "Adaptec ANA-62020 10/100BaseFX"); - return (BUS_PROBE_DEFAULT); - case AD_SUBSYSID_69011: - device_set_desc(dev, - "Adaptec ANA-69011 10/100BaseTX"); - return (BUS_PROBE_DEFAULT); - default: - device_set_desc(dev, t->sf_name); + t = sf_devs; + for (i = 0; i < sizeof(sf_devs) / sizeof(sf_devs[0]); i++, t++) { + if (vid == t->sf_vid && did == t->sf_did) { + if (sdid == t->sf_sdid) { + device_set_desc(dev, t->sf_sname); return (BUS_PROBE_DEFAULT); - break; } } - t++; } - return(ENXIO); + if (vid == AD_VENDORID && did == AD_DEVICEID_STARFIRE) { + /* unkown subdevice */ + device_set_desc(dev, sf_devs[0].sf_name); + return (BUS_PROBE_DEFAULT); + } + + return (ENXIO); } /* @@ -680,38 +735,78 @@ sf_probe(dev) * setup and ethernet/BPF attach. */ static int -sf_attach(dev) - device_t dev; +sf_attach(device_t dev) { int i; struct sf_softc *sc; struct ifnet *ifp; + uint32_t reg; int rid, error = 0; - u_char eaddr[6]; + uint8_t eaddr[ETHER_ADDR_LEN]; sc = device_get_softc(dev); sc->sf_dev = dev; mtx_init(&sc->sf_mtx, device_get_nameunit(dev), MTX_NETWORK_LOCK, MTX_DEF); + callout_init_mtx(&sc->sf_co, &sc->sf_mtx, 0); + TASK_INIT(&sc->sf_link_task, 0, sf_link_task, sc); + /* * Map control/status registers. */ pci_enable_busmaster(dev); - rid = SF_RID; - sc->sf_res = bus_alloc_resource_any(dev, SF_RES, &rid, RF_ACTIVE); - + /* + * Prefer memory space register mapping over I/O space as the + * hardware requires lots of register access to get various + * producer/consumer index during Tx/Rx operation. However this + * requires large memory space(512K) to map the entire register + * space. + */ + sc->sf_rid = PCIR_BAR(0); + sc->sf_restype = SYS_RES_MEMORY; + sc->sf_res = bus_alloc_resource_any(dev, sc->sf_restype, &sc->sf_rid, + RF_ACTIVE); if (sc->sf_res == NULL) { - device_printf(dev, "couldn't map ports\n"); - error = ENXIO; - goto fail; + reg = pci_read_config(dev, PCIR_BAR(0), 4); + if ((reg & PCIM_BAR_MEM_64) == PCIM_BAR_MEM_64) + sc->sf_rid = PCIR_BAR(2); + else + sc->sf_rid = PCIR_BAR(1); + sc->sf_restype = SYS_RES_IOPORT; + sc->sf_res = bus_alloc_resource_any(dev, sc->sf_restype, + &sc->sf_rid, RF_ACTIVE); + if (sc->sf_res == NULL) { + device_printf(dev, "couldn't allocate resources\n"); + mtx_destroy(&sc->sf_mtx); + return (ENXIO); + } } + if (bootverbose) + device_printf(dev, "using %s space register mapping\n", + sc->sf_restype == SYS_RES_MEMORY ? "memory" : "I/O"); - sc->sf_btag = rman_get_bustag(sc->sf_res); - sc->sf_bhandle = rman_get_bushandle(sc->sf_res); + reg = pci_read_config(dev, PCIR_CACHELNSZ, 1); + if (reg == 0) { + /* + * If cache line size is 0, MWI is not used at all, so set + * reasonable default. AIC-6915 supports 0, 4, 8, 16, 32 + * and 64. + */ + reg = 16; + device_printf(dev, "setting PCI cache line size to %u\n", reg); + pci_write_config(dev, PCIR_CACHELNSZ, reg, 1); + } else { + if (bootverbose) + device_printf(dev, "PCI cache line size : %u\n", reg); + } + /* Enable MWI. */ + reg = pci_read_config(dev, PCIR_COMMAND, 2); + reg |= PCIM_CMD_MWRICEN; + pci_write_config(dev, PCIR_COMMAND, reg, 2); - /* Allocate interrupt */ + /* Allocate interrupt. */ rid = 0; sc->sf_irq = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE); @@ -722,7 +817,28 @@ sf_attach(dev) goto fail; } - callout_init_mtx(&sc->sf_stat_callout, &sc->sf_mtx, 0); + SYSCTL_ADD_PROC(device_get_sysctl_ctx(dev), + SYSCTL_CHILDREN(device_get_sysctl_tree(dev)), + OID_AUTO, "stats", CTLTYPE_INT | CTLFLAG_RW, sc, 0, + sf_sysctl_stats, "I", "Statistics"); + + SYSCTL_ADD_PROC(device_get_sysctl_ctx(dev), + SYSCTL_CHILDREN(device_get_sysctl_tree(dev)), + OID_AUTO, "int_mod", CTLTYPE_INT | CTLFLAG_RW, + &sc->sf_int_mod, 0, sysctl_hw_sf_int_mod, "I", + "sf interrupt moderation"); + /* Pull in device tunables. */ + sc->sf_int_mod = SF_IM_DEFAULT; + error = resource_int_value(device_get_name(dev), device_get_unit(dev), + "int_mod", &sc->sf_int_mod); + if (error == 0) { + if (sc->sf_int_mod < SF_IM_MIN || + sc->sf_int_mod > SF_IM_MAX) { + device_printf(dev, "int_mod value out of range; " + "using default: %d\n", SF_IM_DEFAULT); + sc->sf_int_mod = SF_IM_DEFAULT; + } + } /* Reset the adapter. */ sf_reset(sc); @@ -734,28 +850,24 @@ sf_attach(dev) eaddr[i] = sf_read_eeprom(sc, SF_EE_NODEADDR + ETHER_ADDR_LEN - i); - /* Allocate the descriptor queues. */ - sc->sf_ldata = contigmalloc(sizeof(struct sf_list_data), M_DEVBUF, - M_NOWAIT, 0, 0xffffffff, PAGE_SIZE, 0); - - if (sc->sf_ldata == NULL) { - device_printf(dev, "no memory for list buffers!\n"); - error = ENXIO; + /* Allocate DMA resources. */ + if (sf_dma_alloc(sc) != 0) { + error = ENOSPC; goto fail; } - bzero(sc->sf_ldata, sizeof(struct sf_list_data)); + sc->sf_txthresh = SF_MIN_TX_THRESHOLD; ifp = sc->sf_ifp = if_alloc(IFT_ETHER); if (ifp == NULL) { - device_printf(dev, "can not if_alloc()\n"); + device_printf(dev, "can not allocate ifnet structure\n"); error = ENOSPC; goto fail; } /* Do MII setup. */ - if (mii_phy_probe(dev, &sc->sf_miibus, - sf_ifmedia_upd, sf_ifmedia_sts)) { + if (mii_phy_probe(dev, &sc->sf_miibus, sf_ifmedia_upd, + sf_ifmedia_sts)) { device_printf(dev, "MII without any phy!\n"); error = ENXIO; goto fail; @@ -763,24 +875,37 @@ sf_attach(dev) ifp->if_softc = sc; if_initname(ifp, device_get_name(dev), device_get_unit(dev)); - ifp->if_mtu = ETHERMTU; ifp->if_flags = IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST; ifp->if_ioctl = sf_ioctl; ifp->if_start = sf_start; - ifp->if_watchdog = sf_watchdog; ifp->if_init = sf_init; IFQ_SET_MAXLEN(&ifp->if_snd, SF_TX_DLIST_CNT - 1); ifp->if_snd.ifq_drv_maxlen = SF_TX_DLIST_CNT - 1; IFQ_SET_READY(&ifp->if_snd); + /* + * With the help of firmware, AIC-6915 supports + * Tx/Rx TCP/UDP checksum offload. + */ + ifp->if_hwassist = SF_CSUM_FEATURES; + ifp->if_capabilities = IFCAP_HWCSUM; + + /* + * Call MI attach routine. + */ + ether_ifattach(ifp, eaddr); + + /* VLAN capability setup. */ + ifp->if_capabilities |= IFCAP_VLAN_MTU; ifp->if_capenable = ifp->if_capabilities; #ifdef DEVICE_POLLING ifp->if_capabilities |= IFCAP_POLLING; #endif - /* - * Call MI attach routine. + * Tell the upper layer(s) we support long frames. + * Must appear after the call to ether_ifattach() because + * ether_ifattach() sets ifi_hdrlen to the default value. */ - ether_ifattach(ifp, eaddr); + ifp->if_data.ifi_hdrlen = sizeof(struct ether_vlan_header); /* Hook interrupt last to avoid having to lock softc */ error = bus_setup_intr(dev, sc->sf_irq, INTR_TYPE_NET | INTR_MPSAFE, @@ -796,7 +921,7 @@ sf_attach(dev) if (error) sf_detach(dev); - return(error); + return (error); } /* @@ -807,212 +932,749 @@ sf_attach(dev) * allocated. */ static int -sf_detach(dev) - device_t dev; +sf_detach(device_t dev) { struct sf_softc *sc; struct ifnet *ifp; sc = device_get_softc(dev); - KASSERT(mtx_initialized(&sc->sf_mtx), ("sf mutex not initialized")); ifp = sc->sf_ifp; #ifdef DEVICE_POLLING - if (ifp->if_capenable & IFCAP_POLLING) + if (ifp != NULL && ifp->if_capenable & IFCAP_POLLING) ether_poll_deregister(ifp); #endif - + /* These should only be active if attach succeeded */ if (device_is_attached(dev)) { SF_LOCK(sc); + sc->sf_detach = 1; sf_stop(sc); SF_UNLOCK(sc); - callout_drain(&sc->sf_stat_callout); - ether_ifdetach(ifp); + callout_drain(&sc->sf_co); + taskqueue_drain(taskqueue_swi, &sc->sf_link_task); + if (ifp != NULL) + ether_ifdetach(ifp); } - if (sc->sf_miibus) + if (sc->sf_miibus) { device_delete_child(dev, sc->sf_miibus); + sc->sf_miibus = NULL; + } bus_generic_detach(dev); - if (sc->sf_intrhand) + if (sc->sf_intrhand != NULL) bus_teardown_intr(dev, sc->sf_irq, sc->sf_intrhand); - if (sc->sf_irq) + if (sc->sf_irq != NULL) bus_release_resource(dev, SYS_RES_IRQ, 0, sc->sf_irq); - if (sc->sf_res) - bus_release_resource(dev, SF_RES, SF_RID, sc->sf_res); + if (sc->sf_res != NULL) + bus_release_resource(dev, sc->sf_restype, sc->sf_rid, + sc->sf_res); - if (ifp) + sf_dma_free(sc); + if (ifp != NULL) if_free(ifp); - if (sc->sf_ldata) - contigfree(sc->sf_ldata, sizeof(struct sf_list_data), M_DEVBUF); - mtx_destroy(&sc->sf_mtx); - return(0); + return (0); } -static int -sf_init_rx_ring(sc) - struct sf_softc *sc; -{ - struct sf_list_data *ld; - int i; - - ld = sc->sf_ldata; - - bzero((char *)ld->sf_rx_dlist_big, - sizeof(struct sf_rx_bufdesc_type0) * SF_RX_DLIST_CNT); - bzero((char *)ld->sf_rx_clist, - sizeof(struct sf_rx_cmpdesc_type3) * SF_RX_CLIST_CNT); - - for (i = 0; i < SF_RX_DLIST_CNT; i++) { - if (sf_newbuf(sc, &ld->sf_rx_dlist_big[i], NULL) == ENOBUFS) - return(ENOBUFS); - } - - return(0); -} +struct sf_dmamap_arg { + bus_addr_t sf_busaddr; +}; static void -sf_init_tx_ring(sc) - struct sf_softc *sc; +sf_dmamap_cb(void *arg, bus_dma_segment_t *segs, int nseg, int error) { - struct sf_list_data *ld; - int i; - - ld = sc->sf_ldata; - - bzero((char *)ld->sf_tx_dlist, - sizeof(struct sf_tx_bufdesc_type0) * SF_TX_DLIST_CNT); - bzero((char *)ld->sf_tx_clist, - sizeof(struct sf_tx_cmpdesc_type0) * SF_TX_CLIST_CNT); + struct sf_dmamap_arg *ctx; - for (i = 0; i < SF_TX_DLIST_CNT; i++) - ld->sf_tx_dlist[i].sf_id = SF_TX_BUFDESC_ID; - for (i = 0; i < SF_TX_CLIST_CNT; i++) - ld->sf_tx_clist[i].sf_type = SF_TXCMPTYPE_TX; - - ld->sf_tx_dlist[SF_TX_DLIST_CNT - 1].sf_end = 1; - sc->sf_tx_cnt = 0; + if (error != 0) + return; + ctx = arg; + ctx->sf_busaddr = segs[0].ds_addr; } static int -sf_newbuf(sc, c, m) - struct sf_softc *sc; - struct sf_rx_bufdesc_type0 *c; - struct mbuf *m; +sf_dma_alloc(struct sf_softc *sc) { - struct mbuf *m_new = NULL; - - if (m == NULL) { - MGETHDR(m_new, M_DONTWAIT, MT_DATA); - if (m_new == NULL) - return(ENOBUFS); + struct sf_dmamap_arg ctx; + struct sf_txdesc *txd; + struct sf_rxdesc *rxd; + bus_addr_t lowaddr; + bus_addr_t rx_ring_end, rx_cring_end; + bus_addr_t tx_ring_end, tx_cring_end; + int error, i; + + lowaddr = BUS_SPACE_MAXADDR; + +again: + /* Create parent DMA tag. */ + error = bus_dma_tag_create( + bus_get_dma_tag(sc->sf_dev), /* parent */ + 1, 0, /* alignment, boundary */ + lowaddr, /* lowaddr */ + BUS_SPACE_MAXADDR, /* highaddr */ + NULL, NULL, /* filter, filterarg */ + BUS_SPACE_MAXSIZE_32BIT, /* maxsize */ + 0, /* nsegments */ + BUS_SPACE_MAXSIZE_32BIT, /* maxsegsize */ + 0, /* flags */ + NULL, NULL, /* lockfunc, lockarg */ + &sc->sf_cdata.sf_parent_tag); + if (error != 0) { + device_printf(sc->sf_dev, "failed to create parent DMA tag\n"); + goto fail; + } + /* Create tag for Tx ring. */ + error = bus_dma_tag_create(sc->sf_cdata.sf_parent_tag,/* parent */ + SF_RING_ALIGN, 0, /* alignment, boundary */ + BUS_SPACE_MAXADDR, /* lowaddr */ + BUS_SPACE_MAXADDR, /* highaddr */ + NULL, NULL, /* filter, filterarg */ + SF_TX_DLIST_SIZE, /* maxsize */ + 1, /* nsegments */ + SF_TX_DLIST_SIZE, /* maxsegsize */ + 0, /* flags */ + NULL, NULL, /* lockfunc, lockarg */ + &sc->sf_cdata.sf_tx_ring_tag); + if (error != 0) { + device_printf(sc->sf_dev, "failed to create Tx ring DMA tag\n"); + goto fail; + } - MCLGET(m_new, M_DONTWAIT); - if (!(m_new->m_flags & M_EXT)) { - m_freem(m_new); - return(ENOBUFS); - } - m_new->m_len = m_new->m_pkthdr.len = MCLBYTES; - } else { - m_new = m; - m_new->m_len = m_new->m_pkthdr.len = MCLBYTES; - m_new->m_data = m_new->m_ext.ext_buf; + /* Create tag for Tx completion ring. */ + error = bus_dma_tag_create(sc->sf_cdata.sf_parent_tag,/* parent */ + SF_RING_ALIGN, 0, /* alignment, boundary */ + BUS_SPACE_MAXADDR, /* lowaddr */ + BUS_SPACE_MAXADDR, /* highaddr */ + NULL, NULL, /* filter, filterarg */ + SF_TX_CLIST_SIZE, /* maxsize */ + 1, /* nsegments */ + SF_TX_CLIST_SIZE, /* maxsegsize */ + 0, /* flags */ + NULL, NULL, /* lockfunc, lockarg */ + &sc->sf_cdata.sf_tx_cring_tag); + if (error != 0) { + device_printf(sc->sf_dev, + "failed to create Tx completion ring DMA tag\n"); + goto fail; } - m_adj(m_new, sizeof(u_int64_t)); + /* Create tag for Rx ring. */ + error = bus_dma_tag_create(sc->sf_cdata.sf_parent_tag,/* parent */ + SF_RING_ALIGN, 0, /* alignment, boundary */ + BUS_SPACE_MAXADDR, /* lowaddr */ + BUS_SPACE_MAXADDR, /* highaddr */ + NULL, NULL, /* filter, filterarg */ + SF_RX_DLIST_SIZE, /* maxsize */ + 1, /* nsegments */ + SF_RX_DLIST_SIZE, /* maxsegsize */ + 0, /* flags */ + NULL, NULL, /* lockfunc, lockarg */ + &sc->sf_cdata.sf_rx_ring_tag); + if (error != 0) { + device_printf(sc->sf_dev, + "failed to create Rx ring DMA tag\n"); + goto fail; + } - c->sf_mbuf = m_new; - c->sf_addrlo = SF_RX_HOSTADDR(vtophys(mtod(m_new, caddr_t))); - c->sf_valid = 1; + /* Create tag for Rx completion ring. */ + error = bus_dma_tag_create(sc->sf_cdata.sf_parent_tag,/* parent */ + SF_RING_ALIGN, 0, /* alignment, boundary */ + BUS_SPACE_MAXADDR, /* lowaddr */ + BUS_SPACE_MAXADDR, /* highaddr */ + NULL, NULL, /* filter, filterarg */ + SF_RX_CLIST_SIZE, /* maxsize */ + 1, /* nsegments */ + SF_RX_CLIST_SIZE, /* maxsegsize */ + 0, /* flags */ + NULL, NULL, /* lockfunc, lockarg */ + &sc->sf_cdata.sf_rx_cring_tag); + if (error != 0) { + device_printf(sc->sf_dev, + "failed to create Rx completion ring DMA tag\n"); + goto fail; + } - return(0); -} + /* Create tag for Tx buffers. */ + error = bus_dma_tag_create(sc->sf_cdata.sf_parent_tag,/* parent */ + 1, 0, /* alignment, boundary */ + BUS_SPACE_MAXADDR, /* lowaddr */ + BUS_SPACE_MAXADDR, /* highaddr */ + NULL, NULL, /* filter, filterarg */ + MCLBYTES * SF_MAXTXSEGS, /* maxsize */ + SF_MAXTXSEGS, /* nsegments */ + MCLBYTES, /* maxsegsize */ + 0, /* flags */ + NULL, NULL, /* lockfunc, lockarg */ + &sc->sf_cdata.sf_tx_tag); + if (error != 0) { + device_printf(sc->sf_dev, "failed to create Tx DMA tag\n"); + goto fail; + } -/* - * The starfire is programmed to use 'normal' mode for packet reception, - * which means we use the consumer/producer model for both the buffer - * descriptor queue and the completion descriptor queue. The only problem - * with this is that it involves a lot of register accesses: we have to - * read the RX completion consumer and producer indexes and the RX buffer - * producer index, plus the RX completion consumer and RX buffer producer - * indexes have to be updated. It would have been easier if Adaptec had - * put each index in a separate register, especially given that the damn - * NIC has a 512K register space. - * - * In spite of all the lovely features that Adaptec crammed into the 6915, - * it is marred by one truly stupid design flaw, which is that receive - * buffer addresses must be aligned on a longword boundary. This forces - * the packet payload to be unaligned, which is suboptimal on the x86 and - * completely unuseable on the Alpha. Our only recourse is to copy received - * packets into properly aligned buffers before handing them off. - */ + /* Create tag for Rx buffers. */ + error = bus_dma_tag_create(sc->sf_cdata.sf_parent_tag,/* parent */ + SF_RX_ALIGN, 0, /* alignment, boundary */ + BUS_SPACE_MAXADDR, /* lowaddr */ + BUS_SPACE_MAXADDR, /* highaddr */ + NULL, NULL, /* filter, filterarg */ + MCLBYTES, /* maxsize */ + 1, /* nsegments */ + MCLBYTES, /* maxsegsize */ + 0, /* flags */ + NULL, NULL, /* lockfunc, lockarg */ + &sc->sf_cdata.sf_rx_tag); + if (error != 0) { + device_printf(sc->sf_dev, "failed to create Rx DMA tag\n"); + goto fail; + } -static void -sf_rxeof(sc) - struct sf_softc *sc; -{ - struct mbuf *m; - struct ifnet *ifp; - struct sf_rx_bufdesc_type0 *desc; - struct sf_rx_cmpdesc_type3 *cur_rx; - u_int32_t rxcons, rxprod; - int cmpprodidx, cmpconsidx, bufprodidx; + /* Allocate DMA'able memory and load the DMA map for Tx ring. */ + error = bus_dmamem_alloc(sc->sf_cdata.sf_tx_ring_tag, + (void **)&sc->sf_rdata.sf_tx_ring, BUS_DMA_WAITOK | + BUS_DMA_COHERENT | BUS_DMA_ZERO, &sc->sf_cdata.sf_tx_ring_map); + if (error != 0) { + device_printf(sc->sf_dev, + "failed to allocate DMA'able memory for Tx ring\n"); + goto fail; + } - SF_LOCK_ASSERT(sc); + ctx.sf_busaddr = 0; + error = bus_dmamap_load(sc->sf_cdata.sf_tx_ring_tag, + sc->sf_cdata.sf_tx_ring_map, sc->sf_rdata.sf_tx_ring, + SF_TX_DLIST_SIZE, sf_dmamap_cb, &ctx, 0); + if (error != 0 || ctx.sf_busaddr == 0) { + device_printf(sc->sf_dev, + "failed to load DMA'able memory for Tx ring\n"); + goto fail; + } + sc->sf_rdata.sf_tx_ring_paddr = ctx.sf_busaddr; - ifp = sc->sf_ifp; + /* + * Allocate DMA'able memory and load the DMA map for Tx completion ring. + */ + error = bus_dmamem_alloc(sc->sf_cdata.sf_tx_cring_tag, + (void **)&sc->sf_rdata.sf_tx_cring, BUS_DMA_WAITOK | + BUS_DMA_COHERENT | BUS_DMA_ZERO, &sc->sf_cdata.sf_tx_cring_map); + if (error != 0) { + device_printf(sc->sf_dev, + "failed to allocate DMA'able memory for " + "Tx completion ring\n"); + goto fail; + } - rxcons = csr_read_4(sc, SF_CQ_CONSIDX); - rxprod = csr_read_4(sc, SF_RXDQ_PTR_Q1); - cmpprodidx = SF_IDX_LO(csr_read_4(sc, SF_CQ_PRODIDX)); - cmpconsidx = SF_IDX_LO(rxcons); - bufprodidx = SF_IDX_LO(rxprod); + ctx.sf_busaddr = 0; + error = bus_dmamap_load(sc->sf_cdata.sf_tx_cring_tag, + sc->sf_cdata.sf_tx_cring_map, sc->sf_rdata.sf_tx_cring, + SF_TX_CLIST_SIZE, sf_dmamap_cb, &ctx, 0); + if (error != 0 || ctx.sf_busaddr == 0) { + device_printf(sc->sf_dev, + "failed to load DMA'able memory for Tx completion ring\n"); + goto fail; + } + sc->sf_rdata.sf_tx_cring_paddr = ctx.sf_busaddr; - while (cmpconsidx != cmpprodidx) { - struct mbuf *m0; + /* Allocate DMA'able memory and load the DMA map for Rx ring. */ + error = bus_dmamem_alloc(sc->sf_cdata.sf_rx_ring_tag, + (void **)&sc->sf_rdata.sf_rx_ring, BUS_DMA_WAITOK | + BUS_DMA_COHERENT | BUS_DMA_ZERO, &sc->sf_cdata.sf_rx_ring_map); + if (error != 0) { + device_printf(sc->sf_dev, + "failed to allocate DMA'able memory for Rx ring\n"); + goto fail; + } -#ifdef DEVICE_POLLING - if (ifp->if_capenable & IFCAP_POLLING) { - if (sc->rxcycles <= 0) - break; - sc->rxcycles--; - } -#endif + ctx.sf_busaddr = 0; + error = bus_dmamap_load(sc->sf_cdata.sf_rx_ring_tag, + sc->sf_cdata.sf_rx_ring_map, sc->sf_rdata.sf_rx_ring, + SF_RX_DLIST_SIZE, sf_dmamap_cb, &ctx, 0); + if (error != 0 || ctx.sf_busaddr == 0) { + device_printf(sc->sf_dev, + "failed to load DMA'able memory for Rx ring\n"); + goto fail; + } + sc->sf_rdata.sf_rx_ring_paddr = ctx.sf_busaddr; - cur_rx = &sc->sf_ldata->sf_rx_clist[cmpconsidx]; - desc = &sc->sf_ldata->sf_rx_dlist_big[cur_rx->sf_endidx]; - m = desc->sf_mbuf; - SF_INC(cmpconsidx, SF_RX_CLIST_CNT); - SF_INC(bufprodidx, SF_RX_DLIST_CNT); + /* + * Allocate DMA'able memory and load the DMA map for Rx completion ring. + */ + error = bus_dmamem_alloc(sc->sf_cdata.sf_rx_cring_tag, + (void **)&sc->sf_rdata.sf_rx_cring, BUS_DMA_WAITOK | + BUS_DMA_COHERENT | BUS_DMA_ZERO, &sc->sf_cdata.sf_rx_cring_map); + if (error != 0) { + device_printf(sc->sf_dev, + "failed to allocate DMA'able memory for " + "Rx completion ring\n"); + goto fail; + } - if (!(cur_rx->sf_status1 & SF_RXSTAT1_OK)) { - ifp->if_ierrors++; - sf_newbuf(sc, desc, m); + ctx.sf_busaddr = 0; + error = bus_dmamap_load(sc->sf_cdata.sf_rx_cring_tag, + sc->sf_cdata.sf_rx_cring_map, sc->sf_rdata.sf_rx_cring, + SF_RX_CLIST_SIZE, sf_dmamap_cb, &ctx, 0); + if (error != 0 || ctx.sf_busaddr == 0) { + device_printf(sc->sf_dev, + "failed to load DMA'able memory for Rx completion ring\n"); + goto fail; + } + sc->sf_rdata.sf_rx_cring_paddr = ctx.sf_busaddr; + + /* + * Tx desciptor ring and Tx completion ring should be addressed in + * the same 4GB space. The same rule applys to Rx ring and Rx + * completion ring. Unfortunately there is no way to specify this + * boundary restriction with bus_dma(9). So just try to allocate + * without the restriction and check the restriction was satisfied. + * If not, fall back to 32bit dma addressing mode which always + * guarantees the restriction. + */ + tx_ring_end = sc->sf_rdata.sf_tx_ring_paddr + SF_TX_DLIST_SIZE; + tx_cring_end = sc->sf_rdata.sf_tx_cring_paddr + SF_TX_CLIST_SIZE; + rx_ring_end = sc->sf_rdata.sf_rx_ring_paddr + SF_RX_DLIST_SIZE; + rx_cring_end = sc->sf_rdata.sf_rx_cring_paddr + SF_RX_CLIST_SIZE; + if ((SF_ADDR_HI(sc->sf_rdata.sf_tx_ring_paddr) != + SF_ADDR_HI(tx_cring_end)) || + (SF_ADDR_HI(sc->sf_rdata.sf_tx_cring_paddr) != + SF_ADDR_HI(tx_ring_end)) || + (SF_ADDR_HI(sc->sf_rdata.sf_rx_ring_paddr) != + SF_ADDR_HI(rx_cring_end)) || + (SF_ADDR_HI(sc->sf_rdata.sf_rx_cring_paddr) != + SF_ADDR_HI(rx_ring_end))) { + device_printf(sc->sf_dev, + "switching to 32bit DMA mode\n"); + sf_dma_free(sc); + /* Limit DMA address space to 32bit and try again. */ + lowaddr = BUS_SPACE_MAXADDR_32BIT; + goto again; + } + + /* Create DMA maps for Tx buffers. */ + for (i = 0; i < SF_TX_DLIST_CNT; i++) { + txd = &sc->sf_cdata.sf_txdesc[i]; + txd->tx_m = NULL; + txd->ndesc = 0; + txd->tx_dmamap = NULL; + error = bus_dmamap_create(sc->sf_cdata.sf_tx_tag, 0, + &txd->tx_dmamap); + if (error != 0) { + device_printf(sc->sf_dev, + "failed to create Tx dmamap\n"); + goto fail; + } + } + /* Create DMA maps for Rx buffers. */ + if ((error = bus_dmamap_create(sc->sf_cdata.sf_rx_tag, 0, + &sc->sf_cdata.sf_rx_sparemap)) != 0) { + device_printf(sc->sf_dev, + "failed to create spare Rx dmamap\n"); + goto fail; + } + for (i = 0; i < SF_RX_DLIST_CNT; i++) { + rxd = &sc->sf_cdata.sf_rxdesc[i]; + rxd->rx_m = NULL; + rxd->rx_dmamap = NULL; + error = bus_dmamap_create(sc->sf_cdata.sf_rx_tag, 0, + &rxd->rx_dmamap); + if (error != 0) { + device_printf(sc->sf_dev, + "failed to create Rx dmamap\n"); + goto fail; + } + } + +fail: + return (error); +} + +static void +sf_dma_free(struct sf_softc *sc) +{ + struct sf_txdesc *txd; + struct sf_rxdesc *rxd; + int i; + + /* Tx ring. */ + if (sc->sf_cdata.sf_tx_ring_tag) { + if (sc->sf_cdata.sf_tx_ring_map) + bus_dmamap_unload(sc->sf_cdata.sf_tx_ring_tag, + sc->sf_cdata.sf_tx_ring_map); + if (sc->sf_cdata.sf_tx_ring_map && + sc->sf_rdata.sf_tx_ring) + bus_dmamem_free(sc->sf_cdata.sf_tx_ring_tag, + sc->sf_rdata.sf_tx_ring, + sc->sf_cdata.sf_tx_ring_map); + sc->sf_rdata.sf_tx_ring = NULL; + sc->sf_cdata.sf_tx_ring_map = NULL; + bus_dma_tag_destroy(sc->sf_cdata.sf_tx_ring_tag); + sc->sf_cdata.sf_tx_ring_tag = NULL; + } + /* Tx completion ring. */ + if (sc->sf_cdata.sf_tx_cring_tag) { + if (sc->sf_cdata.sf_tx_cring_map) + bus_dmamap_unload(sc->sf_cdata.sf_tx_cring_tag, + sc->sf_cdata.sf_tx_cring_map); + if (sc->sf_cdata.sf_tx_cring_map && + sc->sf_rdata.sf_tx_cring) + bus_dmamem_free(sc->sf_cdata.sf_tx_cring_tag, + sc->sf_rdata.sf_tx_cring, + sc->sf_cdata.sf_tx_cring_map); + sc->sf_rdata.sf_tx_cring = NULL; + sc->sf_cdata.sf_tx_cring_map = NULL; + bus_dma_tag_destroy(sc->sf_cdata.sf_tx_cring_tag); + sc->sf_cdata.sf_tx_cring_tag = NULL; + } + /* Rx ring. */ + if (sc->sf_cdata.sf_rx_ring_tag) { + if (sc->sf_cdata.sf_rx_ring_map) + bus_dmamap_unload(sc->sf_cdata.sf_rx_ring_tag, + sc->sf_cdata.sf_rx_ring_map); + if (sc->sf_cdata.sf_rx_ring_map && + sc->sf_rdata.sf_rx_ring) + bus_dmamem_free(sc->sf_cdata.sf_rx_ring_tag, + sc->sf_rdata.sf_rx_ring, + sc->sf_cdata.sf_rx_ring_map); + sc->sf_rdata.sf_rx_ring = NULL; + sc->sf_cdata.sf_rx_ring_map = NULL; + bus_dma_tag_destroy(sc->sf_cdata.sf_rx_ring_tag); + sc->sf_cdata.sf_rx_ring_tag = NULL; + } + /* Rx completion ring. */ + if (sc->sf_cdata.sf_rx_cring_tag) { + if (sc->sf_cdata.sf_rx_cring_map) + bus_dmamap_unload(sc->sf_cdata.sf_rx_cring_tag, + sc->sf_cdata.sf_rx_cring_map); + if (sc->sf_cdata.sf_rx_cring_map && + sc->sf_rdata.sf_rx_cring) + bus_dmamem_free(sc->sf_cdata.sf_rx_cring_tag, + sc->sf_rdata.sf_rx_cring, + sc->sf_cdata.sf_rx_cring_map); + sc->sf_rdata.sf_rx_cring = NULL; + sc->sf_cdata.sf_rx_cring_map = NULL; + bus_dma_tag_destroy(sc->sf_cdata.sf_rx_cring_tag); + sc->sf_cdata.sf_rx_cring_tag = NULL; + } + /* Tx buffers. */ + if (sc->sf_cdata.sf_tx_tag) { + for (i = 0; i < SF_TX_DLIST_CNT; i++) { + txd = &sc->sf_cdata.sf_txdesc[i]; + if (txd->tx_dmamap) { + bus_dmamap_destroy(sc->sf_cdata.sf_tx_tag, + txd->tx_dmamap); + txd->tx_dmamap = NULL; + } + } + bus_dma_tag_destroy(sc->sf_cdata.sf_tx_tag); + sc->sf_cdata.sf_tx_tag = NULL; + } + /* Rx buffers. */ + if (sc->sf_cdata.sf_rx_tag) { + for (i = 0; i < SF_RX_DLIST_CNT; i++) { + rxd = &sc->sf_cdata.sf_rxdesc[i]; + if (rxd->rx_dmamap) { + bus_dmamap_destroy(sc->sf_cdata.sf_rx_tag, + rxd->rx_dmamap); + rxd->rx_dmamap = NULL; + } + } + if (sc->sf_cdata.sf_rx_sparemap) { + bus_dmamap_destroy(sc->sf_cdata.sf_rx_tag, + sc->sf_cdata.sf_rx_sparemap); + sc->sf_cdata.sf_rx_sparemap = 0; + } + bus_dma_tag_destroy(sc->sf_cdata.sf_rx_tag); + sc->sf_cdata.sf_rx_tag = NULL; + } + + if (sc->sf_cdata.sf_parent_tag) { + bus_dma_tag_destroy(sc->sf_cdata.sf_parent_tag); + sc->sf_cdata.sf_parent_tag = NULL; + } +} + +static int +sf_init_rx_ring(struct sf_softc *sc) +{ + struct sf_ring_data *rd; + int i; + + sc->sf_cdata.sf_rxc_cons = 0; + + rd = &sc->sf_rdata; + bzero(rd->sf_rx_ring, SF_RX_DLIST_SIZE); + bzero(rd->sf_rx_cring, SF_RX_CLIST_SIZE); + + for (i = 0; i < SF_RX_DLIST_CNT; i++) { + if (sf_newbuf(sc, i) != 0) + return (ENOBUFS); + } + + bus_dmamap_sync(sc->sf_cdata.sf_rx_cring_tag, + sc->sf_cdata.sf_rx_cring_map, + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); + bus_dmamap_sync(sc->sf_cdata.sf_rx_ring_tag, + sc->sf_cdata.sf_rx_ring_map, + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); + + return (0); +} + +static void +sf_init_tx_ring(struct sf_softc *sc) +{ + struct sf_ring_data *rd; + int i; + + sc->sf_cdata.sf_tx_prod = 0; + sc->sf_cdata.sf_tx_cnt = 0; + sc->sf_cdata.sf_txc_cons = 0; + + rd = &sc->sf_rdata; + bzero(rd->sf_tx_ring, SF_TX_DLIST_SIZE); + bzero(rd->sf_tx_cring, SF_TX_CLIST_SIZE); + for (i = 0; i < SF_TX_DLIST_CNT; i++) { + rd->sf_tx_ring[i].sf_tx_ctrl = htole32(SF_TX_DESC_ID); + sc->sf_cdata.sf_txdesc[i].tx_m = NULL; + sc->sf_cdata.sf_txdesc[i].ndesc = 0; + } + rd->sf_tx_ring[i].sf_tx_ctrl |= htole32(SF_TX_DESC_END); + + bus_dmamap_sync(sc->sf_cdata.sf_tx_ring_tag, + sc->sf_cdata.sf_tx_ring_map, + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); + bus_dmamap_sync(sc->sf_cdata.sf_tx_cring_tag, + sc->sf_cdata.sf_tx_cring_map, + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); +} + +/* + * Initialize an RX descriptor and attach an MBUF cluster. + */ +static int +sf_newbuf(struct sf_softc *sc, int idx) +{ + struct sf_rx_rdesc *desc; + struct sf_rxdesc *rxd; + struct mbuf *m; + bus_dma_segment_t segs[1]; + bus_dmamap_t map; + int nsegs; + + m = m_getcl(M_DONTWAIT, MT_DATA, M_PKTHDR); + if (m == NULL) + return (ENOBUFS); + m->m_len = m->m_pkthdr.len = MCLBYTES; + m_adj(m, sizeof(uint32_t)); + + if (bus_dmamap_load_mbuf_sg(sc->sf_cdata.sf_rx_tag, + sc->sf_cdata.sf_rx_sparemap, m, segs, &nsegs, 0) != 0) { + m_freem(m); + return (ENOBUFS); + } + KASSERT(nsegs == 1, ("%s: %d segments returned!", __func__, nsegs)); + + rxd = &sc->sf_cdata.sf_rxdesc[idx]; + if (rxd->rx_m != NULL) { + bus_dmamap_sync(sc->sf_cdata.sf_rx_tag, rxd->rx_dmamap, + BUS_DMASYNC_POSTREAD); + bus_dmamap_unload(sc->sf_cdata.sf_rx_tag, rxd->rx_dmamap); + } + map = rxd->rx_dmamap; + rxd->rx_dmamap = sc->sf_cdata.sf_rx_sparemap; + sc->sf_cdata.sf_rx_sparemap = map; + bus_dmamap_sync(sc->sf_cdata.sf_rx_tag, rxd->rx_dmamap, + BUS_DMASYNC_PREREAD); + rxd->rx_m = m; + desc = &sc->sf_rdata.sf_rx_ring[idx]; + desc->sf_addr = htole64(segs[0].ds_addr); + + return (0); +} + +#ifndef __NO_STRICT_ALIGNMENT +static __inline void +sf_fixup_rx(struct mbuf *m) +{ + int i; + uint16_t *src, *dst; + + src = mtod(m, uint16_t *); + dst = src - 1; + + for (i = 0; i < (m->m_len / sizeof(uint16_t) + 1); i++) + *dst++ = *src++; + + m->m_data -= ETHER_ALIGN; +} +#endif + +/* + * The starfire is programmed to use 'normal' mode for packet reception, + * which means we use the consumer/producer model for both the buffer + * descriptor queue and the completion descriptor queue. The only problem + * with this is that it involves a lot of register accesses: we have to + * read the RX completion consumer and producer indexes and the RX buffer + * producer index, plus the RX completion consumer and RX buffer producer + * indexes have to be updated. It would have been easier if Adaptec had + * put each index in a separate register, especially given that the damn + * NIC has a 512K register space. + * + * In spite of all the lovely features that Adaptec crammed into the 6915, + * it is marred by one truly stupid design flaw, which is that receive + * buffer addresses must be aligned on a longword boundary. This forces + * the packet payload to be unaligned, which is suboptimal on the x86 and + * completely unuseable on the Alpha. Our only recourse is to copy received + * packets into properly aligned buffers before handing them off. + */ +static void +sf_rxeof(struct sf_softc *sc) +{ + struct mbuf *m; + struct ifnet *ifp; + struct sf_rxdesc *rxd; + struct sf_rx_rcdesc *cur_cmp; + int cons, eidx, prog; + uint32_t status, status2; + + SF_LOCK_ASSERT(sc); + + ifp = sc->sf_ifp; + + bus_dmamap_sync(sc->sf_cdata.sf_rx_ring_tag, + sc->sf_cdata.sf_rx_ring_map, + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); + bus_dmamap_sync(sc->sf_cdata.sf_rx_cring_tag, + sc->sf_cdata.sf_rx_cring_map, + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); + + /* + * To reduce register access, directly read Receive completion + * queue entry. + */ + eidx = 0; + prog = 0; + for (cons = sc->sf_cdata.sf_rxc_cons; ; SF_INC(cons, SF_RX_CLIST_CNT)) { + cur_cmp = &sc->sf_rdata.sf_rx_cring[cons]; + status = le32toh(cur_cmp->sf_rx_status1); + if (status == 0) + break; +#ifdef DEVICE_POLLING + if ((ifp->if_capenable & IFCAP_POLLING) != 0) { + if (sc->rxcycles <= 0) + break; + sc->rxcycles--; + } +#endif + prog++; + eidx = (status & SF_RX_CMPDESC_EIDX) >> 16; + rxd = &sc->sf_cdata.sf_rxdesc[eidx]; + m = rxd->rx_m; + + /* + * Note, if_ipackets and if_ierrors counters + * are handled in sf_stats_update(). + */ + if ((status & SF_RXSTAT1_OK) == 0) { + cur_cmp->sf_rx_status1 = 0; continue; } - m0 = m_devget(mtod(m, char *), cur_rx->sf_len, ETHER_ALIGN, - ifp, NULL); - sf_newbuf(sc, desc, m); - if (m0 == NULL) { - ifp->if_ierrors++; + if (sf_newbuf(sc, eidx) != 0) { + ifp->if_iqdrops++; + cur_cmp->sf_rx_status1 = 0; continue; } - m = m0; - ifp->if_ipackets++; + /* AIC-6915 supports TCP/UDP checksum offload. */ + if ((ifp->if_capenable & IFCAP_RXCSUM) != 0) { + status2 = le32toh(cur_cmp->sf_rx_status2); + /* + * Sometimes AIC-6915 generates an interrupt to + * warn RxGFP stall with bad checksum bit set + * in status word. I'm not sure what conditioan + * triggers it but recevied packet's checksum + * was correct even though AIC-6915 does not + * agree on this. This may be an indication of + * firmware bug. To fix the issue, do not rely + * on bad checksum bit in status word and let + * upper layer verify integrity of received + * frame. + * Another nice feature of AIC-6915 is hardware + * assistance of checksum calculation by + * providing partial checksum value for received + * frame. The partial checksum value can be used + * to accelerate checksum computation for + * fragmented TCP/UDP packets. Upper network + * stack already takes advantage of the partial + * checksum value in IP reassembly stage. But + * I'm not sure the correctness of the partial + * hardware checksum assistance as frequent + * RxGFP stalls are seen on non-fragmented + * frames. Due to the nature of the complexity + * of checksum computation code in firmware it's + * possible to see another bug in RxGFP so + * ignore checksum assistance for fragmented + * frames. This can be changed in future. + */ + if ((status2 & SF_RXSTAT2_FRAG) == 0) { + if ((status2 & (SF_RXSTAT2_TCP | + SF_RXSTAT2_UDP)) != 0) { + if ((status2 & SF_RXSTAT2_CSUM_OK)) { + m->m_pkthdr.csum_flags = + CSUM_DATA_VALID | + CSUM_PSEUDO_HDR; + m->m_pkthdr.csum_data = 0xffff; + } + } + } +#ifdef SF_PARTIAL_CSUM_SUPPORT + else if ((status2 & SF_RXSTAT2_FRAG) != 0) { + if ((status2 & (SF_RXSTAT2_TCP | + SF_RXSTAT2_UDP)) != 0) { + if ((status2 & SF_RXSTAT2_PCSUM_OK)) { + m->m_pkthdr.csum_flags = + CSUM_DATA_VALID; + m->m_pkthdr.csum_data = + (status & + SF_RX_CMPDESC_CSUM2); + } + } + } +#endif + } + + m->m_pkthdr.len = m->m_len = status & SF_RX_CMPDESC_LEN; +#ifndef __NO_STRICT_ALIGNMENT + sf_fixup_rx(m); +#endif + m->m_pkthdr.rcvif = ifp; + SF_UNLOCK(sc); (*ifp->if_input)(ifp, m); SF_LOCK(sc); + + /* Clear completion status. */ + cur_cmp->sf_rx_status1 = 0; } - csr_write_4(sc, SF_CQ_CONSIDX, - (rxcons & ~SF_CQ_CONSIDX_RXQ1) | cmpconsidx); - csr_write_4(sc, SF_RXDQ_PTR_Q1, - (rxprod & ~SF_RXDQ_PRODIDX) | bufprodidx); + if (prog > 0) { + sc->sf_cdata.sf_rxc_cons = cons; + bus_dmamap_sync(sc->sf_cdata.sf_rx_ring_tag, + sc->sf_cdata.sf_rx_ring_map, + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); + bus_dmamap_sync(sc->sf_cdata.sf_rx_cring_tag, + sc->sf_cdata.sf_rx_cring_map, + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); + + /* Update Rx completion Q1 consumer index. */ + csr_write_4(sc, SF_CQ_CONSIDX, + (csr_read_4(sc, SF_CQ_CONSIDX) & ~SF_CQ_CONSIDX_RXQ1) | + (cons & SF_CQ_CONSIDX_RXQ1)); + /* Update Rx descriptor Q1 ptr. */ + csr_write_4(sc, SF_RXDQ_PTR_Q1, + (csr_read_4(sc, SF_RXDQ_PTR_Q1) & ~SF_RXDQ_PRODIDX) | + (eidx & SF_RXDQ_PRODIDX)); + } } /* @@ -1024,90 +1686,122 @@ sf_rxeof(sc) * index, which is the offset in 8 byte blocks. */ static void -sf_txeof(sc) - struct sf_softc *sc; +sf_txeof(struct sf_softc *sc) { - int txcons, cmpprodidx, cmpconsidx; - struct sf_tx_cmpdesc_type1 *cur_cmp; - struct sf_tx_bufdesc_type0 *cur_tx; + struct sf_txdesc *txd; + struct sf_tx_rcdesc *cur_cmp; struct ifnet *ifp; + uint32_t status; + int cons, idx, prod; + + SF_LOCK_ASSERT(sc); ifp = sc->sf_ifp; - SF_LOCK_ASSERT(sc); - txcons = csr_read_4(sc, SF_CQ_CONSIDX); - cmpprodidx = SF_IDX_HI(csr_read_4(sc, SF_CQ_PRODIDX)); - cmpconsidx = SF_IDX_HI(txcons); - - while (cmpconsidx != cmpprodidx) { - cur_cmp = &sc->sf_ldata->sf_tx_clist[cmpconsidx]; - cur_tx = &sc->sf_ldata->sf_tx_dlist[cur_cmp->sf_index >> 7]; - - if (cur_cmp->sf_txstat & SF_TXSTAT_TX_OK) - ifp->if_opackets++; - else { - if (cur_cmp->sf_txstat & SF_TXSTAT_TX_UNDERRUN) - sf_txthresh_adjust(sc); - ifp->if_oerrors++; - } + bus_dmamap_sync(sc->sf_cdata.sf_tx_cring_tag, + sc->sf_cdata.sf_tx_cring_map, + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); - sc->sf_tx_cnt--; - if (cur_tx->sf_mbuf != NULL) { - m_freem(cur_tx->sf_mbuf); - cur_tx->sf_mbuf = NULL; - } else + cons = sc->sf_cdata.sf_txc_cons; + prod = (csr_read_4(sc, SF_CQ_PRODIDX) & SF_TXDQ_PRODIDX_HIPRIO) >> 16; + if (prod == cons) + return; + + for (; cons != prod; SF_INC(cons, SF_TX_CLIST_CNT)) { + cur_cmp = &sc->sf_rdata.sf_tx_cring[cons]; + status = le32toh(cur_cmp->sf_tx_status1); + if (status == 0) + break; + switch (status & SF_TX_CMPDESC_TYPE) { + case SF_TXCMPTYPE_TX: + /* Tx complete entry. */ + break; + case SF_TXCMPTYPE_DMA: + /* DMA complete entry. */ + idx = status & SF_TX_CMPDESC_IDX; + idx = idx / sizeof(struct sf_tx_rdesc); + /* + * We don't need to check Tx status here. + * SF_ISR_TX_LOFIFO intr would handle this. + * Note, if_opackets, if_collisions and if_oerrors + * counters are handled in sf_stats_update(). + */ + txd = &sc->sf_cdata.sf_txdesc[idx]; + if (txd->tx_m != NULL) { + bus_dmamap_sync(sc->sf_cdata.sf_tx_tag, + txd->tx_dmamap, + BUS_DMASYNC_POSTWRITE); + bus_dmamap_unload(sc->sf_cdata.sf_tx_tag, + txd->tx_dmamap); + m_freem(txd->tx_m); + txd->tx_m = NULL; + } + sc->sf_cdata.sf_tx_cnt -= txd->ndesc; + KASSERT(sc->sf_cdata.sf_tx_cnt >= 0, + ("%s: Active Tx desc counter was garbled\n", + __func__)); + txd->ndesc = 0; + ifp->if_drv_flags &= ~IFF_DRV_OACTIVE; break; - SF_INC(cmpconsidx, SF_TX_CLIST_CNT); + default: + /* It should not happen. */ + device_printf(sc->sf_dev, + "unknown Tx completion type : 0x%08x : %d : %d\n", + status, cons, prod); + break; + } + cur_cmp->sf_tx_status1 = 0; } - ifp->if_timer = 0; - ifp->if_drv_flags &= ~IFF_DRV_OACTIVE; + sc->sf_cdata.sf_txc_cons = cons; + bus_dmamap_sync(sc->sf_cdata.sf_tx_cring_tag, + sc->sf_cdata.sf_tx_cring_map, + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); + if (sc->sf_cdata.sf_tx_cnt == 0) + sc->sf_watchdog_timer = 0; + + /* Update Tx completion consumer index. */ csr_write_4(sc, SF_CQ_CONSIDX, - (txcons & ~SF_CQ_CONSIDX_TXQ) | - ((cmpconsidx << 16) & 0xFFFF0000)); + (csr_read_4(sc, SF_CQ_CONSIDX) & 0xffff) | + ((cons << 16) & 0xffff0000)); } static void -sf_txthresh_adjust(sc) - struct sf_softc *sc; +sf_txthresh_adjust(struct sf_softc *sc) { - u_int32_t txfctl; - u_int8_t txthresh; - - txfctl = csr_read_4(sc, SF_TX_FRAMCTL); - txthresh = txfctl & SF_TXFRMCTL_TXTHRESH; - if (txthresh < 0xFF) { - txthresh++; + uint32_t txfctl; + + device_printf(sc->sf_dev, "Tx underrun -- "); + if (sc->sf_txthresh < SF_MAX_TX_THRESHOLD) { + txfctl = csr_read_4(sc, SF_TX_FRAMCTL); + /* Increase Tx threshold 256 bytes. */ + sc->sf_txthresh += 16; + if (sc->sf_txthresh > SF_MAX_TX_THRESHOLD) + sc->sf_txthresh = SF_MAX_TX_THRESHOLD; txfctl &= ~SF_TXFRMCTL_TXTHRESH; - txfctl |= txthresh; -#ifdef DIAGNOSTIC - device_printf(sc->sf_dev, "tx underrun, increasing " - "tx threshold to %d bytes\n", - txthresh * 4); -#endif + txfctl |= sc->sf_txthresh; + printf("increasing Tx threshold to %d bytes\n", + sc->sf_txthresh * SF_TX_THRESHOLD_UNIT); csr_write_4(sc, SF_TX_FRAMCTL, txfctl); - } + } else + printf("\n"); } #ifdef DEVICE_POLLING static void sf_poll(struct ifnet *ifp, enum poll_cmd cmd, int count) { - struct sf_softc *sc = ifp->if_softc; + struct sf_softc *sc; + uint32_t status; + sc = ifp->if_softc; SF_LOCK(sc); - if (ifp->if_drv_flags & IFF_DRV_RUNNING) - sf_poll_locked(ifp, cmd, count); - SF_UNLOCK(sc); -} - -static void -sf_poll_locked(struct ifnet *ifp, enum poll_cmd cmd, int count) -{ - struct sf_softc *sc = ifp->if_softc; - SF_LOCK_ASSERT(sc); + if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) { + SF_UNLOCK(sc); + return; + } sc->rxcycles = count; sf_rxeof(sc); @@ -1116,80 +1810,103 @@ sf_poll_locked(struct ifnet *ifp, enum poll_cmd cmd, int count) sf_start_locked(ifp); if (cmd == POLL_AND_CHECK_STATUS) { - u_int32_t status; - + /* Reading the ISR register clears all interrrupts. */ status = csr_read_4(sc, SF_ISR); - if (status) - csr_write_4(sc, SF_ISR, status); - if (status & SF_ISR_TX_LOFIFO) - sf_txthresh_adjust(sc); - - if (status & SF_ISR_ABNORMALINTR) { - if (status & SF_ISR_STATSOFLOW) { - callout_stop(&sc->sf_stat_callout); + if ((status & SF_ISR_ABNORMALINTR) != 0) { + if ((status & SF_ISR_STATSOFLOW) != 0) sf_stats_update(sc); - } else + else if ((status & SF_ISR_TX_LOFIFO) != 0) + sf_txthresh_adjust(sc); + else if ((status & SF_ISR_DMAERR) != 0) { + device_printf(sc->sf_dev, + "DMA error, resetting\n"); sf_init_locked(sc); + } else if ((status & SF_ISR_NO_TX_CSUM) != 0) { + sc->sf_statistics.tx_gfp_stall++; +#ifdef SF_GFP_DEBUG + device_printf(sc->sf_dev, + "TxGFP is not responding!\n"); +#endif + } else if ((status & SF_ISR_RXGFP_NORESP) != 0) { + sc->sf_statistics.rx_gfp_stall++; +#ifdef SF_GFP_DEBUG + device_printf(sc->sf_dev, + "RxGFP is not responding!\n"); +#endif + } } } + + SF_UNLOCK(sc); } #endif /* DEVICE_POLLING */ static void -sf_intr(arg) - void *arg; +sf_intr(void *arg) { struct sf_softc *sc; struct ifnet *ifp; - u_int32_t status; + uint32_t status; - sc = arg; + sc = (struct sf_softc *)arg; SF_LOCK(sc); - ifp = sc->sf_ifp; + if (sc->sf_suspended != 0) + goto done_locked; + + /* Reading the ISR register clears all interrrupts. */ + status = csr_read_4(sc, SF_ISR); + if (status == 0 || status == 0xffffffff || + (status & SF_ISR_PCIINT_ASSERTED) == 0) + goto done_locked; + ifp = sc->sf_ifp; #ifdef DEVICE_POLLING - if (ifp->if_capenable & IFCAP_POLLING) { - SF_UNLOCK(sc); - return; - } + if ((ifp->if_capenable & IFCAP_POLLING) != 0) + goto done_locked; #endif - - if (!(csr_read_4(sc, SF_ISR_SHADOW) & SF_ISR_PCIINT_ASSERTED)) { - SF_UNLOCK(sc); - return; - } + if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) + goto done_locked; /* Disable interrupts. */ csr_write_4(sc, SF_IMR, 0x00000000); - for (;;) { - status = csr_read_4(sc, SF_ISR); - if (status) - csr_write_4(sc, SF_ISR, status); - - if (!(status & SF_INTRS)) - break; - - if (status & SF_ISR_RXDQ1_DMADONE) + for (; (status & SF_INTRS) != 0;) { + if ((status & SF_ISR_RXDQ1_DMADONE) != 0) sf_rxeof(sc); - if (status & SF_ISR_TX_TXDONE || - status & SF_ISR_TX_DMADONE || - status & SF_ISR_TX_QUEUEDONE) + if ((status & (SF_ISR_TX_TXDONE | SF_ISR_TX_DMADONE | + SF_ISR_TX_QUEUEDONE)) != 0) sf_txeof(sc); - if (status & SF_ISR_TX_LOFIFO) - sf_txthresh_adjust(sc); - - if (status & SF_ISR_ABNORMALINTR) { - if (status & SF_ISR_STATSOFLOW) { - callout_stop(&sc->sf_stat_callout); + if ((status & SF_ISR_ABNORMALINTR) != 0) { + if ((status & SF_ISR_STATSOFLOW) != 0) sf_stats_update(sc); - } else + else if ((status & SF_ISR_TX_LOFIFO) != 0) + sf_txthresh_adjust(sc); + else if ((status & SF_ISR_DMAERR) != 0) { + device_printf(sc->sf_dev, + "DMA error, resetting\n"); sf_init_locked(sc); + break; + } else if ((status & SF_ISR_NO_TX_CSUM) != 0) { + sc->sf_statistics.sf_tx_gfp_stall++; +#ifdef SF_GFP_DEBUG + device_printf(sc->sf_dev, + "TxGFP is not responding!\n"); +#endif + } + else if ((status & SF_ISR_RXGFP_NORESP) != 0) { + sc->sf_statistics.sf_rx_gfp_stall++; +#ifdef SF_GFP_DEBUG + device_printf(sc->sf_dev, + "RxGFP is not responding!\n"); +#endif + } } + /* Reading the ISR register clears all interrrupts. */ + status = csr_read_4(sc, SF_ISR); } /* Re-enable interrupts. */ @@ -1197,28 +1914,66 @@ sf_intr(arg) if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) sf_start_locked(ifp); - +done_locked: SF_UNLOCK(sc); } static void -sf_init(xsc) - void *xsc; +sf_download_fw(struct sf_softc *sc) +{ + uint32_t gfpinst; + int i, ndx; + uint8_t *p; + + /* + * A FP instruction is composed of 48bits so we have to + * write it with two parts. + */ + p = txfwdata; + ndx = 0; + for (i = 0; i < sizeof(txfwdata) / SF_GFP_INST_BYTES; i++) { + gfpinst = p[2] << 24 | p[3] << 16 | p[4] << 8 | p[5]; + csr_write_4(sc, SF_TXGFP_MEM_BASE + ndx * 4, gfpinst); + gfpinst = p[0] << 8 | p[1]; + csr_write_4(sc, SF_TXGFP_MEM_BASE + (ndx + 1) * 4, gfpinst); + p += SF_GFP_INST_BYTES; + ndx += 2; + } + if (bootverbose) + device_printf(sc->sf_dev, "%d Tx instructions downloaded\n", i); + + p = rxfwdata; + ndx = 0; + for (i = 0; i < sizeof(rxfwdata) / SF_GFP_INST_BYTES; i++) { + gfpinst = p[2] << 24 | p[3] << 16 | p[4] << 8 | p[5]; + csr_write_4(sc, SF_RXGFP_MEM_BASE + (ndx * 4), gfpinst); + gfpinst = p[0] << 8 | p[1]; + csr_write_4(sc, SF_RXGFP_MEM_BASE + (ndx + 1) * 4, gfpinst); + p += SF_GFP_INST_BYTES; + ndx += 2; + } + if (bootverbose) + device_printf(sc->sf_dev, "%d Rx instructions downloaded\n", i); +} + +static void +sf_init(void *xsc) { struct sf_softc *sc; - sc = xsc; + sc = (struct sf_softc *)xsc; SF_LOCK(sc); sf_init_locked(sc); SF_UNLOCK(sc); } static void -sf_init_locked(sc) - struct sf_softc *sc; +sf_init_locked(struct sf_softc *sc) { struct ifnet *ifp; struct mii_data *mii; + uint8_t eaddr[ETHER_ADDR_LEN]; + bus_addr_t addr; int i; SF_LOCK_ASSERT(sc); @@ -1226,22 +1981,24 @@ sf_init_locked(sc) mii = device_get_softc(sc->sf_miibus); sf_stop(sc); + /* Reset the hardware to a known state. */ sf_reset(sc); /* Init all the receive filter registers */ for (i = SF_RXFILT_PERFECT_BASE; - i < (SF_RXFILT_HASH_MAX + 1); i += 4) + i < (SF_RXFILT_HASH_MAX + 1); i += sizeof(uint32_t)) csr_write_4(sc, i, 0); /* Empty stats counter registers. */ - for (i = 0; i < sizeof(struct sf_stats)/sizeof(u_int32_t); i++) - csr_write_4(sc, SF_STATS_BASE + - (i + sizeof(u_int32_t)), 0); + for (i = SF_STATS_BASE; i < (SF_STATS_END + 1); i += sizeof(uint32_t)) + csr_write_4(sc, i, 0); - /* Init our MAC address */ - csr_write_4(sc, SF_PAR0, *(u_int32_t *)(&IF_LLADDR(sc->sf_ifp)[0])); - csr_write_4(sc, SF_PAR1, *(u_int32_t *)(&IF_LLADDR(sc->sf_ifp)[4])); - sf_setperf(sc, 0, IF_LLADDR(sc->sf_ifp)); + /* Init our MAC address. */ + bcopy(IF_LLADDR(sc->sf_ifp), eaddr, sizeof(eaddr)); + csr_write_4(sc, SF_PAR0, + eaddr[2] << 24 | eaddr[3] << 16 | eaddr[4] << 8 | eaddr[5]); + csr_write_4(sc, SF_PAR1, eaddr[0] << 8 | eaddr[1]); + sf_setperf(sc, 0, eaddr); if (sf_init_rx_ring(sc) == ENOBUFS) { device_printf(sc->sf_dev, @@ -1251,149 +2008,228 @@ sf_init_locked(sc) sf_init_tx_ring(sc); - csr_write_4(sc, SF_RXFILT, SF_PERFMODE_NORMAL|SF_HASHMODE_WITHVLAN); - - /* If we want promiscuous mode, set the allframes bit. */ - if (ifp->if_flags & IFF_PROMISC) { - SF_SETBIT(sc, SF_RXFILT, SF_RXFILT_PROMISC); - } else { - SF_CLRBIT(sc, SF_RXFILT, SF_RXFILT_PROMISC); - } - - if (ifp->if_flags & IFF_BROADCAST) { - SF_SETBIT(sc, SF_RXFILT, SF_RXFILT_BROAD); - } else { - SF_CLRBIT(sc, SF_RXFILT, SF_RXFILT_BROAD); - } + /* + * 16 perfect address filtering. + * Hash only multicast destination address, Accept matching + * frames regardless of VLAN ID. + */ + csr_write_4(sc, SF_RXFILT, SF_PERFMODE_NORMAL | SF_HASHMODE_ANYVLAN); /* - * Load the multicast filter. + * Set Rx filter. */ - sf_setmulti(sc); + sf_rxfilter(sc); - /* Init the completion queue indexes */ + /* Init the completion queue indexes. */ csr_write_4(sc, SF_CQ_CONSIDX, 0); csr_write_4(sc, SF_CQ_PRODIDX, 0); - /* Init the RX completion queue */ - csr_write_4(sc, SF_RXCQ_CTL_1, - vtophys(sc->sf_ldata->sf_rx_clist) & SF_RXCQ_ADDR); - SF_SETBIT(sc, SF_RXCQ_CTL_1, SF_RXCQTYPE_3); + /* Init the RX completion queue. */ + addr = sc->sf_rdata.sf_rx_cring_paddr; + csr_write_4(sc, SF_CQ_ADDR_HI, SF_ADDR_HI(addr)); + csr_write_4(sc, SF_RXCQ_CTL_1, SF_ADDR_LO(addr) & SF_RXCQ_ADDR); + if (SF_ADDR_HI(addr) != 0) + SF_SETBIT(sc, SF_RXCQ_CTL_1, SF_RXCQ_USE_64BIT); + /* Set RX completion queue type 2. */ + SF_SETBIT(sc, SF_RXCQ_CTL_1, SF_RXCQTYPE_2); + csr_write_4(sc, SF_RXCQ_CTL_2, 0); - /* Init RX DMA control. */ - SF_SETBIT(sc, SF_RXDMA_CTL, SF_RXDMA_REPORTBADPKTS); + /* + * Init RX DMA control. + * default RxHighPriority Threshold, + * default RxBurstSize, 128bytes. + */ + SF_SETBIT(sc, SF_RXDMA_CTL, + SF_RXDMA_REPORTBADPKTS | + (SF_RXDMA_HIGHPRIO_THRESH << 8) | + SF_RXDMA_BURST); /* Init the RX buffer descriptor queue. */ - csr_write_4(sc, SF_RXDQ_ADDR_Q1, - vtophys(sc->sf_ldata->sf_rx_dlist_big)); - csr_write_4(sc, SF_RXDQ_CTL_1, (MCLBYTES << 16) | SF_DESCSPACE_16BYTES); + addr = sc->sf_rdata.sf_rx_ring_paddr; + csr_write_4(sc, SF_RXDQ_ADDR_HI, SF_ADDR_HI(addr)); + csr_write_4(sc, SF_RXDQ_ADDR_Q1, SF_ADDR_LO(addr)); + + /* Set RX queue buffer length. */ + csr_write_4(sc, SF_RXDQ_CTL_1, + ((MCLBYTES - sizeof(uint32_t)) << 16) | + SF_RXDQCTL_64BITBADDR | SF_RXDQCTL_VARIABLE); + + if (SF_ADDR_HI(addr) != 0) + SF_SETBIT(sc, SF_RXDQ_CTL_1, SF_RXDQCTL_64BITDADDR); csr_write_4(sc, SF_RXDQ_PTR_Q1, SF_RX_DLIST_CNT - 1); + csr_write_4(sc, SF_RXDQ_CTL_2, 0); /* Init the TX completion queue */ - csr_write_4(sc, SF_TXCQ_CTL, - vtophys(sc->sf_ldata->sf_tx_clist) & SF_RXCQ_ADDR); + addr = sc->sf_rdata.sf_tx_cring_paddr; + csr_write_4(sc, SF_TXCQ_CTL, SF_ADDR_LO(addr) & SF_TXCQ_ADDR); + if (SF_ADDR_HI(addr) != 0) + SF_SETBIT(sc, SF_TXCQ_CTL, SF_TXCQ_USE_64BIT); /* Init the TX buffer descriptor queue. */ - csr_write_4(sc, SF_TXDQ_ADDR_HIPRIO, - vtophys(sc->sf_ldata->sf_tx_dlist)); - SF_SETBIT(sc, SF_TX_FRAMCTL, SF_TXFRMCTL_CPLAFTERTX); + addr = sc->sf_rdata.sf_tx_ring_paddr; + csr_write_4(sc, SF_TXDQ_ADDR_HI, SF_ADDR_HI(addr)); + csr_write_4(sc, SF_TXDQ_ADDR_HIPRIO, 0); + csr_write_4(sc, SF_TXDQ_ADDR_LOPRIO, SF_ADDR_LO(addr)); + csr_write_4(sc, SF_TX_FRAMCTL, + SF_TXFRMCTL_CPLAFTERTX | sc->sf_txthresh); csr_write_4(sc, SF_TXDQ_CTL, - SF_TXBUFDESC_TYPE0|SF_TXMINSPACE_128BYTES|SF_TXSKIPLEN_8BYTES); - SF_SETBIT(sc, SF_TXDQ_CTL, SF_TXDQCTL_NODMACMP); + SF_TXDMA_HIPRIO_THRESH << 24 | + SF_TXSKIPLEN_0BYTES << 16 | + SF_TXDDMA_BURST << 8 | + SF_TXBUFDESC_TYPE2 | SF_TXMINSPACE_UNLIMIT); + if (SF_ADDR_HI(addr) != 0) + SF_SETBIT(sc, SF_TXDQ_CTL, SF_TXDQCTL_64BITADDR); + + /* Set VLAN Type register. */ + csr_write_4(sc, SF_VLANTYPE, ETHERTYPE_VLAN); + + /* Set TxPause Timer. */ + csr_write_4(sc, SF_TXPAUSETIMER, 0xffff); /* Enable autopadding of short TX frames. */ SF_SETBIT(sc, SF_MACCFG_1, SF_MACCFG1_AUTOPAD); + SF_SETBIT(sc, SF_MACCFG_2, SF_MACCFG2_AUTOVLANPAD); + /* Make sure to reset MAC to take changes effect. */ + SF_SETBIT(sc, SF_MACCFG_1, SF_MACCFG1_SOFTRESET); + DELAY(1000); + SF_CLRBIT(sc, SF_MACCFG_1, SF_MACCFG1_SOFTRESET); + + /* Enable PCI bus master. */ + SF_SETBIT(sc, SF_PCI_DEVCFG, SF_PCIDEVCFG_PCIMEN); + + /* Load StarFire firmware. */ + sf_download_fw(sc); + + /* Intialize interrupt moderation. */ + csr_write_4(sc, SF_TIMER_CTL, SF_TIMER_IMASK_MODE | SF_TIMER_TIMES_TEN | + (sc->sf_int_mod & SF_TIMER_IMASK_INTERVAL)); #ifdef DEVICE_POLLING /* Disable interrupts if we are polling. */ - if (ifp->if_capenable & IFCAP_POLLING) + if ((ifp->if_capenable & IFCAP_POLLING) != 0) csr_write_4(sc, SF_IMR, 0x00000000); else #endif - /* Enable interrupts. */ csr_write_4(sc, SF_IMR, SF_INTRS); SF_SETBIT(sc, SF_PCI_DEVCFG, SF_PCIDEVCFG_INTR_ENB); /* Enable the RX and TX engines. */ - SF_SETBIT(sc, SF_GEN_ETH_CTL, SF_ETHCTL_RX_ENB|SF_ETHCTL_RXDMA_ENB); - SF_SETBIT(sc, SF_GEN_ETH_CTL, SF_ETHCTL_TX_ENB|SF_ETHCTL_TXDMA_ENB); + csr_write_4(sc, SF_GEN_ETH_CTL, + SF_ETHCTL_RX_ENB | SF_ETHCTL_RXDMA_ENB | + SF_ETHCTL_TX_ENB | SF_ETHCTL_TXDMA_ENB); - /*mii_mediachg(mii);*/ - sf_ifmedia_upd_locked(ifp); + if ((ifp->if_capenable & IFCAP_TXCSUM) != 0) + SF_SETBIT(sc, SF_GEN_ETH_CTL, SF_ETHCTL_TXGFP_ENB); + else + SF_CLRBIT(sc, SF_GEN_ETH_CTL, SF_ETHCTL_TXGFP_ENB); + if ((ifp->if_capenable & IFCAP_RXCSUM) != 0) + SF_SETBIT(sc, SF_GEN_ETH_CTL, SF_ETHCTL_RXGFP_ENB); + else + SF_CLRBIT(sc, SF_GEN_ETH_CTL, SF_ETHCTL_RXGFP_ENB); + + sc->sf_link = 0; + mii_mediachg(mii); ifp->if_drv_flags |= IFF_DRV_RUNNING; ifp->if_drv_flags &= ~IFF_DRV_OACTIVE; - callout_reset(&sc->sf_stat_callout, hz, sf_stats_update, sc); + callout_reset(&sc->sf_co, hz, sf_tick, sc); } static int -sf_encap(sc, c, m_head) - struct sf_softc *sc; - struct sf_tx_bufdesc_type0 *c; - struct mbuf *m_head; +sf_encap(struct sf_softc *sc, struct mbuf **m_head) { - int frag = 0; - struct sf_frag *f = NULL; + struct sf_txdesc *txd; + struct sf_tx_rdesc *desc; struct mbuf *m; + bus_dmamap_t map; + bus_dma_segment_t txsegs[SF_MAXTXSEGS]; + int error, i, nsegs, prod, si; + int avail, nskip; - m = m_head; + SF_LOCK_ASSERT(sc); - for (m = m_head, frag = 0; m != NULL; m = m->m_next) { - if (m->m_len != 0) { - if (frag == SF_MAXFRAGS) - break; - f = &c->sf_frags[frag]; - if (frag == 0) - f->sf_pktlen = m_head->m_pkthdr.len; - f->sf_fraglen = m->m_len; - f->sf_addr = vtophys(mtod(m, vm_offset_t)); - frag++; + m = *m_head; + prod = sc->sf_cdata.sf_tx_prod; + txd = &sc->sf_cdata.sf_txdesc[prod]; + map = txd->tx_dmamap; + error = bus_dmamap_load_mbuf_sg(sc->sf_cdata.sf_tx_tag, map, + *m_head, txsegs, &nsegs, BUS_DMA_NOWAIT); + if (error == EFBIG) { + m = m_collapse(*m_head, M_DONTWAIT, SF_MAXTXSEGS); + if (m == NULL) { + m_freem(*m_head); + *m_head = NULL; + return (ENOBUFS); } + *m_head = m; + error = bus_dmamap_load_mbuf_sg(sc->sf_cdata.sf_tx_tag, + map, *m_head, txsegs, &nsegs, BUS_DMA_NOWAIT); + if (error != 0) { + m_freem(*m_head); + *m_head = NULL; + return (error); + } + } else if (error != 0) + return (error); + if (nsegs == 0) { + m_freem(*m_head); + *m_head = NULL; + return (EIO); } - if (m != NULL) { - struct mbuf *m_new = NULL; - - MGETHDR(m_new, M_DONTWAIT, MT_DATA); - if (m_new == NULL) { - if_printf(sc->sf_ifp, "no memory for tx list\n"); - return(1); + /* Check number of available descriptors. */ + avail = (SF_TX_DLIST_CNT - 1) - sc->sf_cdata.sf_tx_cnt; + if (avail < nsegs) { + bus_dmamap_unload(sc->sf_cdata.sf_tx_tag, map); + return (ENOBUFS); + } + nskip = 0; + if (prod + nsegs >= SF_TX_DLIST_CNT) { + nskip = SF_TX_DLIST_CNT - prod - 1; + if (avail < nsegs + nskip) { + bus_dmamap_unload(sc->sf_cdata.sf_tx_tag, map); + return (ENOBUFS); } + } - if (m_head->m_pkthdr.len > MHLEN) { - MCLGET(m_new, M_DONTWAIT); - if (!(m_new->m_flags & M_EXT)) { - m_freem(m_new); - if_printf(sc->sf_ifp, "no memory for tx list\n"); - return(1); - } - } - m_copydata(m_head, 0, m_head->m_pkthdr.len, - mtod(m_new, caddr_t)); - m_new->m_pkthdr.len = m_new->m_len = m_head->m_pkthdr.len; - m_freem(m_head); - m_head = m_new; - f = &c->sf_frags[0]; - f->sf_fraglen = f->sf_pktlen = m_head->m_pkthdr.len; - f->sf_addr = vtophys(mtod(m_head, caddr_t)); - frag = 1; - } - - c->sf_mbuf = m_head; - c->sf_id = SF_TX_BUFDESC_ID; - c->sf_fragcnt = frag; - c->sf_intr = 1; - c->sf_caltcp = 0; - c->sf_crcen = 1; - - return(0); + bus_dmamap_sync(sc->sf_cdata.sf_tx_tag, map, BUS_DMASYNC_PREWRITE); + + si = prod; + for (i = 0; i < nsegs; i++) { + desc = &sc->sf_rdata.sf_tx_ring[prod]; + desc->sf_tx_ctrl = htole32(SF_TX_DESC_ID | + (txsegs[i].ds_len & SF_TX_DESC_FRAGLEN)); + desc->sf_tx_reserved = 0; + desc->sf_addr = htole64(txsegs[i].ds_addr); + if (i == 0 && prod + nsegs >= SF_TX_DLIST_CNT) { + /* Queue wraps! */ + desc->sf_tx_ctrl |= htole32(SF_TX_DESC_END); + prod = 0; + } else + SF_INC(prod, SF_TX_DLIST_CNT); + } + /* Update producer index. */ + sc->sf_cdata.sf_tx_prod = prod; + sc->sf_cdata.sf_tx_cnt += nsegs + nskip; + + desc = &sc->sf_rdata.sf_tx_ring[si]; + /* Check TDP/UDP checksum offload request. */ + if ((m->m_pkthdr.csum_flags & SF_CSUM_FEATURES) != 0) + desc->sf_tx_ctrl |= htole32(SF_TX_DESC_CALTCP); + desc->sf_tx_ctrl |= + htole32(SF_TX_DESC_CRCEN | SF_TX_DESC_INTR | (nsegs << 16)); + + txd->tx_dmamap = map; + txd->tx_m = m; + txd->ndesc = nsegs + nskip; + + return (0); } static void -sf_start(ifp) - struct ifnet *ifp; +sf_start(struct ifnet *ifp) { struct sf_softc *sc; @@ -1404,91 +2240,88 @@ sf_start(ifp) } static void -sf_start_locked(ifp) - struct ifnet *ifp; +sf_start_locked(struct ifnet *ifp) { struct sf_softc *sc; - struct sf_tx_bufdesc_type0 *cur_tx = NULL; - struct mbuf *m_head = NULL; - int i, txprod; + struct mbuf *m_head; + int enq; sc = ifp->if_softc; SF_LOCK_ASSERT(sc); - if (!sc->sf_link && ifp->if_snd.ifq_len < 10) - return; - - if (ifp->if_drv_flags & IFF_DRV_OACTIVE) + if ((ifp->if_drv_flags & (IFF_DRV_RUNNING | IFF_DRV_OACTIVE)) != + IFF_DRV_RUNNING || sc->sf_link == 0) return; - txprod = csr_read_4(sc, SF_TXDQ_PRODIDX); - i = SF_IDX_HI(txprod) >> 4; - - if (sc->sf_ldata->sf_tx_dlist[i].sf_mbuf != NULL) { - if_printf(ifp, "TX ring full, resetting\n"); - sf_init_locked(sc); - txprod = csr_read_4(sc, SF_TXDQ_PRODIDX); - i = SF_IDX_HI(txprod) >> 4; - } - - while(sc->sf_ldata->sf_tx_dlist[i].sf_mbuf == NULL) { - if (sc->sf_tx_cnt >= (SF_TX_DLIST_CNT - 5)) { - ifp->if_drv_flags |= IFF_DRV_OACTIVE; - cur_tx = NULL; - break; - } + /* + * Since we don't know when descriptor wrap occurrs in advance + * limit available number of active Tx descriptor counter to be + * higher than maximum number of DMA segments allowed in driver. + */ + for (enq = 0; !IFQ_DRV_IS_EMPTY(&ifp->if_snd) && + sc->sf_cdata.sf_tx_cnt < SF_TX_DLIST_CNT - SF_MAXTXSEGS; ) { IFQ_DRV_DEQUEUE(&ifp->if_snd, m_head); if (m_head == NULL) break; - - cur_tx = &sc->sf_ldata->sf_tx_dlist[i]; - if (sf_encap(sc, cur_tx, m_head)) { + /* + * Pack the data into the transmit ring. If we + * don't have room, set the OACTIVE flag and wait + * for the NIC to drain the ring. + */ + if (sf_encap(sc, &m_head)) { + if (m_head == NULL) + break; IFQ_DRV_PREPEND(&ifp->if_snd, m_head); ifp->if_drv_flags |= IFF_DRV_OACTIVE; - cur_tx = NULL; break; } + enq++; /* * If there's a BPF listener, bounce a copy of this frame * to him. */ - BPF_MTAP(ifp, m_head); - - SF_INC(i, SF_TX_DLIST_CNT); - sc->sf_tx_cnt++; - /* - * Don't get the TX DMA queue get too full. - */ - if (sc->sf_tx_cnt > 64) - break; + ETHER_BPF_MTAP(ifp, m_head); } - if (cur_tx == NULL) - return; - - /* Transmit */ - csr_write_4(sc, SF_TXDQ_PRODIDX, - (txprod & ~SF_TXDQ_PRODIDX_HIPRIO) | - ((i << 20) & 0xFFFF0000)); + if (enq > 0) { + bus_dmamap_sync(sc->sf_cdata.sf_tx_ring_tag, + sc->sf_cdata.sf_tx_ring_map, + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); + /* Kick transmit. */ + csr_write_4(sc, SF_TXDQ_PRODIDX, + sc->sf_cdata.sf_tx_prod * (sizeof(struct sf_tx_rdesc) / 8)); - ifp->if_timer = 5; + /* Set a timeout in case the chip goes out to lunch. */ + sc->sf_watchdog_timer = 5; + } } static void -sf_stop(sc) - struct sf_softc *sc; +sf_stop(struct sf_softc *sc) { - int i; + struct sf_txdesc *txd; + struct sf_rxdesc *rxd; struct ifnet *ifp; + int i; SF_LOCK_ASSERT(sc); ifp = sc->sf_ifp; - callout_stop(&sc->sf_stat_callout); + ifp->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE); + sc->sf_link = 0; + callout_stop(&sc->sf_co); + sc->sf_watchdog_timer = 0; + + /* Reading the ISR register clears all interrrupts. */ + csr_read_4(sc, SF_ISR); + /* Disable further interrupts. */ + csr_write_4(sc, SF_IMR, 0); + /* Disable Tx/Rx egine. */ csr_write_4(sc, SF_GEN_ETH_CTL, 0); + csr_write_4(sc, SF_CQ_CONSIDX, 0); csr_write_4(sc, SF_CQ_PRODIDX, 0); csr_write_4(sc, SF_RXDQ_ADDR_Q1, 0); @@ -1497,25 +2330,48 @@ sf_stop(sc) csr_write_4(sc, SF_TXCQ_CTL, 0); csr_write_4(sc, SF_TXDQ_ADDR_HIPRIO, 0); csr_write_4(sc, SF_TXDQ_CTL, 0); - sf_reset(sc); - - sc->sf_link = 0; + /* + * Free RX and TX mbufs still in the queues. + */ for (i = 0; i < SF_RX_DLIST_CNT; i++) { - if (sc->sf_ldata->sf_rx_dlist_big[i].sf_mbuf != NULL) { - m_freem(sc->sf_ldata->sf_rx_dlist_big[i].sf_mbuf); - sc->sf_ldata->sf_rx_dlist_big[i].sf_mbuf = NULL; + rxd = &sc->sf_cdata.sf_rxdesc[i]; + if (rxd->rx_m != NULL) { + bus_dmamap_sync(sc->sf_cdata.sf_rx_tag, + rxd->rx_dmamap, BUS_DMASYNC_POSTREAD); + bus_dmamap_unload(sc->sf_cdata.sf_rx_tag, + rxd->rx_dmamap); + m_freem(rxd->rx_m); + rxd->rx_m = NULL; } - } - + } for (i = 0; i < SF_TX_DLIST_CNT; i++) { - if (sc->sf_ldata->sf_tx_dlist[i].sf_mbuf != NULL) { - m_freem(sc->sf_ldata->sf_tx_dlist[i].sf_mbuf); - sc->sf_ldata->sf_tx_dlist[i].sf_mbuf = NULL; + txd = &sc->sf_cdata.sf_txdesc[i]; + if (txd->tx_m != NULL) { + bus_dmamap_sync(sc->sf_cdata.sf_tx_tag, + txd->tx_dmamap, BUS_DMASYNC_POSTWRITE); + bus_dmamap_unload(sc->sf_cdata.sf_tx_tag, + txd->tx_dmamap); + m_freem(txd->tx_m); + txd->tx_m = NULL; + txd->ndesc = 0; } - } + } +} - ifp->if_drv_flags &= ~(IFF_DRV_RUNNING|IFF_DRV_OACTIVE); +static void +sf_tick(void *xsc) +{ + struct sf_softc *sc; + struct mii_data *mii; + + sc = xsc; + SF_LOCK_ASSERT(sc); + mii = device_get_softc(sc->sf_miibus); + mii_tick(mii); + sf_stats_update(sc); + sf_watchdog(sc); + callout_reset(&sc->sf_co, hz, sf_tick, sc); } /* @@ -1526,71 +2382,187 @@ sf_stop(sc) * be changed out from under us. */ static void -sf_stats_update(xsc) - void *xsc; +sf_stats_update(struct sf_softc *sc) { - struct sf_softc *sc; struct ifnet *ifp; - struct mii_data *mii; - struct sf_stats stats; - u_int32_t *ptr; + struct sf_stats now, *stats, *nstats; int i; - sc = xsc; SF_LOCK_ASSERT(sc); - ifp = sc->sf_ifp; - mii = device_get_softc(sc->sf_miibus); - - ptr = (u_int32_t *)&stats; - for (i = 0; i < sizeof(stats)/sizeof(u_int32_t); i++) - ptr[i] = csr_read_4(sc, SF_STATS_BASE + - (i + sizeof(u_int32_t))); - - for (i = 0; i < sizeof(stats)/sizeof(u_int32_t); i++) - csr_write_4(sc, SF_STATS_BASE + - (i + sizeof(u_int32_t)), 0); - ifp->if_collisions += stats.sf_tx_single_colls + - stats.sf_tx_multi_colls + stats.sf_tx_excess_colls; - - mii_tick(mii); + ifp = sc->sf_ifp; + stats = &now; + + stats->sf_tx_frames = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_FRAMES); + stats->sf_tx_single_colls = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_SINGLE_COL); + stats->sf_tx_multi_colls = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_MULTI_COL); + stats->sf_tx_crcerrs = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_CRC_ERRS); + stats->sf_tx_bytes = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_BYTES); + stats->sf_tx_deferred = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_DEFERRED); + stats->sf_tx_late_colls = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_LATE_COL); + stats->sf_tx_pause_frames = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_PAUSE); + stats->sf_tx_control_frames = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_CTL_FRAME); + stats->sf_tx_excess_colls = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_EXCESS_COL); + stats->sf_tx_excess_defer = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_EXCESS_DEF); + stats->sf_tx_mcast_frames = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_MULTI); + stats->sf_tx_bcast_frames = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_BCAST); + stats->sf_tx_frames_lost = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_FRAME_LOST); + stats->sf_rx_frames = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_FRAMES); + stats->sf_rx_crcerrs = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_CRC_ERRS); + stats->sf_rx_alignerrs = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_ALIGN_ERRS); + stats->sf_rx_bytes = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_BYTES); + stats->sf_rx_pause_frames = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_PAUSE); + stats->sf_rx_control_frames = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_CTL_FRAME); + stats->sf_rx_unsup_control_frames = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_UNSUP_FRAME); + stats->sf_rx_giants = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_GIANTS); + stats->sf_rx_runts = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_RUNTS); + stats->sf_rx_jabbererrs = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_JABBER); + stats->sf_rx_fragments = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_FRAGMENTS); + stats->sf_rx_pkts_64 = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_64); + stats->sf_rx_pkts_65_127 = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_65_127); + stats->sf_rx_pkts_128_255 = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_128_255); + stats->sf_rx_pkts_256_511 = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_256_511); + stats->sf_rx_pkts_512_1023 = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_512_1023); + stats->sf_rx_pkts_1024_1518 = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_1024_1518); + stats->sf_rx_frames_lost = + csr_read_4(sc, SF_STATS_BASE + SF_STATS_RX_FRAME_LOST); + /* Lower 16bits are valid. */ + stats->sf_tx_underruns = + (csr_read_4(sc, SF_STATS_BASE + SF_STATS_TX_UNDERRUN) & 0xffff); - if (!sc->sf_link && mii->mii_media_status & IFM_ACTIVE && - IFM_SUBTYPE(mii->mii_media_active) != IFM_NONE) { - sc->sf_link++; - if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) - sf_start_locked(ifp); - } + /* Empty stats counter registers. */ + for (i = SF_STATS_BASE; i < (SF_STATS_END + 1); i += sizeof(uint32_t)) + csr_write_4(sc, i, 0); - callout_reset(&sc->sf_stat_callout, hz, sf_stats_update, sc); + ifp->if_opackets += (u_long)stats->sf_tx_frames; + + ifp->if_collisions += (u_long)stats->sf_tx_single_colls + + (u_long)stats->sf_tx_multi_colls; + + ifp->if_oerrors += (u_long)stats->sf_tx_excess_colls + + (u_long)stats->sf_tx_excess_defer + + (u_long)stats->sf_tx_frames_lost; + + ifp->if_ipackets += (u_long)stats->sf_rx_frames; + + ifp->if_ierrors += (u_long)stats->sf_rx_crcerrs + + (u_long)stats->sf_rx_alignerrs + + (u_long)stats->sf_rx_giants + + (u_long)stats->sf_rx_runts + + (u_long)stats->sf_rx_jabbererrs + + (u_long)stats->sf_rx_frames_lost; + + nstats = &sc->sf_statistics; + + nstats->sf_tx_frames += stats->sf_tx_frames; + nstats->sf_tx_single_colls += stats->sf_tx_single_colls; + nstats->sf_tx_multi_colls += stats->sf_tx_multi_colls; + nstats->sf_tx_crcerrs += stats->sf_tx_crcerrs; + nstats->sf_tx_bytes += stats->sf_tx_bytes; + nstats->sf_tx_deferred += stats->sf_tx_deferred; + nstats->sf_tx_late_colls += stats->sf_tx_late_colls; + nstats->sf_tx_pause_frames += stats->sf_tx_pause_frames; + nstats->sf_tx_control_frames += stats->sf_tx_control_frames; + nstats->sf_tx_excess_colls += stats->sf_tx_excess_colls; + nstats->sf_tx_excess_defer += stats->sf_tx_excess_defer; + nstats->sf_tx_mcast_frames += stats->sf_tx_mcast_frames; + nstats->sf_tx_bcast_frames += stats->sf_tx_bcast_frames; + nstats->sf_tx_frames_lost += stats->sf_tx_frames_lost; + nstats->sf_rx_frames += stats->sf_rx_frames; + nstats->sf_rx_crcerrs += stats->sf_rx_crcerrs; + nstats->sf_rx_alignerrs += stats->sf_rx_alignerrs; + nstats->sf_rx_bytes += stats->sf_rx_bytes; + nstats->sf_rx_pause_frames += stats->sf_rx_pause_frames; + nstats->sf_rx_control_frames += stats->sf_rx_control_frames; + nstats->sf_rx_unsup_control_frames += stats->sf_rx_unsup_control_frames; + nstats->sf_rx_giants += stats->sf_rx_giants; + nstats->sf_rx_runts += stats->sf_rx_runts; + nstats->sf_rx_jabbererrs += stats->sf_rx_jabbererrs; + nstats->sf_rx_fragments += stats->sf_rx_fragments; + nstats->sf_rx_pkts_64 += stats->sf_rx_pkts_64; + nstats->sf_rx_pkts_65_127 += stats->sf_rx_pkts_65_127; + nstats->sf_rx_pkts_128_255 += stats->sf_rx_pkts_128_255; + nstats->sf_rx_pkts_256_511 += stats->sf_rx_pkts_256_511; + nstats->sf_rx_pkts_512_1023 += stats->sf_rx_pkts_512_1023; + nstats->sf_rx_pkts_1024_1518 += stats->sf_rx_pkts_1024_1518; + nstats->sf_rx_frames_lost += stats->sf_rx_frames_lost; + nstats->sf_tx_underruns += stats->sf_tx_underruns; } static void -sf_watchdog(ifp) - struct ifnet *ifp; +sf_watchdog(struct sf_softc *sc) { - struct sf_softc *sc; + struct ifnet *ifp; - sc = ifp->if_softc; + SF_LOCK_ASSERT(sc); - SF_LOCK(sc); + if (sc->sf_watchdog_timer == 0 || --sc->sf_watchdog_timer) + return; + + ifp = sc->sf_ifp; ifp->if_oerrors++; - if_printf(ifp, "watchdog timeout\n"); + if (sc->sf_link == 0) { + if (bootverbose) + if_printf(sc->sf_ifp, "watchdog timeout " + "(missed link)\n"); + } else + if_printf(ifp, "watchdog timeout, %d Tx descs are active\n", + sc->sf_cdata.sf_tx_cnt); - sf_stop(sc); - sf_reset(sc); sf_init_locked(sc); if (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) sf_start_locked(ifp); +} + +static int +sf_shutdown(device_t dev) +{ + struct sf_softc *sc; + + sc = device_get_softc(dev); + SF_LOCK(sc); + sf_stop(sc); SF_UNLOCK(sc); + + return (0); } static int -sf_shutdown(dev) - device_t dev; +sf_suspend(device_t dev) { struct sf_softc *sc; @@ -1598,7 +2570,147 @@ sf_shutdown(dev) SF_LOCK(sc); sf_stop(sc); + sc->sf_suspended = 1; + bus_generic_suspend(dev); + SF_UNLOCK(sc); + + return (0); +} + +static int +sf_resume(device_t dev) +{ + struct sf_softc *sc; + struct ifnet *ifp; + + sc = device_get_softc(dev); + + SF_LOCK(sc); + bus_generic_resume(dev); + ifp = sc->sf_ifp; + if ((ifp->if_flags & IFF_UP) != 0) + sf_init_locked(sc); + + sc->sf_suspended = 0; SF_UNLOCK(sc); return (0); } + +static int +sf_sysctl_stats(SYSCTL_HANDLER_ARGS) +{ + struct sf_softc *sc; + struct sf_stats *stats; + int error; + int result; + + result = -1; + error = sysctl_handle_int(oidp, &result, 0, req); + + if (error != 0 || req->newptr == NULL) + return (error); + + if (result != 1) + return (error); + + sc = (struct sf_softc *)arg1; + stats = &sc->sf_statistics; + + printf("%s statistics:\n", device_get_nameunit(sc->sf_dev)); + printf("Transmit good frames : %ju\n", + (uintmax_t)stats->sf_tx_frames); + printf("Transmit good octets : %ju\n", + (uintmax_t)stats->sf_tx_bytes); + printf("Transmit single collisions : %u\n", + stats->sf_tx_single_colls); + printf("Transmit multiple collisions : %u\n", + stats->sf_tx_multi_colls); + printf("Transmit late collisions : %u\n", + stats->sf_tx_late_colls); + printf("Transmit abort due to excessive collisions : %u\n", + stats->sf_tx_excess_colls); + printf("Transmit CRC errors : %u\n", + stats->sf_tx_crcerrs); + printf("Transmit deferrals : %u\n", + stats->sf_tx_deferred); + printf("Transmit abort due to excessive deferrals : %u\n", + stats->sf_tx_excess_defer); + printf("Transmit pause control frames : %u\n", + stats->sf_tx_pause_frames); + printf("Transmit control frames : %u\n", + stats->sf_tx_control_frames); + printf("Transmit good multicast frames : %u\n", + stats->sf_tx_mcast_frames); + printf("Transmit good broadcast frames : %u\n", + stats->sf_tx_bcast_frames); + printf("Transmit frames lost due to internal transmit errors : %u\n", + stats->sf_tx_frames_lost); + printf("Transmit FIFO underflows : %u\n", + stats->sf_tx_underruns); + printf("Transmit GFP stalls : %u\n", stats->sf_tx_gfp_stall); + printf("Receive good frames : %ju\n", + (uint64_t)stats->sf_rx_frames); + printf("Receive good octets : %ju\n", + (uint64_t)stats->sf_rx_bytes); + printf("Receive CRC errors : %u\n", + stats->sf_rx_crcerrs); + printf("Receive alignment errors : %u\n", + stats->sf_rx_alignerrs); + printf("Receive pause frames : %u\n", + stats->sf_rx_pause_frames); + printf("Receive control frames : %u\n", + stats->sf_rx_control_frames); + printf("Receive control frames with unsupported opcode : %u\n", + stats->sf_rx_unsup_control_frames); + printf("Receive frames too long : %u\n", + stats->sf_rx_giants); + printf("Receive frames too short : %u\n", + stats->sf_rx_runts); + printf("Receive frames jabber errors : %u\n", + stats->sf_rx_jabbererrs); + printf("Receive frames fragments : %u\n", + stats->sf_rx_fragments); + printf("Receive packets 64 bytes : %ju\n", + (uint64_t)stats->sf_rx_pkts_64); + printf("Receive packets 65 to 127 bytes : %ju\n", + (uint64_t)stats->sf_rx_pkts_65_127); + printf("Receive packets 128 to 255 bytes : %ju\n", + (uint64_t)stats->sf_rx_pkts_128_255); + printf("Receive packets 256 to 511 bytes : %ju\n", + (uint64_t)stats->sf_rx_pkts_256_511); + printf("Receive packets 512 to 1023 bytes : %ju\n", + (uint64_t)stats->sf_rx_pkts_512_1023); + printf("Receive packets 1024 to 1518 bytes : %ju\n", + (uint64_t)stats->sf_rx_pkts_1024_1518); + printf("Receive frames lost due to internal receive errors : %u\n", + stats->sf_rx_frames_lost); + printf("Receive GFP stalls : %u\n", stats->sf_rx_gfp_stall); + + return (error); +} + +static int +sysctl_int_range(SYSCTL_HANDLER_ARGS, int low, int high) +{ + int error, value; + + if (!arg1) + return (EINVAL); + value = *(int *)arg1; + error = sysctl_handle_int(oidp, &value, 0, req); + if (error || !req->newptr) + return (error); + if (value < low || value > high) + return (EINVAL); + *(int *)arg1 = value; + + return (0); +} + +static int +sysctl_hw_sf_int_mod(SYSCTL_HANDLER_ARGS) +{ + + return (sysctl_int_range(oidp, arg1, arg2, req, SF_IM_MIN, SF_IM_MAX)); +} diff --git a/sys/dev/sf/if_sfreg.h b/sys/dev/sf/if_sfreg.h index 29861ef343a..3ccfb8035a1 100644 --- a/sys/dev/sf/if_sfreg.h +++ b/sys/dev/sf/if_sfreg.h @@ -196,13 +196,13 @@ #define SF_TXDQ_CTL 0x0090 #define SF_TXDQ_ADDR_HIPRIO 0x0094 #define SF_TXDQ_ADDR_LOPRIO 0x0098 -#define SF_TXDQ_ADDR_HIADDR 0x009C +#define SF_TXDQ_ADDR_HI 0x009C #define SF_TXDQ_PRODIDX 0x00A0 #define SF_TXDQ_CONSIDX 0x00A4 #define SF_TXDMA_STS1 0x00A8 #define SF_TXDMA_STS2 0x00AC #define SF_TX_FRAMCTL 0x00B0 -#define SF_TXCQ_ADDR_HI 0x00B4 +#define SF_CQ_ADDR_HI 0x00B4 #define SF_TXCQ_CTL 0x00B8 #define SF_RXCQ_CTL_1 0x00BC #define SF_RXCQ_CTL_2 0x00C0 @@ -212,7 +212,7 @@ #define SF_RXDMA_CTL 0x00D0 #define SF_RXDQ_CTL_1 0x00D4 #define SF_RXDQ_CTL_2 0x00D8 -#define SF_RXDQ_ADDR_HIADDR 0x00DC +#define SF_RXDQ_ADDR_HI 0x00DC #define SF_RXDQ_ADDR_Q1 0x00E0 #define SF_RXDQ_ADDR_Q2 0x00E4 #define SF_RXDQ_PTR_Q1 0x00E8 @@ -249,6 +249,11 @@ #define SF_TIMER_RXQ1DONE_DLY 0x40000000 #define SF_TIMER_EARLYRX1_DLY 0x80000000 +/* Timer resolution is 0.8us * 128. */ +#define SF_IM_MIN 0 +#define SF_IM_MAX 0x1F /* 3.276ms */ +#define SF_IM_DEFAULT 1 /* 102.4us */ + /* Interrupt status register */ #define SF_ISR_PCIINT_ASSERTED 0x00000001 #define SF_ISR_GFP_TX 0x00000002 @@ -347,10 +352,11 @@ #define SF_INTRS \ (SF_IMR_RXDQ2_NOBUFS|SF_IMR_RXDQ1_DMADONE|SF_IMR_RXDQ2_DMADONE| \ - SF_IMR_TX_TXDONE|SF_IMR_RXDQ1_NOBUFS|SF_IMR_RXDQ2_DMADONE| \ + SF_IMR_TX_DMADONE|SF_IMR_RXDQ1_NOBUFS|SF_IMR_RXDQ2_DMADONE| \ SF_IMR_NORMALINTR|SF_IMR_ABNORMALINTR|SF_IMR_TXCQ_NOBUFS| \ SF_IMR_RXCQ1_NOBUFS|SF_IMR_RXCQ2_NOBUFS|SF_IMR_STATSOFLOW| \ - SF_IMR_TX_LOFIFO) + SF_IMR_TX_LOFIFO|SF_IMR_DMAERR|SF_IMR_RXGFP_NORESP| \ + SF_IMR_NO_TX_CSUM) /* TX descriptor queue control registers */ #define SF_TXDQCTL_DESCTYPE 0x00000007 @@ -361,6 +367,9 @@ #define SF_TXDQCTL_SKIPLEN 0x001F0000 #define SF_TXDQCTL_HIPRIOTHRESH 0xFF000000 +#define SF_TXDMA_HIPRIO_THRESH 2 +#define SF_TXDDMA_BURST (128 / 32) + #define SF_TXBUFDESC_TYPE0 0x00000000 #define SF_TXBUFDESC_TYPE1 0x00000001 #define SF_TXBUFDESC_TYPE2 0x00000002 @@ -434,9 +443,6 @@ #define SF_CQ_TXTHRMODE_INT_ON 0x80000000 #define SF_CQ_TXTHRMODE_INT_OFF 0x00000000 -#define SF_IDX_LO(x) ((x) & 0x000007FF) -#define SF_IDX_HI(x) (((x) >> 16) & 0x000007FF) - /* RX DMA control register */ #define SF_RXDMA_BURSTSIZE 0x0000007F #define SF_RXDMA_FPTESTMODE 0x00000080 @@ -453,6 +459,8 @@ #define SF_RXDMA_DMABADPKTS 0x20000000 #define SF_RXDMA_DMARUNTS 0x40000000 #define SF_RXDMA_REPORTBADPKTS 0x80000000 +#define SF_RXDMA_HIGHPRIO_THRESH 6 +#define SF_RXDMA_BURST (64 / 32) #define SF_RXDQMODE_Q1ONLY 0x00100000 #define SF_RXDQMODE_Q2_ON_FP 0x00200000 @@ -567,8 +575,8 @@ #define SF_MII_DATAPORT 0x0000FFFF #define SF_PHY_REG(phy, reg) \ - (SF_MIIADDR_BASE + (phy * SF_MII_BLOCKS * sizeof(u_int32_t)) + \ - (reg * sizeof(u_int32_t))) + (SF_MIIADDR_BASE + ((phy) * SF_MII_BLOCKS * sizeof(uint32_t)) + \ + ((reg) * sizeof(uint32_t))) /* * Ethernet extra registers 0x4000 to 0x4FFF @@ -613,6 +621,8 @@ #define SF_MACCFG1_MIILOOPBK 0x00004000 #define SF_MACCFG1_SOFTRESET 0x00008000 +#define SF_MACCFG2_AUTOVLANPAD 0x00000020 + /* * There are the recommended IPG nibble counter settings * specified in the Adaptec manual for full duplex and @@ -643,13 +653,53 @@ #define SF_STATS_BASE 0x7000 #define SF_STATS_END 0x7FFF +#define SF_STATS_TX_FRAMES 0x0000 +#define SF_STATS_TX_SINGLE_COL 0x0004 +#define SF_STATS_TX_MULTI_COL 0x0008 +#define SF_STATS_TX_CRC_ERRS 0x000C +#define SF_STATS_TX_BYTES 0x0010 +#define SF_STATS_TX_DEFERRED 0x0014 +#define SF_STATS_TX_LATE_COL 0x0018 +#define SF_STATS_TX_PAUSE 0x001C +#define SF_STATS_TX_CTL_FRAME 0x0020 +#define SF_STATS_TX_EXCESS_COL 0x0024 +#define SF_STATS_TX_EXCESS_DEF 0x0028 +#define SF_STATS_TX_MULTI 0x002C +#define SF_STATS_TX_BCAST 0x0030 +#define SF_STATS_TX_FRAME_LOST 0x0034 +#define SF_STATS_RX_FRAMES 0x0038 +#define SF_STATS_RX_CRC_ERRS 0x003C +#define SF_STATS_RX_ALIGN_ERRS 0x0040 +#define SF_STATS_RX_BYTES 0x0044 +#define SF_STATS_RX_PAUSE 0x0048 +#define SF_STATS_RX_CTL_FRAME 0x004C +#define SF_STATS_RX_UNSUP_FRAME 0x0050 +#define SF_STATS_RX_GIANTS 0x0054 +#define SF_STATS_RX_RUNTS 0x0058 +#define SF_STATS_RX_JABBER 0x005C +#define SF_STATS_RX_FRAGMENTS 0x0060 +#define SF_STATS_RX_64 0x0064 +#define SF_STATS_RX_65_127 0x0068 +#define SF_STATS_RX_128_255 0x006C +#define SF_STATS_RX_256_511 0x0070 +#define SF_STATS_RX_512_1023 0x0074 +#define SF_STATS_RX_1024_1518 0x0078 +#define SF_STATS_RX_FRAME_LOST 0x007C +#define SF_STATS_TX_UNDERRUN 0x0080 + /* * TX frame processor instruction space 0x8000 to 0x9FFF */ +#define SF_TXGFP_MEM_BASE 0x8000 +#define SF_TXGFP_MEM_END 0x8FFF +/* Number of bytes of an GFP instruction. */ +#define SF_GFP_INST_BYTES 6 /* * RX frame processor instruction space 0xA000 to 0xBFFF */ +#define SF_RXGFP_MEM_BASE 0xA000 +#define SF_RXGFP_MEM_END 0xBFFF /* * Ethernet FIFO access space 0xC000 to 0xDFFF @@ -663,52 +713,31 @@ * Descriptor data structures. */ - -/* Receive descriptor formats. */ -#define SF_RX_MINSPACING 8 -#define SF_RX_DLIST_CNT 256 -#define SF_RX_CLIST_CNT 1024 -#define SF_RX_HOSTADDR(x) (((x) >> 2) & 0x3FFFFFFF) - /* - * RX buffer descriptor type 0, 32-bit addressing. Note that we - * program the RX buffer queue control register(s) to allow a - * descriptor spacing of 16 bytes, which leaves room after each - * descriptor to store a pointer to the mbuf for each buffer. + * RX buffer descriptor type 0, 32-bit addressing. */ struct sf_rx_bufdesc_type0 { - u_int32_t sf_valid:1, - sf_end:1, - sf_addrlo:30; - u_int32_t sf_pad0; -#ifdef __i386__ - u_int32_t sf_pad1; -#endif - struct mbuf *sf_mbuf; + uint32_t sf_addrlo; +#define SF_RX_DESC_VALID 0x00000001 +#define SF_RX_DESC_END 0x00000002 }; /* - * RX buffer descriptor type 0, 64-bit addressing. + * RX buffer descriptor type 1, 64-bit addressing. */ struct sf_rx_bufdesc_type1 { - u_int32_t sf_valid:1, - sf_end:1, - sf_addrlo:30; - u_int32_t sf_addrhi; -#ifdef __i386__ - u_int32_t sf_pad; -#endif - struct mbuf *sf_mbuf; + uint64_t sf_addr; }; /* * RX completion descriptor, type 0 (short). */ struct sf_rx_cmpdesc_type0 { - u_int32_t sf_len:16, - sf_endidx:11, - sf_status1:3, - sf_id:2; + uint32_t sf_rx_status1; +#define SF_RX_CMPDESC_LEN 0x0000ffff +#define SF_RX_CMPDESC_EIDX 0x07ff0000 +#define SF_RX_CMPDESC_STAT1 0x38000000 +#define SF_RX_CMPDESC_ID 0x40000000 }; /* @@ -716,12 +745,10 @@ struct sf_rx_cmpdesc_type0 { * if this is a vlan-addressed packet, plus extended status. */ struct sf_rx_cmpdesc_type1 { - u_int32_t sf_len:16, - sf_endidx:11, - sf_status1:3, - sf_id:2; - u_int16_t sf_status2; - u_int16_t sf_vlanid; + uint32_t sf_rx_status1; + uint32_t sf_rx_status2; +#define SF_RX_CMPDESC_VLAN 0x0000ffff +#define SF_RX_CMPDESC_STAT2 0xffff0000 }; /* @@ -729,12 +756,9 @@ struct sf_rx_cmpdesc_type1 { * checksum instead of vlan tag, plus extended status. */ struct sf_rx_cmpdesc_type2 { - u_int32_t sf_len:16, - sf_endidx:11, - sf_status1:3, - sf_id:2; - u_int16_t sf_status2; - u_int16_t sf_cksum; + uint32_t sf_rx_status1; + uint32_t sf_rx_status2; +#define SF_RX_CMPDESC_CSUM2 0x0000ffff }; /* @@ -742,81 +766,64 @@ struct sf_rx_cmpdesc_type2 { * TCP/IP checksum, vlan tag plus priority, two extended status fields. */ struct sf_rx_cmpdesc_type3 { - u_int32_t sf_len:16, - sf_endidx:11, - sf_status1:3, - sf_id:2; - u_int32_t sf_startidx:10, - sf_status3:6, - sf_status2:16; - u_int16_t sf_cksum; - u_int16_t sf_vlanid_prio; - u_int32_t sf_timestamp; + uint32_t sf_rx_status1; + uint32_t sf_rx_status2; + uint32_t sf_rx_status3; +#define SF_RX_CMPDESC_CSUM3 0xffff0000 +#define SF_RX_CMPDESC_VLANPRI 0x0000ffff + uint32_t sf_rx_timestamp; }; -#define SF_RXSTAT1_QUEUE 0x1 -#define SF_RXSTAT1_FIFOFULL 0x2 -#define SF_RXSTAT1_OK 0x4 - - /* 0=unknown,5=unsupported */ -#define SF_RXSTAT2_FRAMETYPE 0x0007 /* 1=IPv4,2=IPv2,3=IPX,4=ICMP */ -#define SF_RXSTAT2_UDP 0x0008 -#define SF_RXSTAT2_TCP 0x0010 -#define SF_RXSTAT2_FRAG 0x0020 -#define SF_RXSTAT2_PCSUM_OK 0x0040 /* partial checksum ok */ -#define SF_RXSTAT2_CSUM_BAD 0x0080 /* TCP/IP checksum bad */ -#define SF_RXSTAT2_CSUM_OK 0x0100 /* TCP/IP checksum ok */ -#define SF_RXSTAT2_VLAN 0x0200 -#define SF_RXSTAT2_BADRXCODE 0x0400 -#define SF_RXSTAT2_DRIBBLE 0x0800 -#define SF_RXSTAT2_ISL_CRCERR 0x1000 -#define SF_RXSTAT2_CRCERR 0x2000 -#define SF_RXSTAT2_HASH 0x4000 -#define SF_RXSTAT2_PERFECT 0x8000 - -#define SF_RXSTAT3_TRAILER 0x01 -#define SF_RXSTAT3_HEADER 0x02 -#define SF_RXSTAT3_CONTROL 0x04 -#define SF_RXSTAT3_PAUSE 0x08 -#define SF_RXSTAT3_ISL 0x10 - -/* - * Transmit descriptor formats. - * Each transmit descriptor type allows for a skip field at the - * start of each structure. The size of the skip field can vary, - * however we always set it for 8 bytes, which is enough to hold - * a pointer (32 bits on x86, 64-bits on alpha) that we can use - * to hold the address of the head of the mbuf chain for the - * frame or fragment associated with the descriptor. This saves - * us from having to create a separate pointer array to hold - * the mbuf addresses. - */ -#define SF_TX_BUFDESC_ID 0xB -#define SF_MAXFRAGS 14 -#define SF_TX_MINSPACING 128 -#define SF_TX_DLIST_CNT 128 -#define SF_TX_DLIST_SIZE 16384 -#define SF_TX_SKIPLEN 1 -#define SF_TX_CLIST_CNT 1024 +#define SF_RXSTAT1_QUEUE 0x08000000 +#define SF_RXSTAT1_FIFOFULL 0x10000000 +#define SF_RXSTAT1_OK 0x20000000 + +#define SF_RXSTAT2_FRAMETYPE_MASK 0x00070000 +#define SF_RXSTAT2_FRAMETYPE_UNKN 0x00000000 +#define SF_RXSTAT2_FRAMETYPE_IPV4 0x00010000 +#define SF_RXSTAT2_FRAMETYPE_IPV6 0x00020000 +#define SF_RXSTAT2_FRAMETYPE_IPX 0x00030000 +#define SF_RXSTAT2_FRAMETYPE_ICMP 0x00040000 +#define SF_RXSTAT2_FRAMETYPE_UNSPRT 0x00050000 +#define SF_RXSTAT2_UDP 0x00080000 +#define SF_RXSTAT2_TCP 0x00100000 +#define SF_RXSTAT2_FRAG 0x00200000 +#define SF_RXSTAT2_PCSUM_OK 0x00400000 /* partial checksum ok */ +#define SF_RXSTAT2_CSUM_BAD 0x00800000 /* TCP/IP checksum bad */ +#define SF_RXSTAT2_CSUM_OK 0x01000000 /* TCP/IP checksum ok */ +#define SF_RXSTAT2_VLAN 0x02000000 +#define SF_RXSTAT2_BADRXCODE 0x04000000 +#define SF_RXSTAT2_DRIBBLE 0x08000000 +#define SF_RXSTAT2_ISL_CRCERR 0x10000000 +#define SF_RXSTAT2_CRCERR 0x20000000 +#define SF_RXSTAT2_HASH 0x40000000 +#define SF_RXSTAT2_PERFECT 0x80000000 +#define SF_RXSTAT2_MASK 0xFFFF0000 + +#define SF_RXSTAT3_ISL 0x00008000 +#define SF_RXSTAT3_PAUSE 0x00004000 +#define SF_RXSTAT3_CONTROL 0x00002000 +#define SF_RXSTAT3_HEADER 0x00001000 +#define SF_RXSTAT3_TRAILER 0x00000800 +#define SF_RXSTAT3_START_IDX_MASK 0x000007FF struct sf_frag { - u_int32_t sf_addr; - u_int16_t sf_fraglen; - u_int16_t sf_pktlen; + uint32_t sf_addr; + uint16_t sf_fraglen; + uint16_t sf_pktlen; }; struct sf_frag_msdos { - u_int16_t sf_pktlen; - u_int16_t sf_fraglen; - u_int32_t sf_addr; + uint16_t sf_pktlen; + uint16_t sf_fraglen; + uint32_t sf_addr; }; /* * TX frame descriptor type 0, 32-bit addressing. One descriptor can - * be used to map multiple packet fragments. We use this format since - * BSD networking fragments packet data across mbuf chains. Note that - * the number of fragments can be variable depending on how the descriptor - * spacing is specified in the TX descriptor queue control register. + * be used to map multiple packet fragments. Note that the number of + * fragments can be variable depending on how the descriptor spacing + * is specified in the TX descriptor queue control register. * We always use a spacing of 128 bytes, and a skipfield length of 8 * bytes: this means 16 bytes for the descriptor, including the skipfield, * with 121 bytes left for fragment maps. Each fragment requires 8 bytes, @@ -825,20 +832,18 @@ struct sf_frag_msdos { * 128 bytes per descriptor, we have room for 128 descriptors in the queue. */ struct sf_tx_bufdesc_type0 { -#ifdef __i386__ - u_int32_t sf_pad; -#endif - struct mbuf *sf_mbuf; - u_int32_t sf_rsvd0:24, - sf_crcen:1, - sf_caltcp:1, - sf_end:1, - sf_intr:1, - sf_id:4; - u_int8_t sf_fragcnt; - u_int8_t sf_rsvd2; - u_int16_t sf_rsvd1; - struct sf_frag sf_frags[14]; + uint32_t sf_tx_ctrl; +#define SF_TX_DESC_CRCEN 0x01000000 +#define SF_TX_DESC_CALTCP 0x02000000 +#define SF_TX_DESC_END 0x04000000 +#define SF_TX_DESC_INTR 0x08000000 +#define SF_TX_DESC_ID 0xb0000000 + uint32_t sf_tx_frag; + /* + * Depending on descriptor spacing/skip field length it + * can have fixed number of struct sf_frag. + * struct sf_frag sf_frags[14]; + */ }; /* @@ -846,18 +851,10 @@ struct sf_tx_bufdesc_type0 { * maps a single fragment. */ struct sf_tx_bufdesc_type1 { -#ifdef __i386__ - u_int32_t sf_pad; -#endif - struct mbuf *sf_mbuf; - u_int32_t sf_fraglen:16, - sf_fragcnt:8, - sf_crcen:1, - sf_caltcp:1, - sf_end:1, - sf_intr:1, - sf_id:4; - u_int32_t sf_addr; + uint32_t sf_tx_ctrl; +#define SF_TX_DESC_FRAGLEN 0x0000ffff +#define SF_TX_DESC_FRAGCNT 0x00ff0000 + uint32_t sf_addrlo; }; /* @@ -865,19 +862,9 @@ struct sf_tx_bufdesc_type1 { * maps a single fragment. */ struct sf_tx_bufdesc_type2 { -#ifdef __i386__ - u_int32_t sf_pad; -#endif - struct mbuf *sf_mbuf; - u_int32_t sf_fraglen:16, - sf_fragcnt:8, - sf_crcen:1, - sf_caltcp:1, - sf_end:1, - sf_intr:1, - sf_id:4; - u_int32_t sf_addrlo; - u_int32_t sf_addrhi; + uint32_t sf_tx_ctrl; + uint32_t sf_tx_reserved; + uint64_t sf_addr; }; /* TX buffer descriptor type 3 is not defined. */ @@ -889,20 +876,14 @@ struct sf_tx_bufdesc_type2 { * to optimize copies in MS-DOS and OS/2 drivers. */ struct sf_tx_bufdesc_type4 { -#ifdef __i386__ - u_int32_t sf_pad; -#endif - struct mbuf *sf_mbuf; - u_int32_t sf_rsvd0:24, - sf_crcen:1, - sf_caltcp:1, - sf_end:1, - sf_intr:1, - sf_id:4; - u_int8_t sf_fragcnt; - u_int8_t sf_rsvd2; - u_int16_t sf_rsvd1; - struct sf_frag_msdos sf_frags[14]; + uint32_t sf_tx_ctrl; + uint32_t sf_tx_frag; + /* + * Depending on descriptor spacing/skip field length it + * can have fixed number of struct sf_frag_msdos. + * + * struct sf_frag_msdos sf_frags[14]; + */ }; /* @@ -912,139 +893,203 @@ struct sf_tx_bufdesc_type4 { /* * Transmit DMA completion descriptor, type 0. */ -#define SF_TXCMPTYPE_DMA 0x4 +#define SF_TXCMPTYPE_DMA 0x80000000 +#define SF_TXCMPTYPE_TX 0xa0000000 struct sf_tx_cmpdesc_type0 { - u_int32_t sf_index:15, - sf_priority:1, - sf_timestamp:13, - sf_type:3; + uint32_t sf_tx_status1; +#define SF_TX_CMPDESC_IDX 0x00007fff +#define SF_TX_CMPDESC_HIPRI 0x00008000 +#define SF_TX_CMPDESC_STAT 0x1fff0000 +#define SF_TX_CMPDESC_TYPE 0xe0000000 }; /* * Transmit completion descriptor, type 1. */ -#define SF_TXCMPTYPE_TX 0x5 struct sf_tx_cmpdesc_type1 { - u_int32_t sf_index:15, - sf_priority:1, - sf_txstat:13, - sf_type:3; + uint32_t sf_tx_status1; + uint32_t sf_tx_status2; }; -#define SF_TXSTAT_CRCERR 0x0001 -#define SF_TXSTAT_LENCHECKERR 0x0002 -#define SF_TXSTAT_LENRANGEERR 0x0004 -#define SF_TXSTAT_TX_OK 0x0008 -#define SF_TXSTAT_TX_DEFERED 0x0010 -#define SF_TXSTAT_EXCESS_DEFER 0x0020 -#define SF_TXSTAT_EXCESS_COLL 0x0040 -#define SF_TXSTAT_LATE_COLL 0x0080 -#define SF_TXSTAT_TOOBIG 0x0100 -#define SF_TXSTAT_TX_UNDERRUN 0x0200 -#define SF_TXSTAT_CTLFRAME_OK 0x0400 -#define SF_TXSTAT_PAUSEFRAME_OK 0x0800 -#define SF_TXSTAT_PAUSED 0x1000 +#define SF_TXSTAT_CRCERR 0x00010000 +#define SF_TXSTAT_LENCHECKERR 0x00020000 +#define SF_TXSTAT_LENRANGEERR 0x00040000 +#define SF_TXSTAT_TX_OK 0x00080000 +#define SF_TXSTAT_TX_DEFERED 0x00100000 +#define SF_TXSTAT_EXCESS_DEFER 0x00200000 +#define SF_TXSTAT_EXCESS_COLL 0x00400000 +#define SF_TXSTAT_LATE_COLL 0x00800000 +#define SF_TXSTAT_TOOBIG 0x01000000 +#define SF_TXSTAT_TX_UNDERRUN 0x02000000 +#define SF_TXSTAT_CTLFRAME_OK 0x04000000 +#define SF_TXSTAT_PAUSEFRAME_OK 0x08000000 +#define SF_TXSTAT_PAUSED 0x10000000 /* Statistics counters. */ struct sf_stats { - u_int32_t sf_tx_frames; - u_int32_t sf_tx_single_colls; - u_int32_t sf_tx_multi_colls; - u_int32_t sf_tx_crcerrs; - u_int32_t sf_tx_bytes; - u_int32_t sf_tx_defered; - u_int32_t sf_tx_late_colls; - u_int32_t sf_tx_pause_frames; - u_int32_t sf_tx_control_frames; - u_int32_t sf_tx_excess_colls; - u_int32_t sf_tx_excess_defer; - u_int32_t sf_tx_mcast_frames; - u_int32_t sf_tx_bcast_frames; - u_int32_t sf_tx_frames_lost; - u_int32_t sf_rx_rx_frames; - u_int32_t sf_rx_crcerrs; - u_int32_t sf_rx_alignerrs; - u_int32_t sf_rx_bytes; - u_int32_t sf_rx_control_frames; - u_int32_t sf_rx_unsup_control_frames; - u_int32_t sf_rx_giants; - u_int32_t sf_rx_runts; - u_int32_t sf_rx_jabbererrs; - u_int32_t sf_rx_pkts_64; - u_int32_t sf_rx_pkts_65_127; - u_int32_t sf_rx_pkts_128_255; - u_int32_t sf_rx_pkts_256_511; - u_int32_t sf_rx_pkts_512_1023; - u_int32_t sf_rx_pkts_1024_1518; - u_int32_t sf_rx_frames_lost; - u_int16_t sf_tx_underruns; - u_int16_t sf_pad; + uint64_t sf_tx_frames; + uint32_t sf_tx_single_colls; + uint32_t sf_tx_multi_colls; + uint32_t sf_tx_crcerrs; + uint64_t sf_tx_bytes; + uint32_t sf_tx_deferred; + uint32_t sf_tx_late_colls; + uint32_t sf_tx_pause_frames; + uint32_t sf_tx_control_frames; + uint32_t sf_tx_excess_colls; + uint32_t sf_tx_excess_defer; + uint32_t sf_tx_mcast_frames; + uint32_t sf_tx_bcast_frames; + uint32_t sf_tx_frames_lost; + uint64_t sf_rx_frames; + uint32_t sf_rx_crcerrs; + uint32_t sf_rx_alignerrs; + uint64_t sf_rx_bytes; + uint32_t sf_rx_pause_frames; + uint32_t sf_rx_control_frames; + uint32_t sf_rx_unsup_control_frames; + uint32_t sf_rx_giants; + uint32_t sf_rx_runts; + uint32_t sf_rx_jabbererrs; + uint32_t sf_rx_fragments; + uint64_t sf_rx_pkts_64; + uint64_t sf_rx_pkts_65_127; + uint64_t sf_rx_pkts_128_255; + uint64_t sf_rx_pkts_256_511; + uint64_t sf_rx_pkts_512_1023; + uint64_t sf_rx_pkts_1024_1518; + uint32_t sf_rx_frames_lost; + uint32_t sf_tx_underruns; + uint32_t sf_tx_gfp_stall; + uint32_t sf_rx_gfp_stall; }; /* * register space access macros */ #define CSR_WRITE_4(sc, reg, val) \ - bus_space_write_4(sc->sf_btag, sc->sf_bhandle, reg, val) + bus_write_4((sc)->sf_res, reg, val) #define CSR_READ_4(sc, reg) \ - bus_space_read_4(sc->sf_btag, sc->sf_bhandle, reg) + bus_read_4((sc)->sf_res, reg) #define CSR_READ_1(sc, reg) \ - bus_space_read_1(sc->sf_btag, sc->sf_bhandle, reg) + bus_read_1((sc)->sf_res, reg) struct sf_type { - u_int16_t sf_vid; - u_int16_t sf_did; + uint16_t sf_vid; + uint16_t sf_did; char *sf_name; + uint16_t sf_sdid; + char *sf_sname; }; -#define SF_INC(x, y) (x) = (x + 1) % y +/* Use Tx descriptor type 2 : 64bit buffer descriptor */ +#define sf_tx_rdesc sf_tx_bufdesc_type2 +/* Use Rx descriptor type 1 : 64bit buffer descriptor */ +#define sf_rx_rdesc sf_rx_bufdesc_type1 +/* Use Tx completion type 0 */ +#define sf_tx_rcdesc sf_tx_cmpdesc_type0 +/* Use Rx completion type 2 : checksum */ +#define sf_rx_rcdesc sf_rx_cmpdesc_type2 -#define ETHER_ALIGN 2 +#define SF_TX_DLIST_CNT 256 +#define SF_RX_DLIST_CNT 256 +#define SF_TX_CLIST_CNT 1024 +#define SF_RX_CLIST_CNT 1024 +#define SF_TX_DLIST_SIZE (sizeof(struct sf_tx_rdesc) * SF_TX_DLIST_CNT) +#define SF_TX_CLIST_SIZE (sizeof(struct sf_tx_rcdesc) * SF_TX_CLIST_CNT) +#define SF_RX_DLIST_SIZE (sizeof(struct sf_rx_rdesc) * SF_RX_DLIST_CNT) +#define SF_RX_CLIST_SIZE (sizeof(struct sf_rx_rcdesc) * SF_RX_CLIST_CNT) +#define SF_RING_ALIGN 256 +#define SF_RX_ALIGN sizeof(uint32_t) +#define SF_MAXTXSEGS 16 + +#define SF_ADDR_LO(x) ((uint64_t)(x) & 0xffffffff) +#define SF_ADDR_HI(x) ((uint64_t)(x) >> 32) +#define SF_TX_DLIST_ADDR(sc, i) \ + ((sc)->sf_rdata.sf_tx_ring_paddr + sizeof(struct sf_tx_rdesc) * (i)) +#define SF_TX_CLIST_ADDR(sc, i) \ + ((sc)->sf_rdata.sf_tx_cring_paddr + sizeof(struct sf_tx_crdesc) * (i)) +#define SF_RX_DLIST_ADDR(sc, i) \ + ((sc)->sf_rdata.sf_rx_ring_paddr + sizeof(struct sf_rx_rdesc) * (i)) +#define SF_RX_CLIST_ADDR(sc, i) \ + ((sc)->sf_rdata.sf_rx_cring_paddr + sizeof(struct sf_rx_rcdesc) * (i)) + +#define SF_INC(x, y) (x) = ((x) + 1) % y + +#define SF_MAX_FRAMELEN 1536 +#define SF_TX_THRESHOLD_UNIT 16 +#define SF_MAX_TX_THRESHOLD (SF_MAX_FRAMELEN / SF_TX_THRESHOLD_UNIT) +#define SF_MIN_TX_THRESHOLD (128 / SF_TX_THRESHOLD_UNIT) + +struct sf_txdesc { + struct mbuf *tx_m; + int ndesc; + bus_dmamap_t tx_dmamap; +}; -/* - * Note: alignment is important here: each list must be aligned to - * a 256-byte boundary. It turns out that each ring is some multiple - * of 4K in length, so we can stack them all on top of each other - * and just worry about aligning the whole mess. There's one transmit - * buffer ring and two receive buffer rings: one RX ring is for small - * packets and the other is for large packets. Each buffer ring also - * has a companion completion queue. - */ -struct sf_list_data { - struct sf_tx_bufdesc_type0 sf_tx_dlist[SF_TX_DLIST_CNT]; - struct sf_tx_cmpdesc_type1 sf_tx_clist[SF_TX_CLIST_CNT]; - struct sf_rx_bufdesc_type0 sf_rx_dlist_big[SF_RX_DLIST_CNT]; -#ifdef notdef - /* - * Unfortunately, because the Starfire doesn't allow arbitrary - * byte alignment, we have to copy packets in the RX handler in - * order to align the payload correctly. This means that we - * don't gain anything by having separate large and small descriptor - * lists, so for now we don't bother with the small one. - */ - struct sf_rx_bufdesc_type0 sf_rx_dlist_small[SF_RX_DLIST_CNT]; -#endif - struct sf_rx_cmpdesc_type3 sf_rx_clist[SF_RX_CLIST_CNT]; +struct sf_rxdesc { + struct mbuf *rx_m; + bus_dmamap_t rx_dmamap; }; +struct sf_chain_data { + bus_dma_tag_t sf_parent_tag; + bus_dma_tag_t sf_tx_tag; + struct sf_txdesc sf_txdesc[SF_TX_DLIST_CNT]; + bus_dma_tag_t sf_rx_tag; + struct sf_rxdesc sf_rxdesc[SF_RX_DLIST_CNT]; + bus_dma_tag_t sf_tx_ring_tag; + bus_dma_tag_t sf_rx_ring_tag; + bus_dma_tag_t sf_tx_cring_tag; + bus_dma_tag_t sf_rx_cring_tag; + bus_dmamap_t sf_tx_ring_map; + bus_dmamap_t sf_rx_ring_map; + bus_dmamap_t sf_rx_sparemap; + bus_dmamap_t sf_tx_cring_map; + bus_dmamap_t sf_rx_cring_map; + int sf_tx_prod; + int sf_tx_cnt; + int sf_txc_cons; + int sf_rxc_cons; +}; + +struct sf_ring_data { + struct sf_tx_rdesc *sf_tx_ring; + bus_addr_t sf_tx_ring_paddr; + struct sf_tx_rcdesc *sf_tx_cring; + bus_addr_t sf_tx_cring_paddr; + struct sf_rx_rdesc *sf_rx_ring; + bus_addr_t sf_rx_ring_paddr; + struct sf_rx_rcdesc *sf_rx_cring; + bus_addr_t sf_rx_cring_paddr; +}; + + struct sf_softc { struct ifnet *sf_ifp; /* interface info */ device_t sf_dev; /* device info */ - bus_space_handle_t sf_bhandle; /* bus space handle */ - bus_space_tag_t sf_btag; /* bus space tag */ void *sf_intrhand; /* interrupt handler cookie */ struct resource *sf_irq; /* irq resource descriptor */ struct resource *sf_res; /* mem/ioport resource */ + int sf_restype; + int sf_rid; struct sf_type *sf_info; /* Starfire adapter info */ device_t sf_miibus; - struct sf_list_data *sf_ldata; - int sf_tx_cnt; - u_int8_t sf_link; + struct sf_chain_data sf_cdata; + struct sf_ring_data sf_rdata; int sf_if_flags; - struct callout sf_stat_callout; + struct callout sf_co; + int sf_watchdog_timer; + struct task sf_link_task; + int sf_link; + int sf_suspended; + int sf_detach; + uint32_t sf_txthresh; + int sf_int_mod; + struct sf_stats sf_statistics; struct mtx sf_mtx; #ifdef DEVICE_POLLING int rxcycles; -- 2.45.0