# Teaming Tips and Advanced Network Services Driver Installation

## Introduction

iANS.LAN is the Intel software for NetWare 4.11 and higher that provides a variety of advanced networking services (ANS). These benefits include the teaming and VLAN based features:

- Adapter Fault Tolerance (AFT)
- Adaptive Load Balancing (ALB)
- Intel LSO Link Aggregation
- Cisco* Fast EtherChannel* (FEC)
- Gigabit EtherChannel* (GEC)
- 802.3ad Link Aggregation
- VLAN (Virtual LAN), compatible with IEEE 802.1q

### Descriptions of the various modes and their specific requirements are:

AFT (Adapter Fault Tolerance). Default mode. A secondary adapter in the team will automatically take over on any failure of the primary connection (cable, adapter or port). Only one card is active at a time. If there is a failover to a second card, control passes back to the primary adapter when it is restored. May use a mixed speed team. May use a hub or any switch.

ALB (Adaptive Load Balancing or asymmetrical link aggregation). A team of 2-8 adapters share the transmission load while the primary receives and transmits. Includes fault tolerance support. Should be a same speed team. May use any hub or switch.

FEC (Cisco Fast EtherChannel* Technology and Intel Link Aggregation). A team of 2-8 adapters which simultaneously receive and transmit. Includes fault tolerance support. Should be a same-speed team. Requires a switch that is compatible with Cisco's FEC, Intel's LSO Link Aggregation or a compatible link aggregation protocol. Spanning Tree Protocol must be turned off.

GEC - Cisco's FEC equivalent for Gigabit. Requires GEC compatible switch and Gigabit Adapters.

IEEE 802.3ad. IEEE 802.3ad is a performance technology standard that increases your server's throughput. It is similar to the Fast EtherChannel (FEC) standard developed by Cisco. However, whereas FEC works only with FEC-compatible Cisco switches, IEEE 802.3ad works with those switches that support IEEE 802.3ad.

You may also identify which adapters in a team you want to use as the primary and secondary cards. The Primary will be the first card to be used in a team. For fault tolerance, the primary is the only card used until a failure. For ALB, the primary is the card that both transmits and receives. The Secondary is the preferred adapter (in a more than 2 card team) to fill the role of primary adapter upon a failure of the primary.

VLAN (Virtual LAN):

VLANs will be enabled if a VLAN ID is stated when iANS is first loaded. VLAN is a network segment formed through software rather than hardware. VLANs provide secure access, segmented broadcast traffic, and centralized management for improved performance. Requires a VLAN capable switch (10, 100 or 1000). Connection must be made to ports tagged with VLAN IDs that are identical to those assigned to the adapter. 802.1q support is provided by all Intel PRO/100 adapters (except the PRO/100B and the PRO/1000 adapters released prior to the year 2000).

## Requirements

- These advanced networking features only work with Intel Server adapters.
- Each team requires at least one Server adapter.
- All adapters must use the same frame type parameters.

Some features require a connection to a switch that has link aggregation and/or VLAN capability. Link all adapters of an ALB, FEC, GEC or 802.3 AD team to the same segment using the same speed and duplex, and disable Spanning Tree Protocol (STP), otherwise performance will be greatly degraded. AFT teams must also be connected to the same segment but may consist of adapters using different speeds and duplex modes. If using different speed adapters on an AFT team, set the fastest adapter as the "primary". When using iANS.LAN, do NOT bind the network protocols (IPX, IP, etc) directly to the base driver of an adapter used with iANS. Instead, bind iANS to the base driver and the protocol(s) to iANS. Doing otherwise can cause routing error messages, but most likely the protocols bound directly to the base drivers will simply not work.

Some advanced features are also adapter dependent as the table below depicts.

The term "Teaming" used here implies teaming modes of AFT(T), ALB(B), FEC(F), GEC(G) and 802.3ad(8).

Server Adapters Supported |  | Teaming |  | IEEE 802.1q VLAN
PRO/1000 (F or T) Gigabit Server Adapter |  | TBG8 |  | X
PRO/100+ Server Adapter |  | TBF8 |  | X
PRO/100+ Dual Port Server Adapter |  | TBF8 |  | X
PRO/100 and PRO/100 S Server Adapters |  | TBF8 |  | X

Note: Only adapters using Fault Tolerance mode can use mixed speed teams. All the other team types require the same speed and duplex. All PRO/100 adapters except the PRO/100B adapter will work for 802.1q VLAN.

For more information on features, switches, and specific adapter compatibility, see the Intel web site.

## Location of ANS driver (required for teaming and VLAN):

For PRO/100 and PRO/1000 server adapters: \NWSERVER\iANS.LAN

### Prerequisites for ANS

- All adapters that will be part of a team must be installed in the system.
- You must have loaded the base drivers. Do not bind the protocols to these base drivers because the protocols will be bound to the intermediate driver (iANS.LAN).
- Do not use the install or nwconfig programs to load ians.lan. Use them for base driver installation and then use the documentation, especially the examples page, to setup your desired configuration.

## Command Lines

Basic Structure:
LOAD base driver (e.g. CE100B.LAN) with appropriate parameters.

LOAD iANS driver

- For a VLAN you would include a VLAN ID number.
- For a VLAN or multiple frame type configuration, include a name.
- For a multiple team setup include a team number.

BIND iANS driver to adapter name.

- If creating multiple teams, include the team # here to define which team uses which adapter(s).
- If you want to specify which adapter is to be primary or secondary, list "primary" or "secondary" on this line.

LOAD iANS COMMIT <mode>. Creates the team.

- (<Mode> will distinguish among AFT, ALB, FEC, GEC or AD.)
- If creating multiple teams, include the team #.

BIND the protocol to the iANS driver using the name that was assigned when iANS was loaded.

- If you are using only one frame type, a single team and no VLAN, you can bind the protocol directly to the iANS driver without using an assigned name. Include a net=n number (especially in a script since the system will request the number if not given.)

Example of Mixed Speed Fault Tolerance Team:

Load CE100b name=100Meg
Load CE1000 name=Gigabit
Load iANS
Bind iANS Gigabit Primary
Bind iANS 100Meg
Load iANS COMMIT MODE = AFT
Bind IPX iANS Net=2

For more information on specific load commands for AFT, ALB, FEC, GEC, AD or VLANs see the Adapter Teaming Examples page.

## VLAN Command Lines

### Allocating Memory for VLANS:

When using multiple VLANs, the server's default packet buffers will probably need to be increased. To do this, add the following lines to the STARTUP.NCF file which is located in the same directory that NetWare is launched from. Usually C:\NWSERVER\STARTUP.NCF.

SET MINIMUM PACKET RECEIVE BUFFERS = 200 (or higher)
SET MAXIMUM PACKET RECEIVE BUFFERS = 500 (or higher)

"SET MINIMUM PACKET" designates the minimum number of packet receive buffers the system will allocate and "SET MAXIMUM PACKET" designates the maximum. Make sure that the maximum setting is equal to or greater than the minimum setting. The number of buffers required is based on the number of VLANs and whether or not load balancing, FEC, GEC, or AD is in use.

For every VLAN used, iANS will request 64 buffers. When in load sharing modes (ALB, FEC, GEC and AD) 64 buffers are needed for each adapter in the team. The non load sharing fault tolerant mode (AFT) only requires the 64 buffers for one of the adapters in the team.

As an example, an ALB team of 2 adapters that uses 12 VLANs would require 64 [buffers] * 2 [adapters] * 12 [VLANs] = 1536 for the minimum packet receive buffers. This number is in addition to any other buffers that the server may require for other purposes.

The default amount of memory used by each NetWare receive buffer is approximately 4K (varies slightly with different versions). If the extra memory requirement for VLAN is a problem, there are several things that can be done to reduce the impact.

If Ethernet is the only network topology that the server uses, the size of the buffer can be reduced to 2000 bytes (the maximum Ethernet frame size plus some out of band data used by iANS) without impacting the performance of the server. This is done by adding the following line to the STARTUP.NCF file:

SET MAXIMUM PHYSICAL RECEIVE PACKET SIZE = 2048

Note that this will not work if Ethernet "jumbo frames" are in use. In fact, the packet size will have to be increased to accommodate the Jumbo Frames. Jumbo frames are supported only with the Gigabit adapter and requires switch infrastructure that supports Jumbo Frames. A keyword is included for iANS that allows the administrator to reduce the buffer requirement per VLAN from 64 down to as low as 32, however, this will negatively impact the server's performance. The syntax is:

LOAD IANS TX_ECBS_TO_USE = X

Where "X" is the number of buffers to use for each VLAN.

Example for a Single 802.1q VLAN Team Using a Single Adapter:

Load ce100b slot=5 frame=ethernet_802.3 name=e83
Load ians vlanid=2 frame=ethernet_802.3 name=T1-VL2 team=1
Load ians vlanid=3 frame=ethernet_802.3 name=T1-VL3 team=1
Bind ians e83 team=1
Load ians commit mode=AFT team=1
Bind ipx T1-VL2 net=2
Bind ipx T1-VL3 net=3

See the Examples page for several examples of configuring teams, vlans and both.

In order to function properly, the adapters configured for VLAN must be connected to a "tagged" port (called a trunk port by Cisco) on an 802.1q capable switch.

## Usage Notes

1. Gigabit adapters - The PRO/1000 adapter may be teamed with certain Intel(R) Fast Ethernet adapters for AFT mode only. The PRO/1000 should be designated the preferred primary adapter for best performance. Drivers may be found on the Intel web site.

2. Required Settings in STARTUP.NCF

SET MINIMUM PACKET RECEIVE BUFFERS = 200 (or larger)
SET MAXIMUM PACKET RECEIVE BUFFERS = 500 (or larger)

See the section "Allocating Memory for VLANS" in Section V. VLAN COMMAND LINES earlier in this document for more information.

3. If you receive the ERROR MESSAGE (at commit): "Failed to create new team," check the following:

a. All adapters have loaded drivers and have the same frame types loaded.

b. iANS is loaded once for each vlan and frame type.

c. All adapters are connected to the same network segment.

d. An attempt to bind iANS to an unsupported adapter is not being made.

e. An "iANS BIND" command has been issued for each adapter and frame type in the team.

Note: Novell's CONFIG command will not reflect the iANS BIND statements until after the COMMIT has been successfully issued. After the COMMIT, iANS will appear in CONFIG as a protocol bound to the base driver.

4. To display the current status for all adapters in a iANS team:

LOAD iANS STATUS team=n (# of the team)

5. To create the primary adapter:

BIND iANS <adapter> PRIMARY

Example: BIND iANS CE100B_1_E82 PRIMARY

This can only be done during in the BIND command.

6. The default amount of memory used by each NetWare receive buffer is approximately 4K (varies slightly with different versions). If the extra memory requirement for VLAN is a problem, there are several things that can be done to reduce the impact:

a. If Ethernet is the only network topology that the server uses, and you are not using "jumbo frames", add (to STARTUP.ncf):

SET MAXIMUM PHYSICAL RECEIVE PACKET SIZE = 2048

b. Use the iANS keyword TX_ECBS_TO_USE to reduce the buffers per VLAN from 64 down to as low as 32. This WILL negatively impact performance (while reducing system resources required).

Example: LOAD IANS TX_ECBS_TO_USE = n (# buffers for each VLAN)

See the section "Allocating Memory for VLANS" in Section V. VLAN COMMAND LINES earlier in this document for more information.

7. When using iANS with SMP mode enabled, do not use the Stop Processors command after loading iANS.LAN. The Novell OS does not currently have a mechanism for informing the intermediate driver of a change in the number of active processors, and this could lead to a server abend if iANS attempts to use a deactivated processor.

8. Due to some potentially severe problems in registering our driver with NEB.NLM, a hot plug capable server must have iANS.LAN loaded before the hot plug modules get loaded. To do this, make sure the iANS.LAN load lines precede the NCMCON.NLM load lines in the AUTOEXEC.NCF.

9. When a multiple adapter team is created, it uses all the functionality of the least capable adapter in the team. In order to add an adapter to an existing team it must have the same base functionality enabled as that of the least functional adapter in the team. For example, the PRO/100 adapter has TCP checksum offloading enabled by default. The PRO/1000 adapter does not. To add a PRO/1000 to an existing PRO/100 adapter team (assuming the PRO/100 adapters are using their default settings) it is necessary for TCP checksum offloading to be enabled on the PRO/1000. This is done while loading the PRO/1000 adapter's base driver.

10. ANS probes are turned off by default when only Ethernet_802.2 and/or Ethernet_802.3 frame types are in use. This is due to the fact that Ethernet_802.2 and Ethernet_802.3 frame types may not allow ANS probes to be sent correctly and may cause one or more adapters to failover unnecessarily. If you intend to use the probe feature of ANS, Intel recommends that you load the frame types Ethernet_II or Ethernet_Snap. (You do not need to bind any protocols to these frame types.)

11. In NetWare, the following message may be displayed: "IPX received an incomplete packet from network 00001011:00A0C9AADFC3. 85 incomplete packets received." This message occurs when NetWare's internal routing mechanism mistakes the Intel iANS driver's probe packets for NetWare RIP packets. To prevent this message from displaying, make sure that IPXRTR.NLM is loaded on every NetWare server on the LAN. For more information on IPXRTR, see Novell's documentation or their Support web site (support.novell.com).