Installing and Configuring Intel(r) Advanced Networking Services (ANS) on
the Linux 2.2.x and 2.4.0 Kernels

Contents:

Overview & Environment
Installation of Advanced Networking Services 
Configuring the Adapters and Teams
Saving the Configuration across Reboots  ) 
Removing Advanced Networking Services
Modifying Advanced Networking Services Configurations
Available Commands 
Example of Setting Up an AFT team
Troubleshooting (also includes file distribution and conf.mod example)
License

Overview and Environment
=======================
This package was tested on Red Hat* Linux*  (Linux kernel 2.2.16, 2.2.18
and 2.4.0) on both UP and SMP Intel processor based PCs. 
Installation assumes you have all necessary kernels and build tools loaded.

The package supports the following teaming modes: 
* AFT ( 10/100 and 1000) 		 Adapter Fault Tolerance
* ALB ( 10/100 or 1000) 		 Adaptive Load Balancing (includes AFT)
* FEC (10/100) or GEC (1000) 	Link Aggregation/FEC/GEC  

Descriptions of the teaming modes and their specific requirements are:

- AFT (Adapter Fault Tolerance). Default mode. A secondary adapter in the
team will automatically take over on any failure of the primary connection
(cable, adapter or port). Only one card is active at a time. After a
failover, if the connection to the primary adapter is restored, control
passes automatically back to that primary adapter. May mix speed and duplex
capabilities and settings. May use a hub or any switch.

- ALB (Adaptive Load Balancing or asymmetrical link aggregation). A team
of 2-8 adapters share the transmission load while the primary receives and
transmits. Includes fault tolerance. Must match speed/duplex settings on all
team members. May use any switch.

- Link Aggregation (Cisco*'s Fast EtherChannel* Technology (FEC) or Intel
Link Aggregation).  A team of 2-8 10/100 adapters which simultaneously
receive and transmit. Includes fault tolerance and load balancing. Must match
speed/duplex settings on all team members Requires a switch that supports
Intel Link Aggregation or Cisco's FEC. Spanning Tree Protocol must be turned
off. Must match switch aggregation requirements.

- GEC (Gigabit equivalent of FEC). Requires GEC compatible switch and
PRO/1000 Adapters.

Roles - For AFT and ALB, you may specify which adapters in a team you want
to use as the Primary and Secondary adapters. The Primary will be the main
adapter to be used in a team. For fault tolerance, the Primary is the only
adapter used until a failure. For ALB, the Primary is the adapter that both
transmits and receives. The Secondary is the preferred adapter (in a team
consisting of more than 2 adapters) to fill the role of Primary adapter if
it fails. If Priority is not specified, the adapter with the highest
supported speed will be chosen. If all adapters support the same speed,
the last one added will be the primary.

These Advanced Networking Services only work with Intel adapters and each
team requires at least one Server adapter.
Linux is not supported on the Intel(R) PRO/100 Intelligent Server Adapter.
                     

IEEE 802 Virtual LANs (VLANs)  are based on the addition of a tag to the packet header. 
All equipment on the network that will pass this traffic must be capable of handling
the extra length.  Settings on the adapter must match the VLAN settings on the switch.
VLANs also add to load time and require additional memory resources. There is a limit
of 64 VLANs per server. However VLANs may be numbered from 1-4096. 

 
Installation of the ADVANCED NETWORKING SERVICES Package
========================================================
Please install and compile the base driver package(s) per the readme included
with those drivers. Verify that all interfaces are working and that you 
have the basedriver file(s) (e100.o, e1000.o) in lib/modules/<kernelversion>/net/.

1. Login as root.
2. Copy the iANS-x.x.x.tar.gz package file to a directory of your choice. 
3. Open the package file by running 'tar xzf iANS-x.x.x.tar.gz'
   This will create iANS-x.x.x as a subdirectory under the current directory.
4. Enter the iANS-x.x.x directory.

At this point, choose between the 2 separate methods for installation:
1. Manual  (suggested for experienced users)
2. Scripted (suggested ONLY for Red Hat 6.x, SuSE* 6.4, Caldera* 2.2, and
TurboLinux* Server 6.0 users) 

Manual Installation and Configuration  
-------------------------------------

1. From the directory iANS-x.x.x run 'make' to compile the module.
   NOTE: SMP/UP and MODVERSIONS are determined by 'make'. If there is a
   mismatch between the configuration for the kernel source tree and
   the kernel itself, run 'make options" to list options available for      
   overriding the source tree settings. These include SMP, MODVERSIONS, 
   and NODEBUG (not listed). NODEBUG values are 0 (debugging is on), 1 
   (debugging is off). Default is 0.

2. Run 'make install'. (Default file locations may be changed by editing
the paths at the top of Makefile - see Troubleshooting section.) There
will not be any messages after make install.
 
See the Troubleshooting section for a list of files and their default
locations.
  
3. You must use Base Driver version 1.4.18 (or higher) for PRO/100
and/or Base Driver version 2.6.x (or higher) for PRO/1000 for ANS support.

NOTE: Make sure no other base drivers are loaded (nor compiled into the
kernel, for any of the Intel adapters. (e.g. eepro, eepro100, eexpress,
e100,e1000). Be sure the configuration files have been edited to alias 
the new driver(s) and not any previous drivers. Please see the trouble-
shooting section for an example file.

4. After the required base drivers are loaded, down any Intel adapters 
that are to be added to a team. (If they already have an IP address, run
ifconfig ethx 0.0.0.0 to remove the IP number.)
	ifconfig - a        - will list all current network devices
	ifconfig ethx down  - will down adapters so ANS may be loaded

	Note: You should leave other vendor's Ethernet adapters
        configured as well as any Intel adapters that you will not
        adding to teams. This will ensure that network device name
        assignment remains consistent after reboot.
 	   
5. Load the Advanced Networking Services module by running 'insmod ians'.
In the following command lines, optional items are placed in [ ]. All VLAN
settings are shown as optional.

6. Create team(s); 
	ianscfg -a -t<team_name> [-M<mode >] [-V]

    Use the -M flag to specify the teaming mode (default AFT, others NONE | ALB | FEC | GEC). 
    Use the -V flag to specify a VLAN  .  
     Use -M NONE if you wish VLANs on single adapters without link
     aggregation. Any other mode may be specified to use VLANs and link
     aggregation simultaneously.

7. Determine the teaming mode (AFT, ALB, FEC, GEC) for each team.
	ianscfg -C -t<team_name> -M<mode>

NOTE: team mode can be specified in either steps 6 or 7

8. Add Intel adapters to the team and specify role (priority)
	ianscfg -a -t<team_name> -m<ethNN> [-p <priority role>]
   Priority may be None,  Primary, or Secondary

9. Create virtual adapters.
	ianscfg -a -t<team_name> -v<vadapter_name> [-i<vlan_id>] 

	vadapter_name may be anything except ethx as long as it ends with an instance suffix
	specify the -i options (vlan_id) only on VLAN based teams or adapters.
	
10. Activate the team 
	ianscfg -c<team_name>

11. Activate the virtual adapter  	
        Note: This binds the protocol to the VIRTUAL adapter.
        ifconfig <vad_name> <ip#> netmask <mask#> [broadcast <broadcast>]

12. Check team status
	ianscfg -s

Congratulations, the team is now active!! You should be able to send
and receive packets via the virtual adapter.

----------------------------------------
Scripted Installation and Configuration
========================================
For ease of configuration, Intel is supplying interactive scripts - which
are only supported on:
 		Red Hat*	(6.1, 6.2)    
 		SuSE*		(6.4)		  
		Caldera*	(2.2)			
		TurboLinux*  (6.0 server)	

ANS will set up 4 directories named Red Hat, SuSE, Caldera, and Turbo to
match each of these distributions under iANS-x.x.x/install/

To use the install script, enter iANS-x.x.x/install/[dist_name] and type
./INSTALL 

   NOTE: The module is created with support for the SMP/UP and MODVERSIONS
   features as defined by the configuration of the current kernel source
   tree (/usr/src/linux/). On some systems, there might be a mismatch
   between this configuration and the kernel image itself. If there is
   a mismatch, it will not be apparent until further in the process.
   Run 'INSTALL options' to see extra compilation switches that allow
   overriding the kernel source tree settings.

Then follow the instructions displayed on the console. You will be asked
some questions during the Process:

1. "Would you like the configuration utility to remove all network
interfaces automatically?" The installation process must bring down the
network interfaces related to Intel adapters, in order to unload the old
Intel drivers and replace them with the new ones.
Select Yes (default) removes all network interfaces (including other brands). 
Select No to choose for each interface.

2. "Enter team name,(Enter) to end." You must enter either a team name
(often a number) or press Enter (CR) to quit adding teams (skips to Step 6).
"Would you like to VLANs in team (name)" - Yes will create a VLAN-based team.
"Select members for team (name)" will list all applicable interfaces and you
need to type in the number of the adapter to add to the team.
Select the role (none, primary or secondary). 
Repeat for each adapter (enter 0 when done.) 

3. "Enter virtual adapter name for team (name)"   
can be anything but ethx and must end in an instance suffix. 
"Enter VLAN id for virtual adapter (name)" (for VLAN-based teams only)
    enter a number 1-4095 to specify the Virtual LAN.
"Enter boot protocol (none, bootp, dhcp, or other)", 
	If choose none will need to manually enter IP address and netmask. 
	If choose bootp or DHCP must be able to reach another machine
           assigning the IP address and netmask.
	If choose Other must enter protocol name (such as IP)
NOTE: This information will NOT be saved until you confirm your selection
after configuring the team.

4. "Select Teaming Mode for Team (name) - None\AFT\ALB\FEC\GEC" - for
descriptions see the first section of the readme. For Intel Link
Aggregation use FEC or GEC. WARNING: Choosing None will still create a team!

5. "Are you sure?."  Yes commits the team. 
Choosing No will make you start again at step3.

After commit, you may make another team, or press Enter to proceed.

6. "Would you like to save this configuration and load it on boot? "
Yes (default)  - modifies init file. Choose which run-levels will have the
ANS module automatically loaded. (Recommendation: select same level as the
network boot - commonly 2,3,4, or 5)
No - means the configuration will not be saved for boot.

7. "Would you like to start the network now?"
Yes - activates the team 
No - should still show installation and configuration completed successfully.
ANS will start at reboot if configuration was saved in step 4.

----------------------------------------------------------------


Saving ADVANCED NETWORKING SERVICES configuration across reboots 
 ===============================================================
NOTES:  Do not use if setting up via Scripted configuration method.
Sample lines are Red Hat 6.x specific - please modify for your distribution.

Now that the Ethernet and ANS drivers are configured, follow the
instructions below to configure the system to activate ANS the same
way every time it boots.

1. Save the ANS configuration to a file by running:
     ianscfg -w [-f<file_name>]
     (default is /etc/ians/ians.conf)

2. Make sure that the adapters that were selected as members of
the team(s) are NOT configured on boot. To do so, change "ONBOOT=yes"
to "ONBOOT=no" in every matching /etc/sysconfig/network-scripts/ifcfg-ethX
file.

3. Set each virtual adapter to configure on boot by creating a config file
   /etc/sysconfig/network-scripts/ifcfg-<vadapter_name>,
   which should look something like this:
	DEVICE=<vadapter_name>
	BOOTPROTO="none"
	ONBOOT="yes"
	IPADDR="192.168.1.1"   (sample)
	NETMASK="255.255.255.0"  (sample)

4. Create an init script that will load the e100.o, e1000.o (if needed)
and ians.o modules, and configure the ANS topology. You can use the sample
script file by copying iANS-x.x.x/install/Red Hat/ans
  to /etc/rc.d/init.d/ans
Configuration will be read from /etc/ians/ians.conf by default.
Modify the script if you wish to use any other file.

5. Add the ANS script to the init process of the appropriate run levels.
     Run 'chkconfig --level <run levels> ans on' 
where <run levels> is the run levels you would like ANS to start in.

   NOTE: This script must be invoked BEFORE the system starts the network
   services. For example, if your network init script for run level 3 is    
   /etc/rc.d/rc3.d/S10network, you should create a symbolic link 
   /etc/rc.d/rc3.d/S09ans that points to /etc/rc.d/init.d/ans.
   ANS service is automatically stopped when the network service is stopped.

 
----------------------------------------------------------------------------
Removing ADVANCED NETWORKING SERVICES
=====================================
There are several stages to removing Advanced Networking Services.
* Bringing down all vadapter interfaces.
* Removing the ANS configuration from future reboots.
* Removing the ANS module
* Removing the files from the system

There are two methods for removing ANS: manual and scripted.


Manual method of Removing ANS
-----------------------------
1. Login as root.

2. Run 'ifconfig -a' to see every virtual adapter configured in the system.

3. For each virtual adapter, run 'ifconfig <vadapter-name> down' 

4. Remove the ians module by running 'rmmod ians'

5. In each virtual adapter file, either modify the file to contain the line 
ONBOOT="no" instead of ONBOOT="yes"
   or remove the file completely.

6. For each file for stand alone adapters, modify the file to contain the
line ONBOOT="yes" instead of ONBOOT="no".

7. You may leave the Intel base driver or you may edit the modules config
file (usually /etc/conf.modules or /etc/modules.conf) and restore its
original form by removing the lines containing e100 or e1000 as the drivers
that were added and restoring every line that was commented out. 

8. Remove the ANS initialization script and all links to it.
   (e.g. /etc/rc.d/init.d/ans and /etc/rc.d/rc.3/S09ans).

9. From iANS-x.x.x directory, run 'make uninstall' to remove all the files


Scripted Method (only those distributions covered by scripted install)
----------------------------------------------------------
1. Login as root.
2. Enter the iANS-x.x.x/install/[distribution_name] directory. 
3. Run 'UNINSTALL'.
4. The rest of the process is interactive. Follow the instructions displayed
on the console.

Prompts:
Restore member adapters to original role (stand alone).
Answer Yes for each adapter you wish to continue using standalone. This
configuration will be made using Intel's e100(0).o driver although you
may restore other drivers.

Removal of the ANS module from memory will not down the adapters running
in stand alone mode. The script will remove all files, scripts, and links
created except the /etc/ians/ians.conf.

Restarting the Network with the Original Configuration
For the new settings to take, you will have to restart the network
service either by rebooting the system or by running the network init script
with the restart parameter.    /etc/rc.d/init.d/network restart



Modifying ADVANCED NETWORKING SERVICES Configuration
====================================================
ANS configuration and topology may be changed, with a few limitations.

After commit, you may change priority and add or remove a team member
 (but not the last one). You may not:
	Change team mode.
	Remove a team or a virtual adapter.
	Remove the last member of an active team.

Teams, members adapters and virtual adapters may be added on the fly while
the service is running via the hot add and hot remove features (using the
same set of commands). (Not to be confused with Hot PCI Add/Remove.)


There are 2 methods of reconfiguring
 - Semi-automatic - good for changing mode and/or moving members between
 teams (can't create new teams plus all adapters must remain in teams).
 - Manual reconfiguration (see below)

Semi-automatic Reconfiguration
------------------------------
1. Manually modify the ANS topology description file (/etc/ians/ians.conf).
2. Call the ANS initialization script with the restart parameter 
	 /etc/rc.d/init.d/ans restart.
3. To reactivate, run the network service initialization script with the
restart parameter.
	/etc/rc.d/init.d/network restart 
 
Manual Reconfiguration
-----------------------

1. Take down the virtual adapter
ifconfig <vadapter_name> down

2. Remove the ians module
	rmmod ians

3. Create or modify the ians topology file to match the new desired topology
(default location /etc/ians/ians.conf), or create  script containing a set
of calls to ianscfg that will generate the desired topology.

4. Load the ians module
	insmod ians

5. Run the script created in step 3, or run the command:
    ianscfg -r [-f<filename>]

6. Reset each created virtual adapter
    ifconfig <vadapter-name> <IPaddr> netmask <NETMASK> broadcast <BROADCAST>

7. Save the new configuration:
	ianscfg -w [-f<filename>]

    Create an interface configuration file for each new virtual adapter.
      * The files would normally reside in /etc/sysconfig/network-scripts/
      * The name format should be ifcfg-<vadapter-name>
      * Sample file content:
	DEVICE=<vadapter_name>
	BOOTPROTO="none"
	ONBOOT="yes"
	IPADDR="192.168.1.1"
	NETMASK="255.255.255.0"

8.  If any unnecessary virtual adapter configuration file is left, remove
it or modify it.

9.  For any adapter that is desired as stand alone connection, you may
continue to use the Intel base driver.
    

List of Available Commands:
=============================
 
Create a Team:
	Create team(s); 
	ianscfg -a -t<team_name> [-M<mode>] [-V]

    Use the -M flag to specify the teaming mode 
         (default AFT, others NONE | ALB | FEC | GEC). 
    Use the -V flag to specify a VLAN  .  
     
Set/Change team mode: (only before commit)
	ianscfg -C -t<team_name> -M <mode>
   	 

Add team members:
	ianscfg -a -t<team_name> -m<ethNN> [-p Y]
                where Y can be none, primary or secondary, optional 
	(Only existing Intel Ethernet devices may be added to a team)

Add virtual adapter 
	ianscfg -a -t<team_name> -v<vadapter_name> [-i<vlan_id>]
	NOTE: On non-VLAN teams, define ONLY one virtual adapter per team.
	On VLAN teams, the -i options (vlan_id) must be specified. 
	The name of the virtual adapter can be any except ethx  but must 
           include instance number suffix.

Activate (commit) the team:
	ianscfg -c<team_name>
        After a team has been activated you can still add/remove members.

NOTE: At this point the virtual adapter appears (ifconfig -a) in the
adapters list but it is in the down state.

Activate the virtual adapter:
        ifconfig <vadap_name> <ip#> netmask <mask#> broadcast <broadcast>

Get Status:
	ianscfg -s

Get help on the ianscfg utility - Type 'ianscfg'

To remove a member:
  ianscfg -d -m<ethX>
  Note: You cannot remove the last member of a team.
  Don't specify the team here. 

To Disable the Virtual Adapter:
  ifconfig <virtual_adapter_name> down

To remove a team (only before commit):
  ianscfg -d -t<team_name>

To remove a virtual adapter: (only before commit)
  ianscfg -d -v<vadater_name>

To disable a virtual adapter:
   ifconfig <vadapter name> down

To Remove the ANS Module: (down all vadapters first)
  rmmod iANS

To remove ANS files from your system: (after downing vadapters
and resetting the configuration)
   make uninstall  (from the iANS-x.x.x directory)

To Save the current configuration in a file: (only after commit)
    ianscfg -w [-f <filename>]
  (default is /etc/ians/ians.conf - editable text file)

To automatically program a topology configuration from a file:
   ianscfg -r [-f <filename>]
  (Again, the default file is /etc/ians/ians.conf)

Turn off probes
   ianscfg -P -t<team_name> -d

-------------------------------------------------------------------


Example File for Setting up AFT mode with VLANs: (2 PRO/100 adapters)
=====================================================================

  insmod e100
  insmod ians

  ianscfg -a -t team1 -M AFT -V

  ianscfg -at team1 -m eth0 -p primary
  ianscfg -at team1 -m eth1 -p secondary

  ianscfg -at team1 -v vadapt1 -i 10 
  ianscfg -at team1 -v vadapt2 -i 15
 
  ianscfg -c team1

  ianscfg -s

  ifconfig vadapt1 143.185.254.10 netmask 255.255.255.0
  ifconfig vadapt2 143.185.254.10 netmask 255.255.255.0

-------------------------------------------------------------------


Troubleshooting


1. If an adapter has been running standalone with an assigned IP
address you must configure the IP address to 0.0.0.0 before adding
it to a team.                     ifconfig ethx {0.0.0.0}.

2. Always bind the protocol to the virtual adapter and NOT to
the physical adapter for adapters that are in a team. (You still 
bind the protocol to the physical adapter for standalone adapters.)

3. If you do not remove references to other possible base drivers in 
the configurations, you risk multiple interfaces for adapters you
wish to use in the team with an unpredictable outcome.

4. If you receive an error message "module not loaded" during
configuration of the team, you need to start over. To do so type
  ./INSTALL veryclean all SMP=[0|1] MODVERSIONS=[0|1]
  choose 0 or 1 for SMP and MODVERSIONS to match the configuration
  of the current kernel tree.
Cause: This will occur when the target module doesn't match the
kernel version and install was done without parameters
(found using ./INSTALL options)

5. If when adding the ANS script to the init process, you receive
an error message :       'service ans does not support chkconfig'
then create the symbolic link (to the ans file) manually for each
run level desired with the command:
	ln -s /etc/rc.d/init.d/ans /etc/rc.d/rcX.d/S09ans
where X is the number of the desired run level

Structure of the ANS build tree
  * kernel dependant source files (src/*.[ch], src/lib/*.[ch])
  * kernel independent precompiled object file (bin/iANS_core.o).
  * Makefiles.
* Advanced Networking Services configuration utility (bin/ianscfg).
* installation, uninstall and configuration scripts

Distribution of files:
 By default, the components are distributed as follows:
     - /lib/modules/[KERNEL_VERSION]/net/ians.o
     - /usr/sbin/ianscfg
     - /usr/man/man1/ianscfg.1

conf.modules file example
        #alias eth0 eepro100
   	#alias eth1 eepro100
   	#alias eth2 eepro100
   	alias eth0 e100
   	alias eth1 e100
   	alias eth2 e100
   	alias parport_lowlevel parport_pc
   	alias scsi_hostadapter aic7xxx
------------------------------------------------------------------

License
-------
Copyright (c) 1999 - 2000, Intel Corporation 

All rights reserved.

Redistribution and use in source and binary forms, with or without 
modification, are permitted provided that the following conditions
are met:

 1. Redistribution's of source code must retain the above copyright
    notice, this list of conditions and the following disclaimer.

 2. Redistribution's in binary form must reproduce the above
    copyright notice, this list of conditions and the following
    disclaimer in the documentation and/or other materials provided
    with the distribution.

 3. Neither the name of Intel Corporation nor the names of its
    contributors may be used to endorse or promote products derived
    from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE. 


