Document last changed by: Jim Sibley on 7/23/2001
I've attempted to pull together the practical
knowledge I've acquired in implementing these
cards based on information
from Boeblingen, Germany (linux390@de.ibm.com)
and trial and error. This
is by no means complete, but at least I have
gotten these parameters to
work on S/390 Linux.
The are two sets of Linux IBM OCO drivers
to support OSA cards
- LAN Channel Station (LCS) Ethernet and Token
Ring (lcs.o). Normal ESCON protocols.
- Gigabit Ethernet (GBE). QDIO is a new S/390
architecture that allows an OSA-Express
Gigabit
Ethernet feature to communicate directly
with system memory through the use of queues
and a never-ending channel program.
- Queued Direct I/O (QDIO) protocol driver
(qdio.o)
- Gigabit Ethernet driver (qeth.o)
The latest versions of the OCO LCS, QDIO
and QETH modules are found at the IBM developers
works download site (OCO
Download ). The modules must untarred, then be copied
and renamed to /lib/modules/$RELNO/net
as "qdio.o" and "qeth.o".
If you do this before a kernel build, the
"depmod -a -v" should build the
correct "/lib/modules/$RELNO/modules.dep"
file. ($RELNO is the kernel release, such
as 2.2.16).
back to top
Currently, there are two flavors of OSA cards:
OSA2 and OSA-Express. The OSA-Express cards
can only be installed in S/390 G5 or above.
The zSeries OSA-Express (Hydra 1.5 cards)
are functionally the same as two (2) G5+
cards (Hydra 1).
- OSA2-ENTR - OSA2 Ethernet/Token Ring
- OSA2-FE - OSA2 Fast Ethernet
- OSA-FEX - OSA Fast Ethernet Express
- OSA-GBX - OSA Gigabit Express
S/390 Open Systems Adapters (OSA) cards
for Linux
Card
|
Mbps
|
Connection
|
CPU |
Cable
|
HCD
CHP
|
#
CHP |
Ports
/CHP
|
Multiple
IP
|
MAX
Addresses
Entries
/Card
|
Access IP's
other LPARS
|
Linux
Module
|
OSA2-ENTR
|
10
|
T/R or
ETH
|
|
copper
|
OSA
|
1 |
2
|
OSA/SF
|
32
|
yes
|
LCS
|
OSA2-FE
|
100
|
ETH
|
|
copper
|
OSA
|
1 |
1
|
OSA/SF
|
16
|
no
|
LCS
|
OSA-FEX
|
100
|
ETH
|
G5+ |
copper
|
OSE
|
1 |
1
|
auto
|
240
|
yes
|
LCS
|
"
|
100
|
ETH
|
G5+ |
copper
|
OSD
|
1 |
1
|
auto
|
240
|
yes
|
QETH
|
OSA-GBX
|
1000
|
ETH
|
G5+ |
fiber
|
OSD
|
1 |
1
|
auto
|
240
|
yes
|
QETH
|
OSA-FEX
|
100
|
ETH
|
Z |
copper
|
OSE
|
2 |
1
|
auto
|
480
|
yes
|
LCS
|
"
|
100
|
ETH
|
Z |
copper
|
OSD
|
2 |
1
|
auto
|
480
|
yes
|
QETH
|
OSA-GBX
|
1000
|
ETH
|
Z |
fiber
|
OSD
|
2 |
1
|
auto
|
480
|
yes
|
QETH
|
- Mbps - Megabits per second (approximate).
- Connection: The card may support other protocols,
but Linux only supports Token Ring or Ethernet.
- CPU: Minimum CPU required
- HCD CHP: HCD/IOCDS CHP type
- # CHP: Number of CHP addresses per card
- Port/CHP: number of ports per CHP
- Multiple IP: You must use OSA/SF to allow
multiple IP on an OSA2 card. The OSAX card
automatically assigns the correct value
to
the correct LPAR.
- MAX Addresses/port: OSA2 cards allow 16 IP
addresses per port. For OSAX, there are
240
addresses per port. Since an LCS card takes
2 S/390 addresses per IP addresses, the
limit
would be 120 IP addresses per chpid for
LCS
and since a QDIO card takes 4 addresses
(LPAR)
mode, the limit would be 60 IP addresses.
- Access IP's other LPARS: IP addresses of
other LPARS are directly addressable within
the card
- Linux Module: the modules are in "/lib/modules/$RELNO/net".
The release number may change
- Sample /etc/modules.conf
- LCS cards
- The addresses are in even-odd pairs. You
only
code the even addresses.
- Only one LCS options card should be present.
Multiple cards would be coded on the same
options card. Linux detects whether
the card is T/R or Ethernet and it assigns
the interface name sequentially,
as appropriate to the card (tr0,tr1,eth0,eth1,
etc.)
- It assigns the interface name sequentially,
as appropriate to the card (tr0,tr1,eth0,eth1,
etc.).
1 chp, 1 port/chp |
options LCS noauto=1 devno_portno_pairs=0x2000,0 |
1 chp, 2 ports/chp |
options LCS noauto=1 devno_portno_pairs=0x2000,0,0x2000,1 |
2 chp, 1 ports/chp |
options LCS noauto=1 devno_portno_pairs=0x2000,0,0x3000,0 |
- QETH cards
- The addresses are in triplets. However, under
native, the developers currently recommend
quadruplets on EVEN address boundaries;
this
may be relaxed in the future. VM without
autosense may be able to use the triplets.
- Only one QETH options line should be present.
Multiple cards would be coded on the same
options line.
- If no interface name is present, it assumes
eth0. It is probably safer to code
the explicit
interface name
- The QETH modules has a dependency on the
QDIO module in /lib/modules/$RELNO/modules.dep,
so both modlues are loaded when QETH
is loaded
1 chp |
options qeth qeth_options=noauto,0x4000,0x4001,0x4002,eth0 |
2 chp |
options qeth qeth_options=noauto,0x4000,0x4001,0x4002,eth0,0x5000,0x5001,0x5002,eth1 |
back to top
SuSE initializes the network with the "/sbin/init.d/network"
script.
It needs to tie together four things:
1) The module name
2) The module parameters
3) The device addresses
4) The TCP/IP parameters
The interface name from the IFCONFIG command
("eth0", "tr0", etc) is used as the common
thread (as in "train of thought").
- /etc/modules.conf
- "alias eth0 lcs" ties the interface name
with
the module name
- "options lcs" ties the device address and
card options with the module name
- /etc/rc.config
- NETCONFIG="_0 _1" defines the subscripts
for
TCP/IP devices (in this case 2 "_0" and "_1")
- IPADDR specifies the IP address to use for
each interface
- IPADDR_0="9.112.50.115"
- IPADDR_1="10.32.56.119"
- NETDEV assigns the interface (or net device)
names
- NETDEV_0="tr0"
- NETDEV_1="eth0"
- IFCONFIG then specifies the parameters for
the "ifconfig" command
- IFCONFIG_0="9.112.50.115 broadcast 9.112.50.255
netmask 255.255.255.0 up"
- IFCONFIG_1="10.32.56.119 broadcast 10.32.56.255
netmask 255.255.255.0 up"
back to top
Generally, I've coded 16 addresses for each
OSA and the last entry at xFE must be genned
for OSA/SF as "OSAD" to read any
of the cards or update the OSA2 cards. This
number of addresses may vary according to
the number of IP addresses you need for an
LPAR - 2 addresses/IP if LCS, 4 addresses/IP
if QDIO (Linux developers recommendation
- this may be relaxed to 3 addresses in the
future). VM may allow 3 addresses/IP if you
do not use autosense.
CARD |
IOCDS |
OSA2-ENTR
OSA2-FE |
CHPID PATH=(00),SHARED,
*
PARTITION=((LPAR1,LPAR2)),TYPE=OSA
CNTLUNIT CUNUMBR=2000,PATH=00),UNIT=OSA
IODEVICE ADDRESS=(2000,016),CUNUMBR=(2000),UNIT=OSA
IODEVICE ADDRESS=(20FE,001),CUNUMBR=(2000),UNIT=OSAD |
OSA-FEX
LCS |
CHPID PATH=(FD),SHARED,
*
PARTITION=((LPAR1,LPAR2)),TYPE=OSE
CNTLUNIT CUNUMBR=4000,PATH=(FD),UNIT=OSA
IODEVICE ADDRESS=(4000,016),CUNUMBR=(4000),UNIT=OSA
IODEVICE ADDRESS=(40FE,001),CUNUMBR=(4000),UNIT=OSAD |
OSA-FEX
QDIO |
CHPID PATH=(FE),SHARED,
*
PARTITION=((LPAR1,LPAR2)),TYPE=OSD
CNTLUNIT CUNUMBR=5000,PATH=(FE),UNIT=OSA
IODEVICE ADDRESS=(5000,016),CUNUMBR=(5000),UNIT=OSA
IODEVICE ADDRESS=(50FE,001),CUNUMBR=(5000),UNIT=OSAD |
OSA-GBX
QDIO |
CHPID PATH=(FF),SHARED,
*
PARTITION=((LPAR1,LPAR2)),TYPE=OSD
CNTLUNIT CUNUMBR=6000,PATH=(FF),UNIT=OSA
IODEVICE ADDRESS=(6000,016),CUNUMBR=(6000),UNIT=OSA
IODEVICE ADDRESS=(60FE,001),CUNUMBR=(6000),UNIT=OSAD |
back to top
If you IPL native, you need your parmline
to have an ipldelay of at least 3 minutes
for the OSA cards to stabilize.
If you IPL as an LPAR, this is not necessary.
Remember to do a "silo" if
you change any of these parameters.
"/boot/parmline" or "/boot/parmfile"
root=/dev/dasda1 noinitrd dasd=2000-201f
ctc=0,0xfa48,0xfa49,escon0 mem=2048M ipldelay=3m |
Note: "ctc", "mem" ,
and "ipldelay" are optional and
I've coded them for reference.
back to top
- Once an OSA has hung in an LPAR, you have
to re-ipl that LPAR.
- Sometimes, the IPL will NOT fix the hang,
so you must reset the OSA card CHP.
- Caution: Resetting an OSA card or configuring
the CHPID off/on may cause the Linux LPARS
with OSA card genned to hang,
whether you have specified it in /etc/modules.conf
or not!
- In most shops, this is handled by Field Support
for the Processor. Some shops allow the systems
programmer to do this procedure.
- First, you must know the CHPID for the OSA
card you want to reset
- OSA cards can be displayed and reset from
the HCD
- Click on "Defined CPC's"
- Highlight the CPC you want
- Click on "Single Object Operations"
- Click on "CPC"
- right click on CPC to get CHPID's
- Use "Advanced Facilities" to view, set the
parameter on OSA2 cards and reset the cards
- All cards should be full duplex
- OSA2-ENTR - 10 Mbs
- All others 100 Mbs
- After you have finished in "Advanced Facilities",
toggle the CHPID's off, then on under "Configure
ON/OFF"
- log out of "Single Object Operations"
when you're done.
- Certain levels of the OCO drivers (2.2.16,
May 2001), may not reboot correctly (shutdown
-r). The system will shutdown, almost complete
the IPL process, then hand in a multi-cp
loop while trying to process the OSA card.
A complete shutdown (shutdown -h now) and
an ipl (ipl clear) will avoid this problem.
- Older levels of the HCD do not have the option
to reset the OSA2 card to its default
settings.
We would recommend that, if you can,
reset
the card to default settings before
you remove
the card from the processor. This will
simplify
later installs for that card, especially
if you do not have OSA/SF available..
back to top
Contributers
- Jim Sibley (jlsibley@us.ibm.com)
- Daiki Shimizu (Daili Shimizu/Japan/IBM@IBMJP
- Cristoph Arenz (arenz@de.ibm.com)
back to top