Networking Stacked Knowledge

FileZilla FTP Client fails through BlueCoat Proxy

2011-12-05T14:34:00.000+08:00

Symptoms:

When using FileZilla FTP client through a bluecoat proxy, the FTP connection attempt fails with the below error message:

Status: Using proxy ftpproxy.mydomain.com

Status: Resolving address of ftpproxy.mydomain.com

Status: Connecting to 10.192.5.37:21...

Status: Connection established, waiting for welcome message...

Response: 220 Blue Coat FTP Service

Command: USER ftpuser@targetftpsite

Response: 331 Enter password.

Command: PASS ********

Response: 332 Enter proxy password.

Error: Login sequence fully executed yet not logged in. Aborting.

Error: Server might require an account. Try specifying an account using the Site Manager

Error: Critical error

Error: Could not connect to server

Versions:

FileZilla FTP Client v3.5.2
The LAN proxy server is a BlueCoat proxy server

Resolution/Workaround:

1. Go to FileZilla Client > Edit > Settings

2. In Connection > FTP > FTP Proxy

2.1. Click on Custom and do the following:

2.2. Under Custom, enter the following:

USER %u@%h

PASS %p

ACCT %a

2.3. Proxy host: <your_bluecoat_proxy_server>

3. Click OK.

Source:

http://forum.filezilla-project.org/viewtopic.php?p=19457#p19457

MPLS Label Distribution Parameters

2011-05-30T09:46:00.066+08:00

MPLS Label Distribution Parameter Options:

Label Space Options	Per-Interface	Per-Plaform
Label Distribution	Unsolicited downstream	Downstream-on-Demand
Label Allocation	Independent Control Mode	Ordered Control Mode
Label Retention	Liberal Label Retention	Conservative Label Retention

Label Space Options

Per-Platform Labels

LFIB: [Label | Action | Next Hop ]
one label assigned to a destination network and announced to all neighbors
locally unique, valid on all incoming interfaces
smaller LFIB, FIB; faster label exchange
less secure than per-interface labels; label-spoofing
default in Frame-Mode MPLS

Per-Interface Labels

LFIB:[IN intf | IN Label | NH intf | NH Label]
one label for each destination, each device, each interface
secure; prevents label-spoofing; labeled packets/ATM cells only accepted from interface where label is assigned
default in Cell-Mode MPLS

Label Distribution

Unsolicited Downstream

label advertised to all neighbor LSRs, regardless of whether upstream or downstream
Frame-Mode

Downstream-On-Demand

label advertised only upon request of upstream LSR
cell-mode

Label Allocation

Independent Control

LSR can assign a label for a prefix w/o outgoing/downstream label
for edge LSRs only (requires L3 capabilities)
faster label propagation
for unsolicited downstream / frame-mode

Ordered Control

local label allocated and propagated only
if exist(nexthop label) propagae/allocate label, else request label from nexthop
for downstream-on-demand/cell-mode

Label Retention

Liberal Label Retention

all received label stored in LIB, even if not from next-hop LSR
improves convergence speed; allows easy failover for link failures
frame-mode

Conservative Label Retention

only received labels from nexthop LSRs are stored; others are ignored
downstream-on-demand/cell-mode

Standard Parameter Sets for Cisco Platforms

Parameter	Routers, frame interfaces	Routers, ATM interfaces	ATM switches
Label Space	per-platform	per-interface	per-interface
Label Distribution	unsolicited downstream	downstream-on-demand	downstream-on-demand
Label Allocation	independent control	independent control	ordered control
Label Retention	liberal label retention	conservative OR liberal	conservative

BGP Route Dampening

2011-05-25T01:02:00.000+08:00

At a Glance:

designed to reduce router processing load caused by unstable routes;
prevents sustained routing oscillations without affecting well-behaved routes;
RFC 2439: BGP Route Flap Dampening;
minimizes BGP updates by suppressing unstable routes.

Route-Dampening Operation:

EBGP route flaps = 1000 penalty points

IBGP route flaps not dampened;
penalty not user-configurable;

penalty decays via exponential decay algorithm

if penalty > suppress limit, route is dampened

if penalty < reuse limit, dampened route is propagated

if penalty < 1/2 reuse limit, flap history forgotten

route is never dampened more than the maximum suppress time limit
unreachable route with flap history is in history state (still in BGP table to retain flap history)
penalty is applied to an individual path, not the prefix

Configuring Route Dampening:

router(config-router)# bgp dampening [half-life reuse suppress max-suppress-time] [route-map route-map-name]
half-life	time for penalty to halve (default 15 minutes)
reuse	penalty points when dampened route is reused (default 750)
suppress	penalty points when route is suppressed (default 2000)
max-suppress-time	maximum time a route is suppressed (default 1hour; maximum 255 minutes)

router(config-route-map)#
set dampening half-life reuse suppress max-suppress-time

- used for less aggressive dampening of routes towards root DNS servers
- dampening of smaller prefixes more aggressively
- selective dampening based on BGP neighbors and route-map match criteria

Other Commands:

clear ip bgp ip_addr flap-statistics [{regexp regexp} | {filter-list listname} | {ipaddr mask} ]

clear ip bgp dampening [ipaddr mask]

show ip bgp dampened-paths

show ip bgp flap-statistics [{regexp regexp}|{filter-list listname}|{ipaddr mask [longer-prefix]}]

debug ip bgp dampening

BGP: Regular Expressions for AS-PATH Filtering

2011-05-23T18:13:00.001+08:00

Symbol	Description
\|	logical OR
.	match any
[x..y]	match one in range
^	match beginning of string
$	match end of string
_	match any delimiter/white space
( )	group as a single atom
*	match 0 or more instances of previous atom
?	match 0 or 1 instance of previous atom
+	match 1 or more instances of previous atom
\	escape character; if followed by a number n, points to the nth atom

Example 1: Advertise routes with empty AS-PATH (internal routes)

router bgp 123
neighbor 5.6.7.8 remote-as 387
neighbor 5.6.7.8 filter-list 1 out
!
ip as-path access-list 1 permit ^$
!match "blank" atom at the "start" and "end" of string
!

Example 2: Accept only default routes, preferring primary route based on AS PATH:

	AS387 (primary ISP)
	/
AS123
	\
	AS462 (backup ISP)

router bgp 123
neighbor 1.2.3.4 remote-as 462
neighbor 1.2.3.4 route-map FILTER in
neighbor 5.6.7.8 remote-as 387
neighbor 5.6.7.8 route-map FILTER in
!
route-map FILTER permit 10
! default routes from primary ISP (AS 387) accepted are preferred (larger weight)
match ip prefix-list DEFAULT_ONLY
match as-path 10
set weight 150
!
route-map FILTER permit 20
! default routes from backup ISP are accepted, with lower preference than primary ISP routes
match ip prefix-list DEFAULT_ONLY
set weight 100
!
ip as-path access-list 10 permit _387$
ip prefix-list DEFAULT_ONLY seq 10 permit 0.0.0.0/0
!

Example 3: AS PATH Filtering with AS Path Prepending:

- customer in AS123 is performing AS-PATH pre-pending

AS123
10.0.0.1	\
	\
	AS462

router bgp 387
neighbor 10.0.0.1 remote-as 213
neighbor 10.0.0.1 filter-list 10 in
!
ip as-path access-list 10 permit ^123(_123)*$
! accepts "123", "123 123", or "123 123 123"

Example 4: AS PATH Filtering with AS Path Prepending, multiple customers:

- multiple customers performing AS PATH Prepending

Customer 1
	\
Customer 2 -	AS387
	/
Customer 3

router bgp 387
neighbor 10.0.0.1 remote-as 123
neighbor 10.0.0.1 filter-list 10 in

neighbor 20.0.0.1 remote-as 456

neighbor 20.0.0.1 filter-list 10 in

neighbor 30.0.0.1 remote-as 789

neighbor 30.0.0.1 filter-list 10 in

!
ip as-path access-list 10 permit ^([0..9]+)(_\1)*$
! accepts repeating instances of "123", "456", and "789"
! does not accept strings non-repeating strings (e.g. "123 123 100")
!

Notes:
atom 1 = at least one instance of a number at the beginning of the string;
atom 2 = 0 or more instances of a whitespace and atom 1 until the end of the string.

BGP Path Attributes and Route Selection

2011-05-04T12:02:00.000+08:00

BGP Path Attributes

Mandatory Well-Known
Origin (i, e, ?)
AS-Path (sequence of AS-Numbers to access network/IP)
Next-Hop (ip address)

Discretionary Well-Known
Local Preference (for routing policy)
Atomic Aggregate (flags route if aggregated)

Optional Non-Transitive
Multi-Exit Discriminator (MED) - multiple entry pts to one AS
Originator-ID - for route reflector environment
Cluster-List - for route reflector environment

Optional Transitive
Aggregator - IP address & AS of routers that aggregated routes
Community - for route tagging

BGP Route Selection Order:

Prefer highest weight (local to router)
Prefer highest local preference (global within AS)
Prefer routes that the router originated
Prefer shorter AS paths
Prefer lowest origin code (IGP < EGP < Incomplete)
Prefer lowest MED
Prefer external (EBGP) paths over internal (IBGP)

For IBGP paths, prefer nonreflected routes (no originator-ID) over reflected routes
For reflected routes, prefer shorter cluster-list

For IBGP paths, prefer path through closest IGP neighbor
For EBGP paths, prefer oldest (most stable) path
Prefer paths from router with the lower BGP router-ID

BGP: Multihomed Customer to Single ISP in Load-Sharing Setup with Static Routes

2011-05-04T11:22:00.001+08:00

Outbound Traffic (CE to PE):
- each customer router uses closest CE as exit point;
- CE routers must be collocated to have load-sharing;

Inbound/Return Traffic (PE to CE) (pre-IOS 12.2):
- true load-sharing is impossible to achieve with multiple PEs;
- per BGP route selection, only one route will be the best route (to the customer network)
- can be optimized by dividing the customer address space

Customer Network: 11.2.3.0 /24
Customer Network "division" assigned to PE1: 11.2.3.0/25
Customer Network "division" assigned to PE2: 11.2.3.128/25

Configurations:

PE1 Config:
!
ip route 11.2.3.0 255.255.255.128 serial 0 tag 1000
ip route 11.2.3.0 255.255.255.0 serial 0 tag 1000
!
router bgp 123
redistribute static route-map INTOBGP
...
!

PE2 Config:
!
ip route 11.2.3.128 255.255.255.128 serial 0/0 tag 1000
ip route 11.2.3.0 255.255.255.128 serial 0/0 tag 1000
!
router bgp 123
redistribute static route-map INTOBGP
...
!

BGP: Multihomed Customer to Single ISP in Primary-Backup Setup with Static Routes

2011-05-04T11:00:00.003+08:00

Scenario:
Multihomed customer connected to a single service provider on multiple permanent links;
Customer network using OSPF; provider network using BGP; CE-PE via static routing;
Floating static routes configured as backup on both CE and PE;

Floating Static Routes in BGP:
Once active, the floating static routes will be permanently installed in BGP; static routes are locally sourced which is preferred;

admin distance cannot be used in route-maps; hence, use communities
tag floating static routes; tags mapped to specific communities;
use route maps to modify the weight and/or local preference;
default weight: 32768

Sample ISP Route-Community Mapping:

ISP (AS 123) Service Offerings
Advertise CU Routes	Primary?	Route Tag	Community	LocalPref
No	Yes	1000	no-export 123:31000	100
No	No	1010	no-export 123:31000	50
Yes	Yes	1001	123:31000	100
Yes	No	1011	123:31000	50

Customers Configurations:

CE1 Config (Primary link):
ip route 0.0.0.0 0.0.0.0 Serial0
!
router ospf 1
default-information originate
!

CE2 Config (Backup link):
ip route 0.0.0.0 0.0.0.0 Serial0 250
!
router ospf 1
default-information originate
!

Provider Configurations:

PE1 Config (Primary link):
ip route 11.2.3.0 255.255.255.0 serial 0 tag 1000
!
router bgp 123
redistribute static route-map INTOBGP
!

PE2 Config (Backup link):
ip route 11.2.3.0 255.255.255.0 serial 0/0 tag 1010 250
!
router bgp 123
redistribute static route-map INTOBGP
!

Common Route-Map Configurations on both PE1 and PE2:

route-map INTOBGP permit 10
match tag 1000
set community no-export 123:31000
set local-preference 100
!
route-map INTOBGP permit 20
match tag 1001
set community 123:31000
set local-preference 100
!
route-map INTOBGP permit 30
match tag 1010
set community no-export 123:31000
set local-preference 50
set weight 0
!
route-map INTOBGP permit 40
match tag 1011
set community 123:31000
set local-preference 50
set weight 0
!

IOS: EIGRP Peering Flapping, Auth Failure - %DUAL-5-NBRCHANGE: IP-EIGRP: Auth failure

2010-11-04T14:52:00.000+08:00

This is an actual case we have raised recently with Cisco as we are having unexplained EIGRP flaps between two of our devices. It has been working for more than a year -- actually, it never had any issues when this was brought online last year.

Scenario:

EIGRP peering flaps between two devices, due to authentication failure. The output of show logging is flooded with the below syslogs repeatedly:

Nov 2 01:30:43.436 GMT: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 2: Neighbor 10.10.10.2 (GigabitEthernet1/1) is down: Auth failure

Nov 2 01:30:45.040 GMT: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 2: Neighbor 10.10.10.2 (GigabitEthernet1/1) is up: new adjacency

Nov 2 01:30:47.316 GMT: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 2: Neighbor 10.10.10.2 (GigabitEthernet1/1) is down: Auth failure

Nov 2 01:30:48.820 GMT: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 2: Neighbor 10.10.10.2 (GigabitEthernet1/1) is up: new adjacency

Topology is straightforward:

Router1 Gi1/1 <-----> Gi2/2 Router2

Router1 Gi1/1 = 10.10.10.1/24
Router2 Gi2/2 = 10.10.10.2/24

MD5 Authentication is used and the same key string is configured on both devices

Router1#show key chain MYCHAIN

key-chain MYCHAIN
key 1 -- text "myCiscoChain"
accept lifetime (always valid) - (always valid) [valid now]
send lifetime (always valid) - (always valid) [valid now]
Router1#
Router1#
Router1#
Router1# show run | begin key chain
key-chain MYCHAIN
key 1
key string 5 098123456SA679
...
Router1# show run int Gi1/1
interface GigabitEthernet1/1
ip address 10.10.10.1 255.255.255.0
ip authentication mode eigrp 2 md5
ip authentication key-chain eigrp 2 MYCHAIN
...

Problem:
The issue was with a Level2/Severe bug with the IOS image running on one of the devices. Bug details below:

CSCdu73495 - All routes to network not seen because of invalid md5 authentication
http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCdu73495

Enhanced Interior Gateway Routing Protocol (EIGRP) routes cannot be seen even when message digest algorithm 5 (MD5) is authenticated on all routers. This problem is intermittent and may occur when authentication is turned off and subsequently turned back on again. Sometimes, this problem occurs just after authentication is enabled.

Workaround: This problem is intermittent and may be resolved by disabling and reenabling authentication a second time. This problem may automatically be resolved after a few minutes.

EIGRP Authentication problems & flaps on unrelated links

This bug is a duplicate of CSCdu73495, which causes authentication-related breakage in establishing peers, which eventually clears up on it's own after an indeterminate time. It can be triggered by bouncing peers/interfaces. You will not encounter this issue if you disable EIGRP authentication. CSCdu73495 was resolved in later versions of 12.1E IOS.

EIGRP neighbour cant be established if use MD5 authentication

C2610 EIGRP neighbour could be established via md5 authentication first time. After shut/no shut c2610 ethernet interface, it can't established any more. Via serial interface works fine.

EIGRP MD5 Authentication Breaks Neighbor Adjacencies over LANE

In a LANE environment with 3 or more devices running EIGRP, when upgrading from 12.1(6)E4 to 12.1(10)E4 on 7500's, EIGRP neighbor relationships may not be formed between devices running 12.1(10)E4. This is verified by performing a on one of the devices running 12.1(10)E4. The workaround for this scenario is to wait an unpredictable amount of time for the neighbors to converge, or remove and re-add EIGRP authentication from the interfaces on the affected devices. Also, neighbors can be statically configured in order for EIGRP to use unicast, rather than multicast.

2921-EIGRP flap due to bad TLV received on serial interface

Symptom: EIGRP flaps observed due to retransmission retry limit exceeded. Bad TLV error messages are seen in the logs. Conditions: Issue seen when 2921 replaces the 2611 device with similar configs.

Workaround: None. Apart from 2921, customer is using 2611 that works fine.

Known Affected Versions (Not a comprehensive list):
12.1(9)M

12.1(26)M
15.0M
12.1(8b)E15
12.3(12e)M

Fixed-In (Not comprehensive list):

12.1(10.2)M
12.2(4.2)M
12.0(30)SZ4
12.0(32)S6b
12.0(32)S7
12.0(32)SY4
12.0(32.3)S
12.1(6)E11
12.1(10.5)E
12.1(10.5)EC
12.2(4.2)PI
12.2(4.2a)DA
12.2(5.1)S
12.2(6.4)B
12.2(6.4)PB
12.2(15)BW
12.2(15)BX
12.2(15)ZN
12.0(32.11.1)SY

Workarounds:

1. Disable then re-enable EIGRP authentication;
2. Instead of MD5, use clear text authentication; or
3. Disable EIGRP authentication.

Permanent Fix:
Upgrade IOS version.

Due to intermittence/unpredictability, either use clear text authentication or disable authentication outright if IOS upgrade is not possible immediately. However, bouncing (disable/re-enable) the authentication can serve as your quick fix.

IOS: show interface FastEthernet mod/port - Detailed

2010-10-26T12:11:00.000+08:00

The show interface output for physical interface

Router#sh interfaces FastEthernet 6/1

FastEthernet6/1 is up, line protocol is up (connected)
Hardware is C6k 100Mb 802.3, address is 0009.11f3.8848 (bia 0009.11f3.8848)
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Full-duplex, 100Mb/s
input flow-control is off, output flow-control is off
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:14, output 00:00:36, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue :0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
1117058 packets input, 78283238 bytes, 0 no buffer
Received 1117035 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
285811 packets output, 27449284 bytes, 0 underruns
0 output errors, 0 collisions, 2 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out

Show interface output (physical interface) explained

up, line protocol up (connected) - the "up" is the physical layer (OSI layer 1) status of the link; the "line protocol up" is the data link layer (OSI layer 2) status of the link. Possible output are as follows:

up, line protocol up
up, line protocol down
down, line protocol down

Hardware - the interface hardware type, as well as the hardware/MAC address.

Description - the user-specified interface description as configured in the interface configuration mode.

MTU 1500 bytes - Maximum Transmission Unit.
BW - Bandwidth.
DLY - Delay (in microseconds).

reliability - reliability, as fraction of 255 (where 255/255 = 100% reliability), exponential average over 5 minutes.
txload - current output load, as fraction of 255 (where 255/255 = 100% saturation), exponential average over 5 minutes.
rxload - current input load, as fraction of 255 (where 255/255 = 100% saturation), exponential average over 5 minutes.

Encapsulation - current data link/layer 2 encapsulation of the interface.
loopback - defines if loopback (hardware or software) is enabled or disabled.

Full-duplex, 100Mb/s - current duplex and speed settings of the interface.

ARP Type - the Address Resolution Protocol type enabled.
ARP Timeout - the time in hh:mm:ss for each entry remains in ARP cache before being removed.

Last input 00:00:14, output 00:00:36 - the time in hh:mm:ss when the last packet was received (input) or transmitted (output) by the interface.

output hang - the time in hh:mm:ss when the interface was reset because of a transmission that took too long.

Last clearing of "show interface" counters - the time when the interface counters are last cleared via "clear counter" command.

Input queue: 0/2000/0/0 (size/max/drops/flushes) - the input queue counters and thresholds; the first number (size) is the current number of frames in the queue; the second number (max) is the maximum number of frames in the queue before it starts dropping; the third number (drops) is the number of frames dropped because the max was exceeded; the last number (flushes) is the number of low-priority frames dropped due to Selective Packet Discard (SPD) algorithm when CPU is overloaded.

Total output drops: 0 - total number of packets dropped because the output queue is full; high output drops may indicate mismatched bandwidth settings of this and the remote connecting interface.

Queueing strategy - either First-In/First-Out (fifo), priority-list, custom-list, and weighted-fair.

Output queue :0/40 (size/max) - The number of packets in the output queue. Size is the current number of frames in the queue. Max is the number of frames the queue can hold before it starts dropping frames.

5 minute input/output rate - The average input and output rate seen by the interface in the last five minutes. The interval can be changed via the "load-interval " interface command.

packets input, bytes - Total number of error-free packets received by the system. Total number of bytes, including data and MAC encapsulation, in the error free packets received by the system.

no buffer - The number of packets received and discarded because there is no buffer space. Can be caused by broadcast storms.

Received broadcasts - The number of broadcast/multicast packets received by the interface.

runts - The number of packets that are discarded because they are smaller than the minimum packet size.

giants - The number of packets that are discarded because they exceed the maximum packet size (MTU).

throttles - The number of times the receiver on the port was disabled, possibly due to buffer or processor overload.

input errors - Includes runts, giants, no buffer, CRC, frame, overrun, and ignored counts.

CRC - Number of packets where the CRC generated by the originating far-end device does not match the checksum calculated from the data received; usually indicates noise or transmission problems on the LAN interface or the LAN bus itself.

frame - Number of packets received incorrectly having a CRC error and a noninteger number of octets A(elignment errors).

overrun - Number of times the receiver hardware was unable to hand received data to a hardware buffer because the input rate exceeded the receiver's ability to handle the data.

ignored - Number of received packets ignored by the interface because the interface hardware ran low on internal buffers.

watchdog - Number of times watchdog receive timer expired. It happens when receiving a packet with length greater than 2048.

multicast - Number of multicast packets received.

pause input - Number of times the connected device requests for a traffic pause when its receive buffer is almost full. This counter is incremented for informational purposes, since the switch accepts the frame. The pause packets stop when the connected device is able to receive the traffic.

input packets with dribble condition detected - A dribble bit error indicates that a frame is slightly too long. This frame error counter is incremented for informational purposes, since the switch accepts the frame.

packets output, bytes - Total number of error-free packets transmitted by the system. Total number of bytes, including data and MAC encapsulation, in the error free packets transmitted by the system.

underruns - Number of times that the transmitter has been run faster than the switch can handle. This can occur in a high throughput situation where an interface is hit with a high volume of bursty traffic from many other interfaces all at once. Interface resets can occur along with the underruns.

output errors - Sum of all errors that prevented the final transmission of datagrams out of the interface.

collisions - Number of times a collision occurred before the interface transmitted a frame to the media successfully. Collisions are normal for interfaces configured as half duplex but must not be seen on full duplex interfaces. If collisions increase dramatically, this points to a highly utilized link or possibly a duplex mismatch with the attached device.

interface resets - Number of times the interface transitioned from up to down to up.

babbles - Number of times that the transmit jabber timer expired. A jabber is a frame longer than 1518 octets (which exclude framing bits, but include FCS octets), which does not end with an even number of octets (alignment error) or has a bad FCS error.

late collision - Number of times a late collision occured. A late collision occurs when two devices transmit at the same time, and neither side of the connection detects a collision. The reason for this occurrence is because the time to propagate the signal from one end of the network to another is longer than the time to put the entire packet on the network. The two devices that cause the late collision never see that the other is sending until after it puts the entire packet on the network. Late collisions are not detected by the transmitter until after the first 64 byte slot time. This is because they are only detected in transmissions of packets longer than 64 bytes.

deferred - Number of frames that have been transmitted successfully after they wait because the media was busy. This is usually seen in half duplex environments where the carrier is already in use when it tries to transmit a frame.

lost carrier - The number of times the carrier was lost in transmission. This is usually caused by a bad cable. Check the physical connection on both sides.

no carrier - Number of times the carrier was not present in the transmission. This is usually caused by a bad cable. Check the physical connection on both sides.

output buffer failures, output buffers swapped out - Number of failed buffers and the number of buffers swapped out. A port buffers the packets to the Tx buffer when the rate of traffic switched to the port is high and it cannot handle the amount of traffic. The port starts to drop the packets when the Tx buffer is full and thus increases the underruns and the output buffer failure counters. The increase in the output buffer failure counters can be a sign that the ports are run at an inferior speed and/or duplex, or there is too much traffic that goes through the port.

Reference: Troubleshooting Switch Port and Interface Problems
http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note09186a008015bfd6.shtml

CatOS: %SYS-3-PORT_OUT_DISCARD flood on disabled switchports

2010-10-26T04:57:00.000+08:00

Scenario
Multiple %SYS-3-PORT_OUT_DISCARD syslogs are generated for a switchport which is currently disabled/administratively shutdown.

2010 Jan 04 16:04:10 EST -05:00 %SYS-3-PORT_OUT_DISCARD:Port 4/47 detected 6029 output discard error(s) in last 30 minutes

MySwitch> sh port status 4/47
# = 802.1X Authenticated Port Name.

Port Name Status Vlan Duplex Speed Type
----- -------------------- ---------- ---------- ------ ----------- ------------
4/47 disabled 66 full 100 10/100/1000
MySwitch>

MySwitch> (enable) sh run 4
... [output omitted] ...
#module 4 : 48-port 10/100/1000BaseT Ethernet
set port disable 4/3-48

Explanation
This is due to an identified bug on the CatOS version.

CSCeg24345 - WS-X6748-GE-TX: Tx counters increment on not connected ports

WS-X6748-GE-TX: Tx counters increment on not connected ports On a WS-X6748-GE-TX module in a Catalyst 6500 running CatOS 8.2(2), a port that is not-connected may increment Tx counters as well as ifOutErrors, ifOutDiscards and txCRC

This bug impacts CatOS releases prior to 8.6 and occurs on WS-X6748-GE-TX blades. It is a cosmetic bug and is non-service impacting.

Versions:

1st Found-In: 8.2(2)

Fixed-In : 8.4(3.2), 8.4(4), 8.6(0.85)TAL

Note that although this is not service-impacting, it may wreck havoc on your monitoring system, as it will generate one syslog for each disabled port every thirty minutes.

IOS: %SYS-SP-3-CPUHOG: RFSS_server_action

2010-08-16T10:18:00.000+08:00

Scenario:
A Cat6K throws the following syslog messages:

Jul 18 01:48:12.362 EDT: %SYS-SP-3-CPUHOG: Task is running for (4000)msecs, more than (2000)msecs (0/0),process = RFSS_server_action.

-Traceback= 4045D2CC 4045F5F8 4045F504 4047F45C 4047ED38 4047F31C 40481F5C 40489F04 4048A3CC 4048AF5C 40485DE4 4048B1AC 404816A8 402E41D8 40451534 4029A764

Jul 18 01:48:14.366 EDT: %SYS-SP-3-CPUHOG: Task is running for (2000)msecs, more than (2000)msecs (1/0),process = RFSS_server_action.

-Traceback= 4045D2A8 4045F5F8 4045F504 4047F45C 4047ED38 4047F31C 40481F5C 40489F04 4048A3CC 4048AF5C 40485DE4 4048B1AC 404816A8 402E41D8 40451534 4029A764

Jul 18 01:48:18.370 EDT: %SYS-SP-3-CPUHOG: Task is running for (2000)msecs, more than (2000)msecs (2/1),process = RFSS_server_action.

-Traceback= 4045D2A8 4045F5F8 4045F504 4047F45C 4047ED38 4047F31C 40481F5C 40489F04 4048A3CC 4048AF5C 40485DE4 4048B1AC 4048B504 404817C0 402E440C 40451660

Description:

The traceback shown indicates a problem with writing into the flash disk. Running the privileged-mode command "dir disk1:" will cause your login session to apparently hang for a few minutes. After that the logs will be filled up with a new batch of the above %SYS-SP-3-CPUHOG syslogs and traceback messages.

------------------ show disk1: all ------------------
172683264 bytes available (83296256 bytes used)
******** ATA Flash Card Geometry/Format Info ********
ATA CARD GEOMETRY
Number of Heads: 16
Number of Cylinders 978
Sectors per Cylinder 32
Sector Size 512
Total Sectors 500736

ATA CARD FORMAT
Number of FAT Sectors 245
Sectors Per Cluster 8
Number of Clusters 62495
Number of Data Sectors 500596
Base Root Sector 598
Base FAT Sector 108
Base Data Sector 630 %

Error show disk1: (TF I/O failed in data-in phase)

Workaround/Resolution:

Reseat the compact flash card.
If error still occurs, reformat the flash card.
If error still occurs, replace the flash card.

FortiOS v3.00 MR5 - CPU Usage Too High

2010-07-05T20:37:00.001+08:00

Problem:

Fortigate 3600 running version 3.00 MR5 Patch 2 keeps sending high CPU trap SNMP traps to the SNMP trap servers. CPU utilization is confirmed to be high, based from the output of “get system performance status” or from the GUI. From “diag sys top”, confirmed that the “merged_daemons” process is using 99% of the total CPU, then shortly goes down to 14%.

Cause:
This is due to bug documented below:

0062617: race condition in flgd can cause merged_daemons to spin
The merged_daemons was constantly in the 'R' state and consuming 99% of CPU (when top is first started, the usage will display as 99% -- the usage will decrease to 14% while top is running).

Fix: Build: 0566

Workaround:
Restart merged_daemons as follows:

Enter diag sys top and take note of the PID of merged_daemons
Enter diagnose sys kill 11 [pid]

Note that merged_daemons may still climb back up to 99%.

Resolution/Workaround:
Upgrade to FortiOS MR6 or later.

IOS: %EARL_L3_ASIC-SP-3-INTR_WARN: EARL L3 ASIC: Non-fatal interrupt Packet Parser block interrupt

2010-01-04T16:25:00.002+08:00

Dec 18 09:54:43.989 JST: %EARL_L3_ASIC-SP-STDBY-3-INTR_WARN: EARL L3 ASIC: Non-fatal interrupt Packet Parser block interrupt
Dec 18 09:54:43.993 JST: %EARL_L3_ASIC-SP-3-INTR_WARN: EARL L3 ASIC: Non-fatal interrupt Packet Parser block interrupt

Description
These messages are indicating that the switch has received an invalid packet which contained a Layer 3 IP checksum error. These packets are normally being dropped silently within older IOS. In some IOS releases, the switch informs of this condition to warn users that there is (are) devices outside sending IP packets with checksum errors and/or with wrong length.

See CSCdz10360 (Need a CLI to be able to disable L3 error checking in HW) regarding this enhancement.

Workaround
These messages are purely informational. You may either:

SPAN all the Vlans and look at layer3 IP source address then remove the device generating invalid packets (unfortunately the switch doesn't track the IP address. The only way is to sniff every suspected Vlan to find out where those invalid packets are coming from).

Configure (this is a new config option added by means of CSCdz10360):
no mls verify ip checksum ---> to stop to check for packet checksum errors
no mls verify ip length ---> to stop to check for packet length errors
no mls verify ip length minimum ---> to eliminate check for IP packets that are minimum length.
no mls verify ip same-address ---> to stop checking for packet having equal source and destination IP address.

Do nothing as these are pure informational.

IOS: %ETHCNTR-3-LOOP_BACK_DETECTED : Keepalive packet loop-back detected on [chars]

2010-01-04T16:13:00.003+08:00

Scenario
The switch reports this error message, and the port is forced to linkdown:
%ETHCNTR-3-LOOP_BACK_DETECTED : Keepalive packet loop-back detected on [chars]

Oct 2 10:40:13: %ETHCNTR-3-LOOP_BACK_DETECTED: Keepalive packet loop-back detected on GigabitEthernet0/1
Oct 2 10:40:13: %PM-4-ERR_DISABLE: loopback error detected on Gi0/1, putting Gi0/1 in err-disable state

Description
The problem occurs because the keepalive packet is looped back to the port that sent the keepalive. Keepalives are sent on the Catalyst switches in order to prevent loops in the network. Keepalives are enabled by default on all interfaces. You see this problem on the device that detects and breaks the loop, but not on the device that causes the loop.

Workaround
Issue the no keepalive interface command in order to disable keepalives. A disablement of the keepalive prevents errdisablement of the interface, but it does not remove the loop.

Permanent Fix
In Cisco IOS Software Release 12.2(x)SE-based releases and later, keepalives are not sent on fiber and uplink interfaces by default. Upgrading the IOS version to this or later images should prevent the above issue in the first place.

Fortigate: Password recovery

2009-12-01T21:17:00.002+08:00

To reset the FortiGate unit password:

Connect the terminal to the FortiGate unit using the null modem cable.
Log on at the console with the user name "maintainer" and password "bcpb" followed immediately by the unit serial number. You must enter the alphabetic characters of the serial number in upper case.
Enter the following commands:
config system admin
edit admin
set password mypassword
end

Special Notes:

You must first power down the FortiGate unit, and power it up again.
Follow the above steps within one minute of the restart

If the maintainer login is initially unsuccessful, try the following two tips to ensure successful login:

You many not have the correct serial #. Copy the serial # displayed on the console during initial boot-up and paste it into a terminal editor window.
In the terminal editor window, finish composing the full password by adding "bcpb" before the serial # and then copy & paste the entire password into the console.

F5 BIGIP: Verify/Restart SNMP Daemon

2009-11-25T00:24:00.004+08:00

Just in case you need to check the status and/or restart the SNMP daemon of the bigip (i.e., because it has stopped responding to SNMP polling), enter the following commands via the CLI:

For BIGIP v4

Check the SNMP daemon status:
/etc/bigstart/status/S40snmpd status

The correct output should be:
Status snmpd: (pid xxxxx) is running
Status bigsnmpd: (pid yyyyy) is running
Status rlxsnmpd: is not running

If the result is different from above (i.e., bigsnmpd is not running), restart the SNMP daemon:
/etc/bigstart/status/S40snmpd restart

For BIGIP v9

Check the current status of the SNMP daemon:
bigstart status snmpd

Restart the SNMP daemon
bigstart restart snmpd

Verify status of the SNMP daemon:
bigstart status snmpd

[root@bigip:Active]~# bigstart status snmpd
snmpd run (pid 12707) 90 days, 1 start
[root@bigip:Active]~# bigstart restart snmpd
[root@bigip:Active]~# bigstart status snmpd
snmpd run (pid 4822) 6 seconds, 2 starts
[root@bigip:Active]~#

CatOS : SYS-2-MOD_TEMPSENSORFAIL flood from X6148A-GE-45AF

2009-11-11T01:24:00.004+08:00

CSCsl37513
http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCsl37513

SYS-2-MOD_TEMPSENSORFAIL:Module w/ X6148A-GE-45AF and CatOS

Symptom:
Numerous WS-X6148 linecards generate the following error:

%SYS-2-MOD_TEMPSENSORFAIL:Module # temperature sensors failed, please %powercycle the module

Conditions:
No production impact related to this message.

Workaround:
Powercycle module as requested by the error message.

set module power down module_number
set module power up module_number

Permanent Fix:
Upgrade IOS/CatOS to the below versions or later:
8.7(0.22)FW124
8.7(1.62)LAR
8.6(5.7)
8.7(0.22)BUB48
12.2(33.3.13)SXH
12.2(33)SXH4

IOS: IP SLA : SNMP : Router crashes and reloads if up for more than 497 days

2009-11-11T01:24:00.003+08:00

CSCsa57468
rttmon-mib does not return getnext value when queried via snmp

Symptom:
Concord poller crashes when polling a router that has been configured with IP SLA. Infact this DDTS will surface when doing snmp gets for the objects mentioned in the Conditions section below coming from any NMS (e.g. Concord, IPM, Spectrum, etc.)

Conditions:
The SNMP GETNEXT request is sent to the router for the following OIDs:

rttMonJitterStatsCompletions
rttMonStatsCaptureCompletions
rttMonStatsTotalsInitiations
rttMonStatsCaptureEntry (rttMonStatsCaptureCompletion etc.)
rttMonStatsCollectEntry
rttMonStatsTotalsEntry
rttMonJitterStatsEntry
rttMonHTTPStatsEntry.

The router does not return the next index of these OIDs, but the same index. This happens only when the router has been up and running for longer than 497 days.

Affected IOS Versions:

12.2(15)T
12.2SXH

Workaround:
This problem is only happening when polling the CISCO-RTTMON-MIB via snmp get. Use the IOS CLI to avoid it.

Permanent Fix:
Upgrade the IOS version.

Fixed in:

12.3(14.12)M
12.4(1.5)M
12.2(33)SRC
12.2(40)SE
12.2(44)SE
12.3(11)T6
12.3(11)YW
12.3(14)T2
12.4(1.8)T
12.4(1a)M
12.2(33)SXI
12.2(32.8.80)SR
12.2(32.8.11)XID112.9
12.2(33.1.7)SXH
12.2(33)SXH2
12.2(33)SB
12.2(32.8.99a)SR133
12.2(32.8.11)XJC153.1

IOS: %SSH-3-PRIVATEKEY: Unable to retrieve RSA private key

2009-11-08T12:15:00.002+08:00

Symptoms:
The device getting numerous %SSH-3-PRIVATEKEY syslogs, usually followed by a traceback such as the following:

Nov 7 02:40:49.542 GMT: %SSH-3-PRIVATEKEY: Unable to retrieve RSA private key for
-Process= "SSH Process", ipl= 0, pid= 148
-Traceback= 61D48360 61D44B24 61D462C4 6053BD88 6053BD6C
Nov 8 02:16:22.452 GMT: %SSH-3-PRIVATEKEY: Unable to retrieve RSA private key for
-Process= "SSH Process", ipl= 0, pid= 148
-Traceback= 61D48360 61D44B24 61D462C4 6053BD88 6053BD6C

Explanation:
Often seen if hostname or domain name of the router has been changed.

Workaround/Fix:

Remove existing RSA Key:
crypto key zeroize rsa
Gnerate RSA key with the following commands:

show crypto key mypubkey rsa
crypto key gen rsa general-keys label label
ip ssh rsa keypair-name label

where label = unique label/identifier

Wireless: %DTL-1-ARP_POISON_DETECTED

2009-11-04T12:34:00.006+08:00

CSCsm25943 Change label for %DTL-1-ARP_POISON_DETECTED message
http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCsm25943

Symptom:

A Wireless LAN Controller may emit a message similar to the following:

DTL-1-ARP_POISON_DETECTED: STA [00:01:02:0e:54:c4, 0.0.0.0] ARP (op 1) received with invalid SPA 192.168.1.152/TPA 192.168.0.206

However, when one peruses the entry in the Cisco Wireless LAN Controller System Message Guide, 4.2, for this message, he may find it to be misleading and bereft of useful information.

Conditions:

This message does not necessarily imply that any actual "ARP poisoning" (ARP spoofing) is going on. Rather, it is emitted whenever the following conditions pertain:

- WLAN is configured with DHCP Required
- A client, after associating on that WLAN, transmits an ARP message without first DHCPing

This may be normal behavior - for example, when the client is statically addressed, or when the client is holding a valid DHCP lease from a prior association.

The effect of this condition is that the client will be unable to send or receive any data traffic, until it DHCPs thru the WLC.

In more detail, here is how to interpret the example message above:

DTL-1-ARP_POISON_DETECTED: STA [00:01:02:0e:54:c4, 0.0.0.0] ARP (op 1) received with invalid SPA 192.168.1.152/TPA 192.168.0.206

DTL-1-ARP_POISON_DETECTED
- WLC received an ARP packet from a client in DHCP_REQ state

STA [00:01:02:0e:54:c4, 0.0.0.0]
- the client ("STA" - 802.11 wireless station) has a MAC address of 00:01:02:0e:54:c4, and an IP address unknown to the WLC ("0.0.0.0")

ARP (op 1)
- the offending packet received from client was an ARP request (opcode 1)

invalid SPA 192.168.1.152/TPA 192.168.0.206
- the source IP address (SPA - "sender protocol address") of the ARP request was 192.168.1.152
- the target IP address (TPA - "target protocol address") of the ARP request was 192.168.0.206

Workaround:

figure out whether or not you want to force your wireless clients to DHCP first, after associating, before they can send IP packets.

If no, then unconfigure DHCP required, and you won't get this problem.

If yes, then configure all clients to use DHCP.

If the client is configured for DHCP, but still sometimes sends IP packets after associating without re-DHCPing, then:
- See if the client eventually does re-DHCP & if so doesn't suffer an unacceptable outage before re-DHCPing. If the outage before re-DHCPing is acceptable, then you can just ignore this message.
- If the client never does re-DHCP after associating, then it will never be able to pass L3 traffic. So in that case, either figure out how to change the client's behavior so that it always does re-DHCP after associating, or else just accept that this client won't work in this application, or else reconsider your decision to use "DHCP required".

Further Problem Description:

If the source IP address (SPA) of the ARP is an APIPA address (i.e. one in 169.254.0.0 /16), then this may be indicative of the STA's attempting but failing to acquire an address via DHCP. In which case you may want to verify that your DHCP implementation works.

1st Found-In:
4.2(61.0)

Fixed-In:
7.0(63.0)

Wireless: %APF-3-USER_DEL_FAILED

2009-11-04T05:43:00.006+08:00

Event : %APF-3-USER_DEL_FAILED: Unable to delete username unknown for mobile mac-address

Explanation: This error can mean slightly different things depending on EAP method. Basically it is a side effect of an EAP method with identity protection.

EAP authentication is done in two phases. The first phase of authentication uses generic anonymous external identity in order to establish the tunnel. In phase 2, client authentication is done in the established tunnel. The client sends the original username and password to authenticate and establish a client authorization policy. As this authentication method hides the original user name at the first phase of authentication, the controller does not have a way to add the correct username to the authenticated user list. So the controller uses the anonymous username. The end result generates this error.

Further details on the related bug below:

%APF-1-USER_DEL_FAILED: apf_ms.c:5055 flooding msglogs.
http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCsz51403

Symptom:
The "%APF-1-USER_DEL_FAILED: apf_ms.c:5055" message floods msglogs

Conditions:
1. Multiple clients connect to the controller with the same user name, or
2. AAA server returns a user name that is different to what is registered by the client.

Workaround:
No, but it does not affect any controller feature

1st Found-In

5.2(178.12)

5.2(178.13)

Fixed-In

6.0(176.0)

5.2(186.0)

6.1(34.0)

6.0(182.0)

4.2(205.1)

5.2(193.0)

4.2(207.0)

BIG-IP License Error - Permission denied

2009-07-28T11:05:00.002+08:00

Scenario:
After a reboot, the BIG-IP returns a licensing error. Reactivating the license does not work as well.

In the qkview output, we see the following:

2009-07-19 22:56:17,428 ERROR [Thread-15] util.F5Error: - An error has occurred while trying to process your request.

java.io.FileNotFoundException: /var/tmp/bigip.license (Permission denied)

Workaround:

Delete /var/tmp/bigip.license file if it exists.

Ensure that the user you are logged into has full premissions / full admin rights to the BIG-IP box.

Reactivate the license file.

Basic MQC Configuration

2009-06-07T09:06:00.003+08:00

Three easy steps to MQC (Modular QoS CLI) Configuration:
Step 1: Classify traffic via class-map
Step 2: Assign policies to the traffic classes via policy-map
Step 3: Apply above policies to an interface via service-policy

! ------------------------------------
! Sample Scenraio:
! ------------------------------------
For the following traffic going out Serial0/1 of the device, do the following:
* for voice traffic, reserve 256kbps priority bandwidth
* for email traffic (pop3, imap, smtp), reserve 128kbps bandwith
* for telnet traffic coming from 10.10.10.10, limit to 3200bps bandwidth

! ------------------------------------
! BEGIN CONFIGURATION
! ------------------------------------
Router(config)#access-list 101 host 10.10.10.10 any eq 23
Router(config)#class-map VOICE
Router(config-cmap)#match protocol rtp
Router(config-cmap)#exit
Router(config)#class-map match-any EMAIL
Router(config-cmap)#match protocol pop3
Router(config-cmap)#match protocol imap
Router(config-cmap)#match protocol smtp
Router(config-cmap)#exit
Router(config)#class-map ACL_101
Router(config-cmap)#match access-group 101
Router(config-cmap)#exit

Router(config)#policy-map MY_POLICY
Router(config-pmap)#class VOICE
Router(config-pmap-c)#priority 256
Router(config-pmap-c)#exit
Router(config-pmap)#class EMAIL
Router(config-pmap-c)#bandwidth 128
Router(config-pmap-c)#exit
Router(config-pmap)#class ACL_101
Router(config-pmap-c)#police 3200
Router(config-pmap-c)#exit
Router(config-pmap)#exit

Router(config)#interface Serial0/1
Router(config-if)#service-policy output MY_POLICY
Router(config-if)#exit
Router(config)#

! ------------------------------------
! NOTES
! ------------------------------------

Router(config)#class-map [match-all|match-any] class_name

match-all - the class must match all the succeeding criteria

match-any - the class must match any of the succeeding criteria

if not specified, defaults to match-all

Router(config-cmap)#match {protocol|access-group} value

protocol - based on known traffic classes via NBAR

access-group - based on ACLs

not limited to the above criteria; other criteria include class-map (i.e. nested class-maps), CoS, DSCP, IP Precedence, input-interface, MAC address, QoS group, UDP Port Ranges

Router(config-if)#service-policy {input|output} policy-name

only one policy per direction per interface can be applied;

that is, each interface can have at most one inbound policy and one outbound policy.

Other command syntax will be dealt with in another post.

IOS: %BGP_MPLS-3-GEN_ERROR

2009-04-15T15:37:00.002+08:00

Mar 18 20:41:38.892 EDT: %BGP_MPLS-3-GEN_ERROR: BGP: MPLS outlabel changed, MPLS forw not updated, prefix not in routing table -Traceback= 10D36950 10D3709C 10B10388 10B10718 10AEEFD0 10AEF030 10B53A50 10B53DC0 10AF588C 10AFD610 10AFE8E0 10A44524 10A3B6D4
Mar 18 20:41:38.892 EDT: %BGP_MPLS-3-GEN_ERROR: BGP: MPLS outlabel changed, MPLS forw not updated, prefix not in routing table -Traceback= 10D36950 10D3709C 10B10388 10B10718 10AEEFD0 10AEF030 10B53A50 10B53DC0 10AF588C 10AFD610 10AFE8E0 10A44524 10A3B6D4
Mar 18 20:41:38.892 EDT: %BGP_MPLS-3-GEN_ERROR: BGP: MPLS outlabel changed, MPLS forw not updated, prefix not in routing table -Traceback= 10D36950 10D3709C 10B10388 10B10718 10AEEFD0 10AEF030 10B53A50 10B53DC0 10AF588C 10AFD610 10AFE8E0 10A44524 10A3B6D4

Cisco IOS Software, Catalyst 4500 L3 Switch Software (cat4500e-ENTSERVICES-M), Version
12.2(50)SG1, RELEASE SOFTWARE (fc2)
Technical Support:
http://www.cisco.com/techsupport
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Tue 10-Feb-09 00:17 by prod_rel_team
Image text-base: 0x10000000, data-base: 0x124FED8C

ROM: 12.2(44r)SG
Darkside Revision 0, Jawa Revision 11, Tatooine Revision 140, Forerunner Revision 1.74

MyRouter uptime is 5 days, 3 hours, 12 minutes
System returned to ROM by power-on
System restarted at 19:50:40 EDT Fri Mar 13 2009
System image file is "bootflash:/cat4500e-entservices-mz.122-50.SG1.bin"

cisco WS-C4900M (MPC8548) processor (revision 2) with 524288K bytes of memory.
Processor board ID JAE130628BD
MPC8548 CPU at 1.33GHz, Cisco Catalyst 4900M
Last reset from PowerUp
1 Virtual Ethernet interface
36 Gigabit Ethernet interfaces
16 Ten Gigabit Ethernet interfaces
511K bytes of non-volatile configuration memory.

Configuration register is 0x2102

CSCse15707: Trace back seen at bgp_ipv4_mpls_label_change.

http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCse15707

First Found-In:

12.2(32.8.7)SRB

12.2(32.8.8)SRA

12.4(12.15)PI7e

12.2(31)SB9

12.2(31.4.5)SB11

12.2(31.4.11)SB12

12.2(28.5.24)SB13

Fixed-In:

12.2(32.8.63)SR

12.2(33)SRC

12.4(13.8)PI7c

12.2(33)SRB3

12.2(33.2.18)SRB

12.2(33)SB

12.2(32.8.99a)SR133

12.2(31)SB13

12.2(32.8.1)YCA172.24

12.4(21.14.9)PIC1

Symptoms: A router may generate the following error message and a traceback:

%BGP_MPLS-3-GEN_ERROR: BGP: MPLS outlabel changed, MPLS forw not updated, prefix not in routing table

Conditions: This symptom is observed on a Cisco router that functions in a VPN carrier supporting carrier topology and that is configured for BGP and IPv4.

Workaround: This is a cosmetic issue, the traceback is harmless and the functionality of the router is not affected.

CATOS: %SYS-3-PKTBUFFERFAIL_ERRDIS: Packet buffer failure detected.

2009-04-15T15:36:00.002+08:00

%%SYS-3-PKTBUFFERFAIL_ERRDIS: Packet buffer failure detected. Err-disabling port [dec]/[dec]

Description:

This message indicates that the default error-detection packet buffer setting is error disabling the port. Whenever a parity failure is detected on the port, ASIC ports are error disabled. [dec]/[dec] is the module number/port number of the error-disabled port.

Recommended Action:

Power cycle the switching module with the error-disabled port. Note The next message appears as four lines.

Example:

2009 Mar 19 22:19:18 GMT +00:00 %SYS-3-PKTBUFFERFAIL_ERRDIS:Packet buffer failure detected. Err-disabling port 12/11.
2009 Mar 19 22:19:19 GMT +00:00 %SYS-3-PKTBUFFERFAIL_ERRDIS:Packet buffer failure detected. Err-disabling port 12/12.

MySwitch (enable) show port errdisable-timeout 12/12
Module 12 is not a Komodo+ Firewall
Module 12 is not a Venus SLB

Port Status ErrDisable Reason Port ErrDisableTimeout Action on Timeout
---- ---------- ------------------- ---------------------- -----------------
12/12 errdisable packet-buffer-error Enable No Change
MySwitch (enable)

MySwitch (enable) show port errdisable-timeout 12/11
Module 12 is not a Komodo+ Firewall
Module 12 is not a Venus SLB

Port Status ErrDisable Reason Port ErrDisableTimeout Action on Timeout
---- ---------- ------------------- ---------------------- -----------------
12/11 errdisable packet-buffer-error Enable No Change
MySwitch (enable)

MySwitch (enable) sh port status 12
Port Name Status Vlan Duplex Speed Type
----- -------------------- ---------- ---------- ------ ----- ------------
12/1 FOLPT1412 connected 562 full 100 10/100BaseTX
12/2 FOLPT1413 connected 562 full 100 10/100BaseTX
12/3 3/6 splpw232131 errdisable 562 full 100 10/100BaseTX
12/4 ldnpsmeg025 errdisable 565 full 100 10/100BaseTX
12/5 LDNPSM14006 errdisable 565 full 100 10/100BaseTX
12/6 LDNPSM14007 errdisable 565 full 100 10/100BaseTX
12/7 LDNPSM14008 errdisable 565 full 100 10/100BaseTX
12/8 LDNPSM02989 errdisable 565 full 100 10/100BaseTX
12/9 LDNPSM14015 errdisable 565 full 100 10/100BaseTX
12/10 LDNPSM14014 errdisable 565 full 100 10/100BaseTX
12/11 LDNPSM14012 errdisable 565 full 100 10/100BaseTX
12/12 lsc42n02-app2 errdisable 562 full 100 10/100BaseTX

MySwitch (enable) show port errdisable-timeout 12/3
Module 12 is not a Komodo+ Firewall
Module 12 is not a Venus SLB

Port Status ErrDisable Reason Port ErrDisableTimeout Action on Timeout
---- ---------- ------------------- ---------------------- -----------------
12/3 errdisable packet-buffer-error Enable No Change
MySwitch (enable)

Example:Resolution:

set module power down 12
set module power up 12

Related document:

Best Practices for Catalyst 4500/4000, 5500/5000, and 6500/6000 Series Switches Running CatOS Configuration and Management