RE: [nv-l] Cursed Cisco Trap Formats

["Stringfellow, William" <William.Stringfellow@bankofamerica.com>]

Scott,

    We have seen this many times because the particular Cisco device is sending it's own version of Link UP/Down traps (there are many devices that have unique ways of sending what should be a generic trap.)   We learned about this when we first put MLMs in place and started seeing the raw trap varbinds. 

    Anyway, to fix it, figure out what the oid is for the device that is giving you the wrong number of varbinds, create a new trapd.conf entry for it in the enterprise piece, then add LinkUp and LinkDown specific traps to your menu for that enterprise.

    Under the "Event Log Message" use the generic "enterprise: $E args($#):\n$*"

 

    The $* part will give you each of the varbinds in an individual line in your trapd.log.  Then you can see what information is being provided and change the Event Log Message format so that it makes sense to your operators.

     I go through the log once a day looking for "no known format" or "FMT ERROR" messages and massage the trapd.conf to accomodate them.  We have found many traps where the original log entry had nothing to do with the real trap, remember that the definition of the trap stops at the last piece of the oid that NetView can interpret.  So pay attention to the first part of the trap where it says "received from enterprist AAAA" that AAAA is the name you will see in the list of enterprises when you bring up the trap definition window.

 

        Good luck,

        Bill

 

 

 

 

 -----Original Message-----
From: Barr, Scott [mailto:Scott_Barr@csgsystems.com]
Sent: Thursday, October 03, 2002 9:27 AM
To: nv-l@lists.tivoli.com
Subject: [nv-l] Cursed Cisco Trap Formats

NetView 7.1.1 on Solaris 2.8

 

Okay guys, I am looking for a way to skin a Cisco cat. The problem is due to the fact that we run a wide variety of protocols and routers, we often do not run the latest Cisco IOS versions. I recently had a situation where I observed this in trapd.log:

 

1033361376 3 Sun Sep 29 23:49:36 2002 <routernamehere> A Cisco_Link_Down trap received from enterprise cisco with 3 arguments: ifIndex=24; ifDescr=ATM1/0.8-aal5 layer; ifType=49; locIfReason=FMT ERROR: accessing element #4, only 3 available                                                               

Notice the format error. The reason this occurs is because under most circumstances the cisco IOS is delivering only 3 elements and the trap format in trapd.conf has 4 elements defined. So I opened TAC case on this with Cisco and they told me to use the following command on the routers:

 

snmp-server trap link ietf

 

Now, the trap comes in and looks like this:

 

1033478849 3 Tue Oct 01 08:27:29 2002 <routernamehere> A Cisco_Link_Down trap received from enterprise cisco with 5 arguments: ifIndex=26; ifDescr=2; ifType=2;  locIfReason=ATM1/0.9-aal5 layer                  

 

Now we get five arguments (still only 4 defined in trapd.conf) Okay, first problem is the format is still wrong since trapd.conf is not matching up with the IETF standard (which I have not been able to find yet). But thats no big deal, since I assumed I was writing some code to catch the variables and make intelligent decisions about what to do with it.

 

But wait! There is more! A lot of the routers send in link up/down traps in this format:

 

1033480388 3 Tue Oct 01 08:53:08 2002 <routernamehere>  A Cisco_Link_Down trap received from enterprise cisco with 4 arguments: ifIndex=1; ifDescr=Serial0/0;  ifType=22;  locIfReason=administratively down

 

So, to sum it up, I get link up/down traps with either 3, 4, or 5 arguments depending on what router is sending it in. They all have the same cisco enterprise ID so using trapd.conf to bypass the issue is not possible. I use rulesets (not command for automatic action in trapd.conf) to suppress interface outages of less than 5 minutes. I lose this functionality if I just pass the trap via command for automatic action. So what I need is a script that I can run using an action node, that can decipher whether there are 3,4, or 5 arguments and then parse them out. I am paging/emailing in my ruleset using action nodes, I would have to move them to the parsing script (no problem - we use nvpage and mailx)

 

Suggestions on scripts? How to code trapd.conf? Where is Cisco headquarters and what is composition of the materials used to build it? I *am* not a script coder person, so if you send me a perl script write it the way any idiot C programmer could read it and not one of your fancy-only-takes-1-line-of-completely-unreadable code.

 

- Signed: stuck between a rock and  a hard place with a boulder on my head.

Scott Barr

Network Systems Engineer

CSG Systems

Phone: 402-431-7939

Fax: 402-431-7413

Email: Scott_Barr@csgsystems.com