RE: [nv-l] how to block bursts of traps (in theory)

[James Shanks <jshanks@us.ibm.com>]

Right.  it's a band-aid.  It doesn't really do anything about the storm or the processing load it imposes.  It just keeps the other applications, like nvcorrd, netmon, and others from being forced off trapd because they cannot keep up with the resulting workload.

Each connected application has an internal queue in trapd where he sends processed traps as pdu structures.  The apps  in turn must receive them so that they can be deleted from the queue.  If they cannot keep up and the queue fills, trapd forces them to disconnect to protect himself.  Making the default queue size bigger allows the connected apps to stay connected even when they are too busy processing what they already have to keep abreast of what trapd is now giving them.   Eventually they have to process every trap.

James Shanks
Level 3 Support  for Tivoli NetView for UNIX and NT
Tivoli Software / IBM Software Group

"Ken Karasek" <ken.karasek@hewitt.com> 05/22/2003 04:28 PM

To:        "Allison, Jason (JALLISON)" <JALLISON@arinc.com>         cc:        "'nv-l'" <nv-l@lists.tivoli.com>         Subject:        RE: [nv-l] how to block bursts of traps (in theory)

A possible suggestion to buffer, rather than block, the possibility of a trap storm would to be to increase the, "Set Trapd connected applications queue size: (Num)" field from the default 2000 to 10000.  This option can be found under the "Set options for trapd daemon".  It's not a fix more than it would be a band-aid to absorb a flurry of connections to trapd. 

______________________

Ken Karasek
IS Network Management
Hewitt Associates, LLC
100 Half Day Road
Lincolnshire, Illinois  60069
Phone: 847.295.5000
FAX: 847.295.8877

The information transmitted is intended only for the person(s) or entity(ies) to which it is addressed. This information may contain information that is confidential or otherwise protected from disclosure. If you are not the intended recipient of this message, or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message, including any attachments. Any dissemination, distribution or other use of the contents of this message by anyone other than the intended recipient is strictly prohibited. 
______________________

"Allison, Jason (JALLISON)" <JALLISON@arinc.com> 05/22/2003 09:15 AM

To:        "'nv-l'" <nv-l@lists.tivoli.com>        cc:                Subject:        RE: [nv-l] how to block bursts of traps (in theory)

More pontification ...

I would probably include trap information in the ovxecho to that effect.

The possibilities are endless.

Jason Allison
Principal Engineer
ARINC Incorporated
Office:  (410) 266-2006
FAX:  (410) 573-3026


-----Original Message-----
From: James Shanks [mailto:jshanks@us.ibm.com]
Sent: Thursday, May 22, 2003 10:10 AM
To: nv-l@lists.tivoli.com
Subject: RE: [nv-l] how to block bursts of traps (in theory)


But if you are going to do this, have it log the source of the traps so 
that you can get the owner of the offending box to reconfigure it

James Shanks
Level 3 Support  for Tivoli NetView for UNIX and NT
Tivoli Software / IBM Software Group




"Allison, Jason (JALLISON)" <JALLISON@arinc.com>
05/22/2003 09:50 AM


       To:     "'nv-l'" <nv-l@lists.tivoli.com>
       cc: 
       Subject:        RE: [nv-l] how to block bursts of traps (in theory)



Hypohetically ...

You could write an application that registers with trapd.  Monitor the
'througput' of traps over a period of time.  If your throughput exceeds a
defined threshold .... stop trapd.  Send an ovxecho that trapd was 
stopped.
Delay a period of time.  Start trapd, send a popup, start monitoring
throughput again.  Repeat.

That would be a fun app to write.

Jason Allison
Principal Engineer
ARINC Incorporated
Office:  (410) 266-2006
FAX:  (410) 573-3026


-----Original Message-----
From: James Shanks [mailto:jshanks@us.ibm.com]
Sent: Thursday, May 22, 2003 9:23 AM
To: nv-l@lists.tivoli.com
Subject: Re: [nv-l] how to block bursts of traps (in theory)


Once a trap gets into trapd, there is no way to stop the processing of it. 

Even the ones that are configured "Ignore" (Don't log or Display) still 
get passed on to all connected trap receivers who aren't filtering them 
out.  You can see this for yourself if you bring up an Event Window 
(nvevents) and then create a dynamic workspace.  One of the options on the 

workspace creation panel is to see all the "Don't Log or Display" events, 
which proves that they are passed along to nvcorrd, nvserverd, and to 
nvevents.   There is no configuration you can do to trapd to prevent this 
-- all traps are received, and all are processed.

So, once a trap storm is underway, and the entire event stream is loaded 
up so that processing is degraded, your only short term choice is to wait 
it out or to stop trapd.  You would also have to stop nvcorrd (which stops 

nvserverd) if  the event windows are backed up.   The you can wait awhile, 

hopefully until the storm clears, and restart.  Your only defense is to be 

pro-active.  If this happens more than once in your environment, then I 
recommend that you install an MLM as the primary trap receiver and have 
him threshold traps before sending them to NetView (trapd).  That way he's 

the one the with heavy workload and he will reduce the workload on trapd, 
and ultimately the entire system.

James Shanks
Level 3 Support  for Tivoli NetView for UNIX and NT
Tivoli Software / IBM Software Group




"Mildeberger, Thorsten" <thorsten.mildeberger@eds.com>
05/22/2003 02:38 AM


       To:     nv-l@lists.tivoli.com
       cc: 
       Subject:        [nv-l] how to block bursts of traps (in theory)



Hi all, 
this is more a theoretical question, but I would like to know how to take
care of it anyway. 

I know that sophisticated netview rules should be implemented to avoid 
burst
of traps sent to a event server or even better, not to permit components
like routers, switches and so on to forward traps to a trap receiver like
netview that much. 

But what to do best in theory when netview receives a huge amount of traps
for some reason? I could imagine that such a large number of traps would
block the netview box more or less completely. The only solution to get 
rid
of that situation would be to stop the trapd or to reconfigure trapd?

Many thanks.

best regards, 
> Thorsten Mildeberger
> 
> EMS Solutions - SMC Tools & Automation
> GOSD - Central Region
> EDS Deutschland GmbH
> Eisenstr. 43
> 65428 Rüsselsheim
> Tel.: +49 (0) 6142 80-3706 
> Fax.: +49 (0) 6142 80-3030 
> mailto:thorsten.mildeberger@eds.com
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: nv-l-unsubscribe@lists.tivoli.com
For additional commands, e-mail: nv-l-help@lists.tivoli.com

*NOTE*
This is not an Offical Tivoli Support forum. If you need immediate
assistance from Tivoli please call the IBM Tivoli Software Group
help line at 1-800-TIVOLI8(848-6548)





---------------------------------------------------------------------
To unsubscribe, e-mail: nv-l-unsubscribe@lists.tivoli.com
For additional commands, e-mail: nv-l-help@lists.tivoli.com

*NOTE*
This is not an Offical Tivoli Support forum. If you need immediate
assistance from Tivoli please call the IBM Tivoli Software Group
help line at 1-800-TIVOLI8(848-6548)

---------------------------------------------------------------------
To unsubscribe, e-mail: nv-l-unsubscribe@lists.tivoli.com
For additional commands, e-mail: nv-l-help@lists.tivoli.com

*NOTE*
This is not an Offical Tivoli Support forum. If you need immediate
assistance from Tivoli please call the IBM Tivoli Software Group
help line at 1-800-TIVOLI8(848-6548)





---------------------------------------------------------------------
To unsubscribe, e-mail: nv-l-unsubscribe@lists.tivoli.com
For additional commands, e-mail: nv-l-help@lists.tivoli.com

*NOTE*
This is not an Offical Tivoli Support forum. If you need immediate
assistance from Tivoli please call the IBM Tivoli Software Group
help line at 1-800-TIVOLI8(848-6548)

---------------------------------------------------------------------
To unsubscribe, e-mail: nv-l-unsubscribe@lists.tivoli.com
For additional commands, e-mail: nv-l-help@lists.tivoli.com

*NOTE*
This is not an Offical Tivoli Support forum. If you need immediate
assistance from Tivoli please call the IBM Tivoli Software Group
help line at 1-800-TIVOLI8(848-6548)