Re: Rules stop functioning -- some things to check

To:	nv-l@lists.tivoli.com
Subject:	Re: Rules stop functioning -- some things to check
From:	"James Shanks" <jshanks@us.ibm.com>
Date:	Wed, 7 Nov 2001 12:42:22 -0500

This is a multipart message in MIME format.

Scott -

With netstat, the number of lines is not an issue, but what they say is.
netstat shows you information about socket connections and on any function 
system you will have many.
If you have NetView then you will have many more.  And if you run rulesets 
you will see two for each ruleset you are running.  One is for the 
connection between whoever started the ruleset  (usually either nvserverd 
or actionsvr) and one for nvcorrd's connection back to that process.

If the lines which mention nvcorrd do not show high numbers under Recv_Q 
or Send_Q then you do not have a socket backup problem.  So then you 
should be looking at the nvcorrd.alog or blog after issuing the "nvcdebug 
-d all".   You can follow what goes on thee by looking for the "Received a 
Trap" and "Finished with the trap" eyecatchers.  What happens in between 
is nvcorrd processing that trap.    When an action is sent to actionsvr, 
you will see "Action:"  entries.
Without those the event did not result in an action being sent and the 
preceding trace should tell you why.


James Shanks
Level 3 Support  for Tivoli NetView for UNIX and NT
Tivoli Software / IBM Software Group
 





"Scott Bursik" <tivoliesm@hotmail.com>
Sent by: owner-nv-l@tkg.com
11/07/2001 12:26 PM
Please respond to IBM NetView Discussion

 
        To:     nv-l@tkg.com
        cc: 
        Subject:        Re: [NV-L] Rules stop functioning -- some things to 
check

 

James,

When I run the netstat -a command I have 887 lines listed. Of those, about 
a 
dozen have nvcorrd listed. Is that bad to have that many? I have started 
the 
trace but do not see anything going in the nvaction.alog or blog as of 
yet. 
I do have "BLOCK" listed as the choice for the "purple pizza". As far as 
the 
name resolution goes...the node that was causing the error in the nvcorrd 
lof file was deleted during the database cleanup last night and I have not 

seen him come back yet.

Thanks James


>From: "James Shanks" <jshanks@us.ibm.com>
>Reply-To: IBM NetView Discussion <nv-l@tkg.com>
>To: IBM NetView Discussion <nv-l@tkg.com>
>Subject: Re: [NV-L] Rules stop functioning -- some things to check
>Date: Wed, 7 Nov 2001 12:09:01 -0500
>
>The way to determine whether or not your rules are working is to trace
>nvcorrd.
>This you do with the nvcdebug command .  A full trace is achieve by
>issuing "nvcdebug -d all".
>Then you can look in the nvcorrd.alog and see what is happening.  After
>1000 lines have been written there it will switch to the blog.  And back
>gain after another 1000 lines.
>
>You said you were running a lot of rules out of ESE.automation.
>When you do "netstat -a" do you see backed up sockets for nvcorrd?  These
>would be entries in the netstat list which show "nvcorrd" and have high
>numbers in the Send or Receive queues.  The most you can have is 32,767
>but any relatively high number indicates a problem.
>If you see this, check the Initial node (the "purple pizza") of your
>rulesets running out of ESE.automation.  Does it say PASS or BLOCK?  Only
>BLOCK is allowed for a ruleset running in the background out of
>ESE.automation.
>
>
>I also presume that you saw my earlier append about name resolution. Your
>  Query Smartset test will fail if the attribute resolves to an IP 
Address
>rather than hostname.
>
>James Shanks
>Level 3 Support  for Tivoli NetView for UNIX and NT
>Tivoli Software / IBM Software Group
>
>
>
>
>
>
>"Scott Bursik" <tivoliesm@hotmail.com>
>Sent by: owner-nv-l@tkg.com
>11/07/2001 11:56 AM
>Please respond to IBM NetView Discussion
>
>
>         To:     nv-l@tkg.com
>         cc:
>         Subject:        Re: [NV-L] Rules stop functioning
>
>
>
>Thanks for all of your help. I think I will have to open a PMR and I will
>post the resolution.
>
>Scott
>
>
>
> >From: "Stephen Hochstetler" <shochste@us.ibm.com>
> >Reply-To: IBM NetView Discussion <nv-l@tkg.com>
> >To: IBM NetView Discussion <nv-l@tkg.com>
> >Subject: Re: [NV-L] Rules stop functioning
> >Date: Wed, 7 Nov 2001 10:44:19 -0600
> >
> >Scott,
> >
> >Sorry to confuse you.    I was wondering which you were doing, "in-line
> >action" or just "action".    Since you are just doing "action", you
>should
> >NOT be having the problem that you see.   The actionsvr forks the 
process
> >and rules continue.      If you see actionsvr stop, then possibly it is
> >having troubles forking a process.
> >
> >Ideas
> >    -- when you see this happen, are you low in space in /tmp or 
/usr/OV
>?
> >If you run out of space in "/"   or "/tmp"  or "/usr/OV"    it is
>probably
> >related to your problem.
> >
> >   -- if your machine is creating zombies, it is probably related to 
your
> >problem.
> >
> >If neither of these are true, you probably have an issue which is best
> >handled by calling support.
> >
> >Kind regards,
> >Stephen Hochstetler              shochste@us.ibm.com
> >International Technical Support Organization  - Austin
> >Office - 512-436-8564                      FAX - 512-436-8701
> >
> >ITSO redbooks at  http://www.redbooks.ibm.com
> >
> >
> >
> >                     "Scott Bursik"
> >                     <tivoliesm@hot       To:     nv-l@tkg.com
> >                     mail.com>            cc:
> >                     Sent by:             Subject:     Re: [NV-L] Rules
> >stop functioning
> >                     owner-nv-l@tkg
> >                     .com
> >
> >
> >                     11/07/2001
> >                     10:13 AM
> >                     Please respond
> >                     to IBM NetView
> >                     Discussion
> >
> >
> >
> >
> >
> >Stephen
> >
> >I am not sure what you are asking. My rule end in a action block. The
>rule
> >is written "look for trap xxxxx -->  Check Smartset X for node --> 
Action
> >(run script with no return code)
> >
> >Are you stating that i should be using the inline action?
> >
> >Thanks
> >
> > >From: "Stephen Hochstetler" <shochste@us.ibm.com>
> > >Reply-To: IBM NetView Discussion <nv-l@tkg.com>
> > >To: IBM NetView Discussion <nv-l@tkg.com>
> > >Subject: Re: [NV-L] Rules stop functioning
> > >Date: Wed, 7 Nov 2001 09:52:52 -0600
> > >
> > >Scott,
> > >
> > >Is this in-line action blocks or action blocks?      An action block
> >should
> > >be forked.    If you have any in-line, I would guess you are calling
> > >something that does not return.   Is your machine creating a lot of
> > >"zombie" processes which could impact new processes?    When a 
process
> >ends
> > >it stays around until an AIX cleanup routine runs that finds it and
>stops
> > >the process.     If you have something in your inittab that is not
>ending
> > >this keeps the AIX cleanup routine from working and all ended 
processes
> > >will still exist as zombies.   Eventually it will keep new processes
>from
> > >being forked....but first you will see a big performance problem.
> > >
> > >Kind regards,
> > >Stephen Hochstetler              shochste@us.ibm.com
> > >International Technical Support Organization  - Austin
> > >Office - 512-436-8564                      FAX - 512-436-8701
> > >
> > >ITSO redbooks at  http://www.redbooks.ibm.com
> > >
> >
> 
>_________________________________________________________________________
> > >NV-L List information and Archives: http://www.tkg.com/nv-l
> >
> >
> >_________________________________________________________________
> >Get your FREE download of MSN Explorer at 
>http://explorer.msn.com/intl.asp
> >
> 
>_________________________________________________________________________
> >NV-L List information and Archives: http://www.tkg.com/nv-l
> >
> >
> >
> 
>_________________________________________________________________________
> >NV-L List information and Archives: http://www.tkg.com/nv-l
>
>
>_________________________________________________________________
>Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
>
>_________________________________________________________________________
>NV-L List information and Archives: http://www.tkg.com/nv-l
>
>


_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp

_________________________________________________________________________
NV-L List information and Archives: http://www.tkg.com/nv-l

Scott -

With netstat, the number of lines is not an issue, but what they say is.
netstat shows you information about socket connections and on any function system you will have many.
If you have NetView then you will have many more. And if you run rulesets you will see two for each ruleset you are running. One is for the connection between whoever started the ruleset (usually either nvserverd or actionsvr) and one for nvcorrd's connection back to that process.

If the lines which mention nvcorrd do not show high numbers under Recv_Q or Send_Q then you do not have a socket backup problem. So then you should be looking at the nvcorrd.alog or blog after issuing the "nvcdebug -d all". You can follow what goes on thee by looking for the "Received a Trap" and "Finished with the trap" eyecatchers. What happens in between is nvcorrd processing that trap. When an action is sent to actionsvr, you will see "Action:" entries.
Without those the event did not result in an action being sent and the preceding trace should tell you why.

James Shanks
Level 3 Support for Tivoli NetView for UNIX and NT
Tivoli Software / IBM Software Group

"Scott Bursik" <tivoliesm@hotmail.com>
Sent by: owner-nv-l@tkg.com

11/07/2001 12:26 PM
Please respond to IBM NetView Discussion

To: nv-l@tkg.com
cc:
Subject: Re: [NV-L] Rules stop functioning -- some things to check

James,

When I run the netstat -a command I have 887 lines listed. Of those, about a
dozen have nvcorrd listed. Is that bad to have that many? I have started the
trace but do not see anything going in the nvaction.alog or blog as of yet.
I do have "BLOCK" listed as the choice for the "purple pizza". As far as the
name resolution goes...the node that was causing the error in the nvcorrd
lof file was deleted during the database cleanup last night and I have not
seen him come back yet.

Thanks James

>From: "James Shanks" <jshanks@us.ibm.com>
>Reply-To: IBM NetView Discussion <nv-l@tkg.com>
>To: IBM NetView Discussion <nv-l@tkg.com>
>Subject: Re: [NV-L] Rules stop functioning -- some things to check
>Date: Wed, 7 Nov 2001 12:09:01 -0500
>
>The way to determine whether or not your rules are working is to trace
>nvcorrd.
>This you do with the nvcdebug command . A full trace is achieve by
>issuing "nvcdebug -d all".
>Then you can look in the nvcorrd.alog and see what is happening. After
>1000 lines have been written there it will switch to the blog. And back
>gain after another 1000 lines.
>
>You said you were running a lot of rules out of ESE.automation.
>When you do "netstat -a" do you see backed up sockets for nvcorrd? These
>would be entries in the netstat list which show "nvcorrd" and have high
>numbers in the Send or Receive queues. The most you can have is 32,767
>but any relatively high number indicates a problem.
>If you see this, check the Initial node (the "purple pizza") of your
>rulesets running out of ESE.automation. Does it say PASS or BLOCK? Only
>BLOCK is allowed for a ruleset running in the background out of
>ESE.automation.
>
>
>I also presume that you saw my earlier append about name resolution. Your
> Query Smartset test will fail if the attribute resolves to an IP Address
>rather than hostname.
>
>James Shanks
>Level 3 Support for Tivoli NetView for UNIX and NT
>Tivoli Software / IBM Software Group
>
>
>
>
>
>
>"Scott Bursik" <tivoliesm@hotmail.com>
>Sent by: owner-nv-l@tkg.com
>11/07/2001 11:56 AM
>Please respond to IBM NetView Discussion
>
>
> To: nv-l@tkg.com
> cc:
> Subject: Re: [NV-L] Rules stop functioning
>
>
>
>Thanks for all of your help. I think I will have to open a PMR and I will
>post the resolution.
>
>Scott
>
>
>
> >From: "Stephen Hochstetler" <shochste@us.ibm.com>
> >Reply-To: IBM NetView Discussion <nv-l@tkg.com>
> >To: IBM NetView Discussion <nv-l@tkg.com>
> >Subject: Re: [NV-L] Rules stop functioning
> >Date: Wed, 7 Nov 2001 10:44:19 -0600
> >
> >Scott,
> >
> >Sorry to confuse you. I was wondering which you were doing, "in-line
> >action" or just "action". Since you are just doing "action", you
>should
> >NOT be having the problem that you see. The actionsvr forks the process
> >and rules continue. If you see actionsvr stop, then possibly it is
> >having troubles forking a process.
> >
> >Ideas
> > -- when you see this happen, are you low in space in /tmp or /usr/OV
>?
> >If you run out of space in "/" or "/tmp" or "/usr/OV" it is
>probably
> >related to your problem.
> >
> > -- if your machine is creating zombies, it is probably related to your
> >problem.
> >
> >If neither of these are true, you probably have an issue which is best
> >handled by calling support.
> >
> >Kind regards,
> >Stephen Hochstetler shochste@us.ibm.com
> >International Technical Support Organization - Austin
> >Office - 512-436-8564 FAX - 512-436-8701
> >
> >ITSO redbooks at http://www.redbooks.ibm.com
> >
> >
> >
> > "Scott Bursik"
> > <tivoliesm@hot To: nv-l@tkg.com
> > mail.com> cc:
> > Sent by: Subject: Re: [NV-L] Rules
> >stop functioning
> > owner-nv-l@tkg
> > .com
> >
> >
> > 11/07/2001
> > 10:13 AM
> > Please respond
> > to IBM NetView
> > Discussion
> >
> >
> >
> >
> >
> >Stephen
> >
> >I am not sure what you are asking. My rule end in a action block. The
>rule
> >is written "look for trap xxxxx --> Check Smartset X for node --> Action
> >(run script with no return code)
> >
> >Are you stating that i should be using the inline action?
> >
> >Thanks
> >
> > >From: "Stephen Hochstetler" <shochste@us.ibm.com>
> > >Reply-To: IBM NetView Discussion <nv-l@tkg.com>
> > >To: IBM NetView Discussion <nv-l@tkg.com>
> > >Subject: Re: [NV-L] Rules stop functioning
> > >Date: Wed, 7 Nov 2001 09:52:52 -0600
> > >
> > >Scott,
> > >
> > >Is this in-line action blocks or action blocks? An action block
> >should
> > >be forked. If you have any in-line, I would guess you are calling
> > >something that does not return. Is your machine creating a lot of
> > >"zombie" processes which could impact new processes? When a process
> >ends
> > >it stays around until an AIX cleanup routine runs that finds it and
>stops
> > >the process. If you have something in your inittab that is not
>ending
> > >this keeps the AIX cleanup routine from working and all ended processes
> > >will still exist as zombies. Eventually it will keep new processes
>from
> > >being forked....but first you will see a big performance problem.
> > >
> > >Kind regards,
> > >Stephen Hochstetler shochste@us.ibm.com
> > >International Technical Support Organization - Austin
> > >Office - 512-436-8564 FAX - 512-436-8701
> > >
> > >ITSO redbooks at http://www.redbooks.ibm.com
> > >
> >
> >_________________________________________________________________________
> > >NV-L List information and Archives: http://www.tkg.com/nv-l
> >
> >
> >_________________________________________________________________
> >Get your FREE download of MSN Explorer at
>http://explorer.msn.com/intl.asp
> >
> >_________________________________________________________________________
> >NV-L List information and Archives: http://www.tkg.com/nv-l
> >
> >
> >
> >_________________________________________________________________________
> >NV-L List information and Archives: http://www.tkg.com/nv-l
>
>
>_________________________________________________________________
>Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
>
>_________________________________________________________________________
>NV-L List information and Archives: http://www.tkg.com/nv-l
>
>

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp

_________________________________________________________________________
NV-L List information and Archives: http://www.tkg.com/nv-l

<Prev in Thread]	Current Thread	[Next in Thread>
Re: Rules stop functioning -- some things to check, James Shanks Re: Rules stop functioning -- some things to check, Scott Bursik Re: Rules stop functioning -- some things to check, James Shanks <= Re: Rules stop functioning -- some things to check, Scott Bursik

Previous by Date:	Re: Rules stop functioning -- some things to check, Scott Bursik
Next by Date:	Re: Doing SNMP queries on ports other than 161, James Shanks
Previous by Thread:	Re: Rules stop functioning -- some things to check, Scott Bursik
Next by Thread:	Re: Rules stop functioning -- some things to check, Scott Bursik
Indexes:	[Date] [Thread] [Top] [All Lists]