someone PLEASE throw me a bone...
Our customer is reporting large delays (up to 4 hours) of events arriving in
their control desks. I can force a trap to be sent from a network device
and it hits /usr/OV/log/trapd.log with low delay (<1 sec). The event shows
up delayed (10 sec< delay <4 hours) in the control desk. The only rule they
have implemented is forwardall.rls. There are approx 4 ovws running each w/
multiple dynamic workspaces.
netstat -a |grep nvcorrd shows the following:
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp 0 0 loopback.nvcorrd loopback.1339
tcp 156 0 loopback.1339 loopback.nvcorrd
.
.
.
.
There are several of these socket pairs each with a receive queue listed for
the numbered socket. lsof shows these socket numbers are nvserverd
processes. These bytes in queue are VERY slow to clear out. Bursts of
traps cause large numbers (~31k) and queuing delays.
Has anyone seen this behavior? What could be wrong. Any hints, ideas,
suggestions would be greatly appreciated.
--Bryan Brook
|