I'm trying to set up Opscenter for a 5-node Cassandra cluster.
Opscenter is set up on a separate linux box.
When the agent is started, I'm seeing connection problems:
[bwong@cassa001 bin]$ ./opscenter-agent -f
log4j:WARN No appenders could be found for logger (org.eclipse.jetty.util.log).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
INFO [main] 2013-08-21 19:00:48,955 Loading conf files: ./conf/address.yaml
INFO [main] 2013-08-21 19:00:49,019 Java vendor/version: OpenJDK 64-Bit Server VM/1.6.0_24
INFO [main] 2013-08-21 19:00:49,019 Waiting for the config from OpsCenter
INFO [main] 2013-08-21 19:00:49,020 SSL communication is disabled
INFO [main] 2013-08-21 19:00:49,020 Creating stomp connection to 172.30.119.67:61620
INFO [StompConnection receiver] 2013-08-21 19:00:49,027 Reconnecting in 0s.
INFO [main] 2013-08-21 19:00:49,029 Attempting to determine Cassandra's broadcast address through JMX
INFO [Initialization] 2013-08-21 19:00:49,034 Sleeping for 2s before trying to determine IP over JMX again
INFO [StompConnection receiver] 2013-08-21 19:00:49,036 Connected to 172.30.119.67:61620
ERROR [clojure-agent-send-off-pool-0] 2013-08-21 19:00:49,044 failed sending message to /conf-request:
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at java.io.DataOutputStream.write(DataOutputStream.java:107)
at java.io.FilterOutputStream.write(FilterOutputStream.java:97)
at org.jgroups.client.StompConnection.send(StompConnection.java:200)
at org.jgroups.client.StompConnection.send(StompConnection.java:219)
at org.jgroups.client.StompConnection.send(StompConnection.java:223)
at opsagent.messaging$actually_send_message.invoke(messaging.clj:66)
at clojure.core$binding_conveyor_fn$fn__4107.invoke(core.clj:1845)
at clojure.lang.AFn.applyToHelper(AFn.java:167)
at clojure.lang.RestFn.applyTo(RestFn.java:132)
at clojure.lang.Agent$Action.doRun(Agent.java:114)
at clojure.lang.Agent$Action.run(Agent.java:163)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
ERROR [clojure-agent-send-off-pool-1] 2013-08-21 19:00:49,048 failed sending message to /conf-request:
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at java.io.DataOutputStream.write(DataOutputStream.java:107)
at java.io.FilterOutputStream.write(FilterOutputStream.java:97)
at org.jgroups.client.StompConnection.send(StompConnection.java:200)
at org.jgroups.client.StompConnection.send(StompConnection.java:219)
at org.jgroups.client.StompConnection.send(StompConnection.java:223)
at opsagent.messaging$actually_send_message.invoke(messaging.clj:66)
at clojure.core$binding_conveyor_fn$fn__4107.invoke(core.clj:1845)
at clojure.lang.AFn.applyToHelper(AFn.java:167)
at clojure.lang.RestFn.applyTo(RestFn.java:132)
at clojure.lang.Agent$Action.doRun(Agent.java:114)
at clojure.lang.Agent$Action.run(Agent.java:163)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
INFO [main] 2013-08-21 19:00:49,058 Starting Jetty server: {:port 61621, :host nil, :ssl? false, :join? false}
ERROR [StompConnection receiver] 2013-08-21 19:00:49,063 Connection closed unexpectedly:
java.io.EOFException: reading verb
at org.jgroups.protocols.STOMP.readFrame(STOMP.java:244)
at org.jgroups.client.StompConnection.run(StompConnection.java:253)
at java.lang.Thread.run(Thread.java:679)
INFO [StompConnection receiver] 2013-08-21 19:00:49,064 Reconnecting in 2s.
INFO [Jetty] 2013-08-21 19:00:49,126 Jetty server started
INFO [Initialization] 2013-08-21 19:00:51,035 Sleeping for 4s before trying to determine IP over JMX again
WARN [clojure-agent-send-off-pool-1] 2013-08-21 19:00:51,035 Tried to send message while not connected: /conf-request [["fe80:0:0:0:a6ba:dbff:fe1e:59%2","172.29.50.27","0:0:0:0:0:0:0:1%1","127.0.0.1"],["3.2.1","\/812094383\/conf"]]
INFO [StompConnection receiver] 2013-08-21 19:00:51,065 Connected to 172.30.119.67:61620
ERROR [StompConnection receiver] 2013-08-21 19:00:51,066 Connection closed unexpectedly:
java.io.EOFException: reading verb
at org.jgroups.protocols.STOMP.readFrame(STOMP.java:244)
at org.jgroups.client.StompConnection.run(StompConnection.java:253)
at java.lang.Thread.run(Thread.java:679)
INFO [StompConnection receiver] 2013-08-21 19:00:51,067 Reconnecting in 6s.
ERROR [clojure-agent-send-off-pool-1] 2013-08-21 19:00:51,067 failed sending message to /conf-request:
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at java.io.DataOutputStream.write(DataOutputStream.java:107)
at java.io.FilterOutputStream.write(FilterOutputStream.java:97)
at org.jgroups.client.StompConnection.send(StompConnection.java:200)
at org.jgroups.client.StompConnection.send(StompConnection.java:219)
at org.jgroups.client.StompConnection.send(StompConnection.java:223)
at opsagent.messaging$actually_send_message.invoke(messaging.clj:66)
at clojure.core$binding_conveyor_fn$fn__4107.invoke(core.clj:1845)
at clojure.lang.AFn.applyToHelper(AFn.java:167)
at clojure.lang.RestFn.applyTo(RestFn.java:132)
at clojure.lang.Agent$Action.doRun(Agent.java:114)
at clojure.lang.Agent$Action.run(Agent.java:163)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
INFO [Initialization] 2013-08-21 19:00:55,035 Sleeping for 8s before trying to determine IP over JMX again
WARN [clojure-agent-send-off-pool-1] 2013-08-21 19:00:55,036 Tried to send message while not connected: /conf-request [["fe80:0:0:0:a6ba:dbff:fe1e:59%2","172.29.50.27","0:0:0:0:0:0:0:1%1","127.0.0.1"],["3.2.1","\/812094383\/conf"]]