Hi,
I am managing a cluster with two datacenters (each having atleast 5 nodes). I added the seed nodes to start managing from the ops center. All nodes have Ubuntu setup on them with all the relevant port opened for communication. Opscenter agent is running on all cluser nodes. Opscenter daemon is running on a separate host (not a part of the cluster). When I click on Fix to install agents, it pops up a window, I enter node's credentials and click on install on all nodes. This operation succeeds. But the dashboard is still empty and unable to fetch information from the agents.
Following is the log from Opscenter:
2013-06-12 15:11:32-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.17.186.50:61621/cluster/topology?node_ip=10.137.7.172 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: HTTP request http://10.137.7.172:61621/cluster/topology?node_ip=10.17.186.50 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.137.7.172:61621/cluster/topology?node_ip=10.17.186.50 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: HTTP request http://10.137.7.151:61621/cluster/topology?node_ip=10.137.7.154 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.137.7.151:61621/cluster/topology?node_ip=10.137.7.154 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: HTTP request http://10.137.7.171:61621/cluster/topology?node_ip=10.137.7.181 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.137.7.171:61621/cluster/topology?node_ip=10.137.7.181 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: HTTP request http://10.137.7.151:61621/cluster/topology?node_ip=10.137.7.151 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.137.7.151:61621/cluster/topology?node_ip=10.137.7.151 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: HTTP request http://10.137.7.151:61621/cluster/topology?node_ip=10.17.186.55 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.137.7.151:61621/cluster/topology?node_ip=10.17.186.55 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: HTTP request http://10.137.7.154:61621/cluster/topology?node_ip=10.17.186.21 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.137.7.154:61621/cluster/topology?node_ip=10.17.186.21 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: HTTP request http://10.17.186.21:61621/cluster/topology?node_ip=10.17.186.59 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.17.186.21:61621/cluster/topology?node_ip=10.17.186.59 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: HTTP request http://10.17.186.59:61621/cluster/topology?node_ip=10.17.186.27 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.17.186.59:61621/cluster/topology?node_ip=10.17.186.27 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: HTTP request http://10.17.186.50:61621/cluster/topology?node_ip=10.137.7.171 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.17.186.50:61621/cluster/topology?node_ip=10.137.7.171 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: HTTP request http://10.17.186.55:61621/cluster/topology?node_ip=10.137.7.172 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:12:11-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.17.186.55:61621/cluster/topology?node_ip=10.137.7.172 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: HTTP request http://10.137.7.151:61621/cluster/topology?node_ip=10.17.186.21 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.137.7.151:61621/cluster/topology?node_ip=10.17.186.21 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: HTTP request http://10.137.7.172:61621/cluster/topology?node_ip=10.17.186.59 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.137.7.172:61621/cluster/topology?node_ip=10.17.186.59 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: HTTP request http://10.137.7.171:61621/cluster/topology?node_ip=10.17.186.27 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.137.7.171:61621/cluster/topology?node_ip=10.17.186.27 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: HTTP request http://10.137.7.151:61621/cluster/topology?node_ip=10.17.186.55 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.137.7.151:61621/cluster/topology?node_ip=10.17.186.55 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: HTTP request http://10.137.7.181:61621/cluster/topology?node_ip=10.137.7.151 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.137.7.181:61621/cluster/topology?node_ip=10.137.7.151 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: HTTP request http://10.137.7.151:61621/cluster/topology?node_ip=10.137.7.171 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.137.7.151:61621/cluster/topology?node_ip=10.137.7.171 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: HTTP request http://10.17.186.59:61621/cluster/topology?node_ip=10.137.7.172 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.17.186.59:61621/cluster/topology?node_ip=10.137.7.172 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: HTTP request http://10.17.186.50:61621/cluster/topology?node_ip=10.137.7.154 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.17.186.50:61621/cluster/topology?node_ip=10.137.7.154 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: HTTP request http://10.17.186.59:61621/cluster/topology?node_ip=10.137.7.181 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:00-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.17.186.59:61621/cluster/topology?node_ip=10.137.7.181 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:01-0700 [SRMTestCluster01] WARN: HTTP request http://10.17.186.27:61621/cluster/topology?node_ip=10.17.186.50 failed: [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
2013-06-12 15:13:01-0700 [SRMTestCluster01] WARN: Unable to collect datacenter, rack information: Failed query to http://10.17.186.27:61621/cluster/topology?node_ip=10.17.186.50 : [<twisted.python.failure.Failure <class 'twisted.internet.error.ConnectionDone'>>]
Any help and direction on this is appreciated.
Thanks,
Chethan