Quantcast
Viewing all articles
Browse latest Browse all 387

tambalavanar on "Java code to perform CFS file operations from remote system is not working"

I'm writing a java program to read & write files to CFS from a remote system (non DSE machine). As suggested in the DataStax Site, I wrote the following piece of code.


import java.net.URI;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.security.UserGroupInformation;

import com.datastax.bdp.hadoop.cfs.CassandraFileSystem;

public class CassandraFileHelper {

public static void main(String[] args) throws Exception {

FSDataOutputStream o = null;
CassandraFileSystem cfs = null;
String content = "some text content..";

try {
System.setProperty("cassandra.config", "conf/cassandra.yaml");
System.setProperty("dse.config", "conf/dse.yaml");

Configuration conf = new Configuration();
conf.addResource(new Path("conf/core-site.xml"));

UserGroupInformation.createUserForTesting("unixuserid", new String[] { "usergroupname" });
UserGroupInformation.setConfiguration(conf);

cfs = new CassandraFileSystem();
cfs.initialize(URI.create("cfs://hostname:9160/"), conf);

o = cfs.create(new Path("/folder/testfile.txt"), true);
o.write(content.getBytes());
o.flush();

} catch (Exception err) {
System.out.println("Error: " + err.toString());
} finally {
if (o != null)
o.close();
if (cfs != null)
cfs.close();
}
}
}


I've included the listed configuration files and jar from DSE package.

  • cassandra.yaml
  • core-site.xml
  • dse.yaml
  • cassandra-all-1.0.10.jar
  • cassandra-clientutil-1.0.10.jar
  • cassandra-thrift-1.0.10.jar
  • commons-cli-1.1.jar
  • commons-codec-1.2.jar
  • commons-configuration-1.6.jar
  • commons-lang-2.4.jar
  • commons-logging-1.1.1.jar
  • compress-lzf-0.8.4.jar
  • dse.jar
  • guava-r08.jar
  • hadoop-core-1.0.2-dse-20120707.200359-5.jar
  • libthrift-0.6.1.jar
  • log4j-1.2.16.jar
  • slf4j-api-1.6.1.jar
  • snakeyaml-1.6.jar
  • snappy-java-1.0.4.1.jar
  • speed4j-0.9.jar

When I run the program, I get the following error

org.apache.thrift.TApplicationException: Internal error processing batch_mutate

I copied all the config files from a DSE machine and when I added them I get the following error.

Cannot locate conf/cassandra.yaml
Fatal configuration error; unable to start server. See log for stacktrace.

Could anyone please validate my approach and let me know whether this is possible?
Thanks.


Viewing all articles
Browse latest Browse all 387

Trending Articles