I'm writing a java program to read & write files to CFS from a remote system (non DSE machine). As suggested in the DataStax Site, I wrote the following piece of code.
import java.net.URI;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.security.UserGroupInformation;
import com.datastax.bdp.hadoop.cfs.CassandraFileSystem;
public class CassandraFileHelper {
public static void main(String[] args) throws Exception {
FSDataOutputStream o = null;
CassandraFileSystem cfs = null;
String content = "some text content..";
try {
System.setProperty("cassandra.config", "conf/cassandra.yaml");
System.setProperty("dse.config", "conf/dse.yaml");
Configuration conf = new Configuration();
conf.addResource(new Path("conf/core-site.xml"));
UserGroupInformation.createUserForTesting("unixuserid", new String[] { "usergroupname" });
UserGroupInformation.setConfiguration(conf);
cfs = new CassandraFileSystem();
cfs.initialize(URI.create("cfs://hostname:9160/"), conf);
o = cfs.create(new Path("/folder/testfile.txt"), true);
o.write(content.getBytes());
o.flush();
} catch (Exception err) {
System.out.println("Error: " + err.toString());
} finally {
if (o != null)
o.close();
if (cfs != null)
cfs.close();
}
}
}
I've included the listed configuration files and jar from DSE package.
- cassandra.yaml
- core-site.xml
- dse.yaml
- cassandra-all-1.0.10.jar
- cassandra-clientutil-1.0.10.jar
- cassandra-thrift-1.0.10.jar
- commons-cli-1.1.jar
- commons-codec-1.2.jar
- commons-configuration-1.6.jar
- commons-lang-2.4.jar
- commons-logging-1.1.1.jar
- compress-lzf-0.8.4.jar
- dse.jar
- guava-r08.jar
- hadoop-core-1.0.2-dse-20120707.200359-5.jar
- libthrift-0.6.1.jar
- log4j-1.2.16.jar
- slf4j-api-1.6.1.jar
- snakeyaml-1.6.jar
- snappy-java-1.0.4.1.jar
- speed4j-0.9.jar
When I run the program, I get the following error
org.apache.thrift.TApplicationException: Internal error processing batch_mutate
I copied all the config files from a DSE machine and when I added them I get the following error.
Cannot locate conf/cassandra.yaml
Fatal configuration error; unable to start server. See log for stacktrace.
Could anyone please validate my approach and let me know whether this is possible?
Thanks.
