Hi,
I deploy a cassandra in a EC2 instance. I use 1.2.3 cassandra. I not found any response.
1) I have a very big Keyspace (among 10 ColumnFamily, lot of write), I want manage a second keyspace (ClientWorld), small, few write.
2)
> cassandra-cli
CREATE KEYSPACE ClientWorld;
USE ClientWorld;
CREATE COLUMN FAMILY ClientMeta with comparator=UTF8Type and key_validation_class=UTF8Type;
CREATE COLUMN FAMILY ClientMeta2 with comparator=UTF8Type and key_validation_class=UTF8Type;
SET ClientMeta[utf8('0')][utf8('username')]=utf8('toto');
SET ClientMeta2[utf8('0')][utf8('username')]=utf8('toto');
3)
> cassandra-cli
USE ClientWorld;
[default@ClientWorld] GET ClientMeta[utf8('0')];
=> (column=username, value=746f746f, timestamp=1366303494180000)
Returned 1 results.
Elapsed time: 1.32 msec(s).
[default@ClientWorld] GET ClientMeta2[utf8('0')];
=> (column=username, value=746f746f, timestamp=1366303494206000)
Returned 1 results.
Elapsed time: 1.85 msec(s).
4)
> nodetool snapshot ClientWorld
5)
> cassandra-cli
USE ClientWorld;
[default@ClientWorld] GET ClientMeta[utf8('0')];
null
TimedOutException()
at org.apache.cassandra.thrift.Cassandra$get_slice_result.read(Cassandra.java:7874)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at org.apache.cassandra.thrift.Cassandra$Client.recv_get_slice(Cassandra.java:594)
at org.apache.cassandra.thrift.Cassandra$Client.get_slice(Cassandra.java:578)
at org.apache.cassandra.cli.CliClient.doSlice(CliClient.java:548)
at org.apache.cassandra.cli.CliClient.executeGet(CliClient.java:684)
at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:216)
at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:210)
at org.apache.cassandra.cli.CliMain.main(CliMain.java:337)
[default@ClientWorld] GET ClientMeta2[utf8('0')];
=> (column=username, value=746f746f, timestamp=1366303494206000)
Returned 1 results.
Elapsed time: 1.97 msec(s).
6) Cassandra logs:
java.lang.RuntimeException: java.lang.IllegalArgumentException: unable to seek to position 3188 in /mnt/md0/cassandra/data/ClientWorld/ClientMeta/ClientWorld-ClientMeta-ib-1-Data.db (54 bytes) in read-only mode
at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1582)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.IllegalArgumentException: unable to seek to position 3188 in /mnt/md0/cassandra/data/ClientWorld/ClientMeta/ClientWorld-ClientMeta-ib-1-Data.db (54 bytes) in read-only mode
at org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:306)
at org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:42)
at org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:978)
at org.apache.cassandra.db.columniterator.SimpleSliceReader.<init>(SimpleSliceReader.java:60)
at org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:68)
at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:44)
at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:101)
at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:68)
at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:275)
at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65)
at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1363)
at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1220)
at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1132)
at org.apache.cassandra.db.Table.getRow(Table.java:348)
at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:70)
at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1052)
at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1578)
... 3 more
7) If I run
> nodetool compact ClientWorld ClientMeta
, all becomes allright.
8)
> cassandra-cli
USE ClientWorld;
[default@ClientWorld] GET ClientMeta[utf8('0')];
=> (column=username, value=746f746f, timestamp=1366303494180000)
Returned 1 results.
Elapsed time: 2.03 msec(s).
[default@ClientWorld] GET ClientMeta2[utf8('0')];
=> (column=username, value=746f746f, timestamp=1366303494206000)
Returned 1 results.
Elapsed time: 1.45 msec(s).
9) I can reproduce this on my production database but not on my local cassandra.
Thank you for any help.