We are migrating our DSE 2.1.1 cluster to DSE 3.0, we have followed the instruction given in the datastax documentation pages:
http://www.datastax.com/docs/datastax_enterprise3.0/upgrade/dse_upgrade
http://www.datastax.com/docs/datastax_enterprise3.0/upgrade/solr_upgrade
We are encountering an issue with our existing indexes that appear to have columns that are specified as datetime types but are set to the value of a empty string. When we run Solr queries, we get errors if any of the records in the results have these empty datetime fields in the result set.
This problem did not exist in DSE 2.1.1 with identical Solr queries.
How can we update our existing data (10's of millions of records with some invalid value for a datetime field) without causing query errors? Is it possible to lose the validation for datetime parsing in Solr configuration?
The following is an response with error for a Solr query:
<response>
<lst name="error">
<str name="msg">java.text.ParseException: Unparseable date: ""</str>
<str name="trace">
java.io.IOException: java.text.ParseException: Unparseable date: "" at com.datastax.bdp.cassandra.index.solr.CassandraDocumentReader.processValue(CassandraDocumentReader.java:462) at com.datastax.bdp.cassandra.index.solr.CassandraDocumentReader.field(CassandraDocumentReader.java:416) at com.datastax.bdp.cassandra.index.solr.CassandraDocumentReader.field(CassandraDocumentReader.java:353) at com.datastax.bdp.cassandra.index.solr.CassandraDocumentReader.loadDocument(CassandraDocumentReader.java:267) at com.datastax.bdp.cassandra.index.solr.CassandraDocumentReader.document(CassandraDocumentReader.java:99) at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:169) at org.apache.lucene.index.BaseCompositeReader.document(BaseCompositeReader.java:116) at org.apache.lucene.index.IndexReader.document(IndexReader.java:436) at org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:547) at org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:270) at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:177) at org.apache.solr.response.XMLWriter.writeNamedList(XMLWriter.java:227) at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:188) at org.apache.solr.response.XMLWriter.writeArray(XMLWriter.java:273) at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:190) at org.apache.solr.response.XMLWriter.writeNamedList(XMLWriter.java:227) at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:188) at org.apache.solr.response.XMLWriter.writeNamedList(XMLWriter.java:227) at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:188) at org.apache.solr.response.XMLWriter.writeResponse(XMLWriter.java:111) at org.apache.solr.response.XMLResponseWriter.write(XMLResponseWriter.java:37) at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:418) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:289) at com.datastax.bdp.cassandra.index.solr.CassandraDispatchFilter.doFilter(CassandraDispatchFilter.java:134) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.datastax.bdp.cassandra.audit.SolrHttpAuditLogFilter.doFilter(SolrHttpAuditLogFilter.java:194) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.datastax.bdp.cassandra.index.solr.auth.CassandraAuthorizationFilter.doFilter(CassandraAuthorizationFilter.java:94) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.datastax.bdp.cassandra.index.solr.auth.DseAuthenticationFilter.doFilter(DseAuthenticationFilter.java:102) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619) Caused by: java.text.ParseException: Unparseable date: "" at java.text.DateFormat.parse(DateFormat.java:337) at org.apache.solr.schema.DateField.parseDate(DateField.java:284) at com.datastax.bdp.cassandra.index.solr.CassandraDocumentReader.processValue(CassandraDocumentReader.java:457) ... 44 more
</str>
<int name="code">500</int>
</lst>
</response>