Quantcast
Viewing all articles
Browse latest Browse all 387

lin_qiang on "Why the DataStax Hive service deadlock when a complex query was executed?"

I installed a DataStax Enterprise. It worked fine.

sudo service dse start
dse hive --service hiveserver

The most query I commited to Hive service can correctly execute. But when I send a complex query to server, the Hive service may deadlock without any error message.

SELECT
MAX(LINEITEM.Column_2_3) Column_1_1,
SUM(LINEITEM.Column_2_1) Column_1_2,
YEAR(LINEITEM.Column_2_2) Column_1_3
FROM (SELECT
LINEITEM.Column_3_1Column_2_1,
LINEITEM.Column_3_2Column_2_2,
LINEITEM.Column_3_3Column_2_3,
1 Column_2_4
FROM (SELECT
LINEITEM.l_extendedpriceColumn_3_1,
LINEITEM.l_shipdateColumn_3_2,
COALESCE(YEAR(LINEITEM.l_shipdate) IN (1995, 1996), FALSE) Column_3_3
FROM LINEITEM
) LINEITEM) LINEITEM
GROUP BY YEAR(LINEITEM.Column_2_2)

The Last log information is:

Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>

But I execute that SQL in Hive console, that is worked fine.

After the Hive service deadlock, all of request will be block. I must stop the service (press Ctrl+C) and restart it.

How should I resolve this issue?


Viewing all articles
Browse latest Browse all 387

Trending Articles