Resolve the “Memory limit exceeded” issue in IMPALA

“Memory limit exceeded” usually, happens When the query reached its max memory limit and is unable to allocate any more memory

ERROR :

Memory limit exceeded: Failed to allocate row batch EXCHANGE_NODE (id=897) could not allocate 11.00 KB without exceeding limit. Error occurred on backend server :22000
Memory left in process limit: 231.43 GB
Memory left in query limit: -112.25 KB

(Similar issue with respect to memory and the resolution would be the same)

memory limit exceeded. Limit=256.00 MB Consumption=293.21 MB

Troubleshooting:

• We need to check the Impala query profile and Coordinator daemon logs to find where exactly it failed with the Out of Memory issue
• In this case, it failed in the “EXCHANGE_NODE” step, Where it is unable to allocate any more memory to that query, But the impala process still has enough memory left in that node

Memory left in process limit: 231.43 GB

Memory left in query limit: -112.25 KB

• This shows that the Impala process still has ~231GB left and can be allocated to the query limit

Impact:

This will cause Impala query failure

Resolution:

To resolve this problem, We need to increase the mem_limit for that query or to the impala queue (default mem_limit)

 set MEM_LIMIT=
 

Additional points to be noted:

• Make sure you have proper table stats available for all the tables involved in the query else run the below command before the actual query

Compute stats

• Gathering stats Would help Impala to create a more optimized query plan, which results in better performance.

Good Luck with your Learning !!