Qlik Sense Engine Enterprise on Windows: How the memory hard max limit works

[Bastien_Laugiero]

The Qlik Sense Engine allows for a hard max limit to be set on memory consumption, which, if enabled on the Operating System level as well, will make a best effort to remain below the set limit.

The setting is located in the Qlik Sense Management Console > Engine > Advanced and can be configured as an option in the setting Memory usage mode.

See Editing an engine - Qlik Sense for administrators for details on Engine settings. 

• In the drop down list you can choose Hard Max Limit which prevent the Engine to use more than a certain defined limit. 
• This limit is defined by the parameter set in percentage called Max memory usage (%)

This setting requires that the Operating System is configured to support this, as described in the SetProcessWorkingSetSizeEx documentation (QUOTA_LIMITS_HARDWS_MAX_ENABLE parameter).

Even with the hard limit set, it may still be possible for the host operating system to report memory spikes above the Max memory usage (%).

The reason for that is because the Qlik Sense Engine memory limit will be defined based on the total memory available.

Example:

• A server has 10 GB memory in total and the Max memory usage (%) is set to 90%.
• This will allow the process engine.exe to use 9 GB memory. 
• Another application or process may at that point already be consuming 5 GB memory, and this will cause an overload of the system if the engine is set to use 9 GB. 
• The engine.exe will not be able to respect other services

The memory working setting limit is not a hard limit to set on the engine. This is a setting that set how much we allocate and how far we are allowed to go before we start alarming on the working set beyond parameters. 

Internal Investigation IDs:

QLIK-96872

[QFabian]

Excellent @Bastien_Laugiero , thank you

[aadil_madarveet]

Thanks for the article.

Could you please explain what happens if the max memory usage is reached in a multi-node cluster. If the memory usage one node 1 is at max usage, will the resources of node 2 be used to maintain user sessions and performance. My question is basically, does Qlik manage resources spill over to next available node for the same app.

Thanks,
Aadil

[AdamBS]

Hi

Does the help indicate that this Hard max limit is basically void unless the O/S adjustment is done?

https://help.qlik.com/en-US/sense-admin/February2024/Subsystems/DeployAdministerQSE/Content/Sense_De...

As per @aadil_madarveet when the limit is reached, what is the behavior of the engine?  Does it just restart?

[Sonja_Bauernfeind]

Hello @AdamBS 

Thank you so much for the feedback, this helps us keep our articles up to date. I've added the note regarding the Windows setting, as, yes, it is required. 

As for the second question: If an app is opened at 90% usage on this node and hasn't otherwise been load balanced previously, what will happen is that the app will open while Qlik attempts to aggressively clean memory out to get back to the minimum value. This can still result in a memory overload, but does not have to. 

All the best,
Sonja 

[AdamBS]

Interesting.  Thanks for the update, Adam