FIrstly, you need to separate Qlikview Server and Publisher; they have separate behaviors although they work together.
If something is allocating and keeping memory (but not actively using it at the moment), that's Qlikview Server (QVS.exe) caching user behavior, aggregated data like charts, tables and so on from your QVW documents - this is expected behavior.
Publisher (QVB.exe) use memory during reloads, and then release it back to the system. Publisher does not cache anything between sessions and spawn new processes (Execution services) for each new job/task, depending on the configuration.
The number of executions services that you can use simultaneously is license bound, and generally shows in the licence LEF in the QEMC for Publisher, it's not bound to a patch or a specific version of Qlikview components.
Stefan, thanks for the clarification. i didnt mean to mix things up. i should have been more clear.
The problem that we are facing is the memory usage keeps on growing up until the serves stops responding as it should (QVS.exe) We know that this is expected but the problem is that eventhough the server has memory available, the qvs.exe uses virtual memory increasingly.
Concernign the QV engines, according to our consultant, this issue is not associated to the license. We use to think that this matter was associated to the CPU's available in the sever, in the past we had 8 so we new that number of CPU's - 1 was the amount of qV engines (or jobs) could be running simultaneously. A few months ago we made a server upgrade with 24 CPU, then we realized that we werent able to run more than 9 jobs at the same time.
The consultant requested more information about this matter and they found out that 9 was the limit.
Actually, I need to revise my answer, since I've talked to some resources internally.
The Distribution Service is the only part which instances are regulated by any limit set by Qlikview - and that limit is set by the Publisher license information. And each additional Distribution service is equivalent with an additional server for Publisher, basically.
However, each Distribution service may theoretically spawn as many QVB's (let's call them "QV Batches") as it likes on a machine, and run at the same time. BUT, since for some reason Microsoft COM has trouble handling 10 or more of these QVB's (since COM is the backend for the communication from/to them), we don't recommend using that many per machine. In fact, if you try, all QVB's initiated after the 9th or 10th will fail.
You can limit the number of concurrent QVB's running with the "Max number of simultaneous QlikView engines for distribution" setting in QEMC > Distribution Services > QDS@machine > Advanced. This will queue all QVB's that tries to start when this number of concurrent QVB's are running.
I hope that answers your questions.
Lars or Stefan,
Is that Microsoft COM limitation (to the number of simultaneous qv engines) per server or per Execution service or per Publisher? We have 4 XS, one on each of our 4 identical servers. Are we limited to 10 qv engines or 40?
(We've been experieincing issues with jobs repeating themselves across XS. See http://community.qlik.com/forums/t/33239.aspx)
The strand started with concerns over use of RAM. I have 12 gig of RAM and the error message I am seeing in the logs suggest that the service is only using 2 gig of VM. The logs give a warnig when using our larget app of 250 Mb on a qlikview plugin saying working set nearing limits. The set is 75 to 90 with 10 5 cache and it says 1.49 gig is nearing the limit.
Really need a pointer here as the task manager ays that there is 10.5 gig of actual RAM not being used?
Following the above a disconnection occurs and the PF usgae drops back down
You'll have to be more precise when it comes to the QVS memory usage. QVS is set to avoid using virtual memory (as in Page file) as much as possible and will go to much extent to avoid it. If you're having virtual memory usage on the machine (that's more than the usual amounts that Windows itself uses for regular operations), then you have a physical memory shortage and will most probably need more memory in the machine.
Regarding the QVB engines (reload engines in Publisher); I'm not aware of any limit at 9 instances specifically. Please contact email@example.com for more help with this, if it is not license bound.
I've also had some (similar?) problems:I have recently found an issue with a single CPU server at a client site where a model refresh task (10-15 files x 5-10mb in size) did not complete.After some testing I found that when running .cmd files which are linked to .vbs calling the distribution service, that the following occured:
A) Batch task containing ODBC data loaders worked fine (30 x 100kb qvw's)
B) Batch task containing Non-ODBC 8 x 100kb qvw's worked fine
C) As soon as a 900kb .qvw was added to the above refresh task this model would not reload
Other factors to consider are:
1. All batch tasks (regardless of .qvw file size) works fine when executed manually in Windows Explorer but not through the QVS?
2. Multiple QVB's are spawned at one time - causing problems for the one CPU - I did not have the option to change the number of QVB's spawned as suggested in this thread at:
QEMC > Distribution Services > QDS@machine > Advanced.
Must the server be enterprise-level to change this setting?
What techniques can be used to overcome this issue without adding further CPU's?
We regularly run 16 simultaneous reloads without any problem. You have to change the Desktop Heap settings to go past 9 or 10. See below, which was sent to us from QlikTech support.
Currently we're trying to use even more cores, but can't seem to go much past 16. Supposedly this is a COM/MS limit, but I don't have the original source for this information.
There is a workaround to get you past the limit of 9 engines for distribution and since you are running mostly EDX tasks this might be worth trying. Then you can set the number to 16 or less.
**Please note this is a Microsoft function not supported by QlikView**
There is a workaround and you can employ this at your own risk as it involves changing registry settings for the OS:
There are some registry settings you can change to increase the number of com communication threads allowed per machine.
Change the registry key and restart the server
ComException error 80080005 Solution
Change the registry setting:
SharedSection=1024,3072,512 Windows=On SubSystemType=Windows
(Default is 1024,3072,512 in 32bit or 1024,3072,768 in x64)
Change the GDI and User handle max count in the registry to SharedSection=1024,20480,2048
SharedSection=1024,20480,2048 Windows=On SubSystemType=Windows
Read more about Desktop Heap on:
Qliktech support has recommended me to make this changes too. But i'm wondering when to do it. Because i'm getting new servers, so should i make this change:
- Before installing OS or after installing OS?
- Before installing Qlikview or after?
In other hand, i'm wondering what is the meaning of this parameter MaxRequestThreads=16? Is that the number of cores?? should i change it if i have for example 32 cores?
I'm facing some issues concerning the memory usage of QlikView publisher Enterprise. We have been monitoring our server and the results show that even if we have physical memory available, QlikView uses virtual memory without releasing it. Does anyone know how to manage this?
On the other hand, we know that Qlikview server/publisher Enterprise has a limitation in terms of how many QlikView engines can be created simultaneoulsy, we use to think that this matter was associated to the amount of CPU available in ther sever, but after updating the server we now know that we can only have 9 qlikview engines running at the same time, does anyone know if this will continue or if there is a patch that can enlarge this number?
Thanks to any recomendations.