How much data can one QVW hold?

[matthewjbryant]

Hi All,

I have two QVWs running with about 35 users. Both QVWs hold in excess of 3 million rows; one is 50 MB, the other 58 MB. The way I have set these documents up means that the users don't have any issues using them (apart from them being a little slow to populate), but if I accidently create a table box with all data then I need to reboot my PC.

If I limit the load on the same QVWs they run really fast, so this got me thinking: when is a QVW too large?

Do you have a limit that you stick to?

Is there a certain number of rows that once you pass you don't see any further loss of performance?

[marcus_sommer]

There are (nearly) none limitations by qlikview especially not by those rather small amounts of data which you mentioned. The reason will be within your version 32-bit vs 64-bit, the available hardware-ressources (especially ram) and some settings from the working-set: in short I thin you have not enough ram - see within the task-manager.

- Marcus

[sunny_talwar]

I am not an expert, but I don't think there is a limit. As the size of your application increases, it will seek more resources (Memory) from your system and if you are able to give those resources the application will give similar performance as if you were running with less data and less resources.

Others may have better answer, but this what I happen to know

Best,

Sunny

[matthewjbryant]

Thanks for the reply. RAM is definitely my issue. I have 8GB, but it spikes when I create a table box for these rows. So really, RAM is going to always be a limiting factor.

[jonathandienst · Accepted solution]

The size of the qvw is limited by the amount of available RAM and the number of CPUs determined the speed of calculation. So there is no special limit on the size - rather the practical limitations such as the server RAM and CPUs, the number of users (because this increases the RAM and the CPU demands), the time it takes to load (if it cant be preloaded) and the time it takes to reload.

Having said that, 3 million rows is not very large, but your PC may not have the grunt to recalculate the table box - especially as it is probably having to do a number of joins to calculate the table box.

But how do you do that accidentally?

[sunny_talwar]

But how do you do that accidentally?

[matthewjbryant]

[matthewjbryant]

Neither of these QVWs contain many joins (one is a lot of concatenated data - bad, I know - the other is pretty much one table of 5 years of very detailed information from our database). Is this the issue?

I do this accidentally by creating a tablebox while selections are active and then stupidly clearing those selections!

[Anonymous]

Moral of the story:  Do not "accidentally create a table box with all data"

To prevent it on the General tab of your Table object add a Calculation Condition, maybe something like :

     GetPossibleCount ( [YourField] ) < 999

[marcus_sommer]

To concatenate tables isn't bad. Yes you will need more ram with these datamodel approach as with a link-table model but the calculations within the gui are mostly quicker. The joins mentioned from jontydkpi are more related to the efforts to create these tablebox. If qv need to hop through many tables for this it will need time and ram.

But normally it should be possibly to show a tablebox with 3 M rows by 8GB ram if you not include all fields. I assume there are many things which could be optimized within your apps - have a look here, what is meant:

The Optimal QlikView Data Structure

The Importance Of Being Distinct

Major Update to Document Analyzer | Qlikview Cookbook

- Marcus