Incremental load

[#1]

Hi All,

I have 5 fields A, B, C, D, E in that fields time stamp will not be available. First time i have 100 records for this i used full load but next time 50 records are added. Now i have to do incremental load without time field.

Could you please anyone give me some suggestion on this with detailed descrption.

Thanks,

Vamsi

[ashfaq_haseeb]

Hi

If you have primary key use that instead.

Regards

ASHFAQ

[rohan_mulay · Accepted solution]

Hello Reddy,

If you have an ID field u can take max(ID). Also, u can try using rowno() to get the previously loaded last record.

refer following code:

Master:

ID,

Name,

Description,

RowVersion

from Master.qvd;

Max:

load max(ID) as maxID

resident Master;

Let vMax = peek('maxID',0,'Max')

Drop table Max;

Transaction:

join(Master)

Load

ID,

Name,

Description,

RowVersion

From Transaction.qvd

where ID >'$(vMax)';

[raghvendrasingh]

Hi VV Reddy,

Please Find below qvw for Incremental load.hope this will help u.

Regards,

Raghvendra singh

[Anonymous]

Hi v v Reddy,

you can apply incremental load by using the id or primary key of the table , suppose we conside the id then

firstly you load the table by using order by id and make qvd then by using the peek ('id',-1,'tableName') you can get the last value of the table,

let vLastvalue = peek ('id',-1,'tableName')

set vUpdated_value = $(vLastvalue )

load the table where  vUpdated_value >= id

then concatenate this data with the old table

and then store it into the same qvd .

Hope this would be help you.

[#6]

Hi all,

Thanks for suggestions. Present i'm using Primary key and working fine.

Thanks,

VV

[stevedark]

Just to comment on the correct answer here, finding the Max ID from a QVD actually takes quite a chunk of time (as a full non optimised load is required).

If you are loading from a SQL based data source I would suggest using the database to get the max value and then persist that to a separate QVD for quick access next time.

Also, if all the columns in two loads are identical, and you want to add one set of values to another you will want to be using a CONCATENATE statement rather than a JOIN.  Again, this is much more performant.

I've written a post on incremental loading that covers a number of techniques you may find useful:

http://www.quickintelligence.co.uk/qlikview-incremental-load/

Steve