Reg:How to perform Incremental Load

[#1]

Hi All,

I want to do Incremental Load.Thing is I would like to know how to perform Incremental load?

So please share the qvw file or share some doc.Client using Sql Server as Data base

[#2]

Hi , you can do like this

Let ThisExecTime = Date(Today()-1,'YYYY-MM-DD 00:00:00');

ODBC CONNECT TO v;

Paye:

LOAD ID,

DATIME,

……….

………….

……….

………

…….

………

BCBRANCH;

SQL SELECT *

FROM "RLIFE_DW".dbo."TEAM_PAYE_NEW"

WHERE DATIME > '$(LastExecTime)'

AND DATIME <= '$(ThisExecTime)';

Concatenate LOAD * FROM D:\Qview\LifeAsia\QVD\Paye.qvd (qvd)

WHERE NOT EXISTS(ID);

If ScriptErrorCount = 0 then

STORE Paye INTO Paye.QVD;

Let LastExecTime = ThisExecTime;

End If

Where

ID is a primary key

DATIME is a modified date time

above example is of append and update type of increamental load

Rahul

[Anonymous]

When do you establish LastExecTime to begin with?

[#4]

Hi Rahul,

Thanks for your kind reply.Since I have not used Incremental Load,I am having small query on this.Here you used variable

'$(LastExecTime)' where we have to define and what is the input for that.Please let me know the same.


Thanks & Regards

Jai

[#5]

Hi Jai,

Last Execution time should be RloadTime().

-Peterson

[#6]

Hello Rahul,

I am also trying to the same thing as you mentioned here. Did you find a solution for the problem? Please let me know if you did. I also have something like a TimeID instead of actual Date and Time... so I am not sure how to plan the incremental load as without that I will have to reload all the QVDs everyday which will take more than 2 hrs...

Please help.

Thanks,

ANDY

[#7]

The Qlikview Reference Manual has a whole section on this subject. For version 9 it is Book 1, Part IV, pages 505-508. It outlines how to handle combos of insert, update, and delete in the data set. Perhaps this goes without saying (for everyone but me), but in addition to limiting the data, one also has to combine the limited data set with the historical data set via a concatenate statement. Again, the manual covers the different scenarios one could encounter.

[#8]

Hi All,

I am uploading a sample file for incremental load for database Oracle.

Please see and tell if its helpful.

Raman

[#9]

Hello Ramandeep,

Thank you for the file, and I have seen that before. My question is that I have something called a Timeperiod ID and nothing in the Date table, so was wondering how can I do a reload from the DB to QVD with that? I think we need to have something like the last updated time Field in the DB to do an incremental load isnt it? I have sales, orders tables which my client needs to have 3 years data and everytime I reload the QVD its doing a full reload which is taking about 2-3 hours and then an incremental load of QVW which takes about 15 minutes... Is there a solution to avoid the complete reload of the QVDs with just the Time period ID (Primary Key)?

Thanks,

ANDY

[#10]

Hi Andy,

If you have salesorder Tables then you might be having a CreationDate in that table I guess it is a must.

Before using the script above you have to do once a full upload. Save as a Table.qvd. With the Script above you can implement the Incremental load and load only new data to the already present data in Table.Qvd . This is done by the concatenate Load and the Dates.

Regards

Sravan