Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 
qv_testing
Specialist II
Specialist II

incremental performance

Hi Community,

I have done incremental load for only for new records.

Full reload and Incremental load taking same time....

How to improve performance....???

SET vQvdFile='E:\Ledg_Entry.qvd';

SET vTableName='Ledger_Entry';

LET vQvdExists = if(FileSize('$(vQvdFile)') > 0, -1, 0);

IF $(vQvdExists) THEN

maxdateTab:

LOAD Date(max("Posting Date")-1,'YYYY-MM-DD') as maxdate

  FROM $(vQvdFile) (qvd);

LET vIncrementalExpression = 'Where Convert("Posting Date",SQL_CHAR) >=' & Chr(39) & peek('maxdate') & Chr(39);

DROP table maxdateTab;

      ELSE

LET vIncrementalExpression = '';

     END IF

Cust_Ledger_Entry:

LOAD "Entry No",

    "Customer No",

    "Posting Date",

    "Document Type";

SQL SELECT *

FROM "Ledger Entry"

$(vIncrementalExpression);

IF $(vQvdExists) THEN

CONCATENATE ($(vTableName))

LOAD * FROM $(vQvdFile) (qvd);

End IF

STORE $(vTableName) INTO $(vQvdFile);

drop table $(vTableName);

Both of taking same time (Full and Incremental Load).

Any other method to fetch new records..??

Thanks in Advance...

4 Replies
qv_testing
Specialist II
Specialist II
Author

I don't have any Primary_Key.

I am using Views.

sujeetsingh
Master III
Master III

Can you just come up with more words sir ?

qv_testing
Specialist II
Specialist II
Author

Sorry for Delay..

I am fetching data from SQL Views, i don't have any primary key in that.

Tables have huge data, almost i have 20 views..

it was full loading time more than 1 hour.

So, I did Incremental load for fetching new data only.

It's working fine.. but it's taking same time.

Did i anything wrong??

How to reduce reload time....

Please any suggestions???

marcus_sommer

I'm not sure if there are further issues but your pick from the max-date isn't correct then outside from a load you need to specify all parameters also instead of:

peek('maxdate') you need: peek('maxdate', 0, 'maxdateTab')

- Marcus