Task getting failed in QMC because of memory alloc... - Qlik Community

vireshkolagimat · ‎2018-05-23

Dear All,

I have scheduled some tasks which runs on every hour. Sometimes i get the below message and after that the task will fail.

Memory Allocation Delta for this file=50.16 Mb. Available Physical Memory Before Reload=101025.36 Mb.

Available Physical Memory After Reload=98861.09 Mb. Total Physical Memory=98861.09 Mb.

The task runs fine in the next schedule without any issues. Can somebody help me to understand the above message.

Thank you.

Regards,

Viresh

Miguel_Angel_Baeyens · ‎2018-05-23

That is not an error message, could you please post the log of the task? According to that line, there is no issue with the memory, the server has available 100GB before starting and 98GB after, but it mentions "after reload" so we have to look into something else.

vireshkolagimat · ‎2018-05-23

Here is the complete log file.

(2018-05-23 05:55:32) Information: Starting task 'Oracle EBS/Oracle/Extraction/0.General_Qvd_Generator.qvw'. Id:5b22daa5-7f47-42c2-8758-25b7ba02ac5d. Triggered by 'ScheduleTrigger'. Id:b1daffa9-40c1-4517-bbe9-83852117bc94

(2018-05-23 05:55:32) Information: Entering Task Execution.

(2018-05-23 05:55:32) Information: ClusterID=1

(2018-05-23 05:55:32) Information: QDSID=6fde0f7f-2c56-9382-254d-c7c8f3685623

(2018-05-23 05:55:32) Information: TaskID=5b22daa5-7f47-42c2-8758-25b7ba02ac5d

(2018-05-23 05:55:32) Information: MaxRunTime=06:00:00

(2018-05-23 05:55:32) Information: MachineName=OMMUENAP09

(2018-05-23 05:55:32) Information: Max attempts:1

(2018-05-23 05:55:32) Information: Current Attempt=0

(2018-05-23 05:55:32) Information: Task Dependencies are OK

(2018-05-23 05:55:32) Information: Document is marked to be Reloaded with fresh data. Initializing Reload for Distribution.

(2018-05-23 05:55:32) Information: Opening "E:\ORACLEEBS\Oracle\Extraction\0.General_Qvd_Generator.qvw"

(2018-05-23 05:55:32) Information: Allocating new QlikView Engine. Current usage count=1 of 3 (of type non-reader).

(2018-05-23 05:55:32) Information: Max retries:5

(2018-05-23 05:55:32) Information: Attempt:01

(2018-05-23 05:55:33) Information: Opened the QlikView Engine successfully. ProcessID=46180

(2018-05-23 05:55:33) Information: Allocated QlikView Engine successfully. Current usage count=2 of 3 (of type non-reader). Ticket number=145.

(2018-05-23 05:55:33) Information: Loading document "E:\ORACLEEBS\Oracle\Extraction\0.General_Qvd_Generator.qvw" (0.17 Mb)

(2018-05-23 05:55:33) Information: Physical FileSize=0.17 Mb. Memory Allocation Delta for this file=2.56 Mb. Available Physical Memory Before Open=101028.27 Mb. Available Physical Memory After Open=101025.36 Mb. Total Physical Memory=131037.27 Mb.

(2018-05-23 05:55:33) Information: Attempted to load the document without data.

(2018-05-23 05:55:33) Information: The document was loaded successfully.

(2018-05-23 05:55:33) Information: Document was opened successfully

(2018-05-23 05:55:33) Information: Starting reload

(2018-05-23 05:55:33) Information: QlikView->Settings->Document Peferences->Generate Logfile in document is set. Enabling Log Copier.

(2018-05-23 05:55:33) Information: The Source Document is being reloaded. DocumentPath=E:\ORACLEEBS\Enhance Oracle\Extraction\0.General_Qvd_Generator.qvw

(2018-05-23 05:55:33) Information: Waiting for the document log "E:\ORACLEEBS\Enhance Oracle\Extraction\0.General_Qvd_Generator.qvw" to become available.. Check until 23/05/2018 05:56:03 AM

(2018-05-23 05:55:33) Information: Found a document log. FileName=E:\ORACLEEBS\Enhance Oracle\Extraction\0.General_Qvd_Generator.qvw.log

(2018-05-23 05:55:33) Information: Writing documentLog to C:\ProgramData\QlikTech\DistributionService\1\Log\20180523\055532 - Oracle EBS_Oracle_Extraction_0.General_Qvd_Generator.qvw\DocumentLog.txt

(2018-05-23 05:55:34) Information: Reloading.

(2018-05-23 05:55:35) Information: Reloading..

(2018-05-23 05:55:36) Information: Reloading...

(2018-05-23 05:55:37) Information: Reloading....

(2018-05-23 05:55:38) Information: Reloading.....

(2018-05-23 05:55:39) Information: Reloading......

(2018-05-23 05:55:40) Information: Reloading.......

(2018-05-23 05:55:41) Information: Reloading........

(2018-05-23 05:55:42) Information: Reloading.........

(2018-05-23 05:55:43) Information: Reloading..........

(2018-05-23 05:55:44) Information: Slow down logging. Log every <2> seconds.

(2018-05-23 05:55:44) Information: Reloading

(2018-05-23 05:55:46) Information: Reloading.

(2018-05-23 05:55:48) Information: Reloading..

(2018-05-23 05:55:50) Information: Reloading...

(2018-05-23 05:55:52) Information: Reloading....

(2018-05-23 05:55:54) Information: Reloading.....

(2018-05-23 05:55:56) Information: Reloading......

(2018-05-23 05:55:58) Information: Reloading.......

(2018-05-23 05:56:00) Information: Reloading........

(2018-05-23 05:56:02) Information: Reloading.........

(2018-05-23 05:56:04) Information: Reloading..........

(2018-05-23 05:56:06) Information: Slow down logging. Log every <10> seconds.

(2018-05-23 05:56:06) Information: Reloading

(2018-05-23 05:56:16) Information: Reloading.

(2018-05-23 05:56:26) Information: Reloading..

(2018-05-23 05:56:36) Information: Reloading...

(2018-05-23 05:56:46) Information: Reloading....

(2018-05-23 05:56:56) Information: Reloading.....

(2018-05-23 05:57:06) Information: Reloading......

(2018-05-23 05:57:16) Information: Reloading.......

(2018-05-23 05:57:26) Information: Reloading........

(2018-05-23 05:57:36) Information: Reloading.........

(2018-05-23 05:57:46) Information: Reloading..........

(2018-05-23 05:57:56) Information: Slow down logging. Log every <60> seconds.

(2018-05-23 05:57:56) Information: Reloading

(2018-05-23 05:58:05) Information: The Source Document reload complete. DocumentPath=E:\ORACLEEBS\Oracle\Extraction\0.General_Qvd_Generator.qvw

(2018-05-23 05:58:05) Information: Memory Allocation Delta for this file=50.16 Mb. Available Physical Memory Before Reload=101025.36 Mb. Available Physical Memory After Reload=98861.09 Mb. Total Physical Memory=98861.09 Mb.

(2018-05-23 05:58:05) Error: The Source Document was NOT reloaded successfully. DocumentPath=E:\ORACLEEBS\Enhance Oracle\Extraction\0.General_Qvd_Generator.qvw.

(2018-05-23 05:58:05) Information: Closing the document.

(2018-05-23 05:58:06) Information: Closed the QlikView Engine successfully. ProcessID=46180

(2018-05-23 05:58:06) Error: The task "Oracle EBS/Enhance Oracle/Extraction/0.General_Qvd_Generator.qvw" failed. Exception: || QDSMain.Exceptions.TaskFailedException: Task execution failed with errors to follow. ---> QDSMain.Exceptions.ReloadFailedException: Reload failed ---> QDSMain.Exceptions.LogBucketErrorException: The Source Document was NOT reloaded successfully. DocumentPath=E:\ORACLEEBS\Enhance Oracle\Extraction\0.General_Qvd_Generator.qvw. || at QDSMain.ReloadTask.VerifyConditions(TaskResult taskResult) || at QDSMain.ReloadTask.Reload(String fileName, TaskResult taskResult, String sectionAccessUserName, String sectionAccessPassword, eReloadOptions reloadOption, String variableName, String variableValue, Boolean moniterCpuUsage) || --- End of inner exception stack trace --- || at QDSMain.ReloadTask.Reload(String fileName, TaskResult taskResult, String sectionAccessUserName, String sectionAccessPassword, eReloadOptions reloadOption, String variableName, String variableValue, Boolean moniterCpuUsage) || at QDSMain.DistributeTask.Execute(TaskResult currentTaskResult) || --- End of inner exception stack trace --- || at QDSMain.DistributeTask.Execute(TaskResult currentTaskResult) || at QDSMain.Task.AbstractTask.TaskExecution(ILogBucket logBucket, TaskResult taskResult)

(2018-05-23 05:58:06) Information: Task Execute Duration=00:02:33.9898122

(2018-05-23 05:58:06) Information: Sending Alert mail

(2018-05-23 05:58:08) Information: TaskResult.status=Finished

(2018-05-23 05:58:08) Information: Notifying all triggers of new state:FinishedWithErrors

(2018-05-23 05:58:08) Information: Notifying all triggers of new state:FinishedWithErrors - completed

(2018-05-23 05:58:08) Information: Saving Task Result

Thank you.

Miguel_Angel_Baeyens · ‎2018-05-23

The error is not related to memory based on that log. It could be permissions writing the file, but it also says "Closing the Document". What else is doing the task? Is it reducing or distributing the file?

To verify that the script works, check the log at E:\ORACLEEBS\Enhance Oracle\Extraction\0.General_Qvd_Generator.qvw.log

If this log does not exist, open the QVW with QlikView Desktop, go to Settings > Document Properties > General > check Generate Logfile and run the task again.

It will create that log and you will be able to see which line it breaks, if it does. If everything is fine in the log, there is no error, then you will have to verify that the account running the QlikView Distribution Service has permissions to write on the E: drive. If the task also distributes or reduces, check for the permissions in the folders where this happens.

vireshkolagimat · ‎2018-05-23

Hi, the log file looks fine and also i am not using distribution service but the account i am using to log on to the server has access to run these tasks.

The task ran in the next schedule without any issues. Really not able to figure out the issue.

Thanks for the suggestions.

Regards,

Viresh

Peter_Cammaert · ‎2018-05-23

Just to be sure, can you also post the last 10 to 15 lines from this file (from your Tasklog of a failed reload attempt)?

C:\ProgramData\QlikTech\DistributionService\1\Log\20180523\055532 - Oracle EBS_Oracle_Extraction_0.General_Qvd_Generator.qvw\DocumentLog.txt

vireshkolagimat · ‎2018-05-23

Hi Peter,

Here is the log details and it says some General script error.

23/05/2018 05:57:58.3937277 Information 5/23/2018 5:55:40 AM: 0373 when 'LOG MDC OU' then 19

23/05/2018 05:57:58.3937277 Information 5/23/2018 5:55:40 AM: 0374 when 'GM MDC OU' then 20

23/05/2018 05:57:58.4093572 Information 5/23/2018 5:55:40 AM: 0375 when 'TUL OU' then 21

23/05/2018 05:57:58.4093572 Information 5/23/2018 5:55:40 AM: 0376 when 'BPCL OU' then 22

23/05/2018 05:57:58.4093572 Information 5/23/2018 5:55:40 AM: 0377 when 'GM CSTC OU' then 23

23/05/2018 05:57:58.4249823 Information 5/23/2018 5:55:40 AM: 0378 when 'EGO OU' then 24

23/05/2018 05:57:58.4249823 Information 5/23/2018 5:55:40 AM: 0379 when 'CA2 MCS' then 25 end as DIVISIONCODE

23/05/2018 05:57:58.4249823 Information 5/23/2018 5:55:40 AM: 0380 FROM APPS.XXEGC_QV_OVERDUE_TAB_V

23/05/2018 05:57:58.4406050 Information 5/23/2018 5:57:58 AM: | 15 fields found: customer, item_no, refernce, dated, due_date, userdate, kind, SMGroup, SalesmanCode, amount, currency_amount, unall_amount, open_indicator, DivisionCode,

23/05/2018 05:58:05.6751743 Information ReloadTime, 69,336 lines fetched

23/05/2018 05:58:05.6751743 Information 5/23/2018 5:58:05 AM: 0382 STORE OverDues into ..\Data\Qvds\Intermediate_Qvds\OverDues.qvd

23/05/2018 05:58:05.6751743 Information 5/23/2018 5:58:05 AM: General Script Error

23/05/2018 05:58:05.6907979 Information 5/23/2018 5:58:05 AM: Execution Failed

23/05/2018 05:58:05.6907979 Information 5/23/2018 5:58:05 AM: Execution finished.

Thank you.

Miguel_Angel_Baeyens · ‎2018-05-23

It looks like the STORE is the instruction failing, maybe that file is in use and locked by another QVW and cannot be written, or the account does not have permissions. Or even if it's a NAS the file is very big and there is some latency.

Replace the name of the file by OverDues_1.qvd to see whether the issue keeps happening.

Peter_Cammaert · ‎2018-05-23

STORE is one of those QlikView script statements that comes with absolutely no explanation whatsoever in case of failure. But as Miguel already explained, there are only a few checks for you to do in order to figure out what is wrong, and - when that produces inconclusive results - what can be done to eliminate the problem.

"Most of the time", it is due to a permissions problem or due to the fact that two processes try to write or read&write the QVD at the same time. Since you reported that it works *sometimes*, I would vote for the second one.

"Most of the time", simply deleting the original QVD (the one from a previous run that is still there) will solve the issue.

But then, there may be some other cause that is specific to your situation

vireshkolagimat · ‎2018-05-23

Thank you.

Task getting failed in QMC because of memory allocation