Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Search our knowledge base, curated by global Support, for answers ranging from account questions to troubleshooting error messages.
Qlik offers a wide range of channels to assist you in troubleshooting, answering frequently asked questions, and getting in touch with our technical experts. In this article, we guide you through all available avenues to secure your best possible experience.
For details on our terms and conditions, review the Qlik Support Policy.
Index:
We're happy to help! Here's a breakdown of resources for each type of need.
Support | Professional Services (*) | |
Reactively fixes technical issues as well as answers narrowly defined specific questions. Handles administrative issues to keep the product up-to-date and functioning. | Proactively accelerates projects, reduces risk, and achieves optimal configurations. Delivers expert help for training, planning, implementation, and performance improvement. | |
|
|
(*) reach out to your Account Manager or Customer Success Manager
Your first line of support: https://community.qlik.com/
Looking for content? Type your question into our global search bar:
Leverage the enhanced and continuously updated Knowledge Base to find solutions to your questions and best practice guides. Bookmark this page for quick access!
Subscribe to maximize your Qlik experience!
The Support Updates Blog
The Support Updates blog delivers important and useful Qlik Support information about end-of-product support, new service releases, and general support topics. (click)
The Qlik Design Blog
The Design blog is all about product and Qlik solutions, such as scripting, data modelling, visual design, extensions, best practices, and more! (click)
The Product Innovation Blog
By reading the Product Innovation blog, you will learn about what's new across all of the products in our growing Qlik product portfolio. (click)
Q&A with Qlik
Live sessions with Qlik Experts in which we focus on your questions.
Techspert Talks
Techspert Talks is a free webinar to facilitate knowledge sharing held on a monthly basis.
Technical Adoption Workshops
Our in depth, hands-on workshops allow new Qlik Cloud Admins to build alongside Qlik Experts.
Qlik Fix
Qlik Fix is a series of short video with helpful solutions for Qlik customers and partners.
Suggest an idea, and influence the next generation of Qlik features!
Search & Submit Ideas
Ideation Guidelines
Get the full value of the community.
Register a Qlik ID:
Incidents are supported through our Chat, by clicking Chat Now on any Support Page across Qlik Community.
To raise a new issue, all you need to do is chat with us. With this, we can:
Log in to manage and track your active cases in Manage Cases. (click)
Please note: to create a new case, it is easiest to do so via our chat (see above). Our chat will log your case through a series of guided intake questions.
When creating a case, you will be prompted to enter problem type and issue level. Definitions shared below:
Select Account Related for issues with your account, licenses, downloads, or payment.
Select Product Related for technical issues with Qlik products and platforms.
If your issue is account related, you will be asked to select a Priority level:
Select Medium/Low if the system is accessible, but there are some functional limitations that are not critical in the daily operation.
Select High if there are significant impacts on normal work or performance.
Select Urgent if there are major impacts on business-critical work or performance.
If your issue is product related, you will be asked to select a Severity level:
Severity 1: Qlik production software is down or not available, but not because of scheduled maintenance and/or upgrades.
Severity 2: Major functionality is not working in accordance with the technical specifications in documentation or significant performance degradation is experienced so that critical business operations cannot be performed.
Severity 3: Any error that is not Severity 1 Error or Severity 2 Issue. For more information, visit our Qlik Support Policy.
If you require a support case escalation, you have two options:
A collection of useful links.
Qlik Cloud Status Page
Keep up to date with Qlik Cloud's status.
Support Policy
Review our Service Level Agreements and License Agreements.
Live Chat and Case Portal
Your one stop to contact us.
The IBM DB2 for iSeries source endpoint occasionally encounters an error during the CDC stage. This issue appears to be linked to the presence of the IBM i Access ODBC Driver versions 7.1.26 and 7.1.27.
The error message in task log file:
[SOURCE_CAPTURE ]E: Error parsing [1020109] (db2i_endpoint_capture.c:652)
The issue specifically arises during the CDC stage; however, the Full Load stage operates smoothly without any complications.
As a workaround please downgrade IBM i Access ODBC client from versions '07.01.027'/'07.01.026' to '07.01.025'
The most recent version of IBM i Access ODBC Client is '07.01.027' as of today. For compatibility reasons, it's advisable to revert to version '07.01.025', as '07.01.026' exhibits the same issue.
Various factors can contribute to encountering the 'Error parsing' message, including:
• DB2i ODBC Version '07.01.027' (as described in this article)
• In a single task, the total number of captured tables exceeds 300
• The source table is created by DDS
• Garbage data in table
• Special characters in table object identifier (table name, or column name)
If you continue to encounter the error after switching to '07.01.025', please reach out to Qlik Support for further assistance.
The behavior of the IBM DB2i ODBC Versions '07.01.026' & '07.01.027' differ slightly from that of '07.01.025'. In certain scenarios, it may return incorrect column lengths
#00158029, #00160002, QB-26413
This article provides a comprehensive guide to efficiently install the PostgreSQL ODBC client on Linux for a PostgreSQL target endpoint.
If the PostgreSQL serves as Replicate source endpoint, please check: How to Install PostgreSQL ODBC client on Linux for PostgreSQL Source Endpoint
rpm -ivh postgresql13-libs-13.2-1PGDG.rhel8.x86_64.rpm
rpm -ivh postgresql13-odbc-13.02.0000-1PGDG.rhel8.x86_64.rpm
rpm -ivh postgresql13-13.2-1PGDG.rhel8.x86_64.rpm
export LD_LIBRARY_PATH=/usr/pgsql-13/lib:$LD_LIBRARY_PATH
rpm -ivh unixODBC-2.3.7-1.el8.x86_64.rpm
[PostgreSQL]
Description = ODBC for PostgreSQL
Driver = /usr/lib/psqlodbcw.so
Setup = /usr/lib/libodbcpsqlS.so
Driver64 = /usr/pgsql-13/lib/psqlodbcw.so
Setup64 = /usr/lib64/libodbcpsqlS.so
FileUsage = 1
[pg15]
Driver = /usr/pgsql-13/lib/psqlodbcw.so
Database = targetdb
Servername = <targetDBHostName or IP Address>
Port = 5432
UserName = <PG User Name>
Password = <PG user's Password>
The Chart Exploration panel is not shown in straight tables after the upgrade to Qlik Sense May 2023 Patch 3. The option is still visible when you right click on the table, but it does not work: the table is just enlarged or an empty window appears instead of the Chart Exploration.
The problem is not present in any Qlik Sense releases previous to May 2023 Patch 3. It is fixed in Qlik Sense May 2023 Patch 6 and higher releases. The Qlik Sense August release was not affected.
In releases prior to May Patch 6, it is possible to resolve the problem on the affected releases by following these steps:
An incorrect formatting of the paths in the 'import-map.json' file causes the problem.
QB-20719, QB-21856
In this article, we walk you through the requirements and process of how to upgrade and unbundle an existing Qlik Sense Repository Database (see supported scenarios) as well as how to install a brand new Repository based on PostgreSQL. We will use the Qlik PostgreSQL Installer (QPI).
For a manual method, see How to manually upgrade the bundled Qlik Sense PostgreSQL version to 12.5 version.
Using the Qlik Postgres Installer not only upgrades PostgreSQL; it also unbundles PostgreSQL from your Qlik Sense Enterprise on Windows install. This allows for direct control of your PostgreSQL instance and facilitates maintenance without a dependency on Qlik Sense. Further Database upgrades can then be performed independently and in accordance with your corporate security policy when needed, as long as you remain within the supported PostgreSQL versions. See How To Upgrade Standalone PostgreSQL.
Index
The following versions have been tested and verified to work with QPI (1.4.0):
Qlik Sense February 2022 to Qlik Sense November 2023.
If you are on a Qlik Sense version prior to these, upgrade to at least February 2022 before you begin.
Qlik Sense November 2022 and later do not support 9.6, and a warning will be displayed during the upgrade. From Qlik Sense August 2023 a upgrade with a 9.6 database is blocked.
The Qlik PostgreSQL Installer supports installing a new standalone PostgreSQL database with the configurations required for connecting to a Qlik Sense server. This allows setting up a new environment or migrating an existing database to a separate host.
Using the Qlik PostgreSQL Installer on a patched Qlik Sense version can lead to unexpected results. If you have a patch installed, either:
Do not use the standard Qlik Sense folders, such as C:\Program Files\Qlik\Sense\Repository\PostgreSQL\ and C:\Programdata\Qlik\Sense\Repository\PostgreSQL\.
Do not use the standard Qlik Sense folders, such as C:\Program Files\Qlik\Sense\Repository\PostgreSQL\ and C:\Programdata\Qlik\Sense\Repository\PostgreSQL\.
Download the installer here.
Qlik PostgreSQL installer Release Notes
The following versions have been tested and verified to work with QPI (1.4.0):
February 2022 to November 2023.
If you are on any version prior to these, upgrade to at least February 2022 before you begin.
Qlik Sense November 2022 and later do not support 9.6, and a warning will be displayed during the upgrade. From Qlik Sense August 2023 a 9.6 update is blocked.
Uninstall the old Qlik Sense Repository Database service.
This step is required. Failing to remove the old service will lead the upgrade or patching issues.
Failing to reinstall the binaries will lead to errors when executing any number of service configuration scripts.If you do not immediately upgrade:
If the upgrade was unsuccessful and you are missing data in the Qlik Management Console or elsewhere, contact Qlik Support.
Now that your PostgreSQL instance is no longer connected to the Qlik Sense Enterprise on Windows services, all future updates of PostgreSQL are performed independently of Qlik Sense. This allows you to act in accordance with your corporate security policy when needed, as long as you remain within the supported PostgreSQL versions.
Your PostgreSQL database is fully compatible with the official PostgreSQL installers from https://www.enterprisedb.com/downloads/postgres-postgresql-downloads.
See How To Upgrade Standalone PostgreSQL, which documents the upgrade procedure for either a minor version upgrade (example: 14.5 to 14.8) or a major version upgrade (example: 12 to 14). Further information on PostgreSQL upgrades or updates can be obtained from Postgre directly.
Video chapters:
The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support. The video in this article was recorded in a earlier version of QPI, some screens might differ a little bit.
Qlik PostgreSQL installer version 1.3.0 Release Notes
Techspert Talks - Upgrading Qlik Sense Repository Service
Backup and Restore Qlik Sense Enterprise documentation
Migrating Like a Boss
Optimizing Performance for Qlik Sense Enterprise
Qlik Sense Enterprise on Windows: How To Upgrade Standalone PostgreSQL
How-to reset forgotten PostgreSQL password in Qlik Sense
How to configure Qlik Sense to use a dedicated PostgreSQL database
Troubleshooting Qlik Sense Upgrades
The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.
As discussed in this community thread, an object field may disappear after applying a previousily created bookmark (bookmark created with option "Save layout" checked).
Defect "QB-19322" was reported to R&D but was closed as WAD (working as designed) with the following explaination.
------------
This is WAD (working as designed).
When you store layout state you create a patch on the table that restores the appearance of the table to the way it was when the bookmark was saved, this means that changes coming in later will not be visible when the bookmark is used.
The patch is cleared when the session ends, or if you apply a bookmark with another layout state (but not if you apply a bookmark without layout state or change the selections).
If the user needs the new columns, the best (or only) way is to recreate the bookmark with the new content visible.
--------------
Information provided on this defect is given as is at the time of documenting. For up to date information, please review the most recent Release Notes, or contact support with the ID "QB-19322" for reference.
Product Defect ID: QB-19322
Bookmarks with "Include layout state" do not maintain the column order in Straight table
The goal is to dynamically add images in a PixelPerfect report. In this example we will add country flags, that are stored as image files, in a PixelPerfect template.
The information in this article is provided as-is and will be used at your discretion. Depending on the tool(s) used, customization(s), and/or other factors, ongoing support on the solution below may not be provided by Qlik Support.
NOTE: Please feel free to create the 'ideation' and share the link to your ideation feature request entered to the Ideas | Qlik Community in the comments below.
The information in this article is provided as-is and will be used at your discretion. Depending on the tool(s) used, customization(s), and/or other factors, ongoing support on the solution below may not be provided by Qlik Support.
Although there is a 'success' in Qlik Sense or Qlik Cloud but data is not updated in the Qlik Sense or Cloud app as expected.
Qlik Sense Reload Task Success
Qlik Cloud Reload Success
Check the following:
The information in this article is provided as-is and will be used at your discretion. Depending on the tool(s) used, customization(s), and/or other factors, ongoing support on the solution below may not be provided by Qlik Support.
This guide provides the basic instructions on configuring Qlik Cloud with Okta as an identity provider.
This customization is provided as is. Qlik Support cannot provide continued support of the solution. For assistance, reach out to our Professional Services or engage in our active Integrations forum.
This must be the actual tenant name, not the alias.
For additional information on how to create new identity providers in Qlik Cloud, see Creating a new identity provider configuration.
The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.
Note: This application only returns a limited subset of usage information. Qlik is in the process of developing a new "Automation Analyzer" application for the Qlik Cloud Monitoring Application suite, which will replace this app.
This article shows how to use the example Qlik Application Automation Monitoring App. It explains how to set up the load script and how to use the app for monitoring Qlik Application Automation usage statistics for a cloud tenant.
Index:
The app included is an example and not an official app. The app is provided as-is.
There are four steps for the configuration of the load script:
The monitoring app includes four sheets that present various information on the Qlik Application Automation usage in the current tenant.
Filtering is available based on Automation Name, Last Run Status, Run Mode, State & Status.
The Qlik Application Automation Monitoring app facilitates incremental load, that is, only added data is loaded into the app.
Important: Qlik Application Automation runs are only stored for 30 days. When data is loaded into the app for the first time, only 30 days of history is loaded, thus only 30 days of history will be available in the app. After this initial data load, data older than 30 days will be available in the app thanks to the incremental data load. If data is loaded at least once every 30 days, continuous data will be available in the app.
Qlik Cloud
Qlik Application Automation
The information in this article is provided as-is and is to be used at your own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.
Are you looking to download your purchased Qlik Products or download a Trial? All supported on-premise Qlik Products can be downloaded from Qlik's Product Download Site.
To access the Download Site, you need an active QlikID. You will be able to see all products your account is eligible for.
You can access the Download Page directly here, or navigate to it from the Community Home page:
This shows the download page with the Latest release and patch preselected.
Unsupported versions are not available for download. See Product Lifecycle for details on what versions have reached end of support.
Alternatively, instead of selecting your product directly, you can also search all available columns:
If you encounter issues with the download site, start a chat with us and we will be able to help you right away.
Click here for video transcript
Using Open in Server in QlikView Desktop to connect to a QlikView server fails with:
Connected to server OK, No reply received
Switching to HTTPS tunnelling
Connected to server OK, Security settings denies access
Switching to HTTP tunnelling
Connected to server OK, No reply received
The Desktop client is running version May 2023 (12.80) or later, while the QlikView server is a previous version, such as May 2022 (12.70).
If QlikView Desktop and QlikView Server are running different versions, either upgrade the lower version to match the higher version or set the correct and same session algorithm in the Settings.ini file to resolve the discrepancy.
For issues where the QlikView Desktop is running version May 2023 (12.80) and the QlikView Server is running version May 2022 (12.70) or earlier, the QlikView Desktop settings.ini file must contain the entry SessionAlgorithm=CALG_AES_128.
Modify the QlikView desktop client settings.ini to add SessionAlgorithm=CALG_AES_128. For information on how to modify the settings.ini, see: How to change the settings.ini in QlikView Desktop and QlikView.
The discrepancy in the session algorithm causes the error. QlikView May 2023 (12.80) now uses AES256 encryption instead of AES128, which was used in QlikView May 2022 (12.70) and earlier.
What's New in QlikView May 2023?
QlikView 12.80: set a different Session Algorithm for QVP communication
Communication to QlikView server fails: Switching to HTTPS tunneling
QlikView Client connection fails with Connected server OK, negotiations timed out Switching to HTTP tunneling No reply received
QV-24861
Tabular reporting can't create table individual columns with some specific objects. This will lead to an error during the report production, so the option will be removed soon from the Add-Inn.
The fields in these columns can't neither be used in levels: this possibility will be soon removed as well.
It is possible that individual column or levels are currently present in Excel templates, especially if the report is imported from Qlik NPrinting.
In this case, the preview or the report generation by task fails with this message: " Error code: REP-500200"
The affected objects are:
It is necessary to manually remove these objects from any reports by editing them with the Tabular reporting Add-Inn.
It is possible to duplicate the same objects in table format on the Qlik Cloud Sense app and import the table in the Tabular Reporting template.
Tabular Reporting does not support these objects to create table individual column . Since they can be used in Qlik NPrinting, it is possible that they are present in imported reports.
A Qlik Replicate task with a File source endpoint fails with the following error:
00011440: 2022-05-06T09:04:10 [TASK_MANAGER ]I: Start loading table ''.'your_table_name' (Id = 9) by subtask 1. Start load timestamp 0005DE534DB02115 (replicationtask_util.c:752)
00012268: 2022-05-06T09:04:10 [SOURCE_UNLOAD ]E: Failed to write record id: 3293, Number of values: 1 is not equal to number of columns: 2. [1020417] (file_unload.c:344)
00012268: 2022-05-06T09:04:10 [SOURCE_UNLOAD ]E: Failed to init unloading table. [1020417] (file_unload.c:395)
00011440: 2022-05-06T09:04:10 [TASK_MANAGER ]E: Table error occurred. Based on error behavior policy, task will stop. [1021705] (replicationtask.c:3082)
Verify that you are pointing to the direct path of the file you are referencing.
Within your source file endpoint go to
Setting advanced options
Task for CSV source fails - "cannot refresh metadata" and "could not get object"
The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.
A Qlik Replicate task fails in writing to ADLS in parquet format from source DB2 z/os. The error in the task log is:
[TARGET_LOAD ]E: Failed to convert file from csv to parquet
Error:: failed to read csv temp file
The ODBCINST.ini may not be directed to the accurate DB2 64-bit ODBC driver. Resolving the issue may require a manual edit of the ODBCINST.ini file.
Original value:
./opt/ejvdb2odbc/lib/libdb2.so
Corrected to:
./opt/ejvdb2odbc/lib/libdb2o.so
After the changes have been made and saved, reload the task.
The listed commands are unsupported by Qlik Replicate. The commands work, but Qlik cannot give a guarantee that they will continue to function in the future.
If you do not have Qlik Enterprise Manager (QEM), then this is the option to script task starts and stops. If you do have QEM please look into using the restAPI calls to do the same things. You can reference the QEM developer's guide for information on this API.
To note:
You can start the task multiple ways based on the following parameters and flags:
Parameters:
1 = Start Full Load only
2 = Start Change Capture only
3 = Start Both
Flags Values:
0 = Resume
1 = Fresh Start (like starting as of now)
Start a task:
repctl connect; execute task_name 3 Flags=0; disconnect
Get task status:
repctl connect; gettaskstatus task_name; disconnect
Stop a task:
repctl connect; stoptask task_name; disconnect
Notes:
Repctl execute task=<task_name> 1 flags=1
repctl connect ip=192.168.165.11 port=3552 ; stoptask task; disconnect
Alternate (non-default) data directories must be provided using -d <path>
Alternatively in Windows use: ?set AREP_DATA=<path>
If the Replicate data folder is on another location you would need to set the path for repctl.
Example:
repctl.exe -d "E:\Attunity\Replicate\data" connect;
To run this on Linux you may need to add a back slash in front of the semi-colon:
repctl connect\; gettaskstatus task_name\; disconnect
In addition, while using Linux shell, you have to escape the semicolons with backslashes.
Example:
?./repctl connect\; gettaskstatus task_name\; disconnect
Further note on usage: be certain to have blank space following the semi-colon command delimiter.
The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.
This is a problem which on first impressions should not (and you would think logically cannot) happen. Therefore it is important to understand why it does, and what can be done to resolve it when it does.
The situation is that Replicate is doing a Full Load for a table (individually or as part of a task full loading many tables). The source and target tables have identical unique primary keys. There are no uppercasing or other character set issues relating to any of the columns that make up the key which may sometimes cause duplication problems. Yet as the Full Load for the table progresses, probably nearing the end, you get a message indicating that Replicate has failed to insert a row into the target as a result of a duplicate. That is there is already a row in the target table with the unique key for the row that it is trying to insert. The Full Load for that table is terminated (often after several hours); and if you try again the same error, perhaps for a different row, will often occur.
Logically this shouldn’t happen, but it does. The likelihood of it doing so depends on the source DBMS type, the type of columns in the source table, and you will find it is always for a table that is being updated (SQL UPDATEs) as Replicate copies it. The higher the update rate and the bigger the table, the more likely it is to happen.
Note: This article discussed the problems that are related to duplicates in the TARGET_LOAD and not the TARGET_APPLY, that is during Full Load and before starting to apply the cached changes.
To understand the fix we first need to understand why the problem occurs, and this involves understanding some of the internal workings of most conventional Relational Database Management Systems.
RDBMS’s tend to employ different terminology for things that exist in all of them. I’m going to use DB2 terminology and explain each term the first time I use it. With a different RDBMS the terminology may be different, but the concepts are generally the same?
The first concept to introduce is the Tablespace. That’s what it’s called in DB2, but it exists for all databases and is the physical area where the rows that make up the table are stored. Logically it can be considered as a single contiguous data area, split up into blocks, numbered in ascending order.
This is where your database puts the row data when you INSERT rows into the table. What’s also important is that it tries to update the existing data for a row in place when you do an UPDATE, but may not always be able to do so. If that is the case then it will move the updated row to another place in the tablespace, usually at what is then the highest used (the endpoint) block in the tablespace area.
The next point concerns how the DBMS decides to access data from the tablespace in resolving your SQL calls. Each RDBMS has an optimiser, or something similar that makes these decisions. The role of indexes with a relational database is somewhat strange. They are not really part of the standard Relational Database model, although in practice they are used to guarantee uniqueness and support referential integrity. Other than for these roles, they exist only to help the optimiser come up with faster ways of retrieving rows that satisfy your SELECT (database read) statements.
When any piece of SQL (we’ll focus on simple SELECT statements here) is presented to the optimiser, it decides on what method to use to search for and retrieve any matching rows from the tablespace. The default method is to search through all the rows directly in the tablespace looking for rows that match any selection criteria, this is known as a Tablespace Scan.
A Tablespace Scan may be the best way to access rows from a table, particularly if it is likely that many or most of the rows in the table will match the selection criteria. For other SELECTs though that are more specific about what row(s) are required, a suitable matching index may be used (if one exists) to go directly to the row(s) in the tablespace.
The sort of SQL that Replicate generates to execute against the source table when it is doing a Full Load is of the form SELECT * FROM, or SELECT col1, col2, … FROM. Neither of these has any row specific selection criteria, and in fact this is to be expected as a Full Load is in general intended to select all rows from the source table.
As a result the database optimiser is not likely to choose to use an index (even if a unique index on the table exists) to resolve this type of SELECT statement, and instead a Tablespace Scan of the whole tablespace area will take place. This, as you will see later, can be inconvenient to us but is in fact the fastest way of processing all the rows in the table.
When we do a Full Load copy for a table that is ‘live’ (being updated as we copy it), the result we end up with when the SELECT against the source has been completed and we have inserted all the rows into the target is not likely to be consistent with what is then in the source table. The extent of the differences is dependent on the rate of updates and how long the Full Load for that table takes. For high update rates on big tables that take many hours for a Full Load the extent of the differences can be quite considerable.
This all sounds very worrying but it is not as the CDC (Change Data Capture) part of Replicate takes care of this. CDC is mainly known for Replicating changes from source to target after the initial Full Load has been taken, keeping the target copies up to date and in line with the changing source tables. However CDC processing has an equally important role to play in the Full Load process itself, especially when this is being done on ‘live’ tables subject to updates as the Full Load is being processed.
In fact CDC processing doesn’t start when Full Load is finished, but in fact before Full Load starts. This is so that it can collect details of changes that are occurring at the source whilst the Full Load (and it’s associated SELECT statement) are taking place. The changes collected during this period are known as the ‘cached changes’ and they are applied to the newly populated target table before switching into normal ongoing CDC mode to capture all subsequent changes.
This takes care of and fixes all of the table row data inconsistencies that are likely to occur during a table Full Load, but there is one particular situation that can occur and catch us out before the Full Load completes and the cached changes can be applied. This results in Replicate trying to insert details for the same row more than once in the target table; triggering the duplicates error that we are talking about here.
Consider this situation:
That is how the problem occurs. Having variable length columns, and binary object columns in the source table make this (movement of the row to a new location in the tablespace) much more likely to happen and the duplicate insert problem to occur.
So how to fix this, or at least how to find a method to stop it happening.
The solution is to persuade the optimiser in the source database to use the unique index on the table to access the rows in the table’s tablespace rather than scanning sequentially through it. The index (which is unique) will only provide one row to read for each key as the execution of our SELECT statement progresses. We don’t have to worry about whether it is the ‘latest’ version of the row or not because that will be taken care of later by the application of the cached changes.
The optimiser can (generally) be persuaded to use the unique index on the source table if the SELECT statement indicates that there is a requirement to return the rows in the result set in the order given by that index. This requires having a SELECT statement with a order clause matching the columns in the unique index. Something of the form SELECT * FROM ORDER BY col1, col2, col3, etc. Where col1, col2, col3 etc. are the columns that make up the tables unique primary index.
But, how can we do this. Replicate has a undocumented facility that allows the user to configure extra text to be added to the end of the generated SQL for a particular table during Full Load processing specifically to add a WHERE statement to determine which rows are included and excluded during a Full Load extract.
This is not exactly what we want to do (we want to include all rows), but this ‘FILTER’ facility also provides the option to extend the content of the SELECT statement that is generated after the WHERE part of the statement has been added. So we can use it to add the ORDER BY part of the statement that we require.
Here is the format of the FILTER statement that you need to add.
—FILTER: 1=1) ORDER BY col1, col2, coln —
This is inserted in the ‘Record Selection Condition’ box on the individual table filter screen when configuring the Replicate task. If you want to do this for multiple tables in the Replicate task then you need to set up a FILTER for each table individually.
To explain, the —FILTER: keyword indicates the beginning of filter information that is expected to begin with a WHERE clause (which is generated automatically).
The 1=1)) component completes that WHERE clause in a way that all rows are selected (you could put in something to limit the rows selected if required, but that’s not what we are trying yo achieve here)
It is then possible to add other clauses and parameters before terminating the additional text to be added with the final —
In this case an ORDER clause is added that will guarantee that rows are returned in the order selected. This causes the unique index on the table to be used to retrieve rows at the source; assuming that you code col1, col2, etc. to match the columns and their order in the index. If the index has some columns in descending order (rather than ascending) make sure that is coded in the ORDER BY statement as well.
If you code things incorrectly the generated SELECT statement will fail and you will be able to see and debug this through the log.