How to: Get started with the Qlik Enterprise Manager Connector in Qlik Application Automation (QAA)

[AfeefaTk]

This article provides an overview of getting started with the Qlik Enterprise Manager Connector in Qlik Application Automation. 

The “Qlik Enterprise Manager” provides an intuitive management interface for the Qlik Data Integration. A graphical user interface helps you design, execute, and monitor "Qlik Replicate” and “Qlik Compose” tasks across large business landscapes, and Qlik Enterprise Manager Connector in Qlik Application Automation is created to help facilitate the same using automation. 

 

Adding the Connection in QAA 

Provide username, password, and hostname to establish the connection with the Qlik Enterprise Manager Connector connector.

Datasource-connection-setup

The username and password should be the same as the one used to log in via the Qlik Enterprise Manager console

Since Qlik Enterprise Manager is an on-premise product, the console is accessed via a private hostname/URL. To make the connector work, in this example we made the private hostname exposed externally and used the public hostname/URL to get connected with Qlik Enterprise Manager. There are various ways of making your Qlik Enterprise Manager instance publicly accessible. You will need to choose a suitable option that fits your needs and preferences.

QEM-hostname

 

Working with Qlik Enterprise Manager blocks 

To work with the blocks in QAA, please select the connector to your left and either scroll through the list, choose the filter or type the name in the search option to find the block of interest.

The endpoints can be divided into the following categories

1. Tasks 
   • Task-related endpoints can access data related to tasks – get, list, delete, run, stop, export tasks, etc. 
2. Audit Trail
   • Export Audit Trail
     • This block will return audit trail records from all servers.
     • You can retrieve audit trail data within a timeframe by passing Datetime values in Start Timestamp and End Timestamp input parameters.
     • Limitation
       • This block is currently limited to returning only 12MB of audit trail data. 
       • If the block fails with an Out of Memory or Server error, please reduce the timeframe.
   • List Audit Trails Incrementally
     • This block will return audit trail records from all servers incrementally. 
     • By default, Enterprise Manager retains audit files for 1 week or until they reach a total size of 100 MB. The audit trail retention size and age can be changed through the Enterprise Manager command line interface.
     • Limitation 
       • This block is currently limited to returning only 12MB of audit trail data. 
       • This block will become inefficient if the audit trail retention settings have been changed to accommodate maximum retention size and age.
       • If the block fails with an Out of Memory or Server error:
         • Set 'Fetch all records on first run' in the settings tab of the block to No.
         • Schedule the automation to execute this endpoint at least every 5 or 10 minutes.
         • Alternatively, schedule using another interval that accommodates the number of logs that are continuously generated by your Enterprise Manager servers.
3. Tables 
   • Table-related endpoints can access data related to tables – list table, list table statuses, reload table. 
4. Servers
   •  Server-related endpoints can access data related to servers – get, list. 
5. Raw API's 
   • RAW APIs can be made use of, to build new endpoints blocks which are not present currently – get, list, delete, post, put, etc. 

 

Raw API Blocks 

In case of any endpoints missing – you may use RAW API blocks as per the HTTP method mentioned in the documentation. 

Raw-API-Endpoints

A typical block can look like this, enter the fields necessary (as per API documentation) and test to find the successful response:

Raw-API-Request

 

Example Use Case - Send slack notifications based on replicate task failure

With the Qlik Application Automation feature, you are able to send custom notifications based on the replicate task state

Let's walk through a basic example of how to use Slack to send out a notification whenever the task fails due to a fatal error

Full automation: 

Step-by-step description:

1. Use the block 'List Tasks' to retrieve all the tasks in the replicate server
2. Add a condition block to check whether the task failed due to an error.
   
   condition-block
   
   
3. Use a variable block within the condition block to add message details of your choice which need to be sent as a notification across the slack channel. In this example we are adding task name, state, error message, stop reason, etc to the variable
   
   Message-variable
   
   
4. Use the 'Send Message' block in the slack connector to post your message to the desired channel. The detailed message stored in the variable is mapped to the 'Text' input parameter of the 'Send Message' block.
   
   Slack-send-message-block
   
   

Attached to this message you'll find a JSON file containing the above automation.

How to import an automation from a JSON file