Sending e-mail alerts in Qlik Sense Enterprise Task Failures

[Clever_Anjos]

There´s a common request to any Qlik Sense Enterprise Administrator:

"Is there any way to be informed by e-mail when a reload task fails?"  Yes, there is!

Before that, let´s have some technical small talk (you can jump to session "Creating your config xml" if you want)

Qlik Sense Enterprise relies on log4Net a very mature Apache Project  that is very reliable and powerful. It, besides a lot of features  , has the concept of Appender. That concept means that the application being logged (Qlik Sense in our case) does not to take care where the log is being stored, log4net config files handles this.

You can change a config file and choose if your log are meant to be stored into a database, a flat file, a hadoop data lake and you can even send an e-mail for each error message.   Qlik Sense Enterprise help to Appenders

1 - Creating your config xml

Pick your favorite text editor and write your config file according this https://logging.apache.org/log4net/release/config-examples.html

No, wait!

I´m posting here a functional config file, you can copy and paste it.

I´m supposing you´re going to use Gmail as the SMTP but is up to you chose any working SMTP that you want

<?xml version="1.0"?>
<configuration>
    <!-- Mail appender-->
    <appender name="MailAppender" type="log4net.Appender.SmtpAppender">
        <filter type="log4net.Filter.LevelRangeFilter">
            <param name="levelMin" value="WARN" />
            <param name="levelMax" value="ERROR" />
        </filter>
        <filter type="log4net.Filter.DenyAllFilter" />
        <evaluator type="log4net.Core.LevelEvaluator">
            <param name="threshold" value="ERROR"/>
        </evaluator>

        <param name="to" value="admin@domain.com" /> <!-- please use your admin e-mail address here -->
        <param name="cc" value="someone.else@domain.com" />
        <param name="from" value="yourgmailaccount@gmail.com" /> <!-- you´re supposed to change this -->
        <param name="subject" value="Qlik Sense Task Fail!!!" />
        <param name="smtpHost" value="smtp.gmail.com" />
        <param name="port" value="587" />
        <param name="EnableSsl" value="true" />
        <param name="Authentication" value="Basic" />
        <param name="username" value="yourgmailaccount@gmail.com" /> <!-- you´re supposed to change this -->
        <param name="password" value="yourpasswordforGmail" /> <!-- you´re supposed to change this -->
        <param name="bufferSize" value="0" />
        <!-- Set this to 0 to make sure an email is sent on every error -->
        <param name="lossy" value="true" />
        <layout type="log4net.Layout.PatternLayout">7
<!-- You can change below code if you want to pretify your e-mail message -->
            <param name="conversionPattern" value="%newline%date %-5level %newline%property{TaskName}%newline%property{AppName}%newline%message%newline%newline%newline" />
        </layout>
    </appender>
    <!--Send mail on task failure-->
    <logger name="System.Scheduler.Scheduler.Slave.Tasks.ReloadTask"> <!-- Filter what kind of exceptionname is beeing captured -->
        <appender-ref ref="MailAppender" />
    </logger>
</configuration>

2 - Saving your config xml

You must save your file with LocalLogConfig.xml name and under C:\ProgramData\Qlik\Sense\Scheduler (at least until version April2018). Anything different from this will cause your configuration not work.

3 - Restarting your Scheduler Service

Check if any task is running before restarting the service!!

4 - Test

• Create an app and create an very "silly always fail" script like Store notable into [lib://neverland/notable.qvd](qvd)
• Schedule your app as usual (under QMC/Apps/[your app]/More options button/Create New Reload Task

    (https://help.qlik.com/en-US/sense/April2018/Subsystems/ManagementConsole/Content/create-reload-tasks...)

Get a cup of coffee and wait for your task fail and your alert e-mail arrives at your inbox.

This blog post was written with fundamental contribution of ile‌ that sent me a functional config file

[johnmonson]

We've attempted numerous approaches without success. Our design tools system is running on the September 2018 version of Sense, and we're concerned that might be contributing to the problem.

So far, we haven't seen any evidence that an email has been sent out. We reached out to Qlik support, but it seems this issue falls outside their scope, and they've advised us to contact our account representative instead.

[mountaindude]

@johnmonson If you are running Qlik Sense 2018-September, I would first STRONGLY recommend upgrading to a more recent version of Sense. A Sense version from 2018 is long out of support and do not get new security patches etc. 
I.e. you are using a Sense version with multiple security issues and you also do not get to use the many, many new features that have arrived since 2018.

Other than that, regarding failed reload notifications, the fundamental principle has not changed since 2018. The open source/free Butler tool described above has been around longer than 2018 and its core features remain unchanged since the beginning.
New features added, things refined - but the concept of using Log4Net log appenders for detecting and acting on specific bits of info in the Sense log files remain.

I'd first upgrade Sense, then have a think if you're happy with basic email notifications, or if you want to go the full way and get a proper monitoring solution that also (in addition to email) sent alerts and integrate with Teams, Slack, Grafana, New Relic etc.

Edit 1: 
Regarding an upgrade of Sense from a 2018 version... that's an adventure in itself.
There have been a few (2-3? 4?) of upgrade steps since then that require special attention, and that can go pretty wrong if done incorrectly.
There are good docs in the Qlik support site for all this, but those instructions have to be read and followed.

Again, just sharing how I'd go about doing that upgrade.
I would spin up a new (small) Sense environment next to the existing one, using latest Sense version.
Then export all apps from the old environment, then import the apps into the new one.
I honestly have no idea if a current Sense environment will be able to import six-year-old Sense apps... But I suspect it should work.
Definitely worth a try.

Then bring over data connections etc from the old environment, and finally do full reloads in the new environment. 
Once things work in the new environment just shut off the old one. Maybe keeping it around for a while just in case.

[mountaindude]

Cross posting to another thread I just posted in.

https://community.qlik.com/t5/Alerting/Separate-LocalLogConfig-xml-file-for-each-application-require...

It contains some additional context on the XML appender as well as pros and cons for different alerting solutions (specifically the built-in SMTP appender solution vs more feature-rich alerts using external tools).

Btw, here's a sample alert email using the built-in SMTP appender, using this XML file.

... and here is an alert email from the Butler tool for the exact same failed reload task: