STT - Migrating Qlik Sense Client-Managed

[Troy_Raney]

Environment

• Qlik Sense Enterprise on Windows

Transcript

Hello everyone and welcome to the April edition of Techspert Talks. I'm Troy Raney and I'll be your host for today's session. Today's presentation is Migrating Qlik Sense Client-Managed with our own Sebastian Linser. Sebastian, why don't you tell us a little bit about yourself?
Hello, my name is Sebastian Linser. I'm a Principal Technical Engineer at Qlik Support. My main focus there is Qlik Sense Client-Managed, and I have been around with Qlik Support for 7 years now.
Great. Today Sebastian is going to be talking us through the whole process from start to finish when it comes to migrating Qlik Sense implementations from one environment to another; we're going to go through the whole backup process; he's going to demo that migration process; we're going to talk about certificates and highlight some best practices and things to be aware of. Sebastian, what are we going to be demonstrating today in detail?
So, we want to demonstrate how you migrate a Qlik Sense Client-Managed installation from one PC to another. So, in our case we have the scenario that we have a physical machine we need to get rid of; and we need to put it on a virtualized machine.
Okay, and we're going from the same operating system in both machines?
Yes, in our case Windows Server 2019.
So, basically when everything is finished; we should have the same apps, same tasks, everything running, but on different Hardware once we get finished?
Yes, that's the idea.
Great, can you share a little bit about your setup?
Yeah, I do have a Stand-Alone database on a server called Qlik Server 3 you can see it here in the Services list. We do have a PostgreSQL Service. Right now it's PostgreSQL 14.
Okay. So, this is the Repository Database storing all the settings for this implementation; and it's on its own machine?
Exactly, when we check in PgAdmin, the standard Qlik Sense databases: Licenses, QSMQ, QSR, and Sense Services exist; and we can also see that we have the standard user Qlik Sense Repository down here.
Okay, and that's pretty much staying the same right?
Yes, we will still stay in 14, but we have to do some minor adjustments to the database which I will talk about later.
Okay. So, where do we start with this whole migration process?
We start by connecting to the Central Node. I only have a one node system here.
Okay. So, this is the physical machine where we're starting from?
Yes, this here is our Central Node. I have one app and I have the monitoring apps. So, it's not…
Yeah, pretty simple implementation, but that's fine for demo purposes.
And we are on the February 2024 initial release.
All right.
When we scroll down here to Service cluster, we do have a share which basically serves Share for Qlik Sense.
Okay. So, the apps and log files are in a separate machine in a share drive and they're staying put?
Exactly.
Now what are the requirements for the machine will be migrating to?
You have to make sure you fit all the requirement checks: Windows Server 2016, 19, or 22. You can use Windows 10 and 11, but it's not recommended to use in-production.
And that's really just because it hasn't been thoroughly tested yet, right?
Yes.
Okay.
The processor Supporting AVX, at least if you want to use natural language.
Right.
At least 5 Gb of disc space; while I would say you should have more than 5 free before…
Yeah, bare minimum.
An SMB share. Important .NET 4.8.
With .NET can't you just have a later version installed instead?
You can have this installed as well, but you do need to install 4.8.
Right.
It will not work if you only install 6. It's okay to have multiple .NET versions.
Yeah.
You see, we used a 14 database on the second server.
Right.
We would also be able to use a 13 or 12 if we have it as a Stand-Alone, but you should not use the 12.5 bundled anymore, because it has a couple of security vulnerabilities which you want to upgrade away from.
Yeah, there's a lot of material out there about that.
And important is: there is no Internet Explorer Support anymore.
Oh yeah, TLS.
TLS is at the moment at 1.2. So, if you have Windows Server 2022, and you could potentially enable TLS 1.3; for the moment please do not. It's not yet fully TLS 1.3 compliant.
Oh it's good to be aware of. All right. So, assuming we've got all those prerequisites taken care of (especially making sure we got .NET 4.8 installed; what is the first step?
So, the first step now is to shut down Qlik Sense. I have a Stand-Alone system. So, I do not have the Qlik Sense Repository Database Service, my database is on Qlik Server 3; then we have the Service Dispatcher, which we need to stop at the end.
Okay, now the Qlik Sense is stopped is it time to back up?
Yes, up first, we open the MMC to back up our certificates. We click here on File add/remove snap-in: certificates: add my user account. So, you want to be logged in as the Service user and then you want to get the personal account of the Service user, plus the computer account.
Ah. Okay.
If this point here have any restrictions, then please ask your IT to help you out.
Just want to clarify: the Service user is the account that's actually running the Qlik Sense Services right?
Yes, and you can see it here. So, if you double-click on the Service, and you go on the log-on tab; you can find..
There it is.
And we are logged in as domain administrator. So, so now we do have the Qlik client certificate on the current user personal certificates. We would right-click: all tasks: Export. We say “Yes,” with the private key, then all extended properties. We don't need the certification pass; next; we set a password.
This is a password just for the exported package?
That is just for the key, so that not everybody can just import the key again. Then put it on the desktop; and now we call this one: Current User Qlik Client.
That's a good practice. So, you know where to import it again.
Now we need to go to local computer; and all Qlik issued certificates are issued by your server name under the installation minus CA at the end.
Cool.
So, here we have two: we have Qlik Server 1 and Qlik Service Cluster; and Export it with all the properties.
Okay, password again, now…
You're just creating a password when you type that in?
Yes, I use 1-2-3 for simplicity here. So, we call this one here: local computer personal Server.
Okay.
Finish.
And I see: just repeat the process for all those Qlik certificates?
Yes. This cluster.
All right.
And now there is one more which is under Trusted Root. You just press the letter of the server name, like in my case it's Q and you jump right into the list.
Okay.
You see that this one has a private key attached. In front, we have this little key icon. So, we do want to have the private key exported here as well.
These check boxes are really critical.
Call Trusted Root. So, we are done for the MMC part here.
All right, we've got all the certificates exported; what's next?
Next if you have your Qlik Share locally on the disc, you would back up this one and move it over to the new machine in our case we have an external. So, we can skip that step. Then we have the database.
Yeah, we've got those four databases: Licenses, QSMQ, QSR, and Sense Services. What does the documentation say about this?
We see here we would take a backup of all the databases we have for that installation.
Okay, this is listing all these commands to run through a command prompt?
Yes. So, if you would have the Embedded Database, you would run it from Program Files > Qlik Sense Repository > PostgreSQL. If you have as me a Stand-Alone database, then it's usually wherever you have installed it. So, the standard is Program Files > PostgreSQL and database version and Bin. You can copy the file path.
Great.
Open a CMD prompt as admin; CD that; and now the first database is to dump the QSR. Export the QSR database; the user for the export will be PostgreSQL. So, we have to enter the PostgreSQL password; and then the file QSR backup should be created in the C drive.
And there it is. So, just repeat for all the databases using the commands from the documentation. Is there a simpler method?
Yes, if you do have a Stand-Alone database, you have a second way. You go into properties here; and you just rename to Licenses2; QSMQ2.
Okay. So, by renaming you're basically breaking the connection to the old system.
So, I will create fresh databases right after that step. Installation on the second machine into a fresh database; and at the end, I just rename. So, the new system will talk to the old databases; but I skipped the whole import/export set.
Oh, this definitely seems like a easier way to do it.
The first database called Licenses; and then select the user Qlik Sense Repository; we go to the definition here. It's important to select Template0. We now repeat this step for all the other databases: QSMQ, Qlik Sense Repository. The Qlik Sense installer will not create databases in the stand alone instance.
Ah.
If you are Stand-Alone, you have to create those databases up front, and that is why we walk through that process together.
Okay. So, you’re just creating databases with the exact same name?
Exactly, now we have have completely fresh databases set up.
Great, what's the name of the machine in our new environment?
So, that one is a complete fresh machine: Qlik Server 2. We have to start with the MMC. So, right now we're on the new server where we don't have any certificate yet; but we need to restore the certificates before we install so that it uses those certificates during the installation.
Okay. So, that's the first step before we even install?
Yes.
So, the plan then is to take all those certificates that you exported previously and import them to this machine?
Yes, we go to current user – personal, and we import the certificate now. We browse in SenseCert; we have to change this here to PFX; and then we see here current user Qlik clients. So, we open this one up. Now it gets important. You type the password; you do not check the Enable Strong Key Protection feature. You need to check the box Mark this Key as Exportable; and you want to include all the extended properties. Next, already in this store, and we select Finish.
All right.
See now, it created certificates; and we do have a Qlik client; it will still say Qlik Server 1 which is normal because we import the old certificate from the old server even if we are now on Qlik Server 2.
Okay.
Now we go to the local computer – personal; and we do the same thing: import; PFX; say it should be exportable; finish. We can check now: we have the Qlik Service cluster; and we take one more. So, now when we double-click on one of those certificates, we see: yes, it has the private key, but it says: I don't have enough information to verify this certificate. And, yes, the certificate pass isn't there because we didn't put the root certificate yet. So, we go to Trusted Root.
Okay.
Certificates > right-click here; all tasks; import; PFX; Trusted Root. Same drill here; and Finish. Refresh once, in this list we do have the Qlik Server 1.
So, now I guess we get the installer from Qlik Community > product downloads page?
Yeah, we can say latest release and patch; and we know that we need February ’24; Qlik Sense Enterprise on Windows; we have the setup exe, and the last patch. Now we have downloaded our files. Make sure that the unblock box is checked; every time it comes from the internet, you might see the unblock, you might not see; it it's depending on your IT policies. You right-click here, and you say Run as an administrator. We will on purpose, open Task Manager just just to show there is really no Qlik Service installed here on this machine.
Okay.
We always get those kind of warnings messages: your MMC is open, your Task Manager is open, please close it before proceeding. We close the MMC, the warnings go away. You can now; sometimes it still says it's running, then the trick there is: you check here the users tab and you might see another user logged-on on the server.
Okay, good thing to be aware of.
Okay, we close the Task Manager now again, and we hit Next; Install; accept the license agreement; since we're installing a new Central Node, we create lust we leave the name as it is: Qlik Server 2 domain local.
And is that just the name of the machine?
Yes, that's the name of the machine. We do have the option to get a new database created; that would be the Embedded Database option.
Okay.
And we have the second option: to connect to an existing database. Since we know our databases on Qlik Server 3, we use that option; standard port for Qlik Sense: 4432 or whatever your admin tells you; you can change it too. Database user; password…
That all important password.
Now it will try to connect to the databases; as you see, it could. If it couldn't connect or one of the databases here would be missing; you would get an error message.
Okay, but it actually created a connection to those fresh new databas that you created yourself with exact names.
Exactly.
Okay.
In this one here, you have a couple of additional settings; leave them default; point to the root folder; in our case, we make it simple: the root folder never changed; which is the root folder: apps, archived logs, and Static; and if we go from this machine to that one: they are here. They will not be overwritten during the installation so all your apps be the same after the installation.
Is that a good step, to kind of try to navigate to that location from this machine just to make sure?
Yeah, test in this up front given again you're logged in with the Service user is able to access the share from that server you are currently on.
Okay.
Now you choose the installation folder; and now because we are migrating, we are unchecking this box. We want to change back to the other databases; and we hit Next. And we will get the Bundles customer agreement again; and we hit Install.
So, once this is finished, we still need to run that bootstrap command that changes the names from the old server name to the new one?
Yeah.
And test everything out. I like how it tells you what it's doing.
All right, we are installed.
Good.
And you set it to leave the Services unstarted. So, everything is just waiting, what do we need to do next?
And the next step is to go back to the database machine. The QSR database is basically still empty because Qlik Sense hasn't started up while 2 which is our old database that actually has all those those tables already.
Okay. So, we still need to take that step of restoring the old database to the new system?
Exactly, now we rename this one to license one and now you see why I choose the two before. So, I can keep them apart but now we go back to the databases with number two and just rename those back to the original name you could of course also delete the databases and rename them I keep them here simply because of roll backck purposes that's a.
Great, tip I definitely believe in renaming instead of deleting is it time to start up the Services on our new machine we need to start the first one which is the click Send Service Dispatcher and that one is the foundation for the Repository to fully start up. So, once that is done we open a CMD prompt as admin we need to navigate to a specific folder and here we need to have the Repository EXA which is this one.
Okay, okay copy that folder go back to the command promt paste all right. So, we're in that directory what are we going to be doing here?
Yes, we are going to do a backup restore procedure if we scroll do something if your machine has a different host name in our case we had Qlik Server 1 and now we have Qlik Server 2.
Okay. So, we have to go in and change the host name and all these settings and it says here restore the certificates to security Qlik Sense Services we did that.
Okay, install Qlik Sense on the computer we did that too start the Repository Service database in our case we have a Stand-Alone database. So, that is already done. So, which Step are we on there is this tiny bootstrap command oh and step seven.
Okay. So, we copy this one and we paste it here you will see it immediately goes away but then it should usually come back setting up all the certificates to all the correct ports and. So, on all right. So, that's going through and adjusting the certificates adjusting entries in the database as well?
Yes, it said Qlik Server 1 it's replacing it with Qlik Server 2 yeah. So, in this case here close that window we restart the Service Dispatcher once to get the new certificate in place. So, can you clarify for me again what we're checking here the bootstrap command changed the name of the certificate now to Qlik Server 2 is now it does it is Qlik Server 2 issued by Qlik Server 1 now we start the Repository as a Service we can also start the other Services as a Service is it important to start the Repository first?
Yes, the Repository is the base Service for all the others now the important thing is to watch that lock file system Repository I see it has a error contact in the license server well in that case we first need to see if the license Service actually started up it's always the one with the later time stem. So, in this case that one it is actually started up to listen on Port 92900 but it cannot communicate with the Repository Service this is actually kind of.
Great, we get to see how you would troubleshoot this. So, you're opening up net stat to check the ports first I want to see is Port 9,200 up and that one is then I want to see is Port 42 42 up and that one is not. So, there's some issue with the Repository then?
Yes, one of the reasons. So, would you just restart it or what would you try that's a good question we can try to stop it in any case we should have a look at firewall settings firewall is off. So, that it's not an issue.
Okay, then the de here is most likely our problem we did had one Repository Service still running and that might have been our command line oh?
Yeah, mode. So, now let's start this one again there we go. So, now 4242 is working at least sending something to it but we still need to see if this thing starts up. So, we need to go back to the Repository system lock H keep ret trying. So, let's see we all right. So, we restart the Dispatcher Service that tries to create that connection to licensing server and check the log.
All right, we got a new log file listening on 9200 and we see now all the files with the data left. So, the restart of the Service Dispatcher actually fixed it the thing is started up now it actually says I cannot get the other Services is it not started correct and what we also can see is we see startup phase completed which is thing we want to see. So, now back to Services we start all the other Services up.
Okay. So, the cause of this issue in this case was a command prompt that was keeping the Repository Service open and once we stopped that and restarted the Service Dispatcher things started to work as expected?
Exactly, and we might have to press control C and just closing the command prompt might have been holding back something for the license Service to restart now we should have a couple of Qlik Sense icons and we could for example open the Management console we should be able to use the same user as before domain administrator this it looking good. So, far this is working. So, we see Qlik Sense February is up the app list should show the app test one.
All right, the same apps we had on the previous environment that's good now go to the hub we have test one and we had the monitoring apps that's?
Exactly, as we had it before and we can open this app which means our engine is working.
All right, that seems like a successful migration. So, what you now start this data load you check are my data connections all still well it and working are tasks working or do I get failed tasks and I have to invest why is this not working on the new machine M reasons for that is for example obbc drivers which you had on the old server which you didn't install yet on the new server but the app itself tries to connect to a standard obbc driver?
Yeah, there could certainly be a few more details specific to each environment but now let's move on to Q&A please submit your questions to the Q&A panel on the left hand side of your on24 console we've already got a few questions coming in I'll just read them from the top.
All right, first question: after a migration, if work is accidentally continued on the old system, is there a way to migrate the additional work from the old environment to the new one or what issues might that cause?
It could potentially cause some issues if they used an Embedded Database with Qlik Sense.
Okay, you would have the database on the same machine as Qlik Sense is installed and then we would have one database on Qlik Server 1 and a second database on Qlik Server 2 in our case here we have both servers connecting to the database in Qlik Server 3 that's right it's a Stand-Alone.
Yes, you would get other problems because you potentially can't even start up. So, they would have to remigrate the database.
Okay, All right, next question: is it possible to export and import multiple apps at once?
Yeah, import is I think still one by one.
Cool, could you show us where to do that?
We do go to apps and now you see you have a couple of them. So, we can just go here then we say more actions export export with data and Export.
Okay. So, it's exporting all those apps you had highlighted in one bulk run?
Yes, it just puts them in a folder you see the folder displayed here C program data Qlik Sense Repository Export. You see that was the export I started and here are all my apps.
Very cool. Well, next question: how to apply a patch? Is this necessary on every node?
Of course, if you have a multinode system you do need to apply a patch on every node. So, your first step here is shut down all the end Services it makes installation of the patch much easier I have prepared here a little Qlik Sense update. So, we can run this one as administrator which is the February 2024 patch 2 update and here we are patch successfully applied. Now we need to wait a little bit again and then we should be able to open the QMC. that was a pretty simple process here we are now put your eyes on the bottom right click since February 2024 PCH two.
Great, well that was a pretty simple process and we need to repeat that for every node in a multi-node system?
Yes,.
All right, well let's move on to the next question: what if we need to upgrade Qlik Sense as well? Should we migrate first and then upgrade or should we upgrade first and then migrate?
That's an interesting question of course that might come up you have to stay in certain boundaries never go from a four-year-old Qlik Sense release to the latest one but if you are one year back for example you want to migrate from February 2023 to February 2024 that is possible installing directly February 2024 and then just putting the database from February 2023 back just make sure never started up at the beginning that's the most important step to check the box of do not start the Services after the installation.
Okay, it's nice to know that that's possible but for best practice would you suggest migrating first and then upgrading?
Yeah, I would go to the same version and then upgrade slowly to the next one.
Great,.
All right, move on to the next question: what are the prerequisites to migrate an app from On-Prem to Qlik Cloud?
Okay, for this we just open the QMC up again. If you have a signed license key and we do have one in our system mhm then you can set up a cloud deployment. So, you have here the option Cloud distribution you will not get this if you have a left in control number then you just simply don't see this option.
Okay, in this setup you can add your Cloud tenant if you have one if not you can talk to your account manager.
Okay, some distribution policies and you can then have a look at all the apps which are already in the cloud. So, basically the the simplest rule is everything which get reloaded On-Prem and is published you would push in a copy to the Cloud as well ah and then in your Cloud tenant you can assign some rules to say that this app always goes in a specific space for consumption your users every time. So, usually you would need some Cloud accessible data sources to reload your apps in the Cloud or you play around with Qlik Gateway but the best approach here if you need to move your users over and you still have some data On-Prem and some may be already accessible from the Cloud system would be to reload On-Prem as much as you still need and then just publish this app for consumption to the Cloud.
That's a good suggestion, I like that.
All right, next question: what about migrating our reloads and custom reloads?
Yes, the whole task list in our case it were the two operation monitors they will get automatically cuted over because that is all in the Q is all database.
Okay. So, that is part of what we did migrate?
Yes,.
Great, good to know.
Okay, next question: can we create a miror image of the Qlik Sense client manage machine as installed and working there and then reuse it on a new machine which is a newer version of Windows operating system or other new features?
Can you get more specific on the mirror image?
It seems like they're looking for just an exact copy of their implementation on a different operating system maybe to test it out.
I suppose the easiest way here would be if you take the QSR back up and then you put it on this new machine to test the behavior you can always do that you can then just apply a license on the new machine and you would be able to use that machine.
Okay. So, that's the real roadblock is the license for having two machines with the same license?
Exactly, there there's not an automated workflow to create a mirror image of the exact installation and then put it over to another machine you would still have to do the installation Step at some point and then move the database to the other machine.
Okay, next question: are monitoring apps fail to reload after migration any tips or suggestions that sounds like could be complicated.
It's a good question. So, let's go over the obvious points: you see in my case they just work mhm the reason here is in my data connections for monitor apps users is most likely the first one which comes up you have a very long connection Local Host is the machine name I use which works on every machine even though I have a different host name. Now you might not have specified localhost here you might have had Qlik Server 1. domain. local and…
the qualified domain name.
Yeah, down here in the in the proxy you would have the same in Qlik Server 1. domain. local in that case you would have simply to change this one to Qlik Server 2 since we migrated to Qlik Server 2 then you would take this whole string go back to the data connection you paste it in here now and you can hit apply. So, now we changed that connection.
So, that connection string in the the task that doesn't get updated with the bootstrap command?
That does not get updated because your data connections stay untouched.
Thank you. Okay, next question: how would this differ from moving from on Prem to an AWS machine. So, that would be?
Exactly, the same migration my system here is based on theend there for this demo. So,.
Okay, in AWS of course you have to spin up your machine first but other than that it's a Windows operating system and then you can dig in and install Qlik Sense on it.
Okay, next question: is there a similar process for migrating NPrinting? I guess a lot of customers that are On-Prem are certainly using NPrinting.
Let's ask the Oracle. NPrinting, this here backing up and restoring NPrinting that is most likely the one we want to have.
Okay.
Here are much more restrictions: NPrinting can only be restored in the exact same version and if you wanted to migrate you must follow the instructions on on upgrading.
I'll be sure to include these links because this is good documentation.
All right, we have time for one last question: is there a different process for migrating our Web Connectors QWC?
That's a good question they worked really hard to get most of the web connectors now included inside Qlik Sense itself. So, before you might have used the click web connectors different tool than Qlik Sense but nowadays you have the web connector list for SharePoint click cloud storage S3 buckets and Dropbox on that side side and you also can access the metadata storage from S3 buckets for example on the other side and most of the big query connectors and. So, on are now available inside the product itself. So,.
Okay. So, most of those things are actually built in now. So, if you're still using web connectors you're probably on an older system.
Or you have one specific connector which hasn't been ported yet. In this case, as simple as just installing up the web connectors in any case you have to install this one on the second system as well might be as simple as literally just replacing the config files.
Yeah, most of the settings are stored in deploy config which you can copy and replace.
Okay. Great, well Sebastian this has been really great. I hope this will be a good and valuable resource for people going forward when it comes time to migrate their implementation. I appreciate all the time you put into preparing this.
Thank you so much; and if there are any additional questions, feel free to reach out in the community like I will try to answer them as good as I can.
Thank you everyone. We hope you enjoyed this session. And thank you so much to Sebastian. We appreciate getting experts like Sebastian to share with us. Here's our legal disclaimer. And thank you once again. Have a great, rest of your day.