Disaster Recovery

Disaster recovery (DR) ensures availability of ServiceDesk Plus MSP when the region or site hosting the application goes down due to a disaster. It implements a hot-standby system by keeping a secondary server in the standby mode while a primary server runs the application. Moreover, the primary and secondary servers are placed in different geographical regions.

The application runs on the primary server while the secondary server monitors the activity of the primary server. When disaster strikes, the secondary server takes over and hosts the application by replicating the data from the primary server. This ensures application availability for users without any data loss.

Role Required: SDAdmin

Currently, disaster recovery is supported only for servers running on Windows Operating System.

This page discusses disaster recovery in the following topics.

Prerequisites
How does disaster recovery work?
Benefits of Launching ServiceDesk Plus MSP in DR mode
Setup Process
Set Folder Permission
Configure Disaster Recovery
Network Modifications
Upgrade/Restore
Disable DR
History

Prerequisites

Server Requirements

Two 64-bit server machines with high network connectivity for setting up ServiceDesk Plus MSP application.
Two 64-bit database server machines with high network connectivity.
ServiceDesk Plus MSP 64 bit .exe installations (preferred).
Servers used for the DR setup must have two-way read-write access for the ManageEngine folder (where ServiceDesk Plus MSP is installed).

ServiceDesk Plus MSP Configuration Requirements

FOS license must be purchased as an add-on to your application edition. You cannot view HA configurations on the Admin tab without applying FOS license to your application.
The database must be externalized from primary and secondary servers but remain accessible to both. Preferably, use MSSQL setup.
The database must be available in Always On availability groups (AG) setup, preferably in a different geographical location.
The application must be started as a service using Java Service Wrapper.
Polling and retry parameters should be the same in both servers.
Applications installed on the primary and secondary servers must both run on the same port.

How does disaster recovery work?

The disaster recovery setup involves the following components:

Primary Server - The server where the application is hosted.
Secondary Server - The backup server in standby mode that monitors the health of the primary server and also replicates files from the primary server periodically.
Database Server - The primary and secondary servers are connected to a common database server. By default, the database server is used to monitor the primary server's health via DB heartbeat mechanism. The data from the primary server is stored in the database server. This ensures that no data is lost if the primary server fails.
The primary server, secondary server, and database server are connected to a different network.
The primary server's status can be monitored in two ways:
- DB heartbeat Mechanism - Default method for monitoring the health of a peer machine. The primary server updates its status to the database server every minute. If no status update is received, it is assumed that the primary server is unavailable and the secondary server takes over.
- Ping Mechanism - Requires SSL certificate and can be configured only in the HTTPS protocol. In case of database failure, the application automatically sets up this method.
The standby server monitors the health of the primary server and replicates files from the primary server periodically.
When the primary server goes down, no status update is sent to the database server. The primary server status is updated as "Down" in the database server.
The secondary server recognizes primary server failure and instantly begins the takeover process.
The secondary server pulls the latest files from the primary server and starts as the primary server. The time taken to complete the takeover depends on the size of replication data.
When the primary server is restored back online, it starts as the standby server and the other server continues to act as the primary server. Thus, the cycle continues.

Benefits of Launching ServiceDesk Plus MSP in DR mode

Availability of ServiceDesk Plus MSP application at all times.
Smooth and automatic take-over of the failed server.
Prevents data loss in primary server in the event of server failure.
When the primary server goes down and gets restored, it will start in standby mode while the other server continues to act as the primary server. Therefore, it is not essential for administrators to restore DR setup each time a hardware/software failure occurs.

Setup process

You can set up your servers for DR configurations in two ways:
Approach 1: Set up each server individually

Install ServiceDesk Plus MSP separately in both servers and configure them separately to the same database server.

Approach 2: Set up one server and mirror the configurations to the other server

Use robocopy to mirror the server. Install the application in the primary server and execute the following command from <SDPMSP_home>\bin directory: mirrorSetup.bat <IP address of secondary server>

On mirroring the application, the folders relevant to ServiceDesk Plus MSP will be copied from the primary server to the secondary server. However, the application will not be started as a service in the secondary server. To start the application as a service,

In the command prompt, go to {SDPMSP_Home}/bin
Execute the following command: sd_service.bat -i
The service will be created in the secondary server. You can now start ServiceDesk Plus MSP as a service.

Set Folder Permission

Share the ServiceDesk Plus MSP folder from the primary server to secondary server and vice versa.
Ensure that the servers have mutual full read/write permission for this folder. You can set the folder access permission to Everyone.
To access the shared folder on your server, go to from start menu > run > \\<machineIP>\ServiceDesk Plus MSP. Provide the username and password (if needed) to make sure an IPC connection is established between the machines.
If you want to restrict the folder access to one particular user account, set the folder access to that account and open run command (Win+R) 'services.msc'.
Search for "ManageEngine ServiceDesk Plus MSP" and go to Properties.
Select the Log on tab and choose This account option.
Enter login credentials of the domain user to whom the access is given and save it.
Start the application.

Configure Disaster Recovery

Navigate to Admin > General Settings > HA Configuration.
Switch to Disaster Recovery.
Select the Enable DR startup mode check box.

Use the following pointers to configure the application in DR mode.

Attributes	Description
Primary Server IP*	Enter the IP address of the server where the application is running.
Secondary Server IP*	Enter the IP address of the standby server.
Common Alias Name*	Specify the domain name using which you can access the application. You can customize the alias name as needed.
In case of disaster notify to	Specify the email address to which the notifications must be sent in case of server failure. Notifications regarding disaster self-monitoring alerts, data synchronization alerts, and secondary server takeover alerts will be sent to this email address. You can add multiple email addresses by separating them using commas.

* Mandatory fields

The status of the servers is monitored every 5 minutes by default. In the event of server failure, an email is triggered to the configured email address. You can modify the server's listening time by following the steps below:

Navigate to {AE_home} / conf.
Open the ha.conf file and find the entry: #peer.status.check.time.period=
Uncomment the entry by removing the hashtag
Specify the required time in minutes. For example: peer.status.check.time.period=10

For the notification emails to be sent, the outgoing mail server should be configured for the application. You can customize the notification message from Translations.

Click Save and restart the application for the configurations to take effect.

After setting up DR configuration, set up HA file replication configuration.

When the secondary server takes over as the primary server, the configurations in the DR page are not auto-updated accordingly. In other words, DR configurations do not reflect which server is primary/secondary unless manually updated.

Network Modifications

Initially, the primary machine IP is mapped to the domain URL/common alias URL.

After takeover, the application will be started automatically on the secondary server but you cannot access the application via the same domain URL/common alias URL.
Map the secondary machine IP to the common alias URL/domain URL to access the application.

Upgrade/Restore

Upgrade DR

Announcement for Users using ServiceDesk Plus MSP version 6971 and above

Users upgrading to version 6971 and above are not required to mirror the updates in both the servers as the changes are automatically pushed during the upgrade.

To upgrade your application,

Invoke <SDPMSP HOME>\bin\shutdown.bat in the primary and secondary server. This will stop ServiceDesk Plus MSP.
Upgrade the build in the primary server using PPM. Click here to know more.
Mirror the settings in the secondary server via robocopy by invoking <SDPMSP HOME>\bin\mirrorSetup.bat <IP address of secondary server> - pass the secondary server IP as parameter.
Invoke <SDPMSP HOME>\bin\run.bat in both servers to start ServiceDesk Plus MSP.

Alternatively, you can also start ServiceDesk Plus MSP as a service.

Note: Ensure that you back up your files before upgrading your application. Click here to know more.

Restore DR

In the event of upgrade failure, you can restore the application to its previous version by following the steps mentioned below.

Invoke <SDPMSP HOME>\bin\shutdown.bat in the primary and secondary server. This will stop ServiceDesk Plus MSP.
Restore the data in the primary server. Click here to learn how.
Mirror the settings in the secondary server via robocopy by invoking <SDPMSP HOME>\bin\mirrorSetup.bat <IP address of secondary server> - pass the secondary server IP as parameter.
Invoke <SDPMSP HOME>\bin\run.bat in both the servers to start ServiceDesk Plus MSP.

Alternatively, you can also start ServiceDesk Plus MSP as a service.

After successfully restoring the application, DR will be disabled by default. Enable DR and restart the application again.

Why is DR disabled on restoring ServiceDesk Plus MSP and an application reboot is required?

In some cases, the application backup from one machine with DR enabled is restored in another machine. In such cases, the application will fail to start because the IP address will be incorrect in the ha.conf file. To avoid such failures, DR will be disabled by default.

Administrators can enable DR from Admin > General Settings > HA Configuration and restart the application for the changes to take effect.

Disable DR

To disable DR from the application,

Go to Admin > General Settings > HA Configuration
Switch to Disaster Recovery and uncheck the Enable DR Startup Mode option.
Invoke <SDPMSP HOME>\bin\shutdown.bat in the primary and secondary servers. This will stop ServiceDesk Plus MSP.
You can then restart the application in the servers individually by invoking <SDPMSP HOME>\bin\run.bat command. On restarting, the servers will behave individually as two separate machines.

Alternatively, you can also start ServiceDesk Plus MSP as a service in each server.

History

The History tab records the summary of operations performed on both DR and FOS configuration.