Resolving Remote desktop disconnected error : Incorrectly configured authentication and encryption settings

Objective: To resolve the Remote desktop disconnected error due to incorrect configuration of authentication and encryption settings.

Incorrectly configured authentication and encryption settings causes your server to display the 'remote desktop disconnected error' with the following error messages:

or,

This is accompanied by symptoms such as

1. Unable to establish a new RDP connection log
2. The old connection remains active even if the computer is physically disconnected. If the client logs in again, a new connection will be established and the old connection may still remain active, leaving a security loophole.

The below sections detail steps to fix this error for both the symptoms.

Configuring authentication and encryption for a connection:

To perform the following steps, you need to log in as an Administrator in RDSH server or should have been delegated the appropriate privileges.

1. On the RD Session Host server, click Start, click Administrative Tools, and open Remote Desktop Services. Navigate to Remote Desktop Session Host Configuration >> Connections.

2. Right-click the name of the connection, and select Properties.

3. In the Properties dialog box that opens, navigate to the General tab >> Security layer. Select a security method.

4. In Encryption level, select an option from the following.

   1. Low: Encrypts data sent from the client to the server using 56-bit encryption.
   2. High: Encrypts data sent between the server and the client using 128-bit encryption.
   3. Client compatible: Encrypts data sent between the client and the server at the maximum key strength supported by the client.
   4. FIPS Compliant: Encrypts and decrypts data sent between the server and client with the Federal Information Processing Standard (FIPS) 140-1 encryption algorithms, using Microsoft cryptographic modules.

   The new encryption levels takes effect the next time a user logs on.

Also, note that any encryption level settings defined in Group Policy override the above settings. Within group policy itself, System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing setting, overrides the Set client connection encryption level setting.

Note: Certificate issues can be verified by checking the View Certificate option in Remote Desktop Services Configuration console. It can be further monitored by enabling CAPI2 event logs on both the client and server computers.

CryptoAPI2 aka CAPI2 is a tool in Windows that troubleshoots certificate issues, which provides CAPI2 event logs. In the Event Viewer, navigate to Applications and Services Logs >> Microsoft >> Windows >> CAPI2 >> Operational. Right click Operational and select the option Enable logs.

Configuring RDP connection interval :

Sometimes, the session on the Remote Desktop server is not disconnected, even if it has been physically disconnected. This can be rectified by configuring a time interval until which the RDP connections needs to be kept alive without activity. The steps are as follows :

1. Open Group Policy Object Editor, by typing gpedit.msc in Run and click OK.
2. Navigate to Computer Configuration >> Administrative Templates >> Windows Components >> Remote Desktop Services >> Remote Desktop Session Host >> Connections.
3. In the right pane, double-click Configure keep-alive connection interval.
4. Check Enabled, and click OK. Close Group Policy Object Editor.