How to monitor risky sign-ins in Microsoft Entra ID

Identifying compromised accounts plays a crucial role in mitigating security vulnerabilities. Monitoring user activities—especially sign-in behaviors—may reveal patterns that suggest unauthorized access, such as logins from unusual IP addresses, unfamiliar locations, off-peak hours, or unrecognized devices. A cloud IAM solution like Microsoft Entra ID should be actively monitored to address these risks proactively before they become an incident. In this article, we’ll explore how to use Microsoft Entra ID to identify factors of risky sign-ins and how ManageEngine M365 Manager Plus can improve on your risky sign-in monitoring.

How to get a list of high-risk sign-ins in Microsoft Entra ID

Unfamiliar sign-in factors, such as a different sign-in location, anonymous IP addresses, or out of office hours activities, can be a sign your accounts may have been compromised. Microsoft Entra ID provides a built-in Risky Sign-ins report to help administrators identify and monitor any suspicious sign-ins in the last 30 days.

Note: The Microsoft Entra ID P2 license or the Entra ID Protection add-on is required to access the Risky Sign-ins reports

To access a list of high-risk sign-ins in Microsoft Entra ID, follow these steps:

1. Log in to the Microsoft Entra admin center with a user account that has at least the Reports Reader role assigned to it.
2. Navigate to Protection > Identity Protection.
3. Select the Risky sign-ins tab under the Report section.
   

Note: You can also refer to the Risky users tab to view all risky users in your organization.

Create alerts for risky sign-ins in Microsoft Entra ID using M365 Manager Plus

A sign-in report helps detect attempts to bypass your security defenses, but if it's reviewed too late, it may not give you enough time to address the issue. The Risky sign-ins tab in Microsoft Entra ID does display logins from high-risk users. However, any account, even those considered low risk, can be compromised. Low-risk users may log in at unexpected times or with unusual frequency, which cannot be viewed in this report. Even if you can create a filer to identify unusual sign-ins, you’ll need an Azure Monitor subscription to receive real-time alerts for them.

M365 Manager Plus offers a proactive solution with custom alert profiles that send email notifications as soon as your alert thresholds are exceeded. You can configure alerts for logins during non-business hours or unusual login patterns by following the steps outlined below.

1. Log in to M365 Manager Plus, navigate to Settings > Audit Configuration > Alert Profiles, and click Add Profile.
   
2. Type in a Profile Name and Description for your audit profile.
3. Select Azure Active Directory as your Microsoft 365 Service, choose Azure STS Logon as your Category, and select Failed Logins under Actions. (You can also include Successful Logins if you want to receive alerts for untimely successful attempts).
4. Assign a Severity based on how crucial this alert is. We will select Trouble for this case.
5. Configure an alert message using Macros to use specific variables in your alert message. For this case, we will use the following message to display their out of office logins: " %APPLICATION_ID% %RESULT_STATUS% log in at %CREATION_TIME_IN_MILLIS% from %CLIENT_IP%"
   
6. In the Filter Settings tab, you can configure an Alert Profile to alert you when a certain event occurs above a certain frequency. Use the Business Hours Filter to monitor for any alerts outside of working hours and the Filter By Column option to format the report data you will receive in your alerts.
7. Click Add to finish configuring alerts for unusual application sign-ins in Microsoft Entra ID.
   

Limitations of using native tools to monitor risky sign-ins in Microsoft Entra ID

• The Microsoft Entra ID P2 license or the Entra ID Protection add-on is required to access the risky sign-in reports referred to in this article.
• Administrators must be assigned at least the Reports Reader role to access the reports displayed in Microsoft Entra ID.
• Reports generated using Microsoft Entra ID can only be exported in CSV and JSON formats.
• Reports will have to be filtered every time they are generated, and the filters cannot be saved, which can get exhausting if a filter is required to generate data that you require frequently.
• Generating reports and conducting management tasks on services in Microsoft 365 must be done in their respective admin centers, which can be exhausting and time-consuming.

Benefits of using M365 Manager Plus to monitor risky sign-ins in Microsoft Entra ID

• Gain access to detailed and critical Microsoft 365 reports without any exclusive licenses.
• Delegate granular access permissions, individual tasks, and reports to technicians without elevating their Microsoft 365 privileges. Create custom roles with any combination of reporting, management, and auditing tasks.
• Export reports generated in M365 Manager Plus in not just CSV but also other presentable formats, such as HTML, PDF, and XLSX.
• Filter your reports just once and save them as custom reports that you can access in just a few clicks.
• Keep tabs on risky sign-ins and other activities not just in Microsoft Entra ID but also other services such as Exchange Online, SharePoint Online, and Microsoft Teams, all from a single console.
• Manage users, mailboxes, groups, sites, and contacts effortlessly in bulk without PowerShell scripting.
• Monitor the health and performance of Microsoft 365 features and endpoints around the clock.