How to create custom sign-in reports in Entra ID

Effective identity and access management is crucial for any organization, and Microsoft Entra ID (formerly Azure Active Directory) is a leading cloud-based IAM solution offering comprehensive tools for user management, permission assignment, and activity monitoring. To leverage Microsoft Entra ID fully, administrators often require tailored insights beyond standard reports.

Creating custom reports enables deeper analysis of user behavior, access patterns, and security metrics, facilitating more informed decisions to enhance security and operational efficiency. These reports provide detailed views of sign-in activities, application usage, and other critical data points, customized to meet specific organizational needs.

However, the filters offered in Microsoft Entra ID are very specific and require exact values, which are not helpful to generate data for a range of conditions and are not user friendly. What's worse is that you will have to create these filters every time you generate the report, which can get exhausting for reports that you wish to access daily.

ManageEngine M365 Manager Plus, a comprehensive tool used for reporting, managing, monitoring, auditing, and creating alerts for critical activities in your Microsoft 365 environments, can also be used to create custom reports for Microsoft Entra ID, including custom sign-in reports for applications and users alike. You can also set up alerts for unusual sign-ins and schedule your reports to be mailed to you everyday. Curious to see how this works? Keep reading to learn how you can configure Microsoft Entra ID and M365 Manager Plus to create your custom sign-in reports.

Creating custom sign-in reports using Entra ID and M365 Manager Plus

This table is a comparison on how to create custom sign-in reports using Microsoft Entra ID and M365 Manager Plus.

Create alerts for unusual sign-ins in Microsoft Entra ID using M365 Manager Plus

Both Microsoft Entra ID and M365 Manager Plus are capable of generating custom sign-in reports. However, the purpose of a sign-in report is to identify attempts to break through your security measures. Checking the report long after the attack took place will leave you with little time to remediate the problem.

Microsoft Entra ID can notify you if high-risk users try to log in. However, any account can be compromised, including those of users who usually have a low risk. Sometimes, even these users might log in at unusual times or more often than expected. While Microsoft Entra ID provides reports on risky sign-ins, it won't alert you to these specific situations.

M365 Manager Plus is well prepared for this battle, with custom alert profiles that can email you alerts as soon as your threshold for alerts is crossed. You can set alerts for untimely hours or unusual frequencies by following the steps mentioned below:

1. Log in to M365 Manager Plus, navigate to Settings > Audit Configuration > Alert Profiles, and click Add Profile.

   

2. Type in a Profile Name and Description for your audit profile.
3. Select Azure Active Directory as your Microsoft 365 Service, choose Azure STS Logon as your Category, and Failed Logins under Actions. (You can also include Successful Logins if you want to receive alerts for untimely successful attempts).
4. Assign a Severity based on how crucial this alert is. We will select Trouble for this case.
5. Configure an alert message using Macros to use specific variables in your alert message. For this case, we will use the following message:
   "%APPLICATION_ID% %RESULT_STATUS% log in at
   %CREATION_TIME_IN_MILLIS% from %CLIENT_IP%"
6. Expand Advanced Configuration and check the Email every alert corresponding to this profile box to receive email alerts.

   

7. In the Filter Settings tab, you can configure an Alerts Threshold to alert you when a certain event occurs above a certain frequency. Use the Business Hours Filter to monitor for any alerts outside of working hours and the Filter By Column option to format the report data you will receive in your alerts.
8. Click Add to finish configuring alerts for unusual application sign-ins in Microsoft Entra ID.

   

Generate custom Microsoft Entra ID sign-in reports daily using M365 Manager Plus

1. Log in to M365 Manager Plus, navigate to Reports, and select Schedule Report.

   

2. Click Create Schedule.

   

3. Type in a Scheduler Name and select the tenant for which you want to generate and schedule this report.
4. Select My Reports in the Microsoft 365 Service drop-down and select the report that you created previously.

   

5. In the pop-up that opens, you can decide the Period for which data will be generated. You can click More Options to filter the report further and customize the Export FIle Name with macros.

   

6. Configure the schedule using the Scheduled to Run drop-downs.
7. Select the format in which you want the report to be emailed using the Export As drop-down. You can also configure the Storage Path to download the reports locally.
8. In the Notification Settings drop-down, select Notify all reports.
9. If you want the reports emailed to you, use the Select Notification Template to select a format for your emails. This option will be accessible once you configure your email server.
10. Click Save to schedule emails for custom Entra ID sign-in reports daily.

    

Limitations of using native tools to create custom sign-in reports in Entra ID

• The Microsoft Entra ID P2 license is required to access the risky sign-in reports referred in this article.
• Administrators must be assigned at least the Reports Reader role to access the reports displayed in Microsoft Entra ID.
• Reports generated using Microsoft Entra ID can only be exported in CSV and JSON formats.
• Reports will have to be filtered every time they are generated, and the filters cannot be saved, which can get exhausting if a filter is required to generate data that you require frequently.
• Unusual sign-in alerts cannot be generated for users who are at low risk.
• Assigning granular permissions to execute individual tasks in a broad category is not possible.
• Generating reports and conducting management tasks on services in Microsoft 365 have to be carried out in their respective admin centers, which can be exhausting and time-consuming.

Benefits of using M365 Manager Plus to create custom sign-in reports in Entra ID

• Gain a thorough understanding of your environment in Microsoft Entra ID, Exchange Online, SharePoint Online, OneDrive for Business, and other Microsoft 365 services with detailed reports from a single console.
• Keep tabs on even the most granular user activities in your Microsoft 365 environment.
• Delegate granular permissions to technicians without elevating their Microsoft 365 privileges, and create custom roles with any combination of reporting, management, and auditing tasks.
• Manage users, mailboxes, groups, sites, and contacts effortlessly in bulk without PowerShell scripting.
• Monitor the health and performance of Microsoft 365 features and endpoints around the clock.