Detection: M365 Copilot Failed Authentication Patterns

Description

Detects M365 Copilot users with failed authentication attempts, MFA failures, or multi-location access patterns indicating potential credential attacks or account compromise. The detection aggregates M365 Copilot Graph API authentication events per user, calculating metrics like distinct cities/countries accessed, unique IP addresses and browsers, failed login attempts (status containing "fail" or "error"), and MFA failures (error code 50074). Users are flagged when they access Copilot from multiple cities (cities_count > 1), experience any authentication failures (failed_attempts > 0), or encounter MFA errors (mfa_failures > 0), which are indicators of credential stuffing, brute force attacks, or compromised accounts attempting to bypass multi-factor authentication.

Search

1`m365_copilot_graph_api` (appDisplayName="*Copilot*" OR appDisplayName="M365ChatClient" OR appDisplayName="OfficeAIAppChatCopilot") 
2| eval user = userPrincipalName 
3| stats count as events, dc(location.city) as cities_count, values(location.city) as city_list, dc(location.countryOrRegion) as countries_count, values(location.countryOrRegion) as country_list, dc(ipAddress) as ip_count, values(ipAddress) as ip_addresses, sum(eval(if(match(status, "(?i)fail
4|error"), 1, 0))) as failed_attempts, sum(eval(if(match(_raw, "50074"), 1, 0))) as mfa_failures, dc(deviceDetail.browser) as browser_count, values(deviceDetail.browser) as browsers_used, min(_time) as first_seen, max(_time) as last_seen by user 
5| eval first_seen = strftime(first_seen, "%Y-%m-%d %H:%M:%S") 
6| eval last_seen = strftime(last_seen, "%Y-%m-%d %H:%M:%S") 
7| where cities_count > 1 OR failed_attempts > 0 OR mfa_failures > 0 
8| sort -mfa_failures, -failed_attempts, -countries_count 
9| `m365_copilot_failed_authentication_patterns_filter`

Data Source

Macros Used

Annotations

Default Configuration

This detection is configured by default in Splunk Enterprise Security to run with the following settings:

Implementation

This detection requires ingesting M365 Copilot access logs via the Splunk Add-on for Microsoft Office 365. Configure the add-on to collect Azure AD Sign-in logs (AuditLogs.SignIns) through the Graph API data input. Ensure proper authentication and permissions are configured to access sign-in audit logs. The m365_copilot_graph_api macro should be defined to filter for sourcetype o365:graph:api data containing Copilot application activity.

Known False Positives

Legitimate users experiencing network connectivity issues, traveling employees with intermittent VPN connections, users in regions with unstable internet infrastructure, or password reset activities during business travel may trigger false positives.

Associated Analytic Story

• Suspicious Microsoft 365 Copilot Activities

Risk Based Analytics (RBA)

Risk Message:

User $user$ exhibited suspicious M365 Copilot authentication patterns with $failed_attempts$ failed login attempts, $mfa_failures$ MFA failures, and access from $cities_count$ different locations, indicating potential credential compromise or brute force attack.

References

• https://www.splunk.com/en_us/blog/artificial-intelligence/m365-copilot-log-analysis-splunk.html

Detection Testing

Replay any dataset to Splunk Enterprise by using our replay.py tool or the UI. Alternatively you can replay a dataset into a Splunk Attack Range

Source: GitHub | Version: 1