Small Town Sysadmin
Free CJIS checklist

Field notes

Field note: a fake CEO email, a clean purge, and the leftover hybrid connectors it turned up

A CEO-impersonation email reached five people at a small Microsoft 365 shop. The purge had gotchas, and the message trace exposed connectors to a dead Exchange server.

Staff at a small company started forwarding the same message: the CEO needs a task done, reply from your personal email. It came from a random Gmail address using the CEO’s display name, the ordinary kind of impersonation scam. If you want a version of this to hand to office staff, how to spot a scam email covers that side.

It reached five people. Two copies landed in Junk and three failed to deliver. The cleanup was simple on paper, but it hit several command changes along the way, and the message trace turned up something more interesting than the phish: hybrid connectors still pointing at an on-prem Exchange server that had been shut down.

Did anyone reply?

That’s the first question, so I traced outbound mail to the sender’s address. It came back empty.

Get-MessageTraceV2 -RecipientAddress '[email protected]' -StartDate (Get-Date).AddDays(-7) -EndDate (Get-Date)

Know the limit on that answer. An empty trace means nobody replied from a company mailbox. This scam asked people to use their personal email, and a trace of your tenant can’t see that.

Use Get-MessageTraceV2 for this, not Get-MessageTrace. Microsoft’s reference says V2 replaces the old cmdlet. The V2 cmdlet searches up to 90 days back, with a maximum of 10 days in any one query.

Blocking the sender

Two Exchange transport rules: one quarantines the exact sender, and one quarantines outside mail with the CEO’s name in the From header. The parameter for the sender pattern is -FromAddressMatchesPatterns, not -SenderAddressMatchesPatterns, which doesn’t exist. It takes a regular expression, so escape the dots. Here’s the shape of the two rules, with placeholder names:

New-TransportRule -Name 'Quarantine impersonation sender' `
  -FromAddressMatchesPatterns 'scammer@example\.com' -Quarantine $true

New-TransportRule -Name 'Quarantine outside mail using CEO name' `
  -FromScope NotInOrganization -HeaderContainsMessageHeader 'From' `
  -HeaderContainsWords 'Jane Owner' -Quarantine $true

The name rule will also catch real outside mail from anyone who shares that name, including the CEO’s own personal address, so someone needs to watch the quarantine.

Purging what was delivered

A compliance search for the sender found the two delivered copies, and a second search found the internal forward. SoftDelete purges removed all three with zero failures.

The search and the purge both need the search-only session, which takes version 3.9.0 or later of the ExchangeOnlineManagement module:

Update-Module ExchangeOnlineManagement
# Open a fresh PowerShell window, then:
Connect-IPPSSession -EnableSearchOnlySession

Then search, check the count, and purge:

New-ComplianceSearch -Name 'Fake CEO mail' -ExchangeLocation All `
  -ContentMatchQuery '(From:[email protected])'
Start-ComplianceSearch -Identity 'Fake CEO mail'

# When it finishes, confirm what it found
Get-ComplianceSearch -Identity 'Fake CEO mail' | Format-List

New-ComplianceSearchAction -SearchName 'Fake CEO mail' -Purge -PurgeType SoftDelete

# Read the Results line for the item count, not just the Status
Get-ComplianceSearchAction -Identity 'Fake CEO mail_Purge' | Format-List

The last gotcha is the easy one to miss. A purge action can show Completed with an item count of 0, which means it removed nothing. When that happened, I removed the action and ran it again until the count matched the search:

Remove-ComplianceSearchAction -Identity 'Fake CEO mail_Purge'

SoftDelete leaves the message in the user’s Recoverable Items, where they can still get it back until the retention period ends. The Purge action also removes at most 10 items per mailbox and skips unindexed items. It’s meant for a phish like this, not for cleaning out mailboxes.

The side discovery: mail routed to a server that no longer exists

The trace also showed delivery failures for a few users, with an error along these lines:

DNS domain mail.contoso.com does not exist

Those users were AD-synced MailUser objects with no cloud mailbox, left over from an on-prem Exchange server that had been shut down. A HybridWizard outbound connector was still routing their mail to the dead server, and a matching inbound connector was still there too.

To list the MailUsers:

Get-MailUser -ResultSize Unlimited | Format-Table DisplayName, ExternalEmailAddress

Before touching anything, I wanted to know which of those addresses still mattered. One trace counted the bounces, and another showed who was sending and what. That separated the addresses still getting real vendor mail from the ones only getting marketing.

$failed = Get-MessageTraceV2 -Status Failed -StartDate (Get-Date).AddDays(-9) -EndDate (Get-Date) -ResultSize 5000
$failed | Group-Object RecipientAddress | Sort-Object Count -Descending | Format-Table Count, Name
$failed | Format-Table RecipientAddress, SenderAddress, Subject

Check the MX before you delete a connector

An old-looking connector isn’t automatically a dead one. If a third-party mail filter sits in front of the tenant, one of those connectors could be carrying your mail. So I checked two things first.

Resolve-DnsName contoso.com -Type MX

Get-OutboundConnector | Format-List Name, ConnectorSource, SmartHosts, TlsDomain
Get-InboundConnector | Format-List Name, ConnectorSource, TlsSenderCertificateName

The MX pointed at a third-party email filter. Both connectors had ConnectorSource set to HybridWizard, which is how Microsoft says to identify the ones its Hybrid Configuration Wizard created, and they used a certificate for the old server. Neither one belonged to the filter.

I backed up both, removed them, and sent test mail in both directions. Mail kept flowing.

Get-OutboundConnector 'Outbound connector name' | Format-List * | Out-File .\outbound-connector.txt
Get-InboundConnector 'Inbound connector name' | Format-List * | Out-File .\inbound-connector.txt

Remove-OutboundConnector 'Outbound connector name'
Remove-InboundConnector 'Inbound connector name'

This was for a server that was already off. If yours is still running, stop here and work from Microsoft’s decommissioning guides below, because order matters there and some setups need an Exchange server kept for recipient management.

What’s still open

Not everything was closed when I wrote this up. The sender still has to go on the mail filter’s block list, the client needs to tell me which stale MailUser accounts are former employees so they can be disabled in AD (they’re synced, so that’s where the change belongs), and the old Exchange system objects still need cleaning up.

What I took away from it

  • Check the MX before assuming a connector is a leftover. It takes seconds and tells you what’s really carrying mail.
  • Never trust “Completed” without the item count. A finished purge that removed nothing looks the same at a glance.
  • Trace by sender and FromIP to see the path. With a filter in front of the tenant, that shows whether a message came through the filter.
  • Back up connector settings before deleting them. A text file is a small price for knowing exactly what you removed.
  • MailUser objects with no mailbox can bounce quietly for years. A message trace is what brought these to the surface.

Sources

All articles