SPF permerror: an include without an SPF record

The SPF record looked healthy. Four DNS lookups out of ten, valid syntax, one record at the domain apex. Everything a quick SPF check looks at was fine.

The DMARC reports told a different story. Mail from the domain's main email provider passed SPF. Mail from its other services came back with a result that rarely shows up in a report: permerror. Not fail, not softfail. The record ends in ~all, and yet no unknown sender got a softfail either.

The same record, read by the same receivers on the same day, passed some senders and broke for the rest.

First suspects: lookup limit, syntax, outage, a strict receiver

The lookup limit. SPF allows ten DNS lookups per check, and the eleventh produces a permerror. It can strike in the middle of the record: senders matched before it still pass. That fits the symptom, but this record needs four. Not the lookup limit.

The syntax. One v=spf1 record, each mechanism spelled right, a single all at the end. A syntax error, or a second SPF record, breaks the record before any term runs, so the main provider fails too. The main provider passed. Not syntax.

A DNS outage. A server failure or a timeout during the check produces temperror, not permerror, and receivers try again later. These errors came back day after day for weeks. Not an outage.

A strict receiver. This suspect looked guilty. Microsoft reported permerror more often than anyone else, so a strict Microsoft parser looked likely. But Google reported the same permerror on the same domain. So did Yahoo, GMX and WEB.DE. One strict receiver does not explain five agreeing.

The clue in the order of the record

A typical affected record:

 

v=spf1 include:spf.protection.outlook.com include:spf.google.com include:sendgrid.net ~all

Put the report results next to the record. Microsoft 365 servers, covered by the first include, passed. The rest, the Google and SendGrid servers included, got permerror. No sender reached ~all: a sender that reaches it gets softfail, and the reports showed none.

The check stopped in the middle of the record, at the second include. A DNS query for that name returns NXDOMAIN, the answer for a name that does not exist:

 

;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 4211

Google publishes its SPF record at _spf.google.com, with an underscore:

 

"v=spf1 ip4:74.125.0.0/16 ip4:209.85.128.0/17 ... ~all"

The name in the record has no underscore.

The culprit: an include with no SPF record

An include is a hollow include when its target publishes no SPF record. The name may not exist at all (NXDOMAIN), or it may exist with other TXT records but no v=spf1 line. Both count the same.

That one empty lookup should be harmless, at first sight. Section 4.6.4 of RFC 7208 allows two "void lookups", DNS queries that come back empty, and fails only on the third. This record has exactly one.

The allowance does not cover includes. Section 5.2 of RFC 7208 decides what an include does with the result of the domain it points to:

Result for the included domain	The include
pass	matches
fail, softfail, neutral	does not match
temperror	returns temperror
permerror	returns permerror
none (no SPF record)	returns permerror

A domain with no SPF record has the result none, and none turns the include into a permerror on the spot. The redirect= modifier follows the same rule in Section 6.1. The two void lookups only help mechanisms that simply do not match on an empty answer, like a, mx and exists. For include and redirect the allowance is zero.

A receiver checks the terms from left to right and stops at the first one that matches or fails. That decides who is hit:

• A server covered by a term before the hollow include passes. The check does not get that far.

• The other servers, authorized or not, hit the hollow include and get permerror.

• Terms after the hollow include do not run, all included.

For DMARC, permerror is not a pass. A message from an affected server passes DMARC only with an aligned DKIM signature. Under p=quarantine or p=reject, the rest goes to spam or bounces.

The large receivers apply the same rule

One domain could be one receiver's quirk. DmarcDkim.com DNS monitoring flags hollow includes on the SPF records it watches. The DMARC reports for those domains show what each receiver did with the senders after the hollow include:

Receiver	Senders after a hollow include
Google	permerror
Microsoft (Outlook.com and Microsoft 365)	permerror
Yahoo	permerror
GMX, WEB.DE and mail.com	permerror
Zoho Mail	permerror
Fastmail	permerror
Comcast	permerror
Amazon SES	permerror
Cisco Secure Email	permerror
Trend Micro	permerror
Mimecast	softfail or fail, the result of all

In the records behind this table, the hollow include was the first empty lookup, with no void a or mx before it. The receivers did not wait for a third. Where the target has other TXT records but no SPF record, the lookup is not even void, and the result is the same permerror. Mimecast is the exception: its reports show no permerror, and the senders after the hollow include get what all says.

The open-source SPF libraries mostly agree:

Library	Used by	Hollow include
libspf2	Exim	permerror
pyspf	Postfix policyd-spf	permerror
Mail::SPF	SpamAssassin	permerror
mail-auth	Stalwart	permerror
mox spf, blitiri spf	mox, chasquid, maddy	permerror
jSPF	Apache James	permerror
Haraka	Haraka	permerror
Rspamd	mailcow, Mailu	skips the include
OpenDMARC built-in SPF	OpenDMARC without libspf2	fail

Rspamd drops the hollow include and reads on. The senders listed after it still pass, and unknown senders get the softfail or fail that all asks for. A test message to a server that skips the include proves nothing. The DMARC reports from Google and Microsoft show the real damage.

Where hollow includes come from

A hollow include is often a provider's SPF name, written slightly wrong. DmarcDkim.com DNS monitoring found these in watched SPF records:

Found by DmarcDkim.com	Correct name
include:spf.google.com	include:_spf.google.com
include:_spf.googlemail.com	include:_spf.google.com
include:_spf.outlook.com	include:spf.protection.outlook.com
include:spf.mx.cloudflare.net	include:_spf.mx.cloudflare.net
include:_spf.smtp2go.com	include:spf.smtp2go.com
include:_spf.messagingengine.com	include:spf.messagingengine.com

The underscore goes missing in one direction or the other. The other sources:

• A missing space. include:zoho.com~all points at a domain called zoho.com~all, which cannot exist. With the space, include:zoho.com ~all is two correct terms.

• A name that only sounds right. mx.microsoft.com and mx.wix.com sound like part of a provider's mail setup, but no SPF record sits behind them.

• An SPF record nobody published. A record that includes _spf.example.com, a name on the domain itself, needs a TXT record there too. The same goes for the per-domain records of SPF flattening services: set up the include, forget the record, and the include is hollow.

• A provider change that breaks old records. A provider moves its SPF record to a new name, splits it by region or product, or moves customers to a new platform, and stops publishing the old name. The setup guide shows the new name, but domains set up before the change still include the old one. The record was correct on the day it was written, and nothing in it changes. The first sign is permerror in the DMARC reports.

Other causes of permerror

A hollow include is one of several causes of permerror in RFC 7208. Where the permerror starts tells them apart.

When the senders listed first fail too, the receiver rejected the record before any term ran:

Cause	Example
Two SPF records at the same name	two TXT records that both start with v=spf1
A syntax error	ip4:192.0.2.0/33, an unknown mechanism, include: with no domain
A broken macro	exists:%(i).example.com instead of %{i}
redirect= or exp= twice	redirect=a.example.com redirect=b.example.com

When the senders listed first pass and the rest get permerror, a term in the middle failed:

Cause	Example
A hollow include or redirect	include:spf.google.com
More than ten DNS lookups	the eleventh include, a, mx, ptr, exists or redirect
A third void lookup	three a, mx or exists terms that return nothing
An mx term with too many hosts	a domain with more than ten MX hosts
A broken record behind an include	a provider record with a syntax error

Check and fix the record

1. List the includes. Look up the SPF record and note each include: and redirect= target.

 

dig +short TXT example.com

2. Query each target. Each one must return a line that starts with v=spf1.

 

dig +short TXT _spf.google.com

No v=spf1 line in the answer means a hollow include. Or run the free SPF check: it lists each hollow include, marks it in the SPF tree and greys out the rules receivers do not reach.

3. Correct the name. Take the SPF name from the provider's setup guide, letter for letter, underscore included. The correct target can carry includes of its own, so count the lookups again after the fix.

4. Or remove the include. A service the domain no longer uses has no place in the record. Remove its include instead of guessing a name.

5. Put the space back. Each term needs a space before it, all included.

6. Watch for permerror in the DMARC reports. Expect the permerror to disappear from the reports within a day or two. A permerror that stays points at a second cause further down the record: another hollow include, or a lookup count that the fix pushed over ten.

7. Monitor the record. Turn on DmarcDkim.com DNS monitoring. It checks each include when it starts watching the SPF record and again on each change, and raises a notification for each hollow include it finds. Re-run the SPF check when a provider announces a change.

A note for email providers

An email provider that retires or migrates its sending infrastructure can keep past customers' SPF records working. Leave a record at the old SPF name instead of deleting it:

 

v=spf1 ~all

An include of that name now returns softfail, and softfail does not match (the Section 5.2 table above). The check reads on to the next term, so the customer's other senders keep passing, while the retired servers lose their authorization as intended. A deleted name turns the same include into a permerror for the senders listed after it. Announce the new name, and keep the old record published for years, not weeks: customer records change slowly.

Final note

The two void lookups in RFC 7208 are a limit, not a safety margin, and includes do not get them. One include with no SPF record breaks SPF for the senders listed after it.

Three rules follow:

• Check the targets, not just the count. Ten lookups or fewer and valid syntax say nothing about whether each include target exists. Query each one.

• Read permerror as a broken record. Softfail and fail mean a sender is not authorized. Permerror means the record itself could not be read. Fix the record, not the sender. The senders that still pass show how far the check got before it broke.

• Expect the order to hide the problem. The senders listed first keep passing, so the main email provider does not notice. The services further down take the hit.

Run the free SPF check on each sending domain, then the DMARC check to see whether DKIM still carries the mail that SPF lets fall. Let DmarcDkim.com DNS monitoring flag the next edit that leaves an include hollow.