Do You Need SPF Flattening? A Guide For Dmarc And Spf Setup

SPF flattening allows your domain to be better secured and email deliverability improved. A more in depth explanation of when SPF flattening is required, how it works with DMARC and DKIM, how to keep email records up to date, and what DMARC is and why it’s important is provided.

What is SPF Flattening and What is its Purpose?

• As the configuration of email infrastructure becomes more complex, the owner of a domain needs a way to effectively authenticate email and use SPF flattening to remedy this. SPF flattening alters the DNS TXT record to remove the use of multiple “include mechanisms.” Each of these include mechanisms creates a reference to another SPF record and increases the DNS lookup count, resulting in an SPF record that is long and complex. An alternative to this is to use a static list of IP addresses authorized to send email for the domain, known as SPF record flattening.
• The necessity for a SPF record to be flattened arises from the SPF 10 DNS lookup limit detailed in RFC 7208. Some other SPF mechanisms (like a, mx, and ptr), can also trigger a DNS lookup. Exceeding the lookup limit causes mailbox providers like Google and Microsoft to set a PermError (“Permanent Error”). As a result, the email authentication and delivery fails.
• There are several third-party email vendors, like Mailchimp, SendGrid, Mandrill, Salesforce, Zendesk, Amazon SES, and Google Workspace, where domain owners use these email services but host their emails elsewhere. This results in multiple nested include statements for emails. Each include can call a recursive include mechanism, driving up the total lookup count. SPF flattening tools like PowerSPF (by PowerDMARC) automate this process and remove includes by flattening them to vendor IP addresses.

The working of SPF, DKIM and DMARC in Email Authentication

Contemporary email authentication frameworks are typically designed in a layered fashion, and include SPF, DKIM (DomainKeys Identified Mail), and DMARC (Domain-based Message Authentication, Reporting and Conformance). Each component is vital in authentication of sending domain and protecting the domain from unauthorized email spoofing.

SPF:

With SPF, a domain owner declares which IP addresses are authorized to send emails on behalf of that domain. The IP addresses are listed in the DNS TXT record as an include statement, or via an existing CNAME record. During an SPF check, an email receiving server performs a DNS lookup to determine if the sending IP is allowed.

DKIM:

DKIM leverages a cryptographic signature to emails to authenticate the sender and to ensure the integrity of the email content. Like SPF, DKIM records are also published in the DNS as TXT records.

DMARC: Domain-based Message Authentication, Reporting and Conformance

• DMARC extends the use case of DKIM and SPF. While DKIM and SPF are used to validate emails, DMARC provides reporting and instructions on what to do if an email fails validation. Using the reports generated by DMARC gives insights into how well SPF and DKIM are used.
• The interconnection of these standards contributes to the overall security of the standards and strengthens SPF and DKIM validation. With the added protections of DMARC and the challenges of SPF compliance as more SaaS applications are added, best practices like SPF flattening become essential.

SPF 10 DNS Lookup Limit: Risks, Errors and Impact on Email Deliverability

The SPF 10 lookup limit is designed to prevent DNS abuse and bottlenecks on the global DNS infrastructure. Each lookup adds latency and risk. Further, organizations using multiple email services means at least one include in SPF. Other email marketing or CRM services that integrate with email platforms can further add includes, contributing to an SPF lookup explosion.

What Happens When You Go Over The Lookup Limit?

• If your SPF record triggers more than 10 lookups:
  • You will see a PermError (SPF Permanent Error).
  • SPF authentication fails for all senders, even legitimate ones.
  • Issues are compounded if the SPF record is too large or if there are syntax errors.
  • Emails sent from Microsoft 365, Google Workspace, Salesforce, or other email providers can land in spam or be blocked.
• Increased Failure Rates in your DMARC reporting makes it difficult to enforce your DMARC policy.
• The inclusion of large SPF records may compound issues related to SPF record validation due to size limit constraints.

Email Deliverability Impact

Mailbox providers (Google, Microsoft, etc.) strictly enforce the 10 DNS lookup limit for SPF record lookups, thus routine email correspondence from your domain risks being undelivered or marked as suspicious. This is especially true for organizations with a complex email infrastructure and broad domain portfolio.

When is SPF Flattening Necessary and When is it Not?

Signs that SPF Flattening is Needed

• Your SPF record contains multiple includes referencing several third-party services.
• Nested Includes: Some third-party email service providers (SendGrid or Mandrill) have a nested include which increases lookup depth and complexity.
• The 10 lookup limit for SPF records is being approached, exceeded, or deemed inadequate based on an SPF record audit or check.
• SPF record lookups or failures are reported in your DMARC records.
• Frequent changes by your email service providers to IP addresses cause SPF records to be updated frequently.

When is SPF Flattening Unnecessary?

• You may not need SPF flattening if:
  • Your domain sends emails only through your own email infrastructure (no third-party email senders).
  • Your SPF record includes only a few mechanisms based on IP addresses.
  • You have a small, static list of email service providers, making record management simple.
  • Natelanding SPF updates are performed by your email service providers and the lookup depth remains below 10.
  • You use a hosted SPF provider.

Best Practices, Alternatives and Setup Checklist for SPF Flattening

• Automation: Use a tool, such as PowerSPF from PowerDMARC, to automatically identify new vendor IP addresses and rebuild your flattened SPF record.
• Observe Vendor IP Address Changes: Amazon SES, Google Workspace and Microsoft 365 are just a few examples of vendors that change IP addresses. Using a tool that offers automatic SPF record reflattens can help alleviate issues with record obsoleteness.
• Regular SPF Checks and Audits: Regular checks and audits of your SPF records can help identify invalid records and ensure they remain within the DNS lookup limit.
• Remove Includes: Regular record maintenance should include the removal of includes for vendors or 3rd party senders that are no longer in use.
• Limit DNS Lookup: Replace includes for IP addresses. Avoid nested includes if possible.
• Monitoring and Compliance: The use of an SPF management dashboard can assist in monitoring and validating SPF records.

SPF Flattening Risk and Mitigation

SPF Flattening Risk

• Obsoleteness of Static SPF Records: If a vendor changes its IP address, manually flattening SPF records can render them obsolete and cause SPF failures.
• Record Bloat and Size Limit: Flattening records may still be too large and cause SPF record validation to fail.

Managing Risk

• Use dynamic SPF flattening if it is available to automatically update the SPF record based on vendor IP address lists.
• Schedule SPF audits to ensure accuracy.

Other and Additional Methods

• Hosted SPF: Complexity is abstracted out by the SPF host. Lookup count is managed, and SPF records are updated.
• SPF Macros: May reduce record length, but very limiting, and still bounded by lookup limit.
• SPF Folding: A method of formatting SPF records, but does not address lookup limit.
• Optimize SPF Record: Reassess and alter all authentication mechanisms to avoid redundant, unnecessary includes.

SPF Flattening Best Practices and Automation

• Perform a thorough SPF check/audit using a trusted SPF checker.
• Identify all third-party senders, vendors and/or CRM platforms integrated with your email infrastructure.
• Tally the DNS lookup count from all include and nested include mechanisms.
• Remove unnecessary includes and update the SPF record to minimize lookups.
• Use an SPF flattening tool (like PowerSPF) to obtain a flattened SPF record, either static or dynamic.
• Verify SPF record syntax to ensure that it is valid and does not exceed the size limit.
• Update your DNS records to publish the optimized, compliant SPF record.
• Implement SPF monitoring, and automatic SPF reflattening and audits.
• Keep up with your DMARC reporting and adjust your DMARC policy based on the updates to your sending practices.