3

I am working on a tool that helps the users send emails. I plan on using an MTA (Mail Transfer Agent) in the back end like AWS-SES or Sendgrid etc. In order for the emails to arrive successfully in the recipeints inboxes, the users will have to set up DKIM/SPF by configuring the DNS settings of their respective domains.

Now if I take SES for example, I know they have an API that allows me to add an "Identity" and fetch back all the necessary DNS records for it using the API. I am sure Sendgrid and other MTAs have similar APIs that allow to add identities and give back the DNS records for the user to apply.

I show the returned DKIM DNS settings to the user, and they add it to their DNS provider, and after that when they send emails, the recipeints get it correctly (without any "via amazonses.com" stuff in the headers)

Now for the sake of example - let's assume that the tool I am building is hosted on chillybilly.xyz and one of the users that uses my tool, they have a domain called frankthetank.xyz which they want to use to send emails via my platform.

When the user tries to verify his domain via my platform, I will hit that API mentioned above in AWS SES - and show something like this to the user:

enter image description here

After which they can add these CNAMES and TXT records for successful DKIM/SPF and can start sending emails. But if you look closely, they can see I am using SES because of the values of those CNAMES and TXT records. And that is something I want to avoid, instead I want to have those called something like 7nuk24xywyawocu6ctqjxmjasiaiq3vq.dkim.chillybilly.xyz which would show my branding, but in the background it will still point to the correct SES one.

Now I am aware, that such a thing is possible because when I signed up for ConvertKit, they showed me something like so:

enter image description here

Those two values in there, as you can see, are pointing towards converkit.com BUT when I run them through a DNS lookup:

https://dnschecker.org/all-dns-records-of-domain.php?query=spf.dm-5mk8zo6m.sg7.convertkit.com.&rtype=ALL&dns=google

https://dnschecker.org/all-dns-records-of-domain.php?query=dkim.dm-5mk8zo6m.sg7.convertkit.com.&rtype=ALL&dns=google

enter image description here enter image description here enter image description here

I can see that in the background it points to MX and TXT records that belong to Sendgrid. How can I achieve this? (I believe the same principles will apply for SES or any other MTA as well too)

EDIT: I tried a few things - And I set the CNAME, MX and TXT in chillybilly.xyz (my project's domain) and I pointed two CNAMES to it from frankthetank.xyz called spf.frankthetank.xyz and dkim.frankthetankxyz

https://dnschecker.org/all-dns-records-of-domain.php?query=spf.frankthetank.xyz&rtype=ALL&dns=google

enter image description here enter image description here

https://dnschecker.org/all-dns-records-of-domain.php?query=dkim.frankthetank.xyz&rtype=ALL&dns=google

enter image description here

As you can see, I was able to achieve very similar results to what ConvertKit is doing with Sendgrid. But it is not getting verified this way. :(

The only difference I see when I check those DNS lookups (links above) is that the CNAMEs also show up in the lookup for me, but not in the case of convertkit. So I think I am close to a solution, but not sure what I am missing, any ideas? :)

Rohan
  • 125
  • 3
  • 12
  • They will know which provider you outsource to.. when they see a mail delivered by that provider? – anx Mar 11 '22 at 07:12
  • Nope, that's the whole point. To remove the provider branding, and put ours. With the DKIM/SPF stuff, that happens, it says signed by client's domain so it doesn't say "via amazonses.com" or something like that anymore. – Rohan Mar 11 '22 at 07:24

3 Answers3

3

First SPF: if you replace eg. sendgrid's outgoing SPF record, which is included in your SPF-record, with your own branding you have at the same time taken on the job of following all the dynamic things sendgrid might do to their outgoing mailserver's names and addresses. You don't want to do that, so if you are sending via sendgrid you need to include sendgrid's SPF. So since you use sendgrid, there's no way you can get rid of that "include: sendgrid.net" in your SPF-record without inflicting pain on yourself.

DKIM is similar, the amazonses example is creating CNAMEs to records that exist in Amazon's DNS. If you want to replace it with your own (which you can, no problem) you have to put those records into your own DNS. Which is no problem, however, you have to somehow transfer the signing key to whoever signs those emails, and that might be a problem. With the Amazon solution they create the key and give you the link to the public key they put in their DNS that fits the private key they sign your email with.

However, if you want to remove "branding" from Amazon's or sendgrid's mailservers the only way is to buy them out, because that is done by reverse DNS, and they will NEVER point their outgoing servers to names in your DNS namespace.

flinkflonk
  • 654
  • 4
  • 5
  • Okay, can you please elaborate more on the "buy them out" part? Meaning there is another level to SES and SG that I can subscribe to, and they will give me my own dedicated mail sending servers? And then, you're saying that I will have to use "reverse DNS" (I need to check and learn about that as well now), to do what I am setting out to accomplish? The "buy them out" option - Is that like having "Dedicated IPs" option in SES - Like you warm up those IPs and then use them - Is that how I can achieve this? Maybe it gives me more control over the whole thing? Still new & learning, thanks :) – Rohan Mar 11 '22 at 11:04
  • The "buy them out" was more of a joke. Thing is, at least IPv4 addresses are somewhat rare, so I'd not hold my breath for them to give you a dedicated server (or even IP) to send from. And reverse DNS is definitely checked by receiving mailservers, as well as SPF and DKIM and in some cases even DMARC (combination of the two). Of course there's nobody holding you from running your own mail service, but that comes with certain strings attached. I am not familiar with Amazon's offerings, but even gmail and M365 (Microsoft) will use their own outlets and DNS to allow that. – flinkflonk Mar 11 '22 at 15:59
  • It's also to protect their own control over those servers. Think about it in the context of spammers, they'd love to have a service with a lot of oomf to send out their spam from their own domains, but both Microsoft and Google have outgoing mail filtering in place to prevent just that. – flinkflonk Mar 11 '22 at 16:02
  • `The "buy them out" was more of a joke.` I realised that a few mins later :'D `so I'd not hold my breath for them to give you a dedicated server (or even IP) to send from` - But they do, they have dedicated IP options that I can pay more money for, I will link to a doc in the next comment :) I can even create my own pools from these IPs depending on if I want to keep my outgoing emails separate based on categories. What I am still trying to understand is if they will allow me to do the Reverse DNS thing you talked about with the dedicated IPs :) – Rohan Mar 11 '22 at 16:51
  • https://docs.aws.amazon.com/ses/latest/dg/dedicated-ip.html – Rohan Mar 11 '22 at 16:51
  • The only big problem with using dedicated IPs instead of the big shared IP pools they have - Is that I will have to warm up my dedicated IPs and keep them warm throughout in order to sustain whatever sending patterns I want to achieve so that means more work and algos to maintian :) – Rohan Mar 11 '22 at 16:53
  • Added an edit, let me know if you have any idea :) – Rohan Mar 17 '22 at 04:58
1

So you could just do CNAME records that point to the original source record or just put the correct SPF like the convertkit example but under your domain.

or have a middleware that goes through the domain verification process in sendgrid/AWS, creating the DNS records on your domain, which you then tell your customer to CNAME to the new DNS records you created, and then completes the verification once the customer clicks a button to say they have done this

this won't stop email headers or following the DNS records to find the original provider but for an average person this is likely fine.

However I want to caveat this - layers of obfuscation actually hurts at the DNS layer.

For example SPF has an explicit hard coded 10 lookups limit in the RFC - Sendgrid themselves write about it - https://docs.sendgrid.com/ui/account-and-settings/spf-limitations

  • Original RFC. You are at high risk of hitting this issue if you do any kind of obfuscation for SPF

additional DNS lookups will increase response times for any DNS based lookup in the mail delivery flow and/or your application - this could cause much pain at many levels especially as MTA's are processing in real time, any delay has huge potential knock on impacts.

To put some maths into this, imagine a single email takes 20ms to be processed and sent via an MTA with DNS - this is taking the best case example assuming this logic holds true. You can send Approx 180k emails per hour. If we need to double this by adding at least 1 extra dns lookup this halves to 90k emails per hour. If you are reliant on volume you would need to now throw twice the amount of hardware to maintain same throughput.

anthonysomerset
  • 3,983
  • 2
  • 20
  • 24
  • Thanks for the detailed answer :) But the point where you say "have a middelware...". This is the part where I need help really. The "practical" way of creating a CNAME for my customer that points to DNS records in my domain which eventually lead to AWS. How do I achieve this? Are there any guides you can point me to? Especially the part where I can create a middleware for MX and TXT records, that's where I confused at – Rohan Mar 16 '22 at 05:48
  • Added an edit, let me know if you have any idea :) – Rohan Mar 17 '22 at 04:58
  • Hey Anthony, cna you check the EDIT I added, and see if it gives any more ideas? I still haven't found a practical solution to the problem. Thanks :) – Rohan Mar 21 '22 at 05:18
1

SG has an option called automatic_security in the Authenticate a Domain API. What this option does I think in the back end is it lets SG manage/rotate DKIM keys and other records using their DNS records. Consequently when you make this option true, they return CNAME records instead of MX and TXT records. And the MX/TXT records are managed behind the scenes by SG.

enter image description here

The convinient part about CNAME records is that other CNAME records can easily point towards them. So I pointed chillybilly.xyz -> SG CNAME records that I got back from SG, and I pointed frankthetank.xyz records -> chillybilly.xyz CNAME records and thus creating a sort of a middleware in between.

Please not that chillybilly.xyz is my app's domain and frankthetank.xyz is the client's domain. That is the assumption we are operting on while solving thie problem.

Now of course as one of the answers pointed out that this will increase the DNS lookup because of another layer in between, we still need to evaluate if doing this actually makes sense for us or if it will increase email delivery times when scaling up eventually. But the solution does work. I think its an SG only implementation tho I think. I wasn't able to make it work with AWS SES. I have a call with them soon - If I know any better, will edit this answer :)

Rohan
  • 125
  • 3
  • 12