Local POSTFIX cannot send to Gmail

[banderas20]

Hi.

I have set up a POSTFIX server in an AWS EC2 instance. If I try to send emails between local users using Sendmail, it works.

However, if I try with an external account (gmail for example), it doesn't work.

Command:

echo "test email" | sendmail [email protected]

POSTFIX Log:

postfix/smtp[2108]: CD4FE4652F: to=<[email protected]>, relay=none, delay=150, delays=0.02/0.02/150/0, dsn=4.4.1, status=deferred (connect to alt4.gmail-smtp-in.l.google.com[142.250.153.26]:25: Connection timed out)

from=<[email protected]

Telnet output:

telnet gmail-smtp-in.l.google.com 25
Trying 172.253.63.27...

Finally, here is my main.cf file:

smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = abdsrvr.ec2.internal
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mydestination = $myhostname, srvr, localhost.localdomain, , localhost
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
#inet_protocols = all
inet_protocols = ipv4

I have port 25 open in the AWS SG.

My guess is that it has something to do with the domain, since my machine is bound to a local domain (ec2.internal), without any DNS record. Maybe GMAIL and other public SMTP servers reject this kind of connections. Anyway, I'd appreciate some help on this.

Thanks!

 

[blunix]

Hi,

I have administrated postfix mailrelays on hundrets of servers for MANY years and have switched to OpenSMTPD for that functionality about 2-3 years ago. It is ORDERS OF MAGNITUTE more easy to manage and debug than postfix and I highly recommend you to switch to opensmtpd as well. I run a few mailservers as well and there also have ditched postfix for opensmtpd.

I could go on a long rant about how much postfix sucks, even if you want something as simple as a mailrelay, and I have written this blogpost on installing a mailrelay using OpenSMTPD exactly about this topic. It will explain you exactly what you need to do to setup a mailrelay using opensmtpd and how to relay mails via a large provider like gmail.

 

[blunix]

PS: On gmail you will have to flip some settings - which I don't know - in order to be able to relay via smtp. I remember that before you can send the first mail you need to do some kind of authentication, but the last time I did that was about 2 years ago.

I just remember that it was bouncing mails at first, but the bounce message, or the smtp error message with which it rejects mails contained a link on how to do that auth thing.

 

[blunix]

As to your port 25 problem:

postfix/smtp[2108]: CD4FE4652F: to=<[email protected]>, relay=none, delay=150, delays=0.02/0.02/150/0, dsn=4.4.1, status=deferred (connect to alt4.gmail-smtp-in.l.google.com[142.250.153.26]:25: Connection timed out)

That means that packets you send to 142.250.153.26 port 25 are never answered.

This most likely is unrelated to postfix but a network configuration. I have ditched aws for similar reasons I didtched postfix a few years back (because its horrible lol) but I do remember that port 25 is blocked outgoing by default, as this blogpost on aws states as well.
I think for this you have to ping the aws support, as in that you can not just fix by opening your security group, the amazon support has to manually open that for you, as its a default smtp spam prevention thing.

many IaaS providers actually do that, amongst those are hetzner (25 and 475) and IONOS cloud in Germany. A short mail / support ticket will usually do to remove this.

HOWEVER You should not relay to gmail.com over port 25 anyways but use 587 for TLS encrypted transport. That is usually reserved to SMTP Submission via TLS, as in authenticated SMTP, and nobody blocks this outgoing.

Email on servers and email relays are a "bit of a topic".. if this is confusing just ask, I've been in this hell for MANY years now, I can explain you EVERYTHING there is know about relaying emails and the troubles that come with it

 

[blunix]

My guess is that it has something to do with the domain, since my machine is bound to a local domain (ec2.internal), without any DNS record.
telnet gmail-smtp-in.l.google.com 25 | Trying 172.253.63.27... | timeout

This kind of behavior (timeout or connection refused) is pretty much always a firewall or network problem. I am assuming that you login via SSH to the instance, which suggests that the instance itself has a functional "internet connection".

It has nothing to do with the machine having a DNS record (domain) pointing at it.

 

[dcbrown73]

@blunix you will have to explain to me how OpenSMTPd is superior to postfix and primarily how it's more secure. (the primary reason to use postfix) I'm not saying OpenSMTPd is not better, I've personally never used it, but you say that it's far easier which struck me as odd.

That's likely due to coming from sendmail back in the day which was "difficult to use and easily hacked" to say the least. Postfix has always been dead simple which is why I was confused by your statement that it's easier. (it very well maybe, but I cannot image by much!)

As for the problem at hand. I generally manage postfix primarily as a relay server for my current infrastructure. (ie, my servers send email it to to be relayed onto the Internet)

A few questions to ask:

• Do you have your SMTP security setup correctly?
  • MX / SRV
  • DKIM
  • SPF Records
  • DMARC

Some SMTP servers will reverse lookup the connecting mail server to verify they are authoritative SMTP servers before they will even respond. (ie, why even talk to rogue servers)

We currently relay all email out of O365 these days from our mail relay. So I haven't setup connections like this via Postfix in quite a few years.

 

[blunix]

but you say that it's far easier which struck me as odd.
how it's more secure

The configuration language for post-fix (yes, its named fix-after) is extremely complex and split up into multiple config files. This is not required for a task as simple as a mailrelay. Having complex configuration is bad practice, not only in the python coding guidelines but in general Linux administration as well - sometimes things have to be complex, but making them uneccessarily complex often leads to errors and misconfiguration.
OpenSMTPD's config file will be FAR more easy to understand than the config chaos of postfix.

how OpenSMTPd is superior to postfix

Postfix has more functionality, which you don't need. It is only required for VERY large smtp environments, like if you run a mailprovider with hundrets of thousands of human users. Then you "might" not get around postfix in some cases.
For everything else I personally consider postfix deprecated in favor of opensmtpd. (that is an opinion based on >10 years of running mailrelays as well as company sized (~100-500 users) mailservers.

difficult to use and easily hacked"

I am sensing a theme here Complex software is bad, unless complexity is unavoidable.

relay setup

You do not need to configure ANY DNS records for a mailrelay. In fact you should not even configure your mailrelay to write the servers IP address or hostname into the mail header as this can be a security issue. The end user does not need to know that his "booking confirmation email" was send by "web7.example.com".

A mailrelays job is to give the email to a large provider, like gmail, using smtp auth, which then delivers the email. This is exactly the same as when you setup smtp auth in thunderbird (a linux email client) and send mail from there. No DNS records required at all.

Back in the day you used to set SPF records to servers which ran "mini-mailservers", and would send the mail directly to the recipients mailserver. This approach is discouraged, my blogpost explains why in detail.

Some SMTP servers will reverse lookup the connecting mail server to verify they are authoritative SMTP servers before they will even respond. (ie, why even talk to rogue servers)

You are installing a mailRELAY, not a mailserver. Google will not PTR your mailrelay-server, the same way google will not PTR your the IP address of your router when you send a mail from thunderbird.

For clarity: Mailrelays are like send-only mail-clients (like thunderbird). They login to gmail, give it a mail and say send this to [email protected]. JUST LIKE your local thunderbird does. Literally no difference.

I'd highly recommend to read my blogpost (completely, like all of it). It will not only describe in detail what a mailrelay is from a technical POV, but also how to configure one correctly. Your problem will be solved in about 30 to 45 minutes

 

[blunix]

PS: About security. I'll throw in a small rant.

It doesnt matter if you use postfix, opensmtpd or a python library to connect to gmail as that connection is TLS encrypted anyways.

The problem with mail security arises when gmail delivers the mail to "mail.example.com". Sadly a large percentage of self-run mailservers out there do not have proper TLS certificates installed, and hence most SMTP mails are send unencrypted over the internet.

I could go on a large rant about why SMTP is totally broken but this is off topic

If you are a Linux admin running webservers that require mailrelays, here are the security things you need to worry about:

• You have to configure your mailrelay to talk to gmail or office365 via TLS. Imho gmail wont accept connections without TLS, no idea about office365 (I wouldn't be supriesd if they dont).
• If somebody owns your webapp, make sure that /etc/postfix/sasl-passwords or /etc/smtpd/tables/smtp-passwords (or however you call it) is only readable to root and the linux user running your mailserver. If you want to be really secure, you can setup one single server just for relaying emails, and accept relaying connections just from the webservers / webworkers. This isnt redundant then.. This topic I can spin up for a while.. Long story short if somebody owns your webserver and the smtp password is in the postfix / opensmtpd config, you have to change that.

Other than that security here isn't really your problem, which is particularly the reason you run a mailrelay. You outsource delivery problems and delivery-security to google.

 

[blunix]

PS: Very very cool signature xD

 

[blunix]

A mailrelays job is to give the email to a large provider, like gmail, using smtp auth, which then delivers the email. This is exactly the same as when you setup smtp auth in thunderbird (a linux email client) and send mail from there. No DNS records required at all.

About this: "some" large mail providers (I'm looking at you o365) setup blacklists with IP addresses from hetzner, AWS and alike. If you try to relay from such an IP you will get an error message that contains a link to o365, where you have to apply to whitelist your IP.
I'm not sure about google but I think there was something similar.
This doesnt happen with thunderbird from your workstation as microsoft and google have a list of ISPs IP addresses, but with servers its pretty common.

 

[dos2unix]

If I recall, most commercial mail servers ( yahoo, gmail, microsoft,.. etc.. ) will only accept mail from registered DNS domains.

 

[blunix]

If I recall, most commercial mail servers ( yahoo, gmail, microsoft,.. etc.. ) will only accept mail from registered DNS domains.

The OP wants to RELAY emails from servers to gmail, which then delivers to the actual recipeint. like this:

op's server -> smtp auth -> gmail -> mail.example.com -> mail-client of mail.example.com user

This is "the same" as:

thunderbird -> smtp auth -> gmail -> mail.example.com -> mail-client of mail.example.com user

You do not even need a public IP address for this, you can do that behind NAT, like for example your workstation which you use to read this forum right now is most likely behind nat, and you can use a mail-client like thunderbird to send emails.

 

[blunix]

If I recall, most commercial mail servers ( yahoo, gmail, microsoft,.. etc.. ) will only accept mail from registered DNS domains.

Regardless of that being unrelated to the OPs actual inquery, yes, that is a correct statement. You have to configure an array of DNS-based authentication systems to be able to deliver email to pretty much any mail server, not just the large ones.

But that is SMTP delivery, not SMTP relay.

 

[dos2unix]

ou do not even need a public IP address for this, you can do that behind NAT, like for example your workstation which you use to read this forum right now is most likely behind nat, and you can use a mail-client like thunderbird to send emails.

NAT has very little to do with DNS. Your mail client on your workstation is likely in a domain even if you don't know it.
( comcast, xfinity, spectrum or something similar ).

Also my mail cient itself is logged into a domain. I have to log in to my ( yahoo, gmail, etc... ) account in order to send
or receive email.

 

[blunix]

@dos2unix

NAT has very little to do with DNS

If you are behind nat, you can not directly have a PTR record (at least not a public one, a private network one yes, but that is irrelevant for mail routed over the public internet). Yet you can connect to gmail via smtp-auth and send mails.

The same goes for a mailrelay server. It can be behind NAT, actually aws ec2 does offer networks where you are behind nat.

Your mail client on your workstation is likely in a domain even if you don't know it

My mail client on my workstation is behind two NATs - one of my qubes installation and one of my router. Truth be told there's another router in the middle (a raspberry pi). But lets not make things to complicated xD

You do not need any DNS records to smtp-auth with gmail.com to send emails.

@dos2unix I still think you have misunderstood the topic of this thread. Please read this detailed explaination of what a mailrelay does exactly to clarify the topic.

 

[blunix]

Now that I re-read the ops question.. @dcbrown73 , do you want to RELAY mails via gmail, as in you have a gmail account that you want to connect to in order to RELAY mails to [email protected], or do you want to send mails DIRECTLY from your server to [email protected]?

If its the latter you will need (at least) an SPF record, and you will need lots of time and patience, as that approach is largely discouraged by now (see my blogpost section "Another bad approach: sending emails directly from your webservers using SPF records")

I used to send mails from webservers directly via SPF myself for years, but this leads to a large number of mails being rejected, as many mailservers use shady blocklists that just block the aws ec2 public IPs entirely, and you have to manually unblock yourself using shady webforms for that to work.
And then sometimes they just block you for no reason.
I'd highly recommend you to go with the approach described in the blogpost, which will save you time and nerves.

 

[banderas20]

OMG. I was away for a few hours and I found tons of info. Many thanks.

About the outgoing port of AWS, I will replicate the exact setup with local VMs and see the results.

I will read everything carefully and post my results.

Thanks a lot to everyone!

 

[blunix]

I was away for a few hours and I found tons of info

You struck a topic that I love to hate Ask anything, I can write a book about SMTP xD

About the outgoing port of AWS, I will replicate the exact setup with local VMs and see the results.

I'm 99% sure that this is blocked by aws ec2 by default. Don't waste your time, just directly complain to the aws support. If you don't have a support plan with aws it will take them forever to answer. My recommendation here would be to switch to hetzner.de

 

[blunix]

About the outgoing port of AWS, I will replicate the exact setup with local VMs and see the results.

Some ISPs and VPNs block port 25 outgoing as well, so you might get the same result (I just tried and my VPN provider does).

Just complain to aws. I'm extremely sure its that. I used to host with AWS but "banned" them for being a crappy IaaS provider about 2 years ago. I'm very sure I remember we had to do that too.

 

[banderas20]

Now that I re-read the ops question.. @dcbrown73 , do you want to RELAY mails via gmail, as in you have a gmail account that you want to connect to in order to RELAY mails to [email protected], or do you want to send mails DIRECTLY from your server to [email protected]?

If its the latter you will need (at least) an SPF record, and you will need lots of time and patience, as that approach is largely discouraged by now (see my blogpost section "Another bad approach: sending emails directly from your webservers using SPF records")

I used to send mails from webservers directly via SPF myself for years, but this leads to a large number of mails being rejected, as many mailservers use shady blocklists that just block the aws ec2 public IPs entirely, and you have to manually unblock yourself using shady webforms for that to work.
And then sometimes they just block you for no reason.
I'd highly recommend you to go with the approach described in the blogpost, which will save you time and nerves.

Hi!

No. What I want to achieve is more simple (and maybe impossible).

From my POSTFIX mail server I want to send a mail directly to an existing account.

ops mail server -> sendmail to [email protected]