The troubles of Hotmail/Gmail with your own mailserver (postfix) and how to fix them!

So you have setup your own mailserver with Dovecot and Postfix and it's working fine (DKIM, SPF etc), except for sending mails to Hotmail and Gmail.

The problem: Sending mail to Hotmail/Gmail

Guess what, you are not the exception, many people with self small hosted mailservers complain about Gmail and especially Hotmail. Their spamfilters are beyond normal to comply with, and getting it to work is a real pain. As a small mailserver your mail will get flagged as spam in no-time. But here are some very handy tips to get it working in notime!

This solution assumes you already have DKIM/SPF working correctly

Gmail fix: prefer IPv6

  1. edit /etc/postfix/main.cf and add/edit the following lines:
smtp_address_preference = any
smtp_bind_address = <YOUR_IPV4>
smtp_bind_address6 = <YOUR_IPV6>
  1. You can see your IP(v6) info via ipconfig. The lines you added allows Postfix to also use IPv6 (it will by default not apparently).
  2. Execute sudo service postfix reload and send a test mail to Gmail. In Gmail you can press the "more options" menu next to the the reply button and click "Show Original". This is the original mail including headers where you can search for:
Received: from mail.maikel.pro (mail.maikel.pro. [<YOUR_IPV6>])
  1. Gmail might need 3-4 days to fully accept your DKIM/SPF signatures. Wait and you will see that it works, sometimes postfix prefers IPv4, in that case leave it be.

Hotmail/Microsoft fix: use Mandrill/Google Apps/SendGrid and selective relay your mail

Hotmail is one of the biggest nightmares you can think of. They provide a form to unblock your IPv4 address and also to apply for their anti spam programs, but this won't help if you have a small mail server with < 100 mail a day. Believe me, I tried.

The postfix solution is a relay. Since it is impossible to get through the spam filters of Microsoft, a few free relay services apparently can. For this you first need to create an account at Mandrill/Google Apps or SendGrid (I use Mandrill and it's free, even for the huge volumes I send for Gitlab).

  1. Create an account at one of the free services and update the DNS records (DKIM and SPF) to let them send mail on your behalf.
  2. Create the file /etc/postfix/sasl_passwd and put in the following content:
[smtp.service.com] email@mydomain.com:PASSWORD
[SMTP MAIL HOST] USERNAME:PASSWORD
  1. Execute sudo postmap /etc/postfix/sasl_passwd.
  2. Add/edit the following lines in /etc/postfix/main.cf to point to the file you just created:
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd 
smtp_sasl_security_options = noanonymous

Now you have added the relayhost credentials to Postfix, now you need to choose how to relay the mail:

  • Sent all mail via the relay (defeats the purpose of good privacy, but will always work)
  • Sent all mail via the relay based on specific mail accounts (only use specific mail accounts that always will use the relay)
  • Sent only mail via the relay that needs to be sent to a Microsoft mail service (privacy friendliest, but might not always work).

Option 1: Sent all mail via the relay:

Edit /etc/postfix/main.cf and add:

relayhost = [smtp.service.com]:587

Where you need to change [smtp.service.com]:587 to your own settings: [YOUR-RELAY-SMTP-HOST]:PORT.

Option 2: Sent all mail via the relay based on specific mail accounts

Create a file /etc/postfix/relayhost_maps:

mymailaccount@mydomain.com   [smtp.service.com]
.. add 0 or more accounts here

Where you need to change [smtp.service.com] to your own settings: [YOUR-RELAY-SMTP-HOST].

Execute sudo postmap /etc/postfix/relayhost_maps.
And edit /etc/postfix/main.cf and add/edit this line:

sender_dependent_relayhost_maps = hash:/etc/postfix/relayhost_maps

To point to the new relayhost map. Now restart postfix sudo service postfix restart and send a test mail to a Hotmail/Live account. This time it will be in inbox! Finally!

Option 3: Sent only mail via the relay that needs to be sent to a Microsoft mail service (most control/most privacy friendly)

  1. Create the file /etc/postfix/transport and put in the following content:
hotmail.com smtp:[smtp.service.com]
hotmail.co.uk smtp:[smtp.service.com]
hotmail.eu smtp:[smtp.service.com]
hotmail.co smtp:[smtp.service.com]
hotmail.net smtp:[smtp.service.com]
hotmail.org smtp:[smtp.service.com]
outlook.com smtp:[smtp.service.com]
outlook.org smtp:[smtp.service.com]
outlook.co smtp:[smtp.service.com]
outlook.eu smtp:[smtp.service.com]
live.com smtp:[smtp.service.com]
live.co.uk smtp:[smtp.service.com]
live.net smtp:[smtp.service.com]
live.co smtp:[smtp.service.com]
live.org smtp:[smtp.service.com]
live.eu smtp:[smtp.service.com]
office.com smtp:[smtp.service.com]
msn.com smtp:[smtp.service.com]
microsoft.com smtp:[smtp.service.com]
microsoft.org smtp:[smtp.service.com]
microsoft.net smtp:[smtp.service.com]
microsoft.co.uk smtp:[smtp.service.com]
microsoft.nl smtp:[smtp.service.com]
microsoft.co smtp:[smtp.service.com]
bing.com smtp:[smtp.service.com]
bing.net smtp:[smtp.service.com]
bing.co.uk smtp:[smtp.service.com]
bing.co smtp:[smtp.service.com]

Where you need to change [smtp.service.com] to your own settings: [YOUR-RELAY-SMTP-HOST].

Execute sudo postmap /etc/postfix/transport.
And edit /etc/postfix/main.cf and add/edit this line:

transport_maps = hash:/etc/postfix/transport

At last execute sudo service postfix restart to reload postfix and the settings. Now all mail that needs to reach a Microsoft mail server with their Live Smartscan crap will be sent via the relay host.

Other privacy tip: Mail headers

By default Postfix sends headers along with the mail providing the recieving user with information about your mailclient and PC (e.g. Windows or Linux). If you want to disable these headers you can do the following:

  1. Create a file named header_checks in /etc/postfix.
  2. Paste and save the following content in the file:
/^Received:.*with ESMTPSA/              IGNORE
/^X-Originating-IP:/    IGNORE
/^X-Mailer:/            IGNORE
/^User-Agent:.*/ 	IGNORE

  1. Edit /etc/postfix/main.cf and add/edit the following lines to point the to the header_checks file:
mime_header_checks = regexp:/etc/postfix/header_checks
header_checks = regexp:/etc/postfix/header_checks
  1. Execute sudo service postfix reload and try it out!