How to: Postfix as mail relay with greylisting support
Posted in Administration, Linux on January 28th, 2010 by Philipp C. Heckel – Be the first to commentGreylisting is a very efficient technique for fighting spam and can reduce the spam messages in your mailbox by more than 90%. It uses the fact that most spammers only try delivering their spam-mails once, whereas real mail transfer agents (such as the ones regular e-mail service providers are using) try delivering each message up to 4-5 days before they give up.
I have always wondered why most ESPs don’t offer greylisting for their mailboxes, but only rely on less effective and resource-hungry post-retrieval filter methods. Unfortunately, my e-mail provider is one of them so that I get at least a couple of spam mails a day …
Luckily, it is very easy to set up your own mail relay with greylisting support, i.e. a mail server that simply forwards the mail to your real provider once it passes the greylist-filter.
This little tutorial describes how to set up Postfix and SQLgrey as mail relay.
1. What you need
- A dedicated or virtual private server with SSH root access.
- Access to the DNS entries of your domain for adjusting the MX record; in this post called example.com
2. How it works
If you have outsourced all e-mail services to a service provider like I have, the MX record of your domain usually points to your provider’s mail server. That is, your mails go directly to the mail server of your provider, e.g. Google’s mail server → your provider’s mail server. That is, your DNS configuration looks something like this:
$ dig example.com mx ... example.com. IN MX 50 mx0.example.com. mx0.example.com. IN A (your provider's mail server IP) ...
In order to pre-process mails with greylisting and blacklisting, your server will handle mails as intermediary, i.e., mails will always traverse your server first; in the above case something like Google’s mail server → your mail server → your provider’s mail server. Consequently, the MX record has to be changed to the IP address of your server:
$ dig example.com mx ... example.com. IN MX 50 mx0.example.com. mx0.example.com. IN A (your server IP) ...
But, first things first: we need to configure our server before we change the DNS records!
3. Installation & Configuration
If you have a Debian based system, install Postfix, SQLgrey and MySQL using apt-get:
$ apt-get install postfix sqlgrey mysql-server
This will install:
- Postfix: a fully functioning MTA which will be configured as mail relay, i.e., instead of storing arriving mails on the system, it will just greylist them and then forward them to their real destination (your provider’s mail server).
- SQLgrey: a SQL-based greylisting add-on for Postfix. Before accepting mails blindly, Postfix will ask the SQLgrey daemon whether to accept the mail or not. SQLgrey keeps track of mail delivery attempts and only replies with success if the foreign MTA tried delivering the mail at least twice.
- MySQL: a RDBMS which will be used as back-end for storing Postfix’s routing tables and SQLgrey’s caching tables. Both Postfix and SQLgrey also support other back-ends such as PostgreSQL.
3.1. Configuring SQLgrey
SQLgrey’s config files reside in /etc/sqlgrey/, the main configuration happens in /etc/sqlgrey/sqlgrey.conf. The file is well documented and offers many possibilities.
The most important options are:
- inet: IP address and port to bind the daemon to, default is 127.0.0.1:2501
- db_*: database connection details, i.e., database, user and password
- greymethod: defines which IP class to use for greylisting. Especially important for big e-mail service providers since the same mail might be delivered from two different IP addresses (Class C greylisting recommended!).
- optmethod: defines if greylisting is enabled by default (optout), or has to be enabled specifically for each address or domain (optin).
3.1.1. Config file /etc/sqlgrey/sqlgrey.conf
My configuration looks like this:
inet = 10101 # bind to localhost:10101 reconnect_delay = 5 # no reconnect before 5 minutes max_connect_age = 24 # no reconnect after 24 hours # database settings db_type = mysql db_name = sqlgrey db_host = localhost db_user = sqlgrey db_pass = sqlgreypassword db_cleandelay = 1800 clean_method = sync # 'async' is said to be buggy # greylist by class C network. eg: 2.3.4.6 connection # accepted if 2.3.4.145 did connect earlier greymethod = classc # one must optin to have its (incoming) messages being greylisted optmethod = optin
3.1.2. Database
SQLgrey has a fixed database structure which is set up automatically when the script is started. All that needs to be done is to create a new database sqlgrey with a corresponding user. You can do this manually, or with a tool like phpMyAdmin:
CREATE USER 'sqlgrey'@'localhost' IDENTIFIED BY 'sqlgreypassword'; CREATE DATABASE IF NOT EXISTS `sqlgrey` ; GRANT ALL PRIVILEGES ON `sqlgrey` . * TO 'sqlgrey'@'localhost'; FLUSH PRIVILEGES;
3.1.3. Populating the database
SQLgrey automatically creates the required tables when it starts for the first time. So start the daemon using the provided init.d-script:
$ /etc/init.d/sqlgrey start
This creates a couple of tables in the sqlgrey-database. For our purpose and configuration, the tables optin_email and optin_domain are most interesting because only domains and e-mail addresses in these tables will be greylisted.
For our example, we will enable greylisting for the whole domain example.com:
INSERT INTO `sqlgrey`.`optin_domain` (`domain`) VALUES ('example.com');
That’s it for SQLgrey. Once we connect it to Postfix, it’ll provide us with the greylisting service we want.
3.2. Configuring Postfix
Postfix is a very flexible and powerful mail transfer agent (MTA) and can be used as final destination, or mail forwarder (mail relay). For this scenario, Postfix will be a mail relay which only forwards an e-mail if
- its recipient is listed in the database
- and it passes the greylisting-filter
The configuration files of Postfix reside in /etc/postfix/. The most interesting file for our purpose is /etc/postfix/main.cf.
In order to not be confused by all the more or less useful config parameters, the file shown here is minimal, i.e., you cannot remove any parameter without major consequences. Details to each of them can be found in the Postfix configuration man-page.
The most important parameters for the configuration as mail relay are:
- myhostname: this defines your hostname, i.e., in this case relay.example.com. Regarding the hostname, two things must be considered very thoroughly:
- The hostname must resolve to the IP address of your server, i.e., make sure you don’t state a fake host here. Postfix uses this value as HELO/EHLO identification, and some mail servers might reject your mails if this value doesn’t resolve to your server’s IP address.
- If you use your top level domain here, e.g. example.com, some mail servers might additionally perform a MX lookup and match your server’s IP address with the one of the MX record. In case the MX record points to a different IP address than the A record of the TLD, foreign servers might also reject all your mails. In my case, this resulted in log entries like this:
postfix/smtp: to=<john.doe@example.com>, relay=mx.my-esp.tld:25, status=bounced (host mx.somedomain.com[1.2.3.4] refused to talk to me: 550 Forged HELO: you are not example.com)
As you can see, the foreign hosts suspected me of forging the HELO name, and denied relaying my mails.
- relay_domains: this option links to the database table relay_domains and defines the domains managed by this mail server. If a recipient-domain is not in this table, Postfix will reject the mail. For this example, the domain example.com must be added to this DB table.
- relay_recipient_maps: this option links to the database table relay_recipients and defines the e-mail addresses managed by this mail server. If a recipient address is not in this table, Postfix will reject the mail. This option is closely linked to relay_domains and will not work without it!
In this case, only one address will be added to this table: john.doe@example.com.
- transport_maps: this option links to the database table transport and defines to which mail server incoming mails will be forwarded. Routing can happen address- or domain-based.
In this case, mails for example.com shall be forwarded to our provider’s mail server, i.e., this table must have an entry of the form example.com → smtp:[mx.my-esp.tld].
For details on the value format, read the transport man page.
- smtpd_recipient_restrictions: this is where the actual magic happens. This option checks the RCPT TO field of each incoming mail, i.e., the recipient, and then queries the greylisting service. Its options closely relate to the relay_*-tables from above:
- permit_mynetworks allows local applications to send e-mails. If you have web sites running on localhost that may use e-mail, do not remove this option.
- reject_unauth_destination queries the relay_domains SQL table, i.e., it checks whether a the incoming recipient domain is relayed by our server.
- reject_unlisted_recipient queries the relay_recipients SQL table to find out if the exact address is relayed.
- check_policy_service queries the SQLgrey daemon which in turn either allows or rejects the mail.
3.2.1 Config file /etc/postfix/main.cf
Here’s the minimal main.cf config file to make Postfix a mail relay with greylisting support:
# This is a minimal main.cf config file. Make sure to read the above # comments so you understand what each option means. # server name; must resolve to your server's IP address myhostname = relay.example.com # avoid warning message 'dict_nis_init: NIS ...' alias_maps = relay_domains = mysql:/etc/postfix/mysql_relay_domains.cf relay_recipient_maps = mysql:/etc/postfix/mysql_relay_recipient_maps.cf transport_maps = mysql:/etc/postfix/mysql_transport_maps.cf smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination, reject_unlisted_recipient, check_policy_service inet:127.0.0.1:10101
3.2.2. Database
Postfix is very flexible when it comes to address and route handling. In fact, its configuration doesn’t need a database back-end at all. However, using a SQL database makes everything much easier. I decided to use a very straight forward database structure which directly relates to Postfix’ configuration options.
First, create a database postfix and a corresponding read-only user:
CREATE USER 'postfix'@'127.0.0.1' IDENTIFIED BY 'postfixpassword'; CREATE DATABASE IF NOT EXISTS `postfix` ; GRANT SELECT ON `postfix` . * TO 'postfix'@'127.0.0.1'; FLUSH PRIVILEGES;
Note: It is important that you use 127.0.0.1 as host, and not localhost, because Postfix runs in a chroot-environment and wouldn’t be able to access localhost.
After setting up the database, add the following three tables:
CREATE TABLE `relay_domains` ( `domain` VARCHAR(255) NOT NULL, `active` enum('y','n') NOT NULL DEFAULT 'y', PRIMARY KEY (`domain`) ); CREATE TABLE `relay_recipients` ( `email` VARCHAR(255) NOT NULL, `active` enum('y','n') NOT NULL DEFAULT 'y', PRIMARY KEY (`email`) ); CREATE TABLE `transport` ( `pattern` VARCHAR(255) NOT NULL, `relay` VARCHAR(255) NOT NULL, `active` enum('y','n') NOT NULL DEFAULT 'y', PRIMARY KEY (`pattern`) );
In order to connect Postfix with the database, we need to create the three config files specified above: /etc/postfix/mysql_*.cf:
# /etc/postfix/mysql_relay_domains.cf hosts = 127.0.0.1 user = postfix-read password = postfixpassword dbname = postfix query = SELECT domain FROM relay_domains WHERE domain='%s' AND active='y'
# /etc/postfix/mysql_relay_recipient_maps.cf hosts = 127.0.0.1 user = postfix-read password = postfixpassword dbname = postfix query = SELECT email FROM relay_recipients WHERE email='%s' AND active='y'
# /etc/postfix/mysql_transport_maps.cf hosts = 127.0.0.1 user = postfix-read password = postfixpassword dbname = postfix query = SELECT relay FROM transport WHERE pattern='%s' AND active='y'
Before we can now start testing our server, we need to compile these config files to Postfix compatible lookup tables. Do that by running the following command:
$ postmap /etc/postfix/mysql_*.cf
3.2.3. Populate the database
Now we have to fill Postfix’ database with the domains and addresses we’d like to relay. In particular, that means we have to add example.com to relay_domains and transport, and the full addresses to relay_recipients:
INSERT INTO `postfix`.`relay_domains` (`domain` ,`active`) VALUES ('example.com', 'y'); INSERT INTO `postfix`.`relay_recipients` (`email` ,`active`) VALUES ('john.doe@example.com', 'y'); INSERT INTO `postfix`.`transport` (`pattern` ,`relay` ,`active`) VALUES ('example.com', 'smtp:[mx.my-e-mail-service-provider.tld]', 'y');
The entry structure for each table is different. Please refer to the Postfix manual for details (cp. transport, relay_domains, and relay_recipient_maps).
Note: the database structure above is not optimal since it requires redundant entries in three different tables. Even though the structure is not perfect, I have chosen this layout to make it easily understandable!
4. Test your server
After this short configuration period it’s now time to finally start the Postfix server:
$ /etc/init.d/postfix start
To make sure you didn’t make any mistakes in the configuration, you should now check the log files. Postfix and SQLgrey both use syslog so that you should be able to determine the system’s status like this:
$ tail -n 20 -f /var/log/syslog
If the log doesn’t show any errors, we can now try if everything works as expected. To do so, simply connect to the server from your home computer via telnet:
$ telnet relay.example.com 25 Connected to relay.example.com. Escape character is '^]'. 220 relay.example.com ESMTP Postfix HELO somebody 250 relay.example.com MAIL FROM: some@address.tld 250 2.1.0 Ok RCPT TO: john.doe@example.com 450 4.7.1 <john.doe@example.com>: Recipient address rejected: Greylisted for 5 minutes
If Postfix replies with a 450 error code, i.e., relay temporarily denied, everything works just fine. On the server side, the log should output something like this:
postfix/smtpd: connect from 1-2-3-4.your-isp.tld[4.3.2.1]
sqlgrey: grey: new: 4.3.2(4.3.2.1), some@address.tld->john.doe@example.com
postfix/smtpd: NOQUEUE: reject: RCPT from 1-2-3-4.your-isp.tld[4.3.2.1]:
450 4.7.1 <john.doe@example.com>:
Recipient address rejected: Greylisted for 5 minutes;
from=<some@address.tld> to=<john.doe@example.com> ....
postfix/smtpd: disconnect from 1-2-3-4.your-isp.tld[4.3.2.1]Wait 5 minutes and try connecting again via telnet. This time, SQLgrey will detect that this is your second delivery attempt and add the sender e-mail and its IP address to the automatic white list (AWL). Postfix will accept your mail and forward it to your provider’s mail server (according to the transport-table):
postfix/smtpd: connect from 1-2-3-4.your-isp.tld[4.3.2.1]
sqlgrey: grey: reconnect ok: 4.3.2(4.3.2.1),
some@address.tld -> john.doe@example.com (00:22:42)
sqlgrey: grey: from awl: 4.3.2, some@address.tld added
postfix/smtpd: client=1-2-3-4.your-isp.tld[4.3.2.1]
postfix/cleanup: message-id=<201001...@relay.example.com>
postfix/qmgr: from=<some@address.tld>, size=422, ...
postfix/smtp: to=<john.doe@example.com>,
relay=mx.my-esp.tld[12.34.56.78]:25, status=sent, ...
postfix/qmgr: removed
postfix/smtpd: disconnect from 1-2-3-4.your-isp.tld[4.3.2.1]5. Go live: change the DNS record
Play around a little and make sure that everything works as expected. If it does, change the DNS record like described above, i.e., set the MX record of the domains to be relayed to your server’s IP address.
Note: For the first few mails, you should definitely watch the logs. If anything goes wrong, you can always change back the MX record. But be aware that DNS changes might take up to 48h!
If you have any questions, please comment below. I am open for suggestions!
