Debian: Mail Server

From ReceptiveIT
Jump to: navigation, search

Introduction

There are quite a number of options available to us in setting up a mail server, all of which can be quite confusing. What is usually refered to as a mail server, is actually a MTA, or mail transport agent. A MTA is responsible for sending mail from itself to other MTAs, but this does not implicitly include delivery to the recipients mailbox. There is usually a final delivery agent that is responsible for taking delivery of an email from a MTA and storing it in the users inbox.

Under Linux, the main mail server packages, or MTAs, that are in use are sendmail, qmail, postfix and exim. If you have read your history, you would know that sendmail has been around the longest and is probably the most used and is full of rich features. The biggest problem that sendmail has is security, and while it is getting better every day, you only have to search for sendmail vulnerabilities to start seeing red flags. Information on sendmail can be found at www.sendmail.org.

Dan Bernstein was so sick of fixing security problems in sendmail that he wrote his own MTA called qmail, with a focus on security. Dan was so confident in his product, that in March 1997 he offered a cash prize of US$500 for anyone that could find a security vulnerability that would allow a hacker to use qmail to take over another system account. That prize has not been claimed. This sounds like the best candidate for a mail server until you decide to install it. Setting up qmail is not an easy task, and can be quite overwhelming. The licensing of qmail stops the ability of people to pre-package qmail into easily installable binaries for Linux, so while qmail is a really good mail server, it is not for the feint hearted. Information on qmail can be found at www.qmail.org.

Exim is a general purpose mail server that would be sufficient on small volume sites, but my experience with exim has highlighted scalability issues, and I have not revisited this since. Information on exim can be found at www.exim.org.

Postfix started life as the IBM secure mailer, and it attempts to be a fast, easy to administer, secure MTA that is sendmail compatible. By sendmail compatible, I mean that postfix has a sendmail-ish look from the outside, with a completely different inner core, which is great for keeping upgrade paths open. Postfix, like sendmail, is a MTA only and does not handle final delivery, but is substantially more secure, and exponentially easier to install, configure and maintain. Information on postfix can be found at www.postfix.org.

I have personally used sendmail, qmail and postfix in production servers and I have formed my own opinions that may or may not be completely correct. Due to the development process that software receives in the open source community, a software packages shortcomings can be completely turned around in a short amount of time. I therefore urge you to thoroughly test all software, and make your own opinions before attempting to use any software in a production environment.

MTA Installation - Postfix

All *nix opeating systems will have a pre-installed MTA, which will have to be removed before installing Postfix. Luckily the Debian APT tool can handle dependencies correctly and will automatically uninstall any MTA software before installing Postfix.

Note: You must be root to install system wide software

Install Postfix

apollo:~$ apt-get install postfix

An info screen will come up once all the needed files have been downloaded and the installer starts.

Configuration type?
Internet site
Where should mail for root go?
[email protected]
Mail name?
mail.fqdn.com
Append .domain to simple addresses?
NO
Other destinations to accept mail for?
fqdn.com, mail.fqdn.com, localhost.localdomain, localhost
Force syncronous updates on mail queue?
NO

Let's not forget to add the postfix user to the mail group. This is not immediately apparent, but it will help later on when using lmtp to send mail to the mailbox.

apollo:~$ adduser postfix mail

Congratulations. You have successfully installed your first mail transport agent. Wasn't that hard now, was it? How do we know that Postfix is really running? We could connect to the SMTP (simple mail transport protocol) port and find out.

apollo:~$ telnet 127.0.0.1 25

You should see something similar to

Trying 127.0.0.1...
Connecting to 127.0.0.1...
Escape character is '^]'.
220 mail.fqdn.com ESMTP Postfix (Debian/GNU)

If you don't get a successful connection, it is possible that something has gone wrong. We should try to restart Postfix.

apollo:~$ /etc/init.d/postfix restart
Stopping mail transport agent: Postfix
Starting mail transport agent: Postfix
apollo:~$ telnet 127.0.0.1 25

I mentioned above that Postfix is a MTA and does not deal with final delivery, so we need to tell Postfix what software will take care of final delivery. Cyrus is a popular final delivery agent that has many great features. We need to modify the Postfix configuration to send emails for final delivery to Cyrus. For additional security, we will also change the default SMTP banner which would tell any hacker what software we are running, set a maximum message limit and implicitly allow our local subnet to relay mail through our server only.

 apollo:~$ pico /etc/postfix/main.cf

Make sure that the following options are included.

smtpd_banner = mail.fqdn.com ESMTP Avoid the gates of hell, use Linux!
mydomain = fqdn.com
mynetworks = 127.0.0.1/8, 192.168.0.0/24
mailbox_transport = lmtp:unix:/var/run/cyrus/socket/lmtp
message_size_limit = 20000000
local_recipient_maps =

We also need to check the master Postfix configuration to make sure cyrus support is included

apollo:~$ pico /etc/postfix/master.cf

Make sure that the following option is included. Take special note that the n is included, as this forces the lmtp process to be not started in a chroot jail.

lmtp      unix  -       -       n       -       -       lmtp

After applying the configuration changes, we will need to restart Postfix.

apollo:~$ /etc/init.d/postfix restart

Stopping mail transport agent: Postfix
Starting mail transport agent: Postfix

Final Delivery - Cyrus Installation

Since our Postfix SMTP server is now running and waiting for mail to arrive, we had better install and configure a final delivery agent. In my experience, I have found Cyrus to be one of the best all round final delivery agenty.

Install Cyrus

apollo:~$ apt-get install cyrus-imapd-2.2 cyrus-admin-2.2 sasl2-bin

As soon as the installer beings, you will be asked

Hesiod domain for searches?
.fqdn.com

Cyrus executes as the user cyrus, and therefore the user cyrus us a logical administrative account for the mail server. We need to set a password for the cyrus user so we can create mail accounts.

apollo:~$ saslpasswd2 cyrus
Password: ********
Again (for verification): ********

We have to do some minor configuration changes to allow the user cyrus to have access to the mail store. Like all Linux software, configuration is done by editing the correct text file.

apollo:~$ pico /etc/imapd.conf

Make sure that the following options are included.

sasl_mech_list: PLAIN
sasl_pwcheck_method: saslauthd
sasl_saslauthd_path: /var/spool/postfix/var/run/saslauthd/mux
admins: cyrus

After all configuration file changes, we should tell the server software to restart.

apollo:~$ /etc/init.d/cyrus2.2 restart

We also need the authentication server software running to authenticate local and remote user requests. We have told Cyrus that we will be using saslauthd, so let's configure and start it. First we need to edit the default configuration file.

apollo:~$ pico /etc/default/saslauthd

Make sure that the following options are included.

START=yes
MECHANISMS="sasldb"

Next we have to start the authentication daemon.

apollo:~$ /etc/init.d/saslauthd restart
Starting Sasl Authentication Daemon: saslauthd.

Cyrus does not require that a mail user have a system account. This is a big security advantage as a mail user does not necessarily have a shell account on the server. Lets log into the cyrus admin panel. The IMAP password is the sasl password for the user cyrus that we set a little while ago.

apollo:~$ cyradm -u cyrus localhost
IMAP Password: ********

We are now in the Cyrus administration panel. Here we can modify everything to do with final delivery. If you simply type the command help, you will be given a comprehensive list of available commands. For now, lets create a mail user. Note that the command we use is cm (create mailbox) and the user we create is user.test. If we type cm joe, Cyrus would create a shared IMAP folder called joe that potentially any mail user could access. More on this later.

apollo.fqdn> cm user.test
apollo.fqdn> quit

We now have a working mail user account, but the user cannot log in yet. We need to set a password, and because it is for a mail user, we will use saslpasswd2

apollo:~$ saslpasswd2 test
Password ********
Again (for verification): ********

We should now have a fully working SMTP+IMAP mail server with a single user called test. We should test the mail server in a variety of different situations. For a mail server to be included in email delivery, it needs to be listed as an MX record in the domain DNS. If your mail server is not listed as an MX record then remote mail will not get to your mail server directly. If there is another mail server that is collecting mail for you, like your internet service proviers email server, you can use POP3 mail collection to fetch the mail for you. If your intention is for a POP3 mail collection server, your tests for remote SMTP will behave differently.

LDAP Authentication

You can have saslauthd authenticate from LDAP by simple changing the /etc/default/saslauthd file to include

MECHANISMS="ldap"

/etc/saslauthd.conf

ldap_auth_method: bind
ldap_servers: ldap://127.0.0.1:389
ldap_version: 3
ldap_timeout: 10
ldap_time_limit: 10
ldap_scope: sub
ldap_search_base: ou=users,dc=bigdomain

Security Lockdown Content Filtering and Authentication

These days, mail servers are bombarded with SPAM and viruses, and as responsible mail server administrators, we should do something about shielding our users from these nasty things. There are many different viewpoints on just how far a mail server administrator should go to protect their users. By its very nature, content filtering implies that some email messages will get delivered to the end user, and some will not. Whilst users believe that all spam should be blocked, the same user would probably get quite cranky if an important email they were waiting for was marked as bad and never delivered. You must ultimately strike a balance on just how tough you are on spam.

Authenticating SMTP

First up, we need to install some dependencies. This might already be installed, but it doesn't hurt to document it here.

apt-get install libsasl2-modules

Edit /etc/postfix/main.cf and add the following

smtpd_sasl_auth_enable = yes
smtpd_sasl_authenticated_header = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain =
smtpd_sasl_path = smtpd
broken_sasl_auth_clients = yes
smtpd_sender_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unknown_sender_domain
smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination

Create /etc/postfix/sasl/smtpd.conf with the following

pwcheck_method: saslauthd
mech_list: PLAIN LOGIN

We need to allow access for the chrooted Postfix to read the sasldb file. On Debian, the init scripts already copy a whole heap of files at startup, so they are available in the chroot. Edit /etc/init.d/postfix and edit the FILES variable to add etc/sasldb2. It will look something like this.

FILES="etc/localtime etc/services etc/resolv.conf etc/hosts \
 etc/nsswitch.conf etc/nss_mdns.config etc/postfix/sasl/smtpd.conf"

Join user postfix to the sasl group

apollo:~$ adduser postfix sasl

Restart Postfix

apollo:~$ /etc/init.d/postfix restart

Content Filtering using Amavis

The first step to content-filtered bliss is to, you guessed it, install a content filtering application. We will be using Amavis

apollo:~$ apt-get install amavisd-new

We need to modify /etc/postfix/main.cf to include the following line.

content_filter = smtp-amavis:127.0.0.1:10024

We need to modify /etc/postfix/master.cf to include the following lines.

# AV scan filter (used by content_filter)
smtp-amavis      unix  -       -       y       -       2      smtp -o smtp_data_done_timeout=1200

# For injecting mail back into postfix from the filter
127.0.0.1:10025 inet  n -       n       -       16      smtpd
   -o content_filter=
   -o local_recipient_maps=
   -o relay_recipient_maps=
   -o smtpd_restriction_classes=
   -o smtpd_client_restrictions=
   -o smtpd_helo_restrictions=
   -o smtpd_sender_restrictions=
   -o smtpd_recipient_restrictions=permit_mynetworks,reject
   -o mynetworks=127.0.0.0/8
   -o strict_rfc821_envelopes=yes
   -o smtpd_error_sleep_time=0
   -o smtpd_soft_error_limit=1001
   -o smtpd_hard_error_limit=1000

This would be an ideal time to make sure that everything still works ok. If you send an email destined to this mail server, and then check the log file located at /var/log/mail.log, we should see a line with an entry something like

Oct 25 19:01:44 localhost amavis[15618]: (15618-01) Passed CLEAN

That is good news, as it means that postfix has received the message, passed it onto amavis for checking, marked it as good, and hopefully passed it back to postfix for delivery. You should now check to make sure that the email actually got delivered into the mailbox.

Anti-Virus using ClamAV

Whilst our Linux mail server will generally not be infected by viruses attached in an email, our users might not be so lucky. While our users should have their own anti-virus on their PC, it makes sense to not deliver emails with viruses attached, as they are almost always unwanted. We will install a virus scanner on our mail server, with the intent of dropping virus infected emails.

These days, everyone with an email account is affected by spam, it just depends on the magnitude of their spam problem. We can also do some spam filtering at the mail server, to reduce the effect of spam on our users.

We need to install our anti-virus application, ClamAV.

apollo:~$ apt-get install clamav clamav-daemon clamav-freshclam

There will be some post-install questions we need to answer. ClamAV keeps up-to-date using an application called clamav-freshclam. If you have an always-on internet connection, it would be a good idea to install clamav-freshclam as a daemon. You will also need to provide your nearest mirror for ClamAV updates, and any proxy information that may be needed on your network. Answer YES to Should clamd be notified after updates?

We also need some decompression tools to look inside of email attachments

apollo:~$ apt-get install unzip bzip2 gzip cabextract arc p7zip unrar arj

Since clamav, our new virus scanner, runs as its own underprivileged user, we need to add the clamav user to the amavis group.

apollo:~$ adduser clamav amavis

We now need to tell amavis to actually scan for viruses. Amavis knows about clamav by default, and just needs anti-virus turned on. Edit /etc/amavis/conf.d/15-content_filter_mode and uncomment the following lines.

@bypass_virus_checks_maps = (
  \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);

Restart amavis

apollo:~$ /etc/init.d/amavis restart

Restart ClamAV

apollo:~$ /etc/init.d/clamav-daemon restart

Check /var/log/mail.log for the following lines

Oct 25 19:14:01 localhost amavis[15927]: ANTI-VIRUS code      loaded
Oct 25 19:14:01 localhost amavis[15927]: Using internal av scanner code for (primary) ClamAV-clamd
Oct 25 19:14:01 localhost amavis[15927]: Using internal av scanner code for (primary) check-jpeg
Oct 25 19:14:01 localhost amavis[15927]: Found secondary av scanner ClamAV-clamscan at /usr/bin/clamscan

Spam filtering

SpamAssassin

apollo:~$ apt-get install spamassassin

Once SpamAssassin is installed, we need to enable it by editing /etc/default/spamassassin, and set;

ENABLED=1

Start SpamAssassin

apollo:~$ /etc/init.d/spamassassin start

Amavis and spamassassin have fairly good, modest defaults, and therefore we only need to turn spam filtering on in amavis. Edit /etc/amavis/conf.d/15-content_filter_mode and uncomment the following lines.

@bypass_spam_checks_maps = (
  \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);

Restart amavis

apollo:~$ /etc/init.d/amavis restart

Check /var/log/mail.log for the following lines

Oct 25 19:17:35 localhost amavis[15958]: ANTI-SPAM code       loaded
Oct 25 19:17:35 localhost amavis[15958]: ANTI-SPAM-SA code    loaded

Realtime Blackhole List (RBL)

The use of an RBL to filter our mail is a little risky and expensive. Risky because what we are basically doing is giving our trust to the RBL managers over what emails get dropped. Expensive because every email that triggers an RBL sends some traffic to the RBL provider, and on a busy mail server, this can be a substantial amount of traffic. That said, they can be effective in reducing spam, and with spam making up 95% of all email, this is a good thing.

Simply put this entry into your main.cf file to enable spamhaus and spamcop RBLs

smtpd_recipient_restrictions =
  permit_sasl_authenticated,
  permit_mynetworks,
  reject_unauth_pipelining,
  reject_non_fqdn_recipient,
  reject_unknown_recipient_domain,  
  reject_unauth_destination,
  reject_rbl_client zen.spamhaus.org,
  reject_rbl_client bl.spamcop.net,
  permit

Automated script for purging virusmails directory

Solution Your /var/lib/amavis/virusmails directory can grow quite large over time. There are potential company policies on email retention that may include items in these directories and in some cases even local law regulates what can be purged from the server. There are ways you can "automate" purgine the virusmails folder using cron.

  1. Open a command line terminal and log in as root
  2. Edit your crontab by typing:
     # crontab -e <enter>
  3. Move your cursor to the bottom line of the crontab and hit "o" to start a new "insert" line.
  4. Type in the following:
     3 0 * * * find /var/lib/amavis/virusmails/ -mtime +30 -type f -exec rm -f {} \;
  5. Press "ESC" followed by "Shift :wq" <enter> to "write" and "quit" the crontab edit.

The contents of this entry into the crontab are as follows:

   * The first entry - the number "3" represents minutes within the hour - i.e. 3 minutes after the hour.
   * The second entry is the hour.  This figure is in 24 hour format with 0 being midnight, 1 being 1AM on through to 23 being 11PM.
   * The third entry (represented by an *) is the day of the month with values ranging from * (wild card meaning every day), to 1 through 31.
   * The fourth entry is the month.  As in the previous entry an * is a wild card meaning every month.  The acceptable values are *, 1 - 12.
   * The fifth entry is the day of the week.  Acceptable values are *, 0 - 6 (0 = Sunday, 1 = Monday, etc.).
   * The "find" command searches for files within a directory.
   * The next entry is the literal path where the quarantined items are - it is important to use literal paths instead of relative paths when using cron.
   * The "-mtime +30" flag is used in conjunction with the "find" command to identify the items that haven't been modified in 30 days or more.
   * The "-type f" flag is also used in conjunction with the "find" command to determine that the item is a regular file type.
   * The "exec rm -f" string tells cron to "execute" the "rm -f" command, or the "remove" command with the "force (-f)" flag.
   * The last items in the string, "{} \;" ends the command string properly.

Simple syntax checking with Postfix

We can also do some rather simple checks that will stop anything with an obviously bogus domain name. This is a good first line of defence for any mail server, since if the sender does not give us a real domain name, we can't reply to them. Bogus domain names have a 99.9% of being spam. Edit /etc/postfix/main.cf and add the following lines.

smtpd_delay_reject = yes
smtpd_sender_restrictions = permit_mynetworks, reject_unknown_sender_domain
smtpd_helo_required = yes
#smtpd_helo_restrictions = permit_mynetworks, reject_unknown_hostname

NOTE: I have commented out the helo restrictions. This is because there is alot of mail servers out there that are misconfigured. While it would be nice for them to fix their mail servers, the reality is that you will probably have a customers asking were their mail is from a certain user. To only accept mail from mail servers that identify themselves with REAL hostnames, simply un-comment the helo restriction.

While this doesn't stop spammers using a valid domain name to spam from, it is a good idea to include. You can test this using trusty telnet. As you can see, our mail server now requires a helo. Polite mail servers always say helo!

apollo:~$ telnet mail.fqdn.com 25
Trying 203.28.11.1...
Connected to mail.fqdn.com
Escape character is '^]'.
220 mail.fqdn.com ESMTP Avoid the gates of hell, use Linux!
mail from: <[email protected]>
503 Error: send HELO/EHLO first

Our mail server now checks the hostname in the helo for validity

apollo:~$ telnet mail.fqdn.com 25
Trying 203.28.11.1...
Connected to mail.fqdn.com
Escape character is '^]'.
220 mail.fqdn.com ESMTP Avoid the gates of hell, use Linux!
helo mail.bogusdomain.com.au
250 mail.fqdn.com
mail from: <[email protected]>
250 Ok
rcpt to: <[email protected]>
450 <mail.bogusdomain.com.au>: Helo command rejected: Host not found

Our mail server now checks the domain name of the sender for validity

apollo:~$ telnet mail.fqdn.com 25
Trying 203.28.11.1...
Connected to mail.fqdn.com
Escape character is '^]'.
220 mail.fqdn.com ESMTP Avoid the gates of hell, use Linux!
helo mail.validdomain.com
250 mail.fqdn.com
mail from: <[email protected]>
250 Ok
rcpt to: <[email protected]>
450 <[email protected]>: Sender address rejected: Domain not found

Enforce the RFC!

The strict_rfc821_envelopes parameter controls how tolerant Postfix is with respect to addresses given in MAIL FROM or RCPT TO commands. Unfortunately, the widely-used Sendmail program tolerates lots of non-standard behavior, so a lot of software expects to get away with it. Being strict to the RFC not only stops unwanted mail, it also blocks legitimate mail from poorly-written mail applications.

By default, the Postfix SMTP server accepts any address form that it can make sense of, including address forms that contain RFC 822-style comments, or addresses not enclosed in <>. There is a lot of broken or misconfigured software out there on the Internet.

/etc/postfix/main.cf and add the following lines.

strict_rfc821_envelopes = yes 

Locking Down Postfix

There are some simple things we can do to restrict what data can be mined from out mail server from an outside source. These will not have an adverse effect on the mail server as a whole, it just modifies the default behaviour of our SMTP server, and what information it will give out.

disable_vrfy_command = yes
smtpd_etrn_restrictions = reject
unknown_address_reject_code = 554
unknown_hostname_reject_code = 554
unknown_client_reject_code = 554

Building a valid recipient list from Active Directory

This method consists of a simple perl script which uses Net::LDAP to retrieve Active Directory users' "proxyAddresses" which are both primary and secondary SMTP addresses (as opposed to using "mail" which would only retrieve a user's primary SMTP address). Nothing needs to be run on the Active Directory domain controllers; this script requires only TCP port 389 access to your Active Directory domain controllers.

#!/usr/bin/perl -T -w

# Version 1.02

# This script will pull all users' SMTP addresses from your Active Directory
# (including primary and secondary email addresses) and list them in the
# format "[email protected] OK" which Postfix uses with relay_recipient_maps.
# Be sure to double-check the path to perl above.

# This requires Net::LDAP to be installed.  To install Net::LDAP, at a shell
# type "perl -MCPAN -e shell" and then "install Net::LDAP"

use Net::LDAP;
use Net::LDAP::Control::Paged;
use Net::LDAP::Constant ( "LDAP_CONTROL_PAGED" );

# Enter the path/file for the output
$VALID = "/etc/postfix/relay_recipients";

# Enter the FQDN of your Active Directory domain controllers below
$dc1="domaincontroller1.example.com";
$dc2="domaincontroller2.example.com";

# Enter the LDAP container for your userbase.
# The syntax is CN=Users,dc=example,dc=com
# This can be found by installing the Windows 2000 Support Tools
# then running ADSI Edit.
# In ADSI Edit, expand the "Domain NC [domaincontroller1.example.com]" &
# you will see, for example, DC=example,DC=com (this is your base).
# The Users Container will be specified in the right pane as
# CN=Users depending on your schema (this is your container).
# You can double-check this by clicking "Properties" of your user
# folder in ADSI Edit and examining the "Path" value, such as:
# LDAP://domaincontroller1.example.com/CN=Users,DC=example,DC=com
# which would be $hqbase="cn=Users,dc=example,dc=com"
# Note:  You can also use just $hqbase="dc=example,dc=com"
$hqbase="cn=Users,dc=example,dc=com";

# Enter the username & password for a valid user in your Active Directory
# with username in the form cn=username,cn=Users,dc=example,dc=com
# Make sure the user's password does not expire.  Note that this user
# does not require any special privileges.
# You can double-check this by clicking "Properties" of your user in
# ADSI Edit and examining the "Path" value, such as:
# LDAP://domaincontroller1.example.com/CN=user,CN=Users,DC=example,DC=com
# which would be $user="cn=user,cn=Users,dc=example,dc=com"
# Note: You can also use the UPN login: "user\@example.com"
$user="cn=user,cn=Users,dc=example,dc=com";
$passwd="password";

# Connecting to Active Directory domain controllers
$noldapserver=0;
$ldap = Net::LDAP->new($dc1) or
   $noldapserver=1;
if ($noldapserver == 1)  {
   $ldap = Net::LDAP->new($dc2) or
      die "Error connecting to specified domain controllers [email protected] \n";
} 

$mesg = $ldap->bind ( dn => $user,
                     password =>$passwd);
if ( $mesg->code()) {
    die ("error:", $mesg->code(),"\n","error name: ",$mesg->error_name(),
        "\n", "error text: ",$mesg->error_text(),"\n");
}

# How many LDAP query results to grab for each paged round
# Set to under 1000 for Active Directory
$page = Net::LDAP::Control::Paged->new( size => 990 ); 

@args = ( base     => $hqbase,
# Play around with this to grab objects such as Contacts, Public Folders, etc.
# A minimal filter for just users with email would be:
# filter => "(&(sAMAccountName=*)(mail=*))"
         filter => "(& (mailnickname=*) (| (&(objectCategory=person)
                    (objectClass=user)(!(homeMDB=*))(!(msExchHomeServerName=*)))
                    (&(objectCategory=person)(objectClass=user)(|(homeMDB=*)
                    (msExchHomeServerName=*)))(&(objectCategory=person)(objectClass=contact))
                    (objectCategory=group)(objectCategory=publicFolder) ))",
          control  => [ $page ],
          attrs  => "proxyAddresses",
); 

my $cookie;
while(1) {
  # Perform search
  my $mesg = $ldap->search( @args ); 

# Filtering results for proxyAddresses attributes  
  foreach my $entry ( $mesg->entries ) {
    my $name = $entry->get_value( "cn" );
    # LDAP Attributes are multi-valued, so we have to print each one.
    foreach my $mail ( $entry->get_value( "proxyAddresses" ) ) {
     # Test if the Line starts with one of the following lines:
     # proxyAddresses: [smtp|SMTP]:
     # and also discard this starting string, so that $mail is only the
     # address without any other characters...
     if ( $mail =~ s/^(smtp|SMTP)://gs ) {
       push(@valid, $mail." OK\n"); 
     }
    }
  }

  # Only continue on LDAP_SUCCESS
  $mesg->code and last;

  # Get cookie from paged control
  my($resp)  = $mesg->control( LDAP_CONTROL_PAGED ) or last;
  $cookie    = $resp->cookie or last;

  # Set cookie in paged control
  $page->cookie($cookie);
}

if ($cookie) {
  # We had an abnormal exit, so let the server know we do not want any more
  $page->cookie($cookie);
  $page->size(0);
  $ldap->search( @args );
  # Also would be a good idea to die unhappily and inform OP at this point
     die("LDAP query unsuccessful");
}
# Only write the file once the query is successful
open VALID, ">$VALID" or die "CANNOT OPEN $VALID $!";
print VALID @valid;
# Add additional restrictions, users, etc. to the output file below.
#print VALID "user\@example.com OK\n";
#print VALID "user1\@example.com 550 User unknown.\n";
#print VALID "bad.example.com 550 User does not exist.\n";

close VALID;

The resulting output is in the format: "[email protected] OK" which then must be postmap(ped).

Add the following to your Postfix 2.0+ main.cf to use the relay_recipient_maps feature of Postfix, which will now reject unknown users:

relay_recipient_maps = hash:/etc/postfix/relay_recipients

Note: the Exchange domains in question MUST be entered in relay_domains, and NOT in mydestination.

Also note if you would like to prevent Postfix from rejecting with "User unknown in relay recipient table" and would rather Postfix say "User unknown" set the following in main.cf

show_user_unknown_table_name = no

I have the script cronned every hour with the following cron job:

#!/bin/sh

cd /etc/postfix ; ./getadsmtp.pl && postmap relay_recipients

Conceivably this script can be easily modified to support other LDAP servers by changing the M$-specific "proxyAddresses" search base and output modification.

POP3 Mail Collection

Fetchmail

If your mail server is not the MX for your domain, or if you have aditional accounts that need to be polled for email, there is a simple way to do this. Fetchmail is a general purpose POP3/IMAP mail collection application, that can be daemonised. Its method is both elegant and simple;

  • Poll remote servers for email
  • Download any mail
  • Spoof the downloaded mail to the locally running SMTP server

This means that the locally running SMTP server (postfix) will assume the mail is getting delivered in the usual fashion. Brilliant.

To install fetchmail, we simply use apt-get

apollo:~$ apt-get install fetchmail

After installation, we simply need to set up a configuration file in /etc/fetchmailrc, owned by the user fetchmail and chmod to 0600. Below is a sample configuration file.

# /etc/fetchmailrc for system-wide daemon mode
# This file must be chmod 0600, owner fetchmail

# The default for this option is 300, which polls the server every 5
# minutes.
#
set daemon      300

# By default, the system-wide fetchmail will output logging messages to
# syslog; uncomment the line below to disable this. This might be useful
# if you are logging to another file using the 'logfile' option.
#
# set no syslog

# Avoid loss on 4xx errors. On the other hand, 5xx errors get more
# dangerous.
#
set no bouncemail

# The following defaults are used when connecting to any server, and can
# be overridden in the server description below.
#
# Set antispam to -1, since it is far safer to use that together with no
# bouncemail.
#
defaults:
  antispam -1
  batchlimit 100

# Lets get some email
#
poll mail.remoteserver.com.au with protocol pop3
  user remoteuser there with password pass123 is localuser here fetchall;

Purge POP3 Mailboxes

There may come a time when you want to nuke a POP3 mailbox. This is a quick little Perl script that does just that. Supply it your user credentials and it will shamelessly delete all messages in that mailbox.

#! /usr/bin/perl -w
use strict;
use warnings;

use Net::POP3;

my ($popserver, $user, $pass);
my $pop;

# Variables
$popserver = 'mail.bigpond.com';        # pop3 server address
$user='[email protected]';                # enter your pop3 username
$pass='PASSWORD';                       # and password here

# Constructor
$pop = Net::POP3->new($popserver, Timeout => 60); 

print "Connecting to server $popserver...\n";
if ($pop->login($user,$pass)) {
 my $msgnums = $pop->list;              # hashref of msgnum => size
 foreach my $msgnum (keys %$msgnums) {
  print "Deleting message $msgnum...\n";
  $pop->delete($msgnum);
 }
}

$pop->quit;

Virtual Hosting

There comes a time in our mail servers life when one domain is just not enough. Luckily Postfix can easily deal with virtual domains.

Edit /etc/postfix/main.cf and add the following

# Virtual Domain Hosting
virtual_alias_domains = newdomain.com.au
virtual_alias_maps = hash:/etc/postfix/virtual

Create a file /etc/postfix/virtual and add all your virtual email addresses in there. Below are some examples. The entry on the left is the virtual email address, and the entry on the right is the real mailbox.

[email protected]		admin_newdomain
[email protected]		jcitizen

Once you are done, you need to generate a hash file.

cd /etc/postfix
postmap virtual

Restart Postfix

/etc/init.d/postfix restart

Encryption using TLS/SSL

Just because you are paranoid, doesn't mean that people are not watching what you are doing.

Secure SMTP

The current version of Postfix for Debian will set up SMTP/TLS by default. If you don't have this already set up, edit /etc/postfix/main.cf and add the following.

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache

Secure IMAP

On Debian, the Cyrus config does is set up, but the imaps line is commented out. Edit /etc/cyrus.conf and un-comment the line under the SERVICES section that refers to imaps. It will look something like this.

SERVICES {
       # --- Normal cyrus spool, or Murder backends ---
       # add or remove based on preferences
       imap            cmd="imapd -U 30" listen="imap" prefork=0 maxchild=100
       imaps           cmd="imapd -s -U 30" listen="imaps" prefork=0 maxchild=100

We also need to make sure that we have access to a valid certificate. Edit /etc/imap.conf and un-comment the following lines.

tls_cert_file: /etc/ssl/certs/ssl-cert-snakeoil.pem
tls_key_file: /etc/ssl/private/ssl-cert-snakeoil.key

We also need Cyrus to be able to read the snakeoil certificates. An easy way to do this is to add the Cyrus user to the ssl-cert group.

 apollo:~$ adduser cyrus ssl-cert

Genertaing a Certificate

You might also want to generate your own certificate. The main thing you need to be concerned about is the servers FQDN matching the domain name that will be used to access the service. As certificates are also designed to expire, you should make sure that the length of time that it is valid for is acceptable, and that a new certificate is generated just before this one expires. You will also need to modify the filenames of the ssl certificates as shown above, if you want to use your new certificates.

apollo:~$ cd /etc/ssl/private
apollo/etc/ssl/private$ openssl genrsa -out ssl-cert-mail.key 2048
apollo:/etc/ssl/private$ chmod 640 ssl-cert-mail.key
apollo:/etc/ssl/private$ chown root:ssl-cert ssl-cert-mail.key
apollo:/etc/ssl/private$ cd /etc/ssl/certs
apollo:/etc/ssl/certs$ openssl req -new -key ../private/ssl-cert-mail.key -out ssl-cert-mail.csr
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:Country Code Here
State or Province Name (full name) [Some-State]:State Here
Locality Name (eg, city) []:Suburb Here
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Company Name Here
Organizational Unit Name (eg, section) []:Web Services
Common Name (eg, YOUR name) []:fqdn.of.server
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

apollo:/etc/ssl/certs$ openssl x509 -in ssl-cert-mail.csr -out ssl-cert-mail.pem -req -signkey ../private/ssl-cert-mail.key -days 3650

LDAP mail routing

This is the first version of LDAP mail routing, and initially it is a quick and dirty note to say how it is done. I will flesh it out later.

I will assume that the base dn of the directory is dc=bigdomain,dc=local

Populate LDAP

You need to make sure that all your users exist in LDAP, and that they have the following entries.

mailHost - This should contain the internal IP address of your mail server that contains the users mailbox. For example, 192.168.0.10
mailLocalAddress - This should contain the fully qualified email address of the user. This entry is a list, and can contain more than one email address. For example, [email protected]
mailRoutingAddress - This should contain the internal routing emailaddress of the user. For example, [email protected]

Postfix LDAP integration

Configuration Files

/etc/postfix/main.cf

alias_maps = ldap:/etc/postfix/distlist-ldap.cf
virtual_alias_maps = ldap:/etc/postfix/distlist-ldap.cf, ldap:/etc/postfix/virtual-ldap.cf
virtual_alias_domains = ldap:/etc/postfix/virtualdomain-ldap.cf
transport_maps = ldap:/etc/postfix/transport-ldap.cf

/etc/postfix/distlist-ldap.cf

# LDAP distribution list support
server_host = ldap://127.0.0.1:389
search_base = dc=bigdomain,dc=local
query_filter = (&(objectClass=nisMailAlias)(cn=%s))
special_result_attribute = rfc822MailMember 
exclude_internal = yes
result_attribute = rfc822MailMember
result_filter = %s
search_timeout = 15
scope = sub
version = 3

/etc/postfix/transport-ldap.cf

# LDAP transport for multilocation support
server_host = ldap://127.0.0.1:389
search_base = dc=bigdomain,dc=local
query_filter = (&(|(mailLocalAddress=%s)(mailRoutingAddress=%s))(!(mailHost=192.168.0.10)))
result_attribute = mailHost
result_filter = smtp:[%s]
search_timeout = 15
scope = sub
version = 3

/etc/postfix/virtual-ldap.cf

# LDAP alias support
server_host = ldap://127.0.0.1:389
search_base = dc=bigdomain,dc=local
query_filter = (|(mailLocalAddress=%s)(mailLocalAddress=%u)(mail=%s))
special_result_attribute = uniqueMember
result_attribute = mailRoutingAddress
result_filter = %s
search_timeout = 15
scope = sub
version = 3

/etc/postfix/virtualdomain-ldap.cf

# LDAP alias support
server_host = ldap://127.0.0.1:389
search_base = ou=mailDomains,dc=bigdomain,dc=local
query_filter = (dc=%s)
special_result_attribute = uniqueMember
result_attribute = dc
result_filter = %s
search_timeout = 15
scope = sub
version = 3

Testing

You can use the postmap utility to do the same query that postfix would do in the process of mail routing.

>postmap -q [email protected] ldap:/etc/postfix/virtual-ldap.cf
[email protected]
>postmap -q [email protected] ldap:/etc/postfix/virtual-ldap.cf
>

You can also add a -v to get verbose output from the above command.

When things go wrong

Recently, one of my customers mailservers stopped working. The root of the problem was that, in their wisdom, they power-cycled the mail server in an attempt to correct an internet related problem. This mail server in question is sitting on a very old box, and when it came back up, it had a brand new problem.

The first thing to do is check the log files, and on Debian it is /var/log/mail.log. Look for anything that is out of the ordinary, or repeditive.

Corrupted SEEN files

After a system disk hit 100%, I started seeing the following errors in /var/log/system.log. Apr 2 18:53:56 HOSTNAME imap[1981]: DBERROR: skiplist recovery /Volumes/raid1/mail_server/database/user/p/USERNAME.seen: ADD at 1FE8 exists Apr 2 18:53:56 HOSTNAME imap[1981]: DBERROR: opening /Volumes/raid1/mail_server/database/user/p/USERNAME.seen: cyrusdb error Apr 2 18:53:56 HOSTNAME imap[1981]: Could not open seen state for USERNAME (System I/O error) If you truncate the file at this point, it should fix the problem. The users mail read state will be valid up to the point of corruption. To do this, first convert the hex to decimal. You can use the Unix bc command:

$ echo "ibase=16;1FE8" | bc
8168
$

Then, using the Unix dd command, you can truncate the file and replace the corrupted .seen file with the fixed one. Have user log in and check that everything is ok.

# dd if=USERNAME.seen of=USERNAME.seen.fixed bs=1 count=8168
# mv USERNAME.seen USERNAME.seen.corrupt
# mv USERNAME.seen.fixed USERNAME.seen
#!/bin/bash

LOGFILE="/var/log/mail.log"

CORRUPTION=`awk '/ADD at/ { print substr($9, 0, length($9)-1 ),$12 | "sort" }' ${LOGFILE} | uniq`

TMP="/tmp/fixseen"

IFS=$'\n'

if [ ! -d "$TMP" ]; then
  mkdir "$TMP"
fi

if [ -n "$CORRUPTION" ]; then
  echo "Found possible corruption"
  echo "$CORRUPTION"
  echo

  for record in $CORRUPTION; do
    IFS=$' '
    arr=($record)
    fullfilename=${arr[0]}
    filename=$(basename ${fullfilename})
    dbhex=${arr[1]}
    dbdec=$(echo "ibase=16;${dbhex}" | bc)

    if [ -s "$TMP/$filename" ]; then
      echo "$TMP/$filename already exists, aborting"
    else
      echo "fixing for $filename on $dbhex ($dbdec)"
      dd if="$fullfilename" of="$TMP/$filename" bs=1 count="$dbdec"
      if [ $? -eq 0 ]; then
        echo "Done successfully"
      else
        echo "Error"
        exit 1
      fi
      echo

      echo "moving seen database to $TMP/$filename.corrupt"
      mv "$fullfilename" "$TMP/$filename.corrupt"
      if [ $? -eq 0 ]; then
        echo "Done successfully"
      else
        echo "Error"
        exit 1
      fi

      echo "moving $TMP/$filename to $fullfilename"
      mv "$TMP/$filename" "$fullfilename"
      if [ $? -eq 0 ]; then
        echo "Done successfully"
      else
        echo "Error"
        exit 1
      fi
    fi
  done
else
  echo "Corruption not found. Aborting."
fi

Can't read mailboxes file

This is a minor problem, but a major inconvenience. You could restore the mailboxes.db file from a backup, or reconstruct it from the files on the server.

Reconstruct mailboxes.db

Create the script shown below, and run it.

#!/bin/sh
TAB=`echo -e \\\t`
cd /var/spool/imap/user
find . -type d | grep ./ | \
	sed -e "s/\.\///" | \
	sed -e "s/\//\./g" | \
	sed -e "s/\([a-z]*\)\(.*\)/user\.\1\2${TAB}default${TAB}\1${TAB}lrswipcda${TAB}cyrus${TAB}lrswipcda${TAB}/"
cd -

This essentially rebuilt the database in plan text format.

If you want to export this list from a running server, you can run

su cyrus -c "/usr/sbin/ctl_mboxlist -d > mailboxes.txt" 

Then, as the cyrus user, reimported it:

su cyrus
/usr/lib/cyrus-imapd/ctl_mboxlist -u -f /var/lib/imap/mailboxes.db < mailboxes.txt

And now it should work. Verify that you can create and delete folders from your IMAP client, Postfix should be able to deliver messages via LMTP, Sieve filtering should be doing its job.