Throttling outbound emails with Mailman & Exim

Many ISPs have limits on the amount of emails received in a period of time, not respecting this will reduce the reputation of the sender.

Mailman 2 doesn’t support throttling, though there’re some patches in the wild adding this feature. Exim supports rate limiting incoming emails, but not emails going out, though it may be possible to do this with some convoluted configuration.

The easy solution is to rate limit emails coming from Mailman to Exim, so Mailman will retry them later.

This is the Mailman configuration:

DELIVERY_RETRY_WAIT = minutes(15)

Here we tell Mailman to retry sending delayed emails every 15 minutes, this is the minimum possible time, because the retry process only runs every 15 minutes. The default value is 1 hour.

This is the Exim configuration:

acl_smtp_rcpt = acl_check_rcpt

[...]

acl_check_rcpt:
  [...]
  defer hosts = +relay_from_hosts
        ratelimit = 75 / 15m / per_rcpt / $domain
        message = Rate limit exceeded for $domain: \
                  $sender_rate/$sender_rate_period \
                  (max $sender_rate_limit)

Here we’re telling Exim to accept no more than 75 emails every 15 minutes for any given domain. As seen above 15 minutes is the minimum possible for Mailman. The value 75 depends on how many emails you’re sending: look at the exim logs, pick up the domain you’re sending the most emails, and adjust this value accordingly.

Backup MX for Mailman, with Exim

This week a hard disk from one of my servers broke. It has been replaced with just 3 minutes of downtime (kudos to OVH), and right now the RAID array is being rebuilt.

Since this is the first time this happens to me, and the server is running a production mailman service, I decided to take my time and set-up a backup MX server. This is how I did it.

Remember that I am using a Gentoo distribution, this may be slightly different in other distro. Continue reading “Backup MX for Mailman, with Exim”

Gentoo: How to use Munin to monitor Mailman

Update 31/12/2012: Now this is a bit simpler, since bug 448414 has been fixed.

Before we start with the step-by-step procedure to get the mailman plug-in working, lets recall how to test a munin plug-in:

$ sudo munin-run mailman
posts.value 24
members.value 21892

The output above is what you should get once the mailman plug-in is correctly set-up. Continue reading “Gentoo: How to use Munin to monitor Mailman”

Gentoo: Mailman with Nginx & Exim

UPDATE 2017-05-11: Use https, with Let’s Encrypt.

UPDATE 2014-07-23: Forgot to give nginx access to mailman archives. Stop using sudo. Use service.

UPDATE 2012-12-10: the process has been simplified now that bug #37429 is fixed.

This is how I installed Mailman in a Gentoo server, using Nginx as the web server and Exim as the MTA.

Mailman

By default Mailman runs with the Apache user and group. First step is to configure Mailman so it runs with the Nginx user and group. To do so edit /etc/make.conf and add these lines:

/etc/make.conf
MAILMAN_CGIUID="nginx"
MAILMAN_CGIGID="nginx"

Now you can go ahead and emerge Mailman. Next comes to configure it.

Edit /etc/mailman/mm_cfg.py (replace the url and email hosts by yours):

/etc/mailman/mm_cfg.py
DEFAULT_URL_HOST = 'mailman.example.com'
DEFAULT_EMAIL_HOST = 'example.com'

# https
DEFAULT_URL_PATTERN = 'https://%s/mailman/'
PUBLIC_ARCHIVE_URL = 'https://%(hostname)s/pipermail/%(listname)s'

# Let Mailman know that the MTA (Exim) needs no aliases setting
MTA = None

As the mailman user (add the cron jobs, create the site password, and add the main list):

# su - mailman

mailman $ crontab cron/crontab.in
mailman $ bin/mmsitepass
mailman $ bin/newlist mailman

Run mailman:

# rc-update add mailman default
# service mailman start

Nginx

The Mailman web interface works with the CGI interface. To get it working with Nginx start by emerging spawn-fcgi and fcgiwrap:

# emerge spawn-fcgi fcgiwrap

Create the configuration file and edit it:

# cd /etc/conf.d
# cp spawn-fcgi spawn-fcgi.fcgiwrap

These are the changes to the configuration file:

/etc/conf.d/spawn-fcgi.fcgiwrap
FCGI_SOCKET=/run/fcgiwrap.sock
FCGI_ADDRESS=
FCGI_PORT=
FCGI_PROGRAM=/usr/sbin/fcgiwrap
FCGI_USER=nginx
FCGI_GROUP=nginx

Now start the daemon:

# cd /etc/init.d
# ln -s spawn-fcgi spawn-fcgi.fcgiwrap
# rc-update add spawn-fcgi.fcgiwrap default
# service spawn-fcgi.fcgiwrap start

Now add the Nginx server configuration:

/etc/nginx/mailman.conf
server {
    listen 80;
    server_name mailman.example.com;

    location /.well-known/acme-challenge {
        root /var/www/letsencrypt;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name mailman.example.com;
    root /usr/lib/mailman/cgi-bin;

    ssl_certificate /etc/letsencrypt/live/mailman.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/mailman.example.com/privkey.pem;
    # Redirect
    location / {
        rewrite ^ /mailman/listinfo permanent;
    }

    location ~ ^/mailman(/[^/]*)(/.*)?$ {
        fastcgi_split_path_info ^/mailman/([^/]*)(.*)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root/$1;
        fastcgi_param PATH_INFO $fastcgi_path_info;
        fastcgi_pass unix:/run/fcgiwrap.sock-1;
    }

    location /mailman-icons {
        alias /usr/lib/mailman/icons;
    }

    location /pipermail {
        alias /var/lib/mailman/archives/public;
    }
}

Include the file into the main Nginx configuration file:

/etc/nginx/nginx.conf
include mailman.conf;

Add nginx to the mailman group, so it has access rights to the archive:

# gpasswd -a nginx mailman

And restart Nginx:

# service nginx restart

Exim

There is very good documentation on running Mailman with Exim: Using Exim 4 and Mailman 2.1 together.

The only thing I found missing from the docs is a reference to the mailman-loop address. Add an alias for mailman-loop to a routable address, edit the /etc/mail/aliases file, for example:

/etc/mail/aliases
mailman-loop: root