Spam and Postfix

From Zanecorpwiki

The two main tools in your anti-spam toolkit are RBL lists and content analysis.

RBL Lists

Real Time Blackhole List lists are the first line of defense.ref group=notesDue to trademark issue, the better name is DNS Blackhole List, or DNSBL. It's much harder to say and pretty much all postfix docs and the settings themselves use the RBL abbreviation./ref They are relatively quick and keep bad mail from getting in the queue. Setting up an RBL is easy enough, just add a declaration to 'smtpd_recipient_restrictions' and reference the RBL source:

smtpd_recipient_restrictions = ...reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net...

Where the '...' indicates additional options. With this, you should see messages like rejected by zen.spamhous.org start showing up in the mail log.

Content Analysis and SpamAssassin

TODO

Why does Spamassassin filter outgoing mail? I mean, the content filter is for everything so it's expected, but let's turn that off.

Instead of using the crontab solution for clearing the MAILER-DAEMON stuff, I'd love to do a header match and dump them straight to spamking.

Installing and Configuring

TODO: I think this can be simplified and the changes to main.cf can be eliminated; see example config in spampd man page.

For SuSE 10.2. Go to yast, install spamassasin. Also get the 'rayzor-agents' (this is a plugin for sa).

The documentation over at spamassassin.org seems to indicate that it's a good idea to get spampd to. We installed this in the bin of the spamking user. (Had to install packages perl-IO-Multiplex and perl-Net-Server too).

The easiest and (according to the docs I read) better way to get spamassassin going was via the spamd-before-queue route. The docs say this is more complex, but doesn't seem to be. The killer is that the after-queue stuff was referencing sendmail, and it seemed chancy to try and figure out what it was trying to do with sendmail and emulate with something else (esp. since I can't install sendmail with postfix installed).

In any case, the process seems to be this:

1. mail is received by the regular means
2. it's then shipped through spampd (which uses spamd, but is more efficient I guess)
3. spampd marks spam with a header thing
4. a filter is applied on the header; anyone aliasing in the spam alias file gets their spam sent to spamking (otherwise, it goes to the original user with the marked header)

Note, spampd must be started as root.

Also note, we created the spamalias file with the following command (don't forget to run postalias):

for i in `ls /home/`; do echo "$i: spamking"; done >> /etc/postfix/spamalias
postalias -f /etc/postfix/spamalias

You also need to open the ports (10025 and 10026) for the new filters. There should be some way to open these for only local connections, which would be ideal, but for the moment, I just opened them in the SuSE firewall (through yast) as open ports. Don't forget to re-run the route_setup.sh (in conf) if necessary (restarting the firewall clears our custom routings).

# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (50)
# ==========================================================================
...
smtp      inet  n       -       n       -       -       smtpd
...
scan               unix    -        -        n        -        20        smtp
localhost:10026    inet    n        -        n        -        20        smtpd
        -o content_filter=
        -o local_recipient_maps=
        -o relay_recipient_maps=
        -o myhostname=filter.mynetwork.local
        -o smtpd_helo_restrictions=
        -o smtpd_client_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o mynetworks=127.0.0.0/8
spamtnsp          unix  -       n       n       -       -       local
        -o alias_maps=hash:/etc/postfix/spamalias

content_filter =scan:[127.0.0.1]:10025
header_checks = regexp:/etc/postfix/spamheadercheck

/^X-Spam-Status: Yes/ FILTER spamtnsp:local

user1:  spamking
user2:  spamking
...

spampd --port=10025 --relayhost=127.0.0.1:10026 --user=spamking --tagall --children=20 --maxrequests=4
rcpostfix reload

Performance Tuning

TODO: make sure the deployed versions match this; at the time of the note, bumperactive did, but liquid and fusion did not

For some reason spamassassin is slow... probably the RBL checks. The configuration is hella huge and not very clear, so I hesitate to mess with it. In the meantime, you can increase the throughput quite easily and effectively by increasing the number of mails processed in parallel. Hence the "20" as the max procs for the scan and local delivery in the master.cf (rather than the 10 in the docs).

Note that when messing with those numbers, the maxprocs of the scan and local delivery must match. You should also match the spampd to be able to take that many. The default number is 10, so up the children to 20 (which actually allows it to handle 21 requests concurrently, oh well). to better manage memory, we also reduce the maxrequests per child to 4 (see spampd man page for more on this).

Updating

If you need to stop and restart spampd (to load new config and/or change the command line options), do:

start-stop-daemon --stop --pidfile /var/run/spampd.pid
spampd --port=10025 --relayhost=127.0.0.1:10026 --user=spamking --tagall #or with different options
rcpostfix reload

The 'rcpostsfix reload' is necessary, otherwise postfix will continue to think the spampd is down (though I'm not sure why since it should be talking to it through the localhost inet internface...).

Commentary

What follows is my own limited experience. I've tried to find better numbers, but so far no luck. If you have (or are interested in) figuring out the impact of spam on your system, send me an email (zane at the obvious domain) and I'll add in your own notes.

In my experience, RBL lists result in a small number of false positives, and on some systems you could get away without using them. The problem is that content analysis is time consuming and should your server ever be targeted to receive a high volume of spam you'll end up eating up bandwidth and runaway queues.

Before about a week ago, my server was getting about somewhere in the 1000's of spam messages a day. With that volume, I actually relied entirely on SpamAssassin with Thunderbirds local junk mail filter. I didn't use the RBL lists at all because in order to avoid dropping mail due to false positives. I found that SA+Thunderbirds junk controls caught all but a couple emails, and most days I had no visible spam.

About a week before writing this up I was, for some reason, targeted and was getting 1-2 bogus emails a second. This was causing the active postfix queue to grow faster than it could clear and so legitimate messages were taking longer and longer to get through. This had happened before, maybe 1-2 times a year on every server I run (2-3), but didn't last too long. I used a couple scripts to manually clear the queues. This particular wave was less acute, but longer lasting.

The RBL checks are close to instant and keep my queues consistently under control and predictable. All in all, I should have been using RBLs all along. The few false positives I know of (literally two) were easy enough to deal with, and didn't last to long. That small risk is worth stability. Especially as, at least for now, spam traffic seems to be increasing.