Bookmark this on Delicious
Next: The Greylisting approach Up: Under the hood Previous: Under the hood Contents
The software side
Spam control has to invariably fall under one of the following categories.
- Bayesian filtering and contextual analysis
- Heuristical filtering based on known keywords/bad words
- CRM114 Markovian chain based discriminator (related to a)
- Vipul's razor approach of DCC (Distributed checksum computation) with manual interference - gmail uses this heavily
- Greylisting to stop spam right at the MTA level
- IP address blacklisting and e-mail address whitelisting
- TMDA - cure worse than the disease (Only approved senders can send mail)
- RBL lists , spamhaus (politically sensitive spam control techniques)
- SPF from Microsoft (not a bad idea per se) but does not work well
Most of these techniques are based on content scanning/filtering and actually reading e-mails with a computer.
Since this is an activity that requires a high end CPU and memory, spam control software and virus scanning software typically end up grinding your machines to a halt or even slow down your legitimate e-mails.
Also there is the very scary possibility of losing e-mails due to false positives.
My product 'SpamCheetah' uses the technique called OpenBSD greylisting. This is a very smart way to combat spam since it is stopped right at the MTA level. Since this never reads e-mail it is also very fast and highly efficient. It is impossible to get a false positive here though the first mail from a domain will experience a delay.
Basically greylisting forces mail servers to be RFC 2821 compliant and retry mails until the receiving site is ready. This also has an added advantage of hurting spammers sometimes and also stopping the spam that is meant for some other sites.
And you don't waste your storage space and bandwidth receiving spam first and then rejecting them.
Next: The Greylisting approach Up: Under the hood Previous: Under the hood Contents Girish Venkatachalam 2009-08-09
