META Trend: As ad hoc electronic communication grows in importance (e.g., e-mail, instant messaging, Web conferencing), organizations will be challenged to create a hygienic and low-cost infrastructure. Through 2006, special attention will be focused on spam blocking and policy enforcement (e.g., regulatory compliance). By 2007, rising electronic communication volumes will frustrate users coping with information overload and drive organizations to employ common filters, queuing services, and categorization engines to ease communication burdens.
Several years ago, in the halcyon days before the spam epidemic, organizations were often able to block the bulk of unsolicited commercial e-mail with simple content filtering and subscription to one or several RBLs. Spam volumes, of course, have risen exponentially, and RBLs currently are merely one ingredient in the overall cocktail required for effective spam blocking. By tracking mail relays that were known (sometimes incorrectly) for sending spam, and thereby enabling subscribers to block messages from the offending IP addresses, RBLs established the concept of “reputation,” which is currently all the rage in the hypercompetitive antispam vendor wars. In fact, RBLs have improved immensely during the past two years, since the number of open relays has diminished to near-zero levels. Indeed, some ISPs report blocking up to 80% of spam with aggressive use of multiple RBLs.
RBLs have deficiencies, however, compared to the real-time reputation services currently used by many vendors. They are often subjective, for example, when compared with the statistical algorithms used by current reputation services, and therefore are more prone to false positives. They can also be slow to add and drop IP addresses, which may be harmful, given that many IP addresses are hijacked for temporary sending of spam (especially so-called zombie networks). RBLs cannot keep pace with these rapidly moving attacks, since they typically take hours or days to both list and unlist offending relay servers. Furthermore, RBLs are binary – companies are either on or off the list. Newer reputation services typically are far more granular in ranking the “spaminess” of a relay.
Therefore, RBLs are an anachronism that will slowly fade away during the next several years, as more sophisticated forms of reputation services prove to be more effective. In fact, recent advances have made ”reputation” somewhat of a misnomer, since these new tracking services are based on behavior of the relay in question as opposed to the reputation of the relay – and consequently, they are able to block/throttle mail flow in real time. Nonetheless, RBLs still provide value in blocking spam, and we continue to receive many inquiries about various aspects of this seminal reputation service.
Following is a list of frequently asked questions regarding RBLs, along with our responses:
Q: If a message is blocked due to an RBL listing, does the sender receive a response indicating the mail was blocked?
A: It depends on how the mail server owner implements the list. Assuming that the sender uses an RFC-standard compliant mail server, the sender server/client will typically receive a return message saying that their e-mail was rejected, though the message may not always state the reason for the rejection. The reason for the rejection, however, is usually captured in the recipient mail server.
Q: Can the rejection response due to an RBL listing be customized?
A: Not generally. There is a standard “552 Content Rejected” error that is returned. Some mail systems may send the URL returned in the RBL lookup, to give the sender more information specifying the reason for the rejection.
Q: How can a company/person be removed from an RBL?
A: The company must contact the owner of the RBL service. Sometimes this can be done with a quick e-mail; in other cases, the RBL Web site has a form the company can fill out, which then automatically performs a relay check on the domain. Speed of response varies greatly and is usually better with commercial RBLs. It used to be that the RBL owners took the attitude of, “You are a stupid administrator who set up your mail incorrectly, so I’m not taking you off.” There was little a company could do about this, other than hope that people they send mail to would not use that specific RBL server. Currently, however, the RBL sites are much more responsive than in the past.
Q: Is it possible to view the companies listed on an RBL? Is that access in real time?
A: Some services offer a Web lookup function on their site. Organizations can do a DNS lookup against the RBL server. A positive result indicates the IP address is on the blacklist. Some RBLs will also provide a link with pertinent information about the listing; others provide a “yea” or “nay” and nothing more, while some have no public face, and it is impossible to determine whether an address is listed. Most RBLs will throttle connections if pinged too quickly (to prevent DOS attacks). Dnsstuff.com is a good place to do multiple RBL lookups.
Q: How often are RBLs updated?
A: It depends on the service. Some are updated multiple times a day. Others are never updated and are grossly out-of-date, yet are still up and running.
Q: What are the bandwidth implications?
A: An RBL check is just a DNS lookup, so the need for bandwidth is not large. Most servers handling fewer than 10 inbound messages per second can handle RBL lookups, and most RBLs have reasonable capacity (e.g., Spamhaus handles 2,000-3,000 queries per second). Being able to keep large databases and scan through them quickly is important if local caching is used (we recommend caching). RBLs should block messages prior to entering the mail-processing stage, if possible.
Q: What are the tradeoffs between paid RBLs and free RBLs?
A: Paid sites are usually better maintained and updated more frequently than free RBLs. The granularity of the database is also generally better. For example, we have seen cases where free sites block the entire Class B block of addresses when a violation is suspected, instead of blocking only the specific addresses of the relays in question. Paid RBLs also are far more accountable and offer service-level agreements and customer service. In addition, paid sites are better architected to survive a denial-of-service attack via built-in redundancy, which is critical since RBLs are often the target of attacks from angry spammers.
Q: What are some of the more effective RBLs, both commercial and free?
A: With hundreds of lists, choosing a relevant RBL can be tricky. We believe MAPS (which was recently bought by Kelkea) is the best of the paid services, though its effectiveness is said to be diminishing since it went commercial. Commercial services typically cost about $2,000/year and have about 5% more unique addresses compared with no-fee services. SpamCop is a reasonable free service, but it can be a bit aggressive. Spamhaus (SBL-XBL), SORBS, and NJABL are also good choices for no-fee sites (Spamhaus charges for downloading zone files but provides free DNS queries).
Bottom Line: Real-time blackhole lists should continue to be used as part of an overall spam-blocking strategy. However, the value of RBLs is being eclipsed by other more sophisticated forms of reputation services, which are essential to effective mail hygiene programs.
Business Impact: Organizations must take draconian measures to prevent spam from compromising the value of corporate messaging systems.