IP Reputation, the most powerful tool in the fight against unwanted bulk email (UBE) and spam

What is IP Reputation?

It is simple. All email must originate from an IP address, and IP reputation can be used to tell if a certain IP Address is responsible for sending Spam or Unwanted Bulk Email (UBE). And it is extremely effective, stopping between 80-95% of all inbound connections at ISP's and sometimes more.

It is up to the person who owns that IP to be responsible for what comes out of that IP Address, and if they don't stop spam from originating they can get a reputation for that behavior. Almost every large company, ISP and email server uses it to some extent. Otherwise they would have to process almost 20 times as much email as they would without it.

And there are many companies and resources that track that type of behavior, and create IP reputation lists. Often those lists are used to completely reject email coming from that IP Address. And these lists are compiled from data that shows that IP Address is responsible for only certain types of behavior. The following is some examples:

The IP Address belongs to a system that is NOT an email server.
- It might be an infected personal computer
- It might be an compromised web server
The IP Address belongs to an ISP or email server that doesn't prevent outbound spam
The IP Address belongs to an ISP that doesn't respond to Spam complaints
The IP Address belongs a known spammer
The IP Address belongs to an email protection device or server that does 'BackScatter'
The IP Address belongs to a company that allows mass email marketing campaigns

Some of these lists are freely available, some you have to pay for, and some are privately maintained. There are literally hundreds of such IP Reputation lists, operated by companies such as SpamHaus, SpamCop, SpamRats, SORBS, UCE-Protect and others. The nice thing about these lists is that is often how companies find out they are leaking Spam in the first place. It is also a motivator for companies to fix the problem.

And checking the IP Address is the easiest thing for email servers to check before accepting email.

Of course, some times using IP reputation can stop legitimate email. Sometimes an IP Address will change owners, and sometimes an IP Address will be listed because too many other IP Addresses from the same company are responsible for UBE. Or, it could be that another email account on the server you use was compromised, and you are affected. Normally most lists allow removal of IP Addresses easily, quickly and quite simply, except where too many repeated cases occur, some list operator may prevent the removal until the owner of the IP address does something to correct the repeated problem. This actually helps to make the internet a better place.

If you are an end user, and you reached this page the problem is with your ISP and/or email administrator. It is their job to ensure that they don't gain a reputation for sending unwanted email. There are many alerting systems and lookup tools to make sure that they don't get listed on such IP reputation lists. And if your ISP does not do anything to address the reasons they get on such IP reputation lists, consider either changing ISP's or you will have to get all your friends to whitelist your email address in order to get email from you.

Two companies that do this are DNS Stuff and MX Toolbox

If you are the operator of an email server and find that you are listed in one of those lookup tools, then it behooves you not only to remove yourself, but to figure out why you got listed in the first place.
The following are the most common reasons for Spam Leakage from your email servers:

Free Email signups by Spammers
Compromised accounts, bad passwords, stolen passwords
Backscatter
- Filter bounces messages to forged or wrong addresses
- Over Quota or Virus Bounces
- Bad Vacation Message or Email Forwarding system
- Not checking for Valid USers properly
No rate limiters

MagicMail Protection

MagicMail servers take advantage of IP Reputation lists. However they usually only need to check a few lists to get most protection. If you received a message similar to the following:

     10.0.0.1 does not like recipient.
                  
     Remote host said: 550-Your message was rejected by this user and was not delivered.
                   
     550-Reason: This system uses BMS to check your IP address reputation, and was rejected 
     550-Protection provided by: MagicMail version 1.1.1 (http://www.magicmail.com) 
     550-For more information, please visit the URL:
     550-http://www.linuxmagic.com/power_of_ip_reputation.html
     550-or contact your ISP or mail server operator.
                         
     Giving up on 10.0.0.1

You can check your IP Address using the form at BMS Lookup Tool and it should show you what IP reputation list the user or the email server was using that you were listed on, and you can go to that site to get yourself removed. PLEASE, check your server over to find out why you got listed in the first place otherwise you may get listed again. Then you can contact the list administrators via their websites.

An example of how great IP reputation can be, since virus infected home PC's are the single biggest source of Spam on the internet, if you had all the IP addresses of all home style connections (DUL/Dynamic/Dialup/Hotspots) then spammers would not be able to use that to spread new viruses by email. One such database at SpamRats for instance has over 25 million such IP's listed and blocks over 50% of Spam alone. If your ISP is using such lists, this is the single best way block 'BotNet' spam.

Email Marketing Companies

Although sometimes not classed as true 'Spam', this unwanted bulk email (UBE) has become a lot worse over the last 18 months. You may have seen this type of email offering lower mortgages, cheaper airline tickets, even advertising for very legitimate companies. They usually contain a message 'You are receiving this because you have opted in to 3rd party offerings'. Normally the main reason you get these is that you signed up for something online or bought something, and the small fine print said that they are allowed to send you such flyers. The problem is once they get that 'permission' from you, very often the flyers get out of hand; now you are getting 30-40 flyers a day.

Wouldn't it be nice if the same way you call the post office, and ask them not to deliver any more flyers to your door, you could do the same thing with your inbox? Well you can, by using IP reputation and databases of such companies and their IP Addresses that engage in such practices, and networks that allow such behavior. Often these companies are so big, they have thousands of email servers sending out advertising. And this is the fastest growing form of UBE out there.

One such database, MIPSpace tracks this activity at hundreds of ISP's across North America (BTW, contrary to popular belief, most of these companies are located in North America and NOT overseas). If your ISP uses, or allows you to use such a database it can block this with IP reputation.

Compromised/Hacked Servers or Accounts

This used to be a more common problem, but as server security has increased hackers and spammers do look for easier targets. There are still web servers and online forms that hackers can compromise, and use to send out their spam, (often the more vicious types, like viruses and porn) but usually this type of activity is easier to detect, and they get shut down very fast, or blocked by IP reputation lists like SpamRats.

However, recently hackers are using easier targets such as email accounts hijacked from people just like you. Using a real email account is a lot better for hackers as ISP's who need to process millions of emails for their customers have a harder time noticing one account that is sending more than it's normal share. The hackers may only send a few thousand from each account, but if they have thousands of such accounts, it makes for a profitable way to send spam. Usually using a real email account from a reputable ISP means they have less chances of their messages being blocked.

How do they get those accounts? Simple. Too many people using easy to guess passwords. If your email is "john@isp.com" and your password is "john", or "john123" or "test", then they are going to get your email account. With the 'BotNets' mentioned early it doesn't take long when 100,000 computers all try to 'guess' your password. They also run dictionary attacks for commonly used words as passwords.

So how do we stop this? Well, the ISP's have to stop it before it gets out. If they use rate limiters on outbound email, and password policies, they should not have these problems. And the ISP's that don't, well they usually end up on blacklists until they rectify the problem. (Actually, some of the bigger ISP's are often the worst problems, as they are too big to blacklist, and without that pressure they have little motivation to deal with this issue) But using better email technologies like the LinuxMagic's own 'MagicMail' email server, more companies get this capability out of the box, and this type of Spam could be a thing of the past.

Free Email Providers

Yes, a problem. But you have to have a little sympathy on how hard it is. Basically, the hacker use the 'BotNets' described earlier to try to sign up to for thousands of email accounts.. Or some individual signs up for a throwaway account. Often this is the nastier form, sending emails to try to get your bank account information, or to tell you you have won a lottery, or that they want you to help them get millions of dollars out of a foreign company.

They can do some things, like limiting how much email a person can send at a time, but when thousands of accounts all send just a few messages, it is harder detect the spammers. And you can't really point a finger at a single email provider, as all of them have suffered from this at one time or another. They try to stop automated signups, but the hackers keep finding new ways around this. For most people, it is impossible to just block free email providers such as Yahoo, Gmail, or Hotmail (although some people do) therefore there is only one way to stop spam from bad free email accounts. Thankfully this is the lowest percentage of all the types of Spam, but in this case the ISP has to use 'filters'. Not the best way to deal with this as it adds load to the servers, and as soon as one filter stops a message, the spammers change the way they write the emails to get around the filters.

Until the free email companies solve this problem, your ISP is forced to use some spam filtering techniques along with the normal virus and other filters. Luckily, most modern email servers have, or keep up with the latest filtering technologies. If you get this kind of spam, report it to your ISP. It will be up to the ISP to put pressure on the free email providers to make sure they stop it before it leaks out of their servers.