 |
|
|
Since over 95% of spam contains links to servers, we came up with a
different approach to spam control.
We started using URL's as our primary means of spam filtration in 2001.
Many other spam filters now use this method for detecting spam as well, but
they must rely on open source lists of bad sites. We collect and manage
our own list of offensive websites.
We then started using our existing database of bad sites from our web filter,
the
Emerald Web Shield
. Our current spam filter service adds over 10,000 new domains per
week to our database. We also use webbots to crawl the Internet looking
for new objectionable sites for our database.
We currently have over 3 million domains in our
database.
|
How does Stop and Dig tm
work?
|
|
Our spam filter service looks at each email as it arrives and extracts all the
links from it. If the email contains links to known bad domains we know
we can block it. If the site contains links to known good sites, we allow
it. More and more frequently the email contains links to servers we
already know about.
Spammers have started a new tactic in recent months of creating a new domain,
and then sending out a large volume of spam that points to the domain. In
as little as 24 hours they destroy the site and the domain no longer
exists. They may reuse this domain over and over in this manner, but
separated by months of inactivity.
When we find a URL in an email that we do not know about, we stop processing
the email. The email is then put into a special holding area for 15
minutes. An automated tool is then sent out to download the contents of
that site, this tool is called a webbot.
The webbot then attempts to classify the type of site it has crawled. If
the site links to other known bad sites, contains porn content, or redirects to
bad sites it is flagged as a bad site and the email can be tagged as
spam. If the site cannot be classified automatically, it is added to a
special list for review by our site reviewers.
If the webbot has not completed the review of the site within the 15 minute time
period the email is released. We never hold an email longer than 15
minutes (unless your server is offline). We still attempt to classify the
site and will add it to our database once it is classified. For more
information about our webbot visit our
Emerald Shield webbot
information page.
|
|
|
