Questions about the Emerald Spam Shield spam filter

Emerald Spam Shield

Spam Basics

Emerald Spam Shield

1. What is the Emerald Spam Shield?

Emerald Spam Shield is an accurate and efficient anti spam filter system that eliminates spam and prevents needless exposure to objectionable content.  Additionally, this hosted spam filtration solution helps free up vital bandwidth and reduce mail server load by filtering mail on our enterprise-class servers and sending the filtered results to your company's mail server.

back to top

2. How does Emerald Spam Shield work?

Spammers can lie about almost everything in an email.  The return address can be spoofed, the route the message took to reach its destination can be phony, the subject line may not represent the actual content, and content can be disguised as images.

But spammers cannot lie about where images are pulled from across the Internet and what websites they want you to visit.  This information is represented within the email by website links.  Since the rest of the content can be falsified, Emerald Spam Shield begins its comprehensive scanning procedure by matching message links to a database of over 3 million known (catagorized) websites.

Emerald Spam Shield assigns a score to each email which indicates it's spam probability.  Points are added to the score for every known-bad link found in the email.  If the spam threshold hasn't been reached, it then continues it's scan using a variety of other techniques.  Once the threshold is exceeded, the message is flagged as spam.  The message can then be tagged, forwarded to a special mailbox, returned to the sender, or deleted.

For more information visit the  "How Emerald Spam Shield Works" page.

back to top

Spammers change their tactics very quickly, almost daily.  Bayesian spam filters are now overloaded by over 50% of spam.  The use of zombie (hijacked or rented computers) email-generating machines means that identifying spammers by IP address lookup is less reliable.  Markov filter technology is overloading thumbprint and related scoring algorithms.

Since correlating a message's links to known-bad websites is one of the most reliable ways to detect spam, Emerald Spam Shield implements a real-time website rating system for maximum accuracy when identifying bad links.  This is the same proprietary rating system used by the  Emerald Web Shield software.

Emerald's block list includes links associated with pop-ups and other techniques that redirect a user's web browser to objectionable or unsafe websites, many of which attempt to install adware and other malicious programs on company computers. 

back to top

Emerald Spam Shield allows a company to:

• Reduce mail server load by preventing spam from clogging mail servers
• Reduce attacks on your mail infrastructure by using our servers as a frontline of defense
• Provide protection from Denial of Service and Directory Harvest Attacks
• Trust email again as a productive, low-cost business tool
• Only receive email from real people and valid organizations
• Minimize non-productive time spent by users dealing with spam
• Reduce the IT resources needed to backup and store spam in mailboxes
• Reduce IT problems caused by viruses and trojans
• Reduce company liability that can result from employee exposure to inappropriate text and images
• Enforce the email portions of the company Acceptable Network Usage Policy (ANUP) efficiently and fairly
• Increase email disaster recovery and fault tolerance
• Nothing to install or maintain on site

You can do a number of things to allow Emerald Spam Shield to work more efficiently for your company, such as:

• Configure your account to allow mail for valid users only.  Mail directed to addresses not on the list are rejected by the Emerald Spam Shield server.
• Avoid listing your real mail server in your MX records.  Spammers will sometimes send spam messages to every server found in the MX records in an attempt to bypass any spam filter.
• Do not place links to email addresses on any company websites.  Spammers use bots to harvest these links and send spam to them.  Use a contact form instead that sends email on behalf of the customer.
• Do not use first names only for email addresses.  Spammers have huge databases of common first and last names.  They send mail to addresses generated with random combinations of these names trying to find a legitimate email address.
• Research, create and enforce an Acceptable Network Usage Policy (ANUP) for the company including appropriate guidelines for employee email and mailbox use.
• Try to assign email addresses that are easy to remember but complex enough to be difficult to guess.  Some examples for an employee named John Smith:  j.smith, j-smith, john-s, john.s, john-smith, etc.  Avoid using the first name/last initial and the last name without some sort of delimiter as spammers regularly try random addresses using those basic combinations.

The Emerald Spam Shield can modify (or tag) the Subject line of any email that has exceeded the spam threshold.  The messages can then be delivered and be easily identified in a user's inbox by the subject tag.  Users can set up rules to handle these tagged messages in a special way such as sorting them into a specific folder.

If this behavior is not preferred, Emerald Spam Shield can be adjusted to instead forward the spam to a specific mailbox, send a rejection message to the sender and delete the message, or delete the message without notifying the sender.

back to top

The Emerald Spam Shield provides the option to forward any message flagged as spam to a special mailbox on your server for quarantine.

back to top

If an apparent spam message arrives in a recipient's inbox and does not appear to have been flagged by the Emerald Spam Shield, the message can be forwarded to Emerald.  The message can be analyzed to determine why it wasn't detected to help prevent similar spam messages from being overlooked in the future.

back to top

If the Emerald Spam Shield inaccurately marks an email as spam it can be forwarded to Emerald.  The message can be analyzed to determine why it was flagged incorrectly and changes can be made to increase accuracy and prevent future false positives.

back to top

Emerald promotes an aggressive development effort and we are constantly testing new algorithms and new methods to better understand spam and the spammers that propagate it.  We are constantly monitoring the latest tactics employed by spammers.  In addition, we employ spam traps on the Internet to assure a steady supply of new spam for analysis and dissection.

Emerald's spam traps involve NNTP newsgroup postings, sign-up offers, prize giveaways, and more.  We maintain a collection of email links in over 100 spam trap websites, a favorite location for spammers to harvest email addresses.

The spam filter traps also contribute to the efficiency of the Emerald Web Shield  products by making the known-bad list used by our spam filter more comprehensive.  Additionally our web filter helps populate our spam domain list.

back to top

To guarantee that we always have a generous supply of spam to analyze, we do all of the following (and more) to ensure a steady flow of spam:

(WARNING:  The following actions are guaranteed to generate spam.  We strongly suggest you don't try them yourself.)

• We ALWAYS "unsubscribe" using links found in spam emails that arrive at our traps.  Phony unsubscribe pages are one of many techniques used by spammers to harvest email addresses.
• We ALWAYS open our spam email, so spammers can confirm that they reached a valid email address and will send more. 
• We ALWAYS visit the links found in suspicious email.
• We ALWAYS browse the web on a computer with all its ports unprotected so the website can easily interrogate us and determine which domains to send mail harvests attacks against.
• We ALWAYS checkout everybody's "webcam".
• We ALWAYS sign-up for giveaways, contests, freebies, mailing lists, and anything else we can find on every website we visit. 
• We ALWAYS use common first and last names as the mailbox name to make it simple for spammers to figure out our email addresses.
• We ALWAYS bounce mail directed to mailboxes that do not exist so that spammers can use reverse logic to decipher which addresses are valid.  Using this technique they can direct an increased volume of mail to the addresses they determine are valid.
• We visit all newsgroups and freely post messages that include a working email address.  Newsgroups are another favorite location for spammers to harvest addresses.

back to top

Emerald Spam Shield provides an attachment filter that allows an administrator to specify which types of email attachments should be allowed and which should be blocked. 

Emerald Spam Shield also offers an optional anti-virus scanner which scans all messages and attachments to protect against emerging viruses and malware threats.

back to top

The data archived by the Emerald Spam Shield is limited to statistical reports of spam traffic and information related to spam analysis.  If an email contains a link (URL) that we have never seen before we add that domain to our list of domains to crawl.  We do not keep any portion of your actual email.

back to top

Right now remote mail servers connect to your server and send email directly to it.  They find your server through your MX records in DNS.  Once you change your MX records to refer to Emerald Spam Shield servers your email will then be directed to us.  After receiving the mail we will then connect to your server and deliver the filtered messages.

You do not have to change anything in your mail flow or mail server configuration.  However we do recommend that you block requests from other servers and force them to send email through us.  This reduces the footprint of attacks on your server from the Internet.

back to top

Our first layer of defense is our high level of server redundancy.  Our server clusters are housed in multiple datacenters.  An attacker trying to disable our servers would have to target three separate datacenters in three different parts of the country.  This redundancy allows us to be confident that our servers will never be unavailable.  We give each customer three MX records they can use for each of their hosted mail domains.  These are spanned across the datacenters to ensure that a single failure will never stop the flow of email.

back to top

The methods spammers use to get their message out are overwhelming and non-discriminating.  If your company has a mail server then you are a target for spam.

Everybody receives spam and almost always through no fault of their own.  In extreme cases a newly created, unused, and unadvertised mailbox could receive 10 to 20 unsolicited spam messages on the first day of its existence.

The lost time that is spent deleting spam is no longer the primary concern for most companies.  The fact that email can transmit spyware and other malicious programs that install themselves (sometimes without the user even opening the email) is a much more serious issue. The Emerald Spam Shield can help you to stop these threats before they reach the desktop! 

back to top

Exchange 2003's antispam technology incorporates Bayesian filter algorithms, which are dictionary lookups that assign weighting factors and scores to words.  Exchange 2003 performs approximately around ten checks and looks at other message characteristics (for example, the time a message was sent) to calculate the message's spam probability. 

Spammers are successfully bypassing Bayesian filters.  They may sprinkle their messages with hundreds of words or phrases that fool almost all rule-based detection schemes.  Since spammers also have these programs they can use them directly to develop new techniques that allow email to bypass these filters.  They may try hundreds of versions of an email until they find the one that gets past the filter.  Once they discover a technique that works they can publish it and share it with other spammers.

The biggest advantage of a spam filter like the Emerald Spam Shield is that it is actively managed .  When you use built-in filters they are static and do not change as spammers change their tactics.  We are constantly evolving our detection methods and we push out spam filter engine updates to our servers as frequently as every 15 minutes.

back to top

The assets provided by a hosted spam solution include rapid implementation and predictable cost in addition to the central benefit of eliminating spam from your network.  With a hosted solution your IT staff will be spared from the added responsibility of maintaining additional hardware onsite.

The Emerald Spam Shield's hosted model can also provide disaster recovery by mailbagging your mail if your company's Internet connection or mail server goes down, delivering it once your connection and/or mail server is restored.  Since our network of servers is geographically diverse you also minimize the chance of a disaster eliminating your email capabilities. 

Spammers change tactics every few weeks.  Do you have the resources to keep up with the pace of change?  We do.

back to top

Most backup MX services are traditionally setup as a low-priority MX record on your domain.  The theory is that if your mail server (the server with the highest priority) is offline the sender will try again on the lower priority mail server, which will then hold your mail until your primary server becomes active again.  For most legitimate mail senders this works fine.  The lower priority mail server will normally not have any filtration and is there "just in case" you need it.

Spammers know this and have recently started sending spam directly to all of the servers referenced in a domain's MX records, regardless of priority, in the hope of finding a server that will not only accept the message but deliver it unfiltered.  This allows spammers to circumvent some filter mechanisms you may be relying on such as RBL lists.  When email is delivered via your backup mail service your primary mail server cannot identify spammers using RBL checks because the mail is originating from a trusted server.  Therefore you have just lost another way to detect the spammer.

We can act as your primary AND backup MX service.  Spammers can attempt to send you mail using any of the MX records for your domain (we give you three to use) and cannot reach you directly using any of them.  Since we also provide a filtration service you get the additional benefit of detecting and handling spam without the spammer ever being able to contact your server directly.  Plus we provide you disaster recovery by holding your mail in the event that your server is offline for any reason.

If you would like a special price for us to only be a backup MX for your company please use the Contact Us link and tell us your needs.

back to top

In the event that your primary mail server is unavailable at the time of delivery our service can try up to two additional servers to deliver your mail.  For example if you have a T1 line and a DSL for backup we can attempt delivery on the T1 line first and then try the DSL in the event of a problem.  If we cannot reach any of the servers (up to three total) that you have designated we mail bag your mail for up to 4 days (see #21 below).  If you need your mail held for a longer period of time (due to emergency circumstances) we can extend the mail bagging duration for you.

back to top

Mail Bagging is the term we use for holding undeliverable mail.  We put it in a "mail bag" and hold it for you while reattempting delivery at regular intervals.  Our retry schedule is variable and normally involves retrying 3 times within the first 2 hours and once every 4 hours thereafter until delivery is successful or the mail expires.

back to top

It is ETI's opinion that our Confidentiality Agreement exceeds the requirements of HIPAA, especially since no Client email information is stored or used by ETI, no designated client record sets are maintained by ESS, and email is not shared with any third party.  We have several HIPAA companies that use our service and their legal teams have approved our documentation as adequate.

We also offer an addendum to our NDA for HIPAA organizations.  Contact us if you require a signed version of the NDA or the HIPAA document.  We will be happy to work with your HIPAA compliance officer in any way.

back to top

Yes!  We have been ready for the new .travel domain since it was announced.  There is no difference in how we view domains with the new TLD.  We are hopeful that this top level domain does not quickly become a source of spam, but you never know.

back to top

No.  We provide filter services for law firms, medical companies, and even government facilities.  This would be a major security breach for these organizations to allow an outside firm access to their internal network structure.  We require NO firewall access to your network.  We deliver mail using either standard port 25, or any other port you assign. 

back to top

Yes!  You have the ability to view the SMTP logs per domain.  These logs are updated every 5 minutes so you always have current information about the status of your mail.

Screen shot of SMTP Logs. 
Click for a larger image.

back to top

If you process mail for mutiple domains, you MUST purchase filtration for each domain. 

A single account is designed for a single business entity.  Some businesses have more than one domain, this is fine.  Lots of companies have a domain for customer service, another for internal mail, etc.  The intention of the mailboxes is for you to purchase one per user in the company, even if they have mailboxes in all domains. 

The intent is NOT for you to process mail for third parties without purchasing a reseller account.  If you manage email for your customers and bill them yourself please contact us for more information about becoming a reseller.  We offer discounts to resellers who manage their customers email. 

Spam Basics

Spam is unsolicited and unwanted email.  Spam increases expense and lowers efficiency for companies that receive it.  It is estimated that more than half of all email received by a company's mail server is spam.  Spam costs are estimated to range in the 100's of dollars per mailbox per year.  Some of the organizational costs caused by spam include:

• Increase in backup media required to make complete mail server backups
• Time needed daily to delete spam before an "automatic backup" can occur
• Loss of messages resulting from mailboxes cluttered with spam that run out of allocated storage

Spam can not only be a nuisance but a security risk that can introduce viruses, trojans, and worms onto an employee's computer, creating an infection that can propagate across the corporate network and ultimately affect all employees.  Cookies, beacons, spyware, and adware can be surreptitiously installed on company computers, tracking keystrokes, website visits, or other employee activities.  This can result in the pilferage of usernames, passwords, bank account numbers, or access codes.  This potential for information leakage is a significant liability and a security threat.

Spam can also be used to perpetuate identity theft scams.  Some spam messages are designed to mislead employees into logging on to fake but official looking websites that ask them to enter personal information.  This information can then be sold to the highest bidder.

Additionally spam may contain objectionable content or images which can expose a company to hostile workplace lawsuits.  A company's Acceptable Network Usage Policy (ANUP) can also be weakened if nothing is done to prevent the spam which violates it.

back to top

The actual volume of spam received by a specific domain varies.  We have found that the percentage of spam messages to total messages received will range anywhere from 64% to as high as 97% in extreme cases.

The actual size of spam messages also fluctuates.  Some spam is made unnecessarily large in an attempt to hide the actual spam content at the end of the email, while some spam is relatively small in an effort to quickly slip through the filter and deliver its message.

back to top

Even if employees are careful about releasing their email addresses they can still receive multiple spam messages in their inbox everyday.  This is due to the overwhelming volume of spam sent out by various marketing groups. 

Even if email addresses aren't compromised, spammers can still deliver their messages by attempting various combinations of first names, first initials, and last names.  If your company issues email addresses based upon an easy-to-guess format (ex. John Doe = jdoe@yourcompany.com) spammers can use their databases of common first and last names to generate random email address combinations and deliver spam through trial-and-error.

Companies must implement safe mail procedures and communicate these procedures to their employees.  The Emerald Spam Shield is one of the simplest and most effective tools to help companies combat the loss of time and other potential dangers inherent with spam.

Laws that exist in the US will not stop spam.  Spammers just moved their activities outside the US to avoid the law entirely.

back to top

A company can target spam in any one of three basic areas:  at the server level before email is delivered to employees, at the user level, and through a proxy server external to the company.

Targetting spam at the server level involves installing and maintaining various filters and possibly maintaining additional hardware onsite to facilitate the detection and handling of spam.  This can increase company expense but keeps the spam elimination effort centralized.

Filtering spam at the user level requires each employee to configure junk mail rules in their email client and train their spam filter to accurately detect spam.  Because the rules and spam filter require regular maintenance to ensure some degree of accuracy, filtering mail at this level introduces the disadvantage of reduced employee productivity.

The final option involves an external SMTP server hosted by a third-party company that handles the spam filtration and delivers only legitimate email to the internal corporate network.  Emerald Spam Shield is an example of this hosted model.  This option eliminates employee productivity loss as well as the load on the mail server and network because spam is identified and handled before it reaches the company.  Since email received by the company is then restricted to legitimate communication the bandwidth usage incurred by mail traffic can be reduced.  Also because the actual corporate email server is never revealed to spammers or attackers the probabilty of your server becoming a direct target is lower.

back to top

When you signed up to purchase your email domain you probably gave the registrar your primary email address.  Unfortunately spammers abuse the system to get WHOIS information from the master database of registrars.  Spammers have no problem sending email to any address found within the WHOIS database, which will include email addresses provided during domain registration.

We recommend that you setup an alternate email account just for the domain registrar. Assign it a descriptive and uncommon name such as domainregister@yourdomain.com.  This account must be valid and should be checked from time to time for messages from your registrar.

back to top

Typically a pump and dump scam is called that because the people who are pumping the stock (trying to get the price to go up) intend to dump it (sell off stock) when it hits a certain price or volume.

A few years ago most of these scams were sponsored by the companies themselves.  They would pay some "marketing" firm ten to twenty thousand shares of worthless stock to drive up the price.  The firm would spam people to get the price to rise and then dump all their shares when it hit some amount.  Of course this usually impacts the stock in a major way (down).  Perhaps the company had a stockholders meeting, or was planning to reissue stock, and was looking for a way to drive up the stock price even if it was temporary.

Today most of these scams are not funded by the companies themselves.  The stock exchanges here in the US have heavily fined several companies for doing this as an unfair trade practice.  Most of the current scams are independent companies that do it to make money themselves.  They have even less motivation to ensure the stock price does not plummet when they dump their shares.  They usually pick a hot sector that is in the news a lot (oil for instance) and then find a small news article or press release from the company and bloat it way out of proportion. 

Some of these scams are actually being funded by criminal organizations with the intent to disrupt the stock of the company.  In a few cases these companies have been subject to hostile takeovers and the stock scam was used as a way to drive the stock price down and allow the attacker to purchase a major position in the company.

back to top

We have setup a page specifically to discuss the webbot and how it works.  Please visit our Emerald Shield webbot information page.